# Azure Eventhub and Machine Learning
## Description 📝
This architecture combines some of the main resources in Azure , used to process big sources of data and also make predictive analysis based on this data.
In the scenario presented in the template we use eventhub as a source of data and kusto databases to store the data and make them ready for processing. In combination with data factory we also add
# Architecture components 🏛️
1. Resource group for the eventhub and the machine learning resources
2. Eventhub- Azure Event Hubs is a big data streaming platform and event ingestion service. It can receive and process millions of events per second. Data sent to an event hub can be transformed and stored by using any real-time analytics provider or batching/storage adapters.
3. An Event Hubs namespace provides a unique scoping container, in which you create one or more event hubs. It provides DNS-integrated network endpoints and a range of access control and network integration management features such as IP filtering, virtual network service endpoint, and Private Link.
4. A consumer group is a view (state, position, or offset) of an entire event hub. Consumer groups enable multiple consuming applications to each have a separate view of the event stream, and to read the stream independently at their own pace and with their own offsets.
5. An authorization rule has a name, is associated with specific rights, and carries a pair of cryptographic keys. You use the rule's name and key via the Event Hubs clients or in your own code to generate SAS tokens. A client can then pass the token to Event Hubs to prove authorization for the requested operation.
6. Kusto follows a relation model of storing the data where upper-level entity is a database.
Kusto cluster can host several databases, where each database will host its own collection of tables, stored functions, and external tables.
1. Azure Data Factory is a managed cloud service that's built for these complex hybrid extract-transform-load (ETL), extract-load-transform (ELT), and data integration projects.
2. Azure IoT Hub is a managed service hosted in the cloud that acts as a central message hub for communication between an IoT application and its attached devices. You can connect millions of devices and their backend solutions reliably and securely. Almost any device can be connected to an IoT hub
3. A data lake is a centralized repository designed to store, process, and secure large amounts of structured, semistructured, and unstructured data. It can store data in its native format and process any variety of it, ignoring size limits.
4. Azure Synapse is an enterprise analytics service that accelerates time to insight across data warehouses and big data systems. Azure Synapse brings together the best of SQL technologies used in enterprise data warehousing, Spark technologies used for big data, Data Explorer for log and time series analytics, Pipelines for data integration and ETL/ELT, and deep integration with other Azure services such as Power BI, CosmosDB, and AzureML.
5. The workspace is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning. The workspace keeps a history of all training runs, including logs, metrics, output, and a snapshot of your scripts. You use this information to determine which training run produces the best model.
6. Application Insights is an extension of Azure Monitor and provides Application Performance Monitoring. In addition to collecting Metrics and application Telemetry data, which describe application activities and health, Application Insights can also be used to collect and store application trace logging data.
# Requirements
| Name | Configuration |
| --- | --- |
| Terraform | all versions |
| Provider | Azure |
| Provider version | 3.2 |
## How to use the architecture
To use this architecture , clone it within your project and change the following variables:
| Variable | Description |
| --- | --- |
|prefix| Application name |
| admin_pass | Default Hub Subnet Prefix |
