From 18abaa7a030e54975c9efdf8704465c642725833 Mon Sep 17 00:00:00 2001 From: George Guirguis Date: Mon, 22 May 2023 18:40:40 -0700 Subject: [PATCH 1/2] Create Routing_Instructions.md --- scenarios/telemetry/Routing_Instructions.md | 68 +++++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100644 scenarios/telemetry/Routing_Instructions.md diff --git a/scenarios/telemetry/Routing_Instructions.md b/scenarios/telemetry/Routing_Instructions.md new file mode 100644 index 00000000..d5ac05d2 --- /dev/null +++ b/scenarios/telemetry/Routing_Instructions.md @@ -0,0 +1,68 @@ +# Routing Telemetry +Event Grid allows you to route your MQTT messages to Azure services or webhooks for further processing. Accordingly, you can build end-to-end solutions by leveraging your IoT data for data analysis, storage, and visualizations, among other use cases. + +The routing configuration enables you to send all your messages from your clients to an [Event Grid custom topic](https://learn.microsoft.com/en-us/azure/event-grid/custom-topics), and configuring [Event Grid event subscriptions](https://learn.microsoft.com/en-us/azure/event-grid/subscribe-through-portal) to route the messages from that custom topic to the [supported event handlers](https://review.learn.microsoft.com/en-us/azure/event-grid/event-handlers). + +These instructions guide you to route your filtered MQTT messages from your Event Grid namespace to an [Azure Event Hubs](https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-about) for further processing. Consider a use case where you want to route the telemetry from only vehicle1. + +## Configure resources +Configure these resources after the Event Grid namespace configuration to enable the routing flow. + +### Create an Event Hubs instance +Use [these instructions](https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-quickstart-cli) to create an Event Hubs instance. + +### Configure the Event Grid topic +- Configure your Event Grid Topic where your messages will be routed. The region of the topic needs to match the region of your namespace. +```bash +az eventgrid topic create +--name {EG custom topic name} \ +-l {region name} \ +-g $rg \ +--input-schema cloudeventschemav1_0 +``` +- Assign EventGrid Data Sender role on the Event Grid topic to your principal ID +```bash +az role assignment create +--assignee "{Your Principal ID}" \ +--role "EventGrid Data Sender" \ +--scope "/subscriptions/$sub_id/resourcegroups/$rg/providers/Microsoft.EventGrid/topics/{EG Custom Topic Name}" +``` +> Note: +> You can find your principal ID using the command: az ad signed-in-user show + +### Configure your Event Grid event subcription + +Create an Event Grid event subscription to route these messages from the Event Grid topic to your Event Hubs instance. The subscription uses the "Subject Begins With" filter that filters on the MQTT topic used in the MQTT messages. + +```bash +az eventgrid event-subscription create --name contosoEventSubscription \ +--source-resource-id "/subscriptions/$sub_id/resourceGroups/$rg/providers/Microsoft.EventGrid/topics/{Your Event Grid Topic Name}" \ +--endpoint-type eventhub \ +--endpoint /subscriptions/$sub_id/resourceGroups/$rg/providers/Microsoft.EventHub/namespaces/{Event Hub Namespace Name}/eventhubs/{Event Hub Name} +--event-delivery-schema cloudeventschemav1_0 \ +--subject-begins-with vehicles/vehicle1 +``` + +### Configure routing in the Event Grid Namespace +Set the routing configuration on the Event Grid referring to the Event Grid topic. + +```bash +az resource update --id $res_id --is-full-object --properties '{ + "properties": { + "isZoneRedundant": true, + "topicsConfiguration": { + "inputSchema": "CloudEventSchemaV1_0" + }, + "topicSpacesConfiguration": { + "state": "Enabled" + "routeTopicResourceId": "/subscriptions/{Subscription ID}/resourceGroups/{Resource Group ID}/providers/Microsoft.EventGrid/topics/{EG Custom Topic Name}" + } + }, + "location": "eastus2euap" +}' +``` +### View the routed MQTT messages in Azure Event Hubs +After you run the samples to send the MQTT messages, follow these steps to view the routed MQTT messages in Azure Event Hubs using Azure Stream Analytics query +- Navigate to the Event Hubs instance on the Azure portal. +- Go to [**Process data**](https://learn.microsoft.com/en-us/azure/event-hubs/process-data-azure-stream-analytics) using Azure Stream Analytics. +- Observe the routed MQTT messages based on the default query. From dcf01379888abf983a81bbe4b696b19bc23c8485 Mon Sep 17 00:00:00 2001 From: George Guirguis Date: Tue, 23 May 2023 01:46:24 +0000 Subject: [PATCH 2/2] added routing references --- README.md | 2 +- scenarios/telemetry/README.md | 5 ++++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 71410f3d..c342647c 100644 --- a/README.md +++ b/README.md @@ -61,7 +61,7 @@ Follow the instructions in the [Prerequisites](#magic_wand-prerequisites) to con | Scenario | Description | dotnet | C | python | | -------- | ------------|--------|---|------- | | [Getting Started](./scenarios/getting_started/) | This quick start scenario simulates basic MQTT tasks.| :white_check_mark:|:white_check_mark:|:white_check_mark:| -| [Telemetry](./scenarios/telemetry/) | This scenario simulates multiple clients (the producers) sending data to a different set of topics to be consumed by a single application (the consumer). | :white_check_mark:|:white_check_mark:|:soon:| +| [Telemetry](./scenarios/telemetry/) | This scenario simulates multiple clients (the producers) sending data to a different set of topics to be consumed by a single application (the consumer). This scenario also showcases routing the data to an Azure service. | :white_check_mark:|:white_check_mark:|:soon:| | [Command](./scenarios/command/) | This scenario simulates the request-response messaging pattern using MQTT v5. | :white_check_mark:|:soon:|:soon:| | [Alert](./scenarios/alert/) | This scenario simulates a fan-out use case where multiple clients receive a singlemessage from the same topic. | :soon:|:soon:|:soon:| diff --git a/scenarios/telemetry/README.md b/scenarios/telemetry/README.md index 7a86b31a..7189b875 100644 --- a/scenarios/telemetry/README.md +++ b/scenarios/telemetry/README.md @@ -2,7 +2,7 @@ | [Create Client Certificates](#lock-create-client-certificates) | [Configure Event Grid Namespaces](#triangular_ruler-configure-event-grid-namespaces) | [Configure Mosquitto](#fly-configure-mosquitto) | [Run the Sample](#game_die-run-the-sample) | -This scenario shows how multiple clients send data (the producers) to different topics that can be consumed by a single application (the consumer). This scenario also showcases routing the data to an Azure service. +This scenario shows how multiple clients send data (the producers) to different topics that can be consumed by a single application (the consumer). This scenario also showcases routing the data to Azure Event Hubs. Consider a use case where a backend solution needs to identify the location of vehicles on a map. Vehicles should be prohibited from listening to other vehicles location on their behalf. Finally, the location data need to be routed to a storage queue. @@ -152,6 +152,9 @@ echo "MQTT_CERT_FILE=map-app.pem" >> map-app.env echo "MQTT_KEY_FILE=map-app.key" >> map-app.env echo "MQTT_CA_PATH=/etc/ssl/certs" >> map-app.env # required by mosquitto_lib to validate EG Tls cert ``` +### Configure routing + +Follow the [Routing Instructions](./Routing_Instructions.md) to route the telemetry to an Azure Event Hubs instance. ## :fly: Configure Mosquitto