From 78a9498dd5762a2de5fd5d3b80e234a42451c090 Mon Sep 17 00:00:00 2001 From: Kexiang Wang Date: Sun, 8 Dec 2024 21:17:48 -0500 Subject: [PATCH 1/3] add more doc for webhook --- integrations/sources/eventbridge-webhook.mdx | 131 +++++++++++++++++++ integrations/sources/hubspot-webhook.mdx | 108 +++++++++++++++ integrations/sources/rudderstack-webhook.mdx | 117 +++++++++++++++++ integrations/sources/segment-webhook.mdx | 116 ++++++++++++++++ 4 files changed, 472 insertions(+) create mode 100644 integrations/sources/eventbridge-webhook.mdx create mode 100644 integrations/sources/hubspot-webhook.mdx create mode 100644 integrations/sources/rudderstack-webhook.mdx create mode 100644 integrations/sources/segment-webhook.mdx diff --git a/integrations/sources/eventbridge-webhook.mdx b/integrations/sources/eventbridge-webhook.mdx new file mode 100644 index 00000000..11434e3e --- /dev/null +++ b/integrations/sources/eventbridge-webhook.mdx @@ -0,0 +1,131 @@ +--- +title: "Ingest data from Amazon EventBridge webhook" +description: "Ingest Amazon EventBridge events directly into your RisingWave database for real-time processing and analytics." +sidebarTitle: Amazon EventBridge webhook +--- + +Amazon EventBridge is a serverless event bus service that enables you to easily connect applications using data from various event sources. By setting RisingWave as a destination for your EventBridge rule targets, you can route real-time event data generated in your Amazon account directly into your database. This allows you to leverage RisingWave’s capabilities for real-time data processing, analytics, and integrations with downstream systems, all without needing additional intermediate brokers or components. + + +This guide will walk through the steps to set up RisingWave as a destination for Amazon EventBridge webhooks. + +## 1. Create a secret in RisingWave + +First, create a secret in RisingWave to securely store a secret string. This secret will be used to validate incoming webhook requests from Amazon EventBridge . + +```sql +CREATE SECRET test_secret WITH (backend = 'meta') AS 'TEST_WEBHOOK'; +``` + +| Parameter or clause | Description | +| :--------- | :-----------| +|`test_secret`| The name of the secret.| +| `TEST_WEBHOOK`| The secret string used for signing and verifying webhook payloads. Replace this with a secure, random string.| + +## 2. Create a table in RisingWave + +Next, create a table configured to accept webhook data from Amazon EventBridge. + +```sql +create table wbhtable ( + data JSONB +) WITH ( + connector = 'webhook', +) VALIDATE SECRET test_secret AS secure_compare( + headers->>'authorization', + test_secret +); +``` + +| Parameter or clause | Description | +| :--------- | :-----------| +| `data JSONB` | Defines the name of column to store the JSON payload from the webhook. Currently, only `JSONB` type is supported for webhook tables. | +| `headers->>'...'` | Extracts the signature provided by Amazon EventBridge in the `authorization` HTTP header. The header key can be any string value and you can specify it when you create your webhook destination for RisingWave.

In `secure_compare()` function, the whole HTTP header is interpreted as a JSONB object, and you can access the header value using the `->>` operator, but only the lower-case header names in the `->>` operator, otherwise the verification will fail. | +| test_secret | Provides the expected signature. In the example above, we directly compare the secret value of `test_secret` with the signature provided in the headers to verify the requests. | +| `secure_compare(...)` | Validates requests by matching the header signature against the computed signature, ensuring only authenticated requests are processed. The `secure_compare()` function compares two strings in a fixed amount of time, regardless of whether they are equal or not, ensuring that the comparison is secure and resistant to timing attacks. | + +## 3. Set up webhook in Amazon EventBridge + +After configuring RisingWave to accept webhook data, set up Amazon EventBridge to send events to your RisingWave instance. + +### RisingWave webhook URL + +The webhook URL should follow this format: +``` +https:///webhook/// +``` + +| Parameter | Description | +|-----------|-------------| +| `HOST` | The hostname or IP address where your RisingWave instance is accessible. This could be a domain name or an IP address. | +| `database` | The name of the RisingWave database where your table resides | +| `schema_name` | The schema name of your table, typically `public` unless specified otherwise. | +| `table_name` | The name of the table you created to receive webhook data, e.g., `wbhtable`. | + + +### Configure webhook in Amazon EventBridge + +For more detailed instructions, refer to the [Amazon EventBridge documentation](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-setup.html). + +Before configuring RisingWave as a target in EventBridge, it's important to understand two key concepts: + +- **Event Bus**: An EventBridge event bus is a logical collection point for events. Events from various sources flow into an event bus. You can use the default event bus or create your own custom event bus to organize and isolate your events. + +- **Rule**: A rule in EventBridge matches incoming events and routes them to targets based on defined event patterns or schedules. When a rule matches an event, it sends the event to the configured target—in this case, your RisingWave webhook. + + + +Open the Amazon EventBridge Console: +Sign in to the AWS Management Console and navigate to **Amazon EventBridge**. + + +Create or Select an Event Bus: +- In the EventBridge console, go to **Event buses**. +- You can use the default event bus or create a new custom event bus. + - To create a new event bus, click **Create event bus**, provide a name, and click **Create**. + + +Create a Rule for the Event Bus: +- Navigate to **Rules** in the left sidebar. +- Click **Create rule**. +- Enter a name and description for your rule. +- Under **Event bus**, select the event bus you created or the default one if you wish to use it. +- Define the event pattern or schedule that determines when this rule triggers. + + +Add a Target to the Rule under **Select targets**: +- From the **Target** dropdown, select **EventBridge API destination**. Create a new target or use a existing target. +- For a new target, click **Create a new API destination**. In the **API destination endpoint** field, enter your RisingWave webhook URL. Set **HTTP method** to `POST`. + + +Configure Authentication: +- Under **Configure authorization**, select `Custom configuration`. +- For **Authorization type**, select `API Key`. Add an `Authorization` header in **API key name**. The **API key name** should match the key specified in your RisingWave table’s validation clause (e.g., `Authorization`), and the **Value** should be the secret you created in RisingWave (e.g., `TEST_WEBHOOK`). + + +Finish Other Configurations and Save Your Configuration: +- Review your settings to ensure correctness. +- Click **Create rule** to finalize the configuration. + + + +## 4. Push Data from Amazon EventBridge via Webhook + +With the webhook configured, Amazon EventBridge will automatically send HTTP POST requests to your RisingWave webhook URL whenever there are data generated from your AWS platform. RisingWave will receive these requests, validate the signatures, and insert the payload data into the target table. + +## 5. Further event processing + +The data in the table is already ready for further processing. You can access the fields using `data->'field_name'` in SQL queries. + +You can create a materialized view to extract specific fields from the JSON payload. + +```sql +CREATE MATERIALIZED VIEW eventbridge_events AS +SELECT + data->>'id' AS id, + data->'sender'->>'login' AS sender_login, + data->>'created_at' AS event_time +FROM wbhtable; +``` + +You can now query `eventbridge_events` like a regular table to perform analytics, generate reports, or trigger further processing. \ No newline at end of file diff --git a/integrations/sources/hubspot-webhook.mdx b/integrations/sources/hubspot-webhook.mdx new file mode 100644 index 00000000..d76c72c1 --- /dev/null +++ b/integrations/sources/hubspot-webhook.mdx @@ -0,0 +1,108 @@ +--- +title: "Ingest data from HubSpot webhook" +description: "Ingest HubSpot events directly into your RisingWave database for real-time processing and analytics." +sidebarTitle: HubSpot webhook +--- + +HubSpot is a CRM platform that provides various marketing, sales, and customer service features. It allows you to leverage webhooks through its workflow automation features. With these webhooks, you can receive real-time HTTP POST requests whenever specific events occur—such as a contact filling out a form or a deal reaching a certain stage—and forward that data to other systems. For example, you can automatically post a contact’s information to another CRM once they complete a form submission, or send deal details to an external shipment-handling system to initiate a purchase order. By integrating these HubSpot webhooks with RisingWave, you gain the ability to ingest and process this event data within your database, enabling real-time analytics and deeper insights into user behavior and engagement. + +HubSpot Webhooks are not available on all HubSpot plans. You must have the appropriate subscription level (e.g., certain Operations Hub Enterprise plans) to access HubSpot Webhook functionality. Check your HubSpot plan details before proceeding. + +This guide will walk through the steps to set up RisingWave as a destination for HubSpot webhooks. + +## 1. Create a secret in RisingWave + +First, create a secret in RisingWave to securely store a secret string. This secret will be used to validate incoming webhook requests from HubSpot. + +```sql +CREATE SECRET test_secret WITH (backend = 'meta') AS 'TEST_WEBHOOK'; +``` + +| Parameter or clause | Description | +| :--------- | :-----------| +|`test_secret`| The name of the secret.| +| `TEST_WEBHOOK`| The secret string used for signing and verifying webhook payloads. Replace this with a secure, random string. If you are using signature V2, please fill the client secret of your HubSpot APP, the format of which should be `yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy` | + +## 2. Create a table in RisingWave + +Next, create a table configured to accept webhook data from HubSpot. + +```sql +CREATE TABLE wbhtable ( + data JSONB +) WITH ( + connector = 'webhook' +) VALIDATE SECRET test_secret AS secure_compare( + headers->>'x-hubspot-signature', + encode( + sha256( + convert_to( + (test_secret || 'POST' || 'http://127.0.0.1:4560/webhook/dev/public/' || convert_from(data, 'utf8')) + , 'UTF8' + ) + ) + , 'hex' + ) +); +``` + +| Parameter or clause | Description | +| :--------- | :-----------| +| `data JSONB` | Defines the name of column to store the JSON payload from the webhook. Currently, only `JSONB` type is supported for webhook tables. | +| `headers->>'...'` | Extracts the signature provided by HubSpot in the `x-hubspot-signature` HTTP header.

In `secure_compare()` function, the whole HTTP header is interpreted as a JSONB object, and you can access the header value using the `->>` operator, but only the lower-case header names in the `->>` operator, otherwise the verification will fail. | +| encode(...)` | Computes the expected signature by concatenating the secret, HTTP method (`POST`), RisingWave Webhook URL (`http://127.0.0.1:4560/webhook/dev/public/` in this example), and the request body converted from UTF-8 `data`. It then takes a `SHA-256` hash of this string and encodes it in hex. | +| `secure_compare(...)` | Validates requests by matching the header signature against the computed signature, ensuring only authenticated requests are processed. The `secure_compare()` function compares two strings in a fixed amount of time, regardless of whether they are equal or not, ensuring that the comparison is secure and resistant to timing attacks. | + +In HubSpot Webhook, several request validation methods are available, including v1, v2, v3, and API Key–based authentication. Each version employs a different approach to generating and verifying signatures, and may store the signature in a distinct header field. The SQL example above illustrates how to validate a signature generated using the v2 method. + +If you prefer to use an API key, you can simply include a key-value pair in the HTTP headers and validate against that value instead. For comprehensive details on all supported validation methods, refer to [HubSpot’s official documentation](https://developers.hubspot.com/beta-docs/guides/apps/authentication/validating-requests). + +## 3. Set up webhook in HubSpot + +After configuring RisingWave to accept webhook data, set up HubSpot to send events to your RisingWave instance. + +### RisingWave webhook URL + +The webhook URL should follow this format: +``` +https:///webhook/// +``` + +| Parameter | Description | +|-----------|-------------| +| `HOST` | The hostname or IP address where your RisingWave instance is accessible. This could be a domain name or an IP address. | +| `database` | The name of the RisingWave database where your table resides | +| `schema_name` | The schema name of your table, typically `public` unless specified otherwise. | +| `table_name` | The name of the table you created to receive webhook data, e.g., `wbhtable`. | + + +### Configure webhook in HubSpot + +While the official [HubSpot webhook documentation](https://knowledge.hubspot.com/workflows/how-do-i-use-webhooks-with-hubspot-workflows) provides comprehensive guidance, here are a few points to keep in mind: + +- **HTTP Method**: Always use the POST method when configuring your webhook to send data to RisingWave. + +- **Signature Validation**: Ensure that your chosen validation method (v1, v2, v3, or API key) aligns with the header key and `secure_compare` expression defined in RisingWave. Different methods generate signatures in distinct ways, so you may need to adjust your validation logic accordingly. + +- **Testing Your Configuration**: Use the **test actions** feature in HubSpot to verify that your webhook and RisingWave setup work as intended. This helps confirm that the configuration is correct before putting it into production. + +## 4. Push data from HubSpot via webhook + +With the webhook configured, HubSpot will automatically send HTTP POST requests to your RisingWave webhook URL whenever the specified condition is satisfied. RisingWave will receive these requests, validate the signatures, and insert the payload data into the target table. + +## 5. Further event processing + +The data in the table is already ready for further processing. You can access the fields using `data->'field_name'` in SQL queries. + +You can create a materialized view to extract specific fields from the JSON payload. + +```sql +CREATE MATERIALIZED VIEW hubspot_events AS +SELECT + data->>'vid' AS id, + data->'properties'->>'lastname' AS last_name, + data->'properties'->>'hs_object_id' AS hs_object_id +FROM wbhtable; +``` + +You can now query `hubspot_events` like a regular table to perform analytics, generate reports, or trigger further processing. \ No newline at end of file diff --git a/integrations/sources/rudderstack-webhook.mdx b/integrations/sources/rudderstack-webhook.mdx new file mode 100644 index 00000000..480b3489 --- /dev/null +++ b/integrations/sources/rudderstack-webhook.mdx @@ -0,0 +1,117 @@ +--- +title: "Ingest data from RudderStack webhook" +description: "Ingest RudderStack events directly into your RisingWave database for real-time processing and analytics." +sidebarTitle: RudderStack webhook +--- + +RudderStack is a Customer Data Platform (CDP) that enables you to collect, route, and process event data from your websites, mobile apps, and servers to various downstream destinations. Webhooks in RudderStack allow you to send real-time event data to external systems via HTTP requests. By configuring RisingWave as a webhook destination, you can ingest and process RudderStack events directly within your database for real-time analytics and processing. + +This guide will walk through the steps to set up RisingWave as a destination for RudderStack webhooks. + +## 1. Create a secret in RisingWave + +First, create a secret in RisingWave to securely store a secret string. This secret will be used to validate incoming webhook requests from RudderStack. + +```sql +CREATE SECRET test_secret WITH (backend = 'meta') AS 'TEST_WEBHOOK'; +``` + +| Parameter or clause | Description | +| :--------- | :-----------| +|`test_secret`| The name of the secret.| +| `TEST_WEBHOOK`| The secret string used for signing and verifying webhook payloads. Replace this with a secure, random string.| + +## 2. Create a table in RisingWave + +Next, create a table configured to accept webhook data from RudderStack. + +```sql +create table wbhtable ( + data JSONB +) WITH ( + connector = 'webhook', +) VALIDATE SECRET test_secret AS secure_compare( + headers->>'authorization', + test_secret +); +``` + +| Parameter or clause | Description | +| :--------- | :-----------| +| `data JSONB` | Defines the name of column to store the JSON payload from the webhook. Currently, only `JSONB` type is supported for webhook tables. | +| `headers->>'...'` | Extracts the signature provided by RudderStack in the `authorization` HTTP header. The header key can be any string value and you can specify it when you create your webhook destination for RisingWave.

In `secure_compare()` function, the whole HTTP header is interpreted as a JSONB object, and you can access the header value using the `->>` operator, but only the lower-case header names in the `->>` operator, otherwise the verification will fail. | +| test_secret | Provides the expected signature. In the example above, we directly compare the secret value of `test_secret` with the signature provided in the headers to verify the requests. | +| `secure_compare(...)` | Validates requests by matching the header signature against the computed signature, ensuring only authenticated requests are processed. The `secure_compare()` function compares two strings in a fixed amount of time, regardless of whether they are equal or not, ensuring that the comparison is secure and resistant to timing attacks. | + +## 3. Set up webhook in RudderStack + +After configuring RisingWave to accept webhook data, set up RudderStack to send events to your RisingWave instance. + +### RisingWave webhook URL + +The webhook URL should follow this format: +``` +https:///webhook/// +``` + +| Parameter | Description | +|-----------|-------------| +| `HOST` | The hostname or IP address where your RisingWave instance is accessible. This could be a domain name or an IP address. | +| `database` | The name of the RisingWave database where your table resides | +| `schema_name` | The schema name of your table, typically `public` unless specified otherwise. | +| `table_name` | The name of the table you created to receive webhook data, e.g., `wbhtable`. | + + +### Configure webhook in RudderStack + +For more detailed instructions, refer to the [RudderStack documentation](https://www.rudderstack.com/docs/destinations/webhooks/setup-guide/). + + + +Access RudderStack Dashboard: Log in to your RudderStack dashboard. + + +As a CDP, RudderStack allow your build your own data pipeline. To sink data into RisingWave, you first need to have data sources. You can build your sources following the [guidance](https://www.rudderstack.com/docs/sources/overview/). +Upon you have created your source, in the left sidebar, click on **Collect** > **Destinations** > **New Destination**. + + +Choose Webhook as the Destination: +- Search for `webhook` in the list of available destinations. +- Select **webhook** and proceed to configure it. + + +Configure the webhook settings: + - **Name destination**: Specify the name of your webhook destination. + - **Connect Sources**: Choose the data source you want to push to RisingWave from. + - **Webhook URL**: Enter your RisingWave webhook URL. + - **URL Method**: Select `POST`. + - **Headers**: Keep the `content-type: application/json` unchanged. Add an `Authorization` header with the value set to your secret string ('TEST_WEBHOOK' or the secure string you used when creating `test_secret` in RisingWave). + - - Header Key: 'Authorization' (Or replace with the value you like, make sure it's aligned with the `CREATE TABLE` DML SQL) + - - Header Value: 'TEST_WEBHOOK' (replace with your actual secret) + + +Save and Activate the Destination: +- Review your settings. +- Click **continue** to activate the webhook destination. + + +## 4. Push Data from RudderStack via Webhook + +With the webhook configured, RudderStack will automatically send HTTP POST requests to your RisingWave webhook URL whenever there are data generated from the connected source. RisingWave will receive these requests, validate the signatures, and insert the payload data into the target table. + +## 5. Further event processing + +The data in the table is already ready for further processing. You can access the fields using `data->'field_name'` in SQL queries. + +You can create a materialized view to extract specific fields from the JSON payload. + +```sql +CREATE MATERIALIZED VIEW rudderstack_events AS +SELECT + data->>'id' AS id, + data->'sender'->>'login' AS sender_login, + data->>'created_at' AS event_time +FROM wbhtable; +``` + +You can now query `rudderstack_events` like a regular table to perform analytics, generate reports, or trigger further processing. \ No newline at end of file diff --git a/integrations/sources/segment-webhook.mdx b/integrations/sources/segment-webhook.mdx new file mode 100644 index 00000000..41b25bad --- /dev/null +++ b/integrations/sources/segment-webhook.mdx @@ -0,0 +1,116 @@ +--- +title: "Ingest data from Segment webhook" +description: "Ingest Segment events directly into your RisingWave database for real-time processing and analytics." +sidebarTitle: Segment webhook +--- + +Segment is a Customer Data Platform (CDP) that collects event data from various sources and routes it to different destinations for analytics, marketing, and data warehousing. By configuring a webhook as a destination, you can forward events from Segment directly into RisingWave. This setup enables you to leverage RisingWave’s capabilities for real-time data processing and analytics without the need for additional intermediaries. + +This guide will walk through the steps to set up RisingWave as a destination for Segment webhooks. + +## 1. Create a secret in RisingWave + +First, create a secret in RisingWave to securely store a secret string. This secret will be used to validate incoming webhook requests from Segment. + +```sql +CREATE SECRET test_secret WITH (backend = 'meta') AS 'TEST_WEBHOOK'; +``` + +| Parameter or clause | Description | +| :--------- | :-----------| +|`test_secret`| The name of the secret.| +| `TEST_WEBHOOK`| The secret string used for signing and verifying webhook payloads. Replace this with a secure, random string.| + +## 2. Create a table in RisingWave + +Next, create a table configured to accept webhook data from Segment. + +```sql +CREATE TABLE wbhtable ( + data JSONB +) WITH ( + connector = 'webhook' +) VALIDATE SECRET test_secret AS secure_compare( + headers->>'x-signature', + encode(hmac(test_secret, data, 'sha1'), 'hex') +); +``` + + +| Parameter or clause | Description | +| :--------- | :-----------| +| `data JSONB` | Defines the name of column to store the JSON payload from the webhook. Currently, only `JSONB` type is supported for webhook tables. | +| `headers->>'...'` | Extracts the signature provided by Segment in the `x-signature` HTTP header.

In `secure_compare()` function, the whole HTTP header is interpreted as a JSONB object, and you can access the header value using the `->>` operator, but only the lower-case header names in the `->>` operator, otherwise the verification will fail. | +| encode(...)` | Computes the expected signature. In the example above, it generates an `HMAC SHA-1` hash of the payload (`data`) using the secret (`test_secret`), encodes it in hexadecimal. | +| `secure_compare(...)` | Validates requests by matching the header signature against the computed signature, ensuring only authenticated requests are processed. The `secure_compare()` function compares two strings in a fixed amount of time, regardless of whether they are equal or not, ensuring that the comparison is secure and resistant to timing attacks. | + + +## 3. Set up webhook in Segment + +After configuring RisingWave to accept webhook data, set up Segment to send events to your RisingWave instance. + +### RisingWave webhook URL + +The webhook URL should follow this format: +``` +https:///webhook/// +``` + +| Parameter | Description | +|-----------|-------------| +| `HOST` | The hostname or IP address where your RisingWave instance is accessible. This could be a domain name or an IP address. | +| `database` | The name of the RisingWave database where your table resides | +| `schema_name` | The schema name of your table, typically `public` unless specified otherwise. | +| `table_name` | The name of the table you created to receive webhook data, e.g., `wbhtable`. | + + +### Configure webhook in Segment + +For more detailed instructions, refer to the Segment's [Webhooks Destination Documentation](https://segment.com/docs/connections/destinations/catalog/webhooks/) and [Webhooks (Actions) Destination Documentation](https://segment.com/docs/connections/destinations/catalog/actions-webhook/). + + + +Access the Segment Workspace: +Log in to your Segment workspace. + + +In the left sidebar, click on **Connections** > **Add Destination**. +- Search for `webhook` in the list of available destinations. +- Select **Webhooks (Actions)** and click **Add destination** to proceed to configure it. + + +- Select data source, which identify the source you want to send events from and follow the steps to name your destination. +- On the **Settings** > **Basic Settings** > **Shared Secret** tab: Enter the same secret string you used when creating the RisingWave +- secret (e.g., `'TEST_WEBHOOK'`). This ensures that Segment signs the payloads using this secret, allowing RisingWave to validate them. +- Check the **Settings** > **Basic Settings** > **Enable Destination** tab and ensure it's enabled. + + +Configure the webhook description on the **Mappings** tab. +- Click **New Mapping** and then **Send**. +- Choose the event type you want to send to RisingWave +- Fill the configuration for the webhook requests based on the source information. You can also directly fill the values, filling RisingWave webhook URL as `URL, `POST` as `Method`. + +At the end of the page, you can send the test message via **Send test event**. +Review your settings, click **Next** and specify a name. + +## 4. Push data from Segment via webhook + +With the webhook configured, Segment will automatically send HTTP POST requests to your RisingWave webhook URL whenever the specified events occur (e.g., pushes to the repository). RisingWave will receive these requests, validate the signatures, and insert the payload data into the target table. + +## 5. Further event processing + +The data in the table is already ready for further processing. You can access the fields using `data->'field_name'` in SQL queries. + +You can create a materialized view to extract specific fields from the JSON payload. + +```sql +CREATE MATERIALIZED VIEW segmentb_events AS +SELECT + data->>'action' AS action, + data->'repository'->>'full_name' AS repository_name, + data->'sender'->>'login' AS sender_login, + data->>'created_at' AS event_time +FROM wbhtable; +``` + +You can now query `segmentb_events` like a regular table to perform analytics, generate reports, or trigger further processing. \ No newline at end of file From 063e716026a4f601c342cae5ba7c4150ea07a313 Mon Sep 17 00:00:00 2001 From: Kexiang Wang Date: Sun, 8 Dec 2024 21:25:59 -0500 Subject: [PATCH 2/3] fix --- integrations/sources/webhook.mdx | 9 ++++++++- mint.json | 6 +++++- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/integrations/sources/webhook.mdx b/integrations/sources/webhook.mdx index f97d786c..36f4f2f0 100644 --- a/integrations/sources/webhook.mdx +++ b/integrations/sources/webhook.mdx @@ -65,4 +65,11 @@ RisingWave has been verified to work with the following webhook sources and auth While only the above sources have been thoroughly tested, RisingWave can support additional webhook sources and authentication methods. You can integrate other services using similar configurations. ## See also -[Ingest from Github webhook](/integrations/sources/github-webhook): Step-by-step guide to help you set up and configure your webhook sources. \ No newline at end of file + +Step-by-step guide to help you set up and configure your webhook sources: + +[Ingest from Github webhook](/integrations/sources/github-webhook) +[Ingest from Segment webhook](/integrations/sources/segment-webhook) +[Ingest from HubSpot webhook](/integrations/sources/hubspot-webhook) +[Ingest from Amazon EventBridge webhook](/integrations/sources/eventbridge-webhook) +[Ingest from RudderStack webhook](/integrations/sources/rudderstack-webhook) diff --git a/mint.json b/mint.json index 33a700d9..fd00dae0 100644 --- a/mint.json +++ b/mint.json @@ -722,7 +722,11 @@ "group": "Webhook", "pages": [ "integrations/sources/webhook", - "integrations/sources/github-webhook" + "integrations/sources/github-webhook", + "integrations/sources/segment-webhook", + "integrations/sources/hubspot-webhook", + "integrations/sources/eventbridge-webhook", + "integrations/sources/rudderstack-webhook" ] } ] From fc8fdf5c3fad1b9eb1cdc6eaaabfbcd42c323833 Mon Sep 17 00:00:00 2001 From: WanYixian Date: Mon, 9 Dec 2024 11:29:54 +0800 Subject: [PATCH 3/3] minor update --- integrations/sources/eventbridge-webhook.mdx | 5 ++--- integrations/sources/hubspot-webhook.mdx | 6 +++--- integrations/sources/rudderstack-webhook.mdx | 8 ++++---- integrations/sources/segment-webhook.mdx | 15 +++++++-------- integrations/sources/webhook.mdx | 16 ++++++++-------- 5 files changed, 24 insertions(+), 26 deletions(-) diff --git a/integrations/sources/eventbridge-webhook.mdx b/integrations/sources/eventbridge-webhook.mdx index 11434e3e..28917631 100644 --- a/integrations/sources/eventbridge-webhook.mdx +++ b/integrations/sources/eventbridge-webhook.mdx @@ -41,7 +41,7 @@ create table wbhtable ( | :--------- | :-----------| | `data JSONB` | Defines the name of column to store the JSON payload from the webhook. Currently, only `JSONB` type is supported for webhook tables. | | `headers->>'...'` | Extracts the signature provided by Amazon EventBridge in the `authorization` HTTP header. The header key can be any string value and you can specify it when you create your webhook destination for RisingWave.

In `secure_compare()` function, the whole HTTP header is interpreted as a JSONB object, and you can access the header value using the `->>` operator, but only the lower-case header names in the `->>` operator, otherwise the verification will fail. | -| test_secret | Provides the expected signature. In the example above, we directly compare the secret value of `test_secret` with the signature provided in the headers to verify the requests. | +| `test_secret` | Provides the expected signature. In the example above, we directly compare the secret value of `test_secret` with the signature provided in the headers to verify the requests. | | `secure_compare(...)` | Validates requests by matching the header signature against the computed signature, ensuring only authenticated requests are processed. The `secure_compare()` function compares two strings in a fixed amount of time, regardless of whether they are equal or not, ensuring that the comparison is secure and resistant to timing attacks. | ## 3. Set up webhook in Amazon EventBridge @@ -81,8 +81,7 @@ Sign in to the AWS Management Console and navigate to **Amazon EventBridge**. Create or Select an Event Bus: - In the EventBridge console, go to **Event buses**. -- You can use the default event bus or create a new custom event bus. - - To create a new event bus, click **Create event bus**, provide a name, and click **Create**. +- You can use the default event bus or create a new custom event bus. To create a new event bus, click **Create event bus**, provide a name, and click **Create**. Create a Rule for the Event Bus: diff --git a/integrations/sources/hubspot-webhook.mdx b/integrations/sources/hubspot-webhook.mdx index d76c72c1..f233332c 100644 --- a/integrations/sources/hubspot-webhook.mdx +++ b/integrations/sources/hubspot-webhook.mdx @@ -6,7 +6,7 @@ sidebarTitle: HubSpot webhook HubSpot is a CRM platform that provides various marketing, sales, and customer service features. It allows you to leverage webhooks through its workflow automation features. With these webhooks, you can receive real-time HTTP POST requests whenever specific events occur—such as a contact filling out a form or a deal reaching a certain stage—and forward that data to other systems. For example, you can automatically post a contact’s information to another CRM once they complete a form submission, or send deal details to an external shipment-handling system to initiate a purchase order. By integrating these HubSpot webhooks with RisingWave, you gain the ability to ingest and process this event data within your database, enabling real-time analytics and deeper insights into user behavior and engagement. -HubSpot Webhooks are not available on all HubSpot plans. You must have the appropriate subscription level (e.g., certain Operations Hub Enterprise plans) to access HubSpot Webhook functionality. Check your HubSpot plan details before proceeding. +HubSpot webhooks are not available on all HubSpot plans. You must have the appropriate subscription level (e.g., certain Operations Hub Enterprise plans) to access HubSpot Webhook functionality. Check your HubSpot plan details before proceeding. This guide will walk through the steps to set up RisingWave as a destination for HubSpot webhooks. @@ -50,7 +50,7 @@ CREATE TABLE wbhtable ( | :--------- | :-----------| | `data JSONB` | Defines the name of column to store the JSON payload from the webhook. Currently, only `JSONB` type is supported for webhook tables. | | `headers->>'...'` | Extracts the signature provided by HubSpot in the `x-hubspot-signature` HTTP header.

In `secure_compare()` function, the whole HTTP header is interpreted as a JSONB object, and you can access the header value using the `->>` operator, but only the lower-case header names in the `->>` operator, otherwise the verification will fail. | -| encode(...)` | Computes the expected signature by concatenating the secret, HTTP method (`POST`), RisingWave Webhook URL (`http://127.0.0.1:4560/webhook/dev/public/` in this example), and the request body converted from UTF-8 `data`. It then takes a `SHA-256` hash of this string and encodes it in hex. | +| `encode(...)` | Computes the expected signature by concatenating the secret, HTTP method (`POST`), RisingWave Webhook URL (`http://127.0.0.1:4560/webhook/dev/public/` in this example), and the request body converted from UTF-8 `data`. It then takes a `SHA-256` hash of this string and encodes it in hex. | | `secure_compare(...)` | Validates requests by matching the header signature against the computed signature, ensuring only authenticated requests are processed. The `secure_compare()` function compares two strings in a fixed amount of time, regardless of whether they are equal or not, ensuring that the comparison is secure and resistant to timing attacks. | In HubSpot Webhook, several request validation methods are available, including v1, v2, v3, and API Key–based authentication. Each version employs a different approach to generating and verifying signatures, and may store the signature in a distinct header field. The SQL example above illustrates how to validate a signature generated using the v2 method. @@ -71,7 +71,7 @@ https:///webhook/// | Parameter | Description | |-----------|-------------| | `HOST` | The hostname or IP address where your RisingWave instance is accessible. This could be a domain name or an IP address. | -| `database` | The name of the RisingWave database where your table resides | +| `database` | The name of the RisingWave database where your table resides. | | `schema_name` | The schema name of your table, typically `public` unless specified otherwise. | | `table_name` | The name of the table you created to receive webhook data, e.g., `wbhtable`. | diff --git a/integrations/sources/rudderstack-webhook.mdx b/integrations/sources/rudderstack-webhook.mdx index 480b3489..6245e625 100644 --- a/integrations/sources/rudderstack-webhook.mdx +++ b/integrations/sources/rudderstack-webhook.mdx @@ -40,7 +40,7 @@ create table wbhtable ( | :--------- | :-----------| | `data JSONB` | Defines the name of column to store the JSON payload from the webhook. Currently, only `JSONB` type is supported for webhook tables. | | `headers->>'...'` | Extracts the signature provided by RudderStack in the `authorization` HTTP header. The header key can be any string value and you can specify it when you create your webhook destination for RisingWave.

In `secure_compare()` function, the whole HTTP header is interpreted as a JSONB object, and you can access the header value using the `->>` operator, but only the lower-case header names in the `->>` operator, otherwise the verification will fail. | -| test_secret | Provides the expected signature. In the example above, we directly compare the secret value of `test_secret` with the signature provided in the headers to verify the requests. | +| `test_secret` | Provides the expected signature. In the example above, we directly compare the secret value of `test_secret` with the signature provided in the headers to verify the requests. | | `secure_compare(...)` | Validates requests by matching the header signature against the computed signature, ensuring only authenticated requests are processed. The `secure_compare()` function compares two strings in a fixed amount of time, regardless of whether they are equal or not, ensuring that the comparison is secure and resistant to timing attacks. | ## 3. Set up webhook in RudderStack @@ -72,7 +72,7 @@ Access RudderStack Dashboard: Log in to your RudderStack dashboard. As a CDP, RudderStack allow your build your own data pipeline. To sink data into RisingWave, you first need to have data sources. You can build your sources following the [guidance](https://www.rudderstack.com/docs/sources/overview/). -Upon you have created your source, in the left sidebar, click on **Collect** > **Destinations** > **New Destination**. +After creating your source, in the left sidebar, click on **Collect** > **Destinations** > **New Destination**. Choose Webhook as the Destination: @@ -86,8 +86,8 @@ Configure the webhook settings: - **Webhook URL**: Enter your RisingWave webhook URL. - **URL Method**: Select `POST`. - **Headers**: Keep the `content-type: application/json` unchanged. Add an `Authorization` header with the value set to your secret string ('TEST_WEBHOOK' or the secure string you used when creating `test_secret` in RisingWave). - - - Header Key: 'Authorization' (Or replace with the value you like, make sure it's aligned with the `CREATE TABLE` DML SQL) - - - Header Value: 'TEST_WEBHOOK' (replace with your actual secret) + - Header Key: `Authorization` (Or replace with the value you like, make sure it's aligned with the `CREATE TABLE` DML SQL). + - Header Value: `TEST_WEBHOOK` (replace with your actual secret). Save and Activate the Destination: diff --git a/integrations/sources/segment-webhook.mdx b/integrations/sources/segment-webhook.mdx index 41b25bad..005e3fb9 100644 --- a/integrations/sources/segment-webhook.mdx +++ b/integrations/sources/segment-webhook.mdx @@ -41,7 +41,7 @@ CREATE TABLE wbhtable ( | :--------- | :-----------| | `data JSONB` | Defines the name of column to store the JSON payload from the webhook. Currently, only `JSONB` type is supported for webhook tables. | | `headers->>'...'` | Extracts the signature provided by Segment in the `x-signature` HTTP header.

In `secure_compare()` function, the whole HTTP header is interpreted as a JSONB object, and you can access the header value using the `->>` operator, but only the lower-case header names in the `->>` operator, otherwise the verification will fail. | -| encode(...)` | Computes the expected signature. In the example above, it generates an `HMAC SHA-1` hash of the payload (`data`) using the secret (`test_secret`), encodes it in hexadecimal. | +| `encode(...)` | Computes the expected signature. In the example above, it generates an `HMAC SHA-1` hash of the payload (`data`) using the secret (`test_secret`), encodes it in hexadecimal. | | `secure_compare(...)` | Validates requests by matching the header signature against the computed signature, ensuring only authenticated requests are processed. The `secure_compare()` function compares two strings in a fixed amount of time, regardless of whether they are equal or not, ensuring that the comparison is secure and resistant to timing attacks. | @@ -76,19 +76,18 @@ Log in to your Segment workspace. In the left sidebar, click on **Connections** > **Add Destination**. - Search for `webhook` in the list of available destinations. -- Select **Webhooks (Actions)** and click **Add destination** to proceed to configure it. +- Select **Webhooks (Actions)** and click **Add destination** to proceed. -- Select data source, which identify the source you want to send events from and follow the steps to name your destination. -- On the **Settings** > **Basic Settings** > **Shared Secret** tab: Enter the same secret string you used when creating the RisingWave -- secret (e.g., `'TEST_WEBHOOK'`). This ensures that Segment signs the payloads using this secret, allowing RisingWave to validate them. -- Check the **Settings** > **Basic Settings** > **Enable Destination** tab and ensure it's enabled. +- Select the data source that identifies the source you want to send events from and follow the steps to name your destination. +- Click on the tab **Settings** > **Basic Settings** > **Shared Secret**, enter the same secret string you used when creating the RisingWave secret (e.g., `'TEST_WEBHOOK'`). This ensures that Segment signs the payloads using this secret, allowing RisingWave to validate them. +- Click on the tab **Settings** > **Basic Settings** > **Enable Destination** and make sure it is enabled. Configure the webhook description on the **Mappings** tab. - Click **New Mapping** and then **Send**. -- Choose the event type you want to send to RisingWave -- Fill the configuration for the webhook requests based on the source information. You can also directly fill the values, filling RisingWave webhook URL as `URL, `POST` as `Method`. +- Choose the event type you want to send to RisingWave. +- Fill the configuration for the webhook requests based on the source information. You can also directly fill the values, filling RisingWave webhook URL as `URL`, `POST` as `Method`. At the end of the page, you can send the test message via **Send test event**. Review your settings, click **Next** and specify a name. diff --git a/integrations/sources/webhook.mdx b/integrations/sources/webhook.mdx index 36f4f2f0..d18dca7a 100644 --- a/integrations/sources/webhook.mdx +++ b/integrations/sources/webhook.mdx @@ -1,7 +1,7 @@ --- title: "Ingest data from webhook" sidebarTitle: Overview -description: Describes how to ingest data from webhook to RisingWave +description: Describes how to ingest data from webhook to RisingWave. --- A webhook is a mechanism that enables real-time data transfer between applications by sending immediate notifications when specific events occur. Instead of continuously polling for updates, applications can receive data automatically, making it an efficient way to integrate with third-party services. @@ -57,10 +57,10 @@ RisingWave has been verified to work with the following webhook sources and auth |webhook source|Authentication methods| | :-- | :-- | |GitHub| SHA-1 HMAC, SHA-256 HMAC | -|Rudderstack| Bearer Token | |Segment| SHA-1 HMAC | -|AWS EventBridge| Bearer Token | |HubSpot| API Key, Signature V2 | +|AWS EventBridge| Bearer Token | +|Rudderstack| Bearer Token | While only the above sources have been thoroughly tested, RisingWave can support additional webhook sources and authentication methods. You can integrate other services using similar configurations. @@ -68,8 +68,8 @@ RisingWave has been verified to work with the following webhook sources and auth Step-by-step guide to help you set up and configure your webhook sources: -[Ingest from Github webhook](/integrations/sources/github-webhook) -[Ingest from Segment webhook](/integrations/sources/segment-webhook) -[Ingest from HubSpot webhook](/integrations/sources/hubspot-webhook) -[Ingest from Amazon EventBridge webhook](/integrations/sources/eventbridge-webhook) -[Ingest from RudderStack webhook](/integrations/sources/rudderstack-webhook) +- [Ingest from Github webhook](/integrations/sources/github-webhook) +- [Ingest from Segment webhook](/integrations/sources/segment-webhook) +- [Ingest from HubSpot webhook](/integrations/sources/hubspot-webhook) +- [Ingest from Amazon EventBridge webhook](/integrations/sources/eventbridge-webhook) +- [Ingest from RudderStack webhook](/integrations/sources/rudderstack-webhook)