From 1ffb68a29f26d0921db1d6b8fd3bace0262d80f7 Mon Sep 17 00:00:00 2001 From: victoria Date: Fri, 15 Nov 2024 17:27:45 -0500 Subject: [PATCH] (/integrations/webhooks/sync-data) update guide (#1655) Co-authored-by: Alexis Aguilar <98043211+alexisintech@users.noreply.github.com> Co-authored-by: Roy Anger --- docs/integrations/webhooks/sync-data.mdx | 328 +++++++++-------------- 1 file changed, 125 insertions(+), 203 deletions(-) diff --git a/docs/integrations/webhooks/sync-data.mdx b/docs/integrations/webhooks/sync-data.mdx index 50047fe5e7..cbcd88eadc 100644 --- a/docs/integrations/webhooks/sync-data.mdx +++ b/docs/integrations/webhooks/sync-data.mdx @@ -1,95 +1,84 @@ --- -title: Sync Clerk data to your application with webhooks -description: Learn how to sync Clerk data to your application with webhooks. +title: Sync Clerk data to your app with webhooks +description: Learn how to sync Clerk data to your app with webhooks. --- - - Listen for an event in the Clerk Dashboard - - Create a webhook handler that: - - uses Svix to verify the webhook signature - - receives the webhook's payload - - Test the webhook locally using ngrok and the Clerk Dashboard - - Add logic to your application to trigger the webhook + - Set up ngrok + - Set up a webhook endpoint + - Create the webhook + - Get type inference for your webhook events + - Test the webhook + - Configure your production instance -The recommended way to sync data between Clerk and your application is to use webhooks. +The recommended way to sync Clerk data to your app is through webhooks. + +In this guide, you'll set up a webhook in your Next.js app to listen for the `user.created` event, create an endpoint in the Clerk Dashboard, build a handler, and test it locally using ngrok and the Clerk Dashboard. -In this guide, you will learn how to create a Clerk webhook in your Next.js application. You will listen for the `user.updated` event by creating a webhook endpoint in the Clerk Dashboard, creating a webhook handler in your Next.js application, and testing the webhook locally using ngrok and the Clerk Dashboard. +Clerk offers many events, but three key events include: -This guide can be adapted to listen for any Clerk event. +- `user.created`: Triggers when a new user registers in the app or is created via the Clerk Dashboard or Backend API. Listening to this event allows the initial insertion of user information in your database. +- `user.updated`: Triggers when user information is updated via Clerk components, the Clerk Dashboard, or Backend API. Listening to this event keeps data synced between Clerk and your external database. It is recommended to only sync what you need to simplify this process. +- `user.deleted`: Triggers when a user deletes their account, or their account is removed via the Clerk Dashboard or Backend API. Listening to this event allows you to delete the user from your database or add a `deleted: true` flag. + +These steps apply to any Clerk event. To make the setup process easier, it's recommended to keep two browser tabs open: one for your Clerk [**Webhooks**](https://dashboard.clerk.com/last-active?path=webhooks) page and one for your [ngrok dashboard](https://dashboard.ngrok.com). ### Set up ngrok - To test a webhook locally, you will need to expose your local server to the internet. For this guide, you will use [ngrok](https://ngrok.com/). ngrok will create a **forwarding URL** that you can send your webhook payload to and it will forward the payload to your local server. - - 1. Go to the [ngrok website](https://ngrok.com/) and create an account. - 1. Once you have made it to the [ngrok dashboard](https://dashboard.ngrok.com/get-started/setup/macos), in navigation sidebar, select [**Domains**](https://dashboard.ngrok.com/cloud-edge/domains). - 1. Select **Create domain**. - 1. Install ngrok and add your auth token by following steps 1 and 2 in their [install guide](https://ngrok.com/docs/getting-started/#step-1-install). - 1. ngrok will generate a free, non-ephemeral domain for you and a command to start a tunnel with that domain. The command should look something like this: - - `ngrok http --domain=fawn-two-nominally.ngrok-free.app 3000` - 1. Change the port number to whatever your server is running on. For this guide, ensure it is set to 3000 and then run the command in your terminal. - 1. Copy your **forwarding URL**. It should look something like `https://fawn-two-nominally.ngrok-free.app`. + To test a webhook locally, you need to expose your local server to the internet. This guide uses [ngrok](https://ngrok.com/) which creates a **forwarding URL** that sends the webhook payload to your local server. - ### Create an endpoint in the Clerk Dashboard + 1. Navigate to the [ngrok website](https://dashboard.ngrok.com/signup) to create an account. + 1. Follow steps 1 and 2 in [ngrok's install guide](https://ngrok.com/docs/getting-started/#step-1-install). + 1. In the ngrok dashboard, select [**Domains**](https://dashboard.ngrok.com/domains) from the sidebar. + 1. Select **Create Domain**. After the page refreshes, the **Start a Tunnel** panel will open. + 1. In the **Start a Tunnel** panel, select the command generated by ngrok. This command provides a free, non-ephemeral domain and starts a tunnel with that domain. The command should resemble `ngrok http --url=fawn-two-nominally.ngrok-free.app 80`. + 1. Paste the command in your terminal and change the port number to match your server's port. For this guide, replace `80` with `3000`, then run the command in your terminal. It will generate a **Forwarding** URL. It should resemble `https://fawn-two-nominally.ngrok-free.app`. + 1. Save your **Forwarding** URL somewhere secure. Close the panel. - To create a webhook endpoint, you must provide the **Endpoint URL** and then choose the events you want to listen to. For this guide, you will listen to the `user.updated` event. + ### Set up a webhook endpoint - 1. Navigate to the [Clerk Dashboard](https://dashboard.clerk.com/last-active?path=webhooks). - 1. In the navigation sidenav, select **Webhooks**. - 1. Select the **Add Endpoint** button. - 1. In the **Endpoint URL** field, paste the ngrok URL you copied earlier followed by `/api/webhooks`. This is the endpoint that you will later create, that Svix will send the webhook payload to. The full URL should look something like `https://fawn-two-nominally.ngrok-free.app/api/webhooks`. - 1. In the **Message Filtering** section, select the `user.updated` event. - 1. Select the **Create** button. - 1. You will be redirected to your endpoint's settings page. Leave this page open. + 1. In the Clerk Dashboard, navigate to the [**Webhooks**](https://dashboard.clerk.com/last-active?path=webhooks) page. + 1. Select **Add Endpoint**. + 1. In the **Endpoint URL** field, paste the ngrok **Forwarding** URL you saved earlier, followed by `/api/webhooks`. This is the endpoint that Svix uses to send the webhook payload. The full URL should resemble `https://fawn-two-nominally.ngrok-free.app/api/webhooks`. + 1. In the **Subscribe to events** section, scroll down and select `user.created`. + 1. Select **Create**. You'll be redirected to your endpoint's settings page. Keep this page open. - ### Add your signing secret to your `.env.local` file + ### Add your Signing Secret to `.env.local` - To verify the webhook payload, you will need your endpoint's signing secret. However, you do not want to expose this secret in your codebase, so you will want to provide it as an environment variable. In local development, this can be done by storing the secret in the `.env.local` file. + To verify the webhook payload, you'll need your endpoint's **Signing Secret**. Since you don't want this secret exposed in your codebase, store it as an environment variable in your `.env.local` file during local development. - 1. On the endpoint's settings page, copy the **Signing Secret**. It should be on the right side of the page with an eye icon next to it. - 1. In your project's root directory, you should have an `.env.local` file that includes your Clerk API keys. Here, assign your signing secret to `WEBHOOK_SECRET`. Your file should look something like this: + 1. On the endpoint's settings page, copy the **Signing Secret**. + 1. In your project's root directory, open or create an `.env.local` file, which should already include your Clerk API keys. Assign your **Signing Secret** to `SIGNING_SECRET`. The file should resemble: ```env {{ filename: '.env.local' }} NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY={{pub_key}} CLERK_SECRET_KEY={{secret}} - WEBHOOK_SECRET=whsec_123 + SIGNING_SECRET=whsec_123 ``` - ### Set webhook route as public in your Middleware - - Incoming webhook events will never be signed in -- they are coming from a source outside of your application. Since they will be in a signed out state, the route should be public. - - The following example shows the recommended Middleware configuration for your webhook routes. + ### Set the webhook route as public in your Middleware - ```tsx {{ filename: 'middleware.tsx' }} - import { clerkMiddleware } from '@clerk/nextjs/server' - - // Make sure that the `/api/webhooks(.*)` route is not protected here - export default clerkMiddleware() - - export const config = { - matcher: [ - // Skip Next.js internals and all static files, unless found in search params - '/((?!_next|[^?]*\\.(?:html?|css|js(?!on)|jpe?g|webp|png|gif|svg|ttf|woff2?|ico|csv|docx?|xlsx?|zip|webmanifest)).*)', - // Always run for API routes - '/(api|trpc)(.*)', - ], - } - ``` + Incoming webhook events don't contain auth information. They come from an external source and aren't signed in or out, so the route must be public to allow access. If you're using `clerkMiddleware()`, ensure that the `/api/webhooks(.*)` route is set as public. For information on configuring routes, see the [`clerkMiddleware()` guide](docs/references/nextjs/clerk-middleware). ### Install `svix` - You will use [`svix`](https://www.npmjs.com/package/svix) to verify the webhook signature. Run the following command to install it: + Clerk uses [`svix`](https://www.npmjs.com/package/svix) to deliver webhooks, so you'll use it to verify the webhook signature. Run the following command in your terminal to install the package: ```bash {{ filename: 'terminal' }} @@ -105,34 +94,36 @@ This guide can be adapted to listen for any Clerk event. ``` - ### Create the endpoint in your application + ### Create the endpoint - Create a Route Handler that uses `svix` to verify the webhook signature and that receives the webhook's payload. + Set up a Route Handler that uses `svix` to verify the incoming Clerk webhook and process the payload. - For the sake of this guide, you will only log the payload to the console. In a real world application, you would use the payload to trigger some action. For example, you are listening for the `user.updated` event, so you could perform a database `update` or `upsert` to update the user's details. + For this guide, the payload will be logged to the console. In a real app, you'd use the payload to trigger an action. For example, if listening for the `user.created` event, you might perform a database `create` or `upsert` to add the user's details to the user's table. - Your webhook will need to return either an error code, or a success code, like `200` or `201`. If it returns an error code, the webhook will reflect that in the Dashboard log and it will [retry](/docs/integrations/webhooks/overview#retry). When the webhook returns a success code, there will be no retries and the webhook will show a success status in the Dashboard. + If the route handler returns a [4xx](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status#client_error_responses) or [5xx code](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status#server_error_responses), or no code at all, the webhook event will be [retried](/docs/integrations/webhooks/overview#retry). If the route handler returns a [2xx code](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status#successful_responses), the event will be marked as successful, and retries will stop. > [!NOTE] - > The following Route Handler is not specific to the `user.updated` event and can be used for any webhook event you choose to listen to. + > The following Route Handler can be used for any webhook event you choose to listen to, not just `user.created`. - + - + ```ts {{ filename: 'app/api/webhooks/route.ts' }} import { Webhook } from 'svix' import { headers } from 'next/headers' import { WebhookEvent } from '@clerk/nextjs/server' export async function POST(req: Request) { - // You can find this in the Clerk Dashboard -> Webhooks -> choose the endpoint - const WEBHOOK_SECRET = process.env.WEBHOOK_SECRET + const SIGNING_SECRET = process.env.SIGNING_SECRET - if (!WEBHOOK_SECRET) { - throw new Error('Please add WEBHOOK_SECRET from Clerk Dashboard to .env or .env.local') + if (!SIGNING_SECRET) { + throw new Error('Error: Please add SIGNING_SECRET from Clerk Dashboard to .env or .env.local') } - // Get the headers + // Create new Svix instance with secret + const wh = new Webhook(SIGNING_SECRET) + + // Get headers const headerPayload = headers() const svix_id = headerPayload.get('svix-id') const svix_timestamp = headerPayload.get('svix-timestamp') @@ -140,21 +131,18 @@ This guide can be adapted to listen for any Clerk event. // If there are no headers, error out if (!svix_id || !svix_timestamp || !svix_signature) { - return new Response('Error occured -- no svix headers', { + return new Response('Error: Missing Svix headers', { status: 400, }) } - // Get the body + // Get body const payload = await req.json() const body = JSON.stringify(payload) - // Create a new Svix instance with your secret. - const wh = new Webhook(WEBHOOK_SECRET) - let evt: WebhookEvent - // Verify the payload with the headers + // Verify payload with headers try { evt = wh.verify(body, { 'svix-id': svix_id, @@ -162,156 +150,88 @@ This guide can be adapted to listen for any Clerk event. 'svix-signature': svix_signature, }) as WebhookEvent } catch (err) { - console.error('Error verifying webhook:', err) - return new Response('Error occured', { + console.error('Error: Could not verify webhook:', err) + return new Response('Error: Verification error', { status: 400, }) } - // Do something with the payload - // For this guide, you simply log the payload to the console + // Do something with payload + // For this guide, log payload to console const { id } = evt.data const eventType = evt.type - console.log(`Webhook with and ID of ${id} and type of ${eventType}`) - console.log('Webhook body:', body) + console.log(`Received webhook with ID ${id} and event type of ${eventType}`) + console.log('Webhook payload:', body) - return new Response('', { status: 200 }) - } - ``` - - ```ts {{ filename: 'pages/api/webhooks.ts' }} - import { Webhook } from 'svix' - import { WebhookEvent } from '@clerk/nextjs/server' - import { NextApiRequest, NextApiResponse } from 'next' - import { buffer } from 'micro' - - export const config = { - api: { - bodyParser: false, - }, - } - - export default async function handler(req: NextApiRequest, res: NextApiResponse) { - if (req.method !== 'POST') { - return res.status(405) - } - // You can find this in the Clerk Dashboard -> Webhooks -> choose the webhook - const WEBHOOK_SECRET = process.env.WEBHOOK_SECRET - - if (!WEBHOOK_SECRET) { - throw new Error('Please add WEBHOOK_SECRET from Clerk Dashboard to .env or .env.local') - } - - // Get the Svix headers for verification - const svix_id = req.headers['svix-id'] as string - const svix_timestamp = req.headers['svix-timestamp'] as string - const svix_signature = req.headers['svix-signature'] as string - - // If there are no headers, error out - if (!svix_id || !svix_timestamp || !svix_signature) { - return res.status(400).json({ error: 'Error occured -- no svix headers' }) - } - - console.log('headers', req.headers, svix_id, svix_signature, svix_timestamp) - // Get the body - const body = (await buffer(req)).toString() - - // Create a new Svix instance with your secret. - const wh = new Webhook(WEBHOOK_SECRET) - - let evt: WebhookEvent - - // Attempt to verify the incoming webhook - // If successful, the payload will be available from 'evt' - // If the verification fails, error out and return error code - try { - evt = wh.verify(body, { - 'svix-id': svix_id, - 'svix-timestamp': svix_timestamp, - 'svix-signature': svix_signature, - }) as WebhookEvent - } catch (err) { - console.error('Error verifying webhook:', err) - return res.status(400).json({ Error: err }) - } - - // Do something with the payload - // For this guide, you simply log the payload to the console - const { id } = evt.data - const eventType = evt.type - console.log(`Webhook with and ID of ${id} and type of ${eventType}`) - console.log('Webhook body:', body) - - return res.status(200).json({ response: 'Success' }) + return new Response('Webhook received', { status: 200 }) } ``` - ```js {{ filename: 'clerkWebhookHandler.js' }} - import { Webhook } from 'svix' - import bodyParser from 'body-parser' - + ```ts {{ filename: 'index.ts' }} app.post( '/api/webhooks', // This is a generic method to parse the contents of the payload. // Depending on the framework, packages, and configuration, this may be // different or not required. bodyParser.raw({ type: 'application/json' }), - async function (req, res) { - // You can find this in the Clerk Dashboard -> Webhooks -> choose the webhook - const WEBHOOK_SECRET = process.env.WEBHOOK_SECRET - if (!WEBHOOK_SECRET) { - throw new Error('You need a WEBHOOK_SECRET in your .env') + + async (req, res) => { + const SIGNING_SECRET = process.env.SIGNING_SECRET + + if (!SIGNING_SECRET) { + throw new Error('Error: Please add SIGNING_SECRET from Clerk Dashboard to .env') } - // Get the headers and body + // Create new Svix instance with secret + const wh = new Webhook(SIGNING_SECRET) + + // Get headers and body const headers = req.headers const payload = req.body - // Get the Svix headers for verification + // Get Svix headers for verification const svix_id = headers['svix-id'] const svix_timestamp = headers['svix-timestamp'] const svix_signature = headers['svix-signature'] - // If there are no Svix headers, error out + // If there are no headers, error out if (!svix_id || !svix_timestamp || !svix_signature) { - return new Response('Error occured -- no svix headers', { - status: 400, + return void res.status(400).json({ + success: false, + message: 'Error: Missing svix headers', }) } - // Create a new Svix instance with your secret. - const wh = new Webhook(WEBHOOK_SECRET) - let evt // Attempt to verify the incoming webhook // If successful, the payload will be available from 'evt' - // If the verification fails, error out and return error code + // If verification fails, error out and return error code try { evt = wh.verify(payload, { - 'svix-id': svix_id, - 'svix-timestamp': svix_timestamp, - 'svix-signature': svix_signature, + 'svix-id': svix_id as string, + 'svix-timestamp': svix_timestamp as string, + 'svix-signature': svix_signature as string, }) } catch (err) { - console.log('Error verifying webhook:', err.message) - return res.status(400).json({ + console.log('Error: Could not verify webhook:', err.message) + return void res.status(400).json({ success: false, message: err.message, }) } - // Do something with the payload - // For this guide, you simply log the payload to the console + // Do something with payload + // For this guide, log payload to console const { id } = evt.data const eventType = evt.type - console.log(`Webhook with an ID of ${id} and type of ${eventType}`) - console.log('Webhook body:', evt.data) + console.log(`Received webhook with ID ${id} and event type of ${eventType}`) + console.log('Webhook payload:', evt.data) - return res.status(200).json({ + return void res.status(200).json({ success: true, message: 'Webhook received', }) @@ -321,22 +241,22 @@ This guide can be adapted to listen for any Clerk event. - ### Narrow the webhook event to get typing + ### Narrow to a webhook event for type inference - The `WebhookEvent` reflects all possible webhook types. You can narrow down the event to get the types inferred correctly for the event type you are working with. + `WebhookEvent` encompasses all possible webhook types. Narrow down the event type for accurate typing for specific events. - In the following example, the `if` statement will narrow the type to the `user.created` type and using `evt.data` will give you autocompletion and type safety for that event. + In the following example, the `if` statement narrows the type to `user.created`, enabling type-safe access to evt.data with autocompletion. ```ts {{ filename: 'app/api/webhooks/route.ts', del: [1, 2], ins: [[4, 6]] }} - console.log(`Webhook with and ID of ${id} and type of ${eventType}`) - console.log('Webhook body:', body) + console.log(`Received webhook with ID ${id} and event type of ${eventType}`) + console.log('Webhook payload:', body) if (evt.type === 'user.created') { console.log('userId:', evt.data.id) } ``` - If you want to handle types yourself, you can import the following types from your backend SDK, such as `@clerk/nextjs/server`. + To handle types manually, import the following package from your backend SDK (e.g., `@clerk/nextjs/server`): - `DeletedObjectJSON` - `EmailJSON` @@ -347,33 +267,35 @@ This guide can be adapted to listen for any Clerk event. - `SMSMessageJSON` - `UserJSON` - ### Test your webhook + ### Test the webhook 1. Start your Next.js server. - 1. On your endpoint's settings page in the Clerk Dashboard, select the **Testing** tab. - 1. In the **Select event** dropdown, select `user.updated`. - 1. Select the **Send Example** button. - 1. Below that section, in the **Message Attempts** section, you should see a successful attempt with a status of `200`. + 1. In your endpoint's settings page in the Clerk Dashboard, select the **Testing** tab. + 1. In the **Select event** dropdown, select `user.created`. + 1. Select **Send Example**. + 1. In the **Message Attempts** section, confirm that the event is labeled with `Succeeded`. - #### Message failed + #### Handling failed messages - If the message failed: - - 1. Select the message. + 1. In the **Message Attempts** section, select the event labeled with `Failed`. 1. Scroll down to the **Webhook Attempts** section. - 1. Select the arrow icon the left side. - 1. Investigate the error. Your solution is going to depend on the error message. See the [Debug your webhooks](/docs/integrations/webhooks/debug-your-webhooks) guide for more information. + 1. Toggle the arrow next to the **Status** column. + 1. Review the error. Solutions vary by error type. For more information, refer to the [Debug your webhooks](/docs/integrations/webhooks/debug-your-webhooks) guide. - ### Trigger your webhook + ### Trigger the webhook - To trigger the `user.updated` event, you can do one of the following: + To trigger the `user.created` event, you can do either one of the following: - 1. You can edit your user in the Dashboard. - 1. You can use the `` component in your application to edit your profile. + 1. Edit your user in the Clerk Dashboard. + 1. Select the `` component in your app to edit your profile. - Once you have updated a user, you should be able to see the webhook's payload logged to your terminal. You can also check the Clerk Dashboard to see the webhook attempt, the same way you did when [testing the webhook](#test-your-webhook). + You should be able to see the webhook's payload logged to your terminal. You can also check the Clerk Dashboard to see the webhook attempt, the same way you did when [testing the webhook](#test-the-webhook). + - ### Wrap up +### Configure your production instance - In this guide, you learned how to create a Clerk webhook using Svix. You created a webhook in the Clerk Dashboard to listen for the `user.updated` event and created a Route Handler for your webhook endpoint to verify the webhook signature and receive the payload. You tested the webhook locally using ngrok and the Clerk Dashboard to ensure it was configured properly. And lastly, you added logic to your application to actually trigger the `user.updated` event. Now, you can do whatever you want with the webhook payload, such as sending a notification to your users, updating a database, or anything else you can think of. - +1. When you're ready to deploy your app to production, follow [the guide on deploying your Clerk app to production](/docs/deployments/overview). +1. Create your production webhook by following the steps in the previous [Set up a webhook endpoint](#set-up-a-webhook-endpoint) section. In the **Endpoint URL** field, instead of pasting the ngrok URL, paste your production app URL. +1. After you've set up your webhook endpoint, you'll be redirected to your endpoint's settings page. Copy the **Signing Secret**. +1. On your hosting platform, update your environment variables on your hosting platform by adding **Signing Secret** with the key of `SIGNING_SECRET`. +1. Redeploy your app.