Trigger AWS Lambdas from database changes
How to use a webhook sink to trigger AWS Lambdas from database changes.
AWS Lambda is a serverless compute service that lets you run code without provisioning or managing servers. It automatically scales your applications in response to incoming requests.
Often, you want to trigger an AWS Lambda function when a database row changes. For example, you may want to trigger a function as a side-effect of a database change, or fan out work to multiple services.
In this guide, you will learn how to setup a Sequin webhook sink to trigger an AWS Lambda function when a database row changes.
Prerequisites
You are about to create a simple AWS Lambda function that logs a message to the console. You’ll trigger this function by setting up a Sequin webhook sink that sends a HTTP POST request to the function’s URL with the payload of the database row that changed.
You’ll need the following:
- An AWS account
- Sequin installed locally or a Sequin Cloud account
- A database connected to Sequin
Create a Lambda function
Start by creating a new AWS Lambda function that takes in a Sequin change event as a payload and logs the payload to the console.
Create a new Lambda function
- Open the AWS Lambda console and click “Create function”.
- Choose “Author from scratch”.
- Give your function a name (e.g., “newUserHandler”).
- Select a runtime (e.g., “Node.js 20.x”) and which ever architecture you want to support (e.g., “arm64”).
- Click “Create function”:
Add function code
Replace the default code in the Lambda function. Here’s an example:
This function first checks the authorization header to make sure the request is coming from Sequin. Then it processes the payload and logs the name from the payload to the console.
Click “Deploy” to save the function.
Add the SEQUIN_WEBHOOK_SECRET environment variable
- In the Lambda function “Configuration” tab, scroll down to the “Environment variables” section.
- Click “Edit” and then “Add environment variable”.
- Set the key as
SEQUIN_WEBHOOK_SECRET
and the value to a secure secret of your choice. - Click “Save”.
You will need to use this secret value in the Sequin dashboard when you create the push consumer.
Create a Lambda Function URL
To make your Lambda function accessible via HTTP, you need to create a Function URL:
- Go to your Lambda function in the AWS Console.
- In the “Configuration” tab, click on “Function URL” in the left sidebar.
- Click “Create function URL”.
- For “Auth type”, select “NONE”.
- Under “Configure cross-origin resource sharing (CORS)”, check “Configure CORS”.
- In the “Allowed origins” field, enter ”*” (without quotes) to allow all origins for now. You can restrict this later.
- Click “Save”.
After saving, you’ll see a Function URL. This is the URL you’ll use to configure your Sequin consumer in the next section.
You’ve successfully created an AWS Lambda function that logs a message to the console when Sequin sends a HTTP POST request to the function’s URL.
Create a webhook sink
Create a webhook sink that captures changes to your database and sends a HTTP POST request to the Lambda function’s URL:
Create a new sink
Navigate to the “Sinks” tab, click the “Create Sink” button, and select “Webhook Sink”.
Select source table
Select the table you want to capture changes from (i.e public.users
).
Choose message type
Specify whether you want to receive changes or rows from the table.
If you’re not sure which to choose, leave the default setting of “Changes”. Changes are a better fit for event-driven use cases, like triggering workflows, which is usually what you want to do with webhooks.
Specify change types
If you selected changes, in “Records to process”, you can indicate whether you want to receive insert
, update
, and/or delete
changes. Select all three for now. No need to add a filter.
Configure backfill
You can optionally indicate if you want your webhook endpoint to receive a backfill of all or a portion of the table’s existing data. For now, leave “Backfill” toggled off.
Configure message grouping
Under “Message grouping”, leave the default option selected to ensure events for the same row are sent to your webhook endpoint in order.
Configure delivery settings
Under “Delivery configuration”, choose a conservative value for “Request timeout” and set “Batch size” to 1 since we’re just logging messages.
Configure HTTP endpoint
Under “HTTP Endpoint”, enter the Lambda Function URL you obtained earlier. Then click to “Add Encrypted Header” with the key Authorization
and the value Bearer SEQUIN_WEBHOOK_SECRET
, using the secret value you set in your Lambda function’s environment variables.
Name and create sink
Give your sink a name (i.e. users_webhook_sink
) and click “Create Webhook Sink”.
Your Sequin consumer is now created and ready to send events to your Lambda function.
Test end-to-end
Add a row to your table
Insert a row into your table. For example:
Trace the change in the Sequin dashboard
In the Sequin console, open the Messages tab on your webhook sink and confirm that a message was delivered:
Confirm the event was received by your Lambda function
- Open the AWS Lambda console and navigate to your function.
- Click on the “Monitor” tab and then “View CloudWatch logs”.
- In the most recent log stream, you should see a log entry:
Hello John Doe
:
You’ve successfully triggered an AWS Lambda function from a database change!
Next steps
Modify this example to suit your needs:
- If you need to run long-running jobs, consider using AWS Step Functions in tandem with Lambda functions.
- Tune your webhook sink configuration to suit your volume of work.
- Implement additional security measures, such as API Gateway request validation.