Stream Postgres changes to Tinybird
How to use Sequin to capture Postgres changes and stream them to Tinybird in real-time.
Tinybird is a powerful real-time analytics platform built on ClickHouse. It allows you to ingest, transform, and query large volumes of data with low latency.
Often, you want to stream changes from your Postgres database to Tinybird for real-time analytics. For instance, you might want to analyze order data as it’s created or updated in your database:
In this guide, you’ll learn how to use Sequin to stream Postgres changes to Tinybird in real-time.
Prerequisites
To follow this guide, you’ll need:
- A Sequin account
- A Tinybird account
- A Postgres database (Sequin works with any Postgres database version 12 and up)
Connect Sequin to your database
First, you’ll need to connect Sequin to your Postgres database:
Add a new database in Sequin
- Log in to your Sequin account and click the Add New Database button.
- Enter the connection details for your Postgres database.
- Follow the instructions to create a publication and a replication slot by running two SQL commands in your database:
- Name your database and click the Connect Database button.
Sequin will connect to your database and ensure that it’s configured properly.
You’ve successfully connected Sequin to your Postgres database. In the next step, you’ll set up Tinybird to receive data from Sequin.
Setup a Tinybird Event URL
Now, you’ll create a Data Source in Tinybird that will receive the data from Sequin:
Create a Data Source in Tinybird
- Log in to your Tinybird account and open the Data Sources tab.
- Click on Create Data Source.
- Choose Events API as the source type.
- Copy the endpoint URL and Authorization header. You’ll need this in the next step.
You’ve created a Data Source in Tinybird that’s ready to receive data. Next, you’ll configure Sequin to send data to this endpoint.
Create Sequin HTTP Endpoint
Now, you’ll create an HTTP endpoint in Sequin that will send data to the Tinybird Event URL you created in the previous step:
Create an HTTP Endpoint in Sequin
- In the Sequin console, navigate to the HTTP Endpoints tab.
- Click on Create HTTP Endpoint.
- Name your endpoint (e.g., “tinybird_endpoint”).
- Paste the Tinybird endpoint URL you copied earlier into the Base URL field.
- Add an encrypted header:
- Key:
Authorization
- Value:
Bearer YOUR_TINYBIRD_TOKEN
- Key:
- Click Create HTTP Endpoint.
You’ve set up an HTTP endpoint in Sequin that will securely send data to Tinybird. In the final step, you’ll create a consumer to capture database changes and send them to this endpoint.
Create Sequin Consumer
Finally, you’ll create a Sequin consumer to capture changes from your database and send them to Tinybird:
Create a Push Consumer in Sequin
- In the Sequin console, navigate to the Consumers tab.
- Click on Create Consumer.
- Select the table you want to stream (e.g., “orders”).
- Choose to process Rows. This will capture all inserts and updates.
You can also choose to process Changes instead of rows. This will capture changes to the table, such as inserts, updates, and deletes.
- Select a column to sort by.
- Choose Push as the consumer type.
- Select the HTTP endpoint you created in the previous step.
- Name your consumer and click Create Consumer.
Congratulations! You’ve set up a complete pipeline to stream changes from your Postgres database to Tinybird in real-time.
Test the Integration
To test your integration:
- Insert or update some data in your Postgres database.
- Check the Sequin Trace tab to confirm that the changes were captured and sent to Tinybird.
- In Tinybird, query your Data Source to see the new data.
Next Steps
Now that you have real-time data flowing from Postgres to Tinybird, you can:
- Create Pipes in Tinybird to transform and analyze your data.
- Set up API endpoints in Tinybird to query your data.
- Use Tinybird’s dashboard features to visualize your data in real-time.
By combining the transactional power of Postgres with the analytical capabilities of Tinybird, you’ve created a powerful real-time data pipeline for your application.
Was this page helpful?