Build scalable event-driven applications with Azure and Postgres change data capture (CDC).
This guide shows you how to set up Postgres change data capture (CDC) and stream changes to Azure Event Hubs using Sequin.With Postgres data streaming to Event Hubs, you can trigger workflows, keep services in sync, build audit logs, maintain caches, and more.By the end of this how-to, you’ll have database changes flowing to an Event Hub.
This is the how-to guide for streaming Postgres to Azure Event Hubs. See the quickstart for a step-by-step walkthrough or the reference for details on all configuration options.
Under “Source”, select the table or schema you want to stream data from.
2
Add filters (optional)
Add filters to the sink to control which database changes are sent to your Event Hub.
3
Specify backfill
You can optionally indicate if you want Event Hubs to receive a backfill of all or a portion of the table’s existing data. Backfills are useful if you want to use Event Hubs to process historical data.You can backfill at any time. If you don’t want to backfill, toggle “Backfill” off.
4
Specify message grouping
Under “Message grouping”, you can specify how messages should be grouped for ordering. Messages in the same group will be delivered in order according to their commit timestamp.By default, Sequin uses the source row’s primary key(s) for message grouping. You can override this by specifying one or more columns.
Currently, Sequin does not set the partition key when sending messages to Event Hubs. This means that even messages with the same group may be sent to different partitions. If you need partition key support, please upvote the feature request.
Setup a processorNow that your Postgres data is flowing into Event Hubs, you can setup a consumer to read from the Event Hub and process the data. See the Event Hubs documentation for details on consuming events.
Deploy your implementationWhen you’re ready to deploy your implementation, see “How to deploy to production”.
Advanced configurationFor more about how Event Hub sinks work, see the Event Hub sink reference.