A sink in Sequin is a destination for your Postgres data.

​

Common sink behaviors

​

Guaranteed delivery

​

Delivery Guarantees

Sequin provides exactly-once delivery under most circumstances, but exactly-once delivery is not always possible. Therefore, Sequin implements at-least-once delivery with idempotency keys.

If a message is not delivered successfully, Sequin will automatically retry with exponential backoff, increasing the retry interval up to a maximum of 3 minutes between attempts.

By default, Sequin retries messages indefinitely. You can configure a maximum number of retries for a sink by setting the max_retry_count parameter. If a message fails delivery after the specified number of retries, it will be discarded.

​

Message grouping and ordering

Sequin has the ability to group messages by one or more fields. Changes for rows in the same group are guaranteed to arrive in the order they occurred (i.e. the order they were committed in Postgres) to your sink.

By default, Sequin groups messages by the primary key value(s) of the source row. This means that if a row is updated multiple times, each update event will arrive in order to your sink.

You can specify message grouping behavior when configuring a sink. You can group rows by a single field or multiple fields.

So, for example, instead of grouping by primary key, you could instead group orders by account_id. This means that all changes for a given account_id will be sent to your sink in the order they were committed in Postgres.

All this applies to how Sequin delivers messages to your sink. Whether your sink supports ordered/FIFO processing depends on the sink. See the reference page for your sink for details.

​

Backfill support

Sequin supports backfilling to sinks. See the backfills reference for more details.

​

Monitoring and debugging

You can view the status of sinks in the Sequin web console.

All sinks have a “Messages” tab that shows the status of messages currently in flight to the sink, recently delivered messages, and messages that have failed delivery.

You can click into any failed message to view the delivery logs and error details.

​

Advanced configuration

​

Max memory usage

When sink destinations can’t keep up with incoming messages, Sequin continues to advance the Postgres replication slot and buffer messages in memory.

By default, each sink is set to a max memory usage of 128MB. You can configure the max memory usage for a sink by setting the max_memory_mb parameter in the sink configuration.

​

Load shedding policy

If a sink’s in-memory buffer fills up, the load_shedding_policy will be triggered. There are two available load shedding policies:

pause_on_full (default) — pauses replication for all sinks on the slot until the buffer has room again. In this case, Postgres guarantees no lost messages. However, in order to do this, Postgres has to save the messages on disk. This means the replication slot can grow without bound. Unmanaged, this will cause your database to run out of disk space.

discard_on_full — Sequin will continue reading messages from the replication slot. However, messages intended for the overloaded sink(s) will be dropped. Other sinks will not be impacted. This limits possibility that your database will run out of disk space but will cause sinks to be inconsistent with the state of the database.