Reference for the core concepts and behaviors of Sequin sinks. Learn about delivery guarantees, message grouping, ordering, backfill support, and monitoring capabilities.
max_retry_count
parameter. If a message fails delivery after the specified number of retries, it will be discarded.
message_grouping
configuration option:
message_grouping: true
(default): Enable message grouping for ordered deliverymessage_grouping: false
: Disable message grouping for maximum throughputmessage_grouping
is enabled, Sequin groups messages by the primary key value(s) of the source row by default. This means that if a row is updated multiple times, each update event will arrive in order to your sink.
When message_grouping
is disabled, messages are not grouped, allowing for higher throughput batching but without ordering guarantees.
group_column_names
at the table level. You can group rows by a single field or multiple fields.
For example, instead of grouping by primary key, you could group orders
by account_id
. This means that all changes for a given account_id
will be sent to your sink in the order they were committed in Postgres.
load_shedding_policy
will be triggered. There are two available load shedding policies:
pause_on_full
(default) — pauses replication for all sinks on the slot until the buffer has room again. In this case, Postgres guarantees no lost messages. However, in order to do this, Postgres has to save the messages on disk. This means the replication slot can grow without bound. Unmanaged, this will cause your database to run out of disk space.
discard_on_full
— Sequin will continue reading messages from the replication slot. However, messages intended for the overloaded sink(s) will be dropped. Other sinks will not be impacted. This limits possibility that your database will run out of disk space but will cause sinks to be inconsistent with the state of the database.