A backfill is a process that streams current rows from a table to a sink. You can use a backfill to:
  • Seed a sink: Stream the current state of a table to a sink.
  • Replay: Re-process rows that have already been sent to a sink.

​
Creating a backfill

During sink creation, you can optionally specify a backfill. Sequin will run the backfill first, then continue to stream any changes to the sink. After a sink is created, you can manually trigger a backfill on the sink’s backfill tab at any time. You can backfill all tables or specific tables.

​
Backfill messages

Sequin will send a message with the action read for each row in the table.
Backfills only include the current state of the row at the time of the backfill. You cannot backfill or replay atomic inserts, updates and deletes to your table.To enable backfilling of specific inserts, updates and deletes, you can set up change retention.

​
Full and partial backfills

When creating a backfill, you can choose either:
  • Full backfill: Stream the whole table.
  • Partial backfill: Stream starting at a specific point in the table.
In both cases, Sequin will read rows from the table concurrently with capturing new changes.
Read more about how Sequin backfills work in our blog post.

​
Partial backfills

To bacfkill from a specific point in the table you choose both a sort column and a start position for that column. For instance, if you choose updated_at as the sort column, you can specify a start position of 2025-01-01 00:00:00.
We highly recommend using a column that is indexed as the sort column.
Partial backfills are particularly useful for disaster recovery. For instance, let’s say that your Sequin instance is delivering data to AWS SQS. A bug in your SQS consumer code processes messages incorrectly from SQS for several hours. Once processed, messages are removed from SQS. When you deploy a fix for the bug, you can initiate a partial backfill to replay messages that were incorrectly processed. Sequin will re-deliver these messages to your sink, where the updated consumer code will process them correctly.

​
Backfill ordering

By default, backfills order rows by primary keys of the table. This ensures that select queries are performant, as primary keys are always indexed. For partial backfills, you choose a sort column. Sequin will order rows by the sort column. You may choose any column as the sort column, but we strongly recommend using a column that is indexed.

​
Query timeout

Sequin automatically tunes batch sizes in its select queries to maximize throughput during backfills (i.e. messages per second). This setting controls the maximum amount of time each select query should take. Sequin uses a PageSizeOptimizer that will decrease page sizes as necessary to keep select queries below this limit. This ensures consistent performance and prevents timeouts even when processing large batches of data. If you’re unsure, the default of 5000ms (5 seconds) is fine to start.