This is the how-to guide for streaming Postgres to Meilisearch. See the quickstart for a step-by-step walkthrough or the reference for details on all configuration options.
Prerequisites
If you’re self-hosting Sequin, you’ll need:Basic setup
Prepare your Meilisearch instance
Sequin converts Postgres changes and rows into JSON documents and sends them to your Meilisearch index. You’ll need to have a Meilisearch instance running and accessible to Sequin. You can run Meilisearch locally for development or use a hosted version.If you’re developing locally, you can connect Sequin to a local Meilisearch instance.
Mapping Postgres tables to Meilisearch indexes
From the Meilisearch docs:In Meilisearch, each index contains documents with the same structure.Sequin recommends creating one sink per Postgres table. Each sink will stream changes from a single Postgres table to a single Meilisearch index. This ensures schema consistency across indexed documents. For large-scale applications, you may consider sharding indexes—e.g., sharding a
users
index by region into users_us
, users_eu
, users_asia
, etc.
You can use Sequin’s filters to accomplish this. For example, to shard users
by region, create a sink per region with filters to route the correct rows to the corresponding Meilisearch index.
Advanced: Using routing functions
Sequin supports routing functions for dynamically routing rows and performing advanced operations. With routing functions, you can:- Route rows from one table to multiple Meilisearch indexes
- Perform function-based updates (increments, array modifications, etc.)
- Apply conditional logic for complex routing scenarios
Create Meilisearch sink
Navigate to the “Sinks” tab, click “Create Sink”, and select Meilisearch Sink.Configure the source
1
Select source table or schema
Under “Source”, select the table or schema you want to stream data from.
2
Specify filters
Choose whether to include
insert
, update
, and/or delete
actions. For full sync, select all three.For example, if you uncheck deletes
, rows deleted from Postgres won’t be removed from the Meilisearch index.You can also use filters to only include certain rows—e.g., WHERE in_stock = true
for product availability or for sharding by tenant.3
Specify backfill
Optionally trigger a backfill to populate your Meilisearch index with existing data from your table. This is optional at sink creation and can be done later.
4
Transforms
Meilisearch requires a defined schema with:For dynamic schemas or minimal transformation:Or explicitly assign the
- A unique document primary key (string or integer).
- Searchable and filterable fields.
id
:5
Specify message grouping
Keep the default setting to ensure updates to the same row are processed in order. This helps maintain consistent state in Meilisearch.
Configure delivery
1
Specify batch size
The right value for “Batch size” depends on your data and your requirements.Meilisearch supports bulk document addition via the
/documents
endpoint. A good starting point is 100–1000 documents per batch. Very large batches (10k+) may impact memory or throughput depending on your instance size.2
Select the Meilisearch index you created
Under “Meilisearch Index”, select the index you want to stream data to.
3
Create the sink
Give your sink a name, then click “Create Meilisearch Sink”.
Verify & debug
To verify that your Meilisearch sink is working:- Make some changes in your source table.
- Verify that the count of messages for your sink increases in the Sequin web console.
- Search in your Meilisearch index to confirm the document changes. (You can use the meilisearch-ui for a simple UI to interact with your Meilisearch indexes/tasks)
- Click the “Messages” tab to view the state of messages for your sink.
- Click any failed message.
- Check the delivery logs for error details, including the response from Meilisearch.
Advanced use cases
Using function updates for counters and aggregations
Meilisearch’s function update feature, combined with Sequin’s routing functions, enables powerful use cases like maintaining counters or aggregated values without re-indexing entire documents. For example, if you have aproducts
index and want to update view counts or inventory levels: