This guide shows you how to set up Postgres change data capture (CDC) and stream changes to Typesense using Sequin.

With Postgres data streaming to Typesense, you can build real-time search experiences, maintain up-to-date search indexes, and provide fast, typo-tolerant search for your users.

By the end of this how-to, you’ll have a configured Typesense collection that stays in sync with your database.

​
Prerequisites

If you’re self-hosting Sequin, you’ll need:

  1. Sequin installed
  2. A database connected

If you’re using Sequin Cloud, you’ll need:

  1. A Sequin Cloud account
  2. A database connected

​
Basic setup

​
Prepare your Typesense instance

Sequin converts Postgres changes and rows into JSON documents and sends them to your Typesense collection.

You’ll need to have a Typesense instance running and accessible to Sequin. You can run Typesense locally for development or use a hosted instance.

If you’re using Sequin Cloud and developing locally, you can use the Sequin CLI’s tunnel command to connect Sequin’s servers to a local Typesense instance. So, no need to deploy your Typesense instance just yet!

​
Create Typesense collection

Under “Resources”, navigate to the “Typesense Collections” tab and click “Create Collection”.

When setting up a Typesense sink, you’ll need to specify:

  1. The collection name
  2. The schema for your documents
  3. The API key for authentication

​
Create Typesense sink

Navigate to the “Sinks” tab, click “Create Sink”, and select “Typesense Sink”.

​
Configure the source

1

Select source table

Under “Source”, select the table you want to stream data from.

2

Choose message type

Specify whether you want to receive changes or rows from the table.

If you’re not sure which to choose, leave the default setting of “Changes”. Changes are a better fit for keeping your search index in sync with your database.

3

Specify filters

If you selected changes, in “Records to process”, you can indicate whether you want to receive insert, update, and/or delete changes.

You can also specify SQL filters to narrow down the documents you want to index. For example, if you only want to index products that are currently in_stock, you can add a filter on in_stock = true.

4

Specify backfill

You can optionally indicate if you want your Typesense collection to receive a backfill of all or a portion of the table’s existing data. Backfills are useful if you want your search index to include historical data.

You can backfill at any time. If you don’t want to backfill, toggle “Backfill” off.

5

Specify message grouping

Under “Message grouping”, you’ll most likely want to leave the default option selected to ensure events for the same row are processed in order.

While your Typesense collection is processing a message, if a new change or version related to the same row is captured, Sequin will hold the message back until the first message is processed. This is almost always the desired behavior.

​
Configure delivery

1

Specify request timeout

Under “Delivery configuration”, choose a conservative value for “Request timeout”. The right value depends on how long you want to give Typesense to process a batch of documents. When the request timeout is reached, Sequin will mark the message for re-delivery.

If your Typesense instance allows it, we recommend setting a timeout limit below the “Request timeout” specified here. That way, if processing is taking too long, it will be killed before Sequin retries the message.

2

Specify batch size

The right value for “Batch size” depends on your requirements.

You should try to process the full batch atomically.

For indexing documents in Typesense, set this to a value that balances throughput with memory usage. A good starting point is 100-1000 documents per batch.

3

Select the Typesense collection you created

Under “Typesense Collection”, select the collection you created earlier.

4

Create the sink

Give your sink a name, then click “Create Typesense Sink”.

​
Verify & debug

To verify that your Typesense sink is working:

  1. Make some changes in your source table.
  2. Verify that the count of messages for your sink increases in the Sequin web console.
  3. Verify receipt of documents in your Typesense collection by searching for them.

If documents don’t seem to be flowing:

  1. Click the “Messages” tab to view the state of messages for your sink.
  2. Click any failed message.
  3. Check the delivery logs for error details, including the response from Typesense.

​
Next steps

Assuming you’ve followed the steps above for your local environment, “How to deploy to production” will show you how to deploy your implementation to production.