How to maintain caches with Sequin
Learn how to keep Redis compatible caches in sync with your Postgres database. Eliminate stale cache issues with real-time change data capture.
This guide shows you how to keep your Redis-compatible caches in sync with your Postgres database in real-time using Sequin.
By the end of this guide, you’ll have a working sink that automatically creates, updates, and deletes cache entries as your database changes, eliminating stale cache issues.
When to use this approach
This approach works well when:
- You need real-time cache synchronization
- You’re maintaining multiple caches across different regions
- You can tolerate 100ms of latency (changes appear in cache shortly after database)
This approach is not a good fit if:
- Your cache isn’t Redis-compatible. Instead, you should sink to a stream or queue destination and process the messages in your own system.
- You need read-after-write consistency. In this case, you may want to write to the cache directly from your application.
Prerequisites
If you’re self-hosting Sequin, you’ll need:
- Sequin installed or a Sequin Cloud account
- A database connected to Sequin
- A Redis compatible cache (Redis, KeyDB, Dragonfly, Valkey etc.)
Create a Redis String sink
To get started, navigate to the “Sinks” page in the Sequin UI and click “Create Sink”. Select “Redis String” as the sink type. Configure the sink as follows:
Select the source
- Select your database from the connected databases list.
- Choose your table or schema containing the data you want to cache.
Apply filters
Keep all operations enabled (insert, update, delete) to maintain cache consistency.
If there are rows in the source table you don’t need to cache (for instance, rows that are only used for reporting or testing), you can apply a filter to exclude them.
Transform the data
Configure a transform function to store only the data you need in your cache:
You can be more selective and only store certain fields in your cache:
Warm the cache with a backfill
To warm your cache with data currently in your source table, you can start the sink with an initial backfill.
If you only need to warm your cache with a subset of your source table, skip this step for now and run an incremental backfill after the sink is running.
Group by primary key
by default, Sequin will group messages by primary key. This is a good default for most use cases as this will ensure every change to a specific row is processed in order.
If you need to more specific message grouping, you can configure the sink to group messages by a different field.
Configure routing
By default, keys follow the pattern sequin:<table-name>:<primary-key>
To customize keys, enable “Dynamic routing” and add a routing function:
This creates keys like users:123 instead of sequin:users:123.
Enter the connection details for your cache
Enter the configuration parameters for your cache.
When maintaining a cache, you’ll typically want to keep the “Expires” value to 0
so that the cache entries never expire.
Create the sink
Give you sink a name and click “Create Sink”.
Verify your cache is being updated
If you specified a backfill, there should be messages in your sink ready for your system to process:
-
On the sink overview page, click the “Messages” tab. You should see messages flowing to your sink.
-
Query your cache to verify that the data is being updated:
Re-warming your cache
You may need to re-warm your cache in these scenarios:
- After infrastructure changes: Cache server upgrades/migrations
- Data model updates: Schema or business logic changes
- Cache eviction: Unexpected cache flushes
To do so, run a backfill. You can trigger a backfill manually in the Sequin Console or using the Sequin management API.
Next steps
You now have a cache that automatically stays in sync with your database. Changes to your Postgres data will appear in your cache within milliseconds, without any manual intervention or risk of stale data.
Here are some helpful resources to bring this approach to production:
Connect your production database
Learn how to connect your production database to Sequin.
Deploy Sequin in production
Learn how to deploy Sequin in production.
Learn more about Redis String sinks
Learn more about how to configure your Redis String or Redis compatible sink.
Monitor your workflows
Learn how to monitor your system with Sequin’s built-in metrics.