Redis Stream sink

The Redis Stream sink sends messages to a Redis stream using the XADD command.

​

Configuration

• Host

  The hostname of your Redis server (e.g., localhost).

• Port

  The port number for your Redis server (default: 6379).

• Stream Key

  The key name for the Redis stream where messages will be sent. Maximum length is 255 characters.

• Database

  The Redis database number to use (default: 0).

• Username (optional)

  Username for Redis authentication.

• Password (optional)

  Password for Redis authentication.

• TLS

  Enable TLS encryption for connections to Redis.

​

Advanced configuration

• Max Ack Pending

  Sets the maximum number of messages that can be in-flight at any time. This helps control message flow and prevents overwhelming Redis.

​

Message format

Sequin sends messages to Redis streams as JSON using the XADD command. You can find the shape of the messages in the messages reference.

​

Transforms

You can use transforms to customize the fields sent to Redis Streams. The transform function should return a map where:

• Each key becomes a field name in the Redis Stream entry
• String values are stored as-is
• Non-string values are JSON-encoded

Example transform:

def transform(_action, record, changes, metadata) do
  %{
    id: record["id"],
    name: record["name"],
    changes: changes,
    table: metadata.table_name
  }
end

This would result in a Redis Stream entry like:

XADD mystream * id 123 name "John" changes "{\"email\":\"newemail@domain.com\"}" table "users"

​

Retry behavior

If Sequin is unable to deliver a message to Redis, it will retry the message indefinitely. Sequin will exponentially back off the retry interval, with a maximum backoff of roughly 3 minutes.

​

Message Size Limits

The maximum size of a Redis string value is 512MB. However, it’s recommended to keep messages significantly smaller for better performance.

​

Message ordering

Redis streams maintain message order within a single stream. Messages are assigned auto-incrementing IDs of the form <timestamp>-<sequence number> by Redis when they are added to the stream, ensuring strict ordering of messages.

Sequin XADDs messages to the Redis stream in the same order that they are received from your source table. Therefore, you can rely on the ordering of messages by their ID to ensure correct processing order.

For example, if you are using a Redis stream as an audit log, you can process messages in order by their ID to reconstruct the state of the audited object at any point in time. Or if you are maintaining a materialized view of the stream, you can safely upsert to the view as long as the incoming ID is greater than the last processed ID.

​

Routing

The Redis Stream sink supports dynamic routing of the stream_key with routing functions.

Example routing function:

def route(action, record, changes, metadata) do
  %{stream_key: metadata.table_name}
end

When not using a routing function, messages will be appended to the statically configured stream.

​

Security

Sequin supports the following security features for Redis connections:

• TLS encryption
• Username/password authentication

Make sure your Redis server is properly configured to accept the security settings you enable in Sequin.

​

Debugging

You can view the status of your Redis Stream sink in the Sequin web console.

On the “Messages” tab, you can see which messages are in-flight to Redis, which messages Sequin is unable to deliver, and recently delivered messages.

Messages that Sequin is unable to deliver will have a “Deliver count” greater than 1. You can click on a message to see more details, including the last error response received from Redis.