Using Sequin to fan out events
How to fan out events to multiple services.
When a customer places an order, you need to update your inventory, schedule a shipment, and generate an invoice:
- Sequin captures every
insert
on yourorders
table using a durable replication slot. - It then asynchronously queues messages for each service to ensure every order is processed by each service.
- With a simple HTTP interface, replays, and observability built in you can easily scale this implementation.
This is a common “fan out” pattern. This guide shows you how to implement fan outs with Sequin.
You could implement this pattern as a side effect by sending a single message to a single service that then triggers multiple actions. However, by fanning out the order as unique messages you can ensure that each service is scaled and managed independently. You also gain the benefits of durable retries and observability for each service through Sequin.
Implementation
Connect your database to Sequin
- Login to your Sequin account and click the Connect Database button on the dashboard.
- Paste the connection string for your database (or enter your connection credentials).
- Create a replication slot and publication for Sequin using the SQL commands in the console.
- Name your database and click Connect Database to finish.
Your database is now connected to Sequin. If you’d like a step-by-step guide, check out our Quickstart.
Create three webhook subscriptions to fan out messages
Create three webhook subscriptions (one for inventory, one for shipping, and one for invoicing) that capture changes from the orders
table and delivers them to the relevant service:
- Click the Create Consumer button and then select the
users
table. - Because you want to capture every
insert
you can leave the Filter field blank. - Select the Changes option and then choose Push for the consumer type.
- Enter the the HTTP endpoint where Sequin will send the changes. Add any headers you need to authenticate the request.
- Click Create Consumer to finish.
Your consumer is now created. If you’d like a step-by-step guide, check out our Quickstart. You could easily use pull consumers here too.
Handle messages in each service
Sequin is now capturing every insert
on the orders
table and fanning out messages to your HTTP endpoints:
insert into users (name, email) values ('John Doe', 'john.doe@example.com');
Sequin will send the new orders to your webhook endpoints:
{
"record": {
"id": 1,
"customer_id": 42,
"product_id": 123,
"quantity": 2,
"total_price": 99.99,
"shipping_address": "123 Main St, Anytown, USA",
"order_date": "2023-10-15T14:30:00Z"
},
"changes": null,
"action": "insert",
"metadata": {
"table_schema": "public",
"table_name": "orders",
"commit_timestamp": "2023-10-15T14:30:00Z",
"consumer": {
"id": "f3e8b2a1-9d7c-4b6a-8f5e-2d1c3b4a5e6f",
"name": "inventory_consumer" // or shipping_consumer or invoicing_consumer
}
}
}
You can handle this message in your application and send a welcome email:
// Webhook handler for scheduling shipment
app.post('/webhook/schedule-shipment', async (req, res) => {
const { id, customer_id, shipping_address, order_date } = req.body.record;
try {
// Schedule shipment
await scheduleShipment(id, customer_id, shipping_address, order_date);
res.status(200).send('Shipment scheduled successfully');
} catch (error) {
res.status(500).send('Error scheduling shipment');
}
});
Importantly, to ensure that every new order is processed by each service - Sequin will retry message until the HTTP request is acknowledged with a 200 status.
Benefits
Fanning out events to multiple services has several benefits:
- Asynchronous: You can process events asynchronously without blocking your application.
- Scalable: You can easily scale this implementation with a simple HTTP endpoint.
Sequin handles the hard parts for you:
- Transactional guarantees: Sequin ensures that every new order is processed by each service.
- Reliable retries: Sequin will retry failed HTTP requests until they are acknowledged.
- No custom infrastructure: You don’t need to run any custom infrastructure to send emails.
- Observability: You can easily see the status of your messages and re-send failed messages.
Next steps
To add a fan out pattern to your application, you might want to:
- Go through our Quickstart guide to connect your database to Sequin and create a consumer.
- Learn more about how Sequin works.
Was this page helpful?