In this quickstart, you’ll create a real-time data pipeline that streams changes from a Postgres database to a Google Cloud Pub/Sub topic. You’ll:

  • Boot Sequin
  • Connect to a sample playground database
  • Configure a Pub/Sub topic to receive database changes
  • See your changes flow in real-time

By the end, you’ll have hands-on experience setting up Postgres change data capture (CDC) with Sequin. This same pattern can be used to setup your own Postgres CDC pipeline with Google Cloud Pub/Sub.

​
Prerequisites

  • A Google Cloud project with Pub/Sub enabled
  • A service account with Pub/Sub permissions

For this quickstart, you can use an existing service account with broad Pub/Sub permissions. For production use cases, we recommend creating a dedicated service account for Sequin with these specific roles:

  • roles/pubsub.publisher
  • roles/pubsub.viewer

For more information on creating a service account, see the how-to guide for Pub/Sub.

Create a Pub/Sub Topic

If you don’t already have a Pub/Sub topic:

  1. Navigate to the Pub/Sub Topics page in your Google Cloud Console
  2. Click “CREATE TOPIC”
  3. Enter a topic ID for this tutorial (e.g., “products”)
  4. Click “CREATE”

Make note of your:

  • Google Cloud project ID (e.g., “my-project-123”)
  • Pub/Sub topic ID (e.g., “products”)

Run Sequin

The easiest way to get started with Sequin is with our Docker Compose file. This file starts a Postgres database, Redis instance, and Sequin server.

1

Create directory and start services

  1. Download sequin-docker-compose.zip.
  2. Unzip the file.
  3. Navigate to the unzipped directory and start the services:
cd sequin-docker-compose && docker compose up -d
2

Verify services are running

Check that Sequin is running using docker ps:

docker ps

You should see output like the following:

CONTAINER ID   IMAGE                           COMMAND                  CREATED          STATUS                    PORTS                              NAMES
bd5c458cabde   sequin/sequin:latest            "/scripts/start_comm…"   11 seconds ago   Up 9 seconds              4000/tcp, 0.0.0.0:7376->7376/tcp   sequin-sequin-1
3bacd89765e7   grafana/grafana                 "/run.sh"                11 seconds ago   Up 11 seconds             0.0.0.0:3000->3000/tcp             sequin-sequin_grafana-1
3ad41319a66c   postgres:16                     "docker-entrypoint.s…"   11 seconds ago   Up 11 seconds (healthy)   0.0.0.0:7377->5432/tcp             sequin-sequin_postgres-1
6139a5fc4e80   redis:7                         "docker-entrypoint.s…"   11 seconds ago   Up 11 seconds             0.0.0.0:7378->6379/tcp             sequin-sequin_redis-1
7e07a5b052de   prom/prometheus                 "/bin/prometheus --c…"   11 seconds ago   Up 11 seconds             0.0.0.0:9090->9090/tcp             sequin-sequin_prometheus-1

Sequin, Postgres, Redis, Prometheus, and Grafana should be up and running (status: Up).

Login

The Docker Compose file automatically configures Sequin with an admin user and a playground database.

Let’s log in to the Sequin web console:

1

Open the web console

After starting the Docker Compose services, open the Sequin web console at http://localhost:7376:

2

Login with default credentials

Use the following default credentials to login:

  • Email:
admin@sequinstream.com
  • Password:
sequinpassword!

View the playground database

To get you started quickly, Sequin’s Docker Compose file creates a logical database called sequin_playground with a sample dataset in the public.products table.

Let’s take a look:

1

Navigate to Databases

In the Sequin web console, click Databases in the sidebar.

2

Select playground database

Click on the pre-configured sequin-playground database:

The database “Health” should be green.

3

View contents of the products table

Let’s get a sense of what’s in the products table. Run the following command:

docker exec -i sequin-sequin_postgres-1 \
  psql -U postgres -d sequin_playground -c \
  "select id, name, price from products;"

This command connects to the running Postgres container and runs a psql command.

You should see a list of the rows in the products table:

  id |         name          | price 
----+-----------------------+-------
  1 | Avocados (3 pack)     |  5.99
  2 | Flank Steak (1 lb)    |  8.99
  3 | Salmon Fillet (12 oz) | 14.99
  4 | Baby Spinach (16 oz)  |  4.99
  5 | Sourdough Bread       |  6.99
  6 | Blueberries (6 oz)    |  3.99
(6 rows)

We’ll make modifications to this table in a bit.

Create a Pub/Sub Sink

With the playground database connected, you can create a sink. This sink will send changes to the products table to your Pub/Sub topic:

1

Navigate to Sinks

Click “Sinks” in the sidebar navigation, then click “Create Sink”.

2

Select sink type

Select “Google Cloud Pub/Sub” as the sink type and click “Continue”.

3

Note "Source" configuration

In the “Source” card, note that the sequin-playground database and products table are pre-selected. Leave these defaults:

4

Setup a backfill

Toggle open the “Initial backfill” card and select “Backfill all rows”:

5

Configure "Pub/Sub Configuration"

In the “Pub/Sub Configuration” card, enter your Pub/Sub details:

  • Project ID: Your Google Cloud project ID
  • Topic ID: The Pub/Sub topic to stream to (e.g., “products”)
  • Service Account Credentials: Paste your service account JSON key file contents
6

Test the connection

At the bottom of the form, click the “Test Connection” button. If you provided proper credentials, it should succeed.

Sequin can connect to your Pub/Sub topic.

7

Create the sink

You can leave the rest of the defaults. As configured, the Pub/Sub topic will first receive a backfill of all rows currently in the products table. Then, it will receive all changes to the products table in real-time.

Click “Create Sink” to finish setting up your Pub/Sub sink.

See changes flow to your Pub/Sub topic

On the new sink’s overview page, you should see the “Health” status turn green, indicating data is flowing to your Pub/Sub topic.

Let’s confirm messages are flowing:

1

Messages tab

Click the “Messages” tab. You’ll see a list of the recently delivered messages:

Sequin indicates it backfilled the products table to your Pub/Sub topic.

2

View in Pub/Sub Console

In the Google Cloud Console:

  1. Navigate to your Pub/Sub topic
  2. Click on the “MESSAGES” tab
  3. Under “Step 2”, select a Cloud Pub/Sub subscription
  4. Click “PULL” to see messages in the topic

You should see a list of messages appear in the table below. The messages are read events from the initial backfill of the products table.

Messages are flowing from Sequin to your Pub/Sub topic.

3

Make some changes

Let’s make some changes to the products table and see them flow to your topic.

In your terminal, run the following command to insert a new row into the products table:

docker exec -i sequin-sequin_postgres-1 \
  psql -U postgres -d sequin_playground -c \
  "insert into products (name, price) values ('Organic Honey (16 oz)', 12.99);"

In the Pub/Sub Console, click “PULL” again to see the new message appear.

Feel free to try other changes:

Each change will appear in your Pub/Sub topic within a few seconds.

Great work!

You’ve successfully:

  • Set up a complete Postgres change data capture pipeline
  • Loaded existing data through a backfill
  • Made changes to the products table
  • Verified changes are flowing to your Pub/Sub topic

​
Ready to stream

Now you’re ready to connect your own database to Sequin and start streaming changes:

Guide: Connect Postgres

Connect your Postgres database to Sequin.

Guide: Setting up a Pub/Sub sink

Setup a Pub/Sub sink to send changes to your topic.