Overview

When a consumer is configured for pull consumption, your workers will request and acknowledge messages from Sequin. This is in contrast to push consumers, where Sequin pushes messages to your workers.

With a pull consumer, Sequin will provision an HTTP endpoint for your consumer:

https://api.sequinstream.com/api/http_pull_consumers/{{YOUR_CONSUMER_NAME}}/

There are three lifecycle steps to keep in mind when processing messages with a pull consumer:

​

Processing messages

The first step is to make a call to the /receive endpoint to get one or a batch of messages:

curl -X GET https://api.sequinstream.com/api/http_pull_consumers/{{YOUR_CONSUMER_NAME}}/receive?batch_size=10 \
  -H "Authorization: Bearer {your-token}"

This will return a batch of messages. Each message will contain an ack_token and either a row or changes object:

{
  "data": [
    {
      "ack_token": "MTYyeJ7abUjl1pO",
      "record": {
        "id": 2,
        "name": "Chani",
        "title": "Fremen Warrior",
        "spice_allocation": 500,
        "is_sayyadina": true
      },
      "changes": null,
      "action": "insert",
    },
    // more messages...
  ]
}

While your worker is processing this batch of messages, the messages will not be visible to other workers. The amount of time these messages are not visible (i.e. the visibility timeout) defaults to 30 seconds and is configurable in your consumer’s settings.

Once your worker has finished processing the messages, you’ll acknowledge or ack them. This tells Sequin you’re done processing them, and ensures that workers for your consumer won’t see them again:

curl -X POST https://api.sequinstream.com/api/http_pull_consumers/{{YOUR_CONSUMER_NAME}}/ack \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer {your-token}" \
  -d '{
    "ack_tokens": ["MTYyeJ7abUjl1pO", "MTYyeJ0p73hQak"]
  }'

Alternatively, if you’re unable to process the messages, you can nack them. This tells Sequin to make the messages available for processing again:

curl -X POST https://api.sequinstream.com/api/http_pull_consumers/{{YOUR_CONSUMER_NAME}}/nack \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer {your-token}" \
  -d '{
    "ack_tokens": ["MTYyeJ7abUjl1pO", "MTYyeJ0p73hQak"]
  }'

Nacking is a good option if for whatever reason you can’t process the messages right away, but you anticipate they will be processable shortly. For example, if you’re having difficulty connecting to a downstream database, you can nack in the hopes that another worker will pick up the messages that has a working connection.

Instead of nacking, your worker can also do nothing. After the visibility timeout expires, the messages will be made available for processing again.

​

Parallel processing

You can have multiple workers process messages for a single pull consumer. This is a great way to scale your processing throughput in a more deterministic way compared to push consumers.