AWS SQS sink

The SQS sink sends messages to an Amazon SQS queue.

​

Configuration

• Queue URL

  The URL of your Amazon SQS queue. Must be in the format: https://sqs.<region>.amazonaws.com/<account-id>/<queue-name>.

  For FIFO queues, the queue name must end with .fifo.

• Access Key ID

  The AWS access key ID with permissions to send messages to the specified SQS queue.

• Secret Access Key

  The AWS secret access key corresponding to the access key ID.

​

IAM user permissions

Sequin requires an IAM user with the following permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "sqs:SendMessage",
        "sqs:SendMessageBatch",
        "sqs:GetQueueAttributes",
        "sqs:GetQueueUrl",
        "sqs:ListQueues",
        "sqs:ListQueueTags",
        "sqs:ChangeMessageVisibility"
      ],
      "Resource": "<your-queue-arn>"
    }
  ]
}

Replace <your-queue-arn> with your actual queue ARN (e.g., arn:aws:sqs:us-east-1:123456789012:my-queue).

​

Message format

Sequin sends messages to SQS as JSON. You can find the shape of the messages in the messages reference.

​

Retry behavior

If Sequin is unable to deliver a message to SQS, it will retry the message indefinitely. Sequin will exponentially back off the retry interval, with a maximum backoff of roughly 3 minutes.

​

Max message size

SQS allows for a max message size of 256 KB.

If SQS rejects a message, Sequin will cancel the message’s delivery.

​

Grouping and ordering

You can control message grouping behavior using the message_grouping configuration option:

• message_grouping: true (default): Enable message grouping for ordered delivery
• message_grouping: false: Disable message grouping for maximum throughput

When message_grouping is enabled, Sequin will set the MessageGroupId on SQS messages if you’re using a FIFO queue. The default message group for a message is the source row’s primary key(s). You can override this by specifying one or more columns to use for message grouping at the table level.

When message_grouping is disabled, no MessageGroupId is set, allowing for higher throughput but without ordering guarantees.

Sequin will order the delivery of messages with the same group according to their commit timestamp.

sinks:
  - name: "sqs-ordered"
    message_grouping: true  # Enable grouping
    batch_size: 10
    tables:
      - name: "public.orders"
        group_column_names: ["customer_id"]  # Group by customer instead of PK
    destination:
      type: "sqs"
      queue_url: "https://sqs.us-east-1.amazonaws.com/123456789012/orders.fifo"

  - name: "sqs-high-throughput"
    message_grouping: false  # Disable grouping for max throughput
    batch_size: 10
    destination:
      type: "sqs"
      queue_url: "https://sqs.us-east-1.amazonaws.com/123456789012/events"

​

FIFO vs standard queues

FIFO queues are generally recommended. They ensure that messages pertaining to the same row are processed in order.

​

Routing

The AWS SQS sink supports dynamic routing of the queue_url with routing functions.

Example routing function:

def route(action, record, changes, metadata) do
  %{
    queue_url: "https://sqs.us-east-1.amazonaws.com/123456789012/#{metadata.table_name}"
  }
end

When not using a routing function, messages will be published to the queue specified in the sink configuration.

​

Debugging

You can view the status of your SQS sink in the Sequin web console.

On the “Messages” tab, you can see which messages are in-flight to SQS, which messages Sequin is unable to deliver, and recently delivered messages.

Messages that Sequin is unable to deliver will have a “Deliver count” greater than 1. You can click on a message to see more details, including the last error response received from SQS.