Skip to content

The event bus ​

A shared Valkey 9 (Redis-compatible) instance called bus, in the platform project. Services use Redis Streams on it to pass messages to each other, whatever language each one is written in.

  • AOF persistence is on, so messages survive a restart.
  • The memory limit is 512 MB.
  • It's internal only. Apps reach it through BUS_URL.

Streams ​

Names are <domain>.<event>, lower case, separated by dots:

email.inbound
sms.inbound
claude.jobs
claude.results
<app>.<thing>.<happened>

Every message has these fields:

FieldMeaning
typethe stream name
ida uuid, which consumers use to drop duplicates
tsISO 8601 time in UTC
sourcethe app that sent it
dataJSON

Sending ​

XADD <stream> MAXLEN ~ 10000 * type <stream> id <uuid> ts <iso> source <app> data <json>

Always cap the length (MAXLEN ~ 10000), so a stream can't grow forever.

Receiving ​

Each consuming app has its own consumer group, named after itself, so several apps can each read every message:

XGROUP CREATE <stream> <app> $ MKSTREAM
XREADGROUP GROUP <app> <instance> COUNT 10 BLOCK 5000 STREAMS <stream> >
XACK <stream> <app> <message-id>          # only after it's handled

A message stays pending until it's acknowledged, so a crash doesn't lose it.

Retries and dead letters ​

  • On a timer, each consumer runs XAUTOCLAIM to take over messages that have sat unacknowledged for more than 5 minutes.
  • After 5 delivery attempts, the message is copied to <stream>.dead and acknowledged.
  • The ops page shows how long each stream is, and an alert goes out when a .dead stream has anything in it.
  • Handlers must cope with seeing the same message twice, using id to tell.

The infra:bus skill has ready-to-use Node (ioredis) and Python (redis-py) code for all of this.

When not to use it ​

For a job queue inside one app, use that language's normal library (BullMQ, RQ) on the app's own Valkey. The bus is for messages between services.