Appearance
The event bus
A shared Valkey 9 (Redis-compatible) instance called bus, in the platform project. Services use Redis Streams on it to pass messages to each other, whatever language each one is written in.
- AOF persistence is on, so messages survive a restart.
- The memory limit is 512 MB.
- It's internal only. Apps reach it through
BUS_URL.
Streams
Names are <domain>.<event>, lower case, separated by dots:
email.inbound
sms.inbound
claude.jobs
claude.results
<app>.<thing>.<happened>Every message has these fields:
| Field | Meaning |
|---|---|
type | the stream name |
id | a uuid, which consumers use to drop duplicates |
ts | ISO 8601 time in UTC |
source | the app that sent it |
data | JSON |
Sending
XADD <stream> MAXLEN ~ 10000 * type <stream> id <uuid> ts <iso> source <app> data <json>Always cap the length (MAXLEN ~ 10000), so a stream can't grow forever.
Receiving
Each consuming app has its own consumer group, named after itself, so several apps can each read every message:
XGROUP CREATE <stream> <app> $ MKSTREAM
XREADGROUP GROUP <app> <instance> COUNT 10 BLOCK 5000 STREAMS <stream> >
XACK <stream> <app> <message-id> # only after it's handledA message stays pending until it's acknowledged, so a crash doesn't lose it.
Retries and dead letters
- On a timer, each consumer runs
XAUTOCLAIMto take over messages that have sat unacknowledged for more than 5 minutes. - After 5 delivery attempts, the message is copied to
<stream>.deadand acknowledged. - The ops page shows how long each stream is, and an alert goes out when a
.deadstream has anything in it. - Handlers must cope with seeing the same message twice, using
idto tell.
The infra:bus skill has ready-to-use Node (ioredis) and Python (redis-py) code for all of this.
When not to use it
For a job queue inside one app, use that language's normal library (BullMQ, RQ) on the app's own Valkey. The bus is for messages between services.