Realtime presence

The problem

Operators needed to see who was online across a tenant without polling. The first implementation worked until background workers created a new event loop per task. Connections borrowed on one loop were returned to a pool that the next loop could not use, and the failure mode was quiet: tasks looked healthy while the pool was already poisoned.

What it is

Presence looks like a frontend feature until the connection pool starts dying in production. I shipped tenant-wide realtime presence over WebSockets, then tracked a background-worker failure to a per-task event loop that was invalidating async database connections.

What I built

Presence is pushed over WebSockets with Django Channels, scoped to a tenant so one workspace cannot observe another.

The worker bug was not in the presence protocol. Each task was spinning its own event loop, which silently invalidated the async database connection pool. The fix was a persistent loop for the worker process, so borrowed connections go back to a pool that still owns them.

How it fails

A dropped socket is an offline event, not a crashed worker. Pool misuse had been failing quietly; after the loop fix, connection errors surface instead of accumulating as “the site is just slow.”

Focus

• Tenant-wide presence over Django Channels
• A persistent event loop in background workers
• Failures that show up as pool exhaustion, not as exceptions

What changed

• Presence updates without polling the REST API.
• Background workers stopped poisoning the async connection pool.

Built with