Medical imaging ingestion

The problem

A 700MB+ DICOM study cannot sit inside a single HTTP request while it is parsed and handed to a viewer. Manual folder watching and silent failed hand-offs were how studies got stuck. Radiologists also needed the viewer to move when the backend did, not after a refresh.

What it is

Medical imaging files are large, sensitive, and a poor fit for a request that waits on processing. I built backend pipelines that take an upload through background work, cut the folder-watching toil, and keep OHIF and OpenSeaDragon viewers current over Socket.IO.

What I built

Ingestion is a pipeline, not an endpoint. Celery workers, driven in part by Watchdog, pick up arrivals so nobody has to babysit a folder. Failures retry in the open instead of disappearing between “uploaded” and “ready.”

Redis caching sat in front of repeated reads and notification fan-out, which took load off the application servers. Socket.IO pushed processing state to the client so OHIF and OpenSeaDragon were not guessing.

How it fails

A study that cannot be processed stays visible as a failed hand-off, not as a missing file. Workers can be restarted without losing the fact that an upload existed. Viewer updates are events; a missed event can be reconciled from durable state.

Focus

• Asynchronous processing for large imaging files
• Reliable hand-offs between upload and processing
• Realtime viewer updates instead of polling

What changed

• Handled 100+ daily uploads of 700MB+ DICOM files.
• Automated ingestion cut manual operations by about 50%.
• Redis-backed caching and notifications reduced server load by about 30%.

Built with