If I'm honest about what I enjoy most, it isn't the API on top or the dashboard at the end. It's the pipeline — the unglamorous machinery that takes something messy at one end and produces something clean and useful at the other. Data flows, video streams, image processing. I could build these all day.
A pipeline is a series of honest little promises: I will take this, do exactly one thing to it, and hand it on. Compose enough of those and you get something that feels almost alive — bytes moving through stages, each transforming them a little, nothing held up, nothing dropped.
Data pipelines
The classic. Ingest from somewhere noisy, validate, transform, route, land it somewhere queryable. The hard parts are never the transforms — they're backpressure, retries, idempotency, and what happens when stage three is down but stages one and two keep producing. Get the failure modes right and the happy path takes care of itself.
Video pipelines
Video is data with a deadline. Ingest a stream, segment it, transcode to a ladder of bitrates, package it, and push it out — ideally before anyone notices latency. There's a real thrill in watching a live stream come out the far end smooth when you know how much is happening per second to make that true.
Image processing
Pixels are forgiving in some ways and brutal in others. Decode, resize, transform, re-encode — at volume, the per-image cost you ignored becomes the whole bill. I like the discipline of it: small operations, enormous counts, where a 5ms saving per frame is a real win.
A hell of a lot of tools
There's no single stack for this, and that's part of the fun. Queues and brokers, stream processors, object storage, transcoders, image libraries, schedulers and orchestrators to tie it together, plus the cloud plumbing underneath. The toolbox is enormous and always growing, and learning the next piece of it is half the reason I enjoy the work.
Most engineering builds the thing users see. Pipelines build the thing that makes the thing possible. I'll take the plumbing.