Why our captions broke in production
Burned-in captions rendered perfectly on a laptop and came out blank in prod. The bug wasn’t the renderer — it was a missing font in the container.
Burned-in captions worked perfectly on every laptop we tested on. Then they shipped to production and came out blank — a clean video with no captions at all. The renderer wasn't broken. The container was missing a font.
The symptom
Our render path burns captions into the .mp4 with FFmpeg — the words are pixels in the frame, not a sidecar track, so they survive any platform that strips subtitle streams. On a dev machine the captions rendered crisp. In prod the exact same command produced frames with no text where the caption bar should be. No error, no crash — just nothing.
The cause
FFmpeg's text filter draws glyphs using a font it finds on the host. Our dev machines had the font installed system-wide; the production container image did not. When the filter can't resolve the font it asks for, it doesn't fail loudly — it falls back to nothing, or to empty placeholder boxes. Locally everything was fine precisely because the dev environment quietly supplied a dependency prod didn't have.
The fix
Install the fonts the caption filter references into the production image itself, so the renderer finds them in every environment (shipped in PR #80). The font is now a pinned part of the image build, not an ambient assumption about whatever machine runs the render.
The lesson
Anything the renderer reads from the operating system — fonts, codecs, locale data — has to be pinned in the image, not inherited from a developer's laptop. Dev/prod parity is exactly what hides this class of bug: the closer your local box is to a real workstation, the more invisible system dependencies it has that a clean container won't. We caught a couple of neighbors the same week — short clips that froze on the last frame instead of looping, and runs that visually stuck on “Assembling” after the work was actually done — both shipped alongside the font fix.
What we changed in how we work
Render bugs now get reproduced in the production container, not just locally. If it draws to a frame, it gets tested where it actually runs.
