How FogCam is backed up

Fog-Bank, the FogCam archive · fog-bank.org

FogCam keeps one picture and overwrites it with the next. Fog-Bank is the backup: every frame the fog cam publishes is fetched within seconds by four independent pollers on four networks and kept in four places. This page is the plain description of that pipeline, with the numbers it produces. The numbers are live; hover one for how it was made.

Capture

A poller on the archive server asks fogcam.org for the image every 7 seconds, with a cache-busting query string and no-cache headers so FogCam's own server cache cannot hand back an old copy, and a conditional request so an unchanged frame costs nothing to check. A new frame is named by the camera's own Last-Modified timestamp and hashed with sha-256 before anything else touches it. Every attempt, new, unchanged or failed, goes into a permanent ledger.

A second poller, off-site

fogline runs on Cloudflare's network and polls the camera every 15 seconds over a different path to the origin. It keeps a rolling 30 days of raw frames. A reconciler on the archive server pulls anything the archive lacks from it every 5 minutes and after every boot, so a reboot, a full disk or a dead link at the archive loses nothing the second poller saw. It is also the witness: when the archive side goes quiet and fogline is still receiving frames, the fault is ours, and an alert goes out saying so.

A third and a fourth poller

A worker on Cloudflare's network polls the camera every 20 seconds on its own and stores each new frame straight into the permanent mirror (R2). A fourth poller runs on a home connection in another state, writes every frame to its own disk first and pushes it to the mirror within a minute; it keeps 30 days locally, so a Cloudflare or a hosting outage costs it nothing.

A mirror

Every frame from every poller is copied to Cloudflare R2 with its hash and indexed there by day, and the archive reconciles against the mirror every 5 minutes and after every boot, the same way it does against the buffer. The historical archive, the 2019 rescue included, was pushed in September 2026. Losing a frame now takes all four pollers missing the same picture at once, on a camera that holds each one for at least 20 seconds.

Storage

Frames are converted nightly to lossless JPEG XL. The conversion is reversible bit for bit, the original sha-256 is kept, and a frame served from the site is the original JPEG reconstructed. The 27 damaged files in the 2015 and 2019 rescue ZIPs are kept out of the count, not repaired or passed off as frames.

Each layer watched

The buffer's off-site check runs every 5 minutes: it reads the archive's status from outside, compares it with what it is catching itself, and reads the other pollers' pulse (the worker's alarm, the mirror's newest frame, the home vantage's last push). A layer that lapses gets reported on Telegram while the others are still catching.

Gaps, and whose they are

When a day has a drop the record says which side caused it. Our poller silent: ours. FogCam answering with the same image for half an hour or more, or not answering: FogCam's. Both are drawn on the calendar and on every chart with a time axis, and a drop of ours pages the maintainer on Telegram. The full dossier, including the day a stale upstream cache was found taking frames from us, is on the net.

drop: our capture was downdrop: FogCam published nothing

The numbers

Code and configuration

The archive software and its configuration are bundled hourly to an off-site server, so the pipeline can be rebuilt on a clean machine.

More: the net, in full · the frames that were lost before · browse the archive by date · why the archive exists