Somewhere upstream, a storm is intensifying. The question on the table is simple: how much rain has fallen over this basin in the past six hours — and how does it compare to the events that caused flooding before? The answer exists. It is sitting in a weather radar archive. Whether it arrives in time is another matter.
Every weather decision has a window. A flood warning, a reservoir release, an aviation route, a research deadline — each one closes whether or not the data showed up. And too often, the data loses the race: not because the radars didn't observe, but because the archive can't answer questions. It can only hand you files.
The bottleneck isn't the science
In a typical file-based workflow, answering a one-day question about a storm means locating dozens of Level II files, downloading them, decoding each one, reconstructing the timeline in your own code — and only then doing the science. In our benchmark, the science itself took seconds. Almost everything else was preparation. That cost is structural: it repeats for every scientist, every question, every time.
What we measured
That is the problem Radar DataTree was built to remove — and the numbers are now written up: our paper (Ladino, Nesbitt, Grover, Galewsky & Mühlbauer) is under review at IEEE Transactions on Big Data. The idea in one sentence: open an entire radar archive as one time-indexed, cloud-native dataset — ask it questions directly, instead of downloading and decoding thousands of files.
48× faster end-to-end, moving 5.8× less data (809 MB → 139 MB). Same input, same science — the storm profiles come out identical.
48× is the difference between a coffee-break analysis and an interactive one. When one day of storm structure takes 6.5 seconds instead of five minutes, you stop rationing your questions: test five hypotheses before lunch, rerun with different thresholds, follow the data wherever it leads.
And the advantage grows with the question. Rainfall totals across six months of archive — 37,321 files, ~184 GB — run 1,565× faster, because the dataset reads only the exact chunks each query touches, no matter how the observations were originally packed into files.
End-to-end speedup vs the file-based workflow (rainfall estimation, KVNX archive on AWS). No caching tricks — identical science, different plumbing.
Crucially, nothing scientific changed. We reproduced a canonical analysis of the 20 May 2011 Oklahoma storm, and the profiles come out identical — same input, same physics, different plumbing.
Why this matters to us
This is the heart of what AtmoScale does. We care about data being ready inside the decision window — while the flood forecast is still actionable, the study still fundable, the question still worth asking. And we do it on open, standards-aligned foundations: Radar DataTree follows the WMO FM301 data model — we don't replace the standard, we extend it to the cloud — and builds on Zarr, Icechunk, and xarray, so your institution never depends on us, or anyone, to read its own archive. The data is already valuable. Our job is to make sure it answers in time, and that you're the one in control of it.