Evidence

The record behind every forecast

Each published forecast is saved with the data it used, the model version, cryptographic hashes of its inputs and output, and the result of the automatic quality checks. Download any forecast’s replay file to check it or reproduce it.

Replay files
3,640
Ledger entries
3,717

The forecasts on the site now

The forecasts on the site now
HazardForecast and modelIssuedHashes (SHA-256)Quality checks
Earthquakeeq_fcst_20261121_0000eq_operational_S2_gear1_v2-ecb87f020f99in 62663847b38c…
out f09c05d6af1b…
Published with warnings
Hurricanehu_fcst_20261120_2146hurricane_ri_v8_3in 3b5c86df2232…
out 2cd7de19de9b…
Published with warnings
Tornadoto_fcst_20261121_0033tornado_v3-05a06c843c87in 3850ae519b05…
out 3f53894c3b74…
Passed

Quality checks

Quality-check outcomes, last 7 days
HazardPassedPublished with warningsBlocked
Earthquake0290
Hurricane0280
Tornado9710

Most frequent findings: Calibration on live forecasts not yet measured (too few closed forecasts with events) (57); Hashes missing for: model file, input data, receipt (28); Source data is aging (2.6h > 2.5h) (1).

Every forecast goes through these checks when it is published. Each ends in one of three outcomes: passed; published with warnings — a non-blocking check flagged something, and the warning is shown with the forecast; or blocked — the forecast is withheld from every page.

  1. Well-formed. The probability, and any range, are finite numbers.
  2. Fresh inputs. The input data is recent enough for the hazard’s schedule.
  3. Complete record. The model version, the hashes of the model file, inputs and receipt, and the replay file are all recorded.
  4. Calibration. The model’s measured calibration on live forecasts is within bounds (a warning until enough forecasts with events have closed).
  5. Sane values. Coordinates are valid, the probability is between 0 and 1, and the grid is no finer than the published resolution.
  6. No confident alarm on weak evidence. A high chance is never shown on inputs unlike the training data or with a very wide range.
  7. Reproducible. A replay file exists, so the forecast can be re-run.
  8. No official terms. Labels never use terms reserved for official warnings.

Forecast ledger

Every forecast is appended to a ledger in which each entry includes the hash of the one before it. Changing any past forecast would change its hash and break every link after it, so the history cannot be edited without it showing.

The 20 most recent ledger entries
HazardForecastIssuedHeadline chanceEntry hash
Tornadoto_fcst_20261121_00330.2%4b552dcc4a18…
Earthquakeeq_fcst_20261121_000012.5%f25d5411b452…
Tornadoto_fcst_20261120_22330.4%605d4c45807e…
Tornadoto_fcst_20261120_21503.7%80b9f21440a9…
Hurricanehu_fcst_20261120_21460.5%3570d3326638…
Tornadoto_fcst_20261120_19480.9%40bdb861d52e…
Earthquakeeq_fcst_20261120_180010.7%a35dcb00e01e…
Tornadoto_fcst_20261120_17460.6%b0f5447403b3…
Tornadoto_fcst_20261120_15442.4%ee48fef98892…
Hurricanehu_fcst_20261120_15411.0%c07b3991da17…
Tornadoto_fcst_20261120_13451.2%1aee8f27a023…
Earthquakeeq_fcst_20261120_12009.8%6284fc0ad3c5…
Tornadoto_fcst_20261120_11450.8%a32768abc121…
Tornadoto_fcst_20261120_09430.3%e2287023baa0…
Hurricanehu_fcst_20261120_09393.0%d299918aab72…
Tornadoto_fcst_20261120_07470.3%53dbe32442bb…
Earthquakeeq_fcst_20261120_06009.8%f1ff2cf039db…
Tornadoto_fcst_20261120_05400.9%72817fce24c5…
Tornadoto_fcst_20261120_0342<0.1%2344963e28be…
Hurricanehu_fcst_20261120_03381.4%cb3526667917…

The full ledger (JSON) · Input records (JSON) · Every quality-check record (JSON)

Replay files

Every forecast’s replay file holds the inputs, the model version and the output exactly as published; the ledger above links each one. All of them are listed in the replay index (3,640 files, JSON) and served from /data/replay/<forecast id>.json.

Check a forecast yourself

  1. Download a forecast’s replay file from the table above or from /data/replay/.
  2. Recompute its SHA-256 hash and compare it with the output hash in the input record (/api/v1/evidence/prov_<forecast id>).
  3. Check that the ledger entry for the forecast carries the same hash and that each entry’s prev_hash matches the entry before it.
  4. Where a forecast carries a signed receipt (earthquake and tornado forecasts do), verify its signature with our Ed25519 public key: c536bdfe6cbb4cf4….

The scripts that do this are in the source repository (scripts/verify_forecast.py).