Track record

How the forecasts have scored

Two kinds of evidence, kept apart. Test results: how each published model scored on years held back from its development, in tests written down before the data was scored. Live record: how the forecasts we actually published scored once their windows closed. If a live model is not scored yet, this page says so directly. It also shows any scoring backlogs.

Scores as of

In numbers

3,640

forecasts recorded

Every published forecast, frozen with its inputs.

3,490

closed and scored

Windows that have ended and been scored against what happened.

9

waiting to be scored

Windows that have ended but are not scored yet.

0

breaks in the ledgers

Each entry carries the previous entry’s hash, so an edit to history would show here.

Needs attention

  • 9 matured forecast windows are waiting for scoring.

By hazard

Earthquake
Published model
eq_operational_S2_gear1_v2-ecb87f020f99
Window
30 days
Second look at 2023–2025
ranking accuracy 96.9%; information gain 2.61 nats per M6+ cell-window (a cell’s 30 days that held an M6+) over a uniform map
Live record of this version
50 (1 non-overlapping) closed 30-day forecasts (they overlap), 24 distinct M6+ earthquakes; no score yet: 1 non-overlapping 30-day window so far (a score is quoted from 2)
Forecasts recorded
923 (798 closed and scored, 1 waiting)
Ledger
934 chained rows, 0 breaks
Hurricane
Published numbers
NOAA DTOPS (SHIPS-RII as fallback) for the Atlantic, East and Central Pacific; HazardPulse v8.3 elsewhere, tested on 2025–2026 cycles in the basins where it publishes, none used to fit it, with best-track inputs
Window
24 hours
Test, 2025 season (scored once)
DTOPS ranking accuracy 92.8%, Brier skill 0.31; 601 cycles, 44 rapid intensifications
Live record of the published models
NOAA DTOPS (SHIPS-RII as fallback), Atlantic, East and Central Pacific: 181 storm-cycles, 22 rapid intensifications; Brier score 0.062, ranking accuracy 94.4%; HazardPulse v8.3, elsewhere: 174 storm-cycles, 21 rapid intensifications; Brier score 0.082, ranking accuracy 81.2%
Forecasts recorded
256 (247 closed and scored, 5 waiting)
Ledger
197 chained rows, 0 breaks
Tornado
Published model
tornado_v3-05a06c843c87
Window
60 minutes
Test, 2025 (scored once)
ranking accuracy 97.0%, Brier skill 0.11; 1,469,977 storm observations, 1,579 tornadic
Live record of this version
35,125 storm forecasts, 43 tornadic; ranking accuracy 95.7%
Forecasts recorded
2,461 (2,445 closed and scored, 3 waiting)
Ledger
2,527 chained rows, 0 breaks
Row hashes
2,357 rows match their own hash; 170 earlier rows match it with forecast_id removed (they were hashed before that field was added); 0 mismatches

Live record, by model version

Every model version that has published a number, including retired ones. A score is shown only once 10 or more events have been observed for that version: with fewer, a model that always says “no” would look as good as a skilful one. Earthquake forecasts are issued several times a day for 30 days ahead, so their windows overlap: each earthquake is counted once, and a score also needs two windows that do not overlap.

Live record by model version
HazardModel versionStatusForecasts closedEvents observedScore
Earthquakeeq_coherence_v1_0retired717 (7 non-overlapping)87ranking 65.8%; information gain per M6+ earthquake per window vs a uniform map −15.47 (averaged over overlapping windows)
Earthquakeeq_operational_C0_v1-d28c62a2e35bretired3 (1 non-overlapping)23no score yet: 1 non-overlapping 30-day window so far (a score is quoted from 2)
Earthquakeeq_operational_S1_gear1_v1-d409c534bb4cretired28 (1 non-overlapping)26no score yet: 1 non-overlapping 30-day window so far (a score is quoted from 2)
Earthquakeeq_operational_S2_gear1_v2-ecb87f020f99published50 (1 non-overlapping)24no score yet: 1 non-overlapping 30-day window so far (a score is quoted from 2)
Tornadotornado_storm_v1_0retired233,600634ranking 74.2%; Brier skill vs climatology −8.45
Tornadotornado_v3-05a06c843c87published35,12543ranking 95.7%; Brier skill vs climatology 0.08
Tornadotornado_v3-9005905f08b7retired1,8070too few events to score yet (need 10)
Hurricanehurricane_ri_stack_v1-35938d6c09b7published18122ranking 94.4%; Brier score 0.06
Hurricanehurricane_ri_v8_2retired336too few events to score yet (need 10)
Hurricanehurricane_ri_v8_3published17421ranking 81.2%; Brier score 0.08
Hurricanejtwc_warning_ingestretired50too few events to score yet (need 10)

Does a 10% forecast come true 1 time in 10?

Calibration compares what was forecast with what happened. The published tornado model’s 2025 test, split by how high the forecast was. Live calibration is measured once enough forecasts with events have closed.

Tornado model, every storm of 2025 (scored once)
Forecast rangeStorm observationsAverage forecastTornado followed95% interval
0%–0.1%1,340,592<0.01%0.01%<0.01%–0.01%
0.1%–0.2%44,8340.14%0.13%0.10%–0.17%
0.2%–0.5%39,9040.32%0.29%0.24%–0.35%
0.5%–1.0%18,8400.70%0.58%0.48%–0.70%
1.0%–2.0%11,7241.40%1.17%0.99%–1.38%
2.0%–5.0%8,1843.09%2.30%1.99%–2.64%
5.0%–10.0%3,2317.03%6.96%6.12%–7.88%
10.0%–20.0%1,59513.76%16.99%15.21%–18.89%
20.0%–30.0%43024.49%21.86%18.15%–25.95%
30.0%–50.0%64137.65%34.63%31.03%–38.38%
50.0%–100%251.27%50.00%6.08%–93.92%

In a well-calibrated row the two middle numbers agree. Every breakdown of the tornado test