Why hourly forecasts are obsolete
// the demand-side has changedThe hourly-resolution forecast was a sensible abstraction for the grid of 2010. Demand changed slowly. Generation was largely dispatchable. Settlement periods were thirty minutes. The decisions a control-room operator had to make — within-day balancing, reserve activation, day-ahead bid construction — happened on the same hourly cadence as the forecast that informed them. The whole system was internally consistent.
That system has stopped being internally consistent. Electric vehicles charge in bursts of two to fifteen minutes. Heat pumps cycle in response to thermostat dead-bands measured in minutes. Rooftop solar swings on minute-scale cloud passage. Industrial demand-response participants enter and leave the market in five-minute blocks. The aggregate effect, even at GW scale, is that the load curve has acquired a high-frequency component that an hourly forecast cannot see — and increasingly, cannot smooth over without producing dispatch decisions that are systematically wrong on the timescales that now matter.
The decisions that used to be hourly are now minute-level or faster. Battery dispatch is solved every five minutes in any modern stack. Reserve activation runs sub-second on automatic generation control. Congestion management has moved from end-of-day post-mortem to within-hour intervention. Every one of these decisions, when made against an hourly forecast, is being made against a forecast that has already drifted away from reality before the decision finishes computing.
What sub-second buys you
// the latency-revenue linkLatency lets you act on the actual trajectory, not the smoothed average. The single best demonstration we have of this is from an ERCOT pilot deployment last quarter. The same 200 MW / 800 MWh battery, on the same market product, in the same week, recovered 4–7% additional arbitrage margin when the dispatch lane was reading from a 1 Hz forecast versus a 5-minute-average forecast. The difference is not in the model — both forecasts were FATHOM's, with the same calibration target — it is in what the dispatch lane could see. The 1 Hz feed exposes the actual ramp dynamics; the 5-minute-average feed hides them inside a smoothed mean and the optimiser commits capacity to the wrong side of the ramp.
The same dynamic operates in reserve activation. Frequency-response products pay for availability, but penalise for late or partial delivery. An operator who is sitting at the edge of their committed envelope because a smoothed forecast told them the next ten minutes were stable, when in fact a one-minute deviation was developing, will be assessed against the actual delivery and not against the forecast quality. Sub-second forecasting moves the operator's perception of "the next ten minutes" closer to the reality the settlement engine will measure them against.
Probabilistic, not point
// quantile bands · operator agencyA single best-guess forecast is unusable when your downside risk has tripled. The grid of 2026 has more sources of variance — weather-driven generation, EV charging clusters, demand-response activations, inter-area dynamic line ratings — than the grid of 2015, and the best response to higher variance is not a better point forecast but an honest distribution. Our outputs are full quantile bands so dispatch and trading get to choose their own risk appetite per decision. The dispatch lane is risk-neutral by default; the trading desk that consumes the same feed is typically risk-averse on the upside and risk-seeking on the downside; the procurement function that uses the day-ahead band is risk-averse on both. All three consume the same quantile distribution and apply their own risk function.
This is the part of the new operating envelope that organisations struggle with the most. The legacy workflow assumes that "the forecast" is a number — that there is one canonical answer that everyone disagrees with by the same amount. The new workflow accepts that the forecast is a distribution and that different consumers will rationally take different positions against it. That shift requires the operations team, the trading desk and the procurement function to use the same vocabulary about uncertainty. It is more a cultural change than a technical one, and the technical roll-out keeps stalling on the cultural piece.
What control rooms have to change
// HMIs · workflows · escalation pathsMost SCADA HMIs cannot display streaming probabilistic forecasts. The conventional control-room screen shows the forecast as a single line, sometimes with a confidence corridor that updates every fifteen minutes. That representation does not survive contact with a 1 Hz probabilistic feed. We ship an embeddable widget — a Web Component that renders the live quantile bands at 1 Hz against the actual telemetry trace — and a streaming API for HMI vendors who want to build a native rendering. Existing rooms can adopt incrementally without ripping out their EMS.
Workflows change in less visible ways. The escalation path for "the forecast is wrong" used to be a fifteen-minute conversation between the duty operator and the forecasting team. With a 1 Hz feed, the right question is no longer "is the forecast wrong?" — the percentile distribution will be roughly right within its own confidence bounds — but "is the consumer of the forecast making decisions appropriate to its distribution?" That is a different conversation, and most control rooms do not yet have an escalation path for it. We help customers write one in the first month of every deployment.
Procurement teams change least, but most expensively. The day-ahead bid construction process traditionally turns the forecast into a point estimate and bids against it. A probabilistic feed allows the team to bid the distribution — submitting different quantities at different price points to capture the upper tail without exposing the bid book to the lower tail. That is materially more revenue in volatile weeks, but it requires the procurement workflow to model the bid book against a distribution rather than a point, and that requires changes to the procurement IT stack that procurement IT is rarely funded to make.
Where this is going
// next 18 monthsThe next 18 months will see distribution-system operators run their own forecast services, exposing them to retailers and aggregators via standardised APIs. The era of "we run our own ARIMA on a spreadsheet" is over for any DSO with a non-trivial DER population. The Open Grid Forecast initiative — the loose standards-track conversation between DSO technical leads, vendors and aggregators — is converging on an OpenAPI surface that will look very much like the one we already publish. We expect to publish a draft of the conformance test suite later this year.
What will not change is the model-quality question. The standardised API is a good idea; the standardised model is a bad idea. The work of producing a defensible, calibrated, audit-grade forecast against a specific operator's telemetry mix is not commodity work. The vendor ecosystem will probably consolidate, but the right answer for the operator is the same as it has been for ten years: read the calibration report, demand the side-by-side comparison, and never sign a contract that does not allow you to inspect what the model is doing on your own data.
Want to argue with this essay?
FATHOM engineers read every reply. If you disagree with the framing — or have data that contradicts ours — we want to hear it. The position on standardised models is the one that gets the most pushback; we are happy to defend it.
More from the field-notes desk
// other recent essaysStacking revenue on a grid battery: what actually works in 2026. — the four real revenue stacks, the conflicts, and the +14.2% calibrated uplift.
Predicting transformer failure 19 days early: the model behind FATHOM Sentinel. — multi-modal inputs, the graph transformer, and the 14-month UK DNO trial.