A Step Function, Not a Smooth Trend
Fourth entry in this series on the process behind a voltage-screening pipeline. This is the post where the retracted 35/35 claim from an earlier entry gets its real replacement — and where the honest result turned out to be more interesting than the one originally hoped for.
What was needed
Two problems from earlier in this series converged here. First, real SimBench load data produces zero violations at its native scale, so there's no violation population to test a state-aware screening rule against under realistic conditions. Second, the original state-aware threshold was circular — set at the exact minimum of the sample it was validated on.
Fixing both at once meant building a genuine severity experiment: real load shape (not a synthetic table), scaled by an explicit, pre-declared multiplier, swept across a range fixed in advance — not chosen after peeking at which values produce a favorable result — and a screening rule tested with real train/test discipline at every point on that sweep.
Finding the onset
A ScaledLoadProvider wraps a real, time-ordered SimBench load shape and applies one multiplier uniformly across the whole event stream. Sweeping that multiplier from 1.00× to 2.00× of the network's own documented peak load produces a clean, monotonic dose-response on the critical-path population:
| Severity | Violations (of 4,055 critical events) | |---|---| | 1.00×–1.30× | 0 | | 1.35× | 1 | | 1.40× | 2 | | 1.50× | 14 | | 1.75× | 131 | | 2.00× | 372 |
Violations first appear at 1.35× documented peak — higher than the default synthetic table's own 1.26×, and located by sweeping outward from the network's own specification rather than borrowed from any third party's curation. This is itself a second, independent confirmation of the loading-dominates-voltage mechanism from the first post in this series, arrived at by a completely different route than the original regression.
The recall audit, done properly this time
With a real onset located, three severities were selected for a full recall audit: 1.26× (the default table), and two points past the real onset, 1.50× and 1.75×. At each one, the linear voltage-sensitivity regression was refit from scratch on a fresh 70/30 split of that severity's own critical-path data, then applied out-of-sample to a recall-audit population it never touched during fitting — the exact discipline that would have caught the original circular threshold, applied properly this time. And per the sample-size post, every severity was audited against the entire 16,531-event non-critical population, not a sample.
| Severity | FN rate (exact) | State-aware catch rate | 95% CI | |---|---|---|---| | 1.26× (default table) | 7.9% | 20.5% | [18.4%, 22.7%] | | 1.50× (SimBench, scaled) | 0.4% | 26.0% | [17.3%, 37.1%] | | 1.75× (SimBench, scaled) | 3.9% | 74.3% | [70.8%, 77.5%] |
The finding, not the one I expected
Going in, the hoped-for result was a clean, monotonic improvement: state-aware screening gets steadily better as conditions get more severe. That's not what the data shows.
Between 1.26× and 1.50×, the catch rate is statistically indistinguishable — 20.5% versus 26.0%, with heavily overlapping confidence intervals. Then, by 1.75×, it jumps to 74.3%, with a confidence interval that doesn't overlap either of the earlier two at all — 1.50×'s upper bound (37.1%) sits well below 1.75×'s lower bound (70.8%).
That's a genuine step function, not a smooth trend: state-aware screening provides little advantage near the onset of violations, and a large, robust advantage once operating conditions are pushed substantially past it. It's a more specific and more falsifiable claim than "state-aware screening helps" — it says where the value shows up, which is exactly the kind of result that's actually useful for deciding when this kind of screening is worth deploying versus when it isn't.
Why the flat part matters as much as the jump
It would have been easy to report only the 1.75× result and let the flat region go unmentioned — the paper would look cleaner. But the flat region is a real finding on its own: it says the mechanism that makes state-aware screening valuable (background loading dominating voltage) needs enough separation from nominal conditions before a fitted rule can reliably distinguish it from noise. Reporting only the strong result would have hidden exactly the boundary condition a TSO or DSO evaluating this approach would need to know about before trusting it operationally.
Reproducibility: the severity sweep and full-census recall audit
from engine import (
GridSimulator, LocalCsvIngestionLayer,
ScaledLoadProvider, TimeSeriesLoadProvider, load_simbench_profile,
)
simulator = GridSimulator()
stream = LocalCsvIngestionLayer().fetch_stream()
simbench_vals = load_simbench_profile()
base_provider = TimeSeriesLoadProvider(simbench_vals)
# Locate the onset (severities pre-declared before running)
for severity in [1.00, 1.05, 1.10, 1.15, 1.20, 1.25, 1.30, 1.35, 1.40, 1.50, 1.75, 2.00]:
provider = ScaledLoadProvider(base_provider, severity)
cycle_df = simulator.run_streaming_pipeline(stream, load_provider=provider)
critical = cycle_df[cycle_df["critical_event"] == True].dropna(subset=["raw_vm_ref_pu"])
n_violations = (critical["raw_vm_ref_pu"] < 0.90).sum()
print(f"severity={severity:.2f}: {n_violations} violations")
# Full run_severity_recall_audit.py <severity> 16531 for each of 1.50, 1.75
# reproduces the recall table exactly (fresh regression refit, full census).
