Why Does the Model Find Zero Violations on Real Data?
In my ongoing series on grid resilience — from thermal bottlenecks to fault-current analysis — this entry is about a result that looked like a bug and turned out to be a provenance problem: a streaming voltage-screening pipeline that finds hundreds of violations on its own default operating-state table, and exactly zero the moment you swap in real, independently-published load data.
The pipeline, briefly
The setup: one year of real German TSO redispatch instructions (20,586 events), replayed through a streaming screener on the CIGRE medium-voltage benchmark network. A cheap statistical selector (percentile and rate-of-change thresholds on redispatch magnitude) flags the 4,055 most extreme events as "critical" — no power-flow solve, just arithmetic on the stream. Only those flagged events get a full Newton-Raphson AC solve to check whether they actually cause a voltage-limit violation (V < 0.90 p.u.).
Under the pipeline's default operating-state model — a deterministic, hashed hourly load-multiplier table — that AC validation finds 239 confirmed violations among the 4,055 flagged events. A regression against the converged solves shows why: background loading correlates with voltage at r = −0.99, while the redispatch event's own magnitude correlates at only r = 0.06. The event barely matters; the network's background state does almost all the work.
That's a clean, physically sensible result. The problem started when I tried to confirm it wasn't an artifact of the synthetic table's own construction.
The swap that broke everything
SimBench publishes real, independently-curated German load profiles — not synthetic, not tuned to this project. I built two operating-state providers around it: one drawing SimBench values i.i.d. per event (breaking any correlation with event order), one replaying them in their original time-ordered sequence. Both are drop-in replacements for the same 4,055 critical-path events, same selector, same network.
The loading–voltage correlation replicated almost exactly: r = −0.991 under real SimBench data, versus −0.994 under the synthetic default. That's reassuring — the underlying physics isn't an artifact of how the default table was built.
The violation count did not replicate. Zero. Not "fewer" — zero, across both real-data variants, on the same 4,055 events that produce 239 violations under the default table.
Tracing it back
The instinct here is to treat this as a nuisance and move on — pick whichever version supports the paper's headline number. That's exactly the instinct worth resisting. If the mechanism (loading dominates voltage) generalizes but the violation count doesn't, the honest thing to do is find out why, not just note the discrepancy.
The default synthetic table's provenance traces back three hops:
- Rudion et al., 2006 — the original CIGRE Task Force C6.04.02 paper defining this exact medium-voltage benchmark network, including its documented peak-load specification.
- Porsinger et al., 2017 — seasonal load curves simulated for this same network topology, published in Energies.
- A public repository curator who selected specific multi-day windows from that data, explicitly documented as producing network overloads and voltage-limit problems — a stress-testing scenario, not a representative one.
The number that falls out of that chain: the default table's peak load sits 26% above the network's own documented peak-load specification from Rudion et al. That's not a modeling error. It's a deliberately curated overload window, three citations removed from the benchmark's own numbers, quietly doing the work of "producing violations" in a pipeline that inherited it as a default.
What this actually means
Real, representative load data — the kind SimBench exists to provide — never exceeds that documented peak in this benchmark. Which is exactly why it produces zero violations: on this network, with this DER configuration and this real event stream, confirmed violations only occur once loading is pushed above the benchmark's own specified envelope. The 239-violation figure isn't wrong, but it isn't "typical severity" either — it's what happens specifically under a documented, curated overload.
This distinction — mechanism generalizes, headline count doesn't — became the throughline for the rest of the investigation. The next post covers a second, worse problem that surfaced while trying to fix this one: a screening-rule claim that looked airtight and turned out to be circular by construction.
Reproducibility: tracing the correlation under both operating-state models
from engine import (
GridSimulator, LocalCsvIngestionLayer,
SyntheticLoadProvider, TimeSeriesLoadProvider,
load_simbench_profile,
)
simulator = GridSimulator()
stream = LocalCsvIngestionLayer().fetch_stream()
# Default operating-state model (E2): 239 violations, r(loading, V) = -0.994
default_provider = SyntheticLoadProvider(simulator.load_multipliers)
cycle_default = simulator.run_streaming_pipeline(stream, load_provider=default_provider)
# Real SimBench load shape, time-ordered (E4): 0 violations, r(loading, V) = -0.991
simbench_vals = load_simbench_profile()
real_provider = TimeSeriesLoadProvider(simbench_vals)
cycle_real = simulator.run_streaming_pipeline(stream, load_provider=real_provider)
for name, cycle_df, provider in [("default (E2)", cycle_default, default_provider),
("real SimBench (E4)", cycle_real, real_provider)]:
critical = cycle_df[cycle_df["critical_event"] == True]
solved = critical.dropna(subset=["raw_vm_ref_pu"]).copy()
# "load_mw" is the event's own redispatch magnitude, not background loading --
# the background-loading proxy is "multiplier", looked up per event below.
solved["multiplier"] = solved["event_idx"].apply(lambda i: provider.get_multiplier(int(i)))
n_violations = (solved["raw_vm_ref_pu"] < 0.90).sum()
r_loading = solved["multiplier"].corr(solved["raw_vm_ref_pu"])
r_magnitude = solved["load_mw"].corr(solved["raw_vm_ref_pu"])
print(f"{name}: {n_violations} violations, "
f"r(loading, V) = {r_loading:.3f}, r(event magnitude, V) = {r_magnitude:.3f}")
