Files
aimsight-optimizer/archive/batches/2026-11-05/improve/validation.md
T

25 lines
2.3 KiB
Markdown
Raw Normal View History

# Validation
**Verdict**: keep
## Answers
1. The detectors load: no `skipped` line in benchmark-before, S1, S2 or after; the benchmark rerun at HEAD is identical to benchmark-after.
2. No station goes back: benchmark-before, -S1, -S2 and -after are identical, line for line, on all 11 stations and every family.
3. The code stays simple and the gain justifies it. S1 replaces the median gap with a general measure, the cycle per clock hour. It adds no constant and removes 286 false No data instants (0 % correct) on SUSTENHORN. S2 reuses the existing "inside heat shimmer" filter for Poor visibility episodes of point 3, as a one-line change. It drops duplicate events, so the per-instant score does not move, but the same instants are no longer reported twice. Neither change is fitted to the range: no value is cut to its bounds.
4. Nothing forbidden: the six detectors are unchanged in number and family, and `detect` keeps its signature and episode keys. Only the standard library is used and no constant was added. Station names appear only in the docstrings and `thresholds.md` as provenance, never in a rule. The `MISSING` line of `thresholds.md` is updated.
## The range
SUSTENHORN, 2026-11-05 → 2026-11-06:
- No data: correct 0/286 (0 %) → no instants flagged (the false day of No data is gone).
- Every other family on SUSTENHORN, and all of BREITHORN and LENZSPITZE: unchanged.
Total: No data 0/286 → none. The other families are unchanged: Displacement 33/33, 33/36; Obstruction 31/66, 31/37; Thermal displacement 513/513, 513/519; Visibility 20/35, 20/34.
## Non-regression
No line changed, on any station or family, between benchmark-before and benchmark-after (nor in benchmark-S1 or benchmark-S2). The `Total` block is unchanged: Displacement 5746/6345, 5746/20444; No data 75/836, 75/100; Obstruction 1044/4061, 1044/3718; Target issue 19080/22627, 19080/19467; Thermal displacement 10897/16142, 10897/14828; Visibility 3412/7207, 3412/4442.
## Reservations
- No data, S1: the cadence step can only rise, so a station whose hours are only partly filled sees a longer step and needs a longer hole before it reports one. This costs nothing on the reference truth (No data 75/836 and 75/100, unchanged). Watch for missed short holes on stations with an irregular cadence.
## Tools
none