Files
aimsight-labeling/auto/analyses/NIEDERHORN_2026-10-21T1200_2026-11-01T0000/scripts/checks.py
T
maximeandClaude Opus 5.5 ddb17c2189 auto: labels requalified under the 5-noise rules (only what stands out from the noise): 32 analyses refined, the 6 paused periods verified, the long labels requalified with the fixes the period analysts found; 1153 -> 961 labels
- 32 analyses: each label kept, rewritten or removed by its analyst (section Requalification of the report; mm magnitudes where the targets have them)
- SNCF01 x3, WILDSTRUBEL 2026-09-01 -> 2026-10-06 x3: verified from their existing labels under the new rules
- long labels (no analysis): auto/analyses/long-labels/<STATION>/; NORDEND-169 and -195 run to 07/11, so NORDEND-283 and -284 (their continuation) are removed
- reading-rules.md, stations.md: the citations of removed or rewritten labels point to their successor, or to [expert]

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 23:58:00 +01:00

29 lines
1.8 KiB
Python

"""Requalification spot checks: rolling(±3) deviation in own noises of chosen sensors at chosen instants; rejection
counts; sensors present at the No data instants. usage: python3 checks.py"""
import sys, statistics
sys.path.insert(0, 'tools/scripts')
import data
M = data.load('data/NIEDERHORN')
a0, a1 = data.parse_time('2026-10-21T12:00Z'), data.parse_time('2026-11-01T00:00Z')
def win(s): return [m for m in M[s] if a0 <= m['t'] <= a1]
def noise(vals): # median absolute step / robust, same idea as the tools: use MAD of first differences
d = [abs(b - a) for a, b in zip(vals, vals[1:])]
return statistics.median(d) / 0.6745 / 2 ** 0.5 * 1.0
def dev(s, t, c):
ms = [m for m in win(s) if m[c] is not None]
ts = [data.iso(m['t'])[:16] for m in ms]
if t not in ts: return 'rejected/missing'
i = ts.index(t)
nb = [m[c] for m in ms[max(0, i-3):i] + ms[i+1:i+4]]
return f"{(ms[i][c] - statistics.median(nb)):+.4f} px"
for s, t in [('PWD_IMP_B07_016_C09', '2026-10-28T04:00'), ('PWD_IMP_B07_016_C09', '2026-10-26T13:00'),
('PWD_RMP_B15_017_C09', '2026-10-29T13:00'), ('PWD_RMP_B15_017_C09', '2026-10-26T13:00'),
('e01', '2026-10-24T18:00'), ('PWD_IMP_B07_017_C09', '2026-10-22T16:00')]:
print(s, t, 'cx', dev(s, t, 'corr_x'), 'cy', dev(s, t, 'corr_y'))
for s, x, y in [('e10', '2026-10-30T04:00Z', '2026-11-01T00:00Z'), ('PWD_RMP_B16_026_C09', '2026-10-30T02:00Z', '2026-11-01T00:00Z'),
('e10', '2026-10-21T12:00Z', '2026-10-30T01:00Z')]:
ms = [m for m in M[s] if data.parse_time(x) <= m['t'] < data.parse_time(y)]
print(s, x, y, 'rejected', sum(data.rejected(m) for m in ms), 'of', len(ms))
for t in ['2026-10-23T00:00', '2026-10-26T15:00', '2026-10-28T12:00']:
print(t, 'sensors with a reading:', sum(1 for s in M for m in M[s] if data.iso(m['t'])[:16] == t))