Files
aimsight-labeling/auto/analyses/CLARIDEN_2026-10-20T0400_2026-11-01T0010/scripts/requalify.py
T
maximeandClaude Opus 5.5 ddb17c2189 auto: labels requalified under the 5-noise rules (only what stands out from the noise): 32 analyses refined, the 6 paused periods verified, the long labels requalified with the fixes the period analysts found; 1153 -> 961 labels
- 32 analyses: each label kept, rewritten or removed by its analyst (section Requalification of the report; mm magnitudes where the targets have them)
- SNCF01 x3, WILDSTRUBEL 2026-09-01 -> 2026-10-06 x3: verified from their existing labels under the new rules
- long labels (no analysis): auto/analyses/long-labels/<STATION>/; NORDEND-169 and -195 run to 07/11, so NORDEND-283 and -284 (their continuation) are removed
- reading-rules.md, stations.md: the citations of removed or rewritten labels point to their successor, or to [expert]

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 23:58:00 +01:00

35 lines
1.9 KiB
Python

"""Requalify: per label, ranges.py at --k K in a margin window, level/noise outside the label
(and optionally --rolling); keep the ranges overlapping the label; print the max |noises|."""
import json, subprocess, sys, re
from datetime import datetime, timedelta
D = sys.argv[1]; K = sys.argv[2]; mode = sys.argv[3] # mode: outside | rolling:<N>
ids = sys.argv[4].split(',') if len(sys.argv) > 4 else None
F = '%Y-%m-%dT%H:%MZ'
p = lambda s: datetime.strptime(s, F)
for l in open(f'{D}/work/requalify_input.jsonl'):
L = json.loads(l)
if ids and L['id'] not in ids: continue
if L['family'] == 'No data': continue
s, e = p(L['start']), p(L['end'])
m = max(timedelta(hours=12), (e - s))
cmd = ['python3', 'tools/scripts/ranges.py', 'data/CLARIDEN', '--columns', 'corr_x,corr_y', '--k', K,
'--start', (s - m).strftime(F), '--end', (e + m).strftime(F)]
if mode == 'outside': cmd += ['--outside', f"{L['start']}/{L['end']}"]
else: cmd += ['--rolling', mode.split(':')[1]]
out = subprocess.run(cmd, capture_output=True, text=True).stdout
col = None; hits = []
for line in out.splitlines():
mm = re.match(r'(corr_[xy]): ranges', line)
if mm: col = mm.group(1); continue
if line.startswith('=='): col = None; continue
f = line.split()
if col and len(f) >= 7 and re.match(r'\d{4}-', f[1] if len(f) > 1 else ''):
rs, re_ = p(f[1]), p(f[2])
if re_ < s or rs > e: continue
# skip ranges touching the label edges only (start/end = untouched by rule)
if re_ <= s or rs >= e: continue
hits.append((abs(float(f[5])), f[0], col, f[1], f[2], f[3], f[5]))
hits.sort(reverse=True)
print(f"{L['id']} {L['start']}→{L['end']} {L['family']} [{mode} k{K}]: {len(hits)} ranges")
for h in hits[:6]: print(' ', h[1], h[2], h[3], h[4], 'n=' + h[5], h[6], 'noises')