Files
Support-Agent/labeling/auto/revue-croisee/2026-11-04/target-issue/outils/rejets.py
T
maximeandClaude Opus 5.5 35b5ed4d74 Auto labels reviewed per station and cross-reviewed; all labeling knowledge in labeling/ (D-130)
- Every station's auto labels reviewed by an analyst (/review-auto-labels), then a
  cross-review per family across all stations (new skill /cross-review-labels),
  a finishing pass, and descriptions rewritten in words (no numeric values).
- The three never-labeled ranges (WILDSTRUBEL 07/09-01/10, NORDEND 16/08-01/10)
  labeled and reviewed: 991 auto labels on 11 stations, labeling/controle.py at 0 errors.
- Labeling knowledge gathered in labeling/: regles-d-etiquetage.md (new), familles.md
  (settled boundaries), connaissances/ (notes moved from docs/, links updated),
  the label map rebuilt from reviewed labels, events and cases; events tied to their
  stations (PS_DES_COUDRES is JUNGFRAU). The labeler and reviewers read only labeling/.
- labels-map reads auto labels and events; decision D-130; living plan points 73-75.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 08:21:57 +01:00

39 lines
2.7 KiB
Python

"""Rejets par cible et par jour, toutes les stations étiquetées, du 2026-09-01 au 2026-10-31 (fichiers <STATION>_advanced_AAAAMMJJ.csv seulement).
Une cible (pattern_type != global_ref, sans reproj_err ; noms nettoyés de leurs fins de ligne) est rejetée quand error_value > error_threshold.
Écrit <sortie>/rejets.tsv : station, capteur, jours lus, taux de rejet du lot d'octobre, jours d'octobre à plus de 25 % rejetés,
heures UTC où tombent ses rejets (les 6 plus fréquentes), seuil, médiane de error_value / seuil, et la liste jour par jour (taux en %)."""
import csv, glob, os, re, collections, statistics, sys
ici = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
stations = sys.argv[1:] or ['BREITHORN', 'CLARIDEN', 'FLORIDA', 'JUNGFRAU', 'LENZSPITZE', 'MICHIGAN', 'NEBRASKA',
'NIEDERHORN', 'NORDEND', 'SNCF01', 'WILDSTRUBEL']
out = open(f'{ici}/rejets.tsv', 'w')
out.write('station\tcapteur\ttype\tjours_oct\ttaux_oct\tjours_25pc\theures_rejets\tseuil\tev_sur_seuil\tpar_jour\n')
for st in stations:
n = collections.defaultdict(lambda: collections.Counter()); r = collections.defaultdict(lambda: collections.Counter())
h = collections.defaultdict(collections.Counter); ev = collections.defaultdict(list); seuil = {}; typ = {}
for p in sorted(glob.glob(f'data/{st}/displacement/{st}_advanced_*.csv')):
m = re.search(r'_advanced_(\d{8})\.csv$', p)
if not m or not ('20260901' <= m.group(1) <= '20261031'): continue
vus = set()
for l in csv.DictReader(open(p)):
if l['pattern_type'] == 'global_ref' or l['reproj_err'] or not l['error_value']: continue
l['sensor_name'] = l['sensor_name'].strip()
k = (l['time_utc'][:16], l['sensor_name'])
if k in vus: continue
vus.add(k)
c, j = l['sensor_name'], l['time_utc'][:10]
e, s = float(l['error_value']), float(l['error_threshold'])
n[c][j] += 1; seuil[c] = s; typ[c] = l['pattern_type']
if j >= '2026-10-01': ev[c].append(e / s)
if e > s:
r[c][j] += 1
if j >= '2026-10-01': h[c][l['time_utc'][11:13]] += 1
for c in sorted(n):
jo = [j for j in sorted(n[c]) if j >= '2026-10-01']
if not jo: continue
no, ro = sum(n[c][j] for j in jo), sum(r[c][j] for j in jo)
j25 = sum(1 for j in jo if r[c][j] > 0.25 * n[c][j])
pj = ' '.join(f"{j[5:]}:{round(100 * r[c][j] / n[c][j])}" for j in sorted(n[c]))
out.write(f"{st}\t{c}\t{typ[c]}\t{len(jo)}\t{ro / no:.2f}\t{j25}\t{','.join(x for x, _ in h[c].most_common(6))}\t{seuil[c]}\t{statistics.median(ev[c]):.2f}\t{pj}\n")
print(st, len(n))