- Every station's auto labels reviewed by an analyst (/review-auto-labels), then a cross-review per family across all stations (new skill /cross-review-labels), a finishing pass, and descriptions rewritten in words (no numeric values). - The three never-labeled ranges (WILDSTRUBEL 07/09-01/10, NORDEND 16/08-01/10) labeled and reviewed: 991 auto labels on 11 stations, labeling/controle.py at 0 errors. - Labeling knowledge gathered in labeling/: regles-d-etiquetage.md (new), familles.md (settled boundaries), connaissances/ (notes moved from docs/, links updated), the label map rebuilt from reviewed labels, events and cases; events tied to their stations (PS_DES_COUDRES is JUNGFRAU). The labeler and reviewers read only labeling/. - labels-map reads auto labels and events; decision D-130; living plan points 73-75. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
38 lines
2.2 KiB
Python
38 lines
2.2 KiB
Python
"""Table de toutes les étiquettes automatiques, pour la revue croisée.
|
|
Écrit, à côté de outils/ : index.tsv (toutes, description coupée) et par-famille/<famille>.md (en entier).
|
|
Chaque étiquette a un id, <STATION>#<numéro de sa ligne dans labeling/auto/<STATION>.jsonl>. Colonnes ajoutées : durée, heure UTC du début, et les stations qui ont une étiquette au même moment
|
|
(étiquettes de moins de 24 h seulement : une étiquette d'un mois chevauche tout)."""
|
|
import json, glob, os, collections
|
|
from datetime import datetime
|
|
ici = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
|
def t(s): return datetime.fromisoformat(s.replace('Z', '+00:00'))
|
|
es = []
|
|
for p in sorted(glob.glob('labeling/auto/*.jsonl')):
|
|
st = os.path.basename(p)[:-6]
|
|
for n, l in enumerate(open(p), 1):
|
|
if l.strip():
|
|
e = json.loads(l); e['station'] = st; e['id'] = f'{st}#{n}'
|
|
e['_d'], e['_f'] = t(e['debut']), t(e['fin'])
|
|
e['duree_h'] = round((e['_f'] - e['_d']).total_seconds() / 3600, 2)
|
|
es.append(e)
|
|
courtes = [e for e in es if e['duree_h'] < 24]
|
|
for e in es:
|
|
e['simultanees'] = '' if e['duree_h'] >= 24 else ','.join(sorted({
|
|
o['station'] for o in courtes if o['station'] != e['station'] and o['_d'] <= e['_f'] and e['_d'] <= o['_f']}))
|
|
col = ['id', 'debut', 'fin', 'duree_h', 'categorie', 'simultanees', 'description']
|
|
with open(f'{ici}/index.tsv', 'w') as f:
|
|
f.write('\t'.join(col) + '\n')
|
|
for e in sorted(es, key=lambda e: (e['categorie'], e['station'], e['debut'])):
|
|
f.write('\t'.join(str(e[c]) if c != 'description' else e[c][:160].replace('\t', ' ') for c in col) + '\n')
|
|
os.makedirs(f'{ici}/par-famille', exist_ok=True)
|
|
par = collections.defaultdict(list)
|
|
for e in es: par[e['categorie']].append(e)
|
|
for fam, l in par.items():
|
|
with open(f"{ici}/par-famille/{fam.replace(' ', '-')}.md", 'w') as f:
|
|
f.write(f'# {fam} : {len(l)} étiquettes, {len({e["station"] for e in l})} stations\n\n')
|
|
for e in sorted(l, key=lambda e: (e['station'], e['debut'])):
|
|
f.write(f"## {e['id']} {e['debut']} → {e['fin']} ({e['duree_h']} h)\n"
|
|
f"simultanées : {e['simultanees'] or 'aucune'}\n\n{e['description']}\n\n")
|
|
print(fam, len(l))
|
|
print(len(es), 'étiquettes')
|