Files
aimsight-detectors/scripts/no_data.py
T
maximeandClaude Opus 5.5 0de006bcb2 wf-improve generic: the caller gives the range, labels, reference truth, families, context and output folder
Skill and agents no longer name Diagnostic-app paths (analyses, labeling/auto,
demande.json, summary location, git rules of the working repository).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:40:30 +01:00

46 lines
2.2 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""Detector of the No data family (the families document): moments without a measurement (the cadence of the "advanced"
export is broken).
Cadence: median gap between two measurement instants, all sensors together. A hole: more than MISSING missing steps
between two measurements (gap > MISSING + 1 cadences); the episode runs from the last measurement to the next one.
End of the window read (end, when the caller gives it: the chunked run of categories.py): the same rule between the last
measurement and end, the episode running up to end (CLARIDEN, batch 01: 44 h without a measurement up to the end of
the batch, not reported). No measurement in the export does not mean the station stopped: the text only says that the
export has nothing.
Usage: python3 detectors/no_data.py <station folder> (prints the episodes)
"""
import os
import statistics
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) # data.py, next to it
from data import iso, load # noqa: E402
MISSING = 2 # more than 2 missing steps is a hole (user's rule, 2026-10-23)
def duration(s):
h, m = divmod(round(s / 60), 60)
return f"{h} h {m:02d} min" if h else f"{m} min"
def detect(measurements, end=None):
"""end: end of the window read (datetime, excluded), optional; without it, a final hole is not seen."""
ts = sorted({m["t"].replace(second=0, microsecond=0) for ms in measurements.values() for m in ms})
if len(ts) < 3:
return []
gaps = [(b - a).total_seconds() for a, b in zip(ts, ts[1:])]
step = statistics.median(gaps)
out = [(a, b, g, "") for a, b, g in zip(ts, ts[1:], gaps)]
if end: # end excluded: the last expected measurement is at end − 1 step, the gap counts as between two measurements
out.append((ts[-1], end, (end - ts[-1]).total_seconds(), ", up to the end of the period"))
return [{"start": iso(a), "end": iso(b), "kind": "No data", "label": "No data",
"text": f"No measurement in the export for {duration(g)}{tail} (usually one every {duration(step)}).",
"ignore": False, "sensors": []}
for a, b, g, tail in out if g > (MISSING + 1) * step]
if __name__ == "__main__":
for e in detect(load(sys.argv[1])):
print(e)