Files
aimsight-detectors/scripts/no_data.py
T
maximeandClaude Opus 5.5 969637499f Detectors in English, under scripts/; learner tools under scripts/learner/
- data.py (was donnees.py): unused filtrer, heure_image and image sorting removed
- score.py: dead functions (validite, marquer, detectes, score, familles) and the
  validity levels removed; --from/--to
- regression.py: history.csv and --seance/--etape removed
- categories.py: --from/--to/--chunk/--history; process values images/none/check
- benchmark on all stations unchanged (same 132 figures)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 12:19:52 +01:00

46 lines
2.2 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""Detector of the No data family (labeling/familles.md): moments without a measurement (the cadence of the "advanced"
export is broken).
Cadence: median gap between two measurement instants, all sensors together. A hole: more than MISSING missing steps
between two measurements (gap > MISSING + 1 cadences); the episode runs from the last measurement to the next one.
End of the window read (end, when the caller gives it: the chunked run of categories.py): the same rule between the last
measurement and end, the episode running up to end (CLARIDEN, batch 01: 44 h without a measurement up to the end of
the batch, not reported). No measurement in the export does not mean the station stopped: the text only says that the
export has nothing.
Usage: python3 detectors/no_data.py <station folder> (prints the episodes)
"""
import os
import statistics
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) # data.py, next to it
from data import iso, load # noqa: E402
MISSING = 2 # more than 2 missing steps is a hole (user's rule, 2026-10-23)
def duration(s):
h, m = divmod(round(s / 60), 60)
return f"{h} h {m:02d} min" if h else f"{m} min"
def detect(measurements, end=None):
"""end: end of the window read (datetime, excluded), optional; without it, a final hole is not seen."""
ts = sorted({m["t"].replace(second=0, microsecond=0) for ms in measurements.values() for m in ms})
if len(ts) < 3:
return []
gaps = [(b - a).total_seconds() for a, b in zip(ts, ts[1:])]
step = statistics.median(gaps)
out = [(a, b, g, "") for a, b, g in zip(ts, ts[1:], gaps)]
if end: # end excluded: the last expected measurement is at end − 1 step, the gap counts as between two measurements
out.append((ts[-1], end, (end - ts[-1]).total_seconds(), ", up to the end of the period"))
return [{"start": iso(a), "end": iso(b), "kind": "No data", "label": "No data",
"text": f"No measurement in the export for {duration(g)}{tail} (usually one every {duration(step)}).",
"ignore": False, "sensors": []}
for a, b, g, tail in out if g > (MISSING + 1) * step]
if __name__ == "__main__":
for e in detect(load(sys.argv[1])):
print(e)