review is now the date the user validated a label in aimsight-app; the agents' review dates removed from auto/

Quality over quantity: only the analyses discussed and validated in the app are marked. validate.py no longer
requires review; precedents.py gives the provenance as validated by the user or not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
maximeandClaude Opus 5.5 committed 2026-11-17 10:01:22 +01:00
1 parent 542e91ccb2
commit bed0f7d09e
15 files changed
+1007 -1005

No files matched your search

+7 -5
View File
@@ -2,10 +2,12 @@
python3 labeling/scripts/validate.py # every station
python3 labeling/scripts/validate.py NORDEND # one station
python3 labeling/scripts/validate.py <folder>/labels.jsonl # a proposed file, before it is submitted (no `review` required)
python3 labeling/scripts/validate.py <folder>/labels.jsonl # a proposed file, before it is submitted (no `id` required)
`review` (optional): the date the user validated the label in aimsight-app; any change to the label removes it.
Errors (to fix): unreadable JSON, missing field, unknown family, malformed date, start >= end,
label without `review` or without `id` (`python3 labeling/scripts/ids.py`), id not first in the line, duplicate id, exact duplicate,
label without `id` (`python3 labeling/scripts/ids.py`), id not first in the line, duplicate id, exact duplicate,
numeric value, local time, "probably not a movement". "Nothing to worry about" is right only with an established cause: to
judge by reading, not by script.
Flagged (to look at): overlaps of two labels of the same family on the same station.
@@ -27,9 +29,9 @@ FORBIDDEN = re.compile(r"local time|local morning|local evening|local day|probab
def check(e, proposed=False):
"""The errors of one label (a dict) against the rules, without those of its file (duplicates, overlaps) -> [text].
proposed: a label not yet submitted, without `review` nor `id`. For a caller (aimsight-app: a label written in auto/)."""
proposed: a label not yet submitted, without `id`. For a caller (aimsight-app: a label written in auto/)."""
errors = []
missing = ({"start", "end", "family", "description"} | (set() if proposed else {"review", "id"})) - e.keys()
missing = ({"start", "end", "family", "description"} | (set() if proposed else {"id"})) - e.keys()
if missing:
errors.append(f"missing field: {sorted(missing)}")
if "id" in e and next(iter(e)) != "id":
@@ -55,7 +57,7 @@ def main():
errors, flagged, total = [], [], 0
for st in stations:
seen, ids, labels = set(), set(), []
proposed = st.endswith(".jsonl") # a proposed file: its lines have no `review` nor `id` yet
proposed = st.endswith(".jsonl") # a proposed file: its lines have no `id` yet
path = st if proposed else os.path.join(AUTO, f"{st}.jsonl")
for n, line in enumerate(open(path, encoding="utf-8"), 1):
if not line.strip():