review is now the date the user validated a label in aimsight-app; the agents' review dates removed from auto/

Quality over quantity: only the analyses discussed and validated in the app are marked. validate.py no longer
requires review; precedents.py gives the provenance as validated by the user or not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
maximeandClaude Opus 5.5 committed 2026-11-17 10:01:22 +01:00
1 parent 542e91ccb2
commit bed0f7d09e
15 files changed
+1007 -1005

No files matched your search

+5 -5
View File
@@ -2,7 +2,7 @@
uses it.
The analyst describes what it sees on its range; the tool returns the labels already written that speak of it most, all
stations, with their provenance (reviewed, not reviewed): it is up to the analyst to judge whether they fit its case.
stations, with their provenance (validated by the user, not validated): it is up to the analyst to judge whether they fit its case.
python3 labeling/scripts/precedents.py "the targets jump together when a reference is rejected intermittently"
python3 labeling/scripts/precedents.py "<text>" --exclude NORDEND 2026-11-04T21:12Z 2026-11-07T14:12Z --family Obstruction --k 5
@@ -47,8 +47,8 @@ def labels(root=HERE):
def provenance(e):
if e.get("review"):
return f"auto, reviewed on {e['review']}"
return "auto, not reviewed" + (f" (analysis {e['analysis']})" if e.get("analysis") else "")
return f"auto, validated by the user on {e['review']}"
return "auto, not validated" + (f" (analysis {e['analysis']})" if e.get("analysis") else "")
def text(e):
@@ -127,14 +127,14 @@ def check():
assert overlaps(e, "S", "2026-11-03T00:00Z", "2026-11-04T00:00Z") # common edge: excluded
assert not overlaps(e, "S", "2026-11-05T00:01Z", "2026-11-06T00:00Z")
assert not overlaps(e, "T", "2026-11-04T12:00Z", "2026-11-06T00:00Z")
assert provenance({"review": "2026-11-04"}) == "auto, reviewed on 2026-11-04"
assert provenance({"review": "2026-11-04"}) == "auto, validated by the user on 2026-11-04"
import tempfile # only the labels of auto/ are used: manual/ is the user's alone
with tempfile.TemporaryDirectory() as d:
for sub, desc in (("manual", "the user's"), ("auto", "an agent's")):
(Path(d) / sub).mkdir()
(Path(d) / sub / "S.jsonl").write_text(json.dumps({**e, "description": desc}) + "\n", encoding="utf-8")
assert [x["description"] for x in labels(Path(d))] == ["an agent's"]
assert provenance({"analysis": "X"}) == "auto, not reviewed (analysis X)"
assert provenance({"analysis": "X"}) == "auto, not validated (analysis X)"
a, b = {**e, "family": "A", "description": "a"}, {**e, "station": "T", "family": "B", "description": "b"}
vec = {key(text(a)): [1.0, 0.0], key(text(b)): [0.6, 0.8]}
assert [x["family"] for _, x in search("", [a, b], vec, [0.0, 1.0])] == ["B", "A"]
+7 -5
View File
@@ -2,10 +2,12 @@
python3 labeling/scripts/validate.py # every station
python3 labeling/scripts/validate.py NORDEND # one station
python3 labeling/scripts/validate.py <folder>/labels.jsonl # a proposed file, before it is submitted (no `review` required)
python3 labeling/scripts/validate.py <folder>/labels.jsonl # a proposed file, before it is submitted (no `id` required)
`review` (optional): the date the user validated the label in aimsight-app; any change to the label removes it.
Errors (to fix): unreadable JSON, missing field, unknown family, malformed date, start >= end,
label without `review` or without `id` (`python3 labeling/scripts/ids.py`), id not first in the line, duplicate id, exact duplicate,
label without `id` (`python3 labeling/scripts/ids.py`), id not first in the line, duplicate id, exact duplicate,
numeric value, local time, "probably not a movement". "Nothing to worry about" is right only with an established cause: to
judge by reading, not by script.
Flagged (to look at): overlaps of two labels of the same family on the same station.
@@ -27,9 +29,9 @@ FORBIDDEN = re.compile(r"local time|local morning|local evening|local day|probab
def check(e, proposed=False):
"""The errors of one label (a dict) against the rules, without those of its file (duplicates, overlaps) -> [text].
proposed: a label not yet submitted, without `review` nor `id`. For a caller (aimsight-app: a label written in auto/)."""
proposed: a label not yet submitted, without `id`. For a caller (aimsight-app: a label written in auto/)."""
errors = []
missing = ({"start", "end", "family", "description"} | (set() if proposed else {"review", "id"})) - e.keys()
missing = ({"start", "end", "family", "description"} | (set() if proposed else {"id"})) - e.keys()
if missing:
errors.append(f"missing field: {sorted(missing)}")
if "id" in e and next(iter(e)) != "id":
@@ -55,7 +57,7 @@ def main():
errors, flagged, total = [], [], 0
for st in stations:
seen, ids, labels = set(), set(), []
proposed = st.endswith(".jsonl") # a proposed file: its lines have no `review` nor `id` yet
proposed = st.endswith(".jsonl") # a proposed file: its lines have no `id` yet
path = st if proposed else os.path.join(AUTO, f"{st}.jsonl")
for n, line in enumerate(open(path, encoding="utf-8"), 1):
if not line.strip():