review is now the date the user validated a label in aimsight-app; the agents' review dates removed from auto/
Quality over quantity: only the analyses discussed and validated in the app are marked. validate.py no longer requires review; precedents.py gives the provenance as validated by the user or not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
542e91ccb2
commit
bed0f7d09e
15 files changed
+1007
-1005
No files matched your search
+7
-5
@@ -2,10 +2,12 @@
|
||||
|
||||
python3 labeling/scripts/validate.py # every station
|
||||
python3 labeling/scripts/validate.py NORDEND # one station
|
||||
python3 labeling/scripts/validate.py <folder>/labels.jsonl # a proposed file, before it is submitted (no `review` required)
|
||||
python3 labeling/scripts/validate.py <folder>/labels.jsonl # a proposed file, before it is submitted (no `id` required)
|
||||
|
||||
`review` (optional): the date the user validated the label in aimsight-app; any change to the label removes it.
|
||||
|
||||
Errors (to fix): unreadable JSON, missing field, unknown family, malformed date, start >= end,
|
||||
label without `review` or without `id` (`python3 labeling/scripts/ids.py`), id not first in the line, duplicate id, exact duplicate,
|
||||
label without `id` (`python3 labeling/scripts/ids.py`), id not first in the line, duplicate id, exact duplicate,
|
||||
numeric value, local time, "probably not a movement". "Nothing to worry about" is right only with an established cause: to
|
||||
judge by reading, not by script.
|
||||
Flagged (to look at): overlaps of two labels of the same family on the same station.
|
||||
@@ -27,9 +29,9 @@ FORBIDDEN = re.compile(r"local time|local morning|local evening|local day|probab
|
||||
|
||||
def check(e, proposed=False):
|
||||
"""The errors of one label (a dict) against the rules, without those of its file (duplicates, overlaps) -> [text].
|
||||
proposed: a label not yet submitted, without `review` nor `id`. For a caller (aimsight-app: a label written in auto/)."""
|
||||
proposed: a label not yet submitted, without `id`. For a caller (aimsight-app: a label written in auto/)."""
|
||||
errors = []
|
||||
missing = ({"start", "end", "family", "description"} | (set() if proposed else {"review", "id"})) - e.keys()
|
||||
missing = ({"start", "end", "family", "description"} | (set() if proposed else {"id"})) - e.keys()
|
||||
if missing:
|
||||
errors.append(f"missing field: {sorted(missing)}")
|
||||
if "id" in e and next(iter(e)) != "id":
|
||||
@@ -55,7 +57,7 @@ def main():
|
||||
errors, flagged, total = [], [], 0
|
||||
for st in stations:
|
||||
seen, ids, labels = set(), set(), []
|
||||
proposed = st.endswith(".jsonl") # a proposed file: its lines have no `review` nor `id` yet
|
||||
proposed = st.endswith(".jsonl") # a proposed file: its lines have no `id` yet
|
||||
path = st if proposed else os.path.join(AUTO, f"{st}.jsonl")
|
||||
for n, line in enumerate(open(path, encoding="utf-8"), 1):
|
||||
if not line.strip():
|
||||
|
||||
Reference in new issue
Block a user