For aimsight-app: validate.check (the rules of one label), client.py (what the client is told of an event, from families.md); families.md: For the client per variety
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
83f92bb7b9
commit
542e91ccb2
5 files changed
+98
-22
No files matched your search
@@ -4,7 +4,8 @@ labeling: the AIMsight displacement labels and the knowledge to place them, in a
|
|||||||
|
|
||||||
- **The labels**: `manual/<STATION>.jsonl` (the expert, for him alone), `auto/<STATION>.jsonl` (the agents, reviewed by `review`), one per line, `{"id", "start", "end", "family", "description", ...}`. The aimsight-app analyses: `auto/analyses/<STATION>_<start>_<end>/`.
|
- **The labels**: `manual/<STATION>.jsonl` (the expert, for him alone), `auto/<STATION>.jsonl` (the agents, reviewed by `review`), one per line, `{"id", "start", "end", "family", "description", ...}`. The aimsight-app analyses: `auto/analyses/<STATION>_<start>_<end>/`.
|
||||||
- **The knowledge**, one source per kind (`README.md`), maintained by hand: `families.md` (the families, their boundaries, the catalogue of phenomena), `labeling-rules.md` (bounds, a single event, description, stated cause), `reading-rules.md` (what the data mean), `stations.md` (the installations); the cases are the labels, through `precedents.py`; what the data are: `tools/DATA.md`. A piece of knowledge lives in one of them only.
|
- **The knowledge**, one source per kind (`README.md`), maintained by hand: `families.md` (the families, their boundaries, the catalogue of phenomena), `labeling-rules.md` (bounds, a single event, description, stated cause), `reading-rules.md` (what the data mean), `stations.md` (the installations); the cases are the labels, through `precedents.py`; what the data are: `tools/DATA.md`. A piece of knowledge lives in one of them only.
|
||||||
- **The tools**, standard library: `python3 scripts/validate.py [STATION | file.jsonl]` (the rules), `python3 scripts/ids.py [STATION]` (the stable id of new labels), `python3 scripts/precedents.py "<text>" [--exclude STATION START END] [--family F] [--k N]` (the RAG; it starts the Ollama container `ollama-bge-m3` by itself if it does not answer; `--check` without it).
|
- **The tools**, standard library: `python3 scripts/validate.py [STATION | file.jsonl]` (the rules), `python3 scripts/ids.py [STATION]` (the stable id of new labels), `python3 scripts/client.py` (what the client is told of a detectors' event, from `families.md`; its check), `python3 scripts/precedents.py "<text>" [--exclude STATION START END] [--family F] [--k N]` (the RAG; it starts the Ollama container `ollama-bge-m3` by itself if it does not answer; `--check` without it).
|
||||||
|
- **For aimsight-app**, which calls them and holds no rule of its own: `validate.check(label, proposed)` (the rules of one label, before it is written in `auto/`), `ids.assign`, `client.message` and `client.action` (the "For the client" columns of `families.md`, per family and per variety). A rename here needs the change there.
|
||||||
- **Verify**: `python3 scripts/validate.py` (today 19 expected errors: the labels submitted by the app without `review`) and `python3 scripts/precedents.py --check`.
|
- **Verify**: `python3 scripts/validate.py` (today 19 expected errors: the labels submitted by the app without `review`) and `python3 scripts/precedents.py --check`.
|
||||||
|
|
||||||
Rules:
|
Rules:
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ The AIMsight displacement labels and all the domain knowledge to place them, in
|
|||||||
| What the data mean: measurement chain, gaps, reading rules, false friends | `reading-rules.md` |
|
| What the data mean: measurement chain, gaps, reading rules, false friends | `reading-rules.md` |
|
||||||
| The installation of each station | `stations.md` |
|
| The installation of each station | `stations.md` |
|
||||||
| What the data are: files, columns, image quality, images | `tools/DATA.md` (tools repository) |
|
| What the data are: files, columns, image quality, images | `tools/DATA.md` (tools repository) |
|
||||||
- `scripts/`: `validate.py` (the labels against the rules), `ids.py` (the stable id of each).
|
- `scripts/`: `validate.py` (the labels against the rules; `check` for one label), `ids.py` (the stable id of each), `client.py` (what the client is told of a detectors' event).
|
||||||
- `scripts/precedents.py`: the RAG of the labels. The labels of `auto/` closest in meaning to a text, with their provenance (reviewed or not); `--exclude STATION START END` leaves out the analyzed range. bge-m3 model of the Ollama container `ollama-bge-m3` (started by the script itself if it does not answer; port 11435); the `.index/` index, outside git, updates itself at every call (only new or modified labels go through the model; ~8 min to redo everything on CPU).
|
- `scripts/precedents.py`: the RAG of the labels. The labels of `auto/` closest in meaning to a text, with their provenance (reviewed or not); `--exclude STATION START END` leaves out the analyzed range. bge-m3 model of the Ollama container `ollama-bge-m3` (started by the script itself if it does not answer; port 11435); the `.index/` index, outside git, updates itself at every call (only new or modified labels go through the model; ~8 min to redo everything on CPU).
|
||||||
|
|
||||||
aimsight-app and Optimizer reach it through a link `labeling -> ~/aimsight-labeling`, ignored by git: aimsight-app reads and writes there (labels, analyses), Optimizer only reads; commits are made here. The scripts run from here (`python3 scripts/validate.py`) or through the link (`python3 labeling/scripts/validate.py`). Descriptions are written in English from now on.
|
aimsight-app and Optimizer reach it through a link `labeling -> ~/aimsight-labeling`, ignored by git: aimsight-app reads and writes there (labels, analyses), Optimizer only reads; commits are made here. The scripts run from here (`python3 scripts/validate.py`) or through the link (`python3 labeling/scripts/validate.py`). Descriptions are written in English from now on.
|
||||||
|
|||||||
+13
@@ -11,6 +11,19 @@ One label, one anomaly, one family (D-109 to D-111, D-119). An anomaly is what s
|
|||||||
| `Target issue` | `target_issue.py` | a constantly problematic target: rejected almost always, badly oriented, badly installed, badly configured, dirty | act on the target; ignore its curve |
|
| `Target issue` | `target_issue.py` | a constantly problematic target: rejected almost always, badly oriented, badly installed, badly configured, dirty | act on the target; ignore its curve |
|
||||||
| `No data` | `no_data.py` | no measurement for any sensor, all curves missing (acquisition, power, transmission); the cause, if the metrics or the images show it, otherwise not established | the curves are missing over the range; the cause says whether to act |
|
| `No data` | `no_data.py` | no measurement for any sensor, all curves missing (acquisition, power, transmission); the cause, if the metrics or the images show it, otherwise not established | the curves are missing over the range; the cause says whether to act |
|
||||||
|
|
||||||
|
For the client, some varieties of a family (the `label` of the detectors' episode, and its level when it has one) say more than their family; the others take the family's:
|
||||||
|
|
||||||
|
| Variety | For the client |
|
||||||
|
|---|---|
|
||||||
|
| `Poor visibility reduced` | use the data over the range with care: they are less precise |
|
||||||
|
| `Heat shimmer` | nothing to do: the small fast wiggles are not movement |
|
||||||
|
| `Camera thermal movement` | nothing to do: this is not a movement of the structure |
|
||||||
|
| `Brief obstruction` | ignore this single point on the curves |
|
||||||
|
| `Sudden jump, back to normal` | look at the images: the cause is not established |
|
||||||
|
| `Lasting shift` | check whether the structure moved: look at the images |
|
||||||
|
| `Jump, not back yet` | check whether the structure moved: look at the images |
|
||||||
|
| `Slow drift` | keep watching it |
|
||||||
|
|
||||||
## The boundaries
|
## The boundaries
|
||||||
|
|
||||||
- **Visibility or Obstruction**: the nuisance is judged by its extent, not by the light. The whole scene hindered (rain, fog, underexposed image, milky dawn, sun in the field) is Visibility; a few targets hidden by an object, the rest intact, are Obstruction, even if the object is lit (machine, headlights, light of a parked vehicle). A spider web goes by what it does: a thread that hides a target, Obstruction; the web that veils the image, Visibility.
|
- **Visibility or Obstruction**: the nuisance is judged by its extent, not by the light. The whole scene hindered (rain, fog, underexposed image, milky dawn, sun in the field) is Visibility; a few targets hidden by an object, the rest intact, are Obstruction, even if the object is lit (machine, headlights, light of a parked vehicle). A spider web goes by what it does: a thread that hides a target, Obstruction; the web that veils the image, Visibility.
|
||||||
|
|||||||
@@ -0,0 +1,43 @@
|
|||||||
|
"""What the client is told of a detectors' event: what happens (message) and what to do (action), the latter from the
|
||||||
|
"For the client" columns of families.md (per family, refined per variety). For a caller (aimsight-app, the Data page).
|
||||||
|
|
||||||
|
python3 labeling/scripts/client.py # a check
|
||||||
|
|
||||||
|
Standard library.
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
|
||||||
|
FAMILIES_MD = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "families.md")
|
||||||
|
|
||||||
|
|
||||||
|
def advice():
|
||||||
|
"""{key: text}: the rows "| `key` | … | text |" of the tables of families.md, the key a family or a variety."""
|
||||||
|
with open(FAMILIES_MD, encoding="utf-8") as f:
|
||||||
|
return {m.group(1): m.group(2).strip() for line in f if (m := re.match(r"\| `([^`]+)` \|(?:.*\|)? ([^|]+) \|\s*$", line))}
|
||||||
|
|
||||||
|
|
||||||
|
def action(family, variety, level=None, table=None):
|
||||||
|
"""What to do: the variety with its level, otherwise the variety, otherwise the family; "" without. table: advice(),
|
||||||
|
read once by a caller with many events."""
|
||||||
|
t = advice() if table is None else table
|
||||||
|
text = t.get(f"{variety} {level or ''}".strip()) or t.get(variety) or t.get(family, "")
|
||||||
|
return text[:1].upper() + text[1:] + "." * bool(text and not text.endswith("."))
|
||||||
|
|
||||||
|
|
||||||
|
def message(text):
|
||||||
|
"""What happens: the first sentence of the event's text (detectors/scripts/categories.py), up to its ";"; the rest
|
||||||
|
says what to do about it, except the sentence of the absorbed jumps, kept. ". " only: "1.5×" or "0.30 px" do not cut."""
|
||||||
|
same = re.search(r" (The jumps? of .* same cause\.)$", text)
|
||||||
|
return re.split(r"\. |; ", text, maxsplit=1)[0].rstrip(".") + "." + (" " + same.group(1) if same else "")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
t = advice()
|
||||||
|
assert action("Visibility", "Poor visibility", "severe", t) == "Do not trust the curves over the range; it is not a movement.", t
|
||||||
|
assert action("Visibility", "Poor visibility", "reduced", t).startswith("Use the data over the range with care")
|
||||||
|
assert action("Displacement", "Slow drift", None, t) == "Keep watching it."
|
||||||
|
assert action("Unknown", "x", None, t) == ""
|
||||||
|
assert message("A moved; nothing to do. The jump of 2 targets in this period has the same cause.") \
|
||||||
|
== "A moved. The jump of 2 targets in this period has the same cause."
|
||||||
|
print("ok")
|
||||||
+39
-20
@@ -24,6 +24,33 @@ DATE = re.compile(r"^\d{4}-\d\d-\d\dT\d\d:\d\dZ$")
|
|||||||
NUMBER = re.compile(r"(?<![\w.])\d+,\d+|\b\d+\s*(noise|noises|px|°|%|MAD|mm|times)(?!\w)|\b\d+\s+(targets|measurements) (out )?of\b")
|
NUMBER = re.compile(r"(?<![\w.])\d+,\d+|\b\d+\s*(noise|noises|px|°|%|MAD|mm|times)(?!\w)|\b\d+\s+(targets|measurements) (out )?of\b")
|
||||||
FORBIDDEN = re.compile(r"local time|local morning|local evening|local day|probably not a movement", re.I)
|
FORBIDDEN = re.compile(r"local time|local morning|local evening|local day|probably not a movement", re.I)
|
||||||
|
|
||||||
|
|
||||||
|
def check(e, proposed=False):
|
||||||
|
"""The errors of one label (a dict) against the rules, without those of its file (duplicates, overlaps) -> [text].
|
||||||
|
proposed: a label not yet submitted, without `review` nor `id`. For a caller (aimsight-app: a label written in auto/)."""
|
||||||
|
errors = []
|
||||||
|
missing = ({"start", "end", "family", "description"} | (set() if proposed else {"review", "id"})) - e.keys()
|
||||||
|
if missing:
|
||||||
|
errors.append(f"missing field: {sorted(missing)}")
|
||||||
|
if "id" in e and next(iter(e)) != "id":
|
||||||
|
errors.append("id not first in the line (`python3 labeling/scripts/ids.py`)")
|
||||||
|
if e.get("family") not in FAMILIES:
|
||||||
|
errors.append(f"unknown family: {e.get('family')}")
|
||||||
|
if not (DATE.match(e.get("start") or "") and DATE.match(e.get("end") or "")):
|
||||||
|
return errors + ["malformed date"]
|
||||||
|
if e["start"] >= e["end"]:
|
||||||
|
errors.append(f"start >= end ({e['start']})")
|
||||||
|
d = e.get("description") or ""
|
||||||
|
if "description" in e and not d.strip():
|
||||||
|
errors.append("empty description")
|
||||||
|
if m := NUMBER.search(d):
|
||||||
|
errors.append(f"numeric value: \"{m.group(0)}\"")
|
||||||
|
if m := FORBIDDEN.search(d):
|
||||||
|
errors.append(f"forbidden: \"{m.group(0)}\"")
|
||||||
|
return errors
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
stations = sys.argv[1:] or sorted(os.path.basename(p)[:-6] for p in glob.glob(os.path.join(AUTO, "*.jsonl")))
|
stations = sys.argv[1:] or sorted(os.path.basename(p)[:-6] for p in glob.glob(os.path.join(AUTO, "*.jsonl")))
|
||||||
errors, flagged, total = [], [], 0
|
errors, flagged, total = [], [], 0
|
||||||
for st in stations:
|
for st in stations:
|
||||||
@@ -39,35 +66,23 @@ for st in stations:
|
|||||||
e = json.loads(line)
|
e = json.loads(line)
|
||||||
except json.JSONDecodeError:
|
except json.JSONDecodeError:
|
||||||
errors.append(f"{where} unreadable JSON"); continue
|
errors.append(f"{where} unreadable JSON"); continue
|
||||||
missing = ({"start", "end", "family", "description"} | (set() if proposed else {"review", "id"})) - e.keys()
|
found = check(e, proposed)
|
||||||
if missing:
|
|
||||||
errors.append(f"{where} missing field: {sorted(missing)}")
|
|
||||||
if "id" in e:
|
if "id" in e:
|
||||||
if next(iter(e)) != "id":
|
|
||||||
errors.append(f"{where} id not first in the line (`python3 labeling/scripts/ids.py`)")
|
|
||||||
if e["id"] in ids:
|
if e["id"] in ids:
|
||||||
errors.append(f"{where} duplicate id: {e['id']}")
|
found.append(f"duplicate id: {e['id']}")
|
||||||
ids.add(e["id"])
|
ids.add(e["id"])
|
||||||
if e.get("family") not in FAMILIES:
|
errors += [f"{where} {f}" for f in found]
|
||||||
errors.append(f"{where} unknown family: {e.get('family')}")
|
if "malformed date" in found:
|
||||||
if not (DATE.match(e.get("start", "")) and DATE.match(e.get("end", ""))):
|
continue
|
||||||
errors.append(f"{where} malformed date"); continue
|
key = (e["start"], e["end"], e.get("family"), e.get("description"))
|
||||||
if e["start"] >= e["end"]:
|
|
||||||
errors.append(f"{where} start >= end ({e['start']})")
|
|
||||||
key = (e["start"], e["end"], e["family"], e["description"])
|
|
||||||
if key in seen:
|
if key in seen:
|
||||||
errors.append(f"{where} exact duplicate")
|
errors.append(f"{where} exact duplicate")
|
||||||
seen.add(key)
|
seen.add(key)
|
||||||
d = e.get("description", "")
|
|
||||||
if m := NUMBER.search(d):
|
|
||||||
errors.append(f"{where} numeric value: \"{m.group(0)}\"")
|
|
||||||
if m := FORBIDDEN.search(d):
|
|
||||||
errors.append(f"{where} forbidden: \"{m.group(0)}\"")
|
|
||||||
labels.append((where, e))
|
labels.append((where, e))
|
||||||
for i, (a, x) in enumerate(labels):
|
for i, (a, x) in enumerate(labels):
|
||||||
for b, y in labels[i + 1:]:
|
for b, y in labels[i + 1:]:
|
||||||
if x["family"] == y["family"] and x["start"] < y["end"] and y["start"] < x["end"]:
|
if x.get("family") == y.get("family") and x["start"] < y["end"] and y["start"] < x["end"]:
|
||||||
flagged.append(f"{a} and {b}: two {x['family']} that overlap")
|
flagged.append(f"{a} and {b}: two {x.get('family')} that overlap")
|
||||||
|
|
||||||
print(f"{total} labels, {len(stations)} stations: {len(errors)} errors, {len(flagged)} overlaps to look at")
|
print(f"{total} labels, {len(stations)} stations: {len(errors)} errors, {len(flagged)} overlaps to look at")
|
||||||
for l in errors:
|
for l in errors:
|
||||||
@@ -77,3 +92,7 @@ for l in flagged[:40]:
|
|||||||
if len(flagged) > 40:
|
if len(flagged) > 40:
|
||||||
print(f" … and {len(flagged) - 40} other overlaps")
|
print(f" … and {len(flagged) - 40} other overlaps")
|
||||||
sys.exit(1 if errors else 0)
|
sys.exit(1 if errors else 0)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Reference in new issue
Block a user