4.5 KiB
4.5 KiB
detectors
AIMsight anomaly detectors (Python 3, standard library only) and their benchmark (bench/); nothing else. They are improved by Optimizer (optimize skill) with the upgrader and validator of the agents repository. Everything is in English: code, identifiers, docs, outputs. Layout and usage: README.md.
Contract (do not break)
scripts/categories.pyruns the detectors ofDETECTORS(modules ofscripts/:visibility,target_issue,thermal_displacement,no_data,displacement) and makes their episodes events. A module returns every family it finds:visibilityalso gives Brief obstruction andtarget_issueTarget obscured (Obstruction).episodes.pyholds the helpers they share.detect(measurements)(no_data: alsoend, the end of the window read) returns episodes with the keysstart,end,kind(the family),label(the variety),text(the sentence shown),ignore,sensors; a Poor visibility also haslevel.- The measurements are read by
scripts/data.pyof the tools repository, the sibling~/aimsight-tools(categories.pyputs it on the path); a reading fix is made there. - Label files (JSONL):
id,start,end,family,description(andreview,analysis); the project useslabeling/autoonly. - Two repositories use this one through a
detectors -> ~/aimsight-detectorslink (one copy on the workstation: a commit here is live for both at once); a rename here needs the matching change there:- aimsight-app:
categories.events,categories.CATEGORIES(process:images,none,check), the event fields (start,end,category,text,sensors,ignore,episodes), the--from/--toCLIs,bench/score.py(the command shown in the page); - Optimizer (
optimizeskill and its agents):scripts/categories.py --from/--to,bench/(score.py,regression.py,sensors.py); its upgrader changesscripts/*.pyandthresholds.md, one commit per fix.
- aimsight-app:
Rules
- A change to improve the detectors touches
scripts/*.pyandthresholds.mdonly; neverbench/, which is the measure. Anything is possible there: add, remove or change a constant, a rule or a whole detector, as long as the six families remain. - Simple first: a rule earns its place by its score on the benchmark, not by one case; prefer removing or widening a rule to adding a special case.
- Everything is detected: what is shown or not to a client is not the detectors' business. A setting chosen not to show something may be changed, saying so.
- A constant added or changed updates its line of
thresholds.md("Tuned on": the run that changed it) and the detector's docstring in the same change; a constant or rule removed removes its line; check withgrep -nE "^[A-Z_0-9]+ = " scripts/*.py. - A detector that fails is skipped: a
skippedline in the output ofcategories.pyor of the benchmark means the change is broken. - No rule specific to a station, a sensor or a date.
- A change to a detector is checked with the benchmark:
python3 bench/regression.py --labels <labels folder> --data <data folder>(aimsight-app and Optimizer:labeling/auto,data, links to the shared~/aimsight-labelingand~/aimsight-data/data; about 1 min). A refactoring must leave every figure unchanged. - The benchmark's scores, per family of each station: "found", the share of the labelled instants the detectors flag; "correct", the share of the flagged instants a label puts in the family. A regression, on a family of a station: "found" or "correct" loses more than 2 points; it is a regression even if the
Totalrises; under 2 points it is not one; a rise is a plus, never a requirement. Not a regression: a drop that comes from another consistent reading of the detectors that the labels do not retain: labels contradicting each other from one station to another (the same state in two families), the gaps of a few measurements between two labels of the same phenomenon, the bounds of a label, an instant the score (per instant, without the sensors) counted as found by an event on another sensor. Labels have their ambiguities: the goal is detectors that do their best, not perfect, and not overtrained on the labels. - History references in docstrings and
thresholds.md("wf-improve X, mod. 1, S2", "X, batch 01", D-xxx, E-xxx) point to past runs and to decisions recorded in aimsight-app/Support-Agent; keep them short when editing a rule. - The bench tools take their paths as arguments; the architecture of the repositories:
~/aimsight-app/docs/architecture.drawio.