Compare commits

...
5 Commits
3 changed files with 15 additions and 11 deletions

No files matched your search

+12 -8
View File
@@ -1,20 +1,24 @@
# detectors
AIMsight anomaly detectors (Python 3, standard library only) and their benchmark (`bench/`); nothing else. They are improved by Optimizer (`improve` skill) with the upgrader and validator of the agents repository. Everything is in English: code, identifiers, docs, outputs. Layout and usage: `README.md`.
AIMsight anomaly detectors (Python 3, standard library only) and their benchmark (`bench/`); nothing else. They are improved by Optimizer (`optimize` skill) with the upgrader and validator of the agents repository. Everything is in English: code, identifiers, docs, outputs. Layout and usage: `README.md`.
## Contract (do not break)
- One detector per family, in `scripts/`: `visibility.py`, `obstruction.py`, `target_issue.py`, `thermal_displacement.py`, `no_data.py`, `displacement.py` (D-119). `categories.py` loads every `scripts/*.py` that has `detect`: never put another script with a `detect` function, or with side effects at import, directly in `scripts/`; tools go in `bench/`.
- `detect(measurements)` returns episodes with the keys `start`, `end`, `kind` (the family), `label` (the variety), `text`, `ignore`, `sensors`. A detector may also declare `temperature`, `camera`, `metrics` (`categories.CONTEXT`) or `end`; `categories.events` passes them by introspection.
- Label files (JSONL): `id`, `start`, `end`, `family`, `description` (and `review`, `analysis`); the project uses `labeling/auto` only.
- Two repositories use this one through a `detectors -> ~/detectors` link (one copy on the workstation: a commit here is live for both at once); a rename here needs the matching change there:
- Diagnostic-app: `categories.events`, `categories.CATEGORIES` (`process`: `images`, `none`, `check`), the `--from/--to` CLIs, `bench/score.py` (the command shown in the page);
- Optimizer (`improve` skill and its agents): `scripts/categories.py --from/--to`, `bench/` (`score.py`, `regression.py`, `sensors.py`); its upgrader changes `scripts/*.py` and `thresholds.md`, one commit per fix.
- Two repositories use this one through a `detectors -> ~/aimsight-detectors` link (one copy on the workstation: a commit here is live for both at once); a rename here needs the matching change there:
- aimsight-app: `categories.events`, `categories.CATEGORIES` (`process`: `images`, `none`, `check`), the `--from/--to` CLIs, `bench/score.py` (the command shown in the page);
- Optimizer (`optimize` skill and its agents): `scripts/categories.py --from/--to`, `bench/` (`score.py`, `regression.py`, `sensors.py`); its upgrader changes `scripts/*.py` and `thresholds.md`, one commit per fix.
## Rules
- A constant added or changed updates `thresholds.md` in the same change; check with `grep -nE "^[A-Z_0-9]+ = " scripts/*.py`.
- A change to improve the detectors touches `scripts/*.py` and `thresholds.md` only; never `bench/`, which is the measure. Anything is possible there: add, remove or change a constant, a rule or a whole detector; a fix that needs a new family is out of reach (one detector per family, above).
- **Everything is detected**: what is shown or not to a client is not the detectors' business. A setting chosen not to show something may be changed, saying so.
- A constant added or changed updates its line of `thresholds.md` ("Tuned on": the run that changed it) and the detector's docstring in the same change; a constant or detector removed removes its line; check with `grep -nE "^[A-Z_0-9]+ = " scripts/*.py`.
- A detector that no longer loads silently disappears: a `skipped` line in the output of `categories.py` or of the benchmark means the change is broken.
- No rule specific to a station, a sensor or a date.
- A change to a detector is checked with the benchmark: `python3 bench/regression.py --labels <labels folder> --data <data folder>` (Diagnostic-app and Optimizer: `labeling/auto`, `data`, links to the shared `~/labeling` and `~/aimsight-data/data`; about 3 min). A refactoring must leave every figure unchanged.
- History references in docstrings and `thresholds.md` ("wf-improve X, mod. 1, S2", "X, batch 01", D-xxx, E-xxx) point to past runs and to decisions recorded in Diagnostic-app/Support-Agent; keep them when editing a rule.
- The bench tools take their paths as arguments; the architecture of the repositories: `~/Diagnostic-App/docs/architecture.drawio`.
- A change to a detector is checked with the benchmark: `python3 bench/regression.py --labels <labels folder> --data <data folder>` (aimsight-app and Optimizer: `labeling/auto`, `data`, links to the shared `~/aimsight-labeling` and `~/aimsight-data/data`; about 3 min). A refactoring must leave every figure unchanged.
- **The benchmark's scores**, per family of each station: "found", the share of the labelled instants the detectors flag; "correct", the share of the flagged instants a label puts in the family. **A regression**, on a family of a station: "found" or "correct" loses more than 2 points; it is a regression even if the `Total` rises; under 2 points it is not one; a rise is a plus, never a requirement. **Not a regression**: a drop that comes from another consistent reading of the detectors that the labels do not retain: labels contradicting each other from one station to another (the same state in two families), the gaps of a few measurements between two labels of the same phenomenon, the bounds of a label, an instant the score (per instant, without the sensors) counted as found by an event on another sensor. Labels have their ambiguities: the goal is detectors that do their best, not perfect, and not overtrained on the labels.
- History references in docstrings and `thresholds.md` ("wf-improve X, mod. 1, S2", "X, batch 01", D-xxx, E-xxx) point to past runs and to decisions recorded in aimsight-app/Support-Agent; keep them when editing a rule.
- The bench tools take their paths as arguments; the architecture of the repositories: `~/aimsight-app/docs/architecture.drawio`.
+2 -2
View File
@@ -38,6 +38,6 @@ A change to a detector is checked with the benchmark; a refactoring leaves every
## Improving them
The detectors are improved by Optimizer (its `improve` skill), with the `upgrader` and `validator` agents of the agents repository: on the range of an analysis, its labels as the truth, the upgrader fixes the detectors (one commit per fix here) and the validator keeps, removes or cancels the fixes against overfitting, judged by this benchmark. Nothing is pushed.
The detectors are improved by Optimizer (its `optimize` skill, from one analysis), with the `upgrader` and `validator` agents of the agents repository: on the range of an analysis, its labels as the truth, the upgrader fixes the detectors (one commit per fix here) and the validator keeps, removes or cancels the fixes against overfitting, judged by this benchmark. Nothing is pushed.
Copied from Support-Agent on 2026-11-13 (commit 48a3092); the detectors now evolve here. Used by Diagnostic-app and Optimizer, through a `detectors -> ~/detectors` link (one copy on the workstation); their labels come from the `labeling` repository, their data from `aimsight-data`. The architecture of the repositories: `~/Diagnostic-App/docs/architecture.drawio`.
Copied from Support-Agent on 2026-11-13 (commit 48a3092); the detectors now evolve here. Used by aimsight-app and Optimizer, through a `detectors -> ~/aimsight-detectors` link (one copy on the workstation); their labels come from the `labeling` repository, their data from `aimsight-data`. The architecture of the repositories: `~/aimsight-app/docs/architecture.drawio`.
+1 -1
View File
@@ -1,6 +1,6 @@
"""Reading a station folder (standard library only). Every time is UTC.
Column formats: knowledge/displacement-columns.md in the labeling repository.
Column formats: DATA.md in the tools repository.
"""
import csv
import glob