From c4fca737dc08ad1f7f1d94817cca3b311884306b Mon Sep 17 00:00:00 2001 From: maxime Date: Mon, 16 Nov 2026 14:41:17 +0100 Subject: [PATCH] CLAUDE.md: the rules the upgrader and validator read (what changes, regression, everything is detected) Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/CLAUDE.md b/CLAUDE.md index 15c4e10..0cc00bb 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -13,8 +13,12 @@ AIMsight anomaly detectors (Python 3, standard library only) and their benchmark ## Rules -- A constant added or changed updates `thresholds.md` in the same change; check with `grep -nE "^[A-Z_0-9]+ = " scripts/*.py`. +- A change to improve the detectors touches `scripts/*.py` and `thresholds.md` only; never `bench/`, which is the measure. Anything is possible there: add, remove or change a constant, a rule or a whole detector; a fix that needs a new family is out of reach (one detector per family, above). +- **Everything is detected**: what is shown or not to a client is not the detectors' business. A setting chosen not to show something may be changed, saying so. +- A constant added or changed updates its line of `thresholds.md` and the detector's docstring in the same change; a constant or detector removed removes its line; check with `grep -nE "^[A-Z_0-9]+ = " scripts/*.py`. +- A detector that no longer loads silently disappears: a `skipped` line in the output of `categories.py` or of the benchmark means the change is broken. - No rule specific to a station, a sensor or a date. - A change to a detector is checked with the benchmark: `python3 bench/regression.py --labels --data ` (Diagnostic-app and Optimizer: `labeling/auto`, `data`, links to the shared `~/labeling` and `~/aimsight-data/data`; about 3 min). A refactoring must leave every figure unchanged. +- **The benchmark's scores**, per family of each station: "found", the share of the labelled instants the detectors flag; "correct", the share of the flagged instants a label puts in the family. **A regression**, on a family of a station: "found" or "correct" loses more than 2 points; it is a regression even if the `Total` rises; under 2 points it is not one; a rise is a plus, never a requirement. **Not a regression**: a drop that comes from another consistent reading of the detectors that the labels do not retain: labels contradicting each other from one station to another (the same state in two families), the gaps of a few measurements between two labels of the same phenomenon, the bounds of a label, an instant the score (per instant, without the sensors) counted as found by an event on another sensor. Labels have their ambiguities: the goal is detectors that do their best, not perfect, and not overtrained on the labels. - History references in docstrings and `thresholds.md` ("wf-improve X, mod. 1, S2", "X, batch 01", D-xxx, E-xxx) point to past runs and to decisions recorded in Diagnostic-app/Support-Agent; keep them when editing a rule. - The bench tools take their paths as arguments; the architecture of the repositories: `~/Diagnostic-App/docs/architecture.drawio`.