Review fixes; README and CLAUDE.md finalised

- cancel reverts only the run's commits (base..HEAD), never the caller's edits
- validator diffs base..HEAD, not the working tree
- fix commits carry their Co-Authored-By line; no redundant end benchmark
- score.py: clear error on a missing labels file; docstring says end is not passed
- usage lines; README and CLAUDE.md for the generic workflow

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
maximeandClaude Opus 5.5 committed 2026-11-13 12:45:14 +01:00
1 parent 0de006bcb2
commit 388619685a
10 files changed
+63 -30

No files matched your search

+9 -5
View File
@@ -4,14 +4,18 @@ AIMsight anomaly detectors (Python 3, standard library only) and the Claude Code
## Contract (do not break)
- One detector per family of `labeling/familles.md` (D-119), in `scripts/`: `visibility.py`, `obstruction.py`, `target_issue.py`, `thermal_displacement.py`, `no_data.py`, `displacement.py`. `categories.py` loads every `scripts/*.py` that has `detect`: never put another script with a `detect` function, or one with side effects at import, directly in `scripts/` (tools go in `skills/wf-improve/scripts/`).
- One detector per family, in `scripts/`: `visibility.py`, `obstruction.py`, `target_issue.py`, `thermal_displacement.py`, `no_data.py`, `displacement.py` (D-119). `categories.py` loads every `scripts/*.py` that has `detect`: never put another script with a `detect` function, or with side effects at import, directly in `scripts/`; tools go in `skills/wf-improve/scripts/`.
- `detect(measurements)` returns episodes with the keys `start`, `end`, `kind` (the family), `label` (the variety), `text`, `ignore`, `sensors`. A detector may also declare `temperature`, `camera`, `metrics` (`categories.CONTEXT`) or `end`; `categories.events` passes them by introspection.
- Diagnostic-app reads this repository as it is: `categories.events`, `categories.CATEGORIES` (`process`: `images`, `none`, `check`), the `--from/--to` CLIs, `/detectors:wf-improve`, `wf-improve/<session>/summary.md`. A rename here needs the matching change there.
- Label fields come from Diagnostic-app and stay French: `debut`, `fin`, `categorie`, `description`; likewise `demande.json` (`STATION`, `début`, `fin`).
- Label files are external data and keep their French keys: `debut`, `fin`, `categorie`, `description`.
- Diagnostic-app uses this repository as its `detectors/` submodule: `categories.events`, `categories.CATEGORIES` (`process`: `images`, `none`, `check`), the `--from/--to` CLIs, `/detectors:wf-improve`. A rename here needs the matching change there.
## The workflow stays generic
`skills/wf-improve/SKILL.md` and `agents/` describe only the workflow (range + labels → upgrader → validator → summary). Everything specific to a caller (its paths, its files, its git rules, where the summary goes) is given in the request at launch, never written here. The tools take their paths as arguments.
## Rules
- A constant added or changed updates `thresholds.md` in the same change; check with `grep -nE "^[A-Z_0-9]+ = " scripts/*.py`.
- No rule specific to a station, a sensor or a date.
- A change to a detector is checked with the benchmark, from a working repository where this repository is mounted (or copied, not symlinked) as `detectors/` next to `data/` and `labeling/`: `python3 detectors/skills/wf-improve/scripts/regression.py`. A refactoring must leave every figure unchanged.
- History references in docstrings ("wf-improve X, mod. 1, S2", "X, batch 01", D-xxx, E-xxx) point to past sessions and to decisions recorded in Diagnostic-app/Support-Agent; keep them when editing a rule.
- A change to a detector is checked with the benchmark: `python3 skills/wf-improve/scripts/regression.py --labels <labels folder> --data <data folder>` (Diagnostic-app: `labeling/auto`, `data`; about 3 min). A refactoring must leave every figure unchanged.
- History references in docstrings and `thresholds.md` ("wf-improve X, mod. 1, S2", "X, batch 01", D-xxx, E-xxx) point to past runs and to decisions recorded in Diagnostic-app/Support-Agent; keep them when editing a rule.
+41 -15
View File
@@ -1,31 +1,57 @@
# detectors
The AIMsight anomaly detectors, and their improvement. One detector per anomaly family of the displacements, classified into events by `scripts/categories.py`; `scripts/data.py` reads a station folder. Standard library only (Python 3).
The AIMsight anomaly detectors, and the workflow that improves them. Python 3, standard library only.
## The detectors
One detector per anomaly family of the displacements, in `scripts/`:
| Family | Detector |
|---|---|
| Visibility | `visibility.py` |
| Obstruction | `obstruction.py` |
| Target issue | `target_issue.py` |
| Thermal displacement | `thermal_displacement.py` |
| No data | `no_data.py` |
| Displacement | `displacement.py` |
`categories.py` runs them all and classifies their episodes into events, one line per event; `data.py` reads a station folder (`displacement/`, `metrics/`, `camera/`).
```bash
python3 scripts/categories.py <station folder> --from YYYY-MM-DD --to YYYY-MM-DD
python3 scripts/categories.py <station folder> --from YYYY-MM-DD --to YYYY-MM-DD # --chunk N, --history N
python3 scripts/visibility.py <station folder> # one detector alone
```
The station folder: `displacement/`, `metrics/`, `camera/` (the format of `data/<STATION>/` in Diagnostic-app).
`thresholds.md` is the registry of the detectors' thresholds and what each was tuned on.
- `scripts/`: the detectors (`visibility.py`, `obstruction.py`, `target_issue.py`, `thermal_displacement.py`, `no_data.py`, `displacement.py`), `categories.py`, `data.py`.
- `thresholds.md`: the registry of the detectors' thresholds.
## Improving them: `/detectors:wf-improve`
## The wf-improve workflow: a Claude Code plugin
The repository is also a Claude Code plugin (`.claude-plugin/plugin.json`). Its workflow improves the detectors on a range of data where they perform poorly, given labels as the truth on that range:
The repository is also a Claude Code plugin (`.claude-plugin/plugin.json`), to be mounted as `detectors/` in a working repository that has `labeling/` (the labels, `labeling/auto/<STATION>.jsonl`; the analyses, `labeling/auto/analyses/`), `data/` and `agent-tools/`: Diagnostic-app, where it is a submodule.
1. measures before: the score on the range, and a benchmark on a reference truth (every labelled station);
2. the `upgrader` agent compares the detectors' events with the labels and fixes the detectors with general rules, one commit per fix;
3. the `validator` agent, an LLM-as-judge that does not read the upgrader's diagnosis, keeps the fixes, removes some, or cancels them: no station may go back, no overfitting;
4. a summary.
- `skills/wf-improve/`: `/detectors:wf-improve <analysis>` (`SKILL.md`), the session that improves the detectors from an analysis of the Data page, without overfitting; its tools in `scripts/`, run from the working repository: `regression.py` (the benchmark, all labels of `labeling/auto/`), `score.py` (the score of a range), `sensors.py` (the sensors of each event).
- `agents/`: `upgrader` (diagnoses and fixes the detectors) and `validator` (LLM-as-judge: keeps, removes or cancels the fixes), called `detectors:<name>`.
- `wf-improve/`: the sessions, one folder `<session>_<analysis>/` each, written by the skill.
The result: detectors a little better on the range without going back elsewhere, or unchanged. Nothing is pushed.
From the working repository:
The caller gives the context of each run: the range, the labels, the reference truth, the families document, any context documents, the output folder, and its own rules (git branch, what else to commit). For example:
```bash
claude --plugin-dir detectors # then /detectors:wf-improve <analysis>
python3 detectors/skills/wf-improve/scripts/regression.py # the benchmark
claude --plugin-dir detectors -p "/detectors:wf-improve DETECTORS: detectors/. STATION: data/NORDEND. \
FROM: 2026-11-04. TO: 2026-11-08. LABELS: labeling/auto/NORDEND.jsonl. \
REFERENCE: --labels labeling/auto --data data. FAMILIES: labeling/familles.md. \
CONTEXT: labeling/auto/analyses/NORDEND_x/rapport.md. OUT: detectors/runs/NORDEND_x/."
```
The session commits the detectors here, on `main`, without pushing: push `detectors` so that everyone has the updated detectors.
Labels: JSONL, one per line, `{"id", "debut", "fin", "categorie", "description"}` (UTC ISO times, `categorie` = the family).
Copied from Support-Agent on 2026-11-13 (commit 48a3092); the detectors now evolve here.
- `skills/wf-improve/SKILL.md`: the workflow; its tools in `skills/wf-improve/scripts/`: `score.py` (the score of a range), `regression.py` (the benchmark), `sensors.py` (the sensors of each event).
- `agents/`: `upgrader`, `validator` (called `detectors:<name>`).
```bash
python3 skills/wf-improve/scripts/score.py <station folder> --labels <labels.jsonl> --from YYYY-MM-DD --to YYYY-MM-DD
python3 skills/wf-improve/scripts/regression.py --labels <folder of <STATION>.jsonl> --data <folder of <STATION>/>
```
Copied from Support-Agent on 2026-11-13 (commit 48a3092); the detectors now evolve here. Used by Diagnostic-app, where it is the `detectors/` submodule.
+1 -1
View File
@@ -66,7 +66,7 @@ Your working folder is yours alone: `mktemp -d`. To try a variant without touchi
The validator can have a single fix removed without touching the others: each has its commit. For each fix, in order:
1. change, then check as above, against the benchmark of the last committed state (the benchmark before, or that of your previous kept fix); before your output, the whole against the benchmark before; keep the output of the benchmark in `<output folder>/benchmark-S<n>.txt`;
2. kept: `git -C <DETECTORS> add <the changed files> && git -C <DETECTORS> commit -m "wf-improve: S<n>" -- <the same>`, the message ending with an empty line then `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`;
2. kept: `git -C <DETECTORS> add <the changed files> && git -C <DETECTORS> commit -m "wf-improve: S<n>" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" -- <the same>`;
3. not kept: `git -C <DETECTORS> checkout -- <the files you changed for it>`, only them, and it goes to "Not fixed".
Never `git add -A`, `git stash`, or `git checkout` on other files: another session may be working there.
+1 -1
View File
@@ -17,7 +17,7 @@ The detectors repository (`DETECTORS`: the detectors `scripts/*.py`, `thresholds
- the range score before → after: the gain the modification sought;
- in `modification.md`, the parts "What changed" and "Why", nothing else: `sed -n '/^## What changed/,/^## Before/p' <modification>`;
- the diff of the detectors since the base: `git -C <DETECTORS> diff <base> -- 'scripts/*.py' thresholds.md`, and the code it touches; each fix has its commit (`git -C <DETECTORS> log --oneline <base>..HEAD`, message `wf-improve: S<n>`), its diff by `git -C <DETECTORS> show <commit>`;
- the diff of the detectors since the base: `git -C <DETECTORS> diff <base> HEAD -- 'scripts/*.py' thresholds.md`, and the code it touches; each fix has its commit (`git -C <DETECTORS> log --oneline <base>..HEAD`, message `wf-improve: S<n>`), its diff by `git -C <DETECTORS> show <commit>`;
- the benchmarks (the score per family on each station of the reference truth, then their `Total`), all stations in full; the benchmarks after each fix, in order, say which one makes a station go back; you may rerun the benchmark command;
- to understand a regression on a station: the events command on its range (from the header of the station's block in the benchmark), with `2>&1`, at HEAD and, to compare, at the base (`git -C <DETECTORS> archive <base> | tar -x -C <folder>` in your working folder, then the same command with the copy's `scripts/categories.py`), the station's labels, and the families document.
+2 -2
View File
@@ -23,7 +23,7 @@ episodes; their label gives the variety (Poor visibility, Heat shimmer, Brief ob
Each event gets its sentence, in English (the site): what happened and what to do about it, as the "For the client"
column of the families document says; the cause is only given if the variety establishes it, otherwise the images show it.
No detect function: the loader does not take this file for a detector.
Usage: python3 detectors/scripts/categories.py <station folder> (one line per event: family, variety, sensors)
Usage: python3 scripts/categories.py <station folder> (one line per event: family, variety, sensors)
... <folder> [--from YYYY-MM-DD] [--to YYYY-MM-DD] [--chunk 3] [--history N] (batch run: each chunk of N days
classified alone, with the days of history before it if they exist; only the events touching the chunk are
shown; "nothing" for a quiet chunk)
@@ -50,7 +50,7 @@ def _context(folder, key):
return CONTEXT[key](folder)
# The families of the families document, in the order of the timeline rows; ignore: greyed zone (None: as its episodes
# The families, in the order of the timeline rows; ignore: greyed zone (None: as its episodes
# say: a total loss of visibility, strong heat shimmer); process: Images button, nothing, or Images and a link to
# Diagnosis.
CATEGORIES = {
+1 -1
View File
@@ -7,7 +7,7 @@ End of the window read (end, when the caller gives it: the chunked run of catego
measurement and end, the episode running up to end (CLARIDEN, batch 01: 44 h without a measurement up to the end of
the batch, not reported). No measurement in the export does not mean the station stopped: the text only says that the
export has nothing.
Usage: python3 detectors/no_data.py <station folder> (prints the episodes)
Usage: python3 scripts/no_data.py <station folder> (prints the episodes)
"""
import os
import statistics
+1 -1
View File
@@ -7,7 +7,7 @@ A sensor already in an overlapping Target issue episode (Degraded target, Target
its curve is already to be ignored, the family is Target issue (boundary "Target issue or an episode of another family"
of the families document; wf-improve session Obstruction, mod. 1: S1, problem targets of FLORIDA, NORDEND, SNCF01,
NIEDERHORN, JUNGFRAU, CLARIDEN cut into Target obscured; S3, their isolated rejections as Brief obstruction).
Usage: python3 detectors/obstruction.py <station folder> (prints the episodes)
Usage: python3 scripts/obstruction.py <station folder> (prints the episodes)
"""
import os
import sys
+1 -1
View File
@@ -34,7 +34,7 @@ Stuck target (E-011, FLORIDA: a target defined on the prism of a reference): per
corr_y (median of the gaps from one measurement to the next) under STUCK times the median noise of the other targets, on
both axes; one day is enough (a day without noise does not happen by accident); label "Target stuck". On E-011: C2 at
7e-5 px and C3 at 2e-4 against 0.013 for the others.
Usage: python3 detectors/target_issue.py <station folder> (prints the episodes)
Usage: python3 scripts/target_issue.py <station folder> (prints the episodes)
"""
import os
import statistics
+2 -2
View File
@@ -48,13 +48,13 @@ Each agent is called with the Agent tool, its `subagent_type` (`detectors:upgrad
4. **Validator**, which reads neither the diagnosis nor the detail of the modification: `Detectors: <DETECTORS>. Families: <FAMILIES>. Modification: <OUT>/modification.md. Range score: <OUT>/range-before.txt → <OUT>/range-after.txt. Benchmark: <OUT>/benchmark-before.txt → <OUT>/benchmark-after.txt, after each fix: <OUT>/benchmark-S*.txt; command: <…>. Events command for any station: python3 <DETECTORS>/scripts/categories.py <station folder> --from <from> --to <to>. Base: <base>. Write <OUT>/validation.md.`
- `keep`: go to the summary.
- `remove`: for each fix of the `**Remove**:` line, its commits (`git -C <DETECTORS> log --format=%h --grep ': S<n>$' <base>..HEAD`), from the most recent: `git -C <DETECTORS> revert --no-edit <commit>`, then `git -C <DETECTORS> commit --amend -m "wf-improve: S<n> removed" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"`. A conflict: `git -C <DETECTORS> revert --abort`, and do as for `cancel`.
- `cancel`, or anything else: `git -C <DETECTORS> checkout <base> -- 'scripts/*.py' thresholds.md` (without the files already modified), commit of these paths, message `wf-improve: modification cancelled`.
- `cancel`, or anything else: `git -C <DETECTORS> revert --no-commit <base>..HEAD && git -C <DETECTORS> commit -m "wf-improve: modification cancelled" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"` (only what the fix commits changed; files already modified stay as they are).
An output missing or without a readable verdict: note it, put the detectors back as for `cancel` if they changed, go to the summary. Do not rerun an agent.
## Summary
Measures, step `end` (unless nothing changed: then `end` = `before`). Then `<OUT>/summary.md`:
Measures, step `end`, only after a `remove`; otherwise `end` is `after` (`keep`) or `before` (`cancel`, nothing changed). Then `<OUT>/summary.md`:
```markdown
# wf-improve: <station>, <FROM> → <TO>
+4 -1
View File
@@ -25,7 +25,8 @@ HISTORY = timedelta(days=3) # as the batch command of categories.py
def events(measurements, start, end, folder=None):
"""Detectors' events touching the range, classified with its days of history (as the batch command)."""
"""Detectors' events touching the range, classified with its days of history. Unlike the batch command, end is not
passed: no_data does not see a hole touching the end of the range."""
h, b = parse_time(start) - HISTORY, parse_time(end)
chunk = {s: k for s, ms in measurements.items() if (k := [m for m in ms if h <= m["t"] < b])}
return [v for v in (categories.events(chunk, folder=folder) if chunk else []) if v["end"] >= start]
@@ -88,6 +89,8 @@ if __name__ == "__main__":
p.add_argument("--from", dest="start", required=True)
p.add_argument("--to", dest="end", required=True)
a = p.parse_args()
if not os.path.exists(a.labels):
sys.exit(f"No labels file: {a.labels}")
station, start, end = os.path.basename(os.path.normpath(a.folder)), iso(parse_time(a.start)), iso(parse_time(a.end))
n, labels = family_scores(a.folder, a.labels, start, end)
if not labels: