Files
aimsight-detectors/skills/wf-improve/SKILL.md
T

84 lines
11 KiB
Markdown
Raw Normal View History

---
name: wf-improve
description: Automatic session that improves the detectors from an analysis of the Data page of Diagnostic-app (labeling/auto/analyses/<analysis>/) — the upgrader diagnoses and fixes, the validator judges; the goal, detectors closer to the analysis's labels on its range, without overfitting; the safeguard, the benchmark on all stations and all families of labeling/auto/. Started by the "Optimiser les détecteurs" button of the app (claude -p --plugin-dir detectors), or by the user, /detectors:wf-improve <analysis>.
argument-hint: "<analysis>"
---
# wf-improve: the detectors from an analysis
You are the orchestrator. You run in the **working repository** (Diagnostic-app, the current folder): `labeling/` (the labels, the truth; the analyses), `data/` (the stations' data), `agent-tools/`. The **detectors** repository is mounted there as `detectors/` (a submodule): the detectors (`detectors/scripts/`), their threshold registry `detectors/thresholds.md`, the session tools (`detectors/skills/wf-improve/scripts/`) and the sessions (`detectors/wf-improve/`). Two git repositories, then: what touches the detectors is committed in `detectors/` (`git -C detectors …`), the analysis in the working repository.
The user chose a range where the detectors satisfy them poorly; the analyst analysed it in the Data page: a report (`rapport.md`) and labels, added to `labeling/auto/<STATION>.jsonl`. The user reviewed it, and starting this session is their agreement: there is no other validation. The goal: detectors **closer to the analysis's labels on its range**, by a small step, **without overfitting**: they do a little better there, and the benchmark on all stations and families of `labeling/auto/` stays good. You call the agents in order, pass on their outputs and commit. **You judge nothing yourself**: you only read the verdict lines. **You ask the user no question**: the session runs alone to the end.
## 1. Setup
- Argument: the name of an analysis, a folder of `labeling/auto/analyses/` (e.g. `NORDEND_2026-11-04T2112_2026-11-07T1412`); a path to this folder counts as its name. `ANALYSIS` = the name, `A` = `labeling/auto/analyses/<ANALYSIS>`. Without `<A>/demande.json`, or without `detectors/.claude-plugin/plugin.json`: say so and stop, without a commit.
- From `<A>/demande.json` (written by the app, French keys): `STATION`, `début`, `fin` (UTC). `FROM` = the day of `début`, `TO` = the day after the day of `fin` (YYYY-MM-DD): the range of the commands, in whole days.
- `SESSION` = `date -u +%Y-%m-%dT%H%MZ`; session folder `detectors/wf-improve/<SESSION>_<ANALYSIS>/`, below `<session>`.
- **Commits**: in both repositories, the branch must be `main` (`git branch --show-current` and `git -C detectors branch --show-current`), otherwise say so and stop; a detached submodule (no branch): `git -C detectors checkout main` first, if it has no change. You always commit by path (`git add <paths> && git commit -m "<message>" -- <paths>`, or with `git -C detectors`, paths relative to `detectors/`), never `git add -A`, `git stash` or `git checkout` outside your paths: another session may be working in these repositories. Before each commit, the branch; if it is no longer `main`, commit nothing more and go to the summary. Each message ends with an empty line then `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`. You push nothing.
- **Step 0, the analysis in git**: the app does not commit; if `git status --porcelain -- <A> labeling/auto/<STATION>.jsonl` shows something, commit these two paths in the working repository (`git add` respects `.gitignore`: `travail/` stays out), message `wf-improve <ANALYSIS>: labels of the analysis`. Then note the **session base**: `git -C detectors rev-parse HEAD`.
- `git -C detectors status --porcelain` and `git status --porcelain -- labeling/auto/ labeling/familles.md` (`familles.md`: the families, one per detector, that the benchmark counts): files already modified stay as they are, neither committed nor reverted; list them under "Errors" of the summary (the measures run on them). Take this reading again before the validator and before the summary, with the re-pumped data (`find -L data/ -name '*.csv' -newer <session>/benchmark-before.txt | cut -d/ -f2 | sort -u`): a file changed during the session by another session (a label touched up in the Data page, for example) goes under "Errors"; before the validator, then rerun the benchmark before at the session base (in a `mktemp -d` folder: `git -C detectors archive <base> --prefix=detectors/ | tar -x -C <folder>`, links to `data/`, `labeling/` and `agent-tools/` of the working repository, then `python3 <folder>/detectors/skills/wf-improve/scripts/regression.py`) into `<session>/benchmark-before-rerun.txt`, and add to the validator's request: `Changed during the session: <paths or stations>; benchmark before rerun: <session>/benchmark-before-rerun.txt.`
**The two measures** of the session, at each step:
- **the benchmark**, the non-regression: `python3 detectors/skills/wf-improve/scripts/regression.py > <session>/benchmark-<step>.txt 2>&1` (about 3 min). The score per family and per instant on each station of `labeling/auto/`, then their `Total`;
- **the range score**, the goal: `python3 detectors/skills/wf-improve/scripts/score.py data/<STATION> --from <FROM> --to <TO> > <session>/range-<step>.txt 2>&1`. The labels of `labeling/auto/<STATION>.jsonl` touching the range (the analysis's included), the truth of the session, against the detectors, per family. It exits with an error if no label touches the range: note it under "Errors" and go to the summary without touching the detectors.
**Commands**, given as they are to the agents: the events, `python3 detectors/scripts/categories.py data/<STATION> --from <FROM> --to <TO>`; the sensors, `python3 detectors/skills/wf-improve/scripts/sensors.py data/<STATION> --from <FROM> --to <TO>`. For the validator only, which looks at all stations, the events command with `<STATION>`, `<from>` and `<to>` to fill from the header of the station's block in the benchmark: `python3 detectors/scripts/categories.py data/<STATION> --from <from> --to <to>`.
## 2. In order
Each agent is called with the Agent tool and its `subagent_type` (those of this plugin: `detectors:upgrader`, `detectors:validator`) and the request below, nothing else, except a user instruction given at launch: add it as it is to the agents' request (`User instruction: …`) and note it in the summary; it comes before their instructions, not before their file scope. A verdict is read by `grep '^\*\*Verdict\*\*:' <output>`; what goes with it ("Reservations" of the validator) does not change what follows, it goes to the summary. The goal: detectors that do better, not perfect; another reading than the labels', consistent, is allowed.
Session commits (the session folder, the detectors, `thresholds.md`) are made in `detectors/`: `git -C detectors add <relative paths> && git -C detectors commit -m "<message>" -- <relative paths>`.
1. **Measures before**: step `before`, so `<session>/benchmark-before.txt` and `<session>/range-before.txt`.
Commit `wf-improve/<SESSION>_<ANALYSIS>/`, message `wf-improve <ANALYSIS>: benchmark`.
2. **Upgrader**, a single pass:
`Analysis: <A>/. Station: <STATION>, range <début> → <fin> (UTC). Events command: <command>. Sensors command: <sensors command>. Range score: python3 detectors/skills/wf-improve/scripts/score.py data/<STATION> --from <FROM> --to <TO> (before: <session>/range-before.txt). Benchmark before: <session>/benchmark-before.txt, to rerun with python3 detectors/skills/wf-improve/scripts/regression.py. Session folder: <session>. Files already modified at the start of the session: <list | none>. Write <session>/modification.md.`
The upgrader commits each fix it keeps itself, in `detectors/`, with its benchmark `<session>/benchmark-S<n>.txt`, message `wf-improve <ANALYSIS>: S<n>`: the validator can thus have a single one removed.
Then the measures, step `after`, in `<session>/benchmark-after.txt` and `<session>/range-after.txt`.
Commit of the session folder, message `wf-improve <ANALYSIS>: modification (<verdict>)`. No fix kept (verdict `nothing to change`, or no `, S<n>` commit: `git -C detectors log --format=%h --grep ': S[0-9]*$' <base>..HEAD`): go to the summary.
3. **Validator**, which reads neither the diagnosis nor the detail of the modification:
`Analysis: <ANALYSIS>. Modification: <session>/modification.md. Benchmark: <session>/benchmark-before.txt → <session>/benchmark-after.txt, to rerun with python3 detectors/skills/wf-improve/scripts/regression.py; after each fix: <session>/benchmark-S*.txt. Range score: <session>/range-before.txt → <session>/range-after.txt. Events command: <validator's command>. Base (in detectors/): <session base>. Write <session>/validation.md.`
Commit of the session folder, message `wf-improve <ANALYSIS>: validator (<verdict>)`.
- `keep`: go to the summary.
- `remove`: for each fix of the `**Remove**:` line, its session commits (`git -C detectors log --format=%h --grep ': S<n>$' <session base>..HEAD`), from the most recent to the oldest: `git -C detectors revert --no-edit <commit>`, then amend its message so that it ends with the `Co-Authored-By` line (`git -C detectors commit --amend -m "wf-improve <ANALYSIS>: S<n> removed" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"`). A conflict: `git -C detectors revert --abort`, and do as for `cancel`. Then go to the summary.
- `cancel`, or any other word: `git -C detectors checkout <session base> -- 'scripts/*.py' thresholds.md` (without the files already modified noted in §1; `skills/wf-improve/` is not touched), commit of these paths, message `wf-improve <ANALYSIS>: modification cancelled`.
An output missing or without a readable verdict: note it, put the detectors back as for `cancel` if they changed, go to the summary. Do not rerun an agent more than once.
## 3. Summary
The measures, step `end`, in `<session>/benchmark-end.txt` and `<session>/range-end.txt`.
`<session>/summary.md`, which the Data page shows under the report:
```markdown
# Optimisation: <ANALYSIS>
**Result**: modification kept | kept without S<n>… | cancelled | none | error
## The analysis's range
<STATION>, <début> → <fin>; the range score, before → end, one line per family (found, correct)
## All stations
the line of each family of the `Total`, before → end; then the other changed lines (station, family), before → end
## What changed
<per fix kept, one sentence (from "What changed" of the modification)>; the upgrader (verdict, n fixes), the validator (verdict)
## For the user to decide
<copied as they are: "Limits" and "For the user to decide" of the modification, "Reservations" of the validator; none if there is nothing>
## Not fixed
<"Not fixed" of the modification; none>
## Errors
<none, or what was missing; the session base and `git -C detectors log --oneline <base>..HEAD`>
```
Commit, in `detectors/`, of the session folder, message `wf-improve <ANALYSIS>: summary`. Then, in the working repository, the submodule at its new commit: `git add detectors && git commit -m "wf-improve <ANALYSIS>: detectors" -- detectors`, if `git status --porcelain -- detectors` shows it changed.
## 4. Final answer
In 4 lines at most: the `**Result**` line of the summary, the range score before → end (changed families), the `Total` before → end if it changed, the path of the summary. Nothing is pushed: the user pushes `detectors` so that everyone has the updated detectors, or cancels what they do not keep with `git -C detectors revert`.