- skill detector-learner -> learner; agents detectors-upgrader -> upgrader, detectors-control -> validator; detectors-critic merged into the upgrader - releveur and ameliorations-processus.md removed; history.csv removed - archive/visibilite_simple.py removed (kept in the history of Support-Agent) - seuils-du-journal.md -> thresholds.md, in English, constants renamed - "journal du matin" dropped; README, plugin.json in English; CLAUDE.md added Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
85 lines
11 KiB
Markdown
85 lines
11 KiB
Markdown
---
|
|
name: learner
|
|
description: Automatic session that improves the detectors from an analysis of the Data page of Diagnostic-app (labeling/auto/analyses/<analysis>/) — the upgrader diagnoses and fixes, the validator judges; the goal, detectors closer to the analysis's labels on its range, without overfitting; the safeguard, the benchmark on all stations and all families of labeling/auto/. Started by the "Optimiser les détecteurs" button of the app (claude -p --plugin-dir detectors), or by the user, /detectors:learner <analysis>.
|
|
argument-hint: "<analysis>"
|
|
---
|
|
|
|
# learner: the detectors from an analysis
|
|
|
|
You are the orchestrator. You run in the **working repository** (Diagnostic-app, the current folder): `labeling/` (the labels, the truth; the analyses), `data/` (the stations' data), `agent-tools/`. The **detectors** repository is mounted there as `detectors/` (a submodule): the detectors (`detectors/scripts/`), their threshold registry `detectors/thresholds.md`, the session tools (`detectors/scripts/learner/`) and the sessions (`detectors/learner/`). Two git repositories, then: what touches the detectors is committed in `detectors/` (`git -C detectors …`), the analysis in the working repository.
|
|
|
|
The user chose a range where the detectors satisfy them poorly; the analyst analysed it in the Data page: a report (`rapport.md`) and labels, added to `labeling/auto/<STATION>.jsonl`. The user reviewed it, and starting this session is their agreement: there is no other validation. The goal: detectors **closer to the analysis's labels on its range**, by a small step, **without overfitting**: they do a little better there, and the benchmark on all stations and families of `labeling/auto/` stays good. You call the agents in order, pass on their outputs and commit. **You judge nothing yourself**: you only read the verdict lines. **You ask the user no question**: the session runs alone to the end.
|
|
|
|
## 1. Setup
|
|
|
|
- Argument: the name of an analysis, a folder of `labeling/auto/analyses/` (e.g. `NORDEND_2026-11-04T2112_2026-11-07T1412`); a path to this folder counts as its name. `ANALYSIS` = the name, `A` = `labeling/auto/analyses/<ANALYSIS>`. Without `<A>/demande.json`, or without `detectors/.claude-plugin/plugin.json`: say so and stop, without a commit.
|
|
- From `<A>/demande.json` (written by the app, French keys): `STATION`, `début`, `fin` (UTC). `FROM` = the day of `début`, `TO` = the day after the day of `fin` (YYYY-MM-DD): the range of the commands, in whole days.
|
|
- `SESSION` = `date -u +%Y-%m-%dT%H%MZ`; session folder `detectors/learner/<SESSION>_<ANALYSIS>/`, below `<session>`.
|
|
- **Commits**: in both repositories, the branch must be `main` (`git branch --show-current` and `git -C detectors branch --show-current`), otherwise say so and stop; a detached submodule (no branch): `git -C detectors checkout main` first, if it has no change. You always commit by path (`git add <paths> && git commit -m "<message>" -- <paths>`, or with `git -C detectors`, paths relative to `detectors/`), never `git add -A`, `git stash` or `git checkout` outside your paths: another session may be working in these repositories. Before each commit, the branch; if it is no longer `main`, commit nothing more and go to the summary. Each message ends with an empty line then `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`. You push nothing.
|
|
- **Step 0, the analysis in git**: the app does not commit; if `git status --porcelain -- <A> labeling/auto/<STATION>.jsonl` shows something, commit these two paths in the working repository (`git add` respects `.gitignore`: `travail/` stays out), message `Learner <ANALYSIS>: labels of the analysis`. Then note the **session base**: `git -C detectors rev-parse HEAD`.
|
|
- `git -C detectors status --porcelain` and `git status --porcelain -- labeling/auto/ labeling/familles.md` (`familles.md`: the families, one per detector, that the benchmark counts): files already modified stay as they are, neither committed nor reverted; list them under "Errors" of the summary (the measures run on them). Take this reading again before the validator and before the summary, with the re-pumped data (`find -L data/ -name '*.csv' -newer <session>/benchmark-before.txt | cut -d/ -f2 | sort -u`): a file changed during the session by another session (a label touched up in the Data page, for example) goes under "Errors"; before the validator, then rerun the benchmark before at the session base (in a `mktemp -d` folder: `git -C detectors archive <base> --prefix=detectors/ | tar -x -C <folder>`, links to `data/`, `labeling/` and `agent-tools/` of the working repository, then `python3 <folder>/detectors/scripts/learner/regression.py`) into `<session>/benchmark-before-rerun.txt`, and add to the validator's request: `Changed during the session: <paths or stations>; benchmark before rerun: <session>/benchmark-before-rerun.txt.`
|
|
|
|
**The two measures** of the session, at each step:
|
|
- **the benchmark**, the non-regression: `python3 detectors/scripts/learner/regression.py > <session>/benchmark-<step>.txt 2>&1` (about 3 min). The score per family and per instant on each station of `labeling/auto/`, then their `Total`;
|
|
- **the range score**, the goal: `python3 detectors/scripts/learner/score.py data/<STATION> --from <FROM> --to <TO> > <session>/range-<step>.txt 2>&1`. The labels of `labeling/auto/<STATION>.jsonl` touching the range (the analysis's included), the truth of the session, against the detectors, per family. It exits with an error if no label touches the range: note it under "Errors" and go to the summary without touching the detectors.
|
|
|
|
**Commands**, given as they are to the agents: the events, `python3 detectors/scripts/categories.py data/<STATION> --from <FROM> --to <TO>`; the sensors, `python3 detectors/scripts/learner/sensors.py data/<STATION> --from <FROM> --to <TO>`. For the validator only, which looks at all stations, the events command with `<STATION>`, `<from>` and `<to>` to fill from the header of the station's block in the benchmark: `python3 detectors/scripts/categories.py data/<STATION> --from <from> --to <to>`.
|
|
|
|
## 2. In order
|
|
|
|
Each agent is called with the Agent tool and its `subagent_type` (those of this plugin: `detectors:upgrader`, `detectors:validator`) and the request below, nothing else, except a user instruction given at launch: add it as it is to the agents' request (`User instruction: …`) and note it in the summary; it comes before their instructions, not before their file scope. A verdict is read by `grep '^\*\*Verdict\*\*:' <output>`; what goes with it ("Reservations" of the validator) does not change what follows, it goes to the summary. The goal: detectors that do better, not perfect; another reading than the labels', consistent, is allowed.
|
|
|
|
Session commits (the session folder, the detectors, `thresholds.md`) are made in `detectors/`: `git -C detectors add <relative paths> && git -C detectors commit -m "<message>" -- <relative paths>`.
|
|
|
|
1. **Measures before**: step `before`, so `<session>/benchmark-before.txt` and `<session>/range-before.txt`.
|
|
Commit `learner/<SESSION>_<ANALYSIS>/`, message `Learner <ANALYSIS>: benchmark`.
|
|
2. **Upgrader**, a single pass:
|
|
`Analysis: <A>/. Station: <STATION>, range <début> → <fin> (UTC). Events command: <command>. Sensors command: <sensors command>. Range score: python3 detectors/scripts/learner/score.py data/<STATION> --from <FROM> --to <TO> (before: <session>/range-before.txt). Benchmark before: <session>/benchmark-before.txt, to rerun with python3 detectors/scripts/learner/regression.py. Session folder: <session>. Files already modified at the start of the session: <list | none>. Write <session>/modification.md.`
|
|
The upgrader commits each fix it keeps itself, in `detectors/`, with its benchmark `<session>/benchmark-S<n>.txt`, message `Learner <ANALYSIS>: S<n>`: the validator can thus have a single one removed.
|
|
Then the measures, step `after`, in `<session>/benchmark-after.txt` and `<session>/range-after.txt`.
|
|
Commit of the session folder, message `Learner <ANALYSIS>: modification (<verdict>)`. No fix kept (verdict `nothing to change`, or no `, S<n>` commit: `git -C detectors log --format=%h --grep ': S[0-9]*$' <base>..HEAD`): go to the summary.
|
|
3. **Validator**, which reads neither the diagnosis nor the detail of the modification:
|
|
`Analysis: <ANALYSIS>. Modification: <session>/modification.md. Benchmark: <session>/benchmark-before.txt → <session>/benchmark-after.txt, to rerun with python3 detectors/scripts/learner/regression.py; after each fix: <session>/benchmark-S*.txt. Range score: <session>/range-before.txt → <session>/range-after.txt. Events command: <validator's command>. Base (in detectors/): <session base>. Write <session>/validation.md.`
|
|
Commit of the session folder, message `Learner <ANALYSIS>: validator (<verdict>)`.
|
|
- `keep`: go to the summary.
|
|
- `remove`: for each fix of the `**Remove**:` line, its session commits (`git -C detectors log --format=%h --grep ': S<n>$' <session base>..HEAD`), from the most recent to the oldest: `git -C detectors revert --no-edit <commit>`, then amend its message so that it ends with the `Co-Authored-By` line (`git -C detectors commit --amend -m "Learner <ANALYSIS>: S<n> removed" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"`). A conflict: `git -C detectors revert --abort`, and do as for `cancel`. Then go to the summary.
|
|
- `cancel`, or any other word: `git -C detectors checkout <session base> -- 'scripts/*.py' thresholds.md` (without the files already modified noted in §1; `scripts/learner/` is not touched), commit of these paths, message `Learner <ANALYSIS>: modification cancelled`.
|
|
|
|
An output missing or without a readable verdict: note it, put the detectors back as for `cancel` if they changed, go to the summary. Do not rerun an agent more than once.
|
|
|
|
## 3. Summary
|
|
|
|
The measures, step `end`, in `<session>/benchmark-end.txt` and `<session>/range-end.txt`.
|
|
|
|
`<session>/summary.md`, which the Data page shows under the report:
|
|
|
|
```markdown
|
|
# Optimisation: <ANALYSIS>
|
|
|
|
**Result**: modification kept | kept without S<n>… | cancelled | none | error
|
|
|
|
## The analysis's range
|
|
<STATION>, <début> → <fin>; the range score, before → end, one line per family (found, correct)
|
|
|
|
## All stations
|
|
the line of each family of the `Total`, before → end; then the other changed lines (station, family), before → end
|
|
|
|
## What changed
|
|
<per fix kept, one sentence (from "What changed" of the modification)>; the upgrader (verdict, n fixes), the validator (verdict)
|
|
|
|
## For the user to decide
|
|
<copied as they are: "Limits" and "For the user to decide" of the modification, "Reservations" of the validator; none if there is nothing>
|
|
|
|
## Not fixed
|
|
<"Not fixed" of the modification; none>
|
|
|
|
## Errors
|
|
<none, or what was missing; the session base and `git -C detectors log --oneline <base>..HEAD`>
|
|
```
|
|
|
|
Commit, in `detectors/`, of the session folder, message `Learner <ANALYSIS>: summary`. Then, in the working repository, the submodule at its new commit: `git add detectors && git commit -m "Learner <ANALYSIS>: detectors" -- detectors`, if `git status --porcelain -- detectors` shows it changed.
|
|
|
|
## 4. Final answer
|
|
|
|
In 4 lines at most: the `**Result**` line of the summary, the range score before → end (changed families), the `Total` before → end if it changed, the path of the summary. Nothing is pushed: the user pushes `detectors` so that everyone has the updated detectors, or cancels what they do not keep with `git -C detectors revert`.
|