diff --git a/instructions/upgrader-detectors.md b/instructions/upgrader-detectors.md new file mode 100644 index 0000000..a33d6a8 --- /dev/null +++ b/instructions/upgrader-detectors.md @@ -0,0 +1,41 @@ +# Optimizer's instructions to the upgrader: the detectors + +What `/optimize` asks of the upgrader (agents repository, `agents:upgrader`) in part 1, on top of its trade. The target is the detectors repository; its rules (what may change, the regression, everything is detected) are in `/CLAUDE.md`, read in full. + +## Why this range + +An analyst has analysed a range of a station where the detectors perform poorly. The labels of the range are the truth. You compare what the detectors find with these labels, event by event, all families, say what is wrong and which detector causes it, then fix it. Afterwards, the validator judges your modification on the reference truth (all labelled stations, all families): a gain on the range at the cost of another station or another family is refused. What counts is the mechanism behind a label (a rejected reference that makes all targets jump, for example), recognisable wherever it happens; a constant tuned to make these bounds fall exactly, a rule that only recognises the exact shape of these events: no. What the detectors cannot see with these measurements (a cause only an image reveals) goes to "Limits". + +## The evidence + +- The labels file of the station (or several stations on the same range, a labels file each, in one folder: the events and sensors checks run per station, the range score covers them all; a mechanism that comes back from one station to another comes before a single case). A label: one line of JSONL, `{"id", "start", "end", "family", "description"}` (cited by its `id`), a single anomaly, seen on `corr_x` and `corr_y` (they move or are missing); `description` says the sensors touched, the cause and how it was known. Its bounds go from the measurement just before the first point touched to the measurement just after the last. An instant of the range without a label: the corrected curves show nothing there. Read the labels touching the range (`start` < `TO` and `end` ≥ `FROM`). +- The context documents (the analysis report): the mechanism of each event, the sensors touched and those that do not move, the level of proof; a document may say that a label is wrong: a limit to report, not a target. +- The families document **in full**: the families and their boundaries; the category of an event is its family. +- The checks' outputs before: events (run it with `2>&1`), sensors (the cadence, then the sensors of each event), range score, benchmark. +- The code of the detectors, `scripts/categories.py` (how episodes become events: a jump back absorbed by its cause, the varieties) and `thresholds.md`. You may read the station's data to understand why a rule fails. + +## The diagnosis + +Compare, event by event and label by label: correct; wrong family; extra (flagged by the detectors, without a label); wrong bounds (late, too early, too wide); wrong sensors (the event's, by the sensors check, against those the description names); missing. A few measurements of gap at the bounds are not a defect. Several varieties (the column after the family: Poor visibility, Heat shimmer…) fall into one family: between them, the label does not decide. A reading of the detectors other than the label's but consistent (a boundary of the families document that allows both) is not a defect either: an "alternative reading", no fix. + +Fixes to try first: a significant event missed or badly bounded before a small nuisance. The simplest first, for the detectors, ends with: as a last resort, a new variety (a `label`) in the family's detector. + +## The checks + +- **events**: before and after each fix, with `2>&1`; after, no `skipped` line. +- **range**: the range score, what you make rise, without losing a whole event. A hole has no measurement: it only counts its two bounds; for a hole, read the events check. Do not aim for 100 %. +- **benchmark**: all lines of all stations, against the benchmark of the last committed state (the benchmark before, or that of your previous kept fix); before your output, the whole against the benchmark before. A regression (`/CLAUDE.md`): rework your change before committing it, or drop the fix. A drop from another consistent reading: keep it, and say it in "For the user to decide". For a shorter run, `--stations A,B` measures only these stations; the whole benchmark runs in the background, output to a file. To try a variant without touching the repository, copy the target in your working folder: the copy's scripts measure the copy, with the same arguments. + +## Your output + +On top of the skeleton of your trade: + +- `## Diagnosis` opens with `` and the table `| Family | labelled | found | flagged | wrongly |`, then one line per event or label: its judgement and why; same judgement and cause grouped. +- After `## Checks`: `## Before / after on the range` (the changed lines of the two outputs of the events check; the range score before → after, per family) and `## Benchmark` (the lines that changed, station and family, before → after, then the changed lines of the `Total` block). +- "For the user to decide": a reading of the detectors other than the labels', kept: the labels (id, station, moment, family), the detectors' reading, what it changes in the benchmark. "Limits": the label (id, bounds, family), what it would take to see it (image…); a label the context says is wrong (id). + +Times in UTC. The orchestrator reads the `**Verdict**` line as it is. + +## Your answer + +In 3 lines: the verdict and the fixes kept with their commits, what was not fixed, the path of the output. diff --git a/instructions/upgrader-tools.md b/instructions/upgrader-tools.md new file mode 100644 index 0000000..c47a7cb --- /dev/null +++ b/instructions/upgrader-tools.md @@ -0,0 +1,39 @@ +# Optimizer's instructions to the upgrader: the tools + +What `/optimize` asks of the upgrader (agents repository, `agents:upgrader`) in part 2, on top of its trade. The target is the tools repository (the reading harness); its rules (remove nothing, the simplest first, no judgement in a tool's output, `data.py`) are in `/CLAUDE.md`, read in full. + +## Why + +The analyst has just analysed a range as an expert, with the tools of the target, their guide (`GUIDE.md`) and, where they were not enough, its own scripts. It is up to you to decide what deserves to enter the tools, so that the next analysis, on any station, reads better. The analyst follows the guide; the callers read the data through `scripts/data.py`. + +## The evidence + +The analysis folder: `report.md` first, if it exists (the commands behind each fact, hence the tools the analyst used); then `analysis.md` (its "The documentation" section and its "Difficulties" say what it missed or what got in its way); its scripts, in `scripts/`. Then `/GUIDE.md` and the tools of `/scripts/`; the station folder, to try things. The station and the range are in the heading of `analysis.md`. Nothing else: neither the detectors (`detectors/`) nor `CLAUDE.md` of the root you are called from. + +## What you decide + +- **Optimise**: a tool that got in the analyst's way (output too long, unreadable figure, option to guess) is tuned: a better default, an option, a sentence of the guide. +- **Add**: a script of the analyst that showed what the tools did not show, and that would help on other stations, enters `/scripts/`, made general; or an option of an existing tool does the same, which is better. A script that only served this range stays where it is. +- **Nothing to do** is a decision: if the tools were enough and nothing got in the way, touch nothing and say so. + +You write nothing in the analysis folder. + +## The checks + +There is no check command (`Checks: none`). For each fix, before its commit: + +1. Run each changed or added tool on the range, from the root you are called from (`python3 /scripts/.py data/ …`), with `--start` and `--end` of the range and `--out /work/tools/`; look at each PNG with Read. +2. Rerun, on the range, the commands of the analysis report that use a changed tool, with `--out /work/` (replacing theirs, or added if absent): they must still work. + +## Your output + +On top of the skeleton of your trade: + +- after the `**Changed**` line: `**data.py**: unchanged | changed: `; +- `## What the analyst used` (before `## Diagnosis`): the tools and the scripts it used, and what it missed or what got in its way, summarised; +- `## What I decided` (after `## Diagnosis`): `- : , and why`; +- under `## Checks`: the trial commands and what they return, figures in `/work/tools/`; then `Report commands rerun: , and "no failure" or the failures`. + +## Your answer + +In 2 lines: what changed (or "no tool changed"), the path of your output. diff --git a/instructions/validator-detectors.md b/instructions/validator-detectors.md new file mode 100644 index 0000000..f2f275e --- /dev/null +++ b/instructions/validator-detectors.md @@ -0,0 +1,28 @@ +# Optimizer's instructions to the validator: the detectors + +What `/optimize` asks of the validator (agents repository, `agents:validator`) in part 1, on top of its trade. The target is the detectors repository; its rules (what may change, what loads, the regression) are in `/CLAUDE.md`, read in full. + +## Why + +The upgrader has just adapted the detectors to better find the events of a single range of a single station. The risk is overfitting: detectors glued to this range, which lose precision elsewhere, on another station or another family. + +## What you read, on top of your trade + +- The range score before → after: the gain the modification sought. +- The benchmarks (the score per family on each station of the reference truth, then their `Total`), all stations in full; the benchmarks after each fix, in order, say which fix makes a station go back. +- The families document. +- To understand a regression on a station: the events command on its range (from the header of the station's block in the benchmark), with `2>&1`, at HEAD and, to compare, at the base (`git -C archive | tar -x -C ` in your working folder, then the same command with the copy's `scripts/categories.py`), and the station's labels. + +## What you judge, for the detectors + +- A `skipped` line in a benchmark means cancel. +- Regressions: station by station and family by family, the range's station included, by the definition of `/CLAUDE.md`. A drop you do not count as a regression goes to "Reservations". +- A range gain of nothing or almost, at the price of one more rule, means cancel (or remove for the fix in question); a rule removed is better than a rule added. + +## Your output + +On top of the skeleton of your trade: `## The range` (the range score before → after, per family), placed before `## Checks`; under `## Checks`, the non-regression: the lines that changed, station and family, before → after, then the changed lines of the `Total` block. "Reservations": the station, the family, before → after, the detectors' reading against the labels (id). Times in UTC. + +## Your answer + +In 2 lines: the verdict, the path of the output.