From c44b9286c9739f49acc2444b3294273a34de50d9 Mon Sep 17 00:00:00 2001 From: maxime Date: Mon, 16 Nov 2026 14:44:24 +0100 Subject: [PATCH] optimize: generic agents with instructions/; the tools by the upgrader Co-Authored-By: Claude Opus 5.5 --- .claude/skills/optimize/SKILL.md | 18 +++++++++--------- CLAUDE.md | 4 ++-- README.md | 6 +++--- 3 files changed, 14 insertions(+), 14 deletions(-) diff --git a/.claude/skills/optimize/SKILL.md b/.claude/skills/optimize/SKILL.md index 66e059a..e09a161 100644 --- a/.claude/skills/optimize/SKILL.md +++ b/.claude/skills/optimize/SKILL.md @@ -1,12 +1,12 @@ --- name: optimize -description: Workflow that, from one analysis (its labels as the truth on its range, its scripts and what it missed), optimises the detectors — the upgrader fixes, the validator judges; detectors a little better on the range without going back elsewhere (benchmark on a reference truth), or unchanged — then the reading tools — the tool-builder brings the analyst's useful scripts into them. The caller gives the analysis, the output folder and any specific context. Run from Optimizer's root with the agents plugin (--plugin-dir ). /optimize . +description: Workflow that, from one analysis (its labels as the truth on its range, its scripts and what it missed), optimises the detectors — the upgrader fixes, the validator judges; detectors a little better on the range without going back elsewhere (benchmark on a reference truth), or unchanged — then the reading tools — the upgrader brings the analyst's useful scripts into them. Each agent gets its instructions from `instructions/`. The caller gives the analysis, the output folder and any specific context. Run from Optimizer's root with the agents plugin (--plugin-dir ). /optimize . argument-hint: "ANALYSIS: . OUT: . [context]" --- # optimize: the detectors and the tools, from one analysis -You are the orchestrator. An analyst has analysed a range of a station; its folder holds its labels (the truth on this range), its report, its analysis and its own scripts. Two parts follow, in this order. **Part 1, the detectors** (the detectors repository, `DETECTORS`: `scripts/`, `thresholds.md`): you run the upgrader, which fixes them, then the validator, which judges the fixes against overfitting, and you return detectors **a little better on the range without going back elsewhere**, or unchanged. **Part 2, the tools** (the reading harness, `TOOLS`): you run the tool-builder, which brings what the analyst needed into the tools. **You judge nothing yourself**: you only read the verdict lines. **You ask no question**: the workflow runs alone to the end. +You are the orchestrator. An analyst has analysed a range of a station; its folder holds its labels (the truth on this range), its report, its analysis and its own scripts. Two parts follow, in this order. **Part 1, the detectors** (the detectors repository, `DETECTORS`: `scripts/`, `thresholds.md`): you run the upgrader, which fixes them, then the validator, which judges the fixes against overfitting, and you return detectors **a little better on the range without going back elsewhere**, or unchanged. **Part 2, the tools** (the reading harness, `TOOLS`): you run the upgrader on them, which brings what the analyst needed into the tools. **You judge nothing yourself**: you only read the verdict lines. **You ask no question**: the workflow runs alone to the end. ## Inputs @@ -37,20 +37,20 @@ The measures of a step: ` > /range-.txt 2>&1` and ` rev-parse HEAD`. Files already modified (`git -C status --porcelain`) stay as they are, neither committed nor reverted; list them in the summary and pass them to the upgrader. Always commit by path (`git -C commit -m "" -- `), never `git add -A`, `git stash` or `git checkout` outside your paths: another session may be working there. Each message ends with an empty line then `Co-Authored-By: Claude Opus 5.5 `. The tool-builder commits in `TOOLS` itself. Nothing is pushed. The rest (branch, committing `OUT`, another repository) is the caller's. +The fixes are commits in `DETECTORS`, one per fix, so that the validator can have one removed. Note the **base**: `git -C rev-parse HEAD`. Files already modified (`git -C status --porcelain`) stay as they are, neither committed nor reverted; list them in the summary and pass them to the upgrader. The same for `TOOLS` before part 2: its **tools base** (`git -C rev-parse HEAD`) and its files already modified. Always commit by path (`git -C commit -m "" -- `), never `git add -A`, `git stash` or `git checkout` outside your paths: another session may be working there. Each message ends with an empty line then `Co-Authored-By: Claude Opus 5.5 `. The upgrader commits in `TOOLS` itself in part 2, one commit per fix. Nothing is pushed. The rest (branch, committing `OUT`, another repository) is the caller's. ## Steps -Each agent is called with the Agent tool, its `subagent_type` (`agents:upgrader`, `agents:validator`, `agents:tool-builder`, from the agents plugin loaded with `--plugin-dir`; not available: say so and stop, changing nothing) and the request below, plus the caller's context. A verdict is read by `grep '^\*\*Verdict\*\*:' `. +Each agent is called with the Agent tool, its `subagent_type` (`agents:upgrader`, `agents:validator`, from the agents plugin loaded with `--plugin-dir`; not available: say so and stop, changing nothing) and the request below, plus the caller's context. A verdict is read by `grep '^\*\*Verdict\*\*:' `. ### Part 1: the detectors 1. **Measures before**: step `before`. -2. **Upgrader**, a single pass: `Detectors: . Station: , range → (UTC). Labels: /.jsonl. Families: . Context: . Events command: <…>. Sensors command: <…>. Range score command: <…> (before: /range-before.txt). Benchmark command: <…> (before: /benchmark-before.txt). Base: . Files already modified: . Output folder: . Write /modification.md.` +2. **Upgrader**, a single pass: `Target: . Evidence: station , range → (UTC); labels /.jsonl; families ; context . Checks: events: (before: run it); sensors: (before: run it); range: (before: /range-before.txt); benchmark: (before: /benchmark-before.txt). Base: . Files already modified: . Output folder: . Write /modification.md. INSTRUCTIONS: instructions/upgrader-detectors.md.` It commits each fix it keeps, message `improve: S`, with its benchmark `/benchmark-S.txt`. Verdict `nothing to change`, or no fix commit (`git -C log --format=%h --grep ': S[0-9]*$' ..HEAD` empty): go to part 2. 3. **Measures after**: step `after`. -4. **Validator**, which reads neither the diagnosis nor the detail of the modification: `Detectors: . Families: . Modification: /modification.md. Range score: /range-before.txt → /range-after.txt. Benchmark: /benchmark-before.txt → /benchmark-after.txt, after each fix: /benchmark-S*.txt; command: <…>. Events command for any station: python3 /scripts/categories.py --from --to . Base: . Write /validation.md.` +4. **Validator**, which reads neither the diagnosis nor the detail of the modification: `Target: . Modification: /modification.md. Base: . Checks: range: ; before: /range-before.txt; after: /range-after.txt; benchmark: ; before: /benchmark-before.txt; after: /benchmark-after.txt; after each fix: /benchmark-S*.txt; events, for any station: python3 /scripts/categories.py --from --to . Families: . Output: /validation.md. INSTRUCTIONS: instructions/validator-detectors.md.` - `keep`: go to part 2. - `remove`: for each fix of the `**Remove**:` line, its commits (`git -C log --format=%h --grep ': S$' ..HEAD`), from the most recent: `git -C revert --no-edit `, then `git -C commit --amend -m "improve: S removed" -m "Co-Authored-By: Claude Opus 5.5 "`. A conflict: `git -C revert --abort`, and do as for `cancel`. - `cancel`, or anything else: `git -C revert --no-commit ..HEAD && git -C commit -m "improve: modification cancelled" -m "Co-Authored-By: Claude Opus 5.5 "` (only what the fix commits changed; files already modified stay as they are). @@ -62,7 +62,7 @@ An output missing or without a readable verdict: note it, put the detectors back Always, whatever part 1 gave (a stop in part 1 does not stop part 2). -6. **Tool-builder**: `Analysis: /. Tools: /. Write /tools.md.` It reads the analysis, writes only in `TOOLS` and in ``, and commits in `TOOLS`. Its output missing: note it. Do not rerun it. +6. **Upgrader on the tools**: `Target: . Evidence: /. Checks: none. Base: . Files already modified: . Output folder: . Write /tools.md. INSTRUCTIONS: instructions/upgrader-tools.md.` It reads the analysis, writes only in `TOOLS` and in ``, and commits in `TOOLS`, one commit per fix. Its output missing: note it. Do not rerun it. ## Summary @@ -72,7 +72,7 @@ Always, whatever part 1 gave (a stop in part 1 does not stop part 2). # optimize: , → **Result**: modification kept | kept without S… | cancelled | none | error -**Tools changed**: /tools.md> | error +**Tools changed**: /tools.md> | error ## The range the range score, before → end, one line per family (found, correct) @@ -96,7 +96,7 @@ the `Total` line of each family, before → end; then the other changed lines (s /tools/`); none> ## Errors - log --oneline ..HEAD`; `git -C log --oneline -1` if the tools changed> + log --oneline ..HEAD`; `git -C log --oneline ..HEAD` if the tools changed> ``` ## Answer diff --git a/CLAUDE.md b/CLAUDE.md index faf9879..b148b1c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,10 +1,10 @@ # CLAUDE.md -Optimizer: the optimisation of the AIMsight detectors and of the reading tools, from one analysis of `labeling/auto`, and nothing else (`README.md`). The workflow: `.claude/skills/optimize/SKILL.md` (`/optimize`), launched from the root of Optimizer with the agents plugin (`--plugin-dir agents`); its agents `agents:upgrader`, `agents:validator` (the detectors) and `agents:tool-builder` (the tools); the benchmark `detectors/bench/`. +Optimizer: the optimisation of the AIMsight detectors and of the reading tools, from one analysis of `labeling/auto`, and nothing else (`README.md`). The workflow: `.claude/skills/optimize/SKILL.md` (`/optimize`), launched from the root of Optimizer with the agents plugin (`--plugin-dir agents`); its agents `agents:upgrader`, `agents:validator` (the detectors) and `agents:upgrader` again (the tools), each with its file of `instructions/`; the benchmark `detectors/bench/`. - **The links**, out of git: `detectors/` → `~/detectors` and `tools/` → `~/tools` (the session commits there); `data/` → `~/aimsight-data/data` and `labeling/` → `~/labeling` (read-only); `agents/` → `~/agents` (the plugin, loaded read-only). - **The sessions**: `sessions/_/`; Diagnostic-App reads their summary (`summary.md`). -- **The workflow stays generic**: the skill describes only the workflow (analysis → detectors: upgrader → validator; tools: tool-builder → summary); what is specific to a caller (its paths, its git rules, `Caller's context`) arrives in the request. +- **The workflow stays generic**: the skill describes only the workflow (analysis → detectors: upgrader → validator; tools: upgrader → summary); what is specific to a caller (its paths, its git rules, `Caller's context`) arrives in the request. - **Check a change of the skill**: `env -u ANTHROPIC_API_KEY claude -p "/optimize" --plugin-dir agents --model haiku` must say the inputs are missing and stop without touching `detectors/` or `tools/`; then a real session through the app's button. - `archive/`: the former batch and its scanner, stopped; do not run them again. diff --git a/README.md b/README.md index 5beeea8..9dadcd0 100644 --- a/README.md +++ b/README.md @@ -3,10 +3,10 @@ The optimisation of the AIMsight detectors and of the reading tools, from one analysis, and nothing else. An analysis (made in the Diagnostic-App app by the analyst) leaves in its folder the two things needed: its labels, the truth on its range, and its own scripts with what it missed. On that analysis, the `optimize` skill: 1. **the detectors**: measures their score on the range, and the benchmark on every label of `labeling/auto`; has them fixed by the `upgrader`, one fix per commit in `~/detectors`; has them judged by the `validator`, which keeps, has some removed, or cancels: no station goes back, no overfitting; -2. **the tools**: has the `tool-builder` bring the analyst's useful scripts into the reading harness, commits in `~/tools`; always, whatever the detectors gave; +2. **the tools**: has the `upgrader` bring the analyst's useful scripts into the reading harness, one fix per commit in `~/tools`; always, whatever the detectors gave; 3. writes the summary of the session. -The `upgrader`, the `validator` and the `tool-builder` come from the `agents` repository; the benchmark (`bench/`) from the `detectors` repository. Nothing is pushed. +The `upgrader` and the `validator` come from the `agents` repository, each with its instructions from `instructions/` (`upgrader-detectors.md`, `validator-detectors.md`, `upgrader-tools.md`); the rules of each target are its own `CLAUDE.md`; the benchmark (`bench/`) from the `detectors` repository. Nothing is pushed. ## Install @@ -35,6 +35,6 @@ The station and the range come from the analysis's `request.json` (the range's l ## What a session leaves -`sessions/_