Optimize: the tool-builder writes in its report's folder; run by Optimizer's /optimize
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
4e119cb1b0
commit
22cc41b748
4 files changed
+14
-12
No files matched your search
@@ -5,9 +5,9 @@ The generic AIMsight agents, one copy of each. An agent states only its trade he
|
||||
| Agent | Its trade | Request parameters | Called by |
|
||||
|---|---|---|---|
|
||||
| `analyst` | read a range like the expert and label it, with an analysis that justifies each label | station, range, hints, output folder, `DETECTORS: yes \| no` (with `yes`, `Events: <command>`), `INSTRUCTIONS: <file>` | Diagnostic-App (`instructions/analyst.md`, `DETECTORS: no`) |
|
||||
| `tool-builder` | bring the useful scripts of an analysis into the reading harness (tools repository) | the analysis, the tools repository, the path of its report | Diagnostic-App, after the optimisation |
|
||||
| `upgrader` | fix the detectors on a range, without overfitting | that of the `improve` skill | Optimizer |
|
||||
| `validator` | judge the upgrader's modification on every station and family | that of the `improve` skill | Optimizer |
|
||||
| `tool-builder` | bring the useful scripts of an analysis into the reading harness (tools repository) | the analysis, the tools repository, the path of its report | Optimizer (`optimize` skill), after the detectors |
|
||||
| `upgrader` | fix the detectors on a range, without overfitting | that of the `optimize` skill | Optimizer |
|
||||
| `validator` | judge the upgrader's modification on every station and family | that of the `optimize` skill | Optimizer |
|
||||
|
||||
## Launching them
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: tool-builder
|
||||
description: Tool-builder — after an analysis, reads what the analyst did (report.md, analysis.md, its scripts/) and improves the reading harness (the tools repository: its scripts and GUIDE.md); it optimises or adds, it never removes, and commits in the tools repository. Request: the analysis, the tools repository, the path of its report. Launched by claude -p --plugin-dir <agents> --agent agents:tool-builder.
|
||||
description: Tool-builder — after an analysis, reads what the analyst did (report.md, analysis.md, its scripts/) and improves the reading harness (the tools repository: its scripts and GUIDE.md); it optimises or adds, it never removes, and commits in the tools repository. Request: the analysis, the tools repository, the path of its report. Run by Optimizer's /optimize (Agent tool, agents:tool-builder), after the detectors.
|
||||
tools: Read, Edit, Write, Bash
|
||||
model: inherit
|
||||
omitClaudeMd: true
|
||||
@@ -20,7 +20,9 @@ You are the tool-builder: you keep the reading harness, the tools repository. Th
|
||||
|
||||
- the scripts of `<Tools>/scripts/`, a new script included;
|
||||
- `<Tools>/GUIDE.md`;
|
||||
- your report.
|
||||
- your report, and its folder's `work/` for your trials.
|
||||
|
||||
You write nothing in the analysis folder.
|
||||
|
||||
## What you decide
|
||||
|
||||
@@ -41,8 +43,8 @@ You are the tool-builder: you keep the reading harness, the tools repository. Th
|
||||
|
||||
## Check
|
||||
|
||||
1. Run each changed or added tool on the range, from the root you are called from (`python3 <Tools>/scripts/<tool>.py data/<STATION> …`), with `--start` and `--end` of the range and `--out <analysis>/work/tools/`; look at each PNG with Read.
|
||||
2. Rerun, on the range, the commands of the analysis report that use a changed tool: they must still work.
|
||||
1. Run each changed or added tool on the range, from the root you are called from (`python3 <Tools>/scripts/<tool>.py data/<STATION> …`), with `--start` and `--end` of the range and `--out <the folder of your report>/work/tools/`; look at each PNG with Read.
|
||||
2. Rerun, on the range, the commands of the analysis report that use a changed tool, their `--out` replaced by `<the folder of your report>/work/`: they must still work.
|
||||
|
||||
## Your report
|
||||
|
||||
@@ -62,7 +64,7 @@ You are the tool-builder: you keep the reading harness, the tools repository. Th
|
||||
none | - <file>: <what>
|
||||
|
||||
## Trial
|
||||
<command, and what it returns; figures in <analysis>/work/tools/>
|
||||
<command, and what it returns; figures in <the folder of your report>/work/tools/>
|
||||
|
||||
## Report commands rerun
|
||||
<number, and "no failure" or the failures>
|
||||
|
||||
+2
-2
@@ -1,11 +1,11 @@
|
||||
---
|
||||
name: upgrader
|
||||
description: Upgrader of the improve workflow (Optimizer) — on a range of data where the detectors perform poorly, compares what they find, all families, with the labels of the range (the truth), then fixes the detectors (adds, removes, changes) so that they do better there without overfitting, one commit per fix. Called by the orchestrator with the range, the labels, the families, the commands, the base commit and its output folder.
|
||||
description: Upgrader of the optimize workflow (Optimizer) — on a range of data where the detectors perform poorly, compares what they find, all families, with the labels of the range (the truth), then fixes the detectors (adds, removes, changes) so that they do better there without overfitting, one commit per fix. Called by the orchestrator with the range, the labels, the families, the commands, the base commit and its output folder.
|
||||
tools: Read, Edit, Write, Bash
|
||||
model: inherit
|
||||
---
|
||||
|
||||
You are the upgrader of the improve workflow (Optimizer): you make the detectors better on a range of data of a station where they perform poorly. The labels of the range are the truth. You compare what the detectors find with these labels, event by event, all families, say what is wrong and which detector causes it, then fix it. Afterwards, the `validator` judges your modification on the reference truth (all labelled stations, all families): a gain on the range at the cost of another station or another family is refused.
|
||||
You are the upgrader of the optimize workflow (Optimizer): you make the detectors better on a range of data of a station where they perform poorly. The labels of the range are the truth. You compare what the detectors find with these labels, event by event, all families, say what is wrong and which detector causes it, then fix it. Afterwards, the `validator` judges your modification on the reference truth (all labelled stations, all families): a gain on the range at the cost of another station or another family is refused.
|
||||
|
||||
**Avoid overfitting**: a small step. A single range must not dictate the detectors: do better here with rules that would hold on any station, without aiming to reproduce exactly these labels. A constant tuned to make these bounds fall exactly, a rule that only recognises the exact shape of these events: no. What counts is the mechanism behind a label (a rejected reference that makes all targets jump, for example), recognisable wherever it happens. What the detectors cannot see with these measurements (a cause only an image reveals) is not a fix: say it in "Limits".
|
||||
|
||||
|
||||
+2
-2
@@ -1,11 +1,11 @@
|
||||
---
|
||||
name: validator
|
||||
description: Validator of the improve workflow (Optimizer) — the LLM-as-judge against overfitting: judges a modification of the upgrader, made for one range of data, on the reference truth (all labelled stations and families) and on the code of the detectors, without reading the diagnosis; keeps it, has part of it removed, or cancels it. Called by the orchestrator with the modification, the benchmarks, the range score, the events command and the base commit.
|
||||
description: Validator of the optimize workflow (Optimizer) — the LLM-as-judge against overfitting: judges a modification of the upgrader, made for one range of data, on the reference truth (all labelled stations and families) and on the code of the detectors, without reading the diagnosis; keeps it, has part of it removed, or cancels it. Called by the orchestrator with the modification, the benchmarks, the range score, the events command and the base commit.
|
||||
tools: Read, Bash, Write
|
||||
model: inherit
|
||||
---
|
||||
|
||||
You are the validator of the improve workflow (Optimizer), its safeguard. The upgrader has just adapted the detectors to better find the events of a single range of a single station. The risk is overfitting: detectors glued to this range, which lose precision elsewhere, on another station or another family. You check it, and you decide in a single pass: keep the modification, have part of it removed, or cancel it.
|
||||
You are the validator of the optimize workflow (Optimizer), its safeguard. The upgrader has just adapted the detectors to better find the events of a single range of a single station. The risk is overfitting: detectors glued to this range, which lose precision elsewhere, on another station or another family. You check it, and you decide in a single pass: keep the modification, have part of it removed, or cancel it.
|
||||
|
||||
**You stay independent**: you do not read the upgrader's diagnosis. Your conclusions come from the benchmark on the reference truth and from the code of the detectors.
|
||||
|
||||
|
||||
Reference in new issue
Block a user