Optimize: /optimize, the detectors and the tools from one analysis (ex /improve)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
63bebbc415
commit
68768dfd39
7 files changed
+299
-107
No files matched your search
@@ -0,0 +1,90 @@
|
||||
# `/optimize`: one analysis in, detectors and tools optimised
|
||||
|
||||
Date: 2026-11-16. Size: a small change, across five repositories.
|
||||
|
||||
## Intent
|
||||
|
||||
Everything that optimises lives in Optimizer. Today Optimizer optimises the detectors (`/improve`), while the
|
||||
optimisation of the reading tools is a bare `claude -p --agent agents:tool-builder` call written in Diagnostic-App's
|
||||
server. An analysis run by the analyst already leaves, in its folder, the two things both optimisations need: its labels
|
||||
(`labels.jsonl`, the truth for the detectors) and its own scripts plus what it missed (`scripts/`, `analysis.md`, the
|
||||
material for the tools). So the workflow takes **one analysis** and optimises **both**.
|
||||
|
||||
## Goals
|
||||
|
||||
- One skill, `/optimize`, in Optimizer: input an analysis folder, output one session folder and one summary.
|
||||
- Diagnostic-App's "Optimize" button makes a single call, to `/optimize`.
|
||||
- Optimizer's scope becomes "the detectors and the tools, from an analysis".
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No change to the analyst: it already writes its scripts in `<output folder>/scripts/` and says what it missed.
|
||||
- No validator for the tools; the tool-builder stays as it is in its decisions.
|
||||
- No toolbox for the scripts of the upgrader and validator (`sessions/*/tools/`): later.
|
||||
- No change to the benchmark, the agents' judgement, or the labels.
|
||||
|
||||
## Anti-goals
|
||||
|
||||
- Optimizer writing in `labeling/` (read-only): the tool-builder's outputs move to the session folder.
|
||||
- One step's failure blocking the other.
|
||||
|
||||
## Approaches
|
||||
|
||||
1. **Extend `/improve` into `/optimize ANALYSIS: <folder>`** (picked). The skill derives its inputs from the analysis,
|
||||
runs the current detectors steps, then the tool-builder, then writes one summary. One file, one session, one line in
|
||||
Diagnostic-App. `/improve` has a single caller, so nothing needs it alone.
|
||||
2. Keep `/improve`, add `/optimize` calling it then the tool-builder. Nothing existing moves, but two layers and two
|
||||
summaries to merge, for no caller that wants the detectors alone.
|
||||
3. Two skills called in turn by Diagnostic-App. Smallest diff, but the process (the order, the session) stays in
|
||||
Diagnostic-App, against the intent.
|
||||
|
||||
## Design
|
||||
|
||||
**Skill** `.claude/skills/optimize/SKILL.md` (moved from `improve/`). Inputs:
|
||||
|
||||
- `ANALYSIS` (required): the analysis folder (`request.json`, `labels.jsonl`, `report.md`, `analysis.md`, `scripts/`).
|
||||
- `OUT` (required): the session folder, `sessions/<analysis>_<YYYYMMDDTHHMMZ>/`.
|
||||
- Defaults, relative to Optimizer's links, overridable in the request: `DETECTORS: detectors/`, `TOOLS: tools/`,
|
||||
`DATA: data`, `LABELS: labeling/auto` (the folder of `<STATION>.jsonl`), `FAMILIES: labeling/families.md`.
|
||||
- Derived: `STATION` = `<DATA>/<request.json station>`; `FROM` = the day of `start`; `TO` = the day after `end`
|
||||
(excluded); labels = `<LABELS>/<station>.jsonl`; reference = `--labels <LABELS> --data <DATA>`;
|
||||
context = `<ANALYSIS>/report.md`.
|
||||
- `Caller's context`, passed on as today.
|
||||
|
||||
The multi-station mode (`DATA` + `STATIONS`) is dropped: an analysis is one station and no caller uses it (my call).
|
||||
|
||||
**Steps.**
|
||||
1. Detectors: the current steps of `/improve`, unchanged (measures, upgrader, measures, validator, remove/cancel).
|
||||
Commits stay `improve: S<n>` in `DETECTORS` (the agents grep it).
|
||||
2. Tools, always, whatever step 1 gave: the Agent tool, `subagent_type: agents:tool-builder`, request
|
||||
`Analysis: <ANALYSIS>/. Tools: <TOOLS>/. Write <OUT>/tools.md.` It commits in `TOOLS` itself.
|
||||
3. Summary `<OUT>/summary.md`: the current detectors part, then a `## Reading tools` part (the tool-builder's
|
||||
"Tools changed" line and its "What I decided", copied). The current `## Tools` section (the agents' scripts) is renamed
|
||||
`## Scripts of the agents`.
|
||||
|
||||
**agents repository.** `tool-builder.md`: its trial figures and the reruns of the report's commands write in
|
||||
`<the folder of its report>/work/`, never in the analysis folder. The "improve workflow" wording of `upgrader.md`,
|
||||
`validator.md` and `README.md` becomes "optimize workflow"; the tool-builder's caller becomes Optimizer.
|
||||
|
||||
**Optimizer.** Link `tools -> ~/tools`; the launch adds `--add-dir ~/tools`. `README.md` and `CLAUDE.md`: scope "the
|
||||
detectors and the tools", the `/optimize` request, the check command.
|
||||
|
||||
**Diagnostic-App.** `optimize_thread`: one turn, `/optimize ANALYSIS: labeling/auto/analyses/<name>/. OUT: sessions/
|
||||
<name>_<time>/. Caller's context: …`, from Optimizer, with `--add-dir` on detectors and tools. `optimize()` also requires
|
||||
the `tools` link and `optimize/SKILL.md`. `summary()` unchanged: it shows the session's `summary.md`, and the old
|
||||
`<analysis>/tools.md` if one exists. `check.py`: one launch, `/optimize` in Optimizer. Docs (`CLAUDE.md`, `README.md`,
|
||||
`app/README.md`) follow.
|
||||
|
||||
**detectors, tools.** The wording `improve` skill → `optimize` skill in `detectors/CLAUDE.md`, `detectors/README.md`;
|
||||
`tools/README.md` "Who changes it": the tool-builder, run by Optimizer's `/optimize`.
|
||||
|
||||
## Check
|
||||
|
||||
- `env -u ANTHROPIC_API_KEY claude -p "/optimize" --plugin-dir agents --model haiku`: says the inputs are missing, stops,
|
||||
touches neither `detectors/` nor `tools/`.
|
||||
- Diagnostic-App's `check.py` passes.
|
||||
- A real session through the app's button, on an analysis with scripts.
|
||||
|
||||
## Left to the builder
|
||||
|
||||
The exact wording of the skill and of the summary's tools part; the order of the doc edits.
|
||||
Reference in new issue
Block a user