91 lines
5.2 KiB
Markdown
91 lines
5.2 KiB
Markdown
# `/optimize`: one analysis in, detectors and tools optimised
|
|
|
|
Date: 2026-11-16. Size: a small change, across five repositories.
|
|
|
|
## Intent
|
|
|
|
Everything that optimises lives in Optimizer. Today Optimizer optimises the detectors (`/improve`), while the
|
|
optimisation of the reading tools is a bare `claude -p --agent agents:tool-builder` call written in Diagnostic-App's
|
|
server. An analysis run by the analyst already leaves, in its folder, the two things both optimisations need: its labels
|
|
(`labels.jsonl`, the truth for the detectors) and its own scripts plus what it missed (`scripts/`, `analysis.md`, the
|
|
material for the tools). So the workflow takes **one analysis** and optimises **both**.
|
|
|
|
## Goals
|
|
|
|
- One skill, `/optimize`, in Optimizer: input an analysis folder, output one session folder and one summary.
|
|
- Diagnostic-App's "Optimize" button makes a single call, to `/optimize`.
|
|
- Optimizer's scope becomes "the detectors and the tools, from an analysis".
|
|
|
|
## Non-goals
|
|
|
|
- No change to the analyst: it already writes its scripts in `<output folder>/scripts/` and says what it missed.
|
|
- No validator for the tools; the tool-builder stays as it is in its decisions.
|
|
- No toolbox for the scripts of the upgrader and validator (`sessions/*/tools/`): later.
|
|
- No change to the benchmark, the agents' judgement, or the labels.
|
|
|
|
## Anti-goals
|
|
|
|
- Optimizer writing in `labeling/` (read-only): the tool-builder's outputs move to the session folder.
|
|
- One step's failure blocking the other.
|
|
|
|
## Approaches
|
|
|
|
1. **Extend `/improve` into `/optimize ANALYSIS: <folder>`** (picked). The skill derives its inputs from the analysis,
|
|
runs the current detectors steps, then the tool-builder, then writes one summary. One file, one session, one line in
|
|
Diagnostic-App. `/improve` has a single caller, so nothing needs it alone.
|
|
2. Keep `/improve`, add `/optimize` calling it then the tool-builder. Nothing existing moves, but two layers and two
|
|
summaries to merge, for no caller that wants the detectors alone.
|
|
3. Two skills called in turn by Diagnostic-App. Smallest diff, but the process (the order, the session) stays in
|
|
Diagnostic-App, against the intent.
|
|
|
|
## Design
|
|
|
|
**Skill** `.claude/skills/optimize/SKILL.md` (moved from `improve/`). Inputs:
|
|
|
|
- `ANALYSIS` (required): the analysis folder (`request.json`, `labels.jsonl`, `report.md`, `analysis.md`, `scripts/`).
|
|
- `OUT` (required): the session folder, `sessions/<analysis>_<YYYYMMDDTHHMMZ>/`.
|
|
- Defaults, relative to Optimizer's links, overridable in the request: `DETECTORS: detectors/`, `TOOLS: tools/`,
|
|
`DATA: data`, `LABELS: labeling/auto` (the folder of `<STATION>.jsonl`), `FAMILIES: labeling/families.md`.
|
|
- Derived: `STATION` = `<DATA>/<request.json station>`; `FROM` = the day of `start`; `TO` = the day after `end`
|
|
(excluded); labels = `<LABELS>/<station>.jsonl`; reference = `--labels <LABELS> --data <DATA>`;
|
|
context = `<ANALYSIS>/report.md`.
|
|
- `Caller's context`, passed on as today.
|
|
|
|
The multi-station mode (`DATA` + `STATIONS`) is dropped: an analysis is one station and no caller uses it (my call).
|
|
|
|
**Steps.**
|
|
1. Detectors: the current steps of `/improve`, unchanged (measures, upgrader, measures, validator, remove/cancel).
|
|
Commits stay `improve: S<n>` in `DETECTORS` (the agents grep it).
|
|
2. Tools, always, whatever step 1 gave: the Agent tool, `subagent_type: agents:tool-builder`, request
|
|
`Analysis: <ANALYSIS>/. Tools: <TOOLS>/. Write <OUT>/tools.md.` It commits in `TOOLS` itself.
|
|
3. Summary `<OUT>/summary.md`: the current detectors part, then a `## Reading tools` part (the tool-builder's
|
|
"Tools changed" line and its "What I decided", copied). The current `## Tools` section (the agents' scripts) is renamed
|
|
`## Scripts of the agents`.
|
|
|
|
**agents repository.** `tool-builder.md`: its trial figures and the reruns of the report's commands write in
|
|
`<the folder of its report>/work/`, never in the analysis folder. The "improve workflow" wording of `upgrader.md`,
|
|
`validator.md` and `README.md` becomes "optimize workflow"; the tool-builder's caller becomes Optimizer.
|
|
|
|
**Optimizer.** Link `tools -> ~/tools`; the launch adds `--add-dir ~/tools`. `README.md` and `CLAUDE.md`: scope "the
|
|
detectors and the tools", the `/optimize` request, the check command.
|
|
|
|
**Diagnostic-App.** `optimize_thread`: one turn, `/optimize ANALYSIS: labeling/auto/analyses/<name>/. OUT: sessions/
|
|
<name>_<time>/. Caller's context: …`, from Optimizer, with `--add-dir` on detectors and tools. `optimize()` also requires
|
|
the `tools` link and `optimize/SKILL.md`. `summary()` unchanged: it shows the session's `summary.md`, and the old
|
|
`<analysis>/tools.md` if one exists. `check.py`: one launch, `/optimize` in Optimizer. Docs (`CLAUDE.md`, `README.md`,
|
|
`app/README.md`) follow.
|
|
|
|
**detectors, tools.** The wording `improve` skill → `optimize` skill in `detectors/CLAUDE.md`, `detectors/README.md`;
|
|
`tools/README.md` "Who changes it": the tool-builder, run by Optimizer's `/optimize`.
|
|
|
|
## Check
|
|
|
|
- `env -u ANTHROPIC_API_KEY claude -p "/optimize" --plugin-dir agents --model haiku`: says the inputs are missing, stops,
|
|
touches neither `detectors/` nor `tools/`.
|
|
- Diagnostic-App's `check.py` passes.
|
|
- A real session through the app's button, on an analysis with scripts.
|
|
|
|
## Left to the builder
|
|
|
|
The exact wording of the skill and of the summary's tools part; the order of the doc edits.
|