Files
aimsight-optimizer/docs/superpowers/specs/2026-11-16-optimize-design.md
T

5.2 KiB

/optimize: one analysis in, detectors and tools optimised

Date: 2026-11-16. Size: a small change, across five repositories.

Intent

Everything that optimises lives in Optimizer. Today Optimizer optimises the detectors (/improve), while the optimisation of the reading tools is a bare claude -p --agent agents:tool-builder call written in Diagnostic-App's server. An analysis run by the analyst already leaves, in its folder, the two things both optimisations need: its labels (labels.jsonl, the truth for the detectors) and its own scripts plus what it missed (scripts/, analysis.md, the material for the tools). So the workflow takes one analysis and optimises both.

Goals

  • One skill, /optimize, in Optimizer: input an analysis folder, output one session folder and one summary.
  • Diagnostic-App's "Optimize" button makes a single call, to /optimize.
  • Optimizer's scope becomes "the detectors and the tools, from an analysis".

Non-goals

  • No change to the analyst: it already writes its scripts in <output folder>/scripts/ and says what it missed.
  • No validator for the tools; the tool-builder stays as it is in its decisions.
  • No toolbox for the scripts of the upgrader and validator (sessions/*/tools/): later.
  • No change to the benchmark, the agents' judgement, or the labels.

Anti-goals

  • Optimizer writing in labeling/ (read-only): the tool-builder's outputs move to the session folder.
  • One step's failure blocking the other.

Approaches

  1. Extend /improve into /optimize ANALYSIS: <folder> (picked). The skill derives its inputs from the analysis, runs the current detectors steps, then the tool-builder, then writes one summary. One file, one session, one line in Diagnostic-App. /improve has a single caller, so nothing needs it alone.
  2. Keep /improve, add /optimize calling it then the tool-builder. Nothing existing moves, but two layers and two summaries to merge, for no caller that wants the detectors alone.
  3. Two skills called in turn by Diagnostic-App. Smallest diff, but the process (the order, the session) stays in Diagnostic-App, against the intent.

Design

Skill .claude/skills/optimize/SKILL.md (moved from improve/). Inputs:

  • ANALYSIS (required): the analysis folder (request.json, labels.jsonl, report.md, analysis.md, scripts/).
  • OUT (required): the session folder, sessions/<analysis>_<YYYYMMDDTHHMMZ>/.
  • Defaults, relative to Optimizer's links, overridable in the request: DETECTORS: detectors/, TOOLS: tools/, DATA: data, LABELS: labeling/auto (the folder of <STATION>.jsonl), FAMILIES: labeling/families.md.
  • Derived: STATION = <DATA>/<request.json station>; FROM = the day of start; TO = the day after end (excluded); labels = <LABELS>/<station>.jsonl; reference = --labels <LABELS> --data <DATA>; context = <ANALYSIS>/report.md.
  • Caller's context, passed on as today.

The multi-station mode (DATA + STATIONS) is dropped: an analysis is one station and no caller uses it (my call).

Steps.

  1. Detectors: the current steps of /improve, unchanged (measures, upgrader, measures, validator, remove/cancel). Commits stay improve: S<n> in DETECTORS (the agents grep it).
  2. Tools, always, whatever step 1 gave: the Agent tool, subagent_type: agents:tool-builder, request Analysis: <ANALYSIS>/. Tools: <TOOLS>/. Write <OUT>/tools.md. It commits in TOOLS itself.
  3. Summary <OUT>/summary.md: the current detectors part, then a ## Reading tools part (the tool-builder's "Tools changed" line and its "What I decided", copied). The current ## Tools section (the agents' scripts) is renamed ## Scripts of the agents.

agents repository. tool-builder.md: its trial figures and the reruns of the report's commands write in <the folder of its report>/work/, never in the analysis folder. The "improve workflow" wording of upgrader.md, validator.md and README.md becomes "optimize workflow"; the tool-builder's caller becomes Optimizer.

Optimizer. Link tools -> ~/tools; the launch adds --add-dir ~/tools. README.md and CLAUDE.md: scope "the detectors and the tools", the /optimize request, the check command.

Diagnostic-App. optimize_thread: one turn, /optimize ANALYSIS: labeling/auto/analyses/<name>/. OUT: sessions/ <name>_<time>/. Caller's context: …, from Optimizer, with --add-dir on detectors and tools. optimize() also requires the tools link and optimize/SKILL.md. summary() unchanged: it shows the session's summary.md, and the old <analysis>/tools.md if one exists. check.py: one launch, /optimize in Optimizer. Docs (CLAUDE.md, README.md, app/README.md) follow.

detectors, tools. The wording improve skill → optimize skill in detectors/CLAUDE.md, detectors/README.md; tools/README.md "Who changes it": the tool-builder, run by Optimizer's /optimize.

Check

  • env -u ANTHROPIC_API_KEY claude -p "/optimize" --plugin-dir agents --model haiku: says the inputs are missing, stops, touches neither detectors/ nor tools/.
  • Diagnostic-App's check.py passes.
  • A real session through the app's button, on an analysis with scripts.

Left to the builder

The exact wording of the skill and of the summary's tools part; the order of the doc edits.