Files
aimsight-optimizer/.claude/skills/optimize/SKILL.md
T

8.6 KiB

name, description, argument-hint
name description argument-hint
optimize Workflow that, from one analysis (its labels as the truth on its range, its scripts and what it missed), optimises the detectors — the upgrader fixes, the validator judges; detectors a little better on the range without going back elsewhere (benchmark on a reference truth), or unchanged — then the reading tools — the tool-builder brings the analyst's useful scripts into them. The caller gives the analysis, the output folder and any specific context. Run from Optimizer's root with the agents plugin (--plugin-dir <agents>). /optimize <request>. ANALYSIS: <analysis folder>. OUT: <output folder>. [context]

optimize: the detectors and the tools, from one analysis

You are the orchestrator. An analyst has analysed a range of a station; its folder holds its labels (the truth on this range), its report, its analysis and its own scripts. Two parts follow, in this order. Part 1, the detectors (the detectors repository, DETECTORS: scripts/, thresholds.md): you run the upgrader, which fixes them, then the validator, which judges the fixes against overfitting, and you return detectors a little better on the range without going back elsewhere, or unchanged. Part 2, the tools (the reading harness, TOOLS): you run the tool-builder, which brings what the analyst needed into the tools. You judge nothing yourself: you only read the verdict lines. You ask no question: the workflow runs alone to the end.

Inputs

Given by the caller, in the request; any other context or rule it gives (files to read, git rules, where to commit, names) applies to this run and is passed on to the agents as it is (Caller's context: …), before their instructions but not before their file scope.

  • ANALYSIS (required): the analysis folder: request.json (station, start, end, UTC), labels.jsonl, report.md, analysis.md, scripts/.
  • OUT (required): the output folder of this run (sessions/<analysis>_<YYYYMMDDTHHMMZ>/ in Optimizer).
  • Defaults, relative to Optimizer's root (its links), which the request may override: DETECTORS: detectors/, TOOLS: tools/, DATA: data, LABELS: labeling/auto (the folder of the <STATION>.jsonl label files, JSONL: id, start, end, family, description), FAMILIES: labeling/families.md (the families and their boundaries, one detector per family).

Derived from <ANALYSIS>/request.json:

  • STATION: <DATA>/<station> (read by DETECTORS/scripts/data.py);
  • FROM: the day of start; TO: the day after the day of end (UTC days, YYYY-MM-DD, TO excluded);
  • the labels of the range: <LABELS>/<station>.jsonl (the analysis's labels are submitted there);
  • REFERENCE, the reference truth for the non-regression: --labels <LABELS> --data <DATA>;
  • CONTEXT: <ANALYSIS>/report.md.

ANALYSIS or OUT missing, no <ANALYSIS>/request.json, DETECTORS or TOOLS not a git repository: say so and stop, changing nothing.

Commands

  • events: python3 <DETECTORS>/scripts/categories.py <STATION> --from <FROM> --to <TO>
  • sensors: python3 <DETECTORS>/bench/sensors.py <STATION> --from <FROM> --to <TO>
  • range score (the goal): python3 <DETECTORS>/bench/score.py <STATION> --labels <LABELS>/<station>.jsonl --from <FROM> --to <TO>; it exits with an error if no label touches the range: note it in the summary, touch nothing in the detectors and go to part 2.
  • benchmark (the non-regression, about 3 min): python3 <DETECTORS>/bench/regression.py <REFERENCE>

The measures of a step: <range score> > <OUT>/range-<step>.txt 2>&1 and <benchmark> > <OUT>/benchmark-<step>.txt 2>&1.

Git

The fixes are commits in DETECTORS, one per fix, so that the validator can have one removed. Note the base: git -C <DETECTORS> rev-parse HEAD. Files already modified (git -C <DETECTORS> status --porcelain) stay as they are, neither committed nor reverted; list them in the summary and pass them to the upgrader. Always commit by path (git -C <DETECTORS> commit -m "<message>" -- <paths>), never git add -A, git stash or git checkout outside your paths: another session may be working there. Each message ends with an empty line then Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>. The tool-builder commits in TOOLS itself. Nothing is pushed. The rest (branch, committing OUT, another repository) is the caller's.

Steps

Each agent is called with the Agent tool, its subagent_type (agents:upgrader, agents:validator, agents:tool-builder, from the agents plugin loaded with --plugin-dir; not available: say so and stop, changing nothing) and the request below, plus the caller's context. A verdict is read by grep '^\*\*Verdict\*\*:' <output>.

Part 1: the detectors

  1. Measures before: step before.
  2. Upgrader, a single pass: Detectors: <DETECTORS>. Station: <STATION>, range <FROM> → <TO> (UTC). Labels: <LABELS>/<station>.jsonl. Families: <FAMILIES>. Context: <CONTEXT>. Events command: <…>. Sensors command: <…>. Range score command: <…> (before: <OUT>/range-before.txt). Benchmark command: <…> (before: <OUT>/benchmark-before.txt). Base: <base>. Files already modified: <list | none>. Output folder: <OUT>. Write <OUT>/modification.md. It commits each fix it keeps, message improve: S<n>, with its benchmark <OUT>/benchmark-S<n>.txt. Verdict nothing to change, or no fix commit (git -C <DETECTORS> log --format=%h --grep ': S[0-9]*$' <base>..HEAD empty): go to part 2.
  3. Measures after: step after.
  4. Validator, which reads neither the diagnosis nor the detail of the modification: Detectors: <DETECTORS>. Families: <FAMILIES>. Modification: <OUT>/modification.md. Range score: <OUT>/range-before.txt → <OUT>/range-after.txt. Benchmark: <OUT>/benchmark-before.txt → <OUT>/benchmark-after.txt, after each fix: <OUT>/benchmark-S*.txt; command: <…>. Events command for any station: python3 <DETECTORS>/scripts/categories.py <station folder> --from <from> --to <to>. Base: <base>. Write <OUT>/validation.md.
    • keep: go to part 2.
    • remove: for each fix of the **Remove**: line, its commits (git -C <DETECTORS> log --format=%h --grep ': S<n>$' <base>..HEAD), from the most recent: git -C <DETECTORS> revert --no-edit <commit>, then git -C <DETECTORS> commit --amend -m "improve: S<n> removed" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>". A conflict: git -C <DETECTORS> revert --abort, and do as for cancel.
    • cancel, or anything else: git -C <DETECTORS> revert --no-commit <base>..HEAD && git -C <DETECTORS> commit -m "improve: modification cancelled" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" (only what the fix commits changed; files already modified stay as they are).
  5. Measures end, only after a remove; otherwise end is after (keep) or before (cancel, nothing changed).

An output missing or without a readable verdict: note it, put the detectors back as for cancel if they changed, go to part 2. Do not rerun an agent.

Part 2: the tools

Always, whatever part 1 gave (a stop in part 1 does not stop part 2).

  1. Tool-builder: Analysis: <ANALYSIS>/. Tools: <TOOLS>/. Write <OUT>/tools.md. It reads the analysis, writes only in TOOLS and in <OUT>, and commits in TOOLS. Its output missing: note it. Do not rerun it.

Summary

<OUT>/summary.md:

# optimize: <station>, <FROM> → <TO>

**Result**: modification kept | kept without S<n>… | cancelled | none | error
**Tools changed**: <the "Tools changed" line of <OUT>/tools.md> | error

## The range
the range score, before → end, one line per family (found, correct)

## Elsewhere
the `Total` line of each family, before → end; then the other changed lines (station, family), before → end

## What changed
<per fix kept, one sentence from "What changed" of the modification>; the upgrader's verdict, the validator's

## For the user to decide
<copied as they are: "Limits" and "For the user to decide" of the modification, "Reservations" of the validator; none>

## Not fixed
<"Not fixed" of the modification; none>

## Reading tools
<copied as they are: "What I decided" and "What changed" of <OUT>/tools.md, and its "data.py" line if changed; none>

## Scripts of the agents
<copied as they are: "Tools" of the modification and of the validation (the scripts the agents kept in `<OUT>/tools/`); none>

## Errors
<none, or what was missing; files already modified; the base and `git -C <DETECTORS> log --oneline <base>..HEAD`; `git -C <TOOLS> log --oneline -1` if the tools changed>

Answer

In 5 lines at most: the **Result** line, the range score before → end (changed families), the Total before → end if it changed, the **Tools changed** line, the path of the summary.