Files
aimsight-optimizer/.claude/skills/improve/SKILL.md
T

7.8 KiB

name, description, argument-hint
name description argument-hint
improve Workflow that improves the detectors on a range of data where they perform poorly, given labels as the truth on that range — the upgrader fixes, the validator judges; the result, detectors a little better on the range without going back elsewhere (benchmark on a reference truth), or unchanged. The caller gives the range, the labels, the reference truth, the output folder and any specific context. Run from Optimizer's root with the agents plugin (--plugin-dir <agents>). /improve <request>. <range, labels, reference, output folder, context>

improve: improve the detectors on a range

You are the orchestrator. The detectors (the detectors repository, DETECTORS: scripts/, thresholds.md) perform poorly on a range of data of a station, or of several stations on the same range; labels give the truth on this range. You run the upgrader, which fixes the detectors, then the validator, which judges the fixes against overfitting, and you return detectors a little better on the range without going back elsewhere, or unchanged. You judge nothing yourself: you only read the verdict lines. You ask no question: the workflow runs alone to the end.

Inputs

Given by the caller, in the request; any other context or rule it gives (files to read, git rules, where to commit, names) applies to this run and is passed on to the agents as it is (Caller's context: …), before their instructions but not before their file scope.

  • DETECTORS: the path of the detectors repository (default: detectors/, a link in Optimizer).
  • STATION: the station's data folder (read by DETECTORS/scripts/data.py); or, several stations on the same range: DATA, the folder of the station folders, and STATIONS, A,B, the stations to improve.
  • FROM, TO: the range, UTC days (YYYY-MM-DD, TO excluded).
  • LABELS: the station's labels file (JSONL: id, start, end, family, description), the truth on the range; with STATIONS, a folder of <STATION>.jsonl, one per station.
  • REFERENCE: the reference truth for the non-regression, --labels <folder of <STATION>.jsonl> --data <folder of <STATION>/>.
  • FAMILIES: the document defining the families and their boundaries (one detector per family).
  • CONTEXT (optional): documents explaining the range's labels (an analyst's report, for example).
  • OUT: the output folder of this run (default: sessions/<analysis>_<YYYYMMDDTHHMMZ>/ in Optimizer).

A required input missing, or DETECTORS not a git repository: say so and stop, changing nothing.

Commands

  • events: python3 <DETECTORS>/scripts/categories.py <STATION> --from <FROM> --to <TO>
  • sensors: python3 <DETECTORS>/bench/sensors.py <STATION> --from <FROM> --to <TO>
  • range score (the goal): python3 <DETECTORS>/bench/score.py <STATION> --labels <LABELS> --from <FROM> --to <TO>; it exits with an error if no label touches the range: note it in the summary and stop without touching the detectors.
  • with STATIONS: the events and sensors commands once per station, <DATA>/<S> for each S of STATIONS; the range score, all of them at once: python3 <DETECTORS>/bench/regression.py --labels <LABELS> --data <DATA> --stations <STATIONS> --from <FROM> --to <TO>.
  • benchmark (the non-regression, about 3 min): python3 <DETECTORS>/bench/regression.py <REFERENCE>

The measures of a step: <range score> > <OUT>/range-<step>.txt 2>&1 and <benchmark> > <OUT>/benchmark-<step>.txt 2>&1.

Git

The fixes are commits in DETECTORS, one per fix, so that the validator can have one removed. Note the base: git -C <DETECTORS> rev-parse HEAD. Files already modified (git -C <DETECTORS> status --porcelain) stay as they are, neither committed nor reverted; list them in the summary and pass them to the upgrader. Always commit by path (git -C <DETECTORS> commit -m "<message>" -- <paths>), never git add -A, git stash or git checkout outside your paths: another session may be working there. Each message ends with an empty line then Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>. Nothing is pushed. The rest (branch, committing OUT, another repository) is the caller's.

Steps

Each agent is called with the Agent tool, its subagent_type (agents:upgrader, agents:validator, from the agents plugin loaded with --plugin-dir; not available: say so and stop, changing nothing) and the request below, plus the caller's context. A verdict is read by grep '^\*\*Verdict\*\*:' <output>.

  1. Measures before: step before.
  2. Upgrader, a single pass: Detectors: <DETECTORS>. Station: <STATION | <DATA>/<S> for each S of STATIONS>, range <FROM> → <TO> (UTC). Labels: <LABELS>. Families: <FAMILIES>. Context: <CONTEXT | none>. Events command: <…>. Sensors command: <…>. Range score command: <…> (before: <OUT>/range-before.txt). Benchmark command: <…> (before: <OUT>/benchmark-before.txt). Base: <base>. Files already modified: <list | none>. Output folder: <OUT>. Write <OUT>/modification.md. It commits each fix it keeps, message improve: S<n>, with its benchmark <OUT>/benchmark-S<n>.txt. Verdict nothing to change, or no fix commit (git -C <DETECTORS> log --format=%h --grep ': S[0-9]*$' <base>..HEAD empty): go to the summary.
  3. Measures after: step after.
  4. Validator, which reads neither the diagnosis nor the detail of the modification: Detectors: <DETECTORS>. Families: <FAMILIES>. Modification: <OUT>/modification.md. Range score: <OUT>/range-before.txt → <OUT>/range-after.txt. Benchmark: <OUT>/benchmark-before.txt → <OUT>/benchmark-after.txt, after each fix: <OUT>/benchmark-S*.txt; command: <…>. Events command for any station: python3 <DETECTORS>/scripts/categories.py <station folder> --from <from> --to <to>. Base: <base>. Write <OUT>/validation.md.
    • keep: go to the summary.
    • remove: for each fix of the **Remove**: line, its commits (git -C <DETECTORS> log --format=%h --grep ': S<n>$' <base>..HEAD), from the most recent: git -C <DETECTORS> revert --no-edit <commit>, then git -C <DETECTORS> commit --amend -m "improve: S<n> removed" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>". A conflict: git -C <DETECTORS> revert --abort, and do as for cancel.
    • cancel, or anything else: git -C <DETECTORS> revert --no-commit <base>..HEAD && git -C <DETECTORS> commit -m "improve: modification cancelled" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" (only what the fix commits changed; files already modified stay as they are).

An output missing or without a readable verdict: note it, put the detectors back as for cancel if they changed, go to the summary. Do not rerun an agent.

Summary

Measures, step end, only after a remove; otherwise end is after (keep) or before (cancel, nothing changed). Then <OUT>/summary.md:

# improve: <station | STATIONS>, <FROM> → <TO>

**Result**: modification kept | kept without S<n>… | cancelled | none | error

## The range
the range score, before → end, one line per family (found, correct)

## Elsewhere
the `Total` line of each family, before → end; then the other changed lines (station, family), before → end

## What changed
<per fix kept, one sentence from "What changed" of the modification>; the upgrader's verdict, the validator's

## For the user to decide
<copied as they are: "Limits" and "For the user to decide" of the modification, "Reservations" of the validator; none>

## Not fixed
<"Not fixed" of the modification; none>

## Tools
<copied as they are: "Tools" of the modification and of the validation (the scripts the agents kept in `<OUT>/tools/`); none>

## Errors
<none, or what was missing; files already modified; the base and `git -C <DETECTORS> log --oneline <base>..HEAD`>

Answer

In 4 lines at most: the **Result** line, the range score before → end (changed families), the Total before → end if it changed, the path of the summary.