Optimizer

The optimisation of the AIMsight detectors, on the labels of labeling/auto, and nothing else. On the range of an in-depth analysis (made in the Diagnostic-App app), its labels as the truth, the improve skill:

  1. measures the detectors' score on the range, and the benchmark on every label of labeling/auto;
  2. has the detectors fixed by the upgrader, one fix per commit in ~/detectors;
  3. has them judged by the validator, which keeps, has some removed, or cancels: no station goes back, no overfitting;
  4. writes the summary of the session.

The upgrader and the validator come from the agents repository; the benchmark (bench/) from the detectors repository. Nothing is pushed.

Install

ln -s ~/detectors detectors                  # the detectors, shared with Diagnostic-App
ln -s ~/aimsight-data/data data             # the pumped data, out of git
ln -s ~/labeling labeling                    # the labels: the truth and the families
ln -s ~/agents agents                        # the generic agents (Claude Code plugin)

claude (Claude Code) must be on the PATH, logged in to the Claude subscription (/login), never to an API key.

Run

From the "Optimize the detectors" button of Diagnostic-App, on a finished analysis; or by hand, from the root of Optimizer:

env -u ANTHROPIC_API_KEY claude -p "/improve DETECTORS: detectors/. STATION: data/NORDEND. FROM: 2026-11-04. TO: 2026-11-08. \
LABELS: labeling/auto/NORDEND.jsonl. REFERENCE: --labels labeling/auto --data data. FAMILIES: labeling/families.md. \
CONTEXT: labeling/auto/analyses/<analysis>/report.md. OUT: sessions/<analysis>_<YYYYMMDDTHHMMZ>/." \
  --plugin-dir agents --add-dir ~/detectors --add-dir ~/labeling --add-dir ~/aimsight-data/data \
  --setting-sources project,local --strict-mcp-config \
  --allowedTools Read,Write,Edit,Bash,Agent --permission-mode acceptEdits

TO is excluded. Several stations on the same range: DATA: data. STATIONS: A,B. LABELS: <folder of <STATION>.jsonl> instead of STATION and a labels file.

What a session leaves

sessions/<analysis>_<time>/: the measures before and after (range-*.txt, benchmark-*.txt), modification.md (the upgrader), validation.md (the validator), summary.md (the summary, shown by the app under the analysis report), tools/ (the scripts the agents wrote). The fixes: improve: S<n> commits in ~/detectors, on its current branch.

archive/: the former daily batch and its scanner (auto-labeling by the detectors), stopped on 2026-11-16. The architecture of the repositories: ~/Diagnostic-App/docs/architecture.drawio.

S
Description
Optimizing the detectors
Readme
196 KiB
0 Stars 1 Watchers 0 Forks
Languages
Python 100%