43 lines
2.6 KiB
Markdown
43 lines
2.6 KiB
Markdown
# Optimizer
|
|
|
|
The optimisation of the AIMsight detectors, on the labels of `labeling/auto`, and nothing else. On the range of an in-depth analysis (made in the Diagnostic-App app), its labels as the truth, the `improve` skill:
|
|
|
|
1. measures the detectors' score on the range, and the benchmark on every label of `labeling/auto`;
|
|
2. has the detectors fixed by the `upgrader`, one fix per commit in `~/detectors`;
|
|
3. has them judged by the `validator`, which keeps, has some removed, or cancels: no station goes back, no overfitting;
|
|
4. writes the summary of the session.
|
|
|
|
The `upgrader` and the `validator` come from the `agents` repository; the benchmark (`bench/`) from the `detectors` repository. Nothing is pushed.
|
|
|
|
## Install
|
|
|
|
```bash
|
|
ln -s ~/detectors detectors # the detectors, shared with Diagnostic-App
|
|
ln -s ~/aimsight-data/data data # the pumped data, out of git
|
|
ln -s ~/labeling labeling # the labels: the truth and the families
|
|
ln -s ~/agents agents # the generic agents (Claude Code plugin)
|
|
```
|
|
|
|
`claude` (Claude Code) must be on the `PATH`, logged in to the Claude subscription (`/login`), never to an API key.
|
|
|
|
## Run
|
|
|
|
From the "Optimize the detectors" button of Diagnostic-App, on a finished analysis; or by hand, from the root of Optimizer:
|
|
|
|
```bash
|
|
env -u ANTHROPIC_API_KEY claude -p "/improve DETECTORS: detectors/. STATION: data/NORDEND. FROM: 2026-11-04. TO: 2026-11-08. \
|
|
LABELS: labeling/auto/NORDEND.jsonl. REFERENCE: --labels labeling/auto --data data. FAMILIES: labeling/families.md. \
|
|
CONTEXT: labeling/auto/analyses/<analysis>/report.md. OUT: sessions/<analysis>_<YYYYMMDDTHHMMZ>/." \
|
|
--plugin-dir agents --add-dir ~/detectors --add-dir ~/labeling --add-dir ~/aimsight-data/data \
|
|
--setting-sources project,local --strict-mcp-config \
|
|
--allowedTools Read,Write,Edit,Bash,Agent --permission-mode acceptEdits
|
|
```
|
|
|
|
`TO` is excluded. Several stations on the same range: `DATA: data. STATIONS: A,B. LABELS: <folder of <STATION>.jsonl>` instead of `STATION` and a labels file.
|
|
|
|
## What a session leaves
|
|
|
|
`sessions/<analysis>_<time>/`: the measures before and after (`range-*.txt`, `benchmark-*.txt`), `modification.md` (the upgrader), `validation.md` (the validator), `summary.md` (the summary, shown by the app under the analysis report), `tools/` (the scripts the agents wrote). The fixes: `improve: S<n>` commits in `~/detectors`, on its current branch.
|
|
|
|
`archive/`: the former daily batch and its scanner (auto-labeling by the detectors), stopped on 2026-11-16. The architecture of the repositories: `~/Diagnostic-App/docs/architecture.drawio`.
|