Files
aimsight-optimizer/README.md
T

40 lines
3.2 KiB
Markdown
Raw Normal View History

# Optimizer
The optimisation of the AIMsight detectors and of the reading tools, from one analysis, and nothing else. An analysis (made in the aimsight-app app by the analyst) leaves in its folder the two things needed: its labels, the truth on its range, and its own scripts with what it missed. On that analysis, the `optimize` skill:
1. **the detectors**: measures their score on the range, and the benchmark on every label of `labeling/auto`; has them fixed by the `upgrader`, one fix per commit in `~/aimsight-detectors`; has them judged by the `validator`, which keeps, has some removed, or cancels: no station goes back, no overfitting;
2. **the tools**: has the `upgrader` bring the analyst's useful scripts into the reading harness, one fix per commit in `~/aimsight-tools`; always, whatever the detectors gave;
3. writes the summary of the session.
The `upgrader` and the `validator` come from the `agents` repository, each with its instructions from `instructions/` (`upgrader-detectors.md`, `validator-detectors.md`, `upgrader-tools.md`); the rules of each target are its own `CLAUDE.md`; the benchmark (`bench/`) from the `detectors` repository. Nothing is pushed.
## Install
```bash
ln -s ~/aimsight-detectors detectors # the detectors, shared with aimsight-app
ln -s ~/aimsight-tools tools # the reading tools, shared with aimsight-app
ln -s ~/aimsight-data/data data # the pumped data, out of git
ln -s ~/aimsight-labeling labeling # the labels, the analyses and the families
ln -s ~/aimsight-agents agents # the generic agents (Claude Code plugin)
```
`claude` (Claude Code) must be on the `PATH`, logged in to the Claude subscription (`/login`), never to an API key.
## Run
From the "Optimize" button of aimsight-app, on a finished analysis; or by hand, from the root of Optimizer:
```bash
env -u ANTHROPIC_API_KEY claude -p "/optimize ANALYSIS: labeling/auto/analyses/<analysis>/. OUT: sessions/<analysis>_<YYYYMMDDTHHMMZ>/." \
--plugin-dir agents --add-dir ~/aimsight-detectors --add-dir ~/aimsight-tools --add-dir ~/aimsight-labeling --add-dir ~/aimsight-data/data \
--setting-sources project,local --strict-mcp-config \
--allowedTools Read,Write,Edit,Bash,Agent --permission-mode acceptEdits
```
The station and the range come from the analysis's `request.json` (the range's last day included); the labels from `labeling/auto/<STATION>.jsonl`; the context from its `report.md`. `DETECTORS`, `TOOLS`, `DATA`, `LABELS`, `FAMILIES` can be overridden in the request.
## What a session leaves
`sessions/<analysis>_<time>/`: the measures before and after (`range-*.txt`, `benchmark-*.txt`), `modification.md` (the upgrader), `validation.md` (the validator), `tools.md` (the upgrader on the tools) and its trials (`work/`), `summary.md` (the summary, shown by the app under the analysis report), `tools/` (the scripts the upgrader and validator wrote). The fixes: `improve: S<n>` commits in `~/aimsight-detectors`, the tools `improve: S<n>` commits in `~/aimsight-tools`, each on its current branch.
`archive/`: the former daily batch and its scanner (auto-labeling by the detectors), stopped on 2026-11-16. The architecture of the repositories: `~/aimsight-app/docs/architecture.drawio`.