9.5 KiB
Generic agents: design
Date: 2026-11-16. Status: approved by the user.
Intent
agents is the core of the AIMsight processes: one copy of each agent, used by every process of the gitea repositories. An agent knows its trade and the AIMsight domain (stations, labeling/, tools/, the families); it knows nothing of the process that launches it. What is specific to a process reaches it through the request and an INSTRUCTIONS file, as the analyst already does with Diagnostic-App's instructions/analyst.md.
Today only the analyst works this way. The upgrader and the validator are generic in name only: they hard-code the detectors (the six detector files, thresholds.md, the benchmark, the 2-point regression rule, the optimize workflow). The tool-builder is an upgrader whose target is tools/, written separately.
Goals
- Three agents, one trade each:
analyst: reads a time range of a station like the expert and labels it (unchanged);upgrader: improves a target (a git repository) from evidence, without overfitting, one commit per fix;validator: judges an upgrader's change, independently, by the checks' outputs (scripts) and by reading the code (LLM-as-judge); keep, remove some fixes, or cancel.
- The rules of a target are written once, in the target's own
CLAUDE.md, read by the upgrader, the validator and the user alike. - What is specific to a process lives in the caller's
INSTRUCTIONSfile. tool-builderdisappears: Optimizer's part 2 becomes the upgrader ontools/with its own instructions.
Non-goals
- Support-AI (runs its agents through the API, out of scope by the user's decision).
- Support-Agent and its agents (user's decision).
- A
writeragent (reports, mails, documents): the report stays asked of the analyst by Diagnostic-App's instructions until a second process needs a writer. (Claude's call.) - A validator on Optimizer's part 2 (the tools): none today, none added. (Claude's call.)
- A validator judging something other than a git change (a labels file, a document): added when a process needs it.
Anti-goals
- An agent that names a caller, a caller's path, its report format or its git rules.
- A target rule written in two places (agent file and instructions, or upgrader and validator instructions).
- A change of the detectors' or tools' behaviour: this is a move of text, the processes must behave as before.
Constraints
From CLAUDE.md of this repository and the users of the agents: every time UTC; no rule specific to a station, a target or a date; never read .env nor copy SIM_CCID, ip*; labeling/auto only; commit by path, never git add -A; English; the agents run on the Claude subscription (ANTHROPIC_API_KEY removed); the plugin is loaded read-only, nothing is written in agents/ by an agent. One copy of agents/ on the workstation: a change here is live for Optimizer and Diagnostic-App at once.
Approaches
- The rules in the target's repository (picked by the user).
detectors/CLAUDE.mdandtools/CLAUDE.mdcarry what may and may not change in them; the caller's instructions carry only the process. Each rule is written once; a new target must first write its rules. - Everything in the caller's instructions. One file per agent and process; the detectors' rules would be duplicated between the upgrader's and the validator's instructions, and again for a second caller.
- Target profiles in
agents/. Rejected:agents/would know a specific repository, against this repository'sCLAUDE.md.
The target's rules file is its existing CLAUDE.md rather than a new RULES.md: both already hold most of the rules. (Claude's call.)
Precedence
For the upgrader and the validator, from strongest to weakest:
- the target's
CLAUDE.md: what is forbidden in the target; instructions may narrow it (fewer files to touch), never loosen it; - the agent's core: overfitting, one fix one commit, git safety (upgrader); independence, single pass (validator);
INSTRUCTIONS: everything else (why, scope inside the target, extra sections, how to answer); it prevails over the rest of the agent file.
The upgrader
Request: Target: <repository>. Evidence: <files or folder>. Checks: <command: what it measures; output before: <file>>, … Base: <commit>. Files already modified: <list | none>. Output folder: <folder>. Write <folder>/<name>.md. INSTRUCTIONS: <file>.
Trade (taken from today's upgrader and tool-builder, stripped of the detectors):
- read
INSTRUCTIONSin full first, then<Target>/CLAUDE.mdin full, then the evidence; - diagnosis: what the target does wrong on the evidence, and which part of it causes it; most important first; a mechanism that comes back before an isolated case;
- overfitting: the evidence is one sample; fix the mechanism, never fit the sample; what the target cannot see with its inputs goes to "Limits";
- the simplest first: remove or simplify, change a constant or a default, adjust, add a short rule, add a new part as a last resort; a small gain is not worth a contraption; nothing to do is a decision;
- checks: run each
Checkscommand before and after each fix; output per fix in<Output>/check-S<n>.txt; - one fix one commit:
git -C <Target> commit -m "improve: S<n>" -- <paths>; a fix not kept is reverted bygit checkout -- <its files>only; nevergit add -A,stash, or touching a file listed as already modified; - its scripts in
<Output>/tools/, never inline.
Output: the skeleton below; the instructions may add sections or lines.
# Modification
**Verdict**: changed | nothing to change
**Changed**: none | <files, comma-separated>
## Diagnosis
## What changed (per fix S<n> and its commit; stands alone, no data of the evidence)
## Why (stands alone)
## Checks (before → after, the lines that changed)
## For the user to decide
## Not fixed
## Limits
## Tools
## Difficulties
The validator
Request: Target: <repository>. Modification: <file>. Base: <commit>. Checks: <command; outputs before, after, after each fix>. Output: <file>. INSTRUCTIONS: <file>.
Trade (today's validator, stripped of the detectors):
- reads only:
INSTRUCTIONS,<Target>/CLAUDE.md, "What changed" and "Why" of the modification,git diff <base> HEADand the code it touches, the per-fix commits, the checks' outputs (may rerun them), and what the instructions add; never the diagnosis; - judges: the target still loads/runs (as its
CLAUDE.mdor the instructions define it); no regression (the definition comes from the target'sCLAUDE.md); the gain justifies the code; no overfitting; the target's rules respected; - single pass, no sending back; do not cancel for perfection.
Output: **Verdict**: keep | remove | cancel, **Remove**: S<n>, … (with remove), then ## Answers, ## Checks, ## Reservations, ## Tools. The instructions may add sections.
The analyst
Unchanged. Its README line stays.
What moves where
| Today in | Moves to |
|---|---|
upgrader.md: six detector files, detect() contract, episode keys, thresholds.md update, "everything is detected", stdlib |
detectors/CLAUDE.md (most already there; add "everything is detected" and the regression definition) |
upgrader.md / validator.md: regression = "found" or "correct" loses > 2 points on a family of a station, and what is not a regression |
detectors/CLAUDE.md, under the benchmark rule |
upgrader.md: labels format, range score, benchmark, --stations, section "Benchmark", "Before / after on the range" |
Optimizer/instructions/upgrader-detectors.md |
validator.md: range score, events command to explain a regression, base archive trick |
Optimizer/instructions/validator-detectors.md |
tool-builder.md: never remove a tool, option, output or guide sentence; a new tool enters the guide |
tools/CLAUDE.md (partly there) |
tool-builder.md: read the analysis (report.md, analysis.md, scripts/), trial on the range, rerun the report's commands with --out, the data.py line, one commit Tools: <analysis> |
Optimizer/instructions/upgrader-tools.md (one commit per fix replaces the single commit) |
optimize skill: request of each agent |
the generic requests above, with INSTRUCTIONS: instructions/<…>.md; part 2 calls agents:upgrader; the summary reads **Changed** instead of **Tools changed** |
Then agents/tool-builder.md is deleted; README table, plugin.json description and the references in Optimizer and Diagnostic-App READMEs updated in the same change.
Order
The plugin is live for every caller: the Optimizer skill and its instructions land with the new agents, in one session, before anyone runs /optimize. detectors/ and tools/ rules first (additive), then Optimizer's instructions, then the agents and the skill together.
Checks
- Each agent loads:
claude -p "<without reading or running anything, list the fields of your request and the order of precedence>" --plugin-dir ~/agents --agent agents:<name> --model haiku. - Optimizer:
env -u ANTHROPIC_API_KEY claude -p "/optimize" --plugin-dir agents --model haikustops on missing inputs, touching nothing. - A real
/optimizeon an existing analysis:modification.md,validation.md,tools.mdeach have a readable verdict line,summary.mdis filled, commits indetectors/andtools/followimprove: S<n>. grep -nE "visibility.py|thresholds.md|benchmark|regression.py" agents/agents/*.mdreturns only the analyst's mentions, if any.
Left to the builder
The wording of each agent file; how the summary maps the new sections; the exact names of the instruction files.