Files
aimsight-agents/docs/superpowers/specs/2026-11-16-generic-agents-design.md
T

9.5 KiB

Generic agents: design

Date: 2026-11-16. Status: approved by the user.

Intent

agents is the core of the AIMsight processes: one copy of each agent, used by every process of the gitea repositories. An agent knows its trade and the AIMsight domain (stations, labeling/, tools/, the families); it knows nothing of the process that launches it. What is specific to a process reaches it through the request and an INSTRUCTIONS file, as the analyst already does with Diagnostic-App's instructions/analyst.md.

Today only the analyst works this way. The upgrader and the validator are generic in name only: they hard-code the detectors (the six detector files, thresholds.md, the benchmark, the 2-point regression rule, the optimize workflow). The tool-builder is an upgrader whose target is tools/, written separately.

Goals

  • Three agents, one trade each:
    • analyst: reads a time range of a station like the expert and labels it (unchanged);
    • upgrader: improves a target (a git repository) from evidence, without overfitting, one commit per fix;
    • validator: judges an upgrader's change, independently, by the checks' outputs (scripts) and by reading the code (LLM-as-judge); keep, remove some fixes, or cancel.
  • The rules of a target are written once, in the target's own CLAUDE.md, read by the upgrader, the validator and the user alike.
  • What is specific to a process lives in the caller's INSTRUCTIONS file.
  • tool-builder disappears: Optimizer's part 2 becomes the upgrader on tools/ with its own instructions.

Non-goals

  • Support-AI (runs its agents through the API, out of scope by the user's decision).
  • Support-Agent and its agents (user's decision).
  • A writer agent (reports, mails, documents): the report stays asked of the analyst by Diagnostic-App's instructions until a second process needs a writer. (Claude's call.)
  • A validator on Optimizer's part 2 (the tools): none today, none added. (Claude's call.)
  • A validator judging something other than a git change (a labels file, a document): added when a process needs it.

Anti-goals

  • An agent that names a caller, a caller's path, its report format or its git rules.
  • A target rule written in two places (agent file and instructions, or upgrader and validator instructions).
  • A change of the detectors' or tools' behaviour: this is a move of text, the processes must behave as before.

Constraints

From CLAUDE.md of this repository and the users of the agents: every time UTC; no rule specific to a station, a target or a date; never read .env nor copy SIM_CCID, ip*; labeling/auto only; commit by path, never git add -A; English; the agents run on the Claude subscription (ANTHROPIC_API_KEY removed); the plugin is loaded read-only, nothing is written in agents/ by an agent. One copy of agents/ on the workstation: a change here is live for Optimizer and Diagnostic-App at once.

Approaches

  1. The rules in the target's repository (picked by the user). detectors/CLAUDE.md and tools/CLAUDE.md carry what may and may not change in them; the caller's instructions carry only the process. Each rule is written once; a new target must first write its rules.
  2. Everything in the caller's instructions. One file per agent and process; the detectors' rules would be duplicated between the upgrader's and the validator's instructions, and again for a second caller.
  3. Target profiles in agents/. Rejected: agents/ would know a specific repository, against this repository's CLAUDE.md.

The target's rules file is its existing CLAUDE.md rather than a new RULES.md: both already hold most of the rules. (Claude's call.)

Precedence

For the upgrader and the validator, from strongest to weakest:

  1. the target's CLAUDE.md: what is forbidden in the target; instructions may narrow it (fewer files to touch), never loosen it;
  2. the agent's core: overfitting, one fix one commit, git safety (upgrader); independence, single pass (validator);
  3. INSTRUCTIONS: everything else (why, scope inside the target, extra sections, how to answer); it prevails over the rest of the agent file.

The upgrader

Request: Target: <repository>. Evidence: <files or folder>. Checks: <command: what it measures; output before: <file>>, … Base: <commit>. Files already modified: <list | none>. Output folder: <folder>. Write <folder>/<name>.md. INSTRUCTIONS: <file>.

Trade (taken from today's upgrader and tool-builder, stripped of the detectors):

  • read INSTRUCTIONS in full first, then <Target>/CLAUDE.md in full, then the evidence;
  • diagnosis: what the target does wrong on the evidence, and which part of it causes it; most important first; a mechanism that comes back before an isolated case;
  • overfitting: the evidence is one sample; fix the mechanism, never fit the sample; what the target cannot see with its inputs goes to "Limits";
  • the simplest first: remove or simplify, change a constant or a default, adjust, add a short rule, add a new part as a last resort; a small gain is not worth a contraption; nothing to do is a decision;
  • checks: run each Checks command before and after each fix; output per fix in <Output>/check-S<n>.txt;
  • one fix one commit: git -C <Target> commit -m "improve: S<n>" -- <paths>; a fix not kept is reverted by git checkout -- <its files> only; never git add -A, stash, or touching a file listed as already modified;
  • its scripts in <Output>/tools/, never inline.

Output: the skeleton below; the instructions may add sections or lines.

# Modification

**Verdict**: changed | nothing to change
**Changed**: none | <files, comma-separated>

## Diagnosis
## What changed      (per fix S<n> and its commit; stands alone, no data of the evidence)
## Why               (stands alone)
## Checks            (before → after, the lines that changed)
## For the user to decide
## Not fixed
## Limits
## Tools
## Difficulties

The validator

Request: Target: <repository>. Modification: <file>. Base: <commit>. Checks: <command; outputs before, after, after each fix>. Output: <file>. INSTRUCTIONS: <file>.

Trade (today's validator, stripped of the detectors):

  • reads only: INSTRUCTIONS, <Target>/CLAUDE.md, "What changed" and "Why" of the modification, git diff <base> HEAD and the code it touches, the per-fix commits, the checks' outputs (may rerun them), and what the instructions add; never the diagnosis;
  • judges: the target still loads/runs (as its CLAUDE.md or the instructions define it); no regression (the definition comes from the target's CLAUDE.md); the gain justifies the code; no overfitting; the target's rules respected;
  • single pass, no sending back; do not cancel for perfection.

Output: **Verdict**: keep | remove | cancel, **Remove**: S<n>, … (with remove), then ## Answers, ## Checks, ## Reservations, ## Tools. The instructions may add sections.

The analyst

Unchanged. Its README line stays.

What moves where

Today in Moves to
upgrader.md: six detector files, detect() contract, episode keys, thresholds.md update, "everything is detected", stdlib detectors/CLAUDE.md (most already there; add "everything is detected" and the regression definition)
upgrader.md / validator.md: regression = "found" or "correct" loses > 2 points on a family of a station, and what is not a regression detectors/CLAUDE.md, under the benchmark rule
upgrader.md: labels format, range score, benchmark, --stations, section "Benchmark", "Before / after on the range" Optimizer/instructions/upgrader-detectors.md
validator.md: range score, events command to explain a regression, base archive trick Optimizer/instructions/validator-detectors.md
tool-builder.md: never remove a tool, option, output or guide sentence; a new tool enters the guide tools/CLAUDE.md (partly there)
tool-builder.md: read the analysis (report.md, analysis.md, scripts/), trial on the range, rerun the report's commands with --out, the data.py line, one commit Tools: <analysis> Optimizer/instructions/upgrader-tools.md (one commit per fix replaces the single commit)
optimize skill: request of each agent the generic requests above, with INSTRUCTIONS: instructions/<…>.md; part 2 calls agents:upgrader; the summary reads **Changed** instead of **Tools changed**

Then agents/tool-builder.md is deleted; README table, plugin.json description and the references in Optimizer and Diagnostic-App READMEs updated in the same change.

Order

The plugin is live for every caller: the Optimizer skill and its instructions land with the new agents, in one session, before anyone runs /optimize. detectors/ and tools/ rules first (additive), then Optimizer's instructions, then the agents and the skill together.

Checks

  • Each agent loads: claude -p "<without reading or running anything, list the fields of your request and the order of precedence>" --plugin-dir ~/agents --agent agents:<name> --model haiku.
  • Optimizer: env -u ANTHROPIC_API_KEY claude -p "/optimize" --plugin-dir agents --model haiku stops on missing inputs, touching nothing.
  • A real /optimize on an existing analysis: modification.md, validation.md, tools.md each have a readable verdict line, summary.md is filled, commits in detectors/ and tools/ follow improve: S<n>.
  • grep -nE "visibility.py|thresholds.md|benchmark|regression.py" agents/agents/*.md returns only the analyst's mentions, if any.

Left to the builder

The wording of each agent file; how the summary maps the new sections; the exact names of the instruction files.