Files
aimsight-agents/docs/superpowers/plans/2026-11-16-generic-agents.md
T
2026-11-16 14:40:29 +01:00

16 KiB

Generic agents Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: make upgrader and validator generic on a target repository, move each target's rules into its own CLAUDE.md and each process's specifics into Optimizer's instruction files, and fold tool-builder into the upgrader.

Architecture: the agents (~/agents/agents/*.md) keep only their trade; ~/detectors/CLAUDE.md and ~/tools/CLAUDE.md say what may change in them; ~/Optimizer/instructions/*.md say why and how /optimize uses each agent. The optimize skill passes INSTRUCTIONS: instructions/<file>.md in each request. Nothing in the behaviour of the detectors or the tools changes.

Tech Stack: Markdown prompts for Claude Code agents (plugin agents, --plugin-dir), the optimize skill, Python 3 checks of Diagnostic-App.

Spec: ~/agents/docs/superpowers/specs/2026-11-16-generic-agents-design.md

Global Constraints

  • Every time UTC; no rule specific to a station, a target or a date; never read .env nor copy SIM_CCID, ip*; labeling/auto only.
  • Commit by path, never git add -A; each message ends with an empty line then Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>; nothing is pushed.
  • English everywhere.
  • The agents run on the Claude subscription: every claude -p here is prefixed by env -u ANTHROPIC_API_KEY.
  • An agent file names no caller, no caller's path, no caller's report format (spec, anti-goals). A target rule is written in exactly one place.
  • Precedence for upgrader and validator: target's CLAUDE.md (may be narrowed, never loosened) > agent's core > INSTRUCTIONS > rest of the agent file.
  • ~/agents is live for Optimizer and Diagnostic-App at once: Task 4 lands agents and the optimize skill in the same session; nobody runs /optimize between its two commits.
  • The Optimizer summary keeps its **Result** and **Tools changed** lines as they are: Diagnostic-App renders them (app/server/analysis.py:305, app/server/check.py:184-187).

Review Focus

  • The tools upgrader gets Checks: none: it must run the trial its instructions describe, not stall nor invent a command (Task 4, step 6).
  • tools/CLAUDE.md "remove nothing" must beat the upgrader's generic "remove or simplify first" (Task 4, step 6).
  • Per-fix check outputs keep the name benchmark-S<n>.txt, which the validator request and the remove step glob (Task 4, step 2).
  • The validator extracts "What changed" and "Why" with a sed range that ends at ## Checks, the next heading of the new skeleton (Task 4, step 3).
  • TOOLS gets its own base and "files already modified", as DETECTORS has (Task 4, step 4).

Task 1: the detectors' rules in detectors/CLAUDE.md

Files:

  • Modify: ~/detectors/CLAUDE.md (section "Rules")

Interfaces:

  • Produces: the rules the upgrader and validator read as <Target>/CLAUDE.md for the detectors: what may change, the regression definition, "everything is detected".

  • Step 1: add to "Rules", taken word for word from ~/agents/agents/upgrader.md:39-58 and validator.md:147-153, keeping what is already there:

    • what a change to improve the detectors touches: scripts/*.py and thresholds.md; never bench/ (the measure);
    • everything is detected: what is shown to a client is not the detectors' business; a setting that hides something may be changed, saying so;
    • a fix needing a new family is out of reach (one detector per family is already in "Contract");
    • a constant or detector removed removes its thresholds.md line; the docstring follows a changed constant;
    • a detector that no longer loads shows a skipped line in categories.py and the benchmark: such a change is broken;
    • regression: on a family of a station, "found" or "correct" loses more than 2 points, even if Total rises; under 2 points is not one; not a regression: a drop from another consistent reading the labels do not retain (labels contradicting each other across stations, gaps of a few measurements between two labels of one phenomenon, label bounds, an instant counted as found by an event on another sensor); "found" and "correct" defined as in upgrader.md:63.
  • Step 2: check grep -cE "2 points|bench/|skipped|Everything is detected" ~/detectors/CLAUDE.md → 4 or more; git -C ~/detectors diff --stat shows only CLAUDE.md.

  • Step 3: commit git -C ~/detectors commit -m "CLAUDE.md: the rules the upgrader and validator read (what changes, regression, everything is detected)" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" -- CLAUDE.md

Task 2: the tools' rules in tools/CLAUDE.md

Files:

  • Modify: ~/tools/CLAUDE.md, ~/tools/README.md:23, ~/tools/GUIDE.md:3

Interfaces:

  • Produces: the rules the upgrader reads as <Target>/CLAUDE.md for the tools.

  • Step 1: add to "Rules", from ~/agents/agents/tool-builder.md:

    • remove nothing: no tool, option, output or sentence of the guide; a tool useless on one range may help on another;
    • the simplest first, for the tools: a sentence of the guide or a default, then an option of an existing tool, a new tool last;
    • a new script has a docstring at the top like the others and enters the table of GUIDE.md in the same change (merge with the existing sentence of line 3, do not duplicate);
    • the data.py rule already there stays, plus: a change to it is said at the top of the report.
  • Step 2: wording. README.md:23 and GUIDE.md:3: "the tool-builder" becomes "the upgrader (agents repository), run by Optimizer's optimize skill"; "it optimizes or adds, it removes nothing" stays.

  • Step 3: check grep -rn "tool-builder" ~/tools --include=*.md → nothing.

  • Step 4: commit by path the three files: "CLAUDE.md: the rules the upgrader reads; the tool-builder is the upgrader".

Task 3: Optimizer's instruction files

Files:

  • Create: ~/Optimizer/instructions/upgrader-detectors.md, ~/Optimizer/instructions/validator-detectors.md, ~/Optimizer/instructions/upgrader-tools.md

Interfaces:

  • Consumes: Task 1 and 2 rules (do not repeat them: point to <Target>/CLAUDE.md).

  • Produces: the three files Task 4 passes as INSTRUCTIONS:.

  • Step 1: upgrader-detectors.md, from upgrader.md: why (one analysis, its labels are the truth on its range, the validator judges afterwards on all stations); the evidence (labels format upgrader.md:21, labels touching the range, context documents, families document in full, scripts/categories.py); the comparison judgements (upgrader.md:31: correct, wrong family, extra, wrong bounds, wrong sensors, missing, alternative reading; varieties); the detectors' simplest-first step 5 (a new variety label last); the range score and benchmark use (upgrader.md:62-68, including --stations, background run, copy of DETECTORS for variants); scope: only scripts/*.py and thresholds.md; extra output sections: the ## Diagnosis table (upgrader.md:87), ## Before / after on the range and ## Benchmark placed after ## Checks; answer in 3 lines.

  • Step 2: validator-detectors.md, from validator.md: why (overfitting on one range of one station); the range score before → after is the gain sought; to explain a regression, the events command at HEAD and at base (git -C <Target> archive <base> | tar -x -C <folder>), the station's labels, the families document; extra output section ## The range; answer in 2 lines.

  • Step 3: upgrader-tools.md, from tool-builder.md: why (the next analysis, on any station, reads better); the evidence: <analysis>/report.md first, analysis.md ("The documentation", "Difficulties"), scripts/, GUIDE.md, the station folder; never the detectors; no check command: the trial is running each changed tool on the range with --out <output folder>/work/tools/ and looking at each PNG, then rerunning the report's commands that use a changed tool with --out <output folder>/work/ (replacing theirs, added if absent); extra lines after the verdict: **data.py**: unchanged | changed: <one sentence>; extra sections ## What the analyst used, ## What I decided, ## Report commands rerun; answer in 2 lines.

  • Step 4: check grep -c "CLAUDE.md" ~/Optimizer/instructions/*.md → 1 or more for each file (each points to the target's rules); grep -nE "2 points|one detector per family" ~/Optimizer/instructions/*.md → nothing (those rules live in detectors/CLAUDE.md only).

  • Step 5: commit in Optimizer by path: "instructions/: what /optimize asks of the generic upgrader and validator".

Task 4: generic agents and the optimize skill, together

Files:

  • Modify: ~/agents/agents/upgrader.md, ~/agents/agents/validator.md, ~/agents/README.md, ~/agents/.claude-plugin/plugin.json
  • Delete: ~/agents/agents/tool-builder.md
  • Modify: ~/Optimizer/.claude/skills/optimize/SKILL.md, ~/Optimizer/README.md, ~/Optimizer/CLAUDE.md:3,7

Interfaces:

  • Consumes: Task 3 files.

  • Produces:

    • upgrader request: Target: <repository>. Evidence: <files or folder>. Checks: <name>: <command> (before: <file>); … | none. Base: <commit>. Files already modified: <list | none>. Output folder: <folder>. Write <folder>/<name>.md. INSTRUCTIONS: <file>.
    • validator request: Target: <repository>. Modification: <file>. Base: <commit>. Checks: <name>: <command>; before: <file>; after: <file>; after each fix: <glob>. Output: <file>. INSTRUCTIONS: <file>.
    • verdict lines **Verdict**: changed | nothing to change + **Changed**: none | <files> (upgrader); **Verdict**: keep | remove | cancel + **Remove**: S<n>, … (validator).
  • Step 1: baseline. env -u ANTHROPIC_API_KEY claude -p "/optimize" --plugin-dir agents --model haiku from ~/Optimizer → says inputs missing, git -C ~/detectors status --porcelain and git -C ~/tools status --porcelain unchanged.

  • Step 2: upgrader.md. Frontmatter name: upgrader, generic description (improves a target repository from evidence, without overfitting, one commit per fix; request: Target, Evidence, Checks, Base, Output, INSTRUCTIONS), tools: Read, Edit, Write, Bash, model: inherit, omitClaudeMd: true. Body: the request; precedence (Global Constraints); reading order (INSTRUCTIONS, <Target>/CLAUDE.md in full, evidence); diagnosis; overfitting; simplest first; checks before and after each fix, output of each named check after fix n in <Output>/<name>-S<n>.txt; Checks: none → the instructions say how to try; one fix one commit (upgrader.md:72-77, made generic on <Target>); scripts in <Output>/tools/; the output skeleton of the spec; "What changed" and "Why" stand alone. No word detector, thresholds, benchmark, station, Optimizer.

  • Step 3: validator.md. Same frontmatter pattern, tools: Read, Bash, Write. Body from the spec's "The validator": reads only (with sed -n '/^## What changed/,/^## Checks/p' <modification>), judges the five points, single pass, keep / remove / cancel with the per-fix check outputs saying which fix regresses, output skeleton. Same forbidden words as step 2.

  • Step 4: SKILL.md. Line 44: agents:tool-builder removed. Step 2 request becomes the upgrader request with Target: <DETECTORS>, Evidence: <LABELS>/<station>.jsonl, <FAMILIES>, <CONTEXT>, station <STATION>, range <FROM> → <TO>, Checks: range: <range score> (before: <OUT>/range-before.txt); benchmark: <benchmark> (before: <OUT>/benchmark-before.txt); events: <events>; sensors: <sensors>, Write <OUT>/modification.md, INSTRUCTIONS: instructions/upgrader-detectors.md. Step 4 request becomes the validator request with INSTRUCTIONS: instructions/validator-detectors.md, Write <OUT>/validation.md, after each fix <OUT>/benchmark-S*.txt. Step 6: note TOOLS base and files already modified like DETECTORS; request Target: <TOOLS>. Evidence: <ANALYSIS>/. Checks: none. Base: <tools base>. Files already modified: <…>. Output folder: <OUT>. Write <OUT>/tools.md. INSTRUCTIONS: instructions/upgrader-tools.md.; verdict read the same way. Summary: **Tools changed**: <the **Changed** line of <OUT>/tools.md> | error (label kept); "Reading tools" copies "What I decided" and "What changed" and the **data.py** line if changed; Errors: git -C <TOOLS> log --oneline <tools base>..HEAD.

  • Step 5: docs. ~/agents/README.md table: three rows (analyst unchanged; upgrader: Target, Evidence, Checks, Base, Files already modified, Output, INSTRUCTIONS, called by Optimizer with instructions/upgrader-detectors.md and instructions/upgrader-tools.md; validator: Target, Modification, Base, Checks, Output, INSTRUCTIONS, called by Optimizer with instructions/validator-detectors.md), one sentence: a target's rules are its CLAUDE.md. plugin.json description: "analyst, upgrader, validator". Optimizer/README.md:6,9,38 and CLAUDE.md:3,7: tool-builder → upgrader on the tools; Tools: <analysis> commit → improve: S<n> commits in ~/tools.

  • Step 6: check.

    • grep -nE "visibility\.py|thresholds|benchmark|regression\.py|Optimizer|optimize|station" ~/agents/agents/upgrader.md ~/agents/agents/validator.md → nothing.
    • grep -rn "tool-builder" ~/agents ~/Optimizer --include=*.md --include=*.json | grep -v docs/superpowers | grep -v archive/ | grep -v sessions/ → nothing.
    • From ~/Optimizer: env -u ANTHROPIC_API_KEY claude -p "Without running anything, list the fields of your request and the order of precedence of your sources." --plugin-dir agents --agent agents:upgrader --model haiku → names Target, Evidence, Checks, Base, Output, INSTRUCTIONS and the precedence of Global Constraints; same for agents:validator.
    • From ~/Optimizer: env -u ANTHROPIC_API_KEY claude -p "Target: tools/. INSTRUCTIONS: instructions/upgrader-tools.md. Without changing anything: may you remove an option of a tool that did not help? With Checks: none, how do you check a fix?" --plugin-dir agents --agent agents:upgrader --model haiku --add-dir ~/tools → no (from tools/CLAUDE.md); the trial of the instructions.
    • Step 1 command again → same refusal, nothing touched.
  • Step 7: commit both repositories, back to back. ~/agents: git rm agents/tool-builder.md then git commit -m "Generic upgrader and validator on a target; tool-builder folded into the upgrader" -m "Co-Authored-By: …" -- agents/upgrader.md agents/validator.md agents/tool-builder.md README.md .claude-plugin/plugin.json. ~/Optimizer: -- .claude/skills/optimize/SKILL.md README.md CLAUDE.md, message "optimize: generic agents with instructions/; the tools by the upgrader".

Task 5: Diagnostic-App wording and a real session

Files:

  • Modify: ~/Diagnostic-App/README.md:23, ~/Diagnostic-App/app/server/station.py:26 (comment only)

  • Step 1: wording. "the tool-builder writes in it" → "the upgrader writes in it (Optimizer)".

  • Step 2: check cd ~/Diagnostic-App && python3 app/server/check.py → passes as before (the summary lines did not change).

  • Step 3: commit by path: "README: the tools are written by the upgrader".

  • Step 4: real session — ask the user first (it commits in ~/detectors and ~/tools). On their go, from the app's "Optimize" button, or by hand with the command of ~/Optimizer/README.md on labeling/auto/analyses/NORDEND_2026-11-05T0600_2026-11-05T1200/. Expected: modification.md, validation.md (if fixes), tools.md each with a readable **Verdict**: line; summary.md with **Result** and **Tools changed**; commits named improve: S<n> in each repository touched; the page shows the summary. Report the outcome to the user as it is, failures included.