diff --git a/docs/superpowers/plans/2026-11-16-generic-agents.md b/docs/superpowers/plans/2026-11-16-generic-agents.md new file mode 100644 index 0000000..745db84 --- /dev/null +++ b/docs/superpowers/plans/2026-11-16-generic-agents.md @@ -0,0 +1,119 @@ +# Generic agents Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** make `upgrader` and `validator` generic on a target repository, move each target's rules into its own `CLAUDE.md` and each process's specifics into Optimizer's instruction files, and fold `tool-builder` into the upgrader. + +**Architecture:** the agents (`~/agents/agents/*.md`) keep only their trade; `~/detectors/CLAUDE.md` and `~/tools/CLAUDE.md` say what may change in them; `~/Optimizer/instructions/*.md` say why and how `/optimize` uses each agent. The `optimize` skill passes `INSTRUCTIONS: instructions/.md` in each request. Nothing in the behaviour of the detectors or the tools changes. + +**Tech Stack:** Markdown prompts for Claude Code agents (plugin `agents`, `--plugin-dir`), the `optimize` skill, Python 3 checks of Diagnostic-App. + +**Spec:** `~/agents/docs/superpowers/specs/2026-11-16-generic-agents-design.md` + +## Global Constraints + +- Every time UTC; no rule specific to a station, a target or a date; never read `.env` nor copy `SIM_CCID`, `ip*`; `labeling/auto` only. +- Commit by path, never `git add -A`; each message ends with an empty line then `Co-Authored-By: Claude Opus 5.5 `; nothing is pushed. +- English everywhere. +- The agents run on the Claude subscription: every `claude -p` here is prefixed by `env -u ANTHROPIC_API_KEY`. +- An agent file names no caller, no caller's path, no caller's report format (spec, anti-goals). A target rule is written in exactly one place. +- Precedence for upgrader and validator: target's `CLAUDE.md` (may be narrowed, never loosened) > agent's core > `INSTRUCTIONS` > rest of the agent file. +- `~/agents` is live for Optimizer and Diagnostic-App at once: Task 4 lands agents and the `optimize` skill in the same session; nobody runs `/optimize` between its two commits. +- The Optimizer summary keeps its `**Result**` and `**Tools changed**` lines as they are: Diagnostic-App renders them (`app/server/analysis.py:305`, `app/server/check.py:184-187`). + +## Review Focus + +- The tools upgrader gets `Checks: none`: it must run the trial its instructions describe, not stall nor invent a command (Task 4, step 6). +- `tools/CLAUDE.md` "remove nothing" must beat the upgrader's generic "remove or simplify first" (Task 4, step 6). +- Per-fix check outputs keep the name `benchmark-S.txt`, which the validator request and the `remove` step glob (Task 4, step 2). +- The validator extracts "What changed" and "Why" with a `sed` range that ends at `## Checks`, the next heading of the new skeleton (Task 4, step 3). +- `TOOLS` gets its own base and "files already modified", as `DETECTORS` has (Task 4, step 4). + +--- + +### Task 1: the detectors' rules in `detectors/CLAUDE.md` + +**Files:** +- Modify: `~/detectors/CLAUDE.md` (section "Rules") + +**Interfaces:** +- Produces: the rules the upgrader and validator read as `/CLAUDE.md` for the detectors: what may change, the regression definition, "everything is detected". + +- [ ] **Step 1: add to "Rules"**, taken word for word from `~/agents/agents/upgrader.md:39-58` and `validator.md:147-153`, keeping what is already there: + - what a change to improve the detectors touches: `scripts/*.py` and `thresholds.md`; never `bench/` (the measure); + - everything is detected: what is shown to a client is not the detectors' business; a setting that hides something may be changed, saying so; + - a fix needing a new family is out of reach (one detector per family is already in "Contract"); + - a constant or detector removed removes its `thresholds.md` line; the docstring follows a changed constant; + - a detector that no longer loads shows a `skipped` line in `categories.py` and the benchmark: such a change is broken; + - regression: on a family of a station, "found" or "correct" loses more than 2 points, even if `Total` rises; under 2 points is not one; not a regression: a drop from another consistent reading the labels do not retain (labels contradicting each other across stations, gaps of a few measurements between two labels of one phenomenon, label bounds, an instant counted as found by an event on another sensor); "found" and "correct" defined as in `upgrader.md:63`. +- [ ] **Step 2: check** `grep -cE "2 points|bench/|skipped|Everything is detected" ~/detectors/CLAUDE.md` → 4 or more; `git -C ~/detectors diff --stat` shows only `CLAUDE.md`. +- [ ] **Step 3: commit** `git -C ~/detectors commit -m "CLAUDE.md: the rules the upgrader and validator read (what changes, regression, everything is detected)" -m "Co-Authored-By: Claude Opus 5.5 " -- CLAUDE.md` + +### Task 2: the tools' rules in `tools/CLAUDE.md` + +**Files:** +- Modify: `~/tools/CLAUDE.md`, `~/tools/README.md:23`, `~/tools/GUIDE.md:3` + +**Interfaces:** +- Produces: the rules the upgrader reads as `/CLAUDE.md` for the tools. + +- [ ] **Step 1: add to "Rules"**, from `~/agents/agents/tool-builder.md`: + - remove nothing: no tool, option, output or sentence of the guide; a tool useless on one range may help on another; + - the simplest first, for the tools: a sentence of the guide or a default, then an option of an existing tool, a new tool last; + - a new script has a docstring at the top like the others and enters the table of `GUIDE.md` in the same change (merge with the existing sentence of line 3, do not duplicate); + - the `data.py` rule already there stays, plus: a change to it is said at the top of the report. +- [ ] **Step 2: wording.** `README.md:23` and `GUIDE.md:3`: "the `tool-builder`" becomes "the `upgrader` (agents repository), run by Optimizer's `optimize` skill"; "it optimizes or adds, it removes nothing" stays. +- [ ] **Step 3: check** `grep -rn "tool-builder" ~/tools --include=*.md` → nothing. +- [ ] **Step 4: commit** by path the three files: `"CLAUDE.md: the rules the upgrader reads; the tool-builder is the upgrader"`. + +### Task 3: Optimizer's instruction files + +**Files:** +- Create: `~/Optimizer/instructions/upgrader-detectors.md`, `~/Optimizer/instructions/validator-detectors.md`, `~/Optimizer/instructions/upgrader-tools.md` + +**Interfaces:** +- Consumes: Task 1 and 2 rules (do not repeat them: point to `/CLAUDE.md`). +- Produces: the three files Task 4 passes as `INSTRUCTIONS:`. + +- [ ] **Step 1: `upgrader-detectors.md`**, from `upgrader.md`: why (one analysis, its labels are the truth on its range, the validator judges afterwards on all stations); the evidence (labels format `upgrader.md:21`, labels touching the range, context documents, families document in full, `scripts/categories.py`); the comparison judgements (`upgrader.md:31`: correct, wrong family, extra, wrong bounds, wrong sensors, missing, alternative reading; varieties); the detectors' simplest-first step 5 (a new variety `label` last); the range score and benchmark use (`upgrader.md:62-68`, including `--stations`, background run, copy of `DETECTORS` for variants); scope: only `scripts/*.py` and `thresholds.md`; extra output sections: the `## Diagnosis` table (`upgrader.md:87`), `## Before / after on the range` and `## Benchmark` placed after `## Checks`; answer in 3 lines. +- [ ] **Step 2: `validator-detectors.md`**, from `validator.md`: why (overfitting on one range of one station); the range score before → after is the gain sought; to explain a regression, the events command at HEAD and at base (`git -C archive | tar -x -C `), the station's labels, the families document; extra output section `## The range`; answer in 2 lines. +- [ ] **Step 3: `upgrader-tools.md`**, from `tool-builder.md`: why (the next analysis, on any station, reads better); the evidence: `/report.md` first, `analysis.md` ("The documentation", "Difficulties"), `scripts/`, `GUIDE.md`, the station folder; never the detectors; no check command: the trial is running each changed tool on the range with `--out /work/tools/` and looking at each PNG, then rerunning the report's commands that use a changed tool with `--out /work/` (replacing theirs, added if absent); extra lines after the verdict: `**data.py**: unchanged | changed: `; extra sections `## What the analyst used`, `## What I decided`, `## Report commands rerun`; answer in 2 lines. +- [ ] **Step 4: check** `grep -c "CLAUDE.md" ~/Optimizer/instructions/*.md` → 1 or more for each file (each points to the target's rules); `grep -nE "2 points|one detector per family" ~/Optimizer/instructions/*.md` → nothing (those rules live in `detectors/CLAUDE.md` only). +- [ ] **Step 5: commit** in Optimizer by path: `"instructions/: what /optimize asks of the generic upgrader and validator"`. + +### Task 4: generic agents and the `optimize` skill, together + +**Files:** +- Modify: `~/agents/agents/upgrader.md`, `~/agents/agents/validator.md`, `~/agents/README.md`, `~/agents/.claude-plugin/plugin.json` +- Delete: `~/agents/agents/tool-builder.md` +- Modify: `~/Optimizer/.claude/skills/optimize/SKILL.md`, `~/Optimizer/README.md`, `~/Optimizer/CLAUDE.md:3,7` + +**Interfaces:** +- Consumes: Task 3 files. +- Produces: + - upgrader request: `Target: . Evidence: . Checks: : (before: ); … | none. Base: . Files already modified: . Output folder: . Write /.md. INSTRUCTIONS: .` + - validator request: `Target: . Modification: . Base: . Checks: : ; before: ; after: ; after each fix: . Output: . INSTRUCTIONS: .` + - verdict lines `**Verdict**: changed | nothing to change` + `**Changed**: none | ` (upgrader); `**Verdict**: keep | remove | cancel` + `**Remove**: S, …` (validator). + +- [ ] **Step 1: baseline.** `env -u ANTHROPIC_API_KEY claude -p "/optimize" --plugin-dir agents --model haiku` from `~/Optimizer` → says inputs missing, `git -C ~/detectors status --porcelain` and `git -C ~/tools status --porcelain` unchanged. +- [ ] **Step 2: `upgrader.md`.** Frontmatter `name: upgrader`, generic description (improves a target repository from evidence, without overfitting, one commit per fix; request: Target, Evidence, Checks, Base, Output, INSTRUCTIONS), `tools: Read, Edit, Write, Bash`, `model: inherit`, `omitClaudeMd: true`. Body: the request; precedence (Global Constraints); reading order (INSTRUCTIONS, `/CLAUDE.md` in full, evidence); diagnosis; overfitting; simplest first; checks before and after each fix, output of each named check after fix n in `/-S.txt`; `Checks: none` → the instructions say how to try; one fix one commit (`upgrader.md:72-77`, made generic on ``); scripts in `/tools/`; the output skeleton of the spec; "What changed" and "Why" stand alone. No word `detector`, `thresholds`, `benchmark`, `station`, `Optimizer`. +- [ ] **Step 3: `validator.md`.** Same frontmatter pattern, `tools: Read, Bash, Write`. Body from the spec's "The validator": reads only (with `sed -n '/^## What changed/,/^## Checks/p' `), judges the five points, single pass, keep / remove / cancel with the per-fix check outputs saying which fix regresses, output skeleton. Same forbidden words as step 2. +- [ ] **Step 4: `SKILL.md`.** Line 44: `agents:tool-builder` removed. Step 2 request becomes the upgrader request with `Target: `, `Evidence: /.jsonl, , , station , range → `, `Checks: range: (before: /range-before.txt); benchmark: (before: /benchmark-before.txt); events: ; sensors: `, `Write /modification.md`, `INSTRUCTIONS: instructions/upgrader-detectors.md`. Step 4 request becomes the validator request with `INSTRUCTIONS: instructions/validator-detectors.md`, `Write /validation.md`, after each fix `/benchmark-S*.txt`. Step 6: note `TOOLS` base and files already modified like `DETECTORS`; request `Target: . Evidence: /. Checks: none. Base: . Files already modified: <…>. Output folder: . Write /tools.md. INSTRUCTIONS: instructions/upgrader-tools.md.`; verdict read the same way. Summary: `**Tools changed**: /tools.md> | error` (label kept); "Reading tools" copies "What I decided" and "What changed" and the `**data.py**` line if changed; Errors: `git -C log --oneline ..HEAD`. +- [ ] **Step 5: docs.** `~/agents/README.md` table: three rows (`analyst` unchanged; `upgrader`: Target, Evidence, Checks, Base, Files already modified, Output, `INSTRUCTIONS`, called by Optimizer with `instructions/upgrader-detectors.md` and `instructions/upgrader-tools.md`; `validator`: Target, Modification, Base, Checks, Output, `INSTRUCTIONS`, called by Optimizer with `instructions/validator-detectors.md`), one sentence: a target's rules are its `CLAUDE.md`. `plugin.json` description: "analyst, upgrader, validator". `Optimizer/README.md:6,9,38` and `CLAUDE.md:3,7`: tool-builder → upgrader on the tools; `Tools: ` commit → `improve: S` commits in `~/tools`. +- [ ] **Step 6: check.** + - `grep -nE "visibility\.py|thresholds|benchmark|regression\.py|Optimizer|optimize|station" ~/agents/agents/upgrader.md ~/agents/agents/validator.md` → nothing. + - `grep -rn "tool-builder" ~/agents ~/Optimizer --include=*.md --include=*.json | grep -v docs/superpowers | grep -v archive/ | grep -v sessions/` → nothing. + - From `~/Optimizer`: `env -u ANTHROPIC_API_KEY claude -p "Without running anything, list the fields of your request and the order of precedence of your sources." --plugin-dir agents --agent agents:upgrader --model haiku` → names Target, Evidence, Checks, Base, Output, INSTRUCTIONS and the precedence of Global Constraints; same for `agents:validator`. + - From `~/Optimizer`: `env -u ANTHROPIC_API_KEY claude -p "Target: tools/. INSTRUCTIONS: instructions/upgrader-tools.md. Without changing anything: may you remove an option of a tool that did not help? With Checks: none, how do you check a fix?" --plugin-dir agents --agent agents:upgrader --model haiku --add-dir ~/tools` → no (from `tools/CLAUDE.md`); the trial of the instructions. + - Step 1 command again → same refusal, nothing touched. +- [ ] **Step 7: commit both repositories**, back to back. `~/agents`: `git rm agents/tool-builder.md` then `git commit -m "Generic upgrader and validator on a target; tool-builder folded into the upgrader" -m "Co-Authored-By: …" -- agents/upgrader.md agents/validator.md agents/tool-builder.md README.md .claude-plugin/plugin.json`. `~/Optimizer`: `-- .claude/skills/optimize/SKILL.md README.md CLAUDE.md`, message `"optimize: generic agents with instructions/; the tools by the upgrader"`. + +### Task 5: Diagnostic-App wording and a real session + +**Files:** +- Modify: `~/Diagnostic-App/README.md:23`, `~/Diagnostic-App/app/server/station.py:26` (comment only) + +- [ ] **Step 1: wording.** "the tool-builder writes in it" → "the upgrader writes in it (Optimizer)". +- [ ] **Step 2: check** `cd ~/Diagnostic-App && python3 app/server/check.py` → passes as before (the summary lines did not change). +- [ ] **Step 3: commit** by path: `"README: the tools are written by the upgrader"`. +- [ ] **Step 4: real session — ask the user first** (it commits in `~/detectors` and `~/tools`). On their go, from the app's "Optimize" button, or by hand with the command of `~/Optimizer/README.md` on `labeling/auto/analyses/NORDEND_2026-11-05T0600_2026-11-05T1200/`. Expected: `modification.md`, `validation.md` (if fixes), `tools.md` each with a readable `**Verdict**:` line; `summary.md` with `**Result**` and `**Tools changed**`; commits named `improve: S` in each repository touched; the page shows the summary. Report the outcome to the user as it is, failures included.