Plan: generic agents
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
791ac80125
commit
0df04b58ae
1 file changed
+119
@@ -0,0 +1,119 @@
|
||||
# Generic agents Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** make `upgrader` and `validator` generic on a target repository, move each target's rules into its own `CLAUDE.md` and each process's specifics into Optimizer's instruction files, and fold `tool-builder` into the upgrader.
|
||||
|
||||
**Architecture:** the agents (`~/agents/agents/*.md`) keep only their trade; `~/detectors/CLAUDE.md` and `~/tools/CLAUDE.md` say what may change in them; `~/Optimizer/instructions/*.md` say why and how `/optimize` uses each agent. The `optimize` skill passes `INSTRUCTIONS: instructions/<file>.md` in each request. Nothing in the behaviour of the detectors or the tools changes.
|
||||
|
||||
**Tech Stack:** Markdown prompts for Claude Code agents (plugin `agents`, `--plugin-dir`), the `optimize` skill, Python 3 checks of Diagnostic-App.
|
||||
|
||||
**Spec:** `~/agents/docs/superpowers/specs/2026-11-16-generic-agents-design.md`
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Every time UTC; no rule specific to a station, a target or a date; never read `.env` nor copy `SIM_CCID`, `ip*`; `labeling/auto` only.
|
||||
- Commit by path, never `git add -A`; each message ends with an empty line then `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`; nothing is pushed.
|
||||
- English everywhere.
|
||||
- The agents run on the Claude subscription: every `claude -p` here is prefixed by `env -u ANTHROPIC_API_KEY`.
|
||||
- An agent file names no caller, no caller's path, no caller's report format (spec, anti-goals). A target rule is written in exactly one place.
|
||||
- Precedence for upgrader and validator: target's `CLAUDE.md` (may be narrowed, never loosened) > agent's core > `INSTRUCTIONS` > rest of the agent file.
|
||||
- `~/agents` is live for Optimizer and Diagnostic-App at once: Task 4 lands agents and the `optimize` skill in the same session; nobody runs `/optimize` between its two commits.
|
||||
- The Optimizer summary keeps its `**Result**` and `**Tools changed**` lines as they are: Diagnostic-App renders them (`app/server/analysis.py:305`, `app/server/check.py:184-187`).
|
||||
|
||||
## Review Focus
|
||||
|
||||
- The tools upgrader gets `Checks: none`: it must run the trial its instructions describe, not stall nor invent a command (Task 4, step 6).
|
||||
- `tools/CLAUDE.md` "remove nothing" must beat the upgrader's generic "remove or simplify first" (Task 4, step 6).
|
||||
- Per-fix check outputs keep the name `benchmark-S<n>.txt`, which the validator request and the `remove` step glob (Task 4, step 2).
|
||||
- The validator extracts "What changed" and "Why" with a `sed` range that ends at `## Checks`, the next heading of the new skeleton (Task 4, step 3).
|
||||
- `TOOLS` gets its own base and "files already modified", as `DETECTORS` has (Task 4, step 4).
|
||||
|
||||
---
|
||||
|
||||
### Task 1: the detectors' rules in `detectors/CLAUDE.md`
|
||||
|
||||
**Files:**
|
||||
- Modify: `~/detectors/CLAUDE.md` (section "Rules")
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: the rules the upgrader and validator read as `<Target>/CLAUDE.md` for the detectors: what may change, the regression definition, "everything is detected".
|
||||
|
||||
- [ ] **Step 1: add to "Rules"**, taken word for word from `~/agents/agents/upgrader.md:39-58` and `validator.md:147-153`, keeping what is already there:
|
||||
- what a change to improve the detectors touches: `scripts/*.py` and `thresholds.md`; never `bench/` (the measure);
|
||||
- everything is detected: what is shown to a client is not the detectors' business; a setting that hides something may be changed, saying so;
|
||||
- a fix needing a new family is out of reach (one detector per family is already in "Contract");
|
||||
- a constant or detector removed removes its `thresholds.md` line; the docstring follows a changed constant;
|
||||
- a detector that no longer loads shows a `skipped` line in `categories.py` and the benchmark: such a change is broken;
|
||||
- regression: on a family of a station, "found" or "correct" loses more than 2 points, even if `Total` rises; under 2 points is not one; not a regression: a drop from another consistent reading the labels do not retain (labels contradicting each other across stations, gaps of a few measurements between two labels of one phenomenon, label bounds, an instant counted as found by an event on another sensor); "found" and "correct" defined as in `upgrader.md:63`.
|
||||
- [ ] **Step 2: check** `grep -cE "2 points|bench/|skipped|Everything is detected" ~/detectors/CLAUDE.md` → 4 or more; `git -C ~/detectors diff --stat` shows only `CLAUDE.md`.
|
||||
- [ ] **Step 3: commit** `git -C ~/detectors commit -m "CLAUDE.md: the rules the upgrader and validator read (what changes, regression, everything is detected)" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" -- CLAUDE.md`
|
||||
|
||||
### Task 2: the tools' rules in `tools/CLAUDE.md`
|
||||
|
||||
**Files:**
|
||||
- Modify: `~/tools/CLAUDE.md`, `~/tools/README.md:23`, `~/tools/GUIDE.md:3`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: the rules the upgrader reads as `<Target>/CLAUDE.md` for the tools.
|
||||
|
||||
- [ ] **Step 1: add to "Rules"**, from `~/agents/agents/tool-builder.md`:
|
||||
- remove nothing: no tool, option, output or sentence of the guide; a tool useless on one range may help on another;
|
||||
- the simplest first, for the tools: a sentence of the guide or a default, then an option of an existing tool, a new tool last;
|
||||
- a new script has a docstring at the top like the others and enters the table of `GUIDE.md` in the same change (merge with the existing sentence of line 3, do not duplicate);
|
||||
- the `data.py` rule already there stays, plus: a change to it is said at the top of the report.
|
||||
- [ ] **Step 2: wording.** `README.md:23` and `GUIDE.md:3`: "the `tool-builder`" becomes "the `upgrader` (agents repository), run by Optimizer's `optimize` skill"; "it optimizes or adds, it removes nothing" stays.
|
||||
- [ ] **Step 3: check** `grep -rn "tool-builder" ~/tools --include=*.md` → nothing.
|
||||
- [ ] **Step 4: commit** by path the three files: `"CLAUDE.md: the rules the upgrader reads; the tool-builder is the upgrader"`.
|
||||
|
||||
### Task 3: Optimizer's instruction files
|
||||
|
||||
**Files:**
|
||||
- Create: `~/Optimizer/instructions/upgrader-detectors.md`, `~/Optimizer/instructions/validator-detectors.md`, `~/Optimizer/instructions/upgrader-tools.md`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Task 1 and 2 rules (do not repeat them: point to `<Target>/CLAUDE.md`).
|
||||
- Produces: the three files Task 4 passes as `INSTRUCTIONS:`.
|
||||
|
||||
- [ ] **Step 1: `upgrader-detectors.md`**, from `upgrader.md`: why (one analysis, its labels are the truth on its range, the validator judges afterwards on all stations); the evidence (labels format `upgrader.md:21`, labels touching the range, context documents, families document in full, `scripts/categories.py`); the comparison judgements (`upgrader.md:31`: correct, wrong family, extra, wrong bounds, wrong sensors, missing, alternative reading; varieties); the detectors' simplest-first step 5 (a new variety `label` last); the range score and benchmark use (`upgrader.md:62-68`, including `--stations`, background run, copy of `DETECTORS` for variants); scope: only `scripts/*.py` and `thresholds.md`; extra output sections: the `## Diagnosis` table (`upgrader.md:87`), `## Before / after on the range` and `## Benchmark` placed after `## Checks`; answer in 3 lines.
|
||||
- [ ] **Step 2: `validator-detectors.md`**, from `validator.md`: why (overfitting on one range of one station); the range score before → after is the gain sought; to explain a regression, the events command at HEAD and at base (`git -C <Target> archive <base> | tar -x -C <folder>`), the station's labels, the families document; extra output section `## The range`; answer in 2 lines.
|
||||
- [ ] **Step 3: `upgrader-tools.md`**, from `tool-builder.md`: why (the next analysis, on any station, reads better); the evidence: `<analysis>/report.md` first, `analysis.md` ("The documentation", "Difficulties"), `scripts/`, `GUIDE.md`, the station folder; never the detectors; no check command: the trial is running each changed tool on the range with `--out <output folder>/work/tools/` and looking at each PNG, then rerunning the report's commands that use a changed tool with `--out <output folder>/work/` (replacing theirs, added if absent); extra lines after the verdict: `**data.py**: unchanged | changed: <one sentence>`; extra sections `## What the analyst used`, `## What I decided`, `## Report commands rerun`; answer in 2 lines.
|
||||
- [ ] **Step 4: check** `grep -c "CLAUDE.md" ~/Optimizer/instructions/*.md` → 1 or more for each file (each points to the target's rules); `grep -nE "2 points|one detector per family" ~/Optimizer/instructions/*.md` → nothing (those rules live in `detectors/CLAUDE.md` only).
|
||||
- [ ] **Step 5: commit** in Optimizer by path: `"instructions/: what /optimize asks of the generic upgrader and validator"`.
|
||||
|
||||
### Task 4: generic agents and the `optimize` skill, together
|
||||
|
||||
**Files:**
|
||||
- Modify: `~/agents/agents/upgrader.md`, `~/agents/agents/validator.md`, `~/agents/README.md`, `~/agents/.claude-plugin/plugin.json`
|
||||
- Delete: `~/agents/agents/tool-builder.md`
|
||||
- Modify: `~/Optimizer/.claude/skills/optimize/SKILL.md`, `~/Optimizer/README.md`, `~/Optimizer/CLAUDE.md:3,7`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Task 3 files.
|
||||
- Produces:
|
||||
- upgrader request: `Target: <repository>. Evidence: <files or folder>. Checks: <name>: <command> (before: <file>); … | none. Base: <commit>. Files already modified: <list | none>. Output folder: <folder>. Write <folder>/<name>.md. INSTRUCTIONS: <file>.`
|
||||
- validator request: `Target: <repository>. Modification: <file>. Base: <commit>. Checks: <name>: <command>; before: <file>; after: <file>; after each fix: <glob>. Output: <file>. INSTRUCTIONS: <file>.`
|
||||
- verdict lines `**Verdict**: changed | nothing to change` + `**Changed**: none | <files>` (upgrader); `**Verdict**: keep | remove | cancel` + `**Remove**: S<n>, …` (validator).
|
||||
|
||||
- [ ] **Step 1: baseline.** `env -u ANTHROPIC_API_KEY claude -p "/optimize" --plugin-dir agents --model haiku` from `~/Optimizer` → says inputs missing, `git -C ~/detectors status --porcelain` and `git -C ~/tools status --porcelain` unchanged.
|
||||
- [ ] **Step 2: `upgrader.md`.** Frontmatter `name: upgrader`, generic description (improves a target repository from evidence, without overfitting, one commit per fix; request: Target, Evidence, Checks, Base, Output, INSTRUCTIONS), `tools: Read, Edit, Write, Bash`, `model: inherit`, `omitClaudeMd: true`. Body: the request; precedence (Global Constraints); reading order (INSTRUCTIONS, `<Target>/CLAUDE.md` in full, evidence); diagnosis; overfitting; simplest first; checks before and after each fix, output of each named check after fix n in `<Output>/<name>-S<n>.txt`; `Checks: none` → the instructions say how to try; one fix one commit (`upgrader.md:72-77`, made generic on `<Target>`); scripts in `<Output>/tools/`; the output skeleton of the spec; "What changed" and "Why" stand alone. No word `detector`, `thresholds`, `benchmark`, `station`, `Optimizer`.
|
||||
- [ ] **Step 3: `validator.md`.** Same frontmatter pattern, `tools: Read, Bash, Write`. Body from the spec's "The validator": reads only (with `sed -n '/^## What changed/,/^## Checks/p' <modification>`), judges the five points, single pass, keep / remove / cancel with the per-fix check outputs saying which fix regresses, output skeleton. Same forbidden words as step 2.
|
||||
- [ ] **Step 4: `SKILL.md`.** Line 44: `agents:tool-builder` removed. Step 2 request becomes the upgrader request with `Target: <DETECTORS>`, `Evidence: <LABELS>/<station>.jsonl, <FAMILIES>, <CONTEXT>, station <STATION>, range <FROM> → <TO>`, `Checks: range: <range score> (before: <OUT>/range-before.txt); benchmark: <benchmark> (before: <OUT>/benchmark-before.txt); events: <events>; sensors: <sensors>`, `Write <OUT>/modification.md`, `INSTRUCTIONS: instructions/upgrader-detectors.md`. Step 4 request becomes the validator request with `INSTRUCTIONS: instructions/validator-detectors.md`, `Write <OUT>/validation.md`, after each fix `<OUT>/benchmark-S*.txt`. Step 6: note `TOOLS` base and files already modified like `DETECTORS`; request `Target: <TOOLS>. Evidence: <ANALYSIS>/. Checks: none. Base: <tools base>. Files already modified: <…>. Output folder: <OUT>. Write <OUT>/tools.md. INSTRUCTIONS: instructions/upgrader-tools.md.`; verdict read the same way. Summary: `**Tools changed**: <the **Changed** line of <OUT>/tools.md> | error` (label kept); "Reading tools" copies "What I decided" and "What changed" and the `**data.py**` line if changed; Errors: `git -C <TOOLS> log --oneline <tools base>..HEAD`.
|
||||
- [ ] **Step 5: docs.** `~/agents/README.md` table: three rows (`analyst` unchanged; `upgrader`: Target, Evidence, Checks, Base, Files already modified, Output, `INSTRUCTIONS`, called by Optimizer with `instructions/upgrader-detectors.md` and `instructions/upgrader-tools.md`; `validator`: Target, Modification, Base, Checks, Output, `INSTRUCTIONS`, called by Optimizer with `instructions/validator-detectors.md`), one sentence: a target's rules are its `CLAUDE.md`. `plugin.json` description: "analyst, upgrader, validator". `Optimizer/README.md:6,9,38` and `CLAUDE.md:3,7`: tool-builder → upgrader on the tools; `Tools: <analysis>` commit → `improve: S<n>` commits in `~/tools`.
|
||||
- [ ] **Step 6: check.**
|
||||
- `grep -nE "visibility\.py|thresholds|benchmark|regression\.py|Optimizer|optimize|station" ~/agents/agents/upgrader.md ~/agents/agents/validator.md` → nothing.
|
||||
- `grep -rn "tool-builder" ~/agents ~/Optimizer --include=*.md --include=*.json | grep -v docs/superpowers | grep -v archive/ | grep -v sessions/` → nothing.
|
||||
- From `~/Optimizer`: `env -u ANTHROPIC_API_KEY claude -p "Without running anything, list the fields of your request and the order of precedence of your sources." --plugin-dir agents --agent agents:upgrader --model haiku` → names Target, Evidence, Checks, Base, Output, INSTRUCTIONS and the precedence of Global Constraints; same for `agents:validator`.
|
||||
- From `~/Optimizer`: `env -u ANTHROPIC_API_KEY claude -p "Target: tools/. INSTRUCTIONS: instructions/upgrader-tools.md. Without changing anything: may you remove an option of a tool that did not help? With Checks: none, how do you check a fix?" --plugin-dir agents --agent agents:upgrader --model haiku --add-dir ~/tools` → no (from `tools/CLAUDE.md`); the trial of the instructions.
|
||||
- Step 1 command again → same refusal, nothing touched.
|
||||
- [ ] **Step 7: commit both repositories**, back to back. `~/agents`: `git rm agents/tool-builder.md` then `git commit -m "Generic upgrader and validator on a target; tool-builder folded into the upgrader" -m "Co-Authored-By: …" -- agents/upgrader.md agents/validator.md agents/tool-builder.md README.md .claude-plugin/plugin.json`. `~/Optimizer`: `-- .claude/skills/optimize/SKILL.md README.md CLAUDE.md`, message `"optimize: generic agents with instructions/; the tools by the upgrader"`.
|
||||
|
||||
### Task 5: Diagnostic-App wording and a real session
|
||||
|
||||
**Files:**
|
||||
- Modify: `~/Diagnostic-App/README.md:23`, `~/Diagnostic-App/app/server/station.py:26` (comment only)
|
||||
|
||||
- [ ] **Step 1: wording.** "the tool-builder writes in it" → "the upgrader writes in it (Optimizer)".
|
||||
- [ ] **Step 2: check** `cd ~/Diagnostic-App && python3 app/server/check.py` → passes as before (the summary lines did not change).
|
||||
- [ ] **Step 3: commit** by path: `"README: the tools are written by the upgrader"`.
|
||||
- [ ] **Step 4: real session — ask the user first** (it commits in `~/detectors` and `~/tools`). On their go, from the app's "Optimize" button, or by hand with the command of `~/Optimizer/README.md` on `labeling/auto/analyses/NORDEND_2026-11-05T0600_2026-11-05T1200/`. Expected: `modification.md`, `validation.md` (if fixes), `tools.md` each with a readable `**Verdict**:` line; `summary.md` with `**Result**` and `**Tools changed**`; commits named `improve: S<n>` in each repository touched; the page shows the summary. Report the outcome to the user as it is, failures included.
|
||||
Reference in new issue
Block a user