Files
aimsight-labeling/docs/superpowers/plans/2026-11-16-analyst-knowledge.md
T

119 lines
15 KiB
Markdown
Raw Normal View History

# The analyst's domain knowledge: implementation plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** one hand-maintained source per kind of knowledge for the analyst (labels through the RAG, `families.md` with a phenomenon catalogue, `labeling-rules.md`, `reading-rules.md`, `stations.md`, `tools/DATA.md`), the old map and `knowledge/` archived, then the labeling scripts moved to `labeling/scripts/`.
**Architecture:** mostly writing: markdown compiled from existing files, then path updates in four repositories. One throwaway script (in the scratchpad, never committed) converts the map's label keys to label ids. The only code change is the scripts' root path after the move.
**Tech Stack:** markdown; Python 3 standard library; git.
**Spec:** `labeling/docs/superpowers/specs/2026-11-16-analyst-knowledge-design.md`.
## Global Constraints
- Starts only when the English migration (`Diagnostic-App/docs/superpowers/plans/2026-11-16-english-renames.md`) is committed in labeling, tools, agents and Diagnostic-App: `git status --short` shows nothing of it in any of them (the app's own label submissions in `labeling/auto/*.jsonl` may remain; never stage them).
- All new text in English. Times in UTC. No numeric value in label descriptions (not touched here anyway).
- `labeling/manual/` is never read by any step or script.
- No station-specific *rule* in `families.md`, `labeling-rules.md`, `reading-rules.md`; `stations.md` is context only and says so.
- tools holds what the data *are*; labeling what they *mean*. Nothing in `tools/DATA.md` says "normal", a category or a threshold to judge by.
- Commits by path, never `git add -A`; English commit messages ending with `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`.
- Never stage `labeling/auto/*.jsonl` or `labeling/auto/analyses/` (Diagnostic-App writes there concurrently).
## Review Focus
1. Knowledge silently dropped: a bullet of the map's sections 2–3, or a fact of a `knowledge/` note, that lands in no new file. Pinned by the coverage table of Task 3, step 4.
2. A map key converted to the wrong id (two labels share a start). Pinned by the ambiguity report of Task 3, step 2: every ambiguous key is resolved by hand, none guessed.
3. A stale path left in another repository (`label-map`, `knowledge/`, `labeling/<script>.py`). Pinned by the repo-wide greps of Tasks 6 and 8.
4. The RAG index not found after the move, re-embedding ~1,000 labels (~8 min). Pinned by Task 8, step 5.
5. The analyst still told to read a removed file. Pinned by the load check of Task 6.
---
### Task 0: Gate and baseline
- [ ] **Step 1:** check the gate (Global Constraints, first line). If not met, stop and tell the user.
- [ ] **Step 2:** baseline, saved in the scratchpad as `baseline.txt`:
- `cd ~/labeling && python3 validate.py` (count of errors; the app's labels without `review` are expected);
- `python3 ids.py` (must change nothing: `git diff --stat auto/` before = after);
- bytes the analyst reads in full today: `wc -c label-map.md families.md labeling-rules.md knowledge/measurement-chain.md knowledge/displacement-columns.md knowledge/gaps.md ~/tools/GUIDE.md` (plus the map is read only partly: count lines 1–60 and sections 2–4).
### Task 1: The spec and the plan in labeling
**Files:** Create `labeling/docs/superpowers/specs/2026-11-16-analyst-knowledge-design.md`, `labeling/docs/superpowers/plans/2026-11-16-analyst-knowledge.md` (copies of the scratchpad files).
- [ ] **Step 1:** copy both files.
- [ ] **Step 2:** commit: `git add docs/superpowers/specs/2026-11-16-analyst-knowledge-design.md docs/superpowers/plans/2026-11-16-analyst-knowledge.md && git commit -m "Analyst knowledge: design and plan (one source per kind of knowledge)"`.
### Task 2: `tools/DATA.md`
**Files:**
- Create: `tools/DATA.md`
- Modify: `tools/GUIDE.md` (section "Read first", line 7 onwards), `tools/scripts/data.py` (docstring l. 3, l. 129), `tools/scripts/inventory.py` (messages l. 79, 128, 193), `tools/scripts/quality.py` (docstring l. 11, message ~l. 377), `tools/README.md` (the files of the repository), `tools/CLAUDE.md` (one line: `DATA.md` is maintained by hand, `GUIDE.md` by the tool-builder).
- Sources: `labeling/knowledge/displacement-columns.md`, `quality-metrics.md`, `camera-images.md`.
- [ ] **Step 1:** write `DATA.md`, sections: "The exports" (advanced, hourly, camera pose: columns and their meaning, rejected measurement keeps its row with empty `corr`, `corr_x`/`corr_y` in pixels, the per-pattern factor to the hourly mm, `x`/`y` are the pattern position not the spot centroid, `tilt` positive = camera nosing down, `ref` vs `global_ref`); "LENZSPITZE" (one line: `x`/`y` rewritten in image pixels by the extraction; 1023 and 8019 without pattern); "Image quality metrics" (the table of `quality-metrics.md`, as is, and its closing paragraph: quality of a target ≠ health of the reference network); "Camera images" (IRON, DIFF, acquisition modes, what is known per station). Drop: pumping dates and history, the detectors' to-do list, Support-Agent paths, the closing "causes to separate" list and the non-negotiable rules (they go to `reading-rules.md`, Task 3).
- [ ] **Step 2:** "Read first" of `GUIDE.md` becomes two items: `tools/DATA.md` (**first**), and `labeling/reading-rules.md`. Point every docstring and message listed above at `tools/DATA.md`.
- [ ] **Step 3:** verify: `grep -rn 'knowledge/' ~/tools --include=*.py --include=*.md` → nothing; `cd ~/tools && python3 scripts/quality.py --self-test` → passes; from `~/Diagnostic-App`: `python3 tools/scripts/inventory.py data/NORDEND --start 2026-11-05T06:00Z --end 2026-11-05T12:00Z` runs and its messages name `tools/DATA.md`.
- [ ] **Step 4:** commit in tools: `git add DATA.md GUIDE.md README.md CLAUDE.md scripts/data.py scripts/inventory.py scripts/quality.py && git commit -m "DATA.md: what the data are, from labeling/knowledge (formats, columns, quality metrics, images)"`.
### Task 3: `labeling/reading-rules.md`
**Files:** Create `labeling/reading-rules.md`; scratchpad only: `convert_keys.py`, `coverage.md`.
**Sources:** `knowledge/measurement-chain.md`, `knowledge/gaps.md`, `label-map.md` sections 2 and 3, the non-negotiable rules (end of `displacement-columns.md`, section 4 of `measurement-chain.md`), `knowledge/barcelona-incident.md`, `knowledge/expert-annotated-cases.md`.
**Interfaces:**
- Produces: `convert_keys.py` — `convert(text: str) -> tuple[str, list[str]]` returns the text with keys replaced and the list of unresolved or ambiguous keys; reused by Tasks 4 and 5.
- [ ] **Step 1:** write `convert_keys.py` (scratchpad). A key is `STATION DD.MM HH:MM`, optionally followed by ` (qualifier)`, inside `[...]` citations. Year 2026. Look up `labeling/auto/<STATION>.jsonl` labels with `start == "2026-MM-DDTHH:MMZ"`: one → its `id`; several → the one whose `description` contains the qualifier (or whose `family` equals it); none → `[expert]`; still several → leave the key, report it. Ends with an `assert`-based self-check on three keys taken from the real map: one unique, one with a qualifier (`NEBRASKA 16.10 20:00 (COLJ19)`), one with no match in `auto/`.
- [ ] **Step 2:** run it on the map's sections 2–3; resolve every reported key by reading the candidate labels. No guess: if still unclear, write `[expert]`.
- [ ] **Step 3:** write `reading-rules.md`, sections: "How a measurement is made" (chain, from `measurement-chain.md` §1); "Where each cause enters" (its §2 table, without `[H]` cells unless a label confirms them, then cite the label id); "Thermal effects" (§3); "Gaps" (`gaps.md`); "Reading rules" (the map's §2, both lists, converted citations); "False friends" (the map's §3); "Non-negotiable rules" (once); "Lessons from cases" (Barcelona, the two client reports: one bullet per lesson, no narrative). Drop what `families.md` or `labeling-rules.md` already say.
- [ ] **Step 4:** `coverage.md` (scratchpad): every bullet of the map's §2–3 and every section of the five source notes, each with where it went (`reading-rules.md` § / `families.md` / `labeling-rules.md` / `DATA.md` / dropped, and why). No row without a destination.
- [ ] **Step 5:** verify: `grep -nE '[A-Z0-9]{4,} [0-9]{2}\.[0-9]{2} [0-9]{2}:[0-9]{2}' reading-rules.md` → nothing; `grep -n 'Support-Agent\|\[H\]\|E-0' reading-rules.md` → nothing.
### Task 4: `labeling/stations.md`
**Files:** Create `labeling/stations.md`. Source: `label-map.md` section 4.
- [ ] **Step 1:** header (two lines: context for reading a station's data, not a rule; the expert rule on the kind of structure depends on it). Then the table: Station | Structure | Camera (interval, shelter, power) | References (number, twins, local references) | Neighbouring cameras. Drop the columns "what recurs in its labels" and "Events". Convert citations with `convert_keys.py`.
- [ ] **Step 2:** verify: one row per station of `auto/` that the map describes; same key grep as Task 3, step 5 → nothing.
### Task 5: The phenomenon catalogue in `families.md`
**Files:** Modify `labeling/families.md` (new section "Phenomena", after "Boundaries", before "What is not labelled"; the table's "Phenomena of the label map" column keeps the names, now pointing to this section). Source: the map's table of contents and cards (sections A–G).
- [ ] **Step 1:** one subsection per family, then "Family of the cause" (a reference set aside, two references lost, the base changing without a lost reference). One line per phenomenon of the table of contents: **name**: what is seen; how to tell it from its neighbours. No citations, no station names. Each phenomenon goes under exactly one heading.
- [ ] **Step 2:** verify: every phenomenon title of the map's table of contents appears once in the section (`grep -c` on each title, a loop in the scratchpad); the section is ≤ 60 lines; `grep -nE 'NORDEND|JUNGFRAU|NIEDERHORN|WILDSTRUBEL|LENZSPITZE|CLARIDEN|BREITHORN|FLORIDA|SNCF01|MICHIGAN|NEBRASKA' families.md` → nothing new.
### Task 6: Archive, labeling docs, the analyst
**Files:**
- Move: `labeling/label-map.md`, `labeling/knowledge/` → `labeling/docs/archive/` (`git mv`); one line each in `labeling/docs/archive/README.md` (why archived, 2026-11-16, replaced by …).
- Modify: `labeling/README.md`, `labeling/CLAUDE.md` (the layout table of the spec: kind → file → maintained by the user, by hand).
- Modify: `agents/agents/analyst.md`: l. 11 (the map sentence → "the phenomena are catalogued in `labeling/families.md`; the cases are the labels, through the precedents"); l. 28–31 (the reading list: `families.md`, `labeling-rules.md`, `reading-rules.md`, `stations.md`, all in full; `tools/GUIDE.md` and `tools/DATA.md`); the procedure step 2 (the catalogue gives the hypotheses, the precedents give the cases); the analysis template line "Phenomenon of the map" → "Phenomenon of the catalogue"; "Sources read" unchanged.
- [ ] **Step 1:** `git mv` and the archive README.
- [ ] **Step 2:** README, CLAUDE.md of labeling; `analyst.md`.
- [ ] **Step 3:** verify: `grep -rn 'label-map\|knowledge/' ~/labeling ~/agents ~/tools ~/Diagnostic-App ~/Optimizer ~/detectors --include=*.md --include=*.py 2>/dev/null | grep -v '/\.git/\|/archive/\|docs/superpowers\|auto/analyses'` → nothing; `cd ~/labeling && python3 validate.py` → as in the baseline; `python3 precedents.py --check` → ok.
- [ ] **Step 4:** load check, from `~/Diagnostic-App`: `claude -p "List the files you must read in full before analysing a range, and nothing else." --plugin-dir ~/agents --agent agents:analyst --model haiku` → exactly the six files of step 2 (and `instructions/` if it names it), no `label-map`, no `knowledge/`.
- [ ] **Step 5:** commit labeling: `git add reading-rules.md stations.md families.md README.md CLAUDE.md docs/archive/ label-map.md knowledge/ && git commit -m "Analyst knowledge: reading-rules.md, stations.md, a phenomenon catalogue in families.md; the label map and knowledge/ archived"`. Commit agents: `git add agents/analyst.md && git commit -m "Analyst: reads families (with the catalogue), labeling rules, reading rules, stations, tools/DATA.md; the label map is archived"`.
### Task 7: A real run
- [ ] **Step 1:** from `~/Diagnostic-App`, run the analyst on NORDEND 2026-11-05T06:00Z → 2026-11-05T12:00Z, output folder in the scratchpad (`run/`), not in `labeling/auto/analyses/` (nothing is submitted to `auto/`): the request format of `analyst.md`, `DETECTORS : no`, `INSTRUCTIONS: instructions/analyst.md`, with `--add-dir` on the real targets of `data/`, `labeling/`, `tools/` and the scratchpad.
- [ ] **Step 2:** compare with `labeling/auto/analyses/NORDEND_2026-11-05T0600_2026-11-05T1200/`: the labels (count, bounds, families), and section 3 "documentation" of the new analysis: anything it says is missing that the map used to give. Note the bytes read in full against the baseline.
- [ ] **Step 3:** report to the user; a missing piece of knowledge goes into the right file (Tasks 3–5) and Task 6, step 5 is amended, before Task 8.
### Task 8: The scripts in `labeling/scripts/`
**Files:**
- Move: `labeling/validate.py`, `ids.py`, `precedents.py` → `labeling/scripts/`.
- Modify: `scripts/validate.py:19`, `scripts/ids.py:14` (`AUTO` = `../auto` from the script); `scripts/precedents.py:29` (`HERE` → the labeling root, so that `auto/` and `.index/` stay where they are); the usage lines and messages of the three scripts (`python3 labeling/scripts/<script>.py`); `labeling/README.md`, `labeling/CLAUDE.md`, `labeling/.claude/settings.json` (the three permissions); `agents/agents/analyst.md` (precedents and validate lines); `Diagnostic-App/README.md`; the comment of `Diagnostic-App/app/server/station.py:266`.
- [ ] **Step 1:** `git mv` and the path changes.
- [ ] **Step 2:** `python3 scripts/precedents.py --check` → ok.
- [ ] **Step 3:** `python3 scripts/validate.py` → same as the baseline; `python3 labeling/scripts/validate.py` from `~/Diagnostic-App` (through the link) → same.
- [ ] **Step 4:** `python3 scripts/ids.py` → `git diff --stat auto/` unchanged.
- [ ] **Step 5:** `python3 scripts/precedents.py "a reference rejected intermittently makes the targets jump together" --k 3` → three precedents, and no "index: N/M new texts" line on stderr (nothing re-embedded).
- [ ] **Step 6:** `grep -rn 'labeling/\(validate\|ids\|precedents\)\.py\|python3 \(validate\|ids\|precedents\)\.py' ~/labeling ~/agents ~/tools ~/Diagnostic-App ~/Optimizer ~/detectors 2>/dev/null | grep -v '/\.git/\|/archive/\|docs/superpowers\|auto/analyses'` → nothing.
- [ ] **Step 7:** commits by path: labeling (`scripts/ validate.py ids.py precedents.py README.md CLAUDE.md .claude/settings.json`, "The labeling scripts in scripts/"), agents (`agents/analyst.md`), Diagnostic-App (`README.md app/server/station.py`).