Analyst knowledge: design and plan (one source per kind of knowledge)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
maximeandClaude Opus 5.5 committed 2026-11-16 12:40:54 +01:00
1 parent 68a0eefd8b
commit 1c8479d962
2 files changed
+222

No files matched your search

@@ -0,0 +1,119 @@
# The analyst's domain knowledge: implementation plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** one hand-maintained source per kind of knowledge for the analyst (labels through the RAG, `families.md` with a phenomenon catalogue, `labeling-rules.md`, `reading-rules.md`, `stations.md`, `tools/DATA.md`), the old map and `knowledge/` archived, then the labeling scripts moved to `labeling/scripts/`.
**Architecture:** mostly writing: markdown compiled from existing files, then path updates in four repositories. One throwaway script (in the scratchpad, never committed) converts the map's label keys to label ids. The only code change is the scripts' root path after the move.
**Tech Stack:** markdown; Python 3 standard library; git.
**Spec:** `labeling/docs/superpowers/specs/2026-11-16-analyst-knowledge-design.md`.
## Global Constraints
- Starts only when the English migration (`Diagnostic-App/docs/superpowers/plans/2026-11-16-english-renames.md`) is committed in labeling, tools, agents and Diagnostic-App: `git status --short` shows nothing of it in any of them (the app's own label submissions in `labeling/auto/*.jsonl` may remain; never stage them).
- All new text in English. Times in UTC. No numeric value in label descriptions (not touched here anyway).
- `labeling/manual/` is never read by any step or script.
- No station-specific *rule* in `families.md`, `labeling-rules.md`, `reading-rules.md`; `stations.md` is context only and says so.
- tools holds what the data *are*; labeling what they *mean*. Nothing in `tools/DATA.md` says "normal", a category or a threshold to judge by.
- Commits by path, never `git add -A`; English commit messages ending with `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`.
- Never stage `labeling/auto/*.jsonl` or `labeling/auto/analyses/` (Diagnostic-App writes there concurrently).
## Review Focus
1. Knowledge silently dropped: a bullet of the map's sections 2–3, or a fact of a `knowledge/` note, that lands in no new file. Pinned by the coverage table of Task 3, step 4.
2. A map key converted to the wrong id (two labels share a start). Pinned by the ambiguity report of Task 3, step 2: every ambiguous key is resolved by hand, none guessed.
3. A stale path left in another repository (`label-map`, `knowledge/`, `labeling/<script>.py`). Pinned by the repo-wide greps of Tasks 6 and 8.
4. The RAG index not found after the move, re-embedding ~1,000 labels (~8 min). Pinned by Task 8, step 5.
5. The analyst still told to read a removed file. Pinned by the load check of Task 6.
---
### Task 0: Gate and baseline
- [ ] **Step 1:** check the gate (Global Constraints, first line). If not met, stop and tell the user.
- [ ] **Step 2:** baseline, saved in the scratchpad as `baseline.txt`:
- `cd ~/labeling && python3 validate.py` (count of errors; the app's labels without `review` are expected);
- `python3 ids.py` (must change nothing: `git diff --stat auto/` before = after);
- bytes the analyst reads in full today: `wc -c label-map.md families.md labeling-rules.md knowledge/measurement-chain.md knowledge/displacement-columns.md knowledge/gaps.md ~/tools/GUIDE.md` (plus the map is read only partly: count lines 1–60 and sections 2–4).
### Task 1: The spec and the plan in labeling
**Files:** Create `labeling/docs/superpowers/specs/2026-11-16-analyst-knowledge-design.md`, `labeling/docs/superpowers/plans/2026-11-16-analyst-knowledge.md` (copies of the scratchpad files).
- [ ] **Step 1:** copy both files.
- [ ] **Step 2:** commit: `git add docs/superpowers/specs/2026-11-16-analyst-knowledge-design.md docs/superpowers/plans/2026-11-16-analyst-knowledge.md && git commit -m "Analyst knowledge: design and plan (one source per kind of knowledge)"`.
### Task 2: `tools/DATA.md`
**Files:**
- Create: `tools/DATA.md`
- Modify: `tools/GUIDE.md` (section "Read first", line 7 onwards), `tools/scripts/data.py` (docstring l. 3, l. 129), `tools/scripts/inventory.py` (messages l. 79, 128, 193), `tools/scripts/quality.py` (docstring l. 11, message ~l. 377), `tools/README.md` (the files of the repository), `tools/CLAUDE.md` (one line: `DATA.md` is maintained by hand, `GUIDE.md` by the tool-builder).
- Sources: `labeling/knowledge/displacement-columns.md`, `quality-metrics.md`, `camera-images.md`.
- [ ] **Step 1:** write `DATA.md`, sections: "The exports" (advanced, hourly, camera pose: columns and their meaning, rejected measurement keeps its row with empty `corr`, `corr_x`/`corr_y` in pixels, the per-pattern factor to the hourly mm, `x`/`y` are the pattern position not the spot centroid, `tilt` positive = camera nosing down, `ref` vs `global_ref`); "LENZSPITZE" (one line: `x`/`y` rewritten in image pixels by the extraction; 1023 and 8019 without pattern); "Image quality metrics" (the table of `quality-metrics.md`, as is, and its closing paragraph: quality of a target ≠ health of the reference network); "Camera images" (IRON, DIFF, acquisition modes, what is known per station). Drop: pumping dates and history, the detectors' to-do list, Support-Agent paths, the closing "causes to separate" list and the non-negotiable rules (they go to `reading-rules.md`, Task 3).
- [ ] **Step 2:** "Read first" of `GUIDE.md` becomes two items: `tools/DATA.md` (**first**), and `labeling/reading-rules.md`. Point every docstring and message listed above at `tools/DATA.md`.
- [ ] **Step 3:** verify: `grep -rn 'knowledge/' ~/tools --include=*.py --include=*.md` → nothing; `cd ~/tools && python3 scripts/quality.py --self-test` → passes; from `~/Diagnostic-App`: `python3 tools/scripts/inventory.py data/NORDEND --start 2026-11-05T06:00Z --end 2026-11-05T12:00Z` runs and its messages name `tools/DATA.md`.
- [ ] **Step 4:** commit in tools: `git add DATA.md GUIDE.md README.md CLAUDE.md scripts/data.py scripts/inventory.py scripts/quality.py && git commit -m "DATA.md: what the data are, from labeling/knowledge (formats, columns, quality metrics, images)"`.
### Task 3: `labeling/reading-rules.md`
**Files:** Create `labeling/reading-rules.md`; scratchpad only: `convert_keys.py`, `coverage.md`.
**Sources:** `knowledge/measurement-chain.md`, `knowledge/gaps.md`, `label-map.md` sections 2 and 3, the non-negotiable rules (end of `displacement-columns.md`, section 4 of `measurement-chain.md`), `knowledge/barcelona-incident.md`, `knowledge/expert-annotated-cases.md`.
**Interfaces:**
- Produces: `convert_keys.py` — `convert(text: str) -> tuple[str, list[str]]` returns the text with keys replaced and the list of unresolved or ambiguous keys; reused by Tasks 4 and 5.
- [ ] **Step 1:** write `convert_keys.py` (scratchpad). A key is `STATION DD.MM HH:MM`, optionally followed by ` (qualifier)`, inside `[...]` citations. Year 2026. Look up `labeling/auto/<STATION>.jsonl` labels with `start == "2026-MM-DDTHH:MMZ"`: one → its `id`; several → the one whose `description` contains the qualifier (or whose `family` equals it); none → `[expert]`; still several → leave the key, report it. Ends with an `assert`-based self-check on three keys taken from the real map: one unique, one with a qualifier (`NEBRASKA 16.10 20:00 (COLJ19)`), one with no match in `auto/`.
- [ ] **Step 2:** run it on the map's sections 2–3; resolve every reported key by reading the candidate labels. No guess: if still unclear, write `[expert]`.
- [ ] **Step 3:** write `reading-rules.md`, sections: "How a measurement is made" (chain, from `measurement-chain.md` §1); "Where each cause enters" (its §2 table, without `[H]` cells unless a label confirms them, then cite the label id); "Thermal effects" (§3); "Gaps" (`gaps.md`); "Reading rules" (the map's §2, both lists, converted citations); "False friends" (the map's §3); "Non-negotiable rules" (once); "Lessons from cases" (Barcelona, the two client reports: one bullet per lesson, no narrative). Drop what `families.md` or `labeling-rules.md` already say.
- [ ] **Step 4:** `coverage.md` (scratchpad): every bullet of the map's §2–3 and every section of the five source notes, each with where it went (`reading-rules.md` § / `families.md` / `labeling-rules.md` / `DATA.md` / dropped, and why). No row without a destination.
- [ ] **Step 5:** verify: `grep -nE '[A-Z0-9]{4,} [0-9]{2}\.[0-9]{2} [0-9]{2}:[0-9]{2}' reading-rules.md` → nothing; `grep -n 'Support-Agent\|\[H\]\|E-0' reading-rules.md` → nothing.
### Task 4: `labeling/stations.md`
**Files:** Create `labeling/stations.md`. Source: `label-map.md` section 4.
- [ ] **Step 1:** header (two lines: context for reading a station's data, not a rule; the expert rule on the kind of structure depends on it). Then the table: Station | Structure | Camera (interval, shelter, power) | References (number, twins, local references) | Neighbouring cameras. Drop the columns "what recurs in its labels" and "Events". Convert citations with `convert_keys.py`.
- [ ] **Step 2:** verify: one row per station of `auto/` that the map describes; same key grep as Task 3, step 5 → nothing.
### Task 5: The phenomenon catalogue in `families.md`
**Files:** Modify `labeling/families.md` (new section "Phenomena", after "Boundaries", before "What is not labelled"; the table's "Phenomena of the label map" column keeps the names, now pointing to this section). Source: the map's table of contents and cards (sections A–G).
- [ ] **Step 1:** one subsection per family, then "Family of the cause" (a reference set aside, two references lost, the base changing without a lost reference). One line per phenomenon of the table of contents: **name**: what is seen; how to tell it from its neighbours. No citations, no station names. Each phenomenon goes under exactly one heading.
- [ ] **Step 2:** verify: every phenomenon title of the map's table of contents appears once in the section (`grep -c` on each title, a loop in the scratchpad); the section is ≤ 60 lines; `grep -nE 'NORDEND|JUNGFRAU|NIEDERHORN|WILDSTRUBEL|LENZSPITZE|CLARIDEN|BREITHORN|FLORIDA|SNCF01|MICHIGAN|NEBRASKA' families.md` → nothing new.
### Task 6: Archive, labeling docs, the analyst
**Files:**
- Move: `labeling/label-map.md`, `labeling/knowledge/` → `labeling/docs/archive/` (`git mv`); one line each in `labeling/docs/archive/README.md` (why archived, 2026-11-16, replaced by …).
- Modify: `labeling/README.md`, `labeling/CLAUDE.md` (the layout table of the spec: kind → file → maintained by the user, by hand).
- Modify: `agents/agents/analyst.md`: l. 11 (the map sentence → "the phenomena are catalogued in `labeling/families.md`; the cases are the labels, through the precedents"); l. 28–31 (the reading list: `families.md`, `labeling-rules.md`, `reading-rules.md`, `stations.md`, all in full; `tools/GUIDE.md` and `tools/DATA.md`); the procedure step 2 (the catalogue gives the hypotheses, the precedents give the cases); the analysis template line "Phenomenon of the map" → "Phenomenon of the catalogue"; "Sources read" unchanged.
- [ ] **Step 1:** `git mv` and the archive README.
- [ ] **Step 2:** README, CLAUDE.md of labeling; `analyst.md`.
- [ ] **Step 3:** verify: `grep -rn 'label-map\|knowledge/' ~/labeling ~/agents ~/tools ~/Diagnostic-App ~/Optimizer ~/detectors --include=*.md --include=*.py 2>/dev/null | grep -v '/\.git/\|/archive/\|docs/superpowers\|auto/analyses'` → nothing; `cd ~/labeling && python3 validate.py` → as in the baseline; `python3 precedents.py --check` → ok.
- [ ] **Step 4:** load check, from `~/Diagnostic-App`: `claude -p "List the files you must read in full before analysing a range, and nothing else." --plugin-dir ~/agents --agent agents:analyst --model haiku` → exactly the six files of step 2 (and `instructions/` if it names it), no `label-map`, no `knowledge/`.
- [ ] **Step 5:** commit labeling: `git add reading-rules.md stations.md families.md README.md CLAUDE.md docs/archive/ label-map.md knowledge/ && git commit -m "Analyst knowledge: reading-rules.md, stations.md, a phenomenon catalogue in families.md; the label map and knowledge/ archived"`. Commit agents: `git add agents/analyst.md && git commit -m "Analyst: reads families (with the catalogue), labeling rules, reading rules, stations, tools/DATA.md; the label map is archived"`.
### Task 7: A real run
- [ ] **Step 1:** from `~/Diagnostic-App`, run the analyst on NORDEND 2026-11-05T06:00Z → 2026-11-05T12:00Z, output folder in the scratchpad (`run/`), not in `labeling/auto/analyses/` (nothing is submitted to `auto/`): the request format of `analyst.md`, `DETECTORS : no`, `INSTRUCTIONS: instructions/analyst.md`, with `--add-dir` on the real targets of `data/`, `labeling/`, `tools/` and the scratchpad.
- [ ] **Step 2:** compare with `labeling/auto/analyses/NORDEND_2026-11-05T0600_2026-11-05T1200/`: the labels (count, bounds, families), and section 3 "documentation" of the new analysis: anything it says is missing that the map used to give. Note the bytes read in full against the baseline.
- [ ] **Step 3:** report to the user; a missing piece of knowledge goes into the right file (Tasks 3–5) and Task 6, step 5 is amended, before Task 8.
### Task 8: The scripts in `labeling/scripts/`
**Files:**
- Move: `labeling/validate.py`, `ids.py`, `precedents.py` → `labeling/scripts/`.
- Modify: `scripts/validate.py:19`, `scripts/ids.py:14` (`AUTO` = `../auto` from the script); `scripts/precedents.py:29` (`HERE` → the labeling root, so that `auto/` and `.index/` stay where they are); the usage lines and messages of the three scripts (`python3 labeling/scripts/<script>.py`); `labeling/README.md`, `labeling/CLAUDE.md`, `labeling/.claude/settings.json` (the three permissions); `agents/agents/analyst.md` (precedents and validate lines); `Diagnostic-App/README.md`; the comment of `Diagnostic-App/app/server/station.py:266`.
- [ ] **Step 1:** `git mv` and the path changes.
- [ ] **Step 2:** `python3 scripts/precedents.py --check` → ok.
- [ ] **Step 3:** `python3 scripts/validate.py` → same as the baseline; `python3 labeling/scripts/validate.py` from `~/Diagnostic-App` (through the link) → same.
- [ ] **Step 4:** `python3 scripts/ids.py` → `git diff --stat auto/` unchanged.
- [ ] **Step 5:** `python3 scripts/precedents.py "a reference rejected intermittently makes the targets jump together" --k 3` → three precedents, and no "index: N/M new texts" line on stderr (nothing re-embedded).
- [ ] **Step 6:** `grep -rn 'labeling/\(validate\|ids\|precedents\)\.py\|python3 \(validate\|ids\|precedents\)\.py' ~/labeling ~/agents ~/tools ~/Diagnostic-App ~/Optimizer ~/detectors 2>/dev/null | grep -v '/\.git/\|/archive/\|docs/superpowers\|auto/analyses'` → nothing.
- [ ] **Step 7:** commits by path: labeling (`scripts/ validate.py ids.py precedents.py README.md CLAUDE.md .claude/settings.json`, "The labeling scripts in scripts/"), agents (`agents/analyst.md`), Diagnostic-App (`README.md app/server/station.py`).
@@ -0,0 +1,103 @@
# The analyst's domain knowledge: one source per kind
2026-11-16. Approved by the user on 2026-11-16.
## Why
Since `precedents.py` (the RAG over `labeling/auto/`), the analyst gets the same knowledge twice: as cases cited inside `label-map.md` and as cases retrieved by the RAG. The user no longer knows which file to maintain, nor which one prevails. `label-map.md` is frozen: it was regenerated by `/labels-map` in Support-Agent (obsolete), partly from `manual/`, which no process may read any more. `knowledge/` mixes data formats, domain knowledge, project history and an outdated note (`thermal-cycle.md` contradicts D-117).
Success: for each kind of knowledge, one file (or the labels), maintained by hand by the user, and the analyst reads exactly those. Nothing derived, nothing written twice.
## Goals, non-goals, anti-goals
- Goal: one source per kind of knowledge; the analyst's mandatory reading shrinks, and nothing it relied on is lost.
- Non-goal: feeding the gaps reported by analyses (section 3: the "comb", a jolt inside a thermal cycle…) back into the rules. Later, separate work.
- Non-goal: changing `precedents.py`. It already refreshes its index on each call (new or changed labels embedded, removed ones dropped): no hook needed.
- Anti-goal: a file generated from labels (the old `label-map.md` pattern), or the same rule in two files.
- Anti-goal: a station-specific *rule*. `stations.md` describes installations; it states no rule.
## Constraints
- Runs **after** the English migration (`Diagnostic-App/docs/superpowers/plans/2026-11-16-english-renames.md`) is committed; names below are the English ones; new text in English.
- `manual/` stays unread by any process.
- tools: "a tool returns values: no threshold, no category, no 'normal'". So tools holds what the data *are*, labeling what they *mean*.
- Commits by path, never `git add -A`; one commit per repository.
## Approaches considered for the phenomenon cards (decided by the user: B)
- A. Drop them, rely on the RAG. Least upkeep; the analyst only finds the right cases if it already describes what it sees with the right words; rare phenomena get lost.
- **B. A short catalogue**, one line per phenomenon (name, signature, how to tell it from its neighbours), the RAG for the detailed cases. Gives the analyst its list of hypotheses for ~40 lines.
- C. Keep the full cards, maintained by hand. Richest; 120 KB largely redundant with the labels.
## Target layout
| Kind of knowledge | Source | Read by the analyst |
|---|---|---|
| Cases | `labeling/auto/*.jsonl`, through `precedents.py` | on demand |
| Families, boundaries, **phenomenon catalogue** | `labeling/families.md` | in full |
| How to write a label | `labeling/labeling-rules.md` (unchanged) | in full |
| What the data mean: measurement chain, gaps, reading rules, false friends, lessons of cases | `labeling/reading-rules.md` (new) | in full |
| Installation of each station | `labeling/stations.md` (new) | in full (short) |
| What the data are: export formats, columns, quality metrics, IRON/DIFF images | `tools/DATA.md` (new) | in full, with `GUIDE.md` |
### `labeling/families.md`
Unchanged, plus a section "Phenomena": per family, one line per phenomenon of the old map's table of contents (sections A to G): name; what is seen (signature, from the card's "what is seen"); how to tell it from its neighbours (from the card and the false friends). Phenomena whose family is that of their cause (a reference set aside, two references lost) go in a short last group, "Family of the cause". No case citations in this section: cases come from the RAG.
Note: Optimizer's `/improve` (upgrader, validator) also reads `families.md` (`FAMILIES:`); the catalogue is useful to them too.
### `labeling/reading-rules.md`
Compiled from:
- `knowledge/measurement-chain.md`: how a measurement is made (section 1), where each cause enters and what it touches (section 2), the three thermal effects (section 3);
- `knowledge/gaps.md`;
- `label-map.md` sections 2 (reading rules: the expert's, then those established by labels) and 3 (false friends);
- the non-negotiable rules (end of `displacement-columns.md`, section 4 of `measurement-chain.md`), once;
- the lessons of `barcelona-incident.md` and `expert-annotated-cases.md` (neighbouring prisms interfering, vegetation growing over weeks, masked references giving a 24 h cycle on pan/tilt, compensation instability falsifying the amplitude of a real movement), without the case narratives.
Removed: what `families.md` or `labeling-rules.md` already say; `[H]` hypotheses not confirmed by labels; sources and paths of Support-Agent; the "open questions" sections.
Citations: a label key `STATION DD.MM HH:MM` becomes the label's id (`NORDEND-80`), found in `auto/<STATION>.jsonl` by station and `start` (the sensor or family in parentheses disambiguates). A key with no match in `auto/` (an expert label of `manual/`) becomes `[expert]`, without a key. Events `[E-xxx]` are dropped (the events base is in Support-Agent).
### `labeling/stations.md`
From `label-map.md` section 4, one row per station: structure (bridge, rock face, tunnel, building…), camera (measurement interval, shelter, power), references (number, twins, local references read as targets), neighbouring cameras. The column "what recurs in its labels" is dropped: that is the RAG's job. Header: context for reading, not a rule; the expert rule "the kind of structure says whether a thermal movement is possible" depends on it.
### `tools/DATA.md`
Compiled from `displacement-columns.md`, `quality-metrics.md`, `camera-images.md`: the export formats and the meaning of each column, the unit of `corr_x`/`corr_y`, the camera pose and the sign of `tilt`, the six quality metrics and their reading, IRON vs DIFF. Removed: pumping history, the LENZSPITZE conversion (keep one line: its `x`/`y` are rewritten in image pixels by the extraction), the to-do list for the detectors, Support-Agent sources. Maintained by hand, beside `GUIDE.md` (which stays the tool-builder's).
### Archived to `labeling/docs/archive/`
`label-map.md` and all of `knowledge/` (`thermal-cycle.md` outdated; the others once compiled). Git history keeps the rest.
## Consumers to update
- `agents/agents/analyst.md`: reading list (`families.md`, `labeling-rules.md`, `reading-rules.md`, `stations.md`, `tools/GUIDE.md` + `tools/DATA.md`, the RAG, `auto/<STATION>.jsonl` around the range); step 2 of the procedure ("look up the phenomenon in the map") becomes: the catalogue gives the hypotheses, the RAG the cases; the analysis template's "Phenomenon of the map" becomes "Phenomenon of the catalogue".
- `tools/GUIDE.md` ("Read first"), docstrings and messages of `tools/scripts/data.py`, `inventory.py`, `quality.py`: point to `tools/DATA.md` and `labeling/reading-rules.md` (gaps).
- `labeling/README.md`, `labeling/CLAUDE.md`: the target layout table, who maintains what (the user, by hand).
- `Diagnostic-App/consignes` (or its English name) if it cites the map; `Diagnostic-App/app/server/station.py:221` and `analysis.py:31`, `Optimizer/README.md:29` cite `families.md` only: unchanged.
## Last step: the scripts in `labeling/scripts/` (asked by the user)
Done after the knowledge layout is verified, as its own commit.
- `git mv validate.py ids.py precedents.py scripts/`.
- Paths: each script finds `auto/` from its own location (`validate.py:19`, `ids.py:14` `AUTO`; `precedents.py:29` `HERE`): go up one more level. The index stays at `labeling/.index/` (already embedded vectors kept, still ignored by git).
- Callers: `python3 labeling/<script>.py` becomes `python3 labeling/scripts/<script>.py` in `agents/agents/analyst.md` (lines 33, 60), the scripts' own docstrings and messages, `labeling/README.md`, `labeling/CLAUDE.md`, `labeling/.claude/settings.json` (permissions), `Diagnostic-App/README.md` and the comment of `Diagnostic-App/app/server/station.py:266`; then a `grep -r` over the seven repositories (outside archives and `docs/superpowers/`).
- Check: `python3 scripts/validate.py` (same errors as before), `python3 scripts/ids.py` (changes nothing), `python3 scripts/precedents.py --check`, and one real query (the index is found, nothing re-embedded).
## Verification
- `python3 validate.py` (expected errors unchanged), `python3 precedents.py --check`.
- `grep -r` for `label-map`, `knowledge/` across the seven repositories (outside archives and `docs/superpowers/`): nothing left.
- Load check: `claude -p "<short question>" --plugin-dir ~/agents --agent agents:analyst --model haiku`.
- A real run on an already analysed range, NORDEND 2026-11-05 06:00 → 12:00, from Diagnostic-App: compare its labels with the previous analysis, and read its section 3 ("documentation") for anything it now misses.
- Mandatory reading size before and after (bytes of the files read in full).
## Calls made by Claude (to check)
- Catalogue inside `families.md` rather than a separate file (fewer files; already read in full; it has a "Phenomena" column).
- `reading-rules.md` drops unconfirmed `[H]` hypotheses and `[E-xxx]` events.
- Citations converted to label ids; unmatched keys become `[expert]`.
- Everything in `knowledge/` archived; nothing kept there.