Files
aimsight-labeling/docs/superpowers/plans/2026-11-18-displacement-split.md

18 KiB

Displacement split Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Displacement means something moved and never came back; a new family Perturbation takes what comes back without another family's cause, with nothing to do.

Architecture: The knowledge (families.md, labeling-rules.md) and the key list (validate.py) change first, then the 36 labels and their analysis copies, then the consumers: the detectors (a jump back gets kind Perturbation), aimsight-app (a colour), the agents' instructions (the key list).

Tech Stack: Python 3 standard library; React (Vite) for one line of aimsight-app; markdown.

Spec: docs/superpowers/specs/2026-11-18-displacement-split-design.md

Global Constraints

  • Before Task 1: git -C ~/aimsight-labeling status --short families.md labeling-rules.md reading-rules.md README.md CLAUDE.md scripts/ prints nothing. Another session (the cases work, docs/superpowers/plans/2026-11-18-cases.md) is editing these files; a commit by path here would carry its half-done work. If not empty: stop and tell the user.
  • Commit by path, never git add -A (aimsight-app writes in auto/ at the same time); each repository commits its own files; messages end with Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>.
  • manual/ is never read nor written.
  • An id never changes. Descriptions in English, in words, without numeric values (validate.py checks it).
  • review: kept on a change of family only; removed on the 6 reworded labels (spec, "Open, for the user", point 1, approved).
  • aimsight-detectors/bench/ is never touched (it is the measure).
  • The exact keys: Displacement, Perturbation. The variety Sudden jump, back to normal belongs to Perturbation; Lasting shift, Jump, not back yet, Slow drift to Displacement.

Review Focus

  1. aimsight-app writes a label in auto/<STATION>.jsonl while the relabel runs: the app's write must not be lost (Task 2 re-reads each file right before writing it and changes only the target lines).
  2. A label the plan rewords was edited since (its closing sentence no longer matches): the relabel must stop with the id, not skip it (Task 2, Step 1).
  3. An analysis copy missing, or matched twice: stop with the id (Task 2, Step 1).
  4. aimsight-app proposes a Perturbation label: validate.check(label, proposed=True) must accept it (Task 1, Step 1).
  5. A detectors' event of variety Sudden jump, back to normal asks the client: client.action("Perturbation", "Sudden jump, back to normal") gives the variety's text, not the family's (Task 1, Step 1).

Task 1: The families, the rules, the key list

Files:

  • Modify: families.md:10, :22-24, :31-33, new boundary after :33
  • Modify: labeling-rules.md:63, :86
  • Modify: scripts/validate.py:23 (FAMILIES), self_check()
  • Modify: scripts/client.py (__main__ asserts)
  • Add: docs/superpowers/specs/2026-11-18-displacement-split-design.md, this plan

Interfaces:

  • Produces: validate.FAMILIES contains "Perturbation"; client.advice() has keys Displacement, Perturbation and the four varieties with the texts below.

  • Step 1: Write the failing checks

In scripts/client.py, __main__, replace the Displacement assert and add:

    assert action("Displacement", "Slow drift", None, t) == "Keep watching it."
    assert action("Displacement", "Lasting shift", None, t) == "Check what moved: look at the images."
    assert action("Perturbation", "Sudden jump, back to normal", None, t) == "Nothing to do: it came back; ignore these points."
    assert action("Perturbation", "x", None, t) == "Nothing to do: the curves came back; ignore these points."

In scripts/validate.py, self_check(), before print("ok"):

    label = {"start": "2026-11-04T00:00Z", "end": "2026-11-04T01:00Z", "family": "Perturbation", "description": "x"}
    assert not any("unknown family" in e for e in check(label, proposed=True))
  • Step 2: Run them, see them fail

Run: python3 scripts/client.py; python3 scripts/validate.py --check Expected: AssertionError in both.

  • Step 3: families.md

Line 10 becomes the two rows:

| `Displacement` | `displacement.py` (lasting shift, jump not back yet, slow drift) | something moved, by a jump or a drift, and has not come back by the end of the range: one or more targets, a reference, or the camera outside a thermal effect; the description says what moved | check what moved: the structure (look at the images, check on site) or the camera (check its mount) |
| `Perturbation` | `displacement.py` (jump back to normal) | the corrected curves move beyond their noise and come back, and no other family explains it: a jolt of the camera, a reference or the base shifted for a while, a single point, targets that wander and come back; the cause is not established or several factors mix | nothing to do: the curves came back; ignore these points |

Varieties, lines 22-24:

| `Sudden jump, back to normal` | nothing to do: it came back; ignore these points |
| `Lasting shift` | check what moved: look at the images |
| `Jump, not back yet` | check what moved: look at the images |

Boundaries: line 31, "Visibility, or Displacement without a cause." → "Visibility, or Perturbation without a cause; Displacement if the base stays changed."; line 32, "Without an established cause: Displacement." → "Without an established cause: Perturbation."; line 33, append "What does not follow it and comes back is Perturbation." After line 33, add:

- **Displacement or Perturbation**: decided by the return, not by the cause. What moved and has not come back by the end of the range (a target, a reference, the camera outside a thermal effect) is Displacement; what comes back, even after days, is Perturbation. A camera that moves and stays is Displacement even when the compensation absorbs it afterwards; the description says it is the camera. A partial return is not a return.
  • Step 4: labeling-rules.md

Line 63: add `Perturbation` after `Displacement` in the list of keys. Line 86 becomes the two bullets:

- **A Displacement says what moved, checked target by target on the raw position, and that it has not come back by the end of the range**: targets moved: "a movement of these targets is not excluded: to check (what to look at), monitor"; the camera moved and keeps its new position: "it is not a movement of the structure; the camera mount is to be checked"; the base stays changed, nothing moving in the image: "it is not a movement of the structure; to check with the operator"; the whole scene deforms while the camera moves: "movement of the structure not established: to monitor". [cross Displacement, NEBRASKA]
- **A Perturbation says what moved, that it came back and when, and concludes "nothing to do: points to ignore"**: its cause is stated in the three ways of section 6; "nothing to do" rests on the return seen, not on a cause. [the user, 2026-11-18]
  • Step 5: scripts/validate.py:23: FAMILIES gains "Perturbation".

  • Step 6: Run the checks

Run: python3 scripts/client.py && python3 scripts/validate.py --check && python3 scripts/validate.py | tail -1 Expected: ok, ok, and 0 errors on the labels' line.

  • Step 7: Commit
git add families.md labeling-rules.md scripts/validate.py scripts/client.py docs/superpowers/specs/2026-11-18-displacement-split-design.md docs/superpowers/plans/2026-11-18-displacement-split.md
git commit -m "Families: Displacement is something that moved and never came back; Perturbation, new, what comes back without another family's cause, nothing to do (spec 2026-11-18-displacement-split-design)"

Task 2: The 36 labels and the 6 rewordings

Files:

  • Modify: auto/<STATION>.jsonl (ALASKA, FLORIDA, JUNGFRAU, LENZSPITZE, MICHIGAN, NEBRASKA, NIEDERHORN, NORDEND, WILDSTRUBEL, WYOMING)
  • Modify: auto/analyses/<analysis>/labels.jsonl, the folder named by each label's analysis
  • Create (not committed): <scratchpad>/relabel.py

Interfaces:

  • Consumes: Perturbation known to validate.py (Task 1).

  • Step 1: Write relabel.py

Data, from the spec:

PERTURBATION = """ALASKA-1 ALASKA-2 ALASKA-3 ALASKA-5 ALASKA-6 ALASKA-9 FLORIDA-18 FLORIDA-72 JUNGFRAU-110 JUNGFRAU-112
LENZSPITZE-84 LENZSPITZE-98 LENZSPITZE-105 LENZSPITZE-106 LENZSPITZE-108 MICHIGAN-8 NEBRASKA-3 NIEDERHORN-126
NIEDERHORN-144 NIEDERHORN-157 NORDEND-80 NORDEND-112 NORDEND-243 NORDEND-251 NORDEND-257 NORDEND-271 NORDEND-278
NORDEND-279 NORDEND-310 WILDSTRUBEL-136 WILDSTRUBEL-153 WYOMING-9 WYOMING-11 WYOMING-13 WYOMING-16 WYOMING-17""".split()
REWORD = {  # id: (closing sentence now, closing sentence after)
    "NEBRASKA-3": ("For the customer: the camera straightens and the scene deforms with it: a movement of the structure is not established; the camera tilt is to be monitored, and restoring a third reference will make the curves reliable again.",
                   "For the customer: the camera straightens and the scene deforms with it, then both come back on 17 October: nothing to do over these days; restoring a third reference will make the curves reliable again."),
    "NORDEND-257": ("Cause not established: a movement of PR07 itself is not excluded, but it comes back by itself; to monitor.",
                    "Cause not established; the curve comes back by itself the same morning: nothing to do, points to ignore."),
    "NORDEND-310": ("Cause not established. It is not a movement of the structure; the camera is to be monitored.",
                    "Cause not established. The camera comes back after about an hour: it is not a movement of the structure, nothing to do; points to ignore."),
    "FLORIDA-18": ("Small, without established movement of the structure: to be monitored.",
                   "The shift fades on the morning of the 15th (return labelled separately): nothing to do, points to ignore."),
    "WILDSTRUBEL-136": ("This is not an established movement of the road: do not read these days as a movement, to be checked with the following days.",
                        "The shift fades on 04/10 (return labelled separately): nothing to do, do not read these days as a movement."),
    "NORDEND-250": ("Cause not established: this is not a movement of the structure, the camera moved; point to ignore, the camera is to be watched if it happens again.",
                    "Cause not established: this is not a movement of the structure; the camera moved and keeps its new position: its mount is to be checked."),
}

Behaviour: for each station file, read it, and for each line whose id is in PERTURBATION or REWORD: its family must be Displacement (else stop with the id); set family to Perturbation if in PERTURBATION; if in REWORD, the description must end with the "now" sentence (else stop with the id), replace it, delete review. In auto/analyses/<label["analysis"]>/labels.jsonl, the copy is the one line with the label's start, end and old description (zero or two: stop with the id); apply the same change (and delete review if the copy has one). Write each file right after reading it, every other line byte for byte; a changed line is json.dumps(d, ensure_ascii=False) (the files' format: key order kept, no \u escape). Print the count of labels and copies changed.

  • Step 2: Run it

Run (when the user is not validating in aimsight-app): python3 <scratchpad>/relabel.py Expected: 37 labels, 37 copies (the 36 family changes, 5 of them also reworded, plus NORDEND-250, reworded only).

  • Step 3: Check

Run:

python3 - <<'EOF'
import json, glob
lab = {d["id"]: d for f in glob.glob("auto/*.jsonl") for d in map(json.loads, filter(str.strip, open(f)))}
import sys; sys.path.insert(0, "<scratchpad>"); from relabel import PERTURBATION, REWORD
assert all(lab[i]["family"] == "Perturbation" for i in PERTURBATION)
assert all("review" not in lab[i] and lab[i]["description"].endswith(REWORD[i][1]) for i in REWORD)
assert all("review" in lab[i] for i in set(PERTURBATION) - set(REWORD))
assert lab["NORDEND-250"]["family"] == "Displacement"
print("ok")
EOF
python3 scripts/validate.py | tail -2
git diff --numstat -- auto/ | awk '{a+=$1} END {print a}'

Expected: ok; 0 errors; the diff counts the 74 changed lines plus whatever aimsight-app had already left uncommitted in those files (compare with git diff --numstat taken before Step 2).

  • Step 4: Commit
git add auto/ALASKA.jsonl auto/FLORIDA.jsonl auto/JUNGFRAU.jsonl auto/LENZSPITZE.jsonl auto/MICHIGAN.jsonl auto/NEBRASKA.jsonl auto/NIEDERHORN.jsonl auto/NORDEND.jsonl auto/WILDSTRUBEL.jsonl auto/WYOMING.jsonl auto/analyses/*/labels.jsonl
git commit -m "auto: 36 Displacement labels that came back requalified Perturbation (review kept, a change of family the user asked for); 6 closing sentences reworded to their family, review removed (NEBRASKA-3, NORDEND-257, NORDEND-310, FLORIDA-18, WILDSTRUBEL-136, NORDEND-250)"

Task 3: The cases of family Displacement

Files:

  • Modify: auto/cases/<id>.json, only if auto/cases/ exists

  • Step 1: List them

Run: python3 scripts/cases.py --list | grep "| Displacement |" If auto/cases/ does not exist yet: skip this task and say so; the cases build reads the labels, which carry the new families.

  • Step 2: Requalify

A case whose signature is something that comes back (the seed's "the base changes without a reference lost" and "targets move slowly, alone, then come back") gets "family": "Perturbation" and loses review if it has one (the cases rule: a change of family removes it). The others stay.

  • Step 3: Check and commit

Run: python3 scripts/validate.py | tail -1 && python3 scripts/cases.py --index Expected: N cases: 0 errors, the index updated.

git add auto/cases/<the changed files>
git commit -m "cases: the Displacement cases that come back are Perturbation"

Task 4: The detectors

Files (repository ~/aimsight-detectors):

  • Modify: scripts/displacement.py (jumps(), the docstring point 2)
  • Modify: scripts/categories.py (CATEGORIES, docstring line 8)
  • Modify: README.md:7, :15; CLAUDE.md ("the six families" → "the seven families")

Interfaces:

  • Produces: an episode of state "back" has kind == "Perturbation", label == "Sudden jump, back to normal"; categories.CATEGORIES["Perturbation"] == {"ignore": True, "process": "none"}, placed right after "Displacement".

  • Step 1: The benchmark before

Run: python3 bench/regression.py --labels ~/aimsight-labeling/auto --data ~/aimsight-data/data > <scratchpad>/before.txt Expected: no skipped line; no Perturbation row (no detector gives it yet).

  • Step 2: Change

jumps(): the episode's kind is "Perturbation" when state == "back", else "Displacement". Docstring point 2: "greyed, its cause not established here" → "a Perturbation, greyed". categories.py docstring line 8: "otherwise it stays Displacement, its cause to be checked on the images" → "otherwise it is a Perturbation: nothing to do". README.md: "six families" → "seven families"; the displacement.py row: "Displacement (Lasting shift; Jump, not back yet; Slow drift), Perturbation (Sudden jump, back to normal)". CLAUDE.md: "six families" → "seven families".

  • Step 3: The benchmark after

Run: python3 bench/regression.py --labels ~/aimsight-labeling/auto --data ~/aimsight-data/data > <scratchpad>/after.txt && python3 bench/compare.py <scratchpad>/before.txt <scratchpad>/after.txt Expected: no skipped line; a Perturbation row with "found" above zero; the Displacement "correct" rises. A regression that compare.py reports is told to the user with the labels concerned (ids), per this repository's rule, not hidden.

  • Step 4: Commit
git -C ~/aimsight-detectors add scripts/displacement.py scripts/categories.py README.md CLAUDE.md
git -C ~/aimsight-detectors commit -m "Perturbation, the seventh family: a jump back to normal is a Perturbation (nothing to do), Displacement keeps the lasting shift, the jump not back yet and the slow drift (labeling spec 2026-11-18-displacement-split-design)"

Task 5: aimsight-app and the agents' instructions

Files:

  • Modify: ~/aimsight-app/app/web/src/pages/Data.jsx:16 (FAMILIES), :19 (comment)

  • Modify: ~/aimsight-agents/agents/analyst.md:56

  • Modify: ~/Support-Agent/.claude/skills/support/rapport.md:20

  • Step 1: Data.jsx

FAMILIES gains "Perturbation": "#9fd8d0" right after "Displacement"; the comment "one of the six families" → "one of the seven families".

  • Step 2: Build and restart

Run: cd ~/aimsight-app/app/web && npm run build && grep -l Perturbation dist/assets/*.js && systemctl --user restart aimsight-app Expected: one bundle file named; the service active (systemctl --user is-active aimsight-app → active).

  • Step 3: The instructions

analyst.md:56: add `Perturbation` after `Displacement` in the list of keys. rapport.md:20: `Displacement` (à vérifier) → `Displacement` (quelque chose a bougé : à vérifier), and `Perturbation` (rien à faire) added last in the order.

  • Step 4: Commit, one per repository
git -C ~/aimsight-app add app/web/src/pages/Data.jsx
git -C ~/aimsight-app commit -m "Data: the family Perturbation and its colour"
git -C ~/aimsight-agents add agents/analyst.md
git -C ~/aimsight-agents commit -m "analyst: the family Perturbation"
git -C ~/Support-Agent add .claude/skills/support/rapport.md
git -C ~/Support-Agent commit -m "support: la famille Perturbation"

Task 6: The whole

  • Step 1: Run every check

Run: cd ~/aimsight-labeling && python3 scripts/validate.py | tail -2 && python3 scripts/validate.py --check && python3 scripts/client.py && python3 scripts/cases.py --check Expected: 0 errors (labels and cases), ok, ok, ok.

  • Step 2: Look at it in the app

Open a range with a Perturbation label (ALASKA, 2026-11-05) on the Data page: the label has the new colour, the Category menu offers Perturbation, a detectors' jump back shows as Perturbation, greyed.