From 0de006bcb240d1ebf906e7cc1f5267dd2a421b00 Mon Sep 17 00:00:00 2001 From: maxime Date: Fri, 13 Nov 2026 12:40:30 +0100 Subject: [PATCH] wf-improve generic: the caller gives the range, labels, reference truth, families, context and output folder Skill and agents no longer name Diagnostic-app paths (analyses, labeling/auto, demande.json, summary location, git rules of the working repository). Co-Authored-By: Claude Opus 5.5 --- .claude-plugin/plugin.json | 2 +- agents/upgrader.md | 56 +++++++++---------- agents/validator.md | 32 +++++------ scripts/categories.py | 10 ++-- scripts/displacement.py | 2 +- scripts/no_data.py | 2 +- scripts/obstruction.py | 4 +- scripts/target_issue.py | 4 +- scripts/thermal_displacement.py | 2 +- scripts/visibility.py | 4 +- skills/wf-improve/SKILL.md | 97 +++++++++++++++++---------------- thresholds.md | 8 +-- 12 files changed, 110 insertions(+), 113 deletions(-) diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index d359469..14de2a8 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,4 +1,4 @@ { "name": "detectors", - "description": "AIMsight anomaly detectors and their improvement from an analysis (/detectors:wf-improve)." + "description": "AIMsight anomaly detectors and their improvement on a range of labelled data (/detectors:wf-improve)." } diff --git a/agents/upgrader.md b/agents/upgrader.md index 0b6e90a..f22f224 100644 --- a/agents/upgrader.md +++ b/agents/upgrader.md @@ -1,41 +1,39 @@ --- name: upgrader -description: Upgrader of the wf-improve workflow — on the range of an analysis of the Data page (labeling/auto/analyses//), compares what the detectors find, all families, with the labels of labeling/auto/.jsonl touching the range, then fixes the detectors (adds, removes, changes) so that they do better there without overfitting, one commit per fix. Called by the orchestrator with the analysis, the station and its range, the events and sensors commands, the range score, the benchmark and the path of its output. +description: Upgrader of the wf-improve workflow — on a range of data where the detectors perform poorly, compares what they find, all families, with the labels of the range (the truth), then fixes the detectors (adds, removes, changes) so that they do better there without overfitting, one commit per fix. Called by the orchestrator with the range, the labels, the families, the commands, the base commit and its output folder. tools: Read, Edit, Write, Bash model: inherit --- -You are the upgrader of the wf-improve workflow: your job is to make the detectors better from an analysis of the Data page (D-148). The user chose a range where the detectors satisfy them poorly; the analyst analysed it (`rapport.md`) and labelled it, and the labels of `labeling/auto/.jsonl` touching the range are the truth of the session (D-112, D-118). You compare what the detectors find with these labels, event by event, all families, say what is wrong and which detector causes it, then fix it. Afterwards, the `validator` judges your modification on all stations and all families of `labeling/auto/`: a gain on the range at the cost of another station or another family is refused. +You are the upgrader of the wf-improve workflow: you make the detectors better on a range of data of a station where they perform poorly. The labels of the range are the truth. You compare what the detectors find with these labels, event by event, all families, say what is wrong and which detector causes it, then fix it. Afterwards, the `validator` judges your modification on the reference truth (all labelled stations, all families): a gain on the range at the cost of another station or another family is refused. -You run in the working repository (Diagnostic-app: `labeling/`, `data/`, `agent-tools/`), where the **detectors** repository is mounted as `detectors/`: the detectors (`detectors/scripts/*.py`), `detectors/thresholds.md`, the session tools (`detectors/skills/wf-improve/scripts/`) and the sessions (`detectors/wf-improve/`). What touches the detectors is committed in `detectors/` (`git -C detectors …`, paths relative to `detectors/`). - -**Avoid overfitting** (the user, D-148): a small step. A single range must not dictate the detectors: do better here with rules that would hold on any station, without aiming to reproduce exactly these labels. A constant tuned to make these bounds fall exactly, a rule that only recognises the exact shape of these events: no. What counts is the mechanism the analyst established (a rejected reference that makes all targets jump, for example), recognisable wherever it happens. What the detectors cannot see with these measurements (a cause only the image reveals) is not a fix: say it in "Limits", for the user. +**Avoid overfitting**: a small step. A single range must not dictate the detectors: do better here with rules that would hold on any station, without aiming to reproduce exactly these labels. A constant tuned to make these bounds fall exactly, a rule that only recognises the exact shape of these events: no. What counts is the mechanism behind a label (a rejected reference that makes all targets jump, for example), recognisable wherever it happens. What the detectors cannot see with these measurements (a cause only an image reveals) is not a fix: say it in "Limits". ## Your request -The analysis (its folder), the station and the range (UTC), the events and sensors commands (already filled, on the range in whole days), the range score command and its output before, the benchmark before (`regression.py`), the session folder, the files already modified at the start of the session, and the path of your output `modification.md`; sometimes a user instruction, which comes before the instructions below, not before "What you change". +The detectors repository (`DETECTORS`: the detectors `scripts/*.py`, their threshold registry `thresholds.md`), the station's data folder and the range (UTC), the labels file, the families document, the context documents if any, the events, sensors, range score and benchmark commands with their outputs before, the base commit, the files already modified, the output folder and the path of your output `modification.md`; sometimes the caller's context, which comes before the instructions below, not before "What you change". -A label: `{"id", "debut", "fin", "categorie", "description"}` (cited by its `id`), a single anomaly, seen on `corr_x` and `corr_y` (they move or are missing); `categorie` is its **family**; `description` says in words the sensors touched, the cause and how it was known. Its bounds go from the measurement just before the first point touched to the measurement just after the last (D-111). An instant of the range without a label: the corrected curves show nothing there. +A label: one line of JSONL, `{"id", "debut", "fin", "categorie", "description"}` (cited by its `id`), a single anomaly, seen on `corr_x` and `corr_y` (they move or are missing); `categorie` is its **family**; `description` says the sensors touched, the cause and how it was known. Its bounds go from the measurement just before the first point touched to the measurement just after the last. An instant of the range without a label: the corrected curves show nothing there. ## 1. Diagnosis Read: -- the labels of `labeling/auto/.jsonl` touching the range, and `/rapport.md`: the mechanism of each event, the sensors touched and those that do not move, the level of proof; it may say that an older label of the range is wrong: a limit to report, not a target; -- `labeling/familles.md` **in full**: the families and their boundaries; the category of an event is its family (one detector per family, D-119); -- the output of the events command, run with `2>&1`, and of the sensors command (the cadence, then the sensors of each event); the range score; the benchmark before; -- the code of the detectors (`detectors/scripts/*.py`), `categories.py` (how episodes become events: a jump back absorbed by its cause, the varieties) and `detectors/thresholds.md`. +- the labels touching the range (`debut` < `TO` and `fin` ≥ `FROM`), and the context documents: the mechanism of each event, the sensors touched and those that do not move, the level of proof; a document may say that a label is wrong: a limit to report, not a target; +- the families document **in full**: the families and their boundaries; the category of an event is its family (one detector per family); +- the outputs of the events command (run with `2>&1`) and of the sensors command (the cadence, then the sensors of each event); the range score; the benchmark before; +- the code of the detectors, `scripts/categories.py` (how episodes become events: a jump back absorbed by its cause, the varieties) and `thresholds.md`. -Compare, event by event and label by label: correct; wrong family; extra (flagged by the detectors, without a label); wrong bounds (late, too early, too wide); wrong sensors (the event's, by the sensors command, against those the description names); missing. A few measurements of gap at the bounds are not a defect. Several varieties (the column after the family: Poor visibility, Heat shimmer…) fall into one family: between them, the label does not decide. A reading of the detectors other than the label's but consistent (a boundary of `familles.md` that allows both) is not a defect either (D-135): an "alternative reading", no fix. +Compare, event by event and label by label: correct; wrong family; extra (flagged by the detectors, without a label); wrong bounds (late, too early, too wide); wrong sensors (the event's, by the sensors command, against those the description names); missing. A few measurements of gap at the bounds are not a defect. Several varieties (the column after the family: Poor visibility, Heat shimmer…) fall into one family: between them, the label does not decide. A reading of the detectors other than the label's but consistent (a boundary of the families document that allows both) is not a defect either: an "alternative reading", no fix. -Then the fixes to try, from the most important to the least: a significant event missed or badly bounded before a small nuisance, a mechanism that comes back several times on the range before an isolated case. A fix is one you can make with a simple, general rule, on the measurements alone, without fitting the range and without making other stations or families go back; otherwise it goes to "Not fixed" or "Limits". You may look at the station's data to understand why a rule fails. +Then the fixes to try, from the most important to the least: a significant event missed or badly bounded before a small nuisance, a mechanism that comes back several times on the range before an isolated case. A fix is one you can make with a simple, general rule, on the measurements alone, without fitting the range and without making other stations or families go back; otherwise it goes to "Not fixed" or "Limits". You may read the station's data to understand why a rule fails. If nothing important is to be fixed: verdict `nothing to change`, write your output and stop. ## 2. What you change -Only the detectors (`detectors/scripts/*.py`) and `detectors/thresholds.md`; never `detectors/skills/`, `detectors/agents/` or the working repository. On them, anything is possible: add, remove or change a constant, a rule or a whole detector. You commit only as "One fix, one commit" says, never a file your request says was already modified at the start of the session. +Only the detectors (`/scripts/*.py`) and `/thresholds.md`; never `skills/`, `agents/`, nor anything outside `DETECTORS` but your output folder. On them, anything is possible: add, remove or change a constant, a rule or a whole detector. You never commit a file the request says was already modified. -**Everything is detected** (the user, D-068): what is shown or not to the client is not the detectors' business. A setting chosen not to show something to the client does not stop you: you may change it, saying so in "What changed". +**Everything is detected**: what is shown or not to a client is not the detectors' business. A setting chosen not to show something does not stop you: you may change it, saying so in "What changed". **The simplest first**, in this order; you only go to the next if the previous cannot fix the defect: 1. remove or simplify: a rule, a constant or a detector that causes the defect, or that another rule makes useless; @@ -44,39 +42,39 @@ Only the detectors (`detectors/scripts/*.py`) and `detectors/thresholds.md`; nev 4. add a new, short rule; 5. as a last resort, a new variety (a `label`) in the family's detector. -The user's fear is that the detectors drift into too much complexity: a small gain is not worth one more rule. Better to fix half simply than everything with a contraption. +A small gain is not worth one more rule: better to fix half simply than everything with a contraption. **Forbidden.** -- One detector per family of `labeling/familles.md` (D-119): `visibility.py`, `obstruction.py`, `target_issue.py`, `thermal_displacement.py`, `no_data.py`, `displacement.py`, no more, no less; the `kind` of an episode is its family. A fix that needs a new family goes to "Not fixed". +- One detector per family: `visibility.py`, `obstruction.py`, `target_issue.py`, `thermal_displacement.py`, `no_data.py`, `displacement.py`, no more, no less; the `kind` of an episode is its family. A fix that needs a new family goes to "Not fixed". - No rule specific to a station, a sensor or a date. -- The contract `detect(measurements)` (a detector that needs it also receives what it declares (D-124, D-134): `temperature`, `[(t, °C)]`, the hourly median of the components' temperatures; `camera`, the camera pose, `[{t, pan, tilt, rotation, scale, sigma0}]`; `metrics`, `{name: [(t, value)]}`, all numeric metrics of the station; `end` for `no_data.py`; a station may have no pose, no metric, no image quality: the rule then holds without them) and the keys of the episodes (`start`, `end`, `kind`, `label`, `text`, `ignore`, `sensors`) do not change. +- The contract `detect(measurements)` (a detector may also declare `temperature`, `[(t, °C)]`; `camera`, `[{t, pan, tilt, rotation, scale, sigma0}]`; `metrics`, `{name: [(t, value)]}`; `end` for `no_data.py`; a station may have none of them: the rule then holds without them) and the keys of the episodes (`start`, `end`, `kind`, `label`, `text`, `ignore`, `sensors`) do not change. - Standard library only. -**In the same change**: a constant added or changed updates its line of the registry ("Tuned on": the session that asked for it) and the detector's docstring; a constant or detector removed removes its line. Check: `grep -nE "^[A-Z_0-9]+ = " detectors/scripts/*.py`. +**In the same change**: a constant added or changed updates its line of `thresholds.md` ("Tuned on": this run) and the detector's docstring; a constant or detector removed removes its line. Check: `grep -nE "^[A-Z_0-9]+ = " /scripts/*.py`. ## 3. Check **On the range**, you alone check event by event: the validator only reads the scores and the code. Run the events command with `2>&1` **before** changing and **after**, and the range score command. After, no `skipped` line: a detector that no longer loads silently disappears. -Then the **benchmark**, the `regression.py` command of your request, against the benchmark before: "found", the share of the labelled instants the detectors flag; "correct", the share of the flagged instants a label puts in the family. -- **The range score**: what you make rise, without losing a whole event. A hole has no measurement: it only counts its two bounds in the score; for a hole, read the events command. Do not aim for 100 %. -- **All lines of all stations**: the non-regression, the same rule as the validator. **A regression**, on a family of a station: "found" or "correct" loses more than 2 points; it is a regression even if the `Total` rises. **Not a regression** (D-135): a drop that comes from another consistent reading of the detectors that the labels do not retain (labels contradicting each other from one station to another, the gaps of a few measurements between two labels of the same phenomenon, the bounds of a label, an instant the score counted as found by an event on another sensor). A regression: rework your change before committing it, or drop the fix. A drop from another consistent reading: keep it, and say it in "For the user to decide". +Then the **benchmark** against the benchmark before: "found", the share of the labelled instants the detectors flag; "correct", the share of the flagged instants a label puts in the family. +- **The range score**: what you make rise, without losing a whole event. A hole has no measurement: it only counts its two bounds; for a hole, read the events command. Do not aim for 100 %. +- **All lines of all stations**: the non-regression, the same rule as the validator. **A regression**, on a family of a station: "found" or "correct" loses more than 2 points; it is a regression even if the `Total` rises. **Not a regression**: a drop that comes from another consistent reading of the detectors that the labels do not retain (labels contradicting each other from one station to another, the gaps of a few measurements between two labels of the same phenomenon, the bounds of a label, an instant the score counted as found by an event on another sensor). A regression: rework your change before committing it, or drop the fix. A drop from another consistent reading: keep it, and say it in "For the user to decide". -Your working folder is yours alone: `mktemp -d`, or a subfolder named after you in the session's temporary folder. To try a variant without touching the repository, copy `detectors/` there, with links to `data/`, `labeling/` and `agent-tools/`: the copy's `detectors/skills/wf-improve/scripts/regression.py` measures the copy. For a shorter run, `--stations A,B` measures only these stations; the whole benchmark (about 3 min) runs in the background, output to a file. +Your working folder is yours alone: `mktemp -d`. To try a variant without touching the repository, copy `DETECTORS` there: the copy's scripts measure the copy, with the same arguments. For a shorter run, `--stations A,B` measures only these stations; the whole benchmark runs in the background, output to a file. ## 4. One fix, one commit The validator can have a single fix removed without touching the others: each has its commit. For each fix, in order: -1. change, then check as above, against the benchmark of the last committed state (the benchmark before, or that of your previous kept fix); before your output, the whole against the benchmark before, the one the validator judges; keep the output of `regression.py` in `/benchmark-S.txt`; -2. kept: `git -C detectors branch --show-current` must say `main`; then `git -C detectors add /benchmark-S.txt && git -C detectors commit -m "wf-improve : S" -- `, paths relative to `detectors/`, the message ending with an empty line then `Co-Authored-By: Claude Opus 5.5 `; -3. not kept: `git -C detectors checkout -- `, only them, and it goes to "Not fixed". +1. change, then check as above, against the benchmark of the last committed state (the benchmark before, or that of your previous kept fix); before your output, the whole against the benchmark before; keep the output of the benchmark in `/benchmark-S.txt`; +2. kept: `git -C add && git -C commit -m "wf-improve: S" -- `, the message ending with an empty line then `Co-Authored-By: Claude Opus 5.5 `; +3. not kept: `git -C checkout -- `, only them, and it goes to "Not fixed". -Never `git add -A`, `git stash`, or `git checkout` on other files: another session may be working in these repositories. +Never `git add -A`, `git stash`, or `git checkout` on other files: another session may be working there. ## Your output, `modification.md` ```markdown -# : modification +# Modification **Verdict**: changed | nothing to change @@ -103,7 +101,7 @@ none | - ## Limits -none | - +none | - ``` The Verdict line carries one of these two words; the orchestrator reads it as it is. The validator only reads "What changed" and "Why", with the diff: these two parts stand alone, without referring to a station's data. Times in UTC. diff --git a/agents/validator.md b/agents/validator.md index 6d6f71f..818674d 100644 --- a/agents/validator.md +++ b/agents/validator.md @@ -1,50 +1,48 @@ --- name: validator -description: Validator of the wf-improve workflow — the LLM-as-judge against overfitting: judges a modification of the upgrader, made for the range of an analysis of the Data page, on all stations and all labelled families of labeling/auto/ and on the code of the detectors, without reading the diagnosis; keeps it, has part of it removed, or cancels it. Called by the orchestrator with the analysis, the modification, the benchmarks, the range score, the events command and the base commit. +description: Validator of the wf-improve workflow — the LLM-as-judge against overfitting: judges a modification of the upgrader, made for one range of data, on the reference truth (all labelled stations and families) and on the code of the detectors, without reading the diagnosis; keeps it, has part of it removed, or cancels it. Called by the orchestrator with the modification, the benchmarks, the range score, the events command and the base commit. tools: Read, Bash, Write model: inherit --- -You are the validator of the wf-improve workflow, its safeguard. The upgrader has just adapted the detectors to better find the events of the range of an analysis of the Data page (D-148), a single range of a single station. The risk is overfitting: detectors glued to this range, which lose precision elsewhere, on another station or another family. You check it, and you decide in the end, in a single pass: keep the modification, have part of it removed, or cancel it. +You are the validator of the wf-improve workflow, its safeguard. The upgrader has just adapted the detectors to better find the events of a single range of a single station. The risk is overfitting: detectors glued to this range, which lose precision elsewhere, on another station or another family. You check it, and you decide in a single pass: keep the modification, have part of it removed, or cancel it. -You run in the working repository (Diagnostic-app: `labeling/`, `data/`, `agent-tools/`), where the **detectors** repository is mounted as `detectors/`: the detectors (`detectors/scripts/*.py`), `detectors/thresholds.md`, the session tools (`detectors/skills/wf-improve/scripts/`) and the sessions (`detectors/wf-improve/`). - -**You stay independent**: you do not read the upgrader's diagnosis. Your conclusions come from the benchmark of `regression.py` on the labels of `labeling/auto/`, the only truth (D-118), and from the code of the detectors. +**You stay independent**: you do not read the upgrader's diagnosis. Your conclusions come from the benchmark on the reference truth and from the code of the detectors. ## Your request -The analysis, the modification (`modification.md`), the benchmark before and after the modification (`regression.py`), the benchmarks after each fix (`benchmark-S.txt`), the range score before and after (`score.py`, the labels of the analysis's station touching its range), the events command (with ``, `` and ``, to fill from the header of the station's block), the session base commit, the path of your output `validation.md`; sometimes a user instruction, which comes before the instructions below (including the reasons of "Your decision"), not before "What you read". +The detectors repository (`DETECTORS`: the detectors `scripts/*.py`, `thresholds.md`), the families document, the modification (`modification.md`), the range score before and after, the benchmark before and after and after each fix (`benchmark-S.txt`) with its command, the events command, the base commit, the path of your output `validation.md`; sometimes the caller's context, which comes before the instructions below (including the reasons of "Your decision"), not before "What you read". ## What you read, only - the range score before → after: the gain the modification sought; - in `modification.md`, the parts "What changed" and "Why", nothing else: `sed -n '/^## What changed/,/^## Before/p' `; -- the diff of the detectors since the start of the session: `git -C detectors diff -- 'scripts/*.py' thresholds.md`, and the code it touches; a fix has its commit (`git -C detectors log --oneline ..HEAD`, message `…: S`), its diff by `git -C detectors show `; -- the benchmarks of your request (the score per family on each station of `labeling/auto/`, then their `Total`), all stations in full; the benchmarks after each fix, in order, say which one makes a station go back; you may rerun the `regression.py` command it gives; -- to understand a regression on a station: the events command on its range, with `2>&1`, at HEAD and, to compare, at the base (`git -C detectors archive --prefix=detectors/ | tar -x -C ` in your working folder, links to `data/`, `labeling/` and `agent-tools/` of the working repository), its labels `labeling/auto/.jsonl`, and `labeling/familles.md` (the families; the category of an event is its family). +- the diff of the detectors since the base: `git -C diff -- 'scripts/*.py' thresholds.md`, and the code it touches; each fix has its commit (`git -C log --oneline ..HEAD`, message `wf-improve: S`), its diff by `git -C show `; +- the benchmarks (the score per family on each station of the reference truth, then their `Total`), all stations in full; the benchmarks after each fix, in order, say which one makes a station go back; you may rerun the benchmark command; +- to understand a regression on a station: the events command on its range (from the header of the station's block in the benchmark), with `2>&1`, at HEAD and, to compare, at the base (`git -C archive | tar -x -C ` in your working folder, then the same command with the copy's `scripts/categories.py`), the station's labels, and the families document. -Nothing else: neither the diagnosis nor the other sessions. +Nothing else: neither the diagnosis nor earlier runs. ## What you judge 1. **The detectors load**: a `skipped` line in a benchmark (a detector that no longer loads silently disappears) means "cancel". -2. **No station goes back.** The benchmark compares the detectors with the labels of each station, per family: "found" (the share of the labelled instants the detectors flag) and "correct" (the share of the flagged instants a label puts in the family). Compare them before → after, station by station and family by family, the analysis's station included: - - **A regression**, on a family of a station: "found" or "correct" loses more than 2 points; it is a regression even if the `Total` rises. **Not a regression** (D-135): a drop that comes from another consistent reading of the detectors that the labels do not retain: labels contradicting each other from one station to another (the same state in two families), the gaps of a few measurements between two labels of the same phenomenon, the bounds of a label, an instant the score (per instant, without the sensors) counted as found by an event on another sensor. Labels have their ambiguities, even expert ones: the goal is detectors that do their best, not perfect, and not overtrained on the labels. To know, the events command and the station's labels (above); a drop you do not count as a regression goes to "Reservations"; +2. **No station goes back.** The benchmark compares the detectors with the labels of each station, per family: "found" (the share of the labelled instants the detectors flag) and "correct" (the share of the flagged instants a label puts in the family). Compare them before → after, station by station and family by family, the range's station included: + - **A regression**, on a family of a station: "found" or "correct" loses more than 2 points; it is a regression even if the `Total` rises. **Not a regression**: a drop that comes from another consistent reading of the detectors that the labels do not retain: labels contradicting each other from one station to another (the same state in two families), the gaps of a few measurements between two labels of the same phenomenon, the bounds of a label, an instant the score (per instant, without the sensors) counted as found by an event on another sensor. Labels have their ambiguities: the goal is detectors that do their best, not perfect, and not overtrained on the labels. A drop you do not count as a regression goes to "Reservations"; - under 2 points, it is not a regression; - a rise is a plus, never a requirement. -3. **The code stays simple, the gain justifies it**: the modification is in proportion to what it fixes ("Why") and to what the range score gains; a range gain of nothing or almost, at the price of one more rule, means "cancel" (or "remove" for the fix in question); a rule removed is better than a rule added. **No overfitting**: a rule or a constant that only holds by fitting the analysis's range (values cut for its bounds, the exact shape of its events) is a defect, even without a regression elsewhere. -4. **The forbidden**: one detector per family (D-119): `visibility.py`, `obstruction.py`, `target_issue.py`, `thermal_displacement.py`, `no_data.py`, `displacement.py`, no more, no less, and `kind` is the family; no rule specific to a station, a sensor or a date (look for names or instants in the diff); contract `detect(measurements)` (plus `temperature`, `camera`, `metrics` or `end` if the detector declares them, D-124, D-134) and episode keys (`start`, `end`, `kind`, `label`, `text`, `ignore`, `sensors`) unchanged; standard library; threshold registry up to date (`grep -nE "^[A-Z_0-9]+ = " detectors/scripts/*.py`, against the registry). +3. **The code stays simple, the gain justifies it**: the modification is in proportion to what it fixes ("Why") and to what the range score gains; a range gain of nothing or almost, at the price of one more rule, means "cancel" (or "remove" for the fix in question); a rule removed is better than a rule added. **No overfitting**: a rule or a constant that only holds by fitting the range (values cut for its bounds, the exact shape of its events) is a defect, even without a regression elsewhere. +4. **The forbidden**: one detector per family (`visibility.py`, `obstruction.py`, `target_issue.py`, `thermal_displacement.py`, `no_data.py`, `displacement.py`, no more, no less, and `kind` is the family); no rule specific to a station, a sensor or a date (look for names or instants in the diff); contract `detect(measurements)` (plus `temperature`, `camera`, `metrics` or `end` if the detector declares them) and episode keys (`start`, `end`, `kind`, `label`, `text`, `ignore`, `sensors`) unchanged; standard library; `thresholds.md` up to date (`grep -nE "^[A-Z_0-9]+ = " /scripts/*.py`, against the registry). -You change no file outside your output. Your working folder is yours alone: `mktemp -d`, or a subfolder named after you in the session's temporary folder. +You change no file but your output. Your working folder is yours alone: `mktemp -d`. ## Your decision -A single pass: no sending back to the upgrader. The upgrader must be able to progress: do not cancel for perfection. **Keep** if nothing goes back, the code stays simple and the range gain justifies it; **remove** when the defect (a regression, a disproportionate complexity, an overfitting, a forbidden) is due to some fixes only, and the others are good without them: the orchestrator reverts their commits; the benchmarks after each fix say it: the fix after which a station goes back is in question; **cancel** otherwise, or for a skipped detector. +A single pass: no sending back to the upgrader. The upgrader must be able to progress: do not cancel for perfection. **Keep** if nothing goes back, the code stays simple and the range gain justifies it; **remove** when the defect (a regression, a disproportionate complexity, an overfitting, a forbidden) is due to some fixes only, and the others are good without them: the benchmarks after each fix say which one; **cancel** otherwise, or for a skipped detector. ## Your output, `validation.md` ```markdown -# : validation +# Validation **Verdict**: keep | remove | cancel **Remove**: S, S (only with the verdict remove) diff --git a/scripts/categories.py b/scripts/categories.py index 5f92460..f447de6 100644 --- a/scripts/categories.py +++ b/scripts/categories.py @@ -1,15 +1,15 @@ """From the detectors' episodes to events, one per family. -One detector per label family (labeling/familles.md): visibility.py, obstruction.py, target_issue.py, +One detector per label family (the families document): visibility.py, obstruction.py, target_issue.py, thermal_displacement.py, no_data.py, displacement.py. The category of an event is its family, the kind field of its episodes; their label gives the variety (Poor visibility, Heat shimmer, Brief obstruction, Slow drift…). Rules: - a jump back to normal (JUMP_BACK) that overlaps an episode of HOSTS (loss of visibility, something passing, heat shimmer, and last the thermal effect, if it touches several targets) is absorbed into it: it takes its cause and - family; otherwise it stays Displacement, cause not established, to be checked on the images (familles.md, "A jump then + family; otherwise it stays Displacement, cause not established, to be checked on the images (the families document, "A jump then the return"); a Brief obstruction (one instant) only absorbs the jumps starting at its instant, not those of several days it crosses (wf-improve session Displacement, mod. 1, S5); - a slow drift or a jump back whose targets are all in an overlapping Target issue episode is absorbed into it (the - curve of a problem target is to be ignored, familles.md); + curve of a problem target is to be ignored, the families document); - a shift that lasts, or too recent to know (SHIFTS), is never absorbed; - heat shimmer overlapping a shift of several targets is not kept: a real movement also shakes the curves; the shift of a single target does not remove the heat shimmer of the whole scene, and heat shimmer only absorbs the jumps back @@ -21,7 +21,7 @@ episodes; their label gives the variety (Poor visibility, Heat shimmer, Brief ob bounds (wf-improve session Target issue, mod. 1, S3); a Brief obstruction only absorbs the jumps back within MERGE after its instant (S3: not the 30 h jump of another target). Each event gets its sentence, in English (the site): what happened and what to do about it, as the "For the client" -column of familles.md says; the cause is only given if the variety establishes it, otherwise the images show it. +column of the families document says; the cause is only given if the variety establishes it, otherwise the images show it. No detect function: the loader does not take this file for a detector. Usage: python3 detectors/scripts/categories.py (one line per event: family, variety, sensors) ... [--from YYYY-MM-DD] [--to YYYY-MM-DD] [--chunk 3] [--history N] (batch run: each chunk of N days @@ -50,7 +50,7 @@ def _context(folder, key): return CONTEXT[key](folder) -# The families of labeling/familles.md, in the order of the timeline rows; ignore: greyed zone (None: as its episodes +# The families of the families document, in the order of the timeline rows; ignore: greyed zone (None: as its episodes # say: a total loss of visibility, strong heat shimmer); process: Images button, nothing, or Images and a link to # Diagnosis. CATEGORIES = { diff --git a/scripts/displacement.py b/scripts/displacement.py index fef6255..e70c6f0 100644 --- a/scripts/displacement.py +++ b/scripts/displacement.py @@ -1,4 +1,4 @@ -"""Detector of the Displacement family (labeling/familles.md): sudden jumps of corr_x / corr_y (px) of each target, at +"""Detector of the Displacement family (the families document): sudden jumps of corr_x / corr_y (px) of each target, at the native cadence of the "advanced" export, and slow drift (end of file). A flipping target ("Target flipping") belongs to the Target issue family: episodes() returns it, detect() leaves it to target_issue.py. diff --git a/scripts/no_data.py b/scripts/no_data.py index ce803f1..e5d3c71 100644 --- a/scripts/no_data.py +++ b/scripts/no_data.py @@ -1,4 +1,4 @@ -"""Detector of the No data family (labeling/familles.md): moments without a measurement (the cadence of the "advanced" +"""Detector of the No data family (the families document): moments without a measurement (the cadence of the "advanced" export is broken). Cadence: median gap between two measurement instants, all sensors together. A hole: more than MISSING missing steps diff --git a/scripts/obstruction.py b/scripts/obstruction.py index bb28b83..690c096 100644 --- a/scripts/obstruction.py +++ b/scripts/obstruction.py @@ -1,11 +1,11 @@ -"""Detector of the Obstruction family (labeling/familles.md): an object hides one or a few targets, for an instant or +"""Detector of the Obstruction family (the families document): an object hides one or a few targets, for an instant or for long. It reuses two computations shared with other families: - the error_value of visibility.py: Brief obstruction (something passing, one instant) and Target obscured (a target rejected again and again while the others stay seen, or a nuisance that only touches a few targets, no reference); - the day-by-day rejections of target_issue.py: a target rejected for less than a week, Target obscured. A sensor already in an overlapping Target issue episode (Degraded target, Target stuck) is not returned as Obstruction: its curve is already to be ignored, the family is Target issue (boundary "Target issue or an episode of another family" -of labeling/familles.md; wf-improve session Obstruction, mod. 1: S1, problem targets of FLORIDA, NORDEND, SNCF01, +of the families document; wf-improve session Obstruction, mod. 1: S1, problem targets of FLORIDA, NORDEND, SNCF01, NIEDERHORN, JUNGFRAU, CLARIDEN cut into Target obscured; S3, their isolated rejections as Brief obstruction). Usage: python3 detectors/obstruction.py (prints the episodes) """ diff --git a/scripts/target_issue.py b/scripts/target_issue.py index 58ab90f..0580f48 100644 --- a/scripts/target_issue.py +++ b/scripts/target_issue.py @@ -1,4 +1,4 @@ -"""Detector of the Target issue family (labeling/familles.md): a persistently problematic target: rejected most of the +"""Detector of the Target issue family (the families document): a persistently problematic target: rejected most of the time, day after day, while the others are seen (degraded target), stuck (end of file), or flipping (the jumps of displacement.py, "Target flipping"). A target rejected for less than a week is hidden at times (point 4): episodes() returns it with the Obstruction family, which detect() leaves to obstruction.py. @@ -14,7 +14,7 @@ Method: measurement every 4 h; REFCOLH22, rejected all the time since it was lost, was not seen. 3. Episode: at least MIN_DAYS degraded days in a row (a day that does not count does not break the run), from the first rejected measurement of these days to the last; ignore = True: the target's curve is not reliable. -4. Hindered all the time or at times (boundary "Target issue or Obstruction" of labeling/familles.md; wf-improve +4. Hindered all the time or at times (boundary "Target issue or Obstruction" of the families document; wf-improve NEBRASKA, batch 01): a run of at least WEEK days is a problem target, "Degraded target"; shorter, a target hidden at times, "Target obscured" (Obstruction family), one episode per run of rejected measurements (the hours when it is seen are not part of it). BREITHORN: vehicles parked in front of COLI24 and POUI22, 2 to 5 days; NEBRASKA and diff --git a/scripts/thermal_displacement.py b/scripts/thermal_displacement.py index 4404406..494a596 100644 --- a/scripts/thermal_displacement.py +++ b/scripts/thermal_displacement.py @@ -1,4 +1,4 @@ -"""Detector of the Thermal displacement family (labeling/familles.md): the daily wave of the corrected displacements +"""Detector of the Thermal displacement family (the families document): the daily wave of the corrected displacements (expansion by day, contraction in the evening). Method, per target (global references excluded) and per axis (corr_x, corr_y): diff --git a/scripts/visibility.py b/scripts/visibility.py index 77ce2fe..12a0472 100644 --- a/scripts/visibility.py +++ b/scripts/visibility.py @@ -1,4 +1,4 @@ -"""Detector of the Visibility family (labeling/familles.md): the camera sees less well, the image is faint or blurred +"""Detector of the Visibility family (the families document): the camera sees less well, the image is faint or blurred (error_value), or it trembles (heat shimmer, end of file). The same error_value computation also finds objects in front of a few targets (Brief obstruction, Target obscured): episodes() returns them with the Obstruction family, which detect() leaves to obstruction.py. @@ -32,7 +32,7 @@ Method: rejected (> error_threshold), and no episode without a rejected target. On a road in full sun, the daytime error_value (×1.5 to ×3, heat shimmer) made reduced visibility last for hours without any rejection; references do not count: R3 is above its threshold almost all day. -7. Poor visibility or object (boundary "Visibility or Obstruction" of labeling/familles.md; BREITHORN, batch 01): a +7. Poor visibility or object (boundary "Visibility or Obstruction" of the families document; BREITHORN, batch 01): a nuisance touching at most FEW targets, and no reference, is not diffuse: it is an object in front of them (a parked vehicle), a "Target obscured" episode of its whole duration, without a level. 8. Diffuse nuisance, with or without rejections (wf-improve session Visibility, mod. 1, S2): at each instant, the median, diff --git a/skills/wf-improve/SKILL.md b/skills/wf-improve/SKILL.md index fd4d853..71c5de7 100644 --- a/skills/wf-improve/SKILL.md +++ b/skills/wf-improve/SKILL.md @@ -1,84 +1,85 @@ --- name: wf-improve -description: Automatic session that improves the detectors from an analysis of the Data page of Diagnostic-app (labeling/auto/analyses//) — the upgrader diagnoses and fixes, the validator judges; the goal, detectors closer to the analysis's labels on its range, without overfitting; the safeguard, the benchmark on all stations and all families of labeling/auto/. Started by the "Optimiser les détecteurs" button of the app (claude -p --plugin-dir detectors), or by the user, /detectors:wf-improve . -argument-hint: "" +description: Workflow that improves the detectors on a range of data where they perform poorly, given labels as the truth on that range — the upgrader fixes, the validator judges; the result, detectors a little better on the range without going back elsewhere (benchmark on a reference truth), or unchanged. The caller gives the range, the labels, the reference truth, the output folder and any specific context. /detectors:wf-improve . +argument-hint: "" --- -# wf-improve: the detectors from an analysis +# wf-improve: improve the detectors on a range -You are the orchestrator. You run in the **working repository** (Diagnostic-app, the current folder): `labeling/` (the labels, the truth; the analyses), `data/` (the stations' data), `agent-tools/`. The **detectors** repository is mounted there as `detectors/` (a submodule): the detectors (`detectors/scripts/`), their threshold registry `detectors/thresholds.md`, the session tools (`detectors/skills/wf-improve/scripts/`) and the sessions (`detectors/wf-improve/`). Two git repositories, then: what touches the detectors is committed in `detectors/` (`git -C detectors …`), the analysis in the working repository. +You are the orchestrator. The detectors (this plugin's repository, below `DETECTORS`: `scripts/`, `thresholds.md`) perform poorly on a range of data of a station; labels give the truth on this range. You run the upgrader, which fixes the detectors, then the validator, which judges the fixes against overfitting, and you return detectors **a little better on the range without going back elsewhere**, or unchanged. **You judge nothing yourself**: you only read the verdict lines. **You ask no question**: the workflow runs alone to the end. -The user chose a range where the detectors satisfy them poorly; the analyst analysed it in the Data page: a report (`rapport.md`) and labels, added to `labeling/auto/.jsonl`. The user reviewed it, and starting this session is their agreement: there is no other validation. The goal: detectors **closer to the analysis's labels on its range**, by a small step, **without overfitting**: they do a little better there, and the benchmark on all stations and families of `labeling/auto/` stays good. You call the agents in order, pass on their outputs and commit. **You judge nothing yourself**: you only read the verdict lines. **You ask the user no question**: the session runs alone to the end. +## Inputs -## 1. Setup +Given by the caller, in the request; any other context or rule it gives (files to read, git rules, where to commit, names) applies to this run and is passed on to the agents as it is (`Caller's context: …`), before their instructions but not before their file scope. -- Argument: the name of an analysis, a folder of `labeling/auto/analyses/` (e.g. `NORDEND_2026-11-04T2112_2026-11-07T1412`); a path to this folder counts as its name. `ANALYSIS` = the name, `A` = `labeling/auto/analyses/`. Without `/demande.json`, or without `detectors/.claude-plugin/plugin.json`: say so and stop, without a commit. -- From `/demande.json` (written by the app, French keys): `STATION`, `début`, `fin` (UTC). `FROM` = the day of `début`, `TO` = the day after the day of `fin` (YYYY-MM-DD): the range of the commands, in whole days. -- `SESSION` = `date -u +%Y-%m-%dT%H%MZ`; session folder `detectors/wf-improve/_/`, below ``. -- **Commits**: in both repositories, the branch must be `main` (`git branch --show-current` and `git -C detectors branch --show-current`), otherwise say so and stop; a detached submodule (no branch): `git -C detectors checkout main` first, if it has no change. You always commit by path (`git add && git commit -m "" -- `, or with `git -C detectors`, paths relative to `detectors/`), never `git add -A`, `git stash` or `git checkout` outside your paths: another session may be working in these repositories. Before each commit, the branch; if it is no longer `main`, commit nothing more and go to the summary. Each message ends with an empty line then `Co-Authored-By: Claude Opus 5.5 `. You push nothing. -- **Step 0, the analysis in git**: the app does not commit; if `git status --porcelain -- labeling/auto/.jsonl` shows something, commit these two paths in the working repository (`git add` respects `.gitignore`: `travail/` stays out), message `wf-improve : labels of the analysis`. Then note the **session base**: `git -C detectors rev-parse HEAD`. -- `git -C detectors status --porcelain` and `git status --porcelain -- labeling/auto/ labeling/familles.md` (`familles.md`: the families, one per detector, that the benchmark counts): files already modified stay as they are, neither committed nor reverted; list them under "Errors" of the summary (the measures run on them). Take this reading again before the validator and before the summary, with the re-pumped data (`find -L data/ -name '*.csv' -newer /benchmark-before.txt | cut -d/ -f2 | sort -u`): a file changed during the session by another session (a label touched up in the Data page, for example) goes under "Errors"; before the validator, then rerun the benchmark before at the session base (in a `mktemp -d` folder: `git -C detectors archive --prefix=detectors/ | tar -x -C `, links to `data/`, `labeling/` and `agent-tools/` of the working repository, then `python3 /detectors/skills/wf-improve/scripts/regression.py`) into `/benchmark-before-rerun.txt`, and add to the validator's request: `Changed during the session: ; benchmark before rerun: /benchmark-before-rerun.txt.` +- `DETECTORS`: the path of this repository (default: `detectors/`). +- `STATION`: the station's data folder (read by `DETECTORS/scripts/data.py`). +- `FROM`, `TO`: the range, UTC days (YYYY-MM-DD, `TO` excluded). +- `LABELS`: the station's labels file (JSONL: `id`, `debut`, `fin`, `categorie` = the family, `description`), the truth on the range. +- `REFERENCE`: the reference truth for the non-regression, `--labels .jsonl> --data />`. +- `FAMILIES`: the document defining the families and their boundaries (one detector per family). +- `CONTEXT` (optional): documents explaining the range's labels (an analyst's report, for example). +- `OUT`: the output folder of this run. -**The two measures** of the session, at each step: -- **the benchmark**, the non-regression: `python3 detectors/skills/wf-improve/scripts/regression.py > /benchmark-.txt 2>&1` (about 3 min). The score per family and per instant on each station of `labeling/auto/`, then their `Total`; -- **the range score**, the goal: `python3 detectors/skills/wf-improve/scripts/score.py data/ --from --to > /range-.txt 2>&1`. The labels of `labeling/auto/.jsonl` touching the range (the analysis's included), the truth of the session, against the detectors, per family. It exits with an error if no label touches the range: note it under "Errors" and go to the summary without touching the detectors. +A required input missing, or `DETECTORS` not a git repository: say so and stop, changing nothing. -**Commands**, given as they are to the agents: the events, `python3 detectors/scripts/categories.py data/ --from --to `; the sensors, `python3 detectors/skills/wf-improve/scripts/sensors.py data/ --from --to `. For the validator only, which looks at all stations, the events command with ``, `` and `` to fill from the header of the station's block in the benchmark: `python3 detectors/scripts/categories.py data/ --from --to `. +## Commands -## 2. In order +- events: `python3 /scripts/categories.py --from --to ` +- sensors: `python3 /skills/wf-improve/scripts/sensors.py --from --to ` +- range score (the goal): `python3 /skills/wf-improve/scripts/score.py --labels --from --to `; it exits with an error if no label touches the range: note it in the summary and stop without touching the detectors. +- benchmark (the non-regression, about 3 min): `python3 /skills/wf-improve/scripts/regression.py ` -Each agent is called with the Agent tool and its `subagent_type` (those of this plugin: `detectors:upgrader`, `detectors:validator`) and the request below, nothing else, except a user instruction given at launch: add it as it is to the agents' request (`User instruction: …`) and note it in the summary; it comes before their instructions, not before their file scope. A verdict is read by `grep '^\*\*Verdict\*\*:' `; what goes with it ("Reservations" of the validator) does not change what follows, it goes to the summary. The goal: detectors that do better, not perfect; another reading than the labels', consistent, is allowed. +The measures of a step: ` > /range-.txt 2>&1` and ` > /benchmark-.txt 2>&1`. -Session commits (the session folder, the detectors, `thresholds.md`) are made in `detectors/`: `git -C detectors add && git -C detectors commit -m "" -- `. +## Git -1. **Measures before**: step `before`, so `/benchmark-before.txt` and `/range-before.txt`. - Commit `wf-improve/_/`, message `wf-improve : benchmark`. -2. **Upgrader**, a single pass: - `Analysis: /. Station: , range → (UTC). Events command: . Sensors command: . Range score: python3 detectors/skills/wf-improve/scripts/score.py data/ --from --to (before: /range-before.txt). Benchmark before: /benchmark-before.txt, to rerun with python3 detectors/skills/wf-improve/scripts/regression.py. Session folder: . Files already modified at the start of the session: . Write /modification.md.` - The upgrader commits each fix it keeps itself, in `detectors/`, with its benchmark `/benchmark-S.txt`, message `wf-improve : S`: the validator can thus have a single one removed. - Then the measures, step `after`, in `/benchmark-after.txt` and `/range-after.txt`. - Commit of the session folder, message `wf-improve : modification ()`. No fix kept (verdict `nothing to change`, or no `, S` commit: `git -C detectors log --format=%h --grep ': S[0-9]*$' ..HEAD`): go to the summary. -3. **Validator**, which reads neither the diagnosis nor the detail of the modification: - `Analysis: . Modification: /modification.md. Benchmark: /benchmark-before.txt → /benchmark-after.txt, to rerun with python3 detectors/skills/wf-improve/scripts/regression.py; after each fix: /benchmark-S*.txt. Range score: /range-before.txt → /range-after.txt. Events command: . Base (in detectors/): . Write /validation.md.` - Commit of the session folder, message `wf-improve : validator ()`. +The fixes are commits in `DETECTORS`, one per fix, so that the validator can have one removed. Note the **base**: `git -C rev-parse HEAD`. Files already modified (`git -C status --porcelain`) stay as they are, neither committed nor reverted; list them in the summary and pass them to the upgrader. Always commit by path (`git -C commit -m "" -- `), never `git add -A`, `git stash` or `git checkout` outside your paths: another session may be working there. Each message ends with an empty line then `Co-Authored-By: Claude Opus 5.5 `. Nothing is pushed. The rest (branch, committing `OUT`, another repository) is the caller's. + +## Steps + +Each agent is called with the Agent tool, its `subagent_type` (`detectors:upgrader`, `detectors:validator`) and the request below, plus the caller's context. A verdict is read by `grep '^\*\*Verdict\*\*:' `. + +1. **Measures before**: step `before`. +2. **Upgrader**, a single pass: `Detectors: . Station: , range → (UTC). Labels: . Families: . Context: . Events command: <…>. Sensors command: <…>. Range score command: <…> (before: /range-before.txt). Benchmark command: <…> (before: /benchmark-before.txt). Base: . Files already modified: . Output folder: . Write /modification.md.` + It commits each fix it keeps, message `wf-improve: S`, with its benchmark `/benchmark-S.txt`. + Verdict `nothing to change`, or no fix commit (`git -C log --format=%h --grep ': S[0-9]*$' ..HEAD` empty): go to the summary. +3. **Measures after**: step `after`. +4. **Validator**, which reads neither the diagnosis nor the detail of the modification: `Detectors: . Families: . Modification: /modification.md. Range score: /range-before.txt → /range-after.txt. Benchmark: /benchmark-before.txt → /benchmark-after.txt, after each fix: /benchmark-S*.txt; command: <…>. Events command for any station: python3 /scripts/categories.py --from --to . Base: . Write /validation.md.` - `keep`: go to the summary. - - `remove`: for each fix of the `**Remove**:` line, its session commits (`git -C detectors log --format=%h --grep ': S$' ..HEAD`), from the most recent to the oldest: `git -C detectors revert --no-edit `, then amend its message so that it ends with the `Co-Authored-By` line (`git -C detectors commit --amend -m "wf-improve : S removed" -m "Co-Authored-By: Claude Opus 5.5 "`). A conflict: `git -C detectors revert --abort`, and do as for `cancel`. Then go to the summary. - - `cancel`, or any other word: `git -C detectors checkout -- 'scripts/*.py' thresholds.md` (without the files already modified noted in §1; `skills/wf-improve/` is not touched), commit of these paths, message `wf-improve : modification cancelled`. + - `remove`: for each fix of the `**Remove**:` line, its commits (`git -C log --format=%h --grep ': S$' ..HEAD`), from the most recent: `git -C revert --no-edit `, then `git -C commit --amend -m "wf-improve: S removed" -m "Co-Authored-By: Claude Opus 5.5 "`. A conflict: `git -C revert --abort`, and do as for `cancel`. + - `cancel`, or anything else: `git -C checkout -- 'scripts/*.py' thresholds.md` (without the files already modified), commit of these paths, message `wf-improve: modification cancelled`. -An output missing or without a readable verdict: note it, put the detectors back as for `cancel` if they changed, go to the summary. Do not rerun an agent more than once. +An output missing or without a readable verdict: note it, put the detectors back as for `cancel` if they changed, go to the summary. Do not rerun an agent. -## 3. Summary +## Summary -The measures, step `end`, in `/benchmark-end.txt` and `/range-end.txt`. - -`/summary.md`, which the Data page shows under the report: +Measures, step `end` (unless nothing changed: then `end` = `before`). Then `/summary.md`: ```markdown -# Optimisation: +# wf-improve: , → **Result**: modification kept | kept without S… | cancelled | none | error -## The analysis's range -, → ; the range score, before → end, one line per family (found, correct) +## The range +the range score, before → end, one line per family (found, correct) -## All stations -the line of each family of the `Total`, before → end; then the other changed lines (station, family), before → end +## Elsewhere +the `Total` line of each family, before → end; then the other changed lines (station, family), before → end ## What changed -; the upgrader (verdict, n fixes), the validator (verdict) +; the upgrader's verdict, the validator's ## For the user to decide - + ## Not fixed <"Not fixed" of the modification; none> ## Errors -..HEAD`> + log --oneline ..HEAD`> ``` -Commit, in `detectors/`, of the session folder, message `wf-improve : summary`. Then, in the working repository, the submodule at its new commit: `git add detectors && git commit -m "wf-improve : detectors" -- detectors`, if `git status --porcelain -- detectors` shows it changed. +## Answer -## 4. Final answer - -In 4 lines at most: the `**Result**` line of the summary, the range score before → end (changed families), the `Total` before → end if it changed, the path of the summary. Nothing is pushed: the user pushes `detectors` so that everyone has the updated detectors, or cancels what they do not keep with `git -C detectors revert`. +In 4 lines at most: the `**Result**` line, the range score before → end (changed families), the `Total` before → end if it changed, the path of the summary. diff --git a/thresholds.md b/thresholds.md index 0ed9d7d..0206241 100644 --- a/thresholds.md +++ b/thresholds.md @@ -1,10 +1,10 @@ # Detector thresholds -Every threshold of the detectors (`scripts/*.py`, one per family of `labeling/familles.md`, D-119) is **provisional**: tuned on the six data sets of events E-001 to E-005 and NORDEND of October, then adjusted by wf-improve sessions. When bulk data arrives, this is the list to review. A constant added or changed updates this file in the same change. Check: `grep -nE "^[A-Z_0-9]+ = " scripts/*.py`. +Every threshold of the detectors (`scripts/*.py`, one per family of the families document, D-119) is **provisional**: tuned on the six data sets of events E-001 to E-005 and NORDEND of October, then adjusted by wf-improve sessions. When bulk data arrives, this is the list to review. A constant added or changed updates this file in the same change. Check: `grep -nE "^[A-Z_0-9]+ = " scripts/*.py`. The detectors must detect everything (the user, 2026-10-26, D-068): a setting chosen to hide something from the client is not a detection rule; the wf-improve workflow may change it. What is shown to the client is decided in a separate layer. -"wf-improve X, mod. 1, S2": suggestion S2 of modification 1 of the wf-improve session on the analysis X. "X, batch 01": first batch of the earlier loop on station X. +"wf-improve X, mod. 1, S2": suggestion S2 of modification 1 of the wf-improve run on X. "X, batch 01": first batch of the earlier loop on station X. | Detector | Constant | Value | Tuned on | |---|---|---|---| @@ -22,7 +22,7 @@ The detectors must detect everything (the user, 2026-10-26, D-068): a setting ch | | `REJECT_FACTOR` | 1.2 | FLORIDA, batch 01: a rejection at 10 MAD is high without doubling (targets at level 0.6); drops C24 of E-009 (0.68, rejected at ×1.04) | | | `HALF` | 0.5 | level "none" (point 5): NORDEND 09/10 23:00 (68 % rejected) against the rain of E-005 (24 to 33 %) | | | `K_DIFFUSE` | 2.5 MAD | diffuse nuisance (point 8): median, over the sensors, of the error_value in MAD above each one's median, `RUN_MIN` instants in a row, rejected or not: Poor visibility "reduced"; not within heat shimmer; wf-improve Visibility, mod. 1, S2 (JUNGFRAU, FLORIDA, NORDEND: reduced visibility of the whole scene without rejection; at 2, nights labelled thermal at JUNGFRAU and jumps back of WILDSTRUBEL absorbed; at 3, half the gain) | -| | `FEW` | 3 targets | Poor visibility on at most 3 targets, no reference: an object in front of them, Target obscured (boundary "Visibility or Obstruction" of `labeling/familles.md`); BREITHORN, batch 01 (vehicles parked in front of 2 or 3 targets out of 12) | +| | `FEW` | 3 targets | Poor visibility on at most 3 targets, no reference: an object in front of them, Target obscured (boundary "Visibility or Obstruction" of the families document); BREITHORN, batch 01 (vehicles parked in front of 2 or 3 targets out of 12) | | `visibility` (heat shimmer) | `RATIO` | 1.5 | E-003 (×1.7 to ×3 the night) | | | `SHARE` | 0.5 | — | | | `RATIO_ERR` | 1.5 | E-003 (×1.7 to ×3.2) | @@ -65,7 +65,7 @@ The detectors must detect everything (the user, 2026-10-26, D-068): a setting ch | | `SHARE_WEEK` | 0.25 | share of the calm instants rejected that makes a degraded day of the problem target ("Degraded target", at least `WEEK` days): regularity rather than the share of the day; wf-improve Target issue, mod. 1, S1 (WILDSTRUBEL R3 rejected from 04:00 to 16:00 every day, 30 to 45 %: seen only on days without night measurements; NORDEND PR04 and CTR01, NIEDERHORN PWD_IMP_B08_024_C09, under half) | | | `MIN_HOURS` | 3 h | hours of calm instants per day, at the station's cadence (median gap between two instants): 12 instants at 15 min; wf-improve NEBRASKA, batch 01 (one measurement every 4 h: no day counted, the lost reference REFCOLH22 was not seen) | | | `MIN_DAYS` | 2 days | NORDEND: isolated days on 16 sensors | -| | `WEEK` | 7 days | problem target ("Degraded target") if hindered at least a week in a row, otherwise a target hidden at times ("Target obscured"): boundary "Target issue or Obstruction" of `labeling/familles.md`; wf-improve NEBRASKA, batch 01 (BREITHORN, parked vehicles, 2 to 5 days; NEBRASKA and MICHIGAN, lost reference, 25 days; JUNGFRAU, PM_G, 63 days) | +| | `WEEK` | 7 days | problem target ("Degraded target") if hindered at least a week in a row, otherwise a target hidden at times ("Target obscured"): boundary "Target issue or Obstruction" of the families document; wf-improve NEBRASKA, batch 01 (BREITHORN, parked vehicles, 2 to 5 days; NEBRASKA and MICHIGAN, lost reference, 25 days; JUNGFRAU, PM_G, 63 days) | | | `BRIDGE` | 1 day | non-degraded days in a row that do not break the run of a problem target ("Degraded target": `WEEK` days from the first to the last degraded day, `SHARE_WEEK` of rejections over the bridged run); wf-improve Obstruction, mod. 1, S1 (FLORIDA, SNCF01, NORDEND, NIEDERHORN: problem targets cut into Target obscured by a day under `SHARE`; BREITHORN, parked vehicles, unchanged); 2 → 1: wf-improve Target issue, mod. 1, S1 (at a quarter, two bridging days joined the parked vehicles of BREITHORN, POUI22, into a week) | | | `STUCK` | 0.05 | stuck target (D-074): E-011, C2 and C3 at 0.006 and 0.016 of the others' noise; NORDEND, E-001: 0.7 at the lowest | | `categories` (classification) | `MERGE` | 1 h | rule validated by the user: same variety less than an hour apart, a single event; two Obstruction (Target obscured, Brief obstruction) only if they share a sensor (wf-improve Obstruction, mod. 1, S1 and S3), two Target issue too (wf-improve Target issue, mod. 1, S3); a Brief obstruction only absorbs the jumps back within `MERGE` after its instant (S3) |