Skill learner -> wf-improve (/detectors:wf-improve); scripts/wf-improve/, sessions in wf-improve/
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b9595d6011
commit
6df274f580
16 files changed
+93
-93
No files matched your search
@@ -1,4 +1,4 @@
|
||||
{
|
||||
"name": "detectors",
|
||||
"description": "AIMsight anomaly detectors and their improvement from an analysis (/detectors:learner)."
|
||||
"description": "AIMsight anomaly detectors and their improvement from an analysis (/detectors:wf-improve)."
|
||||
}
|
||||
@@ -1,17 +1,17 @@
|
||||
# detectors
|
||||
|
||||
AIMsight anomaly detectors (Python 3, standard library only) and the Claude Code plugin that improves them (`/detectors:learner`). Everything is in English: code, identifiers, docs, outputs. Layout and usage: `README.md`.
|
||||
AIMsight anomaly detectors (Python 3, standard library only) and the Claude Code plugin that improves them (`/detectors:wf-improve`). Everything is in English: code, identifiers, docs, outputs. Layout and usage: `README.md`.
|
||||
|
||||
## Contract (do not break)
|
||||
|
||||
- One detector per family of `labeling/familles.md` (D-119), in `scripts/`: `visibility.py`, `obstruction.py`, `target_issue.py`, `thermal_displacement.py`, `no_data.py`, `displacement.py`. `categories.py` loads every `scripts/*.py` that has `detect`: never put another script with a `detect` function, or one with side effects at import, directly in `scripts/` (tools go in `scripts/learner/`).
|
||||
- One detector per family of `labeling/familles.md` (D-119), in `scripts/`: `visibility.py`, `obstruction.py`, `target_issue.py`, `thermal_displacement.py`, `no_data.py`, `displacement.py`. `categories.py` loads every `scripts/*.py` that has `detect`: never put another script with a `detect` function, or one with side effects at import, directly in `scripts/` (tools go in `scripts/wf-improve/`).
|
||||
- `detect(measurements)` returns episodes with the keys `start`, `end`, `kind` (the family), `label` (the variety), `text`, `ignore`, `sensors`. A detector may also declare `temperature`, `camera`, `metrics` (`categories.CONTEXT`) or `end`; `categories.events` passes them by introspection.
|
||||
- Diagnostic-app reads this repository as it is: `categories.events`, `categories.CATEGORIES` (`process`: `images`, `none`, `check`), the `--from/--to` CLIs, `/detectors:learner`, `learner/<session>/summary.md`. A rename here needs the matching change there.
|
||||
- Diagnostic-app reads this repository as it is: `categories.events`, `categories.CATEGORIES` (`process`: `images`, `none`, `check`), the `--from/--to` CLIs, `/detectors:wf-improve`, `wf-improve/<session>/summary.md`. A rename here needs the matching change there.
|
||||
- Label fields come from Diagnostic-app and stay French: `debut`, `fin`, `categorie`, `description`; likewise `demande.json` (`STATION`, `début`, `fin`).
|
||||
|
||||
## Rules
|
||||
|
||||
- A constant added or changed updates `thresholds.md` in the same change; check with `grep -nE "^[A-Z_0-9]+ = " scripts/*.py`.
|
||||
- No rule specific to a station, a sensor or a date.
|
||||
- A change to a detector is checked with the benchmark, from a working repository where this repository is mounted (or copied, not symlinked) as `detectors/` next to `data/` and `labeling/`: `python3 detectors/scripts/learner/regression.py`. A refactoring must leave every figure unchanged.
|
||||
- History references in docstrings ("learner X, mod. 1, S2", "X, batch 01", D-xxx, E-xxx) point to past sessions and to decisions recorded in Diagnostic-app/Support-Agent; keep them when editing a rule.
|
||||
- A change to a detector is checked with the benchmark, from a working repository where this repository is mounted (or copied, not symlinked) as `detectors/` next to `data/` and `labeling/`: `python3 detectors/scripts/wf-improve/regression.py`. A refactoring must leave every figure unchanged.
|
||||
- History references in docstrings ("wf-improve X, mod. 1, S2", "X, batch 01", D-xxx, E-xxx) point to past sessions and to decisions recorded in Diagnostic-app/Support-Agent; keep them when editing a rule.
|
||||
@@ -9,22 +9,22 @@ python3 scripts/categories.py <station folder> --from YYYY-MM-DD --to YYYY-MM-DD
|
||||
The station folder: `displacement/`, `metrics/`, `camera/` (the format of `data/<STATION>/` in Diagnostic-app).
|
||||
|
||||
- `scripts/`: the detectors (`visibility.py`, `obstruction.py`, `target_issue.py`, `thermal_displacement.py`, `no_data.py`, `displacement.py`), `categories.py`, `data.py`.
|
||||
- `scripts/learner/`: the learner's tools, run from the working repository: `regression.py` (the benchmark, all labels of `labeling/auto/`), `score.py` (the score of a range), `sensors.py` (the sensors of each event).
|
||||
- `scripts/wf-improve/`: the wf-improve workflow's tools, run from the working repository: `regression.py` (the benchmark, all labels of `labeling/auto/`), `score.py` (the score of a range), `sensors.py` (the sensors of each event).
|
||||
- `thresholds.md`: the registry of the detectors' thresholds.
|
||||
|
||||
## The learner: a Claude Code plugin
|
||||
## The wf-improve workflow: a Claude Code plugin
|
||||
|
||||
The repository is also a Claude Code plugin (`.claude-plugin/plugin.json`), to be mounted as `detectors/` in a working repository that has `labeling/` (the labels, `labeling/auto/<STATION>.jsonl`; the analyses, `labeling/auto/analyses/`), `data/` and `agent-tools/`: Diagnostic-app, where it is a submodule.
|
||||
|
||||
- `skills/learner/SKILL.md`: `/detectors:learner <analysis>`, the session that improves the detectors from an analysis of the Data page, without overfitting.
|
||||
- `skills/wf-improve/SKILL.md`: `/detectors:wf-improve <analysis>`, the session that improves the detectors from an analysis of the Data page, without overfitting.
|
||||
- `agents/`: `upgrader` (diagnoses and fixes the detectors) and `validator` (LLM-as-judge: keeps, removes or cancels the fixes), called `detectors:<name>`.
|
||||
- `learner/`: the sessions, one folder `<session>_<analysis>/` each, written by the skill.
|
||||
- `wf-improve/`: the sessions, one folder `<session>_<analysis>/` each, written by the skill.
|
||||
|
||||
From the working repository:
|
||||
|
||||
```bash
|
||||
claude --plugin-dir detectors # then /detectors:learner <analysis>
|
||||
python3 detectors/scripts/learner/regression.py # the benchmark
|
||||
claude --plugin-dir detectors # then /detectors:wf-improve <analysis>
|
||||
python3 detectors/scripts/wf-improve/regression.py # the benchmark
|
||||
```
|
||||
|
||||
The session commits the detectors here, on `main`, without pushing: push `detectors` so that everyone has the updated detectors.
|
||||
|
||||
+6
-6
@@ -1,13 +1,13 @@
|
||||
---
|
||||
name: upgrader
|
||||
description: Upgrader of the learner — on the range of an analysis of the Data page (labeling/auto/analyses/<analysis>/), compares what the detectors find, all families, with the labels of labeling/auto/<STATION>.jsonl touching the range, then fixes the detectors (adds, removes, changes) so that they do better there without overfitting, one commit per fix. Called by the orchestrator with the analysis, the station and its range, the events and sensors commands, the range score, the benchmark and the path of its output.
|
||||
description: Upgrader of the wf-improve workflow — on the range of an analysis of the Data page (labeling/auto/analyses/<analysis>/), compares what the detectors find, all families, with the labels of labeling/auto/<STATION>.jsonl touching the range, then fixes the detectors (adds, removes, changes) so that they do better there without overfitting, one commit per fix. Called by the orchestrator with the analysis, the station and its range, the events and sensors commands, the range score, the benchmark and the path of its output.
|
||||
tools: Read, Edit, Write, Bash
|
||||
model: inherit
|
||||
---
|
||||
|
||||
You are the upgrader of the learner: your job is to make the detectors better from an analysis of the Data page (D-148). The user chose a range where the detectors satisfy them poorly; the analyst analysed it (`rapport.md`) and labelled it, and the labels of `labeling/auto/<STATION>.jsonl` touching the range are the truth of the session (D-112, D-118). You compare what the detectors find with these labels, event by event, all families, say what is wrong and which detector causes it, then fix it. Afterwards, the `validator` judges your modification on all stations and all families of `labeling/auto/`: a gain on the range at the cost of another station or another family is refused.
|
||||
You are the upgrader of the wf-improve workflow: your job is to make the detectors better from an analysis of the Data page (D-148). The user chose a range where the detectors satisfy them poorly; the analyst analysed it (`rapport.md`) and labelled it, and the labels of `labeling/auto/<STATION>.jsonl` touching the range are the truth of the session (D-112, D-118). You compare what the detectors find with these labels, event by event, all families, say what is wrong and which detector causes it, then fix it. Afterwards, the `validator` judges your modification on all stations and all families of `labeling/auto/`: a gain on the range at the cost of another station or another family is refused.
|
||||
|
||||
You run in the working repository (Diagnostic-app: `labeling/`, `data/`, `agent-tools/`), where the **detectors** repository is mounted as `detectors/`: the detectors (`detectors/scripts/*.py`), `detectors/thresholds.md`, the session tools (`detectors/scripts/learner/`) and the sessions (`detectors/learner/`). What touches the detectors is committed in `detectors/` (`git -C detectors …`, paths relative to `detectors/`).
|
||||
You run in the working repository (Diagnostic-app: `labeling/`, `data/`, `agent-tools/`), where the **detectors** repository is mounted as `detectors/`: the detectors (`detectors/scripts/*.py`), `detectors/thresholds.md`, the session tools (`detectors/scripts/wf-improve/`) and the sessions (`detectors/wf-improve/`). What touches the detectors is committed in `detectors/` (`git -C detectors …`, paths relative to `detectors/`).
|
||||
|
||||
**Avoid overfitting** (the user, D-148): a small step. A single range must not dictate the detectors: do better here with rules that would hold on any station, without aiming to reproduce exactly these labels. A constant tuned to make these bounds fall exactly, a rule that only recognises the exact shape of these events: no. What counts is the mechanism the analyst established (a rejected reference that makes all targets jump, for example), recognisable wherever it happens. What the detectors cannot see with these measurements (a cause only the image reveals) is not a fix: say it in "Limits", for the user.
|
||||
|
||||
@@ -33,7 +33,7 @@ If nothing important is to be fixed: verdict `nothing to change`, write your out
|
||||
|
||||
## 2. What you change
|
||||
|
||||
Only the detectors (`detectors/scripts/*.py`, not `scripts/learner/`) and `detectors/thresholds.md`; never `detectors/skills/`, `detectors/agents/` or the working repository. On them, anything is possible: add, remove or change a constant, a rule or a whole detector. You commit only as "One fix, one commit" says, never a file your request says was already modified at the start of the session.
|
||||
Only the detectors (`detectors/scripts/*.py`, not `scripts/wf-improve/`) and `detectors/thresholds.md`; never `detectors/skills/`, `detectors/agents/` or the working repository. On them, anything is possible: add, remove or change a constant, a rule or a whole detector. You commit only as "One fix, one commit" says, never a file your request says was already modified at the start of the session.
|
||||
|
||||
**Everything is detected** (the user, D-068): what is shown or not to the client is not the detectors' business. A setting chosen not to show something to the client does not stop you: you may change it, saying so in "What changed".
|
||||
|
||||
@@ -62,13 +62,13 @@ Then the **benchmark**, the `regression.py` command of your request, against the
|
||||
- **The range score**: what you make rise, without losing a whole event. A hole has no measurement: it only counts its two bounds in the score; for a hole, read the events command. Do not aim for 100 %.
|
||||
- **All lines of all stations**: the non-regression, the same rule as the validator. **A regression**, on a family of a station: "found" or "correct" loses more than 2 points; it is a regression even if the `Total` rises. **Not a regression** (D-135): a drop that comes from another consistent reading of the detectors that the labels do not retain (labels contradicting each other from one station to another, the gaps of a few measurements between two labels of the same phenomenon, the bounds of a label, an instant the score counted as found by an event on another sensor). A regression: rework your change before committing it, or drop the fix. A drop from another consistent reading: keep it, and say it in "For the user to decide".
|
||||
|
||||
Your working folder is yours alone: `mktemp -d`, or a subfolder named after you in the session's temporary folder. To try a variant without touching the repository, copy `detectors/` there, with links to `data/`, `labeling/` and `agent-tools/`: the copy's `detectors/scripts/learner/regression.py` measures the copy. For a shorter run, `--stations A,B` measures only these stations; the whole benchmark (about 3 min) runs in the background, output to a file.
|
||||
Your working folder is yours alone: `mktemp -d`, or a subfolder named after you in the session's temporary folder. To try a variant without touching the repository, copy `detectors/` there, with links to `data/`, `labeling/` and `agent-tools/`: the copy's `detectors/scripts/wf-improve/regression.py` measures the copy. For a shorter run, `--stations A,B` measures only these stations; the whole benchmark (about 3 min) runs in the background, output to a file.
|
||||
|
||||
## 4. One fix, one commit
|
||||
|
||||
The validator can have a single fix removed without touching the others: each has its commit. For each fix, in order:
|
||||
1. change, then check as above, against the benchmark of the last committed state (the benchmark before, or that of your previous kept fix); before your output, the whole against the benchmark before, the one the validator judges; keep the output of `regression.py` in `<session>/benchmark-S<n>.txt`;
|
||||
2. kept: `git -C detectors branch --show-current` must say `main`; then `git -C detectors add <the changed files> <session>/benchmark-S<n>.txt && git -C detectors commit -m "Learner <analysis>: S<n>" -- <the same>`, paths relative to `detectors/`, the message ending with an empty line then `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`;
|
||||
2. kept: `git -C detectors branch --show-current` must say `main`; then `git -C detectors add <the changed files> <session>/benchmark-S<n>.txt && git -C detectors commit -m "wf-improve <analysis>: S<n>" -- <the same>`, paths relative to `detectors/`, the message ending with an empty line then `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`;
|
||||
3. not kept: `git -C detectors checkout -- <the files you changed for it>`, only them, and it goes to "Not fixed".
|
||||
|
||||
Never `git add -A`, `git stash`, or `git checkout` on other files: another session may be working in these repositories.
|
||||
|
||||
+3
-3
@@ -1,13 +1,13 @@
|
||||
---
|
||||
name: validator
|
||||
description: Validator of the learner — the LLM-as-judge against overfitting: judges a modification of the upgrader, made for the range of an analysis of the Data page, on all stations and all labelled families of labeling/auto/ and on the code of the detectors, without reading the diagnosis; keeps it, has part of it removed, or cancels it. Called by the orchestrator with the analysis, the modification, the benchmarks, the range score, the events command and the base commit.
|
||||
description: Validator of the wf-improve workflow — the LLM-as-judge against overfitting: judges a modification of the upgrader, made for the range of an analysis of the Data page, on all stations and all labelled families of labeling/auto/ and on the code of the detectors, without reading the diagnosis; keeps it, has part of it removed, or cancels it. Called by the orchestrator with the analysis, the modification, the benchmarks, the range score, the events command and the base commit.
|
||||
tools: Read, Bash, Write
|
||||
model: inherit
|
||||
---
|
||||
|
||||
You are the validator of the learner, its safeguard. The upgrader has just adapted the detectors to better find the events of the range of an analysis of the Data page (D-148), a single range of a single station. The risk is overfitting: detectors glued to this range, which lose precision elsewhere, on another station or another family. You check it, and you decide in the end, in a single pass: keep the modification, have part of it removed, or cancel it.
|
||||
You are the validator of the wf-improve workflow, its safeguard. The upgrader has just adapted the detectors to better find the events of the range of an analysis of the Data page (D-148), a single range of a single station. The risk is overfitting: detectors glued to this range, which lose precision elsewhere, on another station or another family. You check it, and you decide in the end, in a single pass: keep the modification, have part of it removed, or cancel it.
|
||||
|
||||
You run in the working repository (Diagnostic-app: `labeling/`, `data/`, `agent-tools/`), where the **detectors** repository is mounted as `detectors/`: the detectors (`detectors/scripts/*.py`), `detectors/thresholds.md`, the session tools (`detectors/scripts/learner/`) and the sessions (`detectors/learner/`).
|
||||
You run in the working repository (Diagnostic-app: `labeling/`, `data/`, `agent-tools/`), where the **detectors** repository is mounted as `detectors/`: the detectors (`detectors/scripts/*.py`), `detectors/thresholds.md`, the session tools (`detectors/scripts/wf-improve/`) and the sessions (`detectors/wf-improve/`).
|
||||
|
||||
**You stay independent**: you do not read the upgrader's diagnosis. Your conclusions come from the benchmark of `regression.py` on the labels of `labeling/auto/`, the only truth (D-118), and from the code of the detectors.
|
||||
|
||||
|
||||
@@ -7,18 +7,18 @@ episodes; their label gives the variety (Poor visibility, Heat shimmer, Brief ob
|
||||
shimmer, and last the thermal effect, if it touches several targets) is absorbed into it: it takes its cause and
|
||||
family; otherwise it stays Displacement, cause not established, to be checked on the images (familles.md, "A jump then
|
||||
the return"); a Brief obstruction (one instant) only absorbs the jumps starting at its instant, not those of several
|
||||
days it crosses (learner session Displacement, mod. 1, S5);
|
||||
days it crosses (wf-improve session Displacement, mod. 1, S5);
|
||||
- a slow drift or a jump back whose targets are all in an overlapping Target issue episode is absorbed into it (the
|
||||
curve of a problem target is to be ignored, familles.md);
|
||||
- a shift that lasts, or too recent to know (SHIFTS), is never absorbed;
|
||||
- heat shimmer overlapping a shift of several targets is not kept: a real movement also shakes the curves; the shift of
|
||||
a single target does not remove the heat shimmer of the whole scene, and heat shimmer only absorbs the jumps back
|
||||
starting during it, not the one of the day before coming back within its day (learner session Visibility, mod. 1,
|
||||
starting during it, not the one of the day before coming back within its day (wf-improve session Visibility, mod. 1,
|
||||
S1: WILDSTRUBEL);
|
||||
- two events of the same variety less than MERGE apart make one; two Obstruction (Target obscured, Brief obstruction)
|
||||
only if they share a sensor (learner session Obstruction, mod. 1: S1, different targets chained into an event of
|
||||
only if they share a sensor (wf-improve session Obstruction, mod. 1: S1, different targets chained into an event of
|
||||
several days; S3, passings of a few minutes into a whole hour); two Target issue too: each problem target keeps its
|
||||
bounds (learner session Target issue, mod. 1, S3); a Brief obstruction only absorbs the jumps back within MERGE after
|
||||
bounds (wf-improve session Target issue, mod. 1, S3); a Brief obstruction only absorbs the jumps back within MERGE after
|
||||
its instant (S3: not the 30 h jump of another target).
|
||||
Each event gets its sentence, in English (the site): what happened and what to do about it, as the "For the client"
|
||||
column of familles.md says; the cause is only given if the variety establishes it, otherwise the images show it.
|
||||
@@ -173,7 +173,7 @@ def classify(episodes, references=frozenset()):
|
||||
and datetime.fromisoformat(e["end"]) - datetime.fromisoformat(h["start"]) <= MERGE)), None)
|
||||
for e in eps if e["label"] == JUMP_BACK}
|
||||
# a drift or a jump back whose targets are all in an overlapping Target issue episode: its curve is to be ignored,
|
||||
# it takes its family (learner session Displacement, mod. 1, S6)
|
||||
# it takes its family (wf-improve session Displacement, mod. 1, S6)
|
||||
host.update({id(e): h for e in eps if e["label"] in (JUMP_BACK, "Slow drift") and not host.get(id(e))
|
||||
and (h := next((h for h in eps if h["kind"] == "Target issue" and overlap(e, h)
|
||||
and set(e["sensors"]) <= set(h["sensors"])), None))})
|
||||
@@ -219,7 +219,7 @@ if __name__ == "__main__":
|
||||
p.add_argument("--chunk", type=int, default=0, help="days per chunk (default: a single pass)")
|
||||
# without history, a chunk starting just before a real jump takes it for a jump back (PR17 at NORDEND, 08/10 02:30:
|
||||
# absorbed into the fog); 3 days give it back; user's rule (2026-10-24): 3 days of history by default, in the
|
||||
# learner as in the real application
|
||||
# workflow as in the real application
|
||||
p.add_argument("--history", type=int, default=3, help="days of earlier measurements given to the detectors (default 3)")
|
||||
a = p.parse_args()
|
||||
measurements = load(a.folder)
|
||||
|
||||
@@ -5,7 +5,7 @@ belongs to the Target issue family: episodes() returns it, detect() leaves it to
|
||||
1. Noise of the series = median of the gaps from one measurement to the next; jump = measurement K noises or more away
|
||||
from the level before (median of the BASE previous measurements); the first BASE measurements, lacking a level
|
||||
before, are compared with the level after (median of the BASE next ones), a gap there is a jump back at once.
|
||||
2. Back (an hour of measurements in a row within half the jump: BASE at 15 min, a single one at 4 h; learner
|
||||
2. Back (an hour of measurements in a row within half the jump: BASE at 15 min, a single one at 4 h; wf-improve
|
||||
MICHIGAN, mod. 1) within a day, or one day later back at its usual place (first day of the export) or at the day
|
||||
before's: "Sudden jump, back to normal", greyed; its cause is not established here (categories.py gives it the
|
||||
cause of the category that overlaps it, otherwise: to be checked on the images).
|
||||
@@ -22,7 +22,7 @@ belongs to the Target issue family: episodes() returns it, detect() leaves it to
|
||||
(REPEAT jumps back or more, each within a day of the previous one) makes a "back to normal" episode on its own, from
|
||||
the first to the last flip, outside the common episodes. A target that flips for at least FLIP, with at most
|
||||
MAX_FLIPPING targets at the same time, becomes "Target flipping" (Target issue, D-074); a lasting jump of at least
|
||||
LARGE of the targets, "Large shift" (everything moved), always to be checked. Learner session Target issue, mod. 1,
|
||||
LARGE of the targets, "Large shift" (everything moved), always to be checked. wf-improve session Target issue, mod. 1,
|
||||
S2: FLIP from 6 h to 36 h; a target flipping for a day during a nuisance of the whole scene (rain, fog, night of poor
|
||||
visibility) is not a problem target; its jumps back go to the nuisance overlapping them, otherwise stay
|
||||
Displacement.
|
||||
@@ -206,14 +206,14 @@ def detect(measurements):
|
||||
# A target that moves little by little, day after day. Takes up the trend of the Displacements tab (D-055) on the
|
||||
# "advanced" export (corr_x, corr_y, px). Method, per target (global references excluded) and per axis:
|
||||
# 1. Level of the day: median of the measurements of each UTC day with at least HOURS measured hours out of 24, as a
|
||||
# share of the hours the station measures (learner MICHIGAN, mod. 1: at one measurement every 4 h, 5 of the 6 hours;
|
||||
# share of the hours the station measures (wf-improve MICHIGAN, mod. 1: at one measurement every 4 h, 5 of the 6 hours;
|
||||
# before, no day counted); at least DAYS days.
|
||||
# 2. Theil-Sen slope and Kendall tau of the daily levels; amplitude of the day: gap between its 9th and 1st deciles.
|
||||
# 3. Drift: |slope| × duration ≥ SIZE times the median daily amplitude, |tau| ≥ STEADY, and no step from one day to the
|
||||
# next of more than half the drift (a jump that lasts is not a drift: the jump rule takes it).
|
||||
# 4. One episode per target (one axis is enough), from the first measurement of the first day to the last of the last
|
||||
# day; not greyed.
|
||||
# 4b. Piecewise (learner session Displacement, mod. 1, S1): a target without a drift over the whole window may drift for
|
||||
# 4b. Piecewise (wf-improve session Displacement, mod. 1, S1): a target without a drift over the whole window may drift for
|
||||
# a few days, then settle or go back. On each run of at least DAYS kept days, drift = level of the last day minus
|
||||
# that of the first, at least PIECE daily amplitudes (median of the series), |tau| ≥ STEADY with the same sign, no
|
||||
# step from one kept day to the next of more than half the drift; pieces taken from the largest drift to the
|
||||
|
||||
@@ -5,7 +5,7 @@ for long. It reuses two computations shared with other families:
|
||||
- the day-by-day rejections of target_issue.py: a target rejected for less than a week, Target obscured.
|
||||
A sensor already in an overlapping Target issue episode (Degraded target, Target stuck) is not returned as Obstruction:
|
||||
its curve is already to be ignored, the family is Target issue (boundary "Target issue or an episode of another family"
|
||||
of labeling/familles.md; learner session Obstruction, mod. 1: S1, problem targets of FLORIDA, NORDEND, SNCF01,
|
||||
of labeling/familles.md; wf-improve session Obstruction, mod. 1: S1, problem targets of FLORIDA, NORDEND, SNCF01,
|
||||
NIEDERHORN, JUNGFRAU, CLARIDEN cut into Target obscured; S3, their isolated rejections as Brief obstruction).
|
||||
Usage: python3 detectors/obstruction.py <station folder> (prints the episodes)
|
||||
"""
|
||||
|
||||
@@ -10,11 +10,11 @@ Method:
|
||||
fog, which touch everyone, are thus set aside.
|
||||
2. Per UTC day and per sensor, on its calm instants (at least MIN_HOURS hours of instants at the station's cadence,
|
||||
median gap between two measurement instants; otherwise the day does not count): degraded day if at least SHARE of
|
||||
them are rejected. Cadence (learner NEBRASKA, batch 01): 12 fixed instants, "3 h at 15 min", counted no day at one
|
||||
them are rejected. Cadence (wf-improve NEBRASKA, batch 01): 12 fixed instants, "3 h at 15 min", counted no day at one
|
||||
measurement every 4 h; REFCOLH22, rejected all the time since it was lost, was not seen.
|
||||
3. Episode: at least MIN_DAYS degraded days in a row (a day that does not count does not break the run), from the first
|
||||
rejected measurement of these days to the last; ignore = True: the target's curve is not reliable.
|
||||
4. Hindered all the time or at times (boundary "Target issue or Obstruction" of labeling/familles.md; learner
|
||||
4. Hindered all the time or at times (boundary "Target issue or Obstruction" of labeling/familles.md; wf-improve
|
||||
NEBRASKA, batch 01): a run of at least WEEK days is a problem target, "Degraded target"; shorter, a target hidden at
|
||||
times, "Target obscured" (Obstruction family), one episode per run of rejected measurements (the hours when it is
|
||||
seen are not part of it). BREITHORN: vehicles parked in front of COLI24 and POUI22, 2 to 5 days; NEBRASKA and
|
||||
@@ -22,11 +22,11 @@ Method:
|
||||
For the problem target, the run is bridged: at most BRIDGE non-degraded days in a row do not break it, and it then
|
||||
needs WEEK days from its first to its last degraded day, at SHARE_WEEK of rejections over all its days ("rejections
|
||||
specific to the target almost every day of the range"); the Target obscured episodes it covers no longer exist
|
||||
(learner session Obstruction, mod. 1, S1: FLORIDA Sellette 6 and Massif 2, SNCF01 F4.3 and F4.4, NORDEND CTR01,
|
||||
(wf-improve session Obstruction, mod. 1, S1: FLORIDA Sellette 6 and Massif 2, SNCF01 F4.3 and F4.4, NORDEND CTR01,
|
||||
NIEDERHORN e01, cut into Target obscured by a day under SHARE; BREITHORN, parked vehicles, unchanged).
|
||||
The problem target judges regularity rather than the share of the day: a day is degraded from SHARE_WEEK (a quarter)
|
||||
of its calm instants rejected, and BRIDGE is one day; the target hidden at times keeps SHARE. A target rejected every
|
||||
day at the same hours, by day only or by night only, stays under half and was not seen (learner session Target
|
||||
day at the same hours, by day only or by night only, stays under half and was not seen (wf-improve session Target
|
||||
issue, mod. 1, S1: WILDSTRUBEL R3, 04:00 to 16:00 every day; NORDEND PR04 and CTR01, NIEDERHORN PWD_IMP_B08_024_C09
|
||||
and e01, SNCF01 F4.3).
|
||||
The cause (dirt, vegetation, setting) can only be read on the images.
|
||||
@@ -47,7 +47,7 @@ from data import is_reference, iso, load # noqa: E402
|
||||
CALM = 0.25 # share of the other sensors rejected, at most, for a calm instant
|
||||
SHARE = 0.5 # half of the calm instants rejected: "most of the time"
|
||||
SHARE_WEEK = 0.25 # a quarter of the calm instants rejected, each day of a week or more: a problem target, even when
|
||||
# rejected by day only or by night only (learner session Target issue, mod. 1, S1)
|
||||
# rejected by day only or by night only (wf-improve session Target issue, mod. 1, S1)
|
||||
MIN_HOURS = 3 # hours of calm instants, at the station's cadence: below that, the day does not count
|
||||
MIN_DAYS = 2 # a single day is too often an accident (NORDEND: isolated days on 16 sensors)
|
||||
BRIDGE = 1 # non-degraded days that do not break a run of degraded days (4); at 2, the parked vehicles of BREITHORN
|
||||
|
||||
@@ -6,7 +6,7 @@ Method, per target (global references excluded) and per axis (corr_x, corr_y):
|
||||
2. Profile = median, hour by hour, over the days; "repeatability" = median, over the days, of the correlation between
|
||||
the day and the profile of the other days. Axis concerned if repeatability ≥ REPRO. An incomplete day (at least
|
||||
PARTIAL hours, fewer than MIN_HOURS) does not enter the profile, but its likeness is measured on the hours it has: a
|
||||
hole of half a day no longer takes its episode away (learner session Thermal displacement, mod. 1, S4; at
|
||||
hole of half a day no longer takes its episode away (wf-improve session Thermal displacement, mod. 1, S4; at
|
||||
MIN_HOURS = 12 or 16, the incomplete days distorted the profile, NIEDERHORN went back).
|
||||
3. Cycle of the station if at least SHARE of the targets have an axis concerned, **and** (second method, asked by the
|
||||
user) if a 24 h sinusoid, a + b·sin(2πt/24 h) + c·cos(2πt/24 h) by least squares, explains at least R2_MIN of the
|
||||
@@ -18,7 +18,7 @@ Method, per target (global references excluded) and per axis (corr_x, corr_y):
|
||||
contraction" until the return. The warm hours say there is a cycle (at least MIN_WARM_HOURS of them); its bounds
|
||||
are read lower, at EDGE times the amplitude: the start, going back from the peak while the gap stays ≥ EDGE (at most
|
||||
to the calmest hour of the profile); the return, going down from the peak until it goes back under EDGE, after
|
||||
midnight if needed, at most to the next day's start: a slow return at night chains the cycles (learner session
|
||||
midnight if needed, at most to the next day's start: a slow return at night chains the cycles (wf-improve session
|
||||
Thermal displacement, mod. 1, S1). The direction (a rising or falling curve) depends on the target's orientation:
|
||||
it is not looked at, only the gap from the night level.
|
||||
Reading: the cycle is almost everywhere in the raw positions (common mode) and the compensation removes it, except at
|
||||
@@ -30,13 +30,13 @@ rose by a 3 px block at sunrise, held 3 h and dropped in one step around 09:30,
|
||||
does not explain); in exchange, a single warm hour is not a cycle (MIN_WARM_HOURS: NIEDERHORN, E-005, hourly, would
|
||||
otherwise have its noon hour as a camera contraction). The camera episode covers all sensors, references included: the
|
||||
whole image moves.
|
||||
Common slide, without a cycle (learner MICHIGAN, mod. 1): lacking a cycle in the corrected curves as in the raw
|
||||
Common slide, without a cycle (wf-improve MICHIGAN, mod. 1): lacking a cycle in the corrected curves as in the raw
|
||||
positions, an instant is a camera slide if the raw positions (x or y) of at least SHARE of the sensors, references
|
||||
included, **and** the corrected curves (corr_x or corr_y) of at least SHARE of the targets move away on the same side by
|
||||
at least SLIDE_K noises (median of the steps of the series) from their median of the day before; slides less than a
|
||||
day apart make one episode, "Camera thermal movement", on all sensors, host of the jumps back (categories.HOSTS). The
|
||||
condition on the raw positions leaves to the jumps a flip of the computed pose alone (raw positions still).
|
||||
The camera moves with the cold (learner session Thermal displacement, mod. 1, S3): lacking a cycle, also a "Camera
|
||||
The camera moves with the cold (wf-improve session Thermal displacement, mod. 1, S3): lacking a cycle, also a "Camera
|
||||
thermal movement" where the camera pose (tilt, rotation or scale) moves away from its median of the day before by at
|
||||
least COLD noises, while the station temperature (detect declares it, D-124) falls under its own by at least COLD
|
||||
noises the same hour; noise: the median of the variations over an hour (over one step, at one measurement every 4 h),
|
||||
|
||||
@@ -11,19 +11,19 @@ Method:
|
||||
instants (train, vehicle), with no reference (global_ref) among them: a one-step peak touching the references is not
|
||||
an obstacle in front of targets (peak of the whole image, NORDEND on 01.09 11:15, batch 01); it is not shown. Unless
|
||||
at least BRIEF_MIN targets are rejected at this instant and neither at the instant before nor after: something
|
||||
passing in front of them, a reference touched as well (learner session Displacement, mod. 1, S5: trains of
|
||||
passing in front of them, a reference touched as well (wf-improve session Displacement, mod. 1, S5: trains of
|
||||
NIEDERHORN, 12 and 13 targets rejected).
|
||||
Also, something passing in front of one or two targets, or a single reference: at an instant, fewer than BRIEF_MIN
|
||||
sensors rejected at more than FACTOR times their median (without K_MAD: by day, heat shimmer inflates the MAD), at
|
||||
most one reference, none of them rejected at the instant before or after, and fewer than BRIEF_MIN high sensors
|
||||
(learner session Obstruction, mod. 1, S3: vehicles, passers-by, insects in front of one or two targets of
|
||||
(wf-improve session Obstruction, mod. 1, S3: vehicles, passers-by, insects in front of one or two targets of
|
||||
WILDSTRUBEL, NIEDERHORN, LENZSPITZE; a reference hidden by a truck at JUNGFRAU, BREITHORN).
|
||||
3. Poor visibility: runs of at least RUN_MIN high measurements per sensor, joined across sensors (at least 2), reduced
|
||||
to the instants where at least FRAC of the episode's peak is high.
|
||||
4. Target obscured: a target (not a global_ref) rejected (> error_threshold) and above K_TARGET MAD, at least TARGET_MIN
|
||||
times less than TARGET_GAP apart, mostly outside the episodes of 2 and 3 (leaves, snow, glare on a target). The gap
|
||||
is at least two measurement steps, and two rejections are enough when TARGET_MIN steps do not fit in TARGET_GAP
|
||||
(learner session Obstruction, mod. 1, S6: at one measurement every 4 h, vehicles parked at night in front of COLI22,
|
||||
(wf-improve session Obstruction, mod. 1, S6: at one measurement every 4 h, vehicles parked at night in front of COLI22,
|
||||
COLI24, POUI22 of BREITHORN make only two or three rejections). All episodes: ignore = True.
|
||||
5. Level of a Poor visibility (validated 2026-10-24), measurement by measurement: "none" (nothing is seen) when at least
|
||||
HALF of the usually seen targets (under their threshold at least half the time) are rejected, "reduced" (average
|
||||
@@ -35,7 +35,7 @@ Method:
|
||||
7. Poor visibility or object (boundary "Visibility or Obstruction" of labeling/familles.md; BREITHORN, batch 01): a
|
||||
nuisance touching at most FEW targets, and no reference, is not diffuse: it is an object in front of them (a parked
|
||||
vehicle), a "Target obscured" episode of its whole duration, without a level.
|
||||
8. Diffuse nuisance, with or without rejections (learner session Visibility, mod. 1, S2): at each instant, the median,
|
||||
8. Diffuse nuisance, with or without rejections (wf-improve session Visibility, mod. 1, S2): at each instant, the median,
|
||||
over the sensors, of the error_value in MAD above each one's median; at least RUN_MIN instants in a row at K_DIFFUSE
|
||||
or more make a Poor visibility "reduced". A diffuse nuisance entirely within heat shimmer is the heat shimmer: no
|
||||
episode (otherwise it absorbed the jump back of the day before, WILDSTRUBEL on 04.10). Finds the drops of
|
||||
@@ -69,7 +69,7 @@ TARGET_GAP = timedelta(hours=6) # tuned on E-005 (leaves: 7 rejections in 12 h,
|
||||
REJECT_FACTOR = 1.2 # a rejection at 10 MAD also counts (FLORIDA, batch 01: levels of 0.6, ×2 out of reach); drops C24
|
||||
# of E-009 (level 0.68 just under the threshold 0.7, rejected at ×1.04)
|
||||
HALF = 0.5 # "nothing is seen"; tuned on NORDEND (09.10 23:00: 68 % rejected) and E-005 (rain: 24 to 33 %)
|
||||
K_DIFFUSE = 2.5 # diffuse nuisance (8); learner session Visibility, mod. 1, S2
|
||||
K_DIFFUSE = 2.5 # diffuse nuisance (8); wf-improve session Visibility, mod. 1, S2
|
||||
FEW = 3 # Poor visibility on at most 3 targets, no reference: an object, Target obscured (7); BREITHORN,
|
||||
# batch 01 (vehicles parked in front of 2 or 3 targets out of 12)
|
||||
|
||||
@@ -255,13 +255,13 @@ def detect(measurements):
|
||||
# sensors at ≥ RATIO_ERR times the calm (warm air blurs the spots: E-003).
|
||||
# 4. Neighbouring hours merged (a one-hour hole tolerated), episode of at least MIN_DURATION, dropped if more than
|
||||
# REJECT_MAX of the measurements are rejected beyond their sensor's habit (its share of rejections over the export):
|
||||
# it is then poor visibility, another rule. A target that is always rejected (Target issue) does not count (learner
|
||||
# it is then poor visibility, another rule. A target that is always rejected (Target issue) does not count (wf-improve
|
||||
# session Visibility, mod. 1, S1: WILDSTRUBEL, R3 rejected all day, the days of heat shimmer were dropped).
|
||||
# Greyed if the median agitation ≥ STRONG.
|
||||
RATIO = 1.5 # tuned on E-003 (WILDSTRUBEL: by day, corr ×1.7 to ×3 the night)
|
||||
SHARE = 0.5 # share of the sensors agitated at the same time
|
||||
RATIO_ERR = 1.5 # tuned on E-003 (error_value ×1.7 to ×3.2 the night; ≈ ×1 at noon elsewhere)
|
||||
REJECT_MAX = 0.15 # E-003 ≤ 7 %; cobweb (E-002) and rain (E-005) ≥ 18 %; rejections beyond the habit since learner
|
||||
REJECT_MAX = 0.15 # E-003 ≤ 7 %; cobweb (E-002) and rain (E-005) ≥ 18 %; rejections beyond the habit since wf-improve
|
||||
# session Visibility, mod. 1, S1 (WILDSTRUBEL: 4 to 14 %, against 9 to 22 % raw)
|
||||
STRONG = 2.0 # E-003 (×2.3 to ×2.6) is strong; an episode between ×1.5 and ×2 stays visible
|
||||
MIN_DURATION = timedelta(hours=3) # E-003 lasts 10-11 h; drops the morning of the real movement of E-004 (2 h), which
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
"""Benchmark of the learner (D-116): the detectors against all the labeller's labels (labeling/auto/).
|
||||
"""Benchmark of the wf-improve workflow (D-116): the detectors against all the labeller's labels (labeling/auto/).
|
||||
|
||||
For each station of labeling/auto/ that has its folder in data/, the score per family and per instant of score.py
|
||||
(D-112), over the range of its labels: from the day of the first to the midnight after the last. Then the total, family
|
||||
by family, which says whether the detectors do better overall from one session to the next. The block of the session's
|
||||
station is its score; the other blocks, the non-regression: a family that goes back on a station is a regression, even
|
||||
if the total rises.
|
||||
Usage: python3 detectors/scripts/learner/regression.py [--stations A,B]
|
||||
Usage: python3 detectors/scripts/wf-improve/regression.py [--stations A,B]
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
@@ -8,7 +8,7 @@ range classified with its 3 days of history). The category of an event is its fa
|
||||
The score is a fact, not a verdict: the labels can be wrong too. No label on the range: exits with an error.
|
||||
Label fields (written by Diagnostic-app): debut, fin, categorie (the family), description.
|
||||
Run from the working repository, where detectors/ is mounted.
|
||||
Usage: python3 detectors/scripts/learner/score.py <station folder> --from YYYY-MM-DD --to YYYY-MM-DD
|
||||
Usage: python3 detectors/scripts/wf-improve/score.py <station folder> --from YYYY-MM-DD --to YYYY-MM-DD
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
@@ -1,7 +1,7 @@
|
||||
"""For the upgrader: the cadence of the measurements of the range, the number of targets and references, then each event
|
||||
of the range with its sensors (categories.py only gives their number). Same computation as categories.py without
|
||||
--chunk (3 days of history).
|
||||
Usage: python3 detectors/scripts/learner/sensors.py <folder> --from YYYY-MM-DD --to YYYY-MM-DD
|
||||
Usage: python3 detectors/scripts/wf-improve/sensors.py <folder> --from YYYY-MM-DD --to YYYY-MM-DD
|
||||
"""
|
||||
import argparse
|
||||
import statistics
|
||||
@@ -1,12 +1,12 @@
|
||||
---
|
||||
name: learner
|
||||
description: Automatic session that improves the detectors from an analysis of the Data page of Diagnostic-app (labeling/auto/analyses/<analysis>/) — the upgrader diagnoses and fixes, the validator judges; the goal, detectors closer to the analysis's labels on its range, without overfitting; the safeguard, the benchmark on all stations and all families of labeling/auto/. Started by the "Optimiser les détecteurs" button of the app (claude -p --plugin-dir detectors), or by the user, /detectors:learner <analysis>.
|
||||
name: wf-improve
|
||||
description: Automatic session that improves the detectors from an analysis of the Data page of Diagnostic-app (labeling/auto/analyses/<analysis>/) — the upgrader diagnoses and fixes, the validator judges; the goal, detectors closer to the analysis's labels on its range, without overfitting; the safeguard, the benchmark on all stations and all families of labeling/auto/. Started by the "Optimiser les détecteurs" button of the app (claude -p --plugin-dir detectors), or by the user, /detectors:wf-improve <analysis>.
|
||||
argument-hint: "<analysis>"
|
||||
---
|
||||
|
||||
# learner: the detectors from an analysis
|
||||
# wf-improve: the detectors from an analysis
|
||||
|
||||
You are the orchestrator. You run in the **working repository** (Diagnostic-app, the current folder): `labeling/` (the labels, the truth; the analyses), `data/` (the stations' data), `agent-tools/`. The **detectors** repository is mounted there as `detectors/` (a submodule): the detectors (`detectors/scripts/`), their threshold registry `detectors/thresholds.md`, the session tools (`detectors/scripts/learner/`) and the sessions (`detectors/learner/`). Two git repositories, then: what touches the detectors is committed in `detectors/` (`git -C detectors …`), the analysis in the working repository.
|
||||
You are the orchestrator. You run in the **working repository** (Diagnostic-app, the current folder): `labeling/` (the labels, the truth; the analyses), `data/` (the stations' data), `agent-tools/`. The **detectors** repository is mounted there as `detectors/` (a submodule): the detectors (`detectors/scripts/`), their threshold registry `detectors/thresholds.md`, the session tools (`detectors/scripts/wf-improve/`) and the sessions (`detectors/wf-improve/`). Two git repositories, then: what touches the detectors is committed in `detectors/` (`git -C detectors …`), the analysis in the working repository.
|
||||
|
||||
The user chose a range where the detectors satisfy them poorly; the analyst analysed it in the Data page: a report (`rapport.md`) and labels, added to `labeling/auto/<STATION>.jsonl`. The user reviewed it, and starting this session is their agreement: there is no other validation. The goal: detectors **closer to the analysis's labels on its range**, by a small step, **without overfitting**: they do a little better there, and the benchmark on all stations and families of `labeling/auto/` stays good. You call the agents in order, pass on their outputs and commit. **You judge nothing yourself**: you only read the verdict lines. **You ask the user no question**: the session runs alone to the end.
|
||||
|
||||
@@ -14,16 +14,16 @@ The user chose a range where the detectors satisfy them poorly; the analyst anal
|
||||
|
||||
- Argument: the name of an analysis, a folder of `labeling/auto/analyses/` (e.g. `NORDEND_2026-11-04T2112_2026-11-07T1412`); a path to this folder counts as its name. `ANALYSIS` = the name, `A` = `labeling/auto/analyses/<ANALYSIS>`. Without `<A>/demande.json`, or without `detectors/.claude-plugin/plugin.json`: say so and stop, without a commit.
|
||||
- From `<A>/demande.json` (written by the app, French keys): `STATION`, `début`, `fin` (UTC). `FROM` = the day of `début`, `TO` = the day after the day of `fin` (YYYY-MM-DD): the range of the commands, in whole days.
|
||||
- `SESSION` = `date -u +%Y-%m-%dT%H%MZ`; session folder `detectors/learner/<SESSION>_<ANALYSIS>/`, below `<session>`.
|
||||
- `SESSION` = `date -u +%Y-%m-%dT%H%MZ`; session folder `detectors/wf-improve/<SESSION>_<ANALYSIS>/`, below `<session>`.
|
||||
- **Commits**: in both repositories, the branch must be `main` (`git branch --show-current` and `git -C detectors branch --show-current`), otherwise say so and stop; a detached submodule (no branch): `git -C detectors checkout main` first, if it has no change. You always commit by path (`git add <paths> && git commit -m "<message>" -- <paths>`, or with `git -C detectors`, paths relative to `detectors/`), never `git add -A`, `git stash` or `git checkout` outside your paths: another session may be working in these repositories. Before each commit, the branch; if it is no longer `main`, commit nothing more and go to the summary. Each message ends with an empty line then `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`. You push nothing.
|
||||
- **Step 0, the analysis in git**: the app does not commit; if `git status --porcelain -- <A> labeling/auto/<STATION>.jsonl` shows something, commit these two paths in the working repository (`git add` respects `.gitignore`: `travail/` stays out), message `Learner <ANALYSIS>: labels of the analysis`. Then note the **session base**: `git -C detectors rev-parse HEAD`.
|
||||
- `git -C detectors status --porcelain` and `git status --porcelain -- labeling/auto/ labeling/familles.md` (`familles.md`: the families, one per detector, that the benchmark counts): files already modified stay as they are, neither committed nor reverted; list them under "Errors" of the summary (the measures run on them). Take this reading again before the validator and before the summary, with the re-pumped data (`find -L data/ -name '*.csv' -newer <session>/benchmark-before.txt | cut -d/ -f2 | sort -u`): a file changed during the session by another session (a label touched up in the Data page, for example) goes under "Errors"; before the validator, then rerun the benchmark before at the session base (in a `mktemp -d` folder: `git -C detectors archive <base> --prefix=detectors/ | tar -x -C <folder>`, links to `data/`, `labeling/` and `agent-tools/` of the working repository, then `python3 <folder>/detectors/scripts/learner/regression.py`) into `<session>/benchmark-before-rerun.txt`, and add to the validator's request: `Changed during the session: <paths or stations>; benchmark before rerun: <session>/benchmark-before-rerun.txt.`
|
||||
- **Step 0, the analysis in git**: the app does not commit; if `git status --porcelain -- <A> labeling/auto/<STATION>.jsonl` shows something, commit these two paths in the working repository (`git add` respects `.gitignore`: `travail/` stays out), message `wf-improve <ANALYSIS>: labels of the analysis`. Then note the **session base**: `git -C detectors rev-parse HEAD`.
|
||||
- `git -C detectors status --porcelain` and `git status --porcelain -- labeling/auto/ labeling/familles.md` (`familles.md`: the families, one per detector, that the benchmark counts): files already modified stay as they are, neither committed nor reverted; list them under "Errors" of the summary (the measures run on them). Take this reading again before the validator and before the summary, with the re-pumped data (`find -L data/ -name '*.csv' -newer <session>/benchmark-before.txt | cut -d/ -f2 | sort -u`): a file changed during the session by another session (a label touched up in the Data page, for example) goes under "Errors"; before the validator, then rerun the benchmark before at the session base (in a `mktemp -d` folder: `git -C detectors archive <base> --prefix=detectors/ | tar -x -C <folder>`, links to `data/`, `labeling/` and `agent-tools/` of the working repository, then `python3 <folder>/detectors/scripts/wf-improve/regression.py`) into `<session>/benchmark-before-rerun.txt`, and add to the validator's request: `Changed during the session: <paths or stations>; benchmark before rerun: <session>/benchmark-before-rerun.txt.`
|
||||
|
||||
**The two measures** of the session, at each step:
|
||||
- **the benchmark**, the non-regression: `python3 detectors/scripts/learner/regression.py > <session>/benchmark-<step>.txt 2>&1` (about 3 min). The score per family and per instant on each station of `labeling/auto/`, then their `Total`;
|
||||
- **the range score**, the goal: `python3 detectors/scripts/learner/score.py data/<STATION> --from <FROM> --to <TO> > <session>/range-<step>.txt 2>&1`. The labels of `labeling/auto/<STATION>.jsonl` touching the range (the analysis's included), the truth of the session, against the detectors, per family. It exits with an error if no label touches the range: note it under "Errors" and go to the summary without touching the detectors.
|
||||
- **the benchmark**, the non-regression: `python3 detectors/scripts/wf-improve/regression.py > <session>/benchmark-<step>.txt 2>&1` (about 3 min). The score per family and per instant on each station of `labeling/auto/`, then their `Total`;
|
||||
- **the range score**, the goal: `python3 detectors/scripts/wf-improve/score.py data/<STATION> --from <FROM> --to <TO> > <session>/range-<step>.txt 2>&1`. The labels of `labeling/auto/<STATION>.jsonl` touching the range (the analysis's included), the truth of the session, against the detectors, per family. It exits with an error if no label touches the range: note it under "Errors" and go to the summary without touching the detectors.
|
||||
|
||||
**Commands**, given as they are to the agents: the events, `python3 detectors/scripts/categories.py data/<STATION> --from <FROM> --to <TO>`; the sensors, `python3 detectors/scripts/learner/sensors.py data/<STATION> --from <FROM> --to <TO>`. For the validator only, which looks at all stations, the events command with `<STATION>`, `<from>` and `<to>` to fill from the header of the station's block in the benchmark: `python3 detectors/scripts/categories.py data/<STATION> --from <from> --to <to>`.
|
||||
**Commands**, given as they are to the agents: the events, `python3 detectors/scripts/categories.py data/<STATION> --from <FROM> --to <TO>`; the sensors, `python3 detectors/scripts/wf-improve/sensors.py data/<STATION> --from <FROM> --to <TO>`. For the validator only, which looks at all stations, the events command with `<STATION>`, `<from>` and `<to>` to fill from the header of the station's block in the benchmark: `python3 detectors/scripts/categories.py data/<STATION> --from <from> --to <to>`.
|
||||
|
||||
## 2. In order
|
||||
|
||||
@@ -32,18 +32,18 @@ Each agent is called with the Agent tool and its `subagent_type` (those of this
|
||||
Session commits (the session folder, the detectors, `thresholds.md`) are made in `detectors/`: `git -C detectors add <relative paths> && git -C detectors commit -m "<message>" -- <relative paths>`.
|
||||
|
||||
1. **Measures before**: step `before`, so `<session>/benchmark-before.txt` and `<session>/range-before.txt`.
|
||||
Commit `learner/<SESSION>_<ANALYSIS>/`, message `Learner <ANALYSIS>: benchmark`.
|
||||
Commit `wf-improve/<SESSION>_<ANALYSIS>/`, message `wf-improve <ANALYSIS>: benchmark`.
|
||||
2. **Upgrader**, a single pass:
|
||||
`Analysis: <A>/. Station: <STATION>, range <début> → <fin> (UTC). Events command: <command>. Sensors command: <sensors command>. Range score: python3 detectors/scripts/learner/score.py data/<STATION> --from <FROM> --to <TO> (before: <session>/range-before.txt). Benchmark before: <session>/benchmark-before.txt, to rerun with python3 detectors/scripts/learner/regression.py. Session folder: <session>. Files already modified at the start of the session: <list | none>. Write <session>/modification.md.`
|
||||
The upgrader commits each fix it keeps itself, in `detectors/`, with its benchmark `<session>/benchmark-S<n>.txt`, message `Learner <ANALYSIS>: S<n>`: the validator can thus have a single one removed.
|
||||
`Analysis: <A>/. Station: <STATION>, range <début> → <fin> (UTC). Events command: <command>. Sensors command: <sensors command>. Range score: python3 detectors/scripts/wf-improve/score.py data/<STATION> --from <FROM> --to <TO> (before: <session>/range-before.txt). Benchmark before: <session>/benchmark-before.txt, to rerun with python3 detectors/scripts/wf-improve/regression.py. Session folder: <session>. Files already modified at the start of the session: <list | none>. Write <session>/modification.md.`
|
||||
The upgrader commits each fix it keeps itself, in `detectors/`, with its benchmark `<session>/benchmark-S<n>.txt`, message `wf-improve <ANALYSIS>: S<n>`: the validator can thus have a single one removed.
|
||||
Then the measures, step `after`, in `<session>/benchmark-after.txt` and `<session>/range-after.txt`.
|
||||
Commit of the session folder, message `Learner <ANALYSIS>: modification (<verdict>)`. No fix kept (verdict `nothing to change`, or no `, S<n>` commit: `git -C detectors log --format=%h --grep ': S[0-9]*$' <base>..HEAD`): go to the summary.
|
||||
Commit of the session folder, message `wf-improve <ANALYSIS>: modification (<verdict>)`. No fix kept (verdict `nothing to change`, or no `, S<n>` commit: `git -C detectors log --format=%h --grep ': S[0-9]*$' <base>..HEAD`): go to the summary.
|
||||
3. **Validator**, which reads neither the diagnosis nor the detail of the modification:
|
||||
`Analysis: <ANALYSIS>. Modification: <session>/modification.md. Benchmark: <session>/benchmark-before.txt → <session>/benchmark-after.txt, to rerun with python3 detectors/scripts/learner/regression.py; after each fix: <session>/benchmark-S*.txt. Range score: <session>/range-before.txt → <session>/range-after.txt. Events command: <validator's command>. Base (in detectors/): <session base>. Write <session>/validation.md.`
|
||||
Commit of the session folder, message `Learner <ANALYSIS>: validator (<verdict>)`.
|
||||
`Analysis: <ANALYSIS>. Modification: <session>/modification.md. Benchmark: <session>/benchmark-before.txt → <session>/benchmark-after.txt, to rerun with python3 detectors/scripts/wf-improve/regression.py; after each fix: <session>/benchmark-S*.txt. Range score: <session>/range-before.txt → <session>/range-after.txt. Events command: <validator's command>. Base (in detectors/): <session base>. Write <session>/validation.md.`
|
||||
Commit of the session folder, message `wf-improve <ANALYSIS>: validator (<verdict>)`.
|
||||
- `keep`: go to the summary.
|
||||
- `remove`: for each fix of the `**Remove**:` line, its session commits (`git -C detectors log --format=%h --grep ': S<n>$' <session base>..HEAD`), from the most recent to the oldest: `git -C detectors revert --no-edit <commit>`, then amend its message so that it ends with the `Co-Authored-By` line (`git -C detectors commit --amend -m "Learner <ANALYSIS>: S<n> removed" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"`). A conflict: `git -C detectors revert --abort`, and do as for `cancel`. Then go to the summary.
|
||||
- `cancel`, or any other word: `git -C detectors checkout <session base> -- 'scripts/*.py' thresholds.md` (without the files already modified noted in §1; `scripts/learner/` is not touched), commit of these paths, message `Learner <ANALYSIS>: modification cancelled`.
|
||||
- `remove`: for each fix of the `**Remove**:` line, its session commits (`git -C detectors log --format=%h --grep ': S<n>$' <session base>..HEAD`), from the most recent to the oldest: `git -C detectors revert --no-edit <commit>`, then amend its message so that it ends with the `Co-Authored-By` line (`git -C detectors commit --amend -m "wf-improve <ANALYSIS>: S<n> removed" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"`). A conflict: `git -C detectors revert --abort`, and do as for `cancel`. Then go to the summary.
|
||||
- `cancel`, or any other word: `git -C detectors checkout <session base> -- 'scripts/*.py' thresholds.md` (without the files already modified noted in §1; `scripts/wf-improve/` is not touched), commit of these paths, message `wf-improve <ANALYSIS>: modification cancelled`.
|
||||
|
||||
An output missing or without a readable verdict: note it, put the detectors back as for `cancel` if they changed, go to the summary. Do not rerun an agent more than once.
|
||||
|
||||
@@ -77,7 +77,7 @@ the line of each family of the `Total`, before → end; then the other changed l
|
||||
<none, or what was missing; the session base and `git -C detectors log --oneline <base>..HEAD`>
|
||||
```
|
||||
|
||||
Commit, in `detectors/`, of the session folder, message `Learner <ANALYSIS>: summary`. Then, in the working repository, the submodule at its new commit: `git add detectors && git commit -m "Learner <ANALYSIS>: detectors" -- detectors`, if `git status --porcelain -- detectors` shows it changed.
|
||||
Commit, in `detectors/`, of the session folder, message `wf-improve <ANALYSIS>: summary`. Then, in the working repository, the submodule at its new commit: `git add detectors && git commit -m "wf-improve <ANALYSIS>: detectors" -- detectors`, if `git status --porcelain -- detectors` shows it changed.
|
||||
|
||||
## 4. Final answer
|
||||
|
||||
+22
-22
@@ -1,32 +1,32 @@
|
||||
# Detector thresholds
|
||||
|
||||
Every threshold of the detectors (`scripts/*.py`, one per family of `labeling/familles.md`, D-119) is **provisional**: tuned on the six data sets of events E-001 to E-005 and NORDEND of October, then adjusted by learner sessions. When bulk data arrives, this is the list to review. A constant added or changed updates this file in the same change. Check: `grep -nE "^[A-Z_0-9]+ = " scripts/*.py`.
|
||||
Every threshold of the detectors (`scripts/*.py`, one per family of `labeling/familles.md`, D-119) is **provisional**: tuned on the six data sets of events E-001 to E-005 and NORDEND of October, then adjusted by wf-improve sessions. When bulk data arrives, this is the list to review. A constant added or changed updates this file in the same change. Check: `grep -nE "^[A-Z_0-9]+ = " scripts/*.py`.
|
||||
|
||||
The detectors must detect everything (the user, 2026-10-26, D-068): a setting chosen to hide something from the client is not a detection rule; the learner may change it. What is shown to the client is decided in a separate layer.
|
||||
The detectors must detect everything (the user, 2026-10-26, D-068): a setting chosen to hide something from the client is not a detection rule; the wf-improve workflow may change it. What is shown to the client is decided in a separate layer.
|
||||
|
||||
"Learner X, mod. 1, S2": suggestion S2 of modification 1 of the learner session on the analysis X. "X, batch 01": first batch of the earlier loop on station X.
|
||||
"wf-improve X, mod. 1, S2": suggestion S2 of modification 1 of the wf-improve session on the analysis X. "X, batch 01": first batch of the earlier loop on station X.
|
||||
|
||||
| Detector | Constant | Value | Tuned on |
|
||||
|---|---|---|---|
|
||||
| `visibility` | `K_MAD` | 10 | E-001 (fog 11 to 57 MAD), E-005 (10 to 70 MAD) |
|
||||
| | `FACTOR` | 2 | drops sensors with a near-zero MAD (E-004, E-005); also, alone (without `K_MAD`), something passing in front of one or two targets or a reference, rejected for a single measurement (point 2): learner Obstruction, mod. 1, S3 (WILDSTRUBEL: by day, heat shimmer inflates the MAD; R1 rejected at ×3.6 under 10 MAD) |
|
||||
| | `FACTOR` | 2 | drops sensors with a near-zero MAD (E-004, E-005); also, alone (without `K_MAD`), something passing in front of one or two targets or a reference, rejected for a single measurement (point 2): wf-improve Obstruction, mod. 1, S3 (WILDSTRUBEL: by day, heat shimmer inflates the MAD; R1 rejected at ×3.6 under 10 MAD) |
|
||||
| | `RUN_MIN` | 3 | E-001, rain of E-005 |
|
||||
| | `BRIEF_MIN` | 3 | trains of E-005 (5 to 17 sensors) |
|
||||
| | `SALIENCE` | 3 | trains (5 to 17) against the cobweb of E-002 (2 to 9) |
|
||||
| | `FRAC` | 0.15 | tails of PR09 and CTR01 (NORDEND October) |
|
||||
| | `BREAK` | 2 h | NORDEND October, night of the 13th to the 14th |
|
||||
| | `K_TARGET` | 5 | leaves of E-005 (6 to 25 MAD) |
|
||||
| | `TARGET_MIN` | 3 | —; 2 when `TARGET_MIN` measurement steps do not fit in `TARGET_GAP`: learner Obstruction, mod. 1, S6 (BREITHORN, one measurement every 4 h: vehicles parked at night, two or three rejections) |
|
||||
| | `TARGET_MIN` | 3 | —; 2 when `TARGET_MIN` measurement steps do not fit in `TARGET_GAP`: wf-improve Obstruction, mod. 1, S6 (BREITHORN, one measurement every 4 h: vehicles parked at night, two or three rejections) |
|
||||
| | `TARGET_GAP` | 6 h | leaves of E-005 (7 rejections in 12 h); at least two measurement steps (same S6) |
|
||||
| | `GAP_STEPS` | 2.5 steps | timestamp jitter |
|
||||
| | `REJECT_FACTOR` | 1.2 | FLORIDA, batch 01: a rejection at 10 MAD is high without doubling (targets at level 0.6); drops C24 of E-009 (0.68, rejected at ×1.04) |
|
||||
| | `HALF` | 0.5 | level "none" (point 5): NORDEND 09/10 23:00 (68 % rejected) against the rain of E-005 (24 to 33 %) |
|
||||
| | `K_DIFFUSE` | 2.5 MAD | diffuse nuisance (point 8): median, over the sensors, of the error_value in MAD above each one's median, `RUN_MIN` instants in a row, rejected or not: Poor visibility "reduced"; not within heat shimmer; learner Visibility, mod. 1, S2 (JUNGFRAU, FLORIDA, NORDEND: reduced visibility of the whole scene without rejection; at 2, nights labelled thermal at JUNGFRAU and jumps back of WILDSTRUBEL absorbed; at 3, half the gain) |
|
||||
| | `K_DIFFUSE` | 2.5 MAD | diffuse nuisance (point 8): median, over the sensors, of the error_value in MAD above each one's median, `RUN_MIN` instants in a row, rejected or not: Poor visibility "reduced"; not within heat shimmer; wf-improve Visibility, mod. 1, S2 (JUNGFRAU, FLORIDA, NORDEND: reduced visibility of the whole scene without rejection; at 2, nights labelled thermal at JUNGFRAU and jumps back of WILDSTRUBEL absorbed; at 3, half the gain) |
|
||||
| | `FEW` | 3 targets | Poor visibility on at most 3 targets, no reference: an object in front of them, Target obscured (boundary "Visibility or Obstruction" of `labeling/familles.md`); BREITHORN, batch 01 (vehicles parked in front of 2 or 3 targets out of 12) |
|
||||
| `visibility` (heat shimmer) | `RATIO` | 1.5 | E-003 (×1.7 to ×3 the night) |
|
||||
| | `SHARE` | 0.5 | — |
|
||||
| | `RATIO_ERR` | 1.5 | E-003 (×1.7 to ×3.2) |
|
||||
| | `REJECT_MAX` | 15 % | E-003 ≤ 7 %; cobweb and rain ≥ 18 %; rejections beyond each sensor's habit (its share of rejections over the export): learner Visibility, mod. 1, S1 (WILDSTRUBEL: a reference or targets usually rejected dropped the days of heat shimmer; 4 to 14 % against 9 to 22 % raw) |
|
||||
| | `REJECT_MAX` | 15 % | E-003 ≤ 7 %; cobweb and rain ≥ 18 %; rejections beyond each sensor's habit (its share of rejections over the export): wf-improve Visibility, mod. 1, S1 (WILDSTRUBEL: a reference or targets usually rejected dropped the days of heat shimmer; 4 to 14 % against 9 to 22 % raw) |
|
||||
| | `STRONG` | 2× | E-003 (×2.3 to ×2.6) |
|
||||
| | `MIN_DURATION` | 3 h | E-003 (10-11 h) against the morning of E-004 (2 h); the classification drops this morning too, whatever the duration |
|
||||
| `displacement` (jumps) | `K` | 20 noises | E-001 (at 15, two healthy targets greyed); do not lower for the trains, seen by `visibility` |
|
||||
@@ -34,38 +34,38 @@ The detectors must detect everything (the user, 2026-10-26, D-068): a setting ch
|
||||
| | `PLATEAU` | 8 ordinary variations | same point: gap between the medians of the day after and the day before, in ordinary variations of the level from one day to the next; NIEDERHORN, batch 01 (e09: 13; E-001, healthy targets: 6 at most) |
|
||||
| | `K_PEAK` | 6 noises | one-step peak common to several targets: NORDEND, batch 01 (shakes of the whole image, 6 to 11 noises) |
|
||||
| | `PEAK_MIN` | 3 targets | NORDEND, batch 01 (3 to 5 targets per shake) |
|
||||
| | `BASE` | 4 measurements | 1 h at 15 min; the return of a jump is judged on an hour of measurements, BASE at most, one at least (learner MICHIGAN, mod. 1: at a 4 h step, a peak back at the next measurement became "Large shift") |
|
||||
| | `BASE` | 4 measurements | 1 h at 15 min; the return of a jump is judged on an hour of measurements, BASE at most, one at least (wf-improve MICHIGAN, mod. 1: at a 4 h step, a peak back at the next measurement became "Large shift") |
|
||||
| | `GAP` | 1 h | — |
|
||||
| | `BACK_H` | 24 h | E-002 (plateaus up to 11.5 h) |
|
||||
| | `REPEAT` | 5 jumps back | CLARIDEN, batch 01: PR_BG flips 28 times in 2.5 days, the other targets 3 at most (morning of the movement) |
|
||||
| | `FLIP` | 36 h | problem target (D-074): PR_BG of CLARIDEN flips 2.5 days; C8 and C13 of E-009, 5 flips in 1 h 20 on the morning of the movement, stay brief jumps; 6 h → 36 h: learner Target issue, mod. 1, S2 (NORDEND PR12, PR14, PR08, PR05, JUNGFRAU BJ_G: 12 to 28 h of flips within a loss of visibility, labelled Visibility or Obstruction, absorbed by it) |
|
||||
| | `FLIP` | 36 h | problem target (D-074): PR_BG of CLARIDEN flips 2.5 days; C8 and C13 of E-009, 5 flips in 1 h 20 on the morning of the movement, stay brief jumps; 6 h → 36 h: wf-improve Target issue, mod. 1, S2 (NORDEND PR12, PR14, PR08, PR05, JUNGFRAU BJ_G: 12 to 28 h of flips within a loss of visibility, labelled Visibility or Obstruction, absorbed by it) |
|
||||
| | `MAX_FLIPPING` | 2 targets | problem target: FLORIDA, the two stuck targets; E-002, cobweb on the references, 15 targets flip at once |
|
||||
| | `LARGE` | 0.5 | large shift ("Large shift", D-074): E-004 and CLARIDEN, 18 targets on the morning of the movement; also, slow drift of the camera (LARGE of the targets in corr and of the sensors in raw x, y, on the same side: no "Slow drift" over the whole window, only the pieces of `PIECE`): learner MICHIGAN, mod. 1 (BREITHORN kept); learner Displacement, mod. 1, S2 (NEBRASKA reported in pieces, JUNGFRAU still silent) |
|
||||
| `displacement` (slow drift) | `HOURS` | 20 h out of 24 | as the trend of the Displacements tab (D-055); as a share of the hours the station measures (learner MICHIGAN, mod. 1: 5 out of 6 at one measurement every 4 h, no day counted) |
|
||||
| | `LARGE` | 0.5 | large shift ("Large shift", D-074): E-004 and CLARIDEN, 18 targets on the morning of the movement; also, slow drift of the camera (LARGE of the targets in corr and of the sensors in raw x, y, on the same side: no "Slow drift" over the whole window, only the pieces of `PIECE`): wf-improve MICHIGAN, mod. 1 (BREITHORN kept); wf-improve Displacement, mod. 1, S2 (NEBRASKA reported in pieces, JUNGFRAU still silent) |
|
||||
| `displacement` (slow drift) | `HOURS` | 20 h out of 24 | as the trend of the Displacements tab (D-055); as a share of the hours the station measures (wf-improve MICHIGAN, mod. 1: 5 out of 6 at one measurement every 4 h, no day counted) |
|
||||
| | `DAYS` | 5 days | D-055: a trend needs at least 5 UTC days |
|
||||
| | `SIZE` | 1 | D-055: drift over the period ≥ one daily amplitude; amplitude of the day: 9th and 1st deciles of `corr` |
|
||||
| | `STEADY` | 0.6 | D-055: Kendall tau of the daily levels; a step from one day to the next of more than half the drift drops it (a jump) |
|
||||
| | `PIECE` | 3.5 amplitudes | piecewise drift, for a target without a drift over the whole window (point 4b of `displacement.py`): learner Displacement, mod. 1, S1 (rows of CLARIDEN, 3.8 to 8.3 amplitudes; JUNGFRAU, thermal cycles without a Displacement label, 3.1 at most; at 1, NORDEND and JUNGFRAU overflow) |
|
||||
| | `MAX_HOLE` | 2 days | a drift piece (point 4b) does not go from one kept day to the next more than 2 days apart: learner Displacement, mod. 2, S1 (CLARIDEN, hole from 10-03 to 10-05: the pieces from 09-28 to 11-01 fall; at 1, the missing days of a half-day hole cut the drifts of CLARIDEN and NEBRASKA) |
|
||||
| | `PIECE` | 3.5 amplitudes | piecewise drift, for a target without a drift over the whole window (point 4b of `displacement.py`): wf-improve Displacement, mod. 1, S1 (rows of CLARIDEN, 3.8 to 8.3 amplitudes; JUNGFRAU, thermal cycles without a Displacement label, 3.1 at most; at 1, NORDEND and JUNGFRAU overflow) |
|
||||
| | `MAX_HOLE` | 2 days | a drift piece (point 4b) does not go from one kept day to the next more than 2 days apart: wf-improve Displacement, mod. 2, S1 (CLARIDEN, hole from 10-03 to 10-05: the pieces from 09-28 to 11-01 fall; at 1, the missing days of a half-day hole cut the drifts of CLARIDEN and NEBRASKA) |
|
||||
| `thermal_displacement` | `MIN_HOURS` | 20 h | —; days of the profile |
|
||||
| | `PARTIAL` | 12 h | an incomplete day (fewer than `MIN_HOURS`) gets its episode if it looks like the profile on the hours it has, without entering the profile; learner Thermal displacement, mod. 1, S4 (SNCF01, NORDEND, JUNGFRAU: labelled cycles of days cut by a hole; `MIN_HOURS` at 12 or 16 made NIEDERHORN go back) |
|
||||
| | `PARTIAL` | 12 h | an incomplete day (fewer than `MIN_HOURS`) gets its episode if it looks like the profile on the hours it has, without entering the profile; wf-improve Thermal displacement, mod. 1, S4 (SNCF01, NORDEND, JUNGFRAU: labelled cycles of days cut by a hole; `MIN_HOURS` at 12 or 16 made NIEDERHORN go back) |
|
||||
| | `REPRO` | 0.6 | pure noise ≈ 0 |
|
||||
| | `SHARE` | 0.5 | NORDEND October just under (9/19) |
|
||||
| | `WARM` | 0.3 | —; only says there is a cycle (at least `MIN_WARM_HOURS` warm hours), its bounds are at `EDGE` |
|
||||
| | `EDGE` | 0.1 | bounds of the wave: start and return of the profile, the return after midnight if needed, at most to the next day's start; learner Thermal displacement, mod. 1, S1 (NORDEND, JUNGFRAU, SNCF01: labelled evenings and nights, the wave stopped at the last warm hour; at 0.15 and 0.2, fewer nights found, FLORIDA overflows in the evening anyway) |
|
||||
| | `EDGE` | 0.1 | bounds of the wave: start and return of the profile, the return after midnight if needed, at most to the next day's start; wf-improve Thermal displacement, mod. 1, S1 (NORDEND, JUNGFRAU, SNCF01: labelled evenings and nights, the wave stopped at the last warm hour; at 0.15 and 0.2, fewer nights found, FLORIDA overflows in the evening anyway) |
|
||||
| | `DAY_MATCH` | 0.5 | — |
|
||||
| | `NIGHT` | 8 h | — |
|
||||
| | `R2_MIN` | 0.3 | 24 h sinusoid: PS des Coudres 14 axes out of 22 above, CLARIDEN 0 out of 48; no longer asked for the camera (raw positions) since WILDSTRUBEL, batch 01 (1 target out of 7, the wrong pointings crush the fit) |
|
||||
| | `MIN_WARM_HOURS` | 2 h | WILDSTRUBEL, batch 01: a single warm hour is not a cycle (NIEDERHORN, E-005: noon, trains and holes) |
|
||||
| | `SLIDE_K` | 3 noises | common camera slide without a cycle ("Camera thermal movement"): gap from the median of the day before, raw positions and corrected curves; learner MICHIGAN, mod. 1 (at 2, Displacement instants of BREITHORN taken) |
|
||||
| | `COLD` | 4 one-hour noises | the camera moves with the cold ("Camera thermal movement", without a cycle): the pose (tilt, rotation or scale) moves away from its median of the day before while the temperature falls under its own, the same hour; noise: variations over an hour, one step at 4 h; learner Thermal displacement, mod. 1, S3 (BREITHORN, CLARIDEN, NEBRASKA: targets in opposite directions, or compensated; at 3, Displacement instants of NEBRASKA taken) |
|
||||
| | `SLIDE_K` | 3 noises | common camera slide without a cycle ("Camera thermal movement"): gap from the median of the day before, raw positions and corrected curves; wf-improve MICHIGAN, mod. 1 (at 2, Displacement instants of BREITHORN taken) |
|
||||
| | `COLD` | 4 one-hour noises | the camera moves with the cold ("Camera thermal movement", without a cycle): the pose (tilt, rotation or scale) moves away from its median of the day before while the temperature falls under its own, the same hour; noise: variations over an hour, one step at 4 h; wf-improve Thermal displacement, mod. 1, S3 (BREITHORN, CLARIDEN, NEBRASKA: targets in opposite directions, or compensated; at 3, Displacement instants of NEBRASKA taken) |
|
||||
| `no_data` | `MISSING` | 2 steps | user's rule, 2026-10-23; also applied between the last measurement and the end of the window read (CLARIDEN, batch 01) |
|
||||
| `target_issue` | `CALM` | 0.25 | (detector added on 2026-10-27, see its docstring) |
|
||||
| | `SHARE` | 0.5 | —; the target hidden at times ("Target obscured") |
|
||||
| | `SHARE_WEEK` | 0.25 | share of the calm instants rejected that makes a degraded day of the problem target ("Degraded target", at least `WEEK` days): regularity rather than the share of the day; learner Target issue, mod. 1, S1 (WILDSTRUBEL R3 rejected from 04:00 to 16:00 every day, 30 to 45 %: seen only on days without night measurements; NORDEND PR04 and CTR01, NIEDERHORN PWD_IMP_B08_024_C09, under half) |
|
||||
| | `MIN_HOURS` | 3 h | hours of calm instants per day, at the station's cadence (median gap between two instants): 12 instants at 15 min; learner NEBRASKA, batch 01 (one measurement every 4 h: no day counted, the lost reference REFCOLH22 was not seen) |
|
||||
| | `SHARE_WEEK` | 0.25 | share of the calm instants rejected that makes a degraded day of the problem target ("Degraded target", at least `WEEK` days): regularity rather than the share of the day; wf-improve Target issue, mod. 1, S1 (WILDSTRUBEL R3 rejected from 04:00 to 16:00 every day, 30 to 45 %: seen only on days without night measurements; NORDEND PR04 and CTR01, NIEDERHORN PWD_IMP_B08_024_C09, under half) |
|
||||
| | `MIN_HOURS` | 3 h | hours of calm instants per day, at the station's cadence (median gap between two instants): 12 instants at 15 min; wf-improve NEBRASKA, batch 01 (one measurement every 4 h: no day counted, the lost reference REFCOLH22 was not seen) |
|
||||
| | `MIN_DAYS` | 2 days | NORDEND: isolated days on 16 sensors |
|
||||
| | `WEEK` | 7 days | problem target ("Degraded target") if hindered at least a week in a row, otherwise a target hidden at times ("Target obscured"): boundary "Target issue or Obstruction" of `labeling/familles.md`; learner NEBRASKA, batch 01 (BREITHORN, parked vehicles, 2 to 5 days; NEBRASKA and MICHIGAN, lost reference, 25 days; JUNGFRAU, PM_G, 63 days) |
|
||||
| | `BRIDGE` | 1 day | non-degraded days in a row that do not break the run of a problem target ("Degraded target": `WEEK` days from the first to the last degraded day, `SHARE_WEEK` of rejections over the bridged run); learner Obstruction, mod. 1, S1 (FLORIDA, SNCF01, NORDEND, NIEDERHORN: problem targets cut into Target obscured by a day under `SHARE`; BREITHORN, parked vehicles, unchanged); 2 → 1: learner Target issue, mod. 1, S1 (at a quarter, two bridging days joined the parked vehicles of BREITHORN, POUI22, into a week) |
|
||||
| | `WEEK` | 7 days | problem target ("Degraded target") if hindered at least a week in a row, otherwise a target hidden at times ("Target obscured"): boundary "Target issue or Obstruction" of `labeling/familles.md`; wf-improve NEBRASKA, batch 01 (BREITHORN, parked vehicles, 2 to 5 days; NEBRASKA and MICHIGAN, lost reference, 25 days; JUNGFRAU, PM_G, 63 days) |
|
||||
| | `BRIDGE` | 1 day | non-degraded days in a row that do not break the run of a problem target ("Degraded target": `WEEK` days from the first to the last degraded day, `SHARE_WEEK` of rejections over the bridged run); wf-improve Obstruction, mod. 1, S1 (FLORIDA, SNCF01, NORDEND, NIEDERHORN: problem targets cut into Target obscured by a day under `SHARE`; BREITHORN, parked vehicles, unchanged); 2 → 1: wf-improve Target issue, mod. 1, S1 (at a quarter, two bridging days joined the parked vehicles of BREITHORN, POUI22, into a week) |
|
||||
| | `STUCK` | 0.05 | stuck target (D-074): E-011, C2 and C3 at 0.006 and 0.016 of the others' noise; NORDEND, E-001: 0.7 at the lowest |
|
||||
| `categories` (classification) | `MERGE` | 1 h | rule validated by the user: same variety less than an hour apart, a single event; two Obstruction (Target obscured, Brief obstruction) only if they share a sensor (learner Obstruction, mod. 1, S1 and S3), two Target issue too (learner Target issue, mod. 1, S3); a Brief obstruction only absorbs the jumps back within `MERGE` after its instant (S3) |
|
||||
| `categories` (classification) | `MERGE` | 1 h | rule validated by the user: same variety less than an hour apart, a single event; two Obstruction (Target obscured, Brief obstruction) only if they share a sensor (wf-improve Obstruction, mod. 1, S1 and S3), two Target issue too (wf-improve Target issue, mod. 1, S3); a Brief obstruction only absorbs the jumps back within `MERGE` after its instant (S3) |
|
||||
Reference in new issue
Block a user