Analyst knowledge: reading-rules.md, stations.md, a phenomenon catalogue in families.md; the label map and knowledge/ archived
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
1c8479d962
commit
4688d8ab9e
16 files changed
+201
-17
No files matched your search
@@ -3,9 +3,9 @@
|
||||
labeling: the AIMsight displacement labels and the knowledge to place them, in a single copy on the workstation (`README.md`). Diagnostic-App reads and writes there, Optimizer only reads, both through a link `labeling -> ~/labeling`; commits are made here.
|
||||
|
||||
- **The labels**: `manual/<STATION>.jsonl` (the expert, for him alone), `auto/<STATION>.jsonl` (the agents, reviewed by `review`), one per line, `{"id", "start", "end", "family", "description", ...}`. The Diagnostic-App analyses: `auto/analyses/<STATION>_<start>_<end>/`.
|
||||
- **The knowledge**: `families.md` (the families, their boundaries), `labeling-rules.md` (bounds, a single event, description, stated cause), `label-map.md` (the phenomena, via its summary), `knowledge/`.
|
||||
- **The knowledge**, one source per kind (`README.md`), maintained by hand: `families.md` (the families, their boundaries, the catalogue of phenomena), `labeling-rules.md` (bounds, a single event, description, stated cause), `reading-rules.md` (what the data mean), `stations.md` (the installations); the cases are the labels, through `precedents.py`; what the data are: `tools/DATA.md`. A piece of knowledge lives in one of them only.
|
||||
- **The tools**, standard library: `python3 validate.py [STATION | file.jsonl]` (the rules), `python3 ids.py [STATION]` (the stable id of new labels), `python3 precedents.py "<text>" [--exclude STATION START END] [--family F] [--k N]` (the RAG; it starts the Ollama container `ollama-bge-m3` by itself if it does not answer; `--check` without it).
|
||||
- **Verify**: `python3 validate.py` (today 18 expected errors: the labels submitted by the app without `review`) and `python3 precedents.py --check`.
|
||||
- **Verify**: `python3 validate.py` (today 19 expected errors: the labels submitted by the app without `review`) and `python3 precedents.py --check`.
|
||||
|
||||
Rules:
|
||||
- **`manual/` is the expert's, for him alone**: no agent and no process reads or writes there; the whole project (analyst, precedents, benchmark, optimization) uses only `auto/`.
|
||||
|
||||
@@ -4,12 +4,21 @@ The AIMsight displacement labels and all the domain knowledge to place them, in
|
||||
|
||||
- `manual/<STATION>.jsonl`: the expert's labels, for him alone: no process and no agent uses them; `auto/<STATION>.jsonl`: those of the Support-Agent `etiqueteur`, reviewed (`review`), and those submitted by the Diagnostic-App `analyst`, without `review` until they are reviewed; one label per line, `{"id", "start", "end", "family", "description", ...}`, UTC times, `family` = the family.
|
||||
- `auto/analyses/<STATION>_<start>_<end>/`: the analyses launched from the Data page of Diagnostic-App (`work/` outside git).
|
||||
- `families.md`, `labeling-rules.md`, `label-map.md`, `knowledge/`: how to label, what the labels have taught.
|
||||
- The knowledge, one source per kind, maintained by hand by the user; the analyst reads the four files in full:
|
||||
|
||||
| Kind of knowledge | Source |
|
||||
|---|---|
|
||||
| The cases | the labels of `auto/`, through `precedents.py` |
|
||||
| The families, their boundaries, the catalogue of phenomena | `families.md` |
|
||||
| How to place a label | `labeling-rules.md` |
|
||||
| What the data mean: measurement chain, gaps, reading rules, false friends | `reading-rules.md` |
|
||||
| The installation of each station | `stations.md` |
|
||||
| What the data are: files, columns, image quality, images | `tools/DATA.md` (tools repository) |
|
||||
- `validate.py` (the labels against the rules), `ids.py` (the stable id of each).
|
||||
- `precedents.py`: the RAG of the labels. The labels of `auto/` closest in meaning to a text, with their provenance (reviewed or not); `--exclude STATION START END` leaves out the analyzed range. bge-m3 model of the Ollama container `ollama-bge-m3` (started by the script itself if it does not answer; port 11435); the `.index/` index, outside git, updates itself at every call (only new or modified labels go through the model; ~8 min to redo everything on CPU).
|
||||
|
||||
Diagnostic-App and Optimizer reach it through a link `labeling -> ~/labeling`, ignored by git: Diagnostic-App reads and writes there (labels, analyses), Optimizer only reads; commits are made here. `validate.py` and `ids.py` run from here or through the link (`python3 labeling/validate.py`). Descriptions are written in English from now on.
|
||||
|
||||
History taken from Support-Agent (the `labeling/` folder, up to commit 48a3092), then the state of Diagnostic-App on 2026-11-13 (462ba9e): the archives of the etiqueteur and of the reviews (`auto/etiqueteur/`, `auto/revue/`, `auto/revue-croisee/`) and the old `labeling.py` tool remain in the history. The notes cite their sources as they were in Support-Agent (`docs/…`, `events/`, `run.md`, `agent-diagnostic/`, the skills `/labels-map`, `/review-auto-labels`, `/review-event`…): paths of that repository, obsolete, kept read-only.
|
||||
History taken from Support-Agent (the `labeling/` folder, up to commit 48a3092), then the state of Diagnostic-App on 2026-11-13 (462ba9e): the archives of the etiqueteur and of the reviews (`auto/etiqueteur/`, `auto/revue/`, `auto/revue-croisee/`) and the old `labeling.py` tool remain in the history.
|
||||
|
||||
`docs/archive/`: the design and plan of the labeler, archived on 2026-11-16 (not built).
|
||||
`docs/archive/`: the design and plan of the labeler (not built), the label map and `knowledge/`, archived on 2026-11-16.
|
||||
@@ -1,3 +1,4 @@
|
||||
# Archive
|
||||
|
||||
- `2026-11-14-labeler-design.md`, `2026-11-14-labeler.md`: the design and plan of the "labeler", many low-effort labels with the help of the detectors, never built. Archived on 2026-11-16 by the user: quantity without quality, and a circular risk (labels guided by the detectors that are used to train them). Outside the repository architecture (`~/Diagnostic-App/docs/architecture.drawio`); not maintained.
|
||||
- `label-map.md`, `knowledge/`: the label map (phenomenon cards, reading rules, false friends, context of the stations), regenerated by `/labels-map` in Support-Agent until 2026-11-05, and the notes it relied on. Archived on 2026-11-16 (`docs/superpowers/specs/2026-11-16-analyst-knowledge-design.md`): one source per kind of knowledge. The cases are the labels of `auto/`, through `precedents.py`; the cards became the phenomenon catalogue of `families.md`; the reading rules, false friends, measurement chain, gaps and lessons of cases `reading-rules.md`; the context of the stations `stations.md`; the formats, columns, quality metrics and images `tools/DATA.md`. `knowledge/thermal-cycle.md` was outdated (D-117). Not maintained.
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
+67
-10
@@ -2,20 +2,20 @@
|
||||
|
||||
One label, one anomaly, one family (D-109 to D-111, D-119). An anomaly is what shows up on `corr_x` and `corr_y`: the curves move (a peak, a jump, a drift, noise) or are missing (a gap). The family says the **cause** of what is seen on the corrected curves, or of what is missing from them, and what the client should conclude. Its key, its very name, goes in the `family` field of the label; the labeling tool offers it in its Category menu. Each family has its detector, in `detectors/scripts/` (D-119): the category of a detector event is its family.
|
||||
|
||||
| Key | Detector | What happens | For the client | Label-map phenomena |
|
||||
|---|---|---|---|---|
|
||||
| `Visibility` | `visibility.py` | things are harder to see: a diffuse nuisance between the camera and the scene. The image is faint or blurred (rain, fog, condensation, light, dirty lens: the spots are fainter or blurrier), or it shakes (turbulence: hot air, above a road or in the sun, makes the spots shake, a fast noise on the curves). The description says which is seen | do not trust the curves over the range; it is not a movement | Rain at night; Visibility drops over the whole scene; Atmospheric turbulence |
|
||||
| `Obstruction` | `obstruction.py` | an object hides one or a few targets, for a moment or for a long time (train, vehicle, plant, snow on the target) | ignore the curve of the hidden targets over the range | A passage hides the targets; A plant in front of a target; An obstacle covers part of the spot |
|
||||
| `Thermal displacement` | `thermal_displacement.py` | a component (the structure, the camera and its mount) moves with temperature; it is always labeled, whether the corrected curves show it or the camera compensates it (D-117); the direction does not matter | normal, nothing to do | The structure expands and contracts; The camera moves through a thermal effect; A rise over several days with the temperature drop |
|
||||
| `Displacement` | `displacement.py` | a lasting jump, a slow drift, and anything that changes the corrected curves without an established cause (rule D-074: do not alarm the client, hide nothing) | to check: look at the images, monitor | A real movement of the structure |
|
||||
| `Target issue` | `target_issue.py` | a constantly problematic target: rejected almost always, badly oriented, badly installed, badly configured, dirty | act on the target; ignore its curve | — |
|
||||
| `No data` | `no_data.py` | no measurement for any sensor, all curves missing (acquisition, power, transmission); the cause, if the metrics or the images show it, otherwise not established | the curves are missing over the range; the cause says whether to act | — |
|
||||
| Key | Detector | What happens | For the client |
|
||||
|---|---|---|---|
|
||||
| `Visibility` | `visibility.py` | things are harder to see: a diffuse nuisance between the camera and the scene. The image is faint or blurred (rain, fog, condensation, light, dirty lens: the spots are fainter or blurrier), or it shakes (turbulence: hot air, above a road or in the sun, makes the spots shake, a fast noise on the curves). The description says which is seen | do not trust the curves over the range; it is not a movement |
|
||||
| `Obstruction` | `obstruction.py` | an object hides one or a few targets, for a moment or for a long time (train, vehicle, plant, snow on the target) | ignore the curve of the hidden targets over the range |
|
||||
| `Thermal displacement` | `thermal_displacement.py` | a component (the structure, the camera and its mount) moves with temperature; it is always labeled, whether the corrected curves show it or the camera compensates it (D-117); the direction does not matter | normal, nothing to do |
|
||||
| `Displacement` | `displacement.py` | a lasting jump, a slow drift, and anything that changes the corrected curves without an established cause (rule D-074: do not alarm the client, hide nothing) | to check: look at the images, monitor |
|
||||
| `Target issue` | `target_issue.py` | a constantly problematic target: rejected almost always, badly oriented, badly installed, badly configured, dirty | act on the target; ignore its curve |
|
||||
| `No data` | `no_data.py` | no measurement for any sensor, all curves missing (acquisition, power, transmission); the cause, if the metrics or the images show it, otherwise not established | the curves are missing over the range; the cause says whether to act |
|
||||
|
||||
## The boundaries
|
||||
|
||||
- **Visibility or Obstruction**: the nuisance is judged by its extent, not by the light. The whole scene hindered (rain, fog, underexposed image, milky dawn, sun in the field) is Visibility; a few targets hidden by an object, the rest intact, are Obstruction, even if the object is lit (machine, headlights, light of a parked vehicle). A spider web goes by what it does: a thread that hides a target, Obstruction; the web that veils the image, Visibility.
|
||||
- **Obstruction, whatever the duration**: an obstacle that stays (parked vehicle, machine, plant, animal or insects against the lens, mark on the glass in front of a single spot) is Obstruction.
|
||||
- **A hindered reference** (map: "A reference dropped: the base changes", "Two references lost"): the family of its cause. A passage in front of a reference that makes the base jump is Obstruction when the object is seen, or deduced from a passage signature; a reference right at its threshold, with nothing in front of it, is not a passage: Visibility, or Displacement without a cause. The description says it is a reference, and that the targets move together because of it.
|
||||
- **A hindered reference** (phenomena: "A reference dropped", "Two references lost"): the family of its cause. A passage in front of a reference that makes the base jump is Obstruction when the object is seen, or deduced from a passage signature; a reference right at its threshold, with nothing in front of it, is not a passage: Visibility, or Displacement without a cause. The description says it is a reference, and that the targets move together because of it.
|
||||
- **A jump then the return**: the family of its cause, seen on the images. A passing vehicle, and its reflection, are Obstruction (a reflection on the roadway lasting only one measurement is a vehicle: read the measurement before and the one after); the reflection of the sun on the fixed scene is Visibility. Without an established cause: Displacement.
|
||||
- **Thermal displacement or Displacement**: what follows the temperature of the housing **and comes back with it** (same position at equal temperature), even over several days, or whose thermal cause the expert gives, is Thermal displacement; what follows it only as a trend, without coming back at equal temperature, and what the temperature does not explain, are Displacement.
|
||||
- **Thermal displacement or Visibility**: curves that follow the scale and rotation of the camera with the housing temperature, the `error_value` at its level or rising only a little, nothing on the images, several targets moving apart in opposite directions through the rest of the compensation, are Thermal displacement. References moving apart from each other and from the targets, an `error_value` rising on most sensors, an episode with sharp edges, or a nuisance seen on the images, are Visibility, even at the temperature trough. The two can coexist: two labels. A daytime agitation, the pose shaking without the `error_value` rising, is not turbulence.
|
||||
@@ -23,8 +23,65 @@ One label, one anomaly, one family (D-109 to D-111, D-119). An anomaly is what s
|
||||
- **Target issue or episode of another family**: a Target issue requires rejections specific to the target (outside the episodes shared with its neighbors) almost every day of the range, also on covered or rainy days, under the station's normal regime (the floodlight at night, daylight). A reference rejected like this every day is one, even if its entries and exits do not make the targets jump: the compensation loses it every day. Rejections that come only with a condition seen on the images (sun, rain, night of poor visibility) are episodes of the family of that condition, one label each time. An `error_value` right at the threshold, with few own rejections, is not a Target issue: first count the own rejections of each target and each reference, day by day.
|
||||
- **Target issue or Displacement**: a changed configuration (moved sensors, changed starting position, setting) is Target issue; the jump that results is not a Displacement.
|
||||
|
||||
## The phenomena
|
||||
|
||||
What has been labelled so far, one line each: **name**: what is seen; how to tell it from its neighbours. The hypotheses to weigh for a moment of a range; the cases, their bounds and descriptions come from the labels, through `precedents.py`. A phenomenon not in this list is labelled too, without forcing a cause.
|
||||
|
||||
### No data
|
||||
- **A gap common to several stations**: the measurements missing at the same minutes on every station measuring then; a cause outside the stations, not established. Read the other stations before the station's metrics.
|
||||
- **The station runs, the measurement is missing**: one station, powered and communicating (synchronization, image or `DB_not_uploaded` at the missing instant), misses a measurement.
|
||||
- **The station goes silent on its own, for a long time**: no measurements, no images, no metrics, for hours or days, without the signature of a lack of energy.
|
||||
- **The station shuts down for lack of energy**: the battery does not last the night; voltage below the knee before the gap, a restart with the solar charge.
|
||||
- **Mains cut: the station runs on its battery**: a station on mains stops, or comes back on a drawn-down battery, until the mains return.
|
||||
|
||||
### Obstruction
|
||||
- **A pass-by hides the targets for the length of one measurement**: a train, a vehicle, passers-by: the hidden targets rejected for one acquisition, back at the next; a reference behind it makes the whole base jump. Insects give the same signature: only the image tells.
|
||||
- **A target left under the threshold latches onto a headlight, a reflection or another spot**: during a pass-by, one false point kept in the curve, without rejection; `dominance` rises on a recent camera.
|
||||
- **A vehicle or machine that stays in front of the targets**: the same sensors hidden for hours or days, sometimes a reference; one label per parking.
|
||||
- **An animal very close to the lens**: lit at night, it hides part or all of the scene for one or two measurements; a spider is not its web.
|
||||
- **A plant shaken by the wind in front of a target**: one target, at times, on windy days; it grows over weeks.
|
||||
- **An obstacle covers part of the spot**: an obstacle, or a mark on the glass in front of one spot, hides part of it for hours or weeks: the measured centre slides; the sliding is the artifact, the jump back is the return to normal.
|
||||
|
||||
### Visibility
|
||||
- **Rain**: drops in front of the lens veil the spots during the hours of rain, at night above all; the references, higher up, may suffer less.
|
||||
- **The soiled glass**: after rain, drop marks veil part of the scene durably; under a single spot, it is an obstacle (Obstruction).
|
||||
- **Visibility drops over the whole scene, without an obstacle**: fog, condensation, a milky dawn: flux down on all sensors, `error_value` up, rejections; raw positions without a step.
|
||||
- **Part of the scene goes dark at night**: the distant spots, or a few neighbours at the edge of the group, lose their light, nothing in front.
|
||||
- **A reference loses its brightness, without an obstacle**: one or two references weaken and shrink, most often at night, then come back on their own; the centre does not move.
|
||||
- **Atmospheric turbulence**: by day, air heated above a road, ballast or rails makes the spots tremble: a fast noise, `error_value` up. Without the `error_value` rising, an agitation of the pose is the camera heating (Thermal displacement).
|
||||
- **The sun in the field or behind a spot**: the sun enters the field, or lights the background of a target or a reference as bright as it: rejections at the same hours on sunny days only.
|
||||
- **A faint spot lost when the light is missing**: a faint target crosses its threshold at the change from night to day, under an overcast sky or in a noisy night image; its rejections come with the light, not every day (else Target issue).
|
||||
|
||||
### Target issue
|
||||
- **A target rejected at all its measurements**: never accepted, whatever the weather.
|
||||
- **A target right at the threshold, the whole batch**: its `error_value` keeps crossing the threshold, with many own rejections.
|
||||
- **A target rejected every night or every day**: under the station's ordinary regime (floodlight, daylight), at the same hours, whatever the weather.
|
||||
- **A target that degrades, and sometimes recovers**: seen worse and worse from a date on; back only after weeks, if ever, often at another level.
|
||||
- **The configuration of a sensor has changed**: a sensor moved or reset: its curve changes with nothing moving in the image; the jump is not a Displacement.
|
||||
|
||||
### Thermal displacement
|
||||
- **The camera moves through a thermal effect**: the pose follows the housing temperature, every day, the compensation erasing more or less of it; labelled even when fully compensated.
|
||||
- **The camera drifts away when the temperature of its housing drops, without a daily cycle**: sheltered (a building, a tunnel), it drifts in jerks with the housing temperature and comes back when it warms.
|
||||
- **The structure expands and contracts with the temperature**: a structure that can move (a bridge) moves with its own slow rhythm, each target its amplitude, the references stable; two labels a day, expansion and contraction.
|
||||
- **A rise over several days with the drop in temperature**: the temperature keeps falling for days and the structure keeps rising; it comes back at equal temperature (else Displacement).
|
||||
|
||||
### Displacement
|
||||
- **A step of the camera that the compensation does not erase**: the camera moves at once and a remainder passes into the curves; a step that leaves nothing is not labelled.
|
||||
- **The camera tilts little by little**: over weeks, the tilt rises and the compensation corrects some targets better than others; the targets do not move relative to each other.
|
||||
- **The base changes without a reference lost**: all references kept, the distance between them changes a little; all targets shift together without moving in the image.
|
||||
- **A real movement of the structure**: targets shift and stay, with references and pose stable and a good image; if the camera and references moved too, the amplitude is not established.
|
||||
- **Targets move slowly, alone, then come back**: one or a few targets, hours or days, without a camera movement or an obstructed view.
|
||||
- **A target or a small group jumps and stays shifted**: at once, to another level, without the camera moving or a visible cause.
|
||||
- **A step taken during a measurement outage**: on resumption, targets do not find their previous level.
|
||||
|
||||
### Family of the cause
|
||||
- **A reference dropped: the base changes**: one reference above its threshold, dropped: all targets jump together, the reprojection error at zero; the family is that of what hindered the reference.
|
||||
- **Two references lost: the camera is blind**: two of three references dropped one after the other: every point of the range is invalid; the family of the cause.
|
||||
- **A compensation on two references**: a station built with two, or having lost one for good: the slightest difference between them passes into all the targets; the family of what disturbs them.
|
||||
- **A spider web in front of the lens**: Visibility when it veils the image, Obstruction when a thread hides a few spots; no signal at all is possible, only the images show it.
|
||||
|
||||
## What is not labeled
|
||||
|
||||
- What touches neither `corr_x` nor `corr_y`: an `error_value` that rises without effect on them; a camera step well compensated, even on two references, as long as the `corr` keep nothing of it (if some remains, the label bounds the step and concludes "the camera moves"). Thermal movement, on the other hand, is always labeled (D-117).
|
||||
- The station's intended configuration (map: "Targets that are twins of their references").
|
||||
- The station's intended configuration (targets that are twins of their references: the same prism serves as target and reference; they are named with their reference in its labels).
|
||||
- The expert can say so with the key `None` (labeling tool: "None (curves untouched)"): such a label is not counted.
|
||||
+2
-2
@@ -1,6 +1,6 @@
|
||||
# The labeling rules
|
||||
|
||||
How to place a label: the only source of these rules for the Diagnostic-App `analyst` (before it, the Support-Agent `etiqueteur`) and for those who review their labels. The families and their boundaries are in `families.md`; the phenomena, in `label-map.md`.
|
||||
How to place a label: the only source of these rules for the Diagnostic-App `analyst` (before it, the Support-Agent `etiqueteur`) and for those who review their labels. The families, their boundaries and the catalogue of phenomena are in `families.md`; what the data mean, in `reading-rules.md`.
|
||||
|
||||
Sources in brackets: [instruction] the section "Les étiquettes" of `.claude/agents/etiqueteur.md`; [cross <family>] the cross review of 2026-11-04, `auto/revue-croisee/2026-11-04/<family>/revue.md`; [finishing <folder>] its finishing pass, `auto/revue-croisee/2026-11-04/finition/<folder>/revue.md`; [<STATION>] the station review, `auto/revue/<STATION>_<start>/revue.md`; [consistency] that of BREITHORN, MICHIGAN and NEBRASKA; [expert] a label or an answer of the expert.
|
||||
|
||||
@@ -91,7 +91,7 @@ Sources in brackets: [instruction] the section "Les étiquettes" of `.claude/age
|
||||
- **A Target issue says when the rejections stop before the end of the label**: the end is read on the curves, the rejections may stop before. [cross Target issue]
|
||||
- **A No data states the metrics and the image of the instant**: present or not, checked, for each gap. [NIEDERHORN, cross No data]
|
||||
|
||||
- **On a recent camera (image quality filled, `tools/scripts/quality.py`), an occultation is written only after reading the flux of the instant**: a cut flux proves it, a full flux denies it; the hidden targets are named from their flux, each time, and a hidden target that stayed under its threshold is stated (its point stays, wrong). The metrics: `labeling/knowledge/quality-metrics.md`. [LENZSPITZE quality review, D-129]
|
||||
- **On a recent camera (image quality filled, `tools/scripts/quality.py`), an occultation is written only after reading the flux of the instant**: a cut flux proves it, a full flux denies it; the hidden targets are named from their flux, each time, and a hidden target that stayed under its threshold is stated (its point stays, wrong). The metrics: `tools/DATA.md`. [LENZSPITZE quality review, D-129]
|
||||
- **"Bigger spots" is written only after comparison with a calm day of the same image regime and reading of the `fwhm`**: a spot that has lost its light can look swollen. [LENZSPITZE quality review, D-129]
|
||||
- **A jump without rejection is in no list of rejections**: on a recent camera, `quality.py --labels --outside` gives the degraded instants outside the labels; `corr` is checked at each. [LENZSPITZE quality review, D-129]
|
||||
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
# Reading rules
|
||||
|
||||
What the data of a station mean: where each cause enters the measurement, what it touches, and the rules and false friends that the expert and the reviewed labels have established. What the files and columns are: `tools/DATA.md`. How to place a label: `labeling-rules.md`; the families and their boundaries: `families.md`, whose rules are not repeated here.
|
||||
|
||||
Cases are cited by label id (`NORDEND-80`, in `auto/<STATION>.jsonl`; `precedents.py` finds others); `[expert]`: the expert's word, from a label that is not in `auto/`. Maintained by hand.
|
||||
|
||||
## 1. How a measurement is made
|
||||
|
||||
1. **The scene**: targets on the structure, references around it, assumed fixed.
|
||||
2. **The view**: the air between the camera and the targets, and what can pass or settle in front.
|
||||
3. **The image**: the lens, then the capture (IRON, or DIFF which combines several captures).
|
||||
4. **The pattern measurement**: each sensor is found in the image: `x`, `y`, `error_value`; on recent cameras, the image quality. Above `error_threshold`, the measurement is rejected.
|
||||
5. **The compensation**: a 2D similarity (translation, rotation, scale) fitted on the references (`global_ref`) gives the camera pose (`pan`, `tilt`, `rotation`, `scale`, `sigma0`) and, per reference, `reproj_err` and `inlier`.
|
||||
6. **The corrected curve**: `corr_x`, `corr_y`, the position of the target once the estimated camera movement is removed.
|
||||
|
||||
A displayed value is the end of this chain: find the stage that gave way.
|
||||
|
||||
## 2. Where each cause enters, and what it touches
|
||||
|
||||
| Stage | Cause | What it touches | Time | Witnesses |
|
||||
|---|---|---|---|---|
|
||||
| View (air) | fog, rain, condensation: **reduced visibility** | a homogeneous drop in flux; `error_value` rises, rejections; raw positions without a step | hours | quality (`flux`), images |
|
||||
| View (air) | **heat haze** (turbulence) | a fast noise; only when the structure really heats up; not under a bridge | the hot hours | temperature, `error_value` by day versus night |
|
||||
| View (obstacle) | object, leaves, snow on a target: **occultation**; a passage | one or a few targets; on a recent camera, their flux cut, the neighbours intact (LENZSPITZE-9); a passage: one measurement | from one measurement to weeks | quality, images |
|
||||
| Target | problem target (badly oriented, dirty, glued) | one target, lastingly | days and more | `error_value` of the target alone |
|
||||
| Lens | **thermal deformation of the optics** (sun on the station) | the whole image deforms, which a similarity cannot represent | short: follows the sun on the station | housing temperature |
|
||||
| Camera and mount | **expansion of the mount**; wind, jolt, shock | raw positions of all sensors, references included, together; the compensation absorbs it (`corr` flat) or badly: `corr` follows the inflections of the camera movement | specific to the mount | camera pose, housing temperature |
|
||||
| References | **reference hidden or less well seen** | the transformation is redone without it: points shifted at once, often grouped in one part of the image, the shift growing with the distance to the references; back when it returns: reversible | the time of the nuisance | `error_value`, `reproj_err` of the references, `sigma0`, camera pose |
|
||||
| References | a reference moved | the `corr` of the targets shifted lastingly | lasting | `reproj_err` of the reference |
|
||||
| Structure | **thermal expansion of the structure** | `corr`, an amplitude specific to each target, references stable | the longest: slow heating, late cooling | no air temperature (section 3) |
|
||||
| Structure | real movement | `corr` of one or several targets, which lasts; references and pose stable, good image quality | lasting | all of the above, at calm |
|
||||
|
||||
**A jump that returns** to normal is not serious: the return is what makes it benign. Saying "jump" is not enough; the question is its cause, quite often a reference hidden or less well seen.
|
||||
|
||||
## 3. Thermal effects
|
||||
|
||||
- **Three movement effects**: the lens, the mount, the structure, told apart by their heating and cooling times; the structure is the slowest. Heat haze is apart: a noise, not a movement.
|
||||
- **The cases cross**: a camera on a bridge has two structures, the one that carries it (the mount) and the one it monitors.
|
||||
- **In waves**: expansion, then contraction; often stronger in Y than in X, depending on the orientation.
|
||||
- **The kind of structure says whether a thermal movement is possible**: a bridge yes; a rock face, too massive, or a road, no: the movement is then that of the camera (`stations.md`). [expert, WILDSTRUBEL-102]
|
||||
- **The camera can move through a thermal effect at a different rhythm from the structure**, not having the same heating time. [expert]
|
||||
- **The temperature that counts is that of the camera housing, not that of the air around**, even sheltered. The temperatures available are those of the compute board (`sensor_*_deg_c`), inside the housing, heated by the sun and by the computing, as hourly means: a witness for the lens and the mount, not for the structure nor the air. `temperature_deg_c`, on a few stations: unknown sensor. [expert, BREITHORN-14]
|
||||
- **A change of regime is understood through the temperature**: look at it. [expert]
|
||||
- **The direction of a thermal movement is read on the housing temperature, not on the hour**; when the scale of the camera follows it, the effect is on the camera. [NORDEND-155, CLARIDEN-13]
|
||||
|
||||
## 4. Gaps
|
||||
|
||||
- **A gap is not necessarily an outage**: acquisition, transmission, ingestion and display are distinct layers.
|
||||
- At the end of a file, empty values can be only a synchronization delay (`last_sync`): a delay only if the other stations do not have their data yet either.
|
||||
- At the start of a file, empty values precede the first measurement: not an outage.
|
||||
- A gap surrounded by a normal voltage is not an energy problem.
|
||||
- **A gap is read with the other stations and the metrics** (the boundary "A gap" of `families.md`). [JUNGFRAU-52, NORDEND-50, WILDSTRUBEL-201, CLARIDEN-19]
|
||||
- **A power cut leaves a signature**: the voltage below the knee (about 12.8 V) just before or just after the gap; the temperature drops during the gap more than at the same hours on nights without a gap, which alone does not settle it (a common gap can do the same, an outage that comes back after sunrise shows nothing).
|
||||
- **A station on mains that comes back with its battery drawn down has lost the mains.**
|
||||
- **A `NaN` or `inf` value** in an export is an absence of measurement.
|
||||
|
||||
## 5. Reading rules
|
||||
|
||||
- **What moves in the image is read before the cause, on the raw positions against the overall movement** (`overview.py --rigid`): the targets do not move relative to each other, it is the camera [BREITHORN-2]; targets move alone, a movement is not excluded [CLARIDEN-14]; nothing moves and the curves jump, it is the base or the configuration [WILDSTRUBEL-136, CLARIDEN-38].
|
||||
- **A jump is judged against the neighbours**: if it brings the target back toward them, it is a return to normal, and the artifact was before; if it takes it away from them, it is to be treated quite differently. [expert]
|
||||
- **A level is also judged against the past of the target, over weeks**: a jump that brings it back to an old level is a return. [NORDEND-94, NORDEND-169]
|
||||
- **A high `error_value` is not a problem in itself: its variations count.** [expert]
|
||||
- **A very stable `error_value` says the targets stay well visible**: while they move, it rather suggests a poor compensation than an obstruction of the view. [expert]
|
||||
- **A rejected measurement is removed from the display, not a measurement just under the threshold**: a target left under the threshold keeps its jump. [expert, NIEDERHORN-13]
|
||||
- **At least three references are needed; dropping a single one is enough to distort everything**: a kind of triangulation. **The position of the targets rests on the network of references**: references badly seen, targets badly anchored. [expert]
|
||||
- **The reprojection error says how many references hold**: at zero, only two remain, a very bad sign, not a perfect measurement; absent, only one, and no measurement is possible. Except on a station built with two references, where it is always at zero. [JUNGFRAU-22, FLORIDA-4, expert]
|
||||
- **The pose of the camera is reliable only if the references hold**: a reference rejected, dropped or sliding against the targets distorts it; a single hindered target does not. [FLORIDA-58]
|
||||
- **A drift specific to a few targets that follows the camera, on the same dates, is the scene deforming with it**, not a second movement; it is separate only if it has its own dates. [NEBRASKA-6, NEBRASKA-5]
|
||||
- **Neighbouring cameras are read together**: the same movement, the same loss or the same gap on the cameras of one building have a common cause; what touches only one is on its line of sight. [MICHIGAN-3, BREITHORN-20]
|
||||
|
||||
## 6. False friends
|
||||
|
||||
- **Jumps of many targets, all in the same direction, are not a movement** when they fall at the reference switches: raw positions without a step, daily cycle that continues. [expert]
|
||||
- **A target that slides slowly away from its neighbours is not moving** when its `error_value` rises with the sliding and a step brings it back toward them. [expert]
|
||||
- **A rise over several days is not necessarily a real movement**: it may follow a lasting drop in temperature. [expert]
|
||||
- **Corrections stuck at zero are not a failure**: a twin of a reference. [expert]
|
||||
- **No signal does not mean no obstruction**: only the images show the web. [expert]
|
||||
- **A web is not the spider**: the web veils in part for hours or days; the spider hides everything for the length of one measurement. [expert]
|
||||
- **References that do not move in the rain do not contradict the rain**: higher up, they suffer it less. [NIEDERHORN-12]
|
||||
- **The step of a real movement is not its amplitude** when the camera and the references have moved too. [expert]
|
||||
- **A swarm of insects is not a train**: same signature; only the image tells them apart. [NIEDERHORN-66]
|
||||
- **In a tunnel, a temperature drop at the same moment does not make a thermal effect**: the bump of a group comes with the `error_value` and returns before the scale of the camera. [CLARIDEN-12, CLARIDEN-13]
|
||||
- **The return of the `error_value` is not the end of an obstruction**: after a mark on the glass, the curves stay shifted. [NORDEND-169]
|
||||
- **A spot on a thumbnail that follows the measurement is not necessarily there**: the frame follows a measurement perhaps latched onto the neighbour; only a fixed frame shows it. [FLORIDA-43]
|
||||
|
||||
## 7. Non-negotiable rules
|
||||
|
||||
- Judge nothing on a binary `inlier`: use continuous weights, residuals, influence.
|
||||
- Neither `sigma0` alone nor the number of references is ever a sufficient criterion.
|
||||
- The image quality of a target is not the quality of the reference network; a small residual is not a well-constrained transformation.
|
||||
- A common-mode movement is not a structural movement; coherent references are not physically fixed references: the camera cannot prove that a network moving as a block is fixed.
|
||||
|
||||
## 8. Lessons from cases outside the labels
|
||||
|
||||
- **When everything moves together, the measurements alone do not separate the structure from the camera** (Barcelona, a bridge whose vault subsided: the references, the camera and the structure moved; the expert called it a real movement, confirmed by other monitoring systems of the site). An unstable compensation then distorts the amplitude of a real movement: the movement is reported, its amplitude is not established. [expert]
|
||||
- **Two prisms too close interfere**: a 24 h cycle correlated between the two, or isolated peaks on one target at a time, too large for the structure, with no obstruction on the images; a neighbour much brighter than the others. On a recent camera, `dominance` shows it. [client report]
|
||||
- **Vegetation grows over days and weeks**: a partial, progressive occultation, not reversible from one measurement to the next like a passage; the hardest obstruction according to the expert. [client report]
|
||||
- **References masked (two of four, by vegetation)**: a 24 h cycle on pan, roll and tilt, badly compensated, is then not a movement of the camera but a base too weak for the transformation. [client report]
|
||||
- **A target of low contrast**: by day, square waves on one axis, back at night, `error_value` rising by day without reaching the threshold. [client report]
|
||||
- **Snow**: noise and apparent movements on all sensors and the camera, a brief loss of data, the signal quickly back. [client report]
|
||||
+17
@@ -0,0 +1,17 @@
|
||||
# The stations
|
||||
|
||||
What each station is: its structure, its camera, its references, its neighbours. Context for reading a station's data, not a rule: what happens at a station is in its labels (`auto/<STATION>.jsonl`, `precedents.py`). The kind of structure says whether a thermal movement of the structure is possible (`reading-rules.md`, section 3). Cases are cited by label id; `[expert]`: the expert's word. Maintained by hand: a new station gets its row when its first range is analysed.
|
||||
|
||||
| Station | Structure | Camera | References | Neighbouring cameras |
|
||||
|---|---|---|---|---|
|
||||
| BREITHORN | in a building (a factory, indoors) [expert] | one measurement every four hours; COLJ19 and POUI19 much closer to the camera than the other targets [expert]; the camera nods forward over October, compensation incomplete [expert] | REFH22, REFPOU22 among the sensors at the bottom of the image | MICHIGAN and NEBRASKA, in the same building: a drift common to the three [BREITHORN-2] |
|
||||
| CLARIDEN | a tunnel under construction, with other measurement systems [expert] | images always dark; on mains | four references, D-302-C, PR-291, PR-293, PR-295; PR-284 is a local reference, read as a target; D-302-B and PR_BG on the same prism [CLARIDEN-38] | — |
|
||||
| FLORIDA | — | — | two references only, p1_r and r1_r, each with its twin target (Sellette 3, Pont Ernen) [expert] | its one-measurement gaps at night fall at the same instants as SNCF01's |
|
||||
| JUNGFRAU | a bridge, which rises as it cools [expert] | under the deck, fixed to the bridge | three references off the bridge, REF_D, REF_G, REF_CT | — |
|
||||
| LENZSPITZE | a railway seen from under a bridge, a work site and machines next to it | recent camera, with image quality; `x`, `y` relative in the export (`tools/DATA.md`); only `temperature_deg_c` is filled | nine references, including the distant 8011 to 8014 | — |
|
||||
| MICHIGAN | in the same building as BREITHORN, indoors [expert] | one measurement every four hours | two references since the loss of REFPOU22 on 09.10 [MICHIGAN-3]; COLH21' and REFH22' on the same point until 05.10 [MICHIGAN-1] | BREITHORN, NEBRASKA: lost a reference at the same measurement as NEBRASKA |
|
||||
| NEBRASKA | in the same building as BREITHORN [expert] | one measurement every four hours | two references since the loss of REFCOLH22 [NEBRASKA-2] | BREITHORN, MICHIGAN |
|
||||
| NIEDERHORN | railway tracks | floodlight at night | higher up, on the road above the tracks (PWD_REF08 to PWD_REF12) | — |
|
||||
| NORDEND | a quarry: a rock face too massive to move with the temperature [expert] | infrared lighting at night; DIFF images | three references, CTR02, CTR03, CTR04; CTR01 is a local reference, read as a target | — |
|
||||
| SNCF01 | a masonry wall along railway tracks | no system metrics, hence no temperature | three references, BG.3.1, R1, R2 | FLORIDA (gaps at the same instants) |
|
||||
| WILDSTRUBEL | a road, in a town [WILDSTRUBEL-102] | solar battery | six references, R1 and R2 at the edge of the road; R3 invisible by day [WILDSTRUBEL-130] | — |
|
||||
Reference in new issue
Block a user