Files
aimsight-labeling/auto/analyses/NORDEND_2026-09-04T0615_2026-09-17T0545/analysis.md
T
maximeandClaude Opus 5.5 ddb17c2189 auto: labels requalified under the 5-noise rules (only what stands out from the noise): 32 analyses refined, the 6 paused periods verified, the long labels requalified with the fixes the period analysts found; 1153 -> 961 labels
- 32 analyses: each label kept, rewritten or removed by its analyst (section Requalification of the report; mm magnitudes where the targets have them)
- SNCF01 x3, WILDSTRUBEL 2026-09-01 -> 2026-10-06 x3: verified from their existing labels under the new rules
- long labels (no analysis): auto/analyses/long-labels/<STATION>/; NORDEND-169 and -195 run to 07/11, so NORDEND-283 and -284 (their continuation) are removed
- reading-rules.md, stations.md: the citations of removed or rewritten labels point to their successor, or to [expert]

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 23:58:00 +01:00

13 KiB
Raw Blame History

Labels: NORDEND, 2026-09-04T06:15Z → 2026-09-17T05:45Z (UTC)

Request: requalify the 39 labels of this analysis (work/requalify_input.jsonl, the same as the previous labels.jsonl) under the new rules: only what stands out from the noise of the corrected curves is labelled (labeling/labeling-rules.md, section 1). No new reading, no new event, no bound moved. DETECTORS: no.

Result: 35 labels in labels.jsonl: 29 kept exactly as they were (fields and review untouched), 6 rewritten (NORDEND-95, 109, 118, 120, 121, 125: right in substance, but their text was about a missing curve; it now describes the effect on the other curves, so the review field is dropped), 4 removed (NORDEND-104, 115, 117, 124: missing points with no effect beyond the noise).

Weather: none for NORDEND (no weather/ folder). Image quality: the six quality columns are empty (old camera). The previous reading (work/requal/analysis.before-requalify.md) still stands for everything not touched here.

1. My labels

The previous analysis justified each label (work/requal/analysis.before-requalify.md, verdict table and section 1). Here I only give what the requalification measured. Deviations are in each sensor's own noises, from tools/scripts/ranges.py --k 5 (see "Requalification"). "Script" is my cross-check (scripts/requalify.py): it uses the noise over the whole range, and for thermal labels a 5-measurement running median against the night before.

Kept unchanged (29)

  • Phenomenon of the catalogue: as before. Camera thermal cycle (11 labels), loss of a reference (89, 90), fog (107), cobweb (91), the "No data" gaps (92, 96, 98, 100), a jump that stays (94), slow dip of one target (257), night jumps (112, 256, 260), agitation (262), a target departing alone (263), sunny afternoon with a group effect (258).
  • Precedents: none new; the precedents of the previous analysis.
  • Family: unchanged.
  • Measured (max target deviation, ranges.py / script):
    • Thermal, night level from the 6 h before, ranges of ≥3 measurements by day: 88 9.6 (PR17 y, 8 measurements; script 5.4), 93 17.5 (PR18 x; 9.2), 97 27.1 (PR18 x; 20.3), 99 39.8 (PR18 x; 16.5), 103 17.0 (PR18 x; 6.7), 105 21.3 (PR18 x; 11.8), 108 24.0 (PR18 x; 17.5), 110 27.5 (PR06 y, 94 measurements; PR18 x 12.8; script 9.3), 259 21.1 (PR18 x; 11.7), 261 8.3 (PR12 y, 3 measurements; PR08 x 7.4 over 27; script 4.6), 116 19.7 (PR08 y, 11 measurements; PR15 x 15.3; script 6.3), 119 27.1 (PR18 x; 18.4), 123 10.2 (PR18 x, 6 measurements; 6.7). In every case the corrected curves keep the cycle: the camera movement is not fully absorbed.
    • Movements: 89 39.0, 90 47.9, 91 58.4, 94 6.3 at rolling ±4, which understates a step (13.7 script), 107 130.2, 112 16.1 (targets; references up to 18), 256 5.6 (PR08 y; PR16 x −9.2 with a 6 h window), 257 8.9 (PR07 x with --outside, a slow dip that a ±4 rolling level hides: 4.7), 258 7.9, 260 11.0, 262 34.0, 263 7.2.
    • No data (92, 96, 98, 100): every sensor is missing at those steps (inventory, previous analysis). They are kept: the station delivers no measurement at all.
  • Images: none new.
  • Confidence: high, except 261 and 88: medium. Both are weak days where the two estimates straddle the threshold. ranges.py, the reference tool, puts them above it.

2026-09-05T14:15Z → 2026-09-05T16:30Z (NORDEND-95, rewritten)

  • Phenomenon of the catalogue: the sun in the field or behind a spot (CTR01).
  • Precedents: NORDEND-258 in the range (same hour; the group effect is written in its text).
  • Family: Visibility, unchanged.
  • Measured: CTR01 rejected 14:30–15:15 and 16:00–16:15. PR01 corr_y +7.6 noises at 15:00 (+3.0 mm; 9.5 → 12.8 mm, back to 11.3), PR14 corr_x −5.4 at 15:45.
  • Images: the reviewed 15:00 reading is kept (rock in full sun around CTR01).
  • Confidence: medium. The effect is one measurement of one target, but it recurs on the other sunny afternoons (PR01 y on 10, 14 and 15/09).

2026-09-10T14:30Z → 2026-09-10T16:30Z (NORDEND-109, rewritten)

  • Phenomenon / precedents / family: as NORDEND-95; Visibility.
  • Measured: at 14:45 (first CTR01 rejection) PR14 x +6.7, PR18 y −6.0, PR18 x +5.6, PR07 y +5.3. At 15:00 PR07 x +5.8, PR17 x +5.0. PR01 corr_y 12.0, 11.9 mm at 15:30–15:45 against about 9.3 mm (script +8.6 at 14:45).
  • Images: none new. Confidence: medium.

2026-09-14T14:15Z → 2026-09-14T16:30Z (NORDEND-118, rewritten)

  • Phenomenon / precedents / family: as NORDEND-95; Visibility.
  • Measured: CTR01 corr_x −7.8 at 14:30, just before its first rejection (no mm factor for CTR01). PR01 corr_y +6.9 at 15:00–15:15 (2 measurements, +2.7 mm). PR09 corr_y −5.2 at 16:00–16:15, the last rejection (8.0 → 5.5 mm).
  • Images: none new. Confidence: medium.

2026-09-15T14:45Z → 2026-09-15T16:15Z (NORDEND-120, rewritten)

  • Phenomenon / precedents / family: as NORDEND-95; Visibility.
  • Measured: PR18 corr_x −6.6 at 16:00 (−1.7 mm). PR01 corr_y +5.9 at 15:15 (8.7 → 10.7, 11.3, 11.2 mm at 15:00–15:30, back at 15:45). PR14 x +5.0 at 15:30.
  • Images: none new. Confidence: medium.

2026-09-16T01:30Z → 2026-09-16T03:15Z (NORDEND-121, rewritten)

  • Phenomenon of the catalogue: a cobweb in front of a target (deduced, as in the reviewed label).
  • Precedents: none new.
  • Family: Obstruction, unchanged.
  • Measured: PR08 corr_x −7.6 at 01:45, the first PR07 rejection (0.057 → 0.038 px, back to 0.057). PR15 corr_x 0.133 → 0.121, 0.112 at 01:45–02:00, back at 02:30. With the whole-range noise (larger, from the day), the same PR08 point is 4.1 noises: the night noise is what ranges.py reads around the label.
  • Images: the reviewed reading is kept. Confidence: medium (the threshold depends on whose noise; the tool's window says above).

2026-09-16T14:15Z → 2026-09-16T16:30Z (NORDEND-125, rewritten)

  • Phenomenon / precedents / family: as NORDEND-95; Visibility.
  • Measured: PR13 corr_y +5.5 at 15:30 (0.069 → 0.093 px) and −5.5 at 16:15 (0.042 px). These are exactly CTR01's rejection instants. At 16:15 PR02 corr_y also dips (0.162 → 0.132 px, script −5.6, −1.7 mm), and CTR02, CTR03, CTR04 shift 5–6 noises together.
  • Images: none new. Confidence: low. The deviations sit just above the threshold. Kept because ranges.py puts them above 5 noises, and they fall on the episode's instants.

2. What I left without a label

Removed by this requalification:

  • NORDEND-104 (2026-09-08T17:00Z → 17:30Z, Visibility): PR07 rejected once. ranges.py finds no other target beyond 5 noises; the script's largest is PR14 x 4.8 at 17:15. Missing point only.
  • NORDEND-115 (2026-09-13T15:30Z → 16:00Z, Visibility): CTR01 rejected once. PR18 x +5.4 at 15:45 (script 4.4) and PR07 x −7.7 at 15:30 belong to the agitation and PR07 return that NORDEND-262 (11:30 → 16:00) already covers. CTR01's own episode is only a missing point.
  • NORDEND-117 (2026-09-14T08:15Z → 08:45Z, Visibility): PR07 rejected once. ranges.py finds nothing beyond 5 noises; the script's largest is 4.3 (PR16 y). Missing point only.
  • NORDEND-124 (2026-09-16T09:45Z → 11:30Z, Visibility): CTR01 rejected at times in the morning. ranges.py finds nothing beyond 5 noises; the script's largest is 4.2 (PR18 x). Missing points only.

Still unlabelled, from the previous reading (unchanged): PR10 at night on 16–17/09 (about 2 noises); small group deviations at the pose jerks (inside the thermal labels); single-sensor deviations of 6–7 noises for one measurement; the missing image of 15/09 02:15; error_value rises with no effect on the corr. See work/requal/analysis.before-requalify.md, section 2.

3. The documentation

  • labeling-rules.md section 1 decided everything: the 5-noise threshold, "a missing point is not labeled for itself", "thermal movement is labeled only when the corrected curves keep it", and "No data only when the station delivers nothing".
  • One thing is not settled by the rules: which noise. ranges.py takes the noise of the window it is given. Around a night label that is the quiet night noise, and a point can be 7.6 noises there and 4.1 against the whole-range noise (NORDEND-121). For thermal labels with --outside the noise is the night's, which roughly doubles the deviations compared to the whole-range noise (88, 261). I followed the tool as the instructions name it, and wrote both values. A sentence in the rules saying which window the noise comes from would make the decision reproducible.
  • A slow dip lasting several measurements (257) or a step (94) is understated by --rolling 4. --outside the label measures it correctly. The rules say --rolling "under a drift or a thermal cycle", which is right, but the rolling window must be longer than the event.
  • The CTR01 sunny-afternoon labels turned out to carry a repeated effect: PR01 corr_y rises 2–3 mm on 05, 10, 14 and 15/09 during CTR01's rejections. The old texts did not say so; the rewritten ones do.

Difficulties

  • The per-label ranges.py loop took about 3 minutes, so it ran in the background.

Requalification

Commands (from the repository root):

  • python3 labeling/auto/analyses/NORDEND_2026-09-04T0615_2026-09-17T0545/scripts/ranges_per_label.py <…>/work/requalify_input.jsonl → work/requal/ranges_per_label.txt. For each label it runs python3 tools/scripts/ranges.py data/NORDEND --columns corr_x,corr_y --k 5 --sensors <all but PR04> with:

    • movements: --start <start−6h> --end <end+6h> --rolling 4;
    • slow ones (91, 257, 262): --outside <start>/<end> instead;
    • thermal: --start <start−6h> --end <end> --outside <start>/<end> --min-measurements 3, ranges by day (06–20 UTC), outside 91, 107, 257, 262.

    It keeps the ranges whose extreme lies strictly inside the label, targets only. PR04 is a Target issue target, rejected almost always (NORDEND-24).

  • python3 <…>/scripts/requalify.py <…>/work/requalify_input.jsonl 4 → work/requal/requalify_N4.txt: the cross-check with whole-range noise. scripts/requalify_mm.py gives the same in mm → work/requal/requalify_mm.txt.

  • python3 tools/scripts/summary.py data/NORDEND --column corr_x|corr_y|corr_y_mm --step 15min|1h --level window --mark on 13/09, 16/09 01:00–04:00 and 14:00–17:00, and on the four CTR01 afternoons.

  • python3 <…>/scripts/write_requalified.py writes labels.jsonl. python3 labeling/scripts/validate.py <…>/labels.jsonl gives 35 labels, 0 errors.

Values (max target deviation in noises, ranges.py; script in brackets):

Id Family ranges.py script Verdict
88 Thermal 9.6 5.4 kept
89 Visibility 39.0 40.8 kept
90 Visibility 47.9 51.7 kept
91 Obstruction 58.4 24.2 kept
92, 96, 98, 100 No data all sensors missing – kept
93 Thermal 17.5 9.2 kept
94 Displacement 6.3 13.7 kept
95 Visibility 7.6 8.4 rewritten
97 Thermal 27.1 20.3 kept
99 Thermal 39.8 16.5 kept
256 Visibility 5.6 (PR16 −9.2, 6 h window) 5.4 kept
257 Displacement 8.9 (--outside) 4.7 kept
103 Thermal 17.0 6.7 kept
104 Visibility 0 4.8 removed
105 Thermal 21.3 11.8 kept
258 Visibility 7.9 7.7 kept
107 Visibility 130.2 123.4 kept
108 Thermal 24.0 17.5 kept
109 Visibility 6.7 8.6 rewritten
110 Thermal 27.5 9.3 kept
259 Thermal 21.1 11.7 kept
260 Visibility 11.0 8.5 kept
112 Displacement 16.1 8.0 kept
261 Thermal 8.3 4.6 kept
262 Visibility 34.0 17.6 kept
115 Visibility 5.4 (inside 262) 4.4 removed
116 Thermal 19.7 6.3 kept
117 Visibility 0 4.3 removed
118 Visibility 7.8 8.6 rewritten
119 Thermal 27.1 18.4 kept
120 Visibility 6.6 6.5 rewritten
121 Obstruction 7.6 4.1 rewritten
263 Visibility 7.2 9.3 kept
123 Thermal 10.2 6.7 kept
124 Visibility 0 4.2 removed
125 Visibility 5.5 5.6 rewritten

Sources read

  • instructions/analyst.md, the request, work/requalify_input.jsonl, the previous analysis.md and report.md.
  • labeling/labeling-rules.md (section 1 and the bounds of section 2), tools/GUIDE.md (ranges, summary), tools/scripts/data.py (functions, for my scripts), data/NORDEND/targets.csv (distances, mm per px), work/inventory.txt.
  • Tools: ranges.py (per label, see above), summary.py (15 min and hourly, px and mm).
  • My scripts: scripts/requalify.py, scripts/requalify_mm.py, scripts/ranges_per_label.py, scripts/write_requalified.py.