Files
aimsight-labeling/docs/archive/knowledge/displacement-columns.md
T

41 lines
7.5 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Displacement columns (next use case)
According to the FAQ, the columns of the CSV export are prefixed by the target (e.g. `target_06_rch`):
- `rch` / `rcv`: displacement corrected for camera movement [mm]. This is the main signal.
- `x` / `y`: raw position in the image [px].
- `rx` / `ry`: relative displacement before camera correction. **It is not** `x - x0`.
- `error`: target recognition error, **not** an uncertainty. Default threshold: 0.7.
- `camera_pan|tilt|rotation` [deg], `camera_scale`, `camera_sigma0` (residual of the reference-network fit).
- `inlier_{ch}`, `reproj_err_{ch}`: contribution and residual of each reference.
- Sharpness, FWHM, Dominance, Peak Intensity, Flux, SNR: in the "advanced" export of recent cameras only (see below); definitions: `quality-metrics.md`.
## Exports received (PS des Coudres, D-045)
Two other formats, observed on the data (not documented in the FAQ):
- **"advanced"** (`<Station>_advanced_<start>_<end>.csv`), the source of the analyses: one row per sensor and per measurement, at the native rate (15 min, 10 min depending on the station), `time_utc` in explicit UTC.
- `sensor_name`, `pattern_type` (`global_ref` for a compensation reference; `ref` for a *local reference*, treated as a target, outside the compensation, according to the station manual and without `reproj_err`, seen on NORDEND), `x` / `y` [px], `error_value`, `error_threshold` (specific to each sensor: 0.3 to 0.7);
- `x` / `y` are not the centroid of the spot in the image (NORDEND, 2026-10-16): by day (12:30 UTC, seven days), centroid − (`x`, `y`) is constant per sensor and differs from one sensor to another (PR17 −2.5 / −0.5 px, PR18 −3.1 / +1.6, PR16 −5.8 / −0.1, CTR02 −1.4 / −2.4, CTR04 −1.0 / −1.4). Centroid weighted above the median of a ±20 px frame, on a ±8 px window. At night, the spot is saturated and the offset changes (PR17 −0.7 to −1.3 px in x). Hypothesis: `x`, `y` designate the position of the reference pattern (station manual), not the spot itself.
- `relative_x` / `relative_y` (presumably `rx` / `ry`), `corr_x` / `corr_y` (presumably `rch` / `rcv`, but not in mm, see below);
- `reproj_err` and `inlier`: references only; `sigma0`: one value per measurement, repeated on each row; no `camera_pan|tilt|rotation|scale`.
- `peak`, `flux`, `sharpness`, `fwhm`, `dominance`, `snr`: the image quality of each target, computed by the GPU of recent cameras (`run.md`); "Empty for most devices today" (handoff G, `docs/aimsight/AIMsight_handoff_G_site_health.md`). Filled on LENZSPITZE and NADELHORN, empty elsewhere (pumping of October 2026); `inventory.py` says, on a batch, which columns carry values. For older cameras, `~/optical_emulator` recomputes them from the DIFF images in `quality/` (`run.md`), with `stacking`, `exposure`, `gain`, `rotation` and the image `flag`, to be joined on `time_utc` and `sensor_name`. Definitions and reading: `quality-metrics.md`. Flux, SNR and sharpness relate to the observability of a target in the image, to be separated from the health of the reference network (`AIMsight_agentic_support_project_context.md`, §5.5 and §5.6).
- A rejected measurement (`error_value` > `error_threshold`) keeps its row and its `error_value`; `corr_x` / `corr_y` are empty, not interpolated. "Rejected" is a judgment by AIMsight, not a sorting by the camera: the camera delivers all its measurements, and the platform of a client that integrates the data receives them and can display them (the user, 2026-10-28).
- **LENZSPITZE, relative `x` / `y`** (2026-11-04): `x` / `y` there are the offset from the pattern's reference position (medians ~0 px), `relative_x` / `relative_y` are equal to `corr_x` / `corr_y`, `pattern_type` is empty. Provisional conversion, for this station only: `~/aimsight-data/extraction/lenzspitze_absolute.py` rewrites `data/LENZSPITZE/displacement/` in image px (pattern center from `lentzpitze_patterns.json` + the measured offset +330.5 / +15.8 px between the JSON frame and the IRON images + `x` / `y`) and fills `pattern_type`. To rerun after a re-pump. Limits: 1023 and 8019 have no pattern in the JSON, their `x` / `y` are emptied (8019 classed `global_ref`, by its `reproj_err`); `relative_x` in px is not reconstructed. To fix in due course: the frame of positions on the pumping side, and `visibility.py`, `thermal_displacement.py`, `categories.py`, which test `pattern_type` directly instead of `data.is_reference` (empty, they see no reference).
- **Camera movements** (`camera/<Station>_camera_<YYYYMMDD>.csv`, pumping only, absent from the exports received): the camera pose at each registration, `time_utc` in UTC, `median_|min_|max_` of `pan`, `tilt`, `rotation` [deg], `scale`, `sigma0` at the minute; one row per measurement (min = max), at the instants of the displacements. A measurement instant without a row: the camera was not registered (NORDEND: 101 instants out of 6,071, of which 49 on 29 and 30/07). Orders of magnitude: `pan`, `tilt` of a few hundredths of a degree, `scale` within 10⁻³ of 1; differences of several degrees on NORDEND (`pan` up to 3.1°, `tilt` −2.8 to 4.0°), not explained. Sign: a positive `tilt` means the camera is nose-diving (the expert, 2026-11-03, BREITHORN: the `tilt` rises all October while the camera tilts). Read by `overview.py` and `inventory.py` (D-073).
- **Hourly** (`<Station>_<start>_to_<end>_1hour.csv`): one row per hour, targets only. Per target: `horizontal_displacement`, `vertical_displacement`, their `_min` / `_max`, `_error`, `_count_val`.
- `timestamp` **without time zone**: Swiss time (UTC+2 in October), at the start of the hour. Verified: `_error` = mean of the `error_value` of the hour, to within 0.0006 (median).
- `count_val` counts the measurements of the hour, rejected ones included.
- **Unit of `corr_x` / `corr_y`**: `horizontal_displacement` = factor × hourly mean of `corr_x` + constant offset (likewise `vertical` and `corr_y`, correlation ≥ 0.99). The factor depends on the pattern: `target05` 0.61, `target` 1.6, `target02` 4.3, `target03` 6.85, `target04` 7.9. If the hourly is in mm (FAQ), `corr_x` / `corr_y` are not. According to the user (2026-10-16), they are surely in **px**, the factor then being mm per px. This is compatible with NORDEND: at the jump of PR17 (08/10 02:30), `corr_y` drops by about 0.15, and `relative_y` (in px) by 0.26 px against 0.08 px for the other sensors; over the window, `corr` of PR17 ≈ 0.8 × (`relative` − mean of the `relative` of the global references), but a mean ignores the rotation and scale of the compensation. To confirm. The factor depends on the station: on NORDEND, `vertical_displacement` of PR17 = 55.5 × hourly mean of `corr_y` (r = 0.994, 244 hours), i.e. about 9 mm for its jump of 0.15.
Causes to tell apart for a displacement anomaly:
- real movement of the structure;
- camera movement;
- instability of the compensation (Barcelona case);
- local degradation of a target;
- parallax or unsuitable model.
The authoritative reference is `AIMsight_reference_quality_event_explainer_roadmap.md` (AIMsight documentation, stored in Support-Agent, `docs/aimsight/`). The non-negotiable rules that relate to it:
- build nothing on a binary `inlier`: RANSAC will be replaced by a weighted estimator; use continuous weights, `N_eff`, residuals, influence;
- `sigma0` alone and the number of references are never sufficient criteria;
- do not confuse the image quality of a target with the quality of the reference network, nor a small residual with a well-constrained transformation;
- do not confuse common-mode movement with structural movement, nor coherent references with physically fixed references.