Files
aimsight-labeling/knowledge/displacement-columns.md
T

40 lines
7.5 KiB
Markdown
Raw Normal View History

# Displacement columns (next use case)
According to the FAQ, the columns of the CSV export are prefixed by the target (e.g. `target_06_rch`):
- `rch` / `rcv`: displacement corrected for camera movement [mm]. This is the main signal.
- `x` / `y`: raw position in the image [px].
- `rx` / `ry`: relative displacement before camera correction. **It is not** `x - x0`.
- `error`: target recognition error, **not** an uncertainty. Default threshold: 0.7.
- `camera_pan|tilt|rotation` [deg], `camera_scale`, `camera_sigma0` (residual of the reference-network fit).
- `inlier_{ch}`, `reproj_err_{ch}`: contribution and residual of each reference.
- Sharpness, FWHM, Dominance, Peak Intensity, Flux, SNR: in the "advanced" export of recent cameras only (see below); definitions: `quality-metrics.md`.
## Exports received (PS des Coudres, D-045)
Two other formats, observed on the data (not documented in the FAQ):
- **"advanced"** (`<Station>_advanced_<start>_<end>.csv`), the source of the analyses: one row per sensor and per measurement, at the native rate (15 min, 10 min depending on the station), `time_utc` in explicit UTC.
- `sensor_name`, `pattern_type` (`global_ref` for a compensation reference; `ref` for a *local reference*, treated as a target, outside the compensation, according to the station manual and without `reproj_err`, seen on NORDEND), `x` / `y` [px], `error_value`, `error_threshold` (specific to each sensor: 0.3 to 0.7);
- `x` / `y` are not the centroid of the spot in the image (NORDEND, 2026-10-16): by day (12:30 UTC, seven days), centroid − (`x`, `y`) is constant per sensor and differs from one sensor to another (PR17 −2.5 / −0.5 px, PR18 −3.1 / +1.6, PR16 −5.8 / −0.1, CTR02 −1.4 / −2.4, CTR04 −1.0 / −1.4). Centroid weighted above the median of a ±20 px frame, on a ±8 px window. At night, the spot is saturated and the offset changes (PR17 −0.7 to −1.3 px in x). Hypothesis: `x`, `y` designate the position of the reference pattern (station manual), not the spot itself.
- `relative_x` / `relative_y` (presumably `rx` / `ry`), `corr_x` / `corr_y` (presumably `rch` / `rcv`, but not in mm, see below);
- `reproj_err` and `inlier`: references only; `sigma0`: one value per measurement, repeated on each row; no `camera_pan|tilt|rotation|scale`.
- `peak`, `flux`, `sharpness`, `fwhm`, `dominance`, `snr`: the image quality of each target, computed by the GPU of recent cameras (`run.md`); "Empty for most devices today" (handoff G, `docs/aimsight/AIMsight_handoff_G_site_health.md`). Filled on LENZSPITZE and NADELHORN, empty elsewhere (pumping of October 2026); `inventory.py` says, on a batch, which columns carry values. For older cameras, `~/optical_emulator` recomputes them from the DIFF images in `quality/` (`run.md`), with `stacking`, `exposure`, `gain`, `rotation` and the image `flag`, to be joined on `time_utc` and `sensor_name`. Definitions and reading: `quality-metrics.md`. Flux, SNR and sharpness relate to the observability of a target in the image, to be separated from the health of the reference network (`AIMsight_agentic_support_project_context.md`, §5.5 and §5.6).
- A rejected measurement (`error_value` > `error_threshold`) keeps its row and its `error_value`; `corr_x` / `corr_y` are empty, not interpolated. "Rejected" is a judgment by AIMsight, not a sorting by the camera: the camera delivers all its measurements, and the platform of a client that integrates the data receives them and can display them (the user, 2026-10-28).
- **LENZSPITZE, relative `x` / `y`** (2026-11-04): `x` / `y` there are the offset from the pattern's reference position (medians ~0 px), `relative_x` / `relative_y` are equal to `corr_x` / `corr_y`, `pattern_type` is empty. Provisional conversion, for this station only: `~/aimsight-data/extraction/lenzspitze_absolute.py` rewrites `data/LENZSPITZE/displacement/` in image px (pattern center from `lentzpitze_patterns.json` + the measured offset +330.5 / +15.8 px between the JSON frame and the IRON images + `x` / `y`) and fills `pattern_type`. To rerun after a re-pump. Limits: 1023 and 8019 have no pattern in the JSON, their `x` / `y` are emptied (8019 classed `global_ref`, by its `reproj_err`); `relative_x` in px is not reconstructed. To fix in due course: the frame of positions on the pumping side, and `visibility.py`, `thermal_displacement.py`, `categories.py`, which test `pattern_type` directly instead of `data.is_reference` (empty, they see no reference).
- **Camera movements** (`camera/<Station>_camera_<YYYYMMDD>.csv`, pumping only, absent from the exports received): the camera pose at each registration, `time_utc` in UTC, `median_|min_|max_` of `pan`, `tilt`, `rotation` [deg], `scale`, `sigma0` at the minute; one row per measurement (min = max), at the instants of the displacements. A measurement instant without a row: the camera was not registered (NORDEND: 101 instants out of 6,071, of which 49 on 29 and 30/07). Orders of magnitude: `pan`, `tilt` of a few hundredths of a degree, `scale` within 10⁻³ of 1; differences of several degrees on NORDEND (`pan` up to 3.1°, `tilt` −2.8 to 4.0°), not explained. Sign: a positive `tilt` means the camera is nose-diving (the expert, 2026-11-03, BREITHORN: the `tilt` rises all October while the camera tilts). Read by `overview.py` and `inventory.py` (D-073).
- **Hourly** (`<Station>_<start>_to_<end>_1hour.csv`): one row per hour, targets only. Per target: `horizontal_displacement`, `vertical_displacement`, their `_min` / `_max`, `_error`, `_count_val`.
- `timestamp` **without time zone**: Swiss time (UTC+2 in October), at the start of the hour. Verified: `_error` = mean of the `error_value` of the hour, to within 0.0006 (median).
- `count_val` counts the measurements of the hour, rejected ones included.
- **Unit of `corr_x` / `corr_y`**: `horizontal_displacement` = factor × hourly mean of `corr_x` + constant offset (likewise `vertical` and `corr_y`, correlation ≥ 0.99). The factor depends on the pattern: `target05` 0.61, `target` 1.6, `target02` 4.3, `target03` 6.85, `target04` 7.9. If the hourly is in mm (FAQ), `corr_x` / `corr_y` are not. According to the user (2026-10-16), they are surely in **px**, the factor then being mm per px. This is compatible with NORDEND: at the jump of PR17 (08/10 02:30), `corr_y` drops by about 0.15, and `relative_y` (in px) by 0.26 px against 0.08 px for the other sensors; over the window, `corr` of PR17 ≈ 0.8 × (`relative` − mean of the `relative` of the global references), but a mean ignores the rotation and scale of the compensation. To confirm. The factor depends on the station: on NORDEND, `vertical_displacement` of PR17 = 55.5 × hourly mean of `corr_y` (r = 0.994, 244 hours), i.e. about 9 mm for its jump of 0.15.
Causes to tell apart for a displacement anomaly:
- real movement of the structure;
- camera movement;
- instability of the compensation (Barcelona case);
- local degradation of a target;
- parallax or unsuitable model.
The authoritative reference is `AIMsight_reference_quality_event_explainer_roadmap.md` (AIMsight documentation, stored in Support-Agent, `docs/aimsight/`). The non-negotiable rules that relate to it:
- build nothing on a binary `inlier`: RANSAC will be replaced by a weighted estimator; use continuous weights, `N_eff`, residuals, influence;
- `sigma0` alone and the number of references are never sufficient criteria;
- do not confuse the image quality of a target with the quality of the reference network, nor a small residual with a well-constrained transformation;
- do not confuse common-mode movement with structural movement, nor coherent references with physically fixed references.