What the files of a station folder are, and what each column means. How to read them with the tools: `GUIDE.md`. What they mean for a label (causes, reading rules, false friends): `labeling/reading-rules.md`. Maintained by hand; `GUIDE.md` is the tool-builder's.
## The displacements: the "advanced" export
`displacement/<Station>_advanced_<start>_<end>.csv`, the source of the analyses: one row per sensor and per measurement, at the native rate of the station (10 or 15 min), `time_utc` in explicit UTC.
| Column | Meaning |
|---|---|
| `sensor_name` | the sensor (target or reference) |
| `pattern_type` | `global_ref`: a reference of the compensation; `ref`: a *local reference*, treated as a target, outside the compensation, without `reproj_err`; empty: a target |
| `x`, `y` | raw position in the image [px]. Not the centroid of the spot: the offset centroid − (`x`, `y`) is constant per sensor by day and differs between sensors; it changes at night, when the spot is saturated. Probably the position of the reference pattern |
| `relative_x`, `relative_y` | relative displacement before the camera compensation [px]; not `x − x0` |
| `corr_x`, `corr_y` | the position of the target once the estimated camera movement is removed: **the curves that are labelled**. In px, not mm (below) |
| `corr_x_mm`, `corr_y_mm` | `corr_x`, `corr_y` in mm: × the sensor's `mm_per_px` (`targets.csv`, below), computed by `data.load`; empty without a sure factor. The same curves in another unit: the noise threshold of a label is read in noises, whatever the unit; mm say how far a target moves, for the reader |
| `error_value` | recognition error of the pattern, not an uncertainty |
| `error_threshold` | specific to each sensor (0.3 to 0.7). Above it, the measurement is rejected: the row and its `error_value` stay, `corr_x` and `corr_y` are empty, not interpolated |
| `reproj_err`, `inlier` | references only: residual and contribution of each reference in the compensation |
| `sigma0` | residual of the fit on the references: one value per measurement, repeated on every row |
| `peak`, `flux`, `sharpness`, `fwhm`, `dominance`, `snr` | image quality of each sensor, recent cameras only (below) |
- "Rejected" is a judgment of the AIMsight platform, not of the camera: the camera delivers all its measurements, and a client's platform that integrates the data receives them all.
- **Unit of `corr_x`, `corr_y`**: px. The hourly export's `horizontal_displacement` (mm) is a factor × the hourly mean of `corr_x` + a constant (likewise `vertical` and `corr_y`, correlation ≥ 0.99); the factor (mm per px) depends on the pattern and on the station.
- **LENZSPITZE**: its export gives `x`, `y` relative to the pattern; the extraction rewrites them in image px and fills `pattern_type`. Sensors 1023 and 8019 have no pattern: their `x`, `y` are empty. The tools' crosses may sit slightly off the spots there.
-`inventory.py` says, on a batch, which columns carry values.
`targets.csv`, from the pumping: one line per sensor of the platform, `sensor_name`, `pattern_type`, `distance_mm` (from the camera) and `mm_per_px` (`distance_convertion_factor`, the factor of the dashboard's exports in mm), their current values. Read by `data.load_targets`, shown by `inventory.py` (sensor table).
- The references are not in it yet (pumped from the next pumping on): no distance, no mm for them. They are assumed fixed.
-`distance_mm` 1000: not filled in on the platform; the line is ignored.
- A name may have several lines (a target counted twice, an older configuration of the unit, another lens): when they disagree, `data.load_targets` keeps no value (`inventory.py`: `?(n)`) and that target has no mm; the distance stays when the lines agree on it.
- The distance says which targets are near and far. A far target moves few px for many mm: targets are compared with each other in mm, or in px with their distance. A camera that turns shifts every target by about the same px (the mm grow with the distance); a movement of the structure of the same mm gives px that shrink with the distance; a parallax shows between targets at different distances.
`camera/<Station>_camera_<YYYYMMDD>.csv`, from the pumping only: the pose at each registration, `time_utc` in UTC, `median_|min_|max_` of `pan`, `tilt`, `rotation` [deg], `scale`, `sigma0`, one row per measurement (min = max). A measurement instant without a row: the camera was not registered.
- Orders of magnitude: `pan`, `tilt` of a few hundredths of a degree, `scale` within 10⁻³ of 1; some stations show several degrees, not explained.
- **A positive `tilt` means the camera is nosing down** (the expert).
`weather/<Station>_weather_<YYYYMMDD>.csv`, from the pumping, for a station with a Swiss position only: the hourly measurements of the nearest MeteoSwiss station, `time_utc` in UTC (start of the hour); `position`, `place`, `station`, `station_name`, `distance_km`, `height_m` say where; then `precipitation_mm`, `temperature_c`, `humidity_pct`, `wind_speed_kmh`, `wind_gust_kmh`, `wind_direction_deg`, `pressure_hpa`, `sunshine_min`, `global_radiation_wm2`, `snow_depth_cm` (empty: not measured there). The weather a few km away, not at the camera: what the images show prevails. Read by `data.load_weather`.
`<Station>_<start>_to_<end>_1hour.csv`: one row per hour, targets only; per target `horizontal_displacement`, `vertical_displacement`, their `_min` / `_max`, `_error` (mean `error_value` of the hour), `_count_val` (measurements of the hour, rejected ones included).
- **`timestamp` has no time zone: Swiss local time** (UTC+2 in summer), at the start of the hour.
## Image quality of a target
Six columns of the advanced export, computed by the GPU of recent cameras on the sensor's window in the DIFF image, in units of that image. Filled only where `inventory.py` says so ("Export image quality"); for older cameras, `~/optical_emulator` can recompute them from the DIFF images. "Disc": the neighbourhood of the peak, radius `min(H, W)/6` (at least 3 px); "background": the rest of the window.
| Metric | Definition | What it measures | Reading |
|---|---|---|---|
| `peak` | `max(I)` | the brightness of the return | high, saturation; low, underexposure or missed flash. All targets dropping: the lighting; a single one: that target (fouling, misalignment) |
| `flux` | flux of the disc minus the background | the light of the target, background removed | reflectivity of the target (fouling, ageing, flash), not ambient light; drops well before `error_value` exceeds its threshold |
| `sharpness` | `Σ(I²) / (ΣI)²` | concentration of the energy (higher = sharper) | drops: drifting focus, turbulence, degrading target. Independent of the gain, not of the background: follow it over time on one target, do not compare cameras |
| `fwhm` | width at half height around the peak, background removed [px] | width of the spot | rises: defocus, turbulence, motion blur. Truncated to the disc: under strong defocus it flattens instead of rising |
| `dominance` | `(second peak − background) / (peak − background)`, 0 (clean) toward 1 (ambiguous) | does a single spot dominate the window? | high: a comparable second reflection, a stray bright point, or the target has gone and something else is the brightest: the correlation can latch onto the wrong pattern, which `error_value` does not see. One hot pixel outside the disc is enough |
| `snr` | `(peak − background) / bg_sigma` | the signal above the background noise | low: marginal return, even when `peak` looks normal. Independent of exposure and gain. The emulator marks `empty` below 8 |
`peak` and `flux` give the brightness, `sharpness` and `fwhm` the sharpness, `dominance` and `snr` the validity: together they say **why** a measurement degrades, where `error_value` only says that it does. They describe the image of **one target**, not the health of the reference network.
## Camera images: IRON and DIFF
- **IRON** (`images/`): the ordinary image of the scene, JPG. The cause of an event is seen on it (condensation, rain, train, snow).
- **DIFF** (`images_diff/`): several captures combined to raise the contrast of the reflectors; **the image the displacement algorithms measure**, by default. Not a difference between two instants. 16-bit PNG.
- Acquisition modes (station manual): "Single with infrared light", one image per measurement; "Differential", several combined (`Stacking` 20 to 40 when light is difficult); option "Work on IRON images", to follow a texture without reflector.
- Not every station has DIFF images; the pumping keeps about one image per hour. The tools look only at IRON images; `inventory.py` counts the DIFF ones, which your own scripts may read. What is very close to the lens (a web) may stand out better on DIFF.