# Configurations: recomputing the stations' data from the differential images 2026-11-19. Approved in conversation. Extends `2026-11-19-pump-diff-design.md` (its layout of `data_diff/` changes: section "data_diff"). ## Intent Recompute a station's data from its differential images under configurations of our own (patterns redrawn, thresholds, references, camera correction), compare them, and let every other repository (aimsight-app, tools, detectors, models, optimizer) read the result exactly as it reads `data/`. `data/` stays as it is: the cloud's version, the reference to compare with. ## Layout ``` aimsight-data/ data/ the cloud, unchanged (pump.sh) data_diff// raw material shared by every configuration (pump_diff.sh), out of git images_diff/ the DIFF images; and, invisible to the tools (data.find_files skips images_diff/): index/_diff_.csv, sensors.json, unit.json images/ metrics/ weather/ targets.csv as in data/ (targets.csv: the cloud's patterns) configs/.yaml one configuration per file, in git runs// the result of a configuration, out of git, the same tree as data/ / displacement/ camera/ targets.csv recomputed, the columns of data/ images/ metrics/ weather/ symbolic links to data_diff// (nothing copied) run.yaml the configuration used, optical's commit, the date (outside the station folders: invisible to the tools) diagnostics/ the emulator's raw rows (edge margin, second spot) ``` A component reads a configuration by pointing its `data` link at `~/aimsight-data/runs/` instead of `~/aimsight-data/data`. Checked on 2026-11-19: `data.py` (aimsight-tools) reads through symbolic links (images, metrics, temperature, weather). ## Configuration A configuration states only what differs from the cloud's patterns (`data_diff//images_diff/sensors.json`) and from the unit's properties (`unit.json`); the configuration `cloud`, with nothing in it, must reproduce `data/` (the reference: on JUNGFRAU 2026-10-12..16 the emulator matched the cloud within 0.05 px, errors identical). ```yaml stations: JUNGFRAU: patterns: # by sensor_name, over sensors.json PM_G: {error_threshold: 0.7} BL_C: {corners: [[1700, 929], [1724, 953]]} REF_CT: {pattern_type: target} camera: {focal_distance: 6.3} # over unit.json and the common camera block camera: # common to the stations of the file algorithm: ransac # ransac | danish_hybrid | trusted_danish mode: "off" # off | rotation | rotation_scale | relative ``` Running a configuration again replaces `runs//`: to compare two settings, two files. No cache between the re-measurement and the correction (431 images in 11 s; about 15 min for a year of one station): added if it gets slow. Rejected: a full copy per configuration (the DIFF images duplicated, 0.6 GB per station-day); a configuration suffix in file names (the tools recognise files by their columns and would mix configurations). ## Work, in order 1. **`pump_diff`** (this repository's `extraction/`): the extras of the DIFF images move under `images_diff/`; per day, before the DIFF images and their index, the metrics (`pump.py`'s `pump_metrics`) and the IRON images (`pump_images`, `--every`, default 60 min); per station, `targets.csv` (`write_targets`, on the DIFF patterns); at the end, the weather (`weather.py`, given the root folder). A day whose metrics failed gets no index: the next run reads it again. `pump.py`'s only change: `pump_metrics` returns whether every day succeeded. 2. **`run_config`**: re-measurement (optical's emulator), camera correction (optical's `remove_camera_movement` and the all-point modes), written in the format of `data/`. Its own design once optical's correction outputs are mapped to the columns of `data/` (`relative_x/y, corr_x/y, reproj_err, inlier, sigma0`, the camera pose).