Patterns by sensor_name (a property, or a window redrawn from a DIFF image as the device's pattern editor does), re-measured with optical's emulator, camera correction as the cloud's process_shifts (one measurement at a time, or the series), written in the files of data/ (displacement/, camera/, targets.csv; images/, metrics/, weather/ linked to data_diff/), with run.yaml and diagnostics/ beside the stations. configs/cloud.yaml, empty, reproduces data/JUNGFRAU 2026-10-12..16: corr_x/y within 0.03 px, the same rejections, the camera pose within 5e-4 deg; 16 s. configs/jungfrau_example.yaml: PM_G back to 0.7, BL_C's window redrawn. .gitignore: data_diff/, runs/, remeasure/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
4.1 KiB
run_config: a configuration recomputed into runs//
2026-11-19. Approved in conversation. Step 2 of 2026-11-19-configurations-design.md.
What the cloud does (monitoring-dashboard app/processing/process_shifts.py, branch explo)
Per DIFF measurement, alone: the non-archived DIFF patterns (load_pattern_from_params) are found in the image; raw
x, y, error (and the metrics) are stored; targets above their error_threshold get x/y blanked; a rejected
global_ref gets back the x/y of the latest stored frame whose error is not above its own (in practice its own
raw position, inserted just before), else its pattern position; remove_camera_movement(r, focal_distance (6.0), pixel_size (0.0024), algorithm (ransac, ransac_threshold 0.99)), which rejects those references from the fit again
(error columns given) but still corrects their rows; corr -= (x_ref, y_ref); cam_info (pan, tilt, scale,
rotation, sigma0) and ref_stats (reproj_err, inlier) stored. The Advanced export adds relative = x - pattern x
and copies sigma0 onto every row; the camera export is the per-minute median / min / max of cam_info.
Design
processing/run_config.py, through processing/run_config.sh (uv: numpy, pandas, pillow, pyyaml, scipy,
scikit-image, opencv-python-headless; optical's code from $OPTICAL, default ~/optical):
processing/run_config.sh <config> [SERIAL...] [--from D] [--to E] [--jobs N]
<config>: configs/<name>.yaml or a path; stations: those named, else those of the configuration; days: those
indexed in data_diff/<SERIAL>/images_diff/index/ (within --from/--to). For each station, runs/<name>/<SERIAL>/
is replaced:
- Patterns:
images_diff/sensors.json(DIFF type, a pattern, not archived; latest value of each property), then the configuration bysensor_name(an unknown name or key is an error): any property replaced (error_threshold,pattern_type,distance_convertion_factor,x,y...);redraw: {image, corners}cuts the template from that DIFF image as the device's pattern editor does (estimate_camera_posefrom the references,saved_anchor_and_reference:x_refkept,ImagePatternMono.from_image). Written todiagnostics/<SERIAL>_sensor_properties.json. - Re-measurement: optical's
reprocess_images.main(emulator) on the days' images,--timesfrom the index; rows kept indiagnostics/<SERIAL>_remeasured.csv. - Correction: as the cloud, one measurement at a time (
per_epoch: true, default): targets above their threshold blanked, references kept raw (rejected in the fit byremove_camera_movement),corr -= x_ref;per_epoch: false: the whole series in one call (a rejected reference interpolated in time, algorithms with memory). Camera block:algorithm(ransac),ransac_threshold(0.99, ransac only),mode(off|rotation|rotation_scale|relative, optical'srefine_allpoints),focal_distance,pixel_size(unit.json, else 6.0 / 0.0024),per_epoch; station block over common block over unit.json over the cloud's defaults. - Output, the files of
data/:displacement/<S>_advanced_<YYYYMMDD>.csv(columns ofdata/:x, y, error_value, peak..snrfrom the emulator,relative = x - pattern x,corrfrom the correction,reproj_err, inlierof the references,sigma0of the epoch on every row,corr_*_mm = corr * distance_convertion_factor),camera/<S>_camera_<YYYYMMDD>.csv(median / min / max per minute),targets.csv;images/,metrics/,weather/as relative symbolic links todata_diff/<SERIAL>/;run.yamlatruns/<name>/(configuration as resolved per station, optical's commit and state, days, date).
Acceptance: configs/cloud.yaml (empty) on JUNGFRAU 2026-10-12..16 against data/JUNGFRAU: positions and errors
within 0.05 px (done by the emulator), corr_x/y and the camera pose close to the cloud, the differences counted.
Check: run_config.py --check in the uv environment: the configuration rules (unknown keys, order of the camera
blocks, overrides by name) and a synthetic correction (three references, two targets, one rejected) through
optical's remove_camera_movement.