Runs in data_diff/<SERIAL>/runs/<run>/: new_run.sh creates one, run_config.sh <SERIAL> <run> computes it

- The user's layout: the station's root holds only what is the same for every run (images_diff/, images/, metrics/,
  weather/); each run's folder holds what depends on it: config.yaml, displacement/, camera/, targets.csv,
  patterns.json.
- processing/new_run.sh <SERIAL> <run>: config.yaml of a new, empty run, the cloud's processing written in full (the
  patterns of sensors.json, the camera, optical's current commit); never over an existing one.
- processing/run_config.sh <SERIAL> <run>: reads the run's config.yaml (patterns, camera, emulator.version, checked
  against $OPTICAL's commit and local changes) and writes its displacement/, camera/, targets.csv (from its patterns)
  and patterns.json (the patterns as given to the emulator). No configs/, no runs/<name>/ outside data_diff/.
- pump_diff no longer writes targets.csv: it depends on the run.

JUNGFRAU run01 (2026-09-01, 2026-10-12..16, 525 images, 18 s): 7875 rows as data/, the same rejections, corr within
0.034 px; a run created by new_run.sh and run as is gives run01 exactly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
maximeandClaude Opus 5.5 committed 2026-11-19 10:52:46 +01:00
1 parent d3bf4b6860
commit ca366c150e
8 files changed
+185 -123

No files matched your search

@@ -3,6 +3,24 @@
2026-11-19. Approved in conversation. Extends `2026-11-19-pump-diff-design.md` (its layout of `data_diff/` changes:
section "data_diff").
**Replaced the same day by the user's own layout** (the layout below, `configs/` and `runs/<name>/`, is not used):
```
data_diff/<SERIAL>/
images_diff/ (index/, sensors.json, unit.json) images/ metrics/ weather/ pumped, the same for every run
runs/<run>/ one run: whatever depends on it
config.yaml the run's configuration: patterns (by sensor_name), camera and emulator.version (optical's
commit, checked), over the cloud's processing
displacement/ camera/ targets.csv its result, the files of data/ (targets.csv from its patterns)
patterns.json the patterns exactly as given to the emulator, templates included
```
`processing/run_config.sh <SERIAL> <run>` reads the run's `config.yaml` and writes its `displacement/`, `camera/`,
`targets.csv` and `patterns.json`, nothing else. Rule: what is not the same for every possible run is in the run's
folder. `processing/new_run.sh <SERIAL> <run>` creates a run, empty: its `config.yaml`, the cloud's processing written
in full (patterns of `sensors.json`, camera, optical's current commit); never over an existing one. Note: the reading tools (aimsight-tools `data.find_files`) read a station folder whole, subfolders
included: pointed at `data_diff/<SERIAL>/`, they see every run's curves together.
## Intent
Recompute a station's data from its differential images under configurations of our own (patterns redrawn,