- The user's layout: the station's root holds only what is the same for every run (images_diff/, images/, metrics/, weather/); each run's folder holds what depends on it: config.yaml, displacement/, camera/, targets.csv, patterns.json. - processing/new_run.sh <SERIAL> <run>: config.yaml of a new, empty run, the cloud's processing written in full (the patterns of sensors.json, the camera, optical's current commit); never over an existing one. - processing/run_config.sh <SERIAL> <run>: reads the run's config.yaml (patterns, camera, emulator.version, checked against $OPTICAL's commit and local changes) and writes its displacement/, camera/, targets.csv (from its patterns) and patterns.json (the patterns as given to the emulator). No configs/, no runs/<name>/ outside data_diff/. - pump_diff no longer writes targets.csv: it depends on the run. JUNGFRAU run01 (2026-09-01, 2026-10-12..16, 525 images, 18 s): 7875 rows as data/, the same rejections, corr within 0.034 px; a run created by new_run.sh and run as is gives run01 exactly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3.3 KiB
3.3 KiB
aimsight-data
The data of the AIMsight stations, in a single copy on the workstation, and their pumping.
data/<SERIAL>/: filled by the pumping, out of git (metrics/,displacement/,camera/,images/,weather/for a station with a Swiss position…). A lost folder is pumped again.data_diff/<SERIAL>/, out of git: pumped byextraction/pump_diff.sh(never touchingdata/): the differential images full size underimages_diff/(with their index, the patternssensors.jsonand the unitunit.json),images/(IRON),metrics/,weather/, as indata/: only what is the same for every run. Its runs inruns/<run>/, with everything that depends on the run:config.yaml(the run's configuration),displacement/,camera/andtargets.csv(its result, the files ofdata/),patterns.json. Design:docs/superpowers/specs/2026-11-19-configurations-design.md.processing/run_config.sh <SERIAL> <run> [--from D] [--to E]: runsdata_diff/<SERIAL>/runs/<run>/config.yaml(optical's emulator and camera correction, from$OPTICAL, default~/optical; 18 s for 6 days of JUNGFRAU) and writes the run'sdisplacement/,camera/,targets.csv(from its patterns) andpatterns.json(the patterns exactly as given to the emulator, templates included);--checkwithout data.config.yamlstates what differs from the cloud's processing (an empty file: the cloud's):patternsbysensor_name(a property, or a window redrawn from a DIFF image),camera(algorithm, ransac_threshold, mode, focal_distance, pixel_size, per_epoch) andemulator.version(optical's commit: the run stops if$OPTICALis at another one or has local changes); its keys are checked.processing/new_run.sh <SERIAL> <run>creates a run, empty: itsconfig.yamlonly, the cloud's processing written in full (every pattern ofsensors.json, the camera, optical's current commit), to edit before running it; it never overwrites an existingconfig.yaml. The standard configuration reproducesdata/(JUNGFRAU, 6 days:corr_x/ywithin 0.034 px, the same rejections). A window redrawn on a target restarts its displacement at zero at the image it is cut from (x_refis kept, the template is new). Design:docs/superpowers/specs/2026-11-19-run-config-design.md.extraction/: the pumping, read-only on the platform; how to use it:extraction/README.md. VS Code tasks "pump" and "pump a range".extraction/pumping.py: the tracked stations, launching the pumping and its state, for the callers (aimsight-app, page Stations).weather/: the weather measured by MeteoSwiss at a place over a past range (weather/meteoswiss.py), written intodata/<SERIAL>/weather/at each pumping (extraction/weather.py); its doc inweather/README.md..env: the credentials of the database and of S3, at the root, never versioned.
The other repositories (aimsight-app, Optimizer) read the data through a link data -> ~/aimsight-data/data, ignored by git. The methods that bring data in live here; the app only calls them. Pumping is done from here only: the lock out/pump.lock holds for this folder alone.
The labels are in the neighbouring repository ~/aimsight-labeling.
Extracted from aimsight-app on 2026-11-13 (commit 462ba9e); the earlier history of the pumping is in Support-Agent.