Energy code moves to src/energy/ and its docs to docs/energy/; shared code (CSV loading, venv switch, browser, zoom script) goes to src/common/, and the AIMsight originals to docs/aimsight/. Analyses are no longer numbered: two Energy scripts remain, daily.py (delivered state and trend) and all_analyses.py (comparison of the four). Calculations and pages unchanged (D-026). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
51 KiB
AIMsight Agentic Support — Project Context
0. Purpose of this document
This document is the working context for development of an AIMsight support and diagnostic agent.
It extends, but does not replace, the existing roadmap:
AIMsight_reference_quality_event_explainer_roadmap.md
That roadmap remains the main reference for:
- planning / installation assessment for geometry, precision and stability;
- deterministic event explanation;
- reference observation, reference-network quality, automatic exclusion/downweighting and integrity alarms;
- estimator-independent diagnostics compatible with the current RANSAC similarity estimator and a future Danish-like robust weighted similarity adjustment.
The support-agent project adds a broader operational layer:
- system health;
- acquisition health;
- communications / export health;
- backend / interface health;
- individual point / target health;
- reference observation and reference-network health;
- camera / transform / model-adequacy diagnostics;
- data provenance and reprocessing state;
- synthesis of these layers into a support-oriented diagnosis.
At the initial POC stage, data access is deliberately simple and manual. The user will provide or query evidence such as:
- CSV exports;
- screenshots / captures of AIMsight interfaces;
- screenshots or saved images from the camera;
- configuration extracts;
- manually copied status information;
- incident notes and timestamps;
- later, selected logs when available.
The POC should therefore optimize the diagnostic reasoning model, evidence requirements, deterministic feature extraction and support workflow before building live station/backend integrations.
1. Core objective
The support agent should help answer support questions such as:
- Is this AIMsight station healthy?
- Why is there no data after a given time?
- Was the image acquired but not uploaded?
- Is the problem local to one monitored point?
- Is a point poorly visible, saturated, occluded or badly tracked?
- Is a corrected displacement trustworthy?
- Did the camera or station move?
- Did the reference compensation change?
- Is the reference network weak or unstable?
- Is the observed event probably physical motion, a compensation event, a point-quality problem, a model mismatch or a data-pipeline problem?
- Why does the web interface disagree with a CSV export?
- Why is a plot empty or returning an error although the station may still be operating?
- Are timestamps consistent with the station, export and UI clocks?
- Was data lost, delayed, buffered, re-uploaded or merely not displayed?
- What changed relative to a known-good period?
- What evidence is still required before a conclusion can be reached?
- What is the next most informative support check?
The agent should reduce the number of ad-hoc manual investigations while making its reasoning traceable and evidence-based.
2. Relationship to the reference-quality / event-explainer roadmap
The existing roadmap establishes a shared geometric and diagnostic layer around:
- target image quality;
- per-reference residuals;
- robust contribution weights;
- leverage and influence;
- effective reference count;
- weighted reference geometry;
- transform covariance;
- target-specific propagated uncertainty;
- transform / camera state;
- temporal weight-state changes;
- model mismatch / parallax diagnostics;
- deterministic event classification;
- project-specific alarms.
The support agent is a fourth consumer of the same diagnostic model:
common evidence / diagnostic layer
|
+---------------------------+---------------------------+
| | |
planning / commissioning event explainer runtime quality
| | |
+---------------------------+---------------------------+
|
v
support diagnostic agent
|
+---------------------------+---------------------------+
| | | |
system / pipeline point health reference health event synthesis
The support agent must not create a separate, incompatible quality model.
In particular, it must preserve the roadmap principles:
- do not confuse target image quality with reference-network quality;
- do not confuse a small residual with a well-constrained transform;
- do not use reference count without geometry and weight information;
- do not build new architecture around binary RANSAC inlier/outlier logic;
- make corrected measurement quality target-specific where possible;
- preserve deterministic integrity logic;
- use the LLM/agent for evidence collection, synthesis and support reasoning;
- keep raw evidence and provenance;
- keep the architecture estimator-independent.
3. Important estimator constraint
The current system uses a RANSAC-based similarity estimator, but work is in parallel to improve or replace it, potentially with a Danish-like robust weighted similarity adjustment.
Therefore, support logic must not be built around:
reference.inlier = true / false
Preferred concepts are:
- image-quality eligibility;
- image-quality weight;
- geometric residual;
- normalized residual;
- robust contribution weight;
- effective weight;
- leverage;
- influence;
- effective reference count;
- weight concentration / entropy;
- weighted geometry;
- transform covariance;
- target-specific propagated uncertainty;
- temporal contribution-state changes.
RANSAC may initially expose weights of 0 or 1 through the same generic interface. A future robust estimator may expose continuous weights between 0 and 1.
A binary accepted state may exist for compatibility or UI display, but it should be derived, not the architectural primitive.
4. Main diagnostic principle: traverse the dependency chain
A value shown to the user is the end of a chain. Support diagnosis must identify which layer failed or degraded.
A useful dependency chain is:
physical scene / structure
|
v
camera + illumination + target visibility
|
v
image acquisition
|
v
image storage / local availability
|
v
point / pattern measurement
|
v
reference similarity / compensation
|
v
corrected target displacement + quality
|
v
local persistence / queue / buffering
|
v
network / modem / VPN
|
v
FTP / SFTP / MQTT / future API transport
|
v
backend ingestion / processing
|
v
CSV export / API
|
v
web interface / plot / customer platform
This prevents a common support error:
“No value is visible” does not imply “the camera did not measure.”
It may instead mean:
- no acquisition;
- acquisition succeeded but processing failed;
- processing succeeded but local persistence failed;
- data exist locally but were not uploaded;
- upload is delayed or queued;
- backend ingestion failed;
- export failed;
- API/UI failed;
- timestamps make the data appear absent;
- a point was measured but rejected;
- reference compensation was degraded or invalid;
- the value exists in a reprocessed dataset rather than the original stream.
The agent should explicitly locate the failure/degradation in this chain.
5. Diagnostic domains
5.1 System / station health
Questions:
- Is the station powered and running?
- Is the SBC / processing service alive?
- Is the camera reachable?
- Is storage available?
- Is local data still being created?
- Is the station clock plausible?
- Did the system reboot?
- Are services repeatedly restarting?
- Is there a hardware or thermal issue?
- Is the modem alive?
- Is the station accessible locally even if remote communication is unavailable?
Candidate evidence, depending on availability:
- last acquisition timestamp;
- uptime / boot timestamp;
- service state;
- disk usage;
- CPU / memory / temperature;
- camera connectivity;
- local image count;
- local measurement-file timestamps;
- modem state;
- VPN state;
- power / battery status;
- system log excerpts;
- application log excerpts.
For the POC, much of this may be absent. The correct output in that situation is unknown / not observed, not an invented healthy state.
5.2 Acquisition health
This domain asks whether measurement opportunities are actually being generated.
Key checks:
- expected cadence vs observed acquisition cadence;
- missing epochs;
- duplicated epochs;
- irregular timing;
- images present or absent;
- acquisition gaps vs communication gaps;
- timestamps monotonic and correctly spaced;
- relationship between acquisition timestamp and export timestamp.
Important distinction:
NO_ACQUISITION
!=
NO_IMAGE_UPLOAD
!=
NO_CSV_EXPORT
!=
NO_UI_DISPLAY
The roadmap provenance vocabulary should be extended/reused where useful:
ACQUIRED_VALID
ACQUIRED_TARGET_REJECTED
ACQUIRED_REFERENCE_SOLUTION_DEGRADED
ACQUIRED_REFERENCE_SOLUTION_INVALID
NO_IMAGE
NO_ACQUISITION
REPROCESSED
Operational provenance may later add states such as:
ACQUIRED_LOCAL_ONLY
UPLOAD_PENDING
UPLOAD_FAILED
BACKEND_INGEST_FAILED
EXPORT_FAILED
UI_UNAVAILABLE
These should describe what is known to have happened, not speculate about root cause.
5.3 Communication / transport health
Relevant channels include current or planned:
- FTP / SFTP;
- internal MQTT;
- future external MQTT;
- VPN / Nebula;
- direct Ethernet access;
- future REST/API mechanisms.
Questions:
- Is the station connected?
- Is the transport service connected?
- When was the last successful transmission?
- Is there a queue/backlog?
- Are older data being re-uploaded after reconnection?
- Is transport FIFO behaviour causing undesirable latency or data loss?
- Are images and numerical data handled consistently?
- Do different export mechanisms have inconsistent recovery semantics?
- Is the station locally healthy but remote data stale?
The desired long-term behaviour should be coherent across transport methods. Support diagnosis should not assume that FTP, MQTT and future APIs necessarily share identical current behaviour.
5.4 Backend / interface health
The web interface is one diagnostic surface, not the ground truth for the entire station.
Relevant symptoms include:
- site not reachable;
- HTTP 404 for a plot/resource;
- plot not updating;
- apparent data gap;
- page available but export unavailable;
- export available but UI stale;
- staging and production behaving differently;
- timestamp rendering/timezone discrepancies.
The support agent should distinguish:
station health
data production
transport health
backend ingestion
export generation
UI rendering
A UI problem must not automatically be classified as a station problem.
5.5 Point / target observation health
This is a major extension beyond the reference-network roadmap.
A target can be locally unhealthy even when the station and reference network are healthy.
Relevant signals include:
Error Value;- visibility / occlusion;
- flux / brightness;
- saturation;
- SNR;
- sharpness;
- template consistency;
- raw H/V coordinates;
- local scatter / noise;
- unexpected raw jumps;
- missing measurements;
- repeated rejection;
- distance / mm-per-pixel;
- image location;
- prism/reflector brightness relative to distance;
- obstruction by other prisms or tunnel geometry.
Point health must separate at least:
A. Can the image feature be measured?
Image / tracking quality.
B. Is the raw measurement temporally plausible?
Raw coordinate behaviour before reference correction.
C. Is the corrected measurement trustworthy?
Raw point uncertainty plus reference-transform uncertainty.
For target (i):
corrected quality
=
target measurement quality
+
reference compensation quality propagated to target i
Conceptually:
[ \Sigma_{corrected,i}
\Sigma_{measurement,i} + J_i \Sigma_T J_i^T ]
Therefore, two targets in the same image can have different corrected-measurement quality even when they share one global transform.
5.6 Reference observation health
Each reference is also an image observation.
Keep separate:
Image observability
- Error Value;
- flux;
- saturation;
- SNR;
- sharpness;
- visibility;
- template consistency.
Geometric consistency
- residual;
- normalized residual;
- robust weight;
- leverage;
- influence;
- leave-one-reference-out effect.
A poor Error Value is not the same failure as a geometrically inconsistent but visually well-tracked reference.
5.7 Reference-network health
Use the existing roadmap directly.
Important quantities include:
- configured reference count;
- eligible reference count;
- effective reference count;
- per-reference contributions;
- weighted horizontal / vertical span;
- convex-hull coverage;
- weighted coordinate covariance;
- collinearity;
- condition number;
- transform covariance;
- residual scale /
sigma0; - weight concentration;
- temporal weight-state stability;
- leverage / influence;
- leave-one-reference-out sensitivity;
- depth distribution;
- target-specific propagated uncertainty.
Avoid conclusions based solely on:
- number of references;
- RANSAC inlier count;
- global
sigma0.
5.8 Camera / compensation state
The agent should be able to relate point behaviour to camera/reference compensation.
Relevant signals:
- translation;
- rotation;
- scale;
- parameter covariance;
- raw camera-state changes;
- abrupt transform jumps;
- persistent drift;
- coincidence between transform changes and common-mode point motion;
- changes in reference contribution state.
Useful support question:
Did the monitored object move, or did the compensation state change?
Evidence for a compensation event may include:
common-mode corrected-target jump
+ reference contribution change
+ transform jump
+ high reversibility
Evidence for probable object motion may include:
target displacement
+ stable reference contributions
+ stable transform
+ good target image quality
+ persistence
These remain deterministic evidence patterns, not semantic guesses.
5.9 Model adequacy / parallax
A statistically clean similarity transform can still be physically inadequate after significant 3D camera translation.
Potential evidence:
- large camera translation;
- residuals correlated with point depth;
- residuals correlated with image position/radius;
- target displacement correlated with target depth;
- abnormal scale/rotation;
- richer diagnostic model explains residual structure substantially better.
At first, model mismatch should be a diagnostic hypothesis / explainer feature, not an automatic rejection.
The support agent should be able to say:
The evidence is consistent with model mismatch / parallax, but this is not proven from the available data.
5.10 Datum / installation stability
A complete reference datum can move coherently and remain internally consistent.
The camera cannot prove physical stability of the entire datum using internal residuals alone.
This must be treated as an observability limitation.
Relevant external context can include:
- known construction work;
- station/support movement;
- mounting changes;
- geological motion;
- GNSS or other external sensor observations;
- manual survey;
- site intervention.
The agent must not overclaim:
“References are internally consistent” is not equivalent to “references are physically fixed.”
5.11 Data provenance and reprocessing
Support conclusions depend strongly on provenance.
The agent should determine, where possible:
- original online processing vs reprocessing;
- configuration version;
- pattern/reference configuration;
- reset/reinitialization events;
- whether motion compensation was applied;
- source CSV format;
- source station/backend;
- staging vs production;
- original acquisition time;
- processing time;
- export time;
- timezone interpretation.
A corrected value without provenance can be misleading during incident analysis.
6. POC data-access model
6.1 Deliberate limitation
For the POC, there is no requirement for automatic station querying.
Evidence will be supplied manually by the support engineer/user.
Typical workflow:
support question
|
v
agent identifies minimum useful evidence
|
v
user exports / pastes / uploads CSV or screenshots
|
v
agent parses and normalizes evidence
|
v
deterministic computations / comparisons
|
v
diagnostic synthesis
|
v
next evidence request OR support conclusion
The POC should therefore make the evidence request precise and economical.
Bad:
Please send all logs.
Better:
Please export the measurement CSV from 00:30–09:30 on 19 Sep, including Raw H/V and Relative H/V for all targets and references, and send a screenshot of the reference status around the jump.
6.2 Expected manual inputs
CSV
Potential columns, depending on current export format:
- timestamp;
- point/pattern name;
- role if available;
- Relative H / V [mm];
- Raw H / V [px];
- validity;
- Error Value;
- distance;
- image quality fields;
- reference state;
sigma0;- transform parameters;
- future residual / weight / influence / covariance fields.
The agent must tolerate different historical CSV schemas.
It should first identify:
available columns
missing expected columns
timestamp format
timezone
sampling cadence
point names
units
data source / export type if known
It should not silently assume a schema.
Interface captures
Useful examples:
- station overview;
- live plot;
- point table;
- reference state;
- configuration;
- current image;
- image with pattern overlays;
- system/network status;
- export status;
- error pages;
- timestamps visible in UI.
The capture itself is evidence and should be associated with:
- station/site;
- approximate timestamp;
- interface/environment;
- screenshot creation time if known;
- whether it represents live or historical state.
Camera / scene images
Can be used to inspect:
- point visibility;
- obstruction;
- saturation;
- illumination;
- condensation/dirt if visible;
- pattern placement;
- major camera viewpoint changes;
- possible scene changes.
Image interpretation remains qualitative unless deterministic image metrics are also provided.
Manual incident context
Examples:
- construction activity;
- known power interruption;
- local network outage;
- station physically touched;
- camera mount adjustment;
- pattern reset;
- pattern configuration changed;
- reference excluded;
- software update;
- server migration;
- reprocessing performed.
This context should be stored as evidence with source operator_note, not mixed into measured facts.
7. Suggested POC evidence bundle
For repeatable investigations, organize each case approximately as:
case_name/
context.md
measurements.csv
diagnostics.csv # optional
configuration.json # optional
notes.md
screenshots/
...
images/
...
logs/
... # optional at POC
Example context.md:
site: Barcelona tunnel
station: <station-id>
question: Why did D-300-A jump?
period_start: 2026-09-19T00:00:00+02:00
period_end: 2026-09-19T12:00:00+02:00
known_actions:
- local pattern reset later performed
files:
- measurements.csv
- screenshots/reference_state.png
This need not be mandatory for ad-hoc use, but it is useful for regression datasets.
8. Normalized evidence model
The agent should normalize heterogeneous manual inputs into a generic evidence structure.
Example:
{
"case": {
"site": "Barcelona",
"station_id": null,
"time_window": {
"start": "2026-09-19T00:00:00+02:00",
"end": "2026-09-19T12:00:00+02:00"
},
"question": "Why did D-300-A jump?"
},
"sources": [
{
"source_id": "csv_01",
"type": "measurement_csv",
"provenance": {
"original_or_reprocessed": "unknown",
"export_path": "SFTP",
"timezone": "unknown"
}
},
{
"source_id": "ui_01",
"type": "interface_capture",
"timestamp": "2026-09-19T08:10:00+02:00"
}
],
"observations": {},
"derived_features": {},
"diagnostic_assessment": {}
}
The normalization layer matters because historical AIMsight data and future APIs will not necessarily expose the same fields.
9. Health-state representation
Do not collapse everything into one global Boolean healthy.
Each domain should expose:
- raw metrics / evidence;
- a derived state if a deterministic rule exists;
- evidence completeness;
- diagnostic notes.
Suggested operational state vocabulary:
HEALTHY
DEGRADED
FAILED
UNKNOWN
NOT_OBSERVED
NOT_APPLICABLE
Suggested measurement/reference states can retain the roadmap terminology:
VALID
DEGRADED
INVALID
MODEL_MISMATCH_SUSPECTED
DATUM_STABILITY_UNVERIFIED
These states should remain domain-specific.
Example:
{
"station": {
"state": "UNKNOWN",
"reason": "No system status or local acquisition evidence supplied."
},
"transport": {
"state": "DEGRADED",
"reason": "15-minute export gap while local acquisition evidence continues."
},
"target:D-300-A": {
"observation_state": "HEALTHY",
"corrected_measurement_state": "DEGRADED"
},
"reference_network": {
"state": "DEGRADED"
}
}
This avoids contradictions such as “the station is bad” when only one point is poorly tracked.
10. Evidence completeness and epistemic rules
The support agent must explicitly distinguish:
- observed fact;
- deterministic derived feature;
- known operator context;
- hypothesis;
- unresolved question.
Recommended confidence wording:
confirmed by supplied evidence
strongly supported
consistent with
possible
insufficient evidence
not observed
Avoid pseudo-probabilities unless a calibrated model exists.
The agent must not claim:
- station downtime from UI downtime alone;
- data loss from a missing plot alone;
- physical reference stability from internal geometric consistency alone;
- structural movement from corrected displacement without considering reference/camera state;
- camera motion from a transform change alone if the evidence is ambiguous;
- network outage from an export gap without a network signal/log.
11. Deterministic analysis available at POC
Even with manual CSV input, the support workflow can compute useful deterministic features.
11.1 Time / cadence
- median cadence;
- expected vs observed intervals;
- missing epochs;
- longest gap;
- duplicated timestamps;
- non-monotonic timestamps;
- time offset between related files;
- timezone-like constant offsets;
- freshness / last sample.
11.2 Point behaviour
- data completeness per point;
- local median / MAD;
- robust z-score;
- step amplitude;
- trend / rate;
- short-term scatter;
- persistence;
- reversibility;
- correlation between Raw H/V and Relative H/V;
- image quality changes coincident with displacement.
11.3 Spatial behaviour
- fraction of targets moving;
- sign agreement;
- common-mode component;
- pairwise correlation;
- clustering of affected points;
- correlation with image position or distance.
11.4 Reference behaviour
As fields become available:
- per-reference residual;
- normalized residual;
- robust weight;
- effective reference count;
- weight concentration;
- reference-state changes;
- geometry metrics;
sigma0/ robust scale;- transform change;
- transform covariance;
- influence / leave-one-out sensitivity.
11.5 Before/after comparison
Compare:
- known-good baseline;
- incident window;
- post-intervention / reprocessed window.
This is particularly useful for regression cases.
12. Diagnostic hypothesis families
The agent should organize evidence around a limited set of support hypotheses.
12.1 Target-local observation degradation
Typical evidence:
one/few targets affected
+ Error Value / brightness / SNR / visibility degradation
+ healthy reference solution
+ no matching common-mode response
Potential causes to investigate:
- occlusion;
- obstruction by another target;
- low or excessive reflected intensity;
- illumination issue;
- poor template;
- scene change;
- target physically altered.
12.2 Reference-compensation instability
Typical evidence:
common-mode corrected-target jump
+ reference contribution change
+ transform jump
+ possible reversibility
Barcelona is the primary regression case.
12.3 Weak reference-network geometry
Evidence can include:
- low effective reference count;
- concentrated weights;
- poor span / collinearity;
- high condition number;
- high transform covariance;
- high target-specific propagated uncertainty;
- high influence of one reference.
This can occur even when residuals are small.
12.4 Camera/station movement
Possible evidence:
- strong transform change;
- many raw point coordinates move coherently;
- reference diagnostics change;
- target response common-mode;
- known physical/site event.
A camera movement may still be compensated successfully. The agent should distinguish:
camera moved but compensation remained valid
from:
camera moved and compensation became unstable
and from:
camera moved enough for similarity-model mismatch / parallax
12.5 Model mismatch / parallax
Possible evidence:
- large camera movement;
- depth-dependent residuals;
- target displacement related to target depth;
- structured residual field;
- similarity transform statistically insufficient.
Initially diagnostic only.
12.6 Acquisition failure
Evidence:
- no new local images;
- no local measurement epochs;
- acquisition service failure;
- simultaneous local and remote data stop.
Must be distinguished from transport failure.
12.7 Transport / export failure
Evidence:
- local acquisition continues;
- remote/export data stop or lag;
- network/modem problem;
- upload queue/backlog;
- data later appear after recovery.
Important future diagnostic: whether data are actually recoverable and re-uploaded after outages.
12.8 Backend / UI failure
Evidence:
- source data/export exists;
- web page, plot or resource fails;
- 404 or rendering issue;
- staging/production discrepancy.
12.9 Timestamp / timezone issue
Evidence:
- otherwise regular data consistently offset by a fixed amount;
- UI timestamps differ from CSV/acquisition times;
- future timestamps;
- mismatch close to timezone/DST offset.
Do not modify timestamps silently. Identify the assumed time basis.
12.10 Coherent datum motion
Internal reference consistency may remain good.
Requires external/contextual evidence.
The correct diagnosis may remain:
reference datum internally consistent;
absolute datum stability cannot be verified from AIMsight image data alone.
13. Support query workflow
For each question:
Step 1 — Parse the question
Identify:
- site / station;
- point(s);
- time window;
- symptom;
- expected behaviour;
- comparison period if given.
Step 2 — Map symptom to diagnostic layers
Example:
“Point D-300-A jumped at 08:00.”
Relevant layers:
- target observation quality;
- Raw H/V;
- corrected H/V;
- other targets;
- reference network;
- transform;
- acquisition timestamps;
- intervention context.
Step 3 — Inventory available evidence
Report:
- what is available;
- what is missing;
- which missing evidence materially limits diagnosis.
Step 4 — Run deterministic checks
Prefer calculations over semantic interpretation where possible.
Step 5 — Build candidate explanations
Rank only by supplied evidence.
Step 6 — Try to falsify the leading explanation
Example:
If “target tracking problem” is suspected:
- did Raw H/V jump?
- did Error Value degrade?
- did neighboring/other targets jump?
- did the transform change?
- did the same target recover after reprocessing?
Step 7 — Produce support diagnosis
Include:
- affected layer;
- leading diagnosis;
- evidence;
- alternative explanations;
- unknowns;
- exact next check.
Step 8 — Preserve case for regression
Once resolved, store:
- evidence bundle;
- final expert diagnosis;
- intervention;
- expected agent result.
14. Recommended support-agent response format
Use a compact but traceable format:
Assessment
----------
Most likely issue: reference-compensation instability.
Measurement availability: data present.
Target-local tracking: no clear degradation.
Reference network: unstable around event.
System / communication: no evidence of failure in supplied files.
Evidence
--------
- D-300-A and multiple targets jump together.
- Raw target positions do not show an equivalent local-only tracking failure.
- Reference contribution/state changes at the same epoch.
- Similarity transform changes at the same epoch.
- Event is reversible.
What is not established
-----------------------
- Physical stability of the complete reference datum.
- Whether a network interruption also occurred; no network log supplied.
Next check
----------
Provide the reference residual/weight table for 07:30–08:30 if available.
For machine use, the same result can be represented as structured JSON.
15. Suggested structured output
{
"query": {
"site": "Barcelona",
"station": null,
"targets": ["D-300-A"],
"time_window": {},
"question": "Why did the point jump?"
},
"evidence_completeness": {
"measurement_csv": true,
"raw_coordinates": true,
"target_quality": true,
"reference_diagnostics": true,
"system_status": false,
"transport_logs": false
},
"domain_health": {
"system": {"state": "UNKNOWN"},
"acquisition": {"state": "HEALTHY"},
"transport": {"state": "UNKNOWN"},
"backend_ui": {"state": "NOT_OBSERVED"},
"target_observation": {"state": "HEALTHY"},
"reference_network": {"state": "DEGRADED"},
"model_adequacy": {"state": "NOT_OBSERVED"}
},
"diagnosis": {
"primary": "REFERENCE_COMPENSATION_INSTABILITY",
"support": "strongly_supported",
"alternatives": [
{
"type": "TARGET_LOCAL_DEGRADATION",
"support": "weak"
}
]
},
"evidence": [],
"unknowns": [],
"next_checks": []
}
Do not treat this schema as fixed. It should evolve with the POC.
16. Relevant previous AIMsight support contexts
16.1 Barcelona tunnel — core regression case
Barcelona remains the most important initial incident dataset.
Known context includes:
- 1-minute SFTP CSV export;
- images at a lower cadence (15 minutes in the incident workflow);
- a quiet baseline around 12 Sep;
- station/camera movement during the 19 Sep incident;
- correlated movement later in the morning;
- reference instability involving PR_284 / PR_291;
- repeated switching between plausible compensation states;
- monitored targets remaining individually measurable while compensation changed;
- local pattern reset returning values to zero;
- later pattern reconfiguration stabilizing the system;
- discussion of excluding PR_284 as a fallback;
- loss of references on the left in a later episode;
- proposal for an additional reference;
- questions about dedicated FTP export;
- need to separate target tracking, reference quality, camera motion and export/acquisition behaviour.
Expected support-agent value:
- diagnose a target jump without assuming target failure;
- identify common-mode behaviour;
- identify reference-state changes;
- distinguish station/camera movement from structural motion;
- distinguish original from reprocessed data;
- explain missing data around network/export events;
- request the minimum extra evidence needed.
The Barcelona incident should remain a regression suite, not a special-case rule.
16.2 Barcelona network-loss / MQTT / export behaviour
A separate but related support thread highlighted disappointing behaviour during loss of network connectivity:
- holes in exported data;
- holes in images;
- suspicion around the newer MQTT chunk implementation;
- questions about whether FTP data remained locally recoverable;
- questions about missing re-upload/recovery logic;
- undesirable FIFO behaviour;
- desire to unify semantics across SFTP/FTP, internal MQTT and future external MQTT.
This led to the broader concept of full agentic observability.
Desired future observability includes:
- application/system logs;
- modem/network logs;
- acquisition state;
- local stored files;
- upload queue/backlog;
- last successful transport;
- requested on-demand diagnostics;
- local USB/Ethernet access for diagnosing a station when remote communication itself is unavailable.
This thread is the main context for the system-health and data-pipeline side of the support agent.
16.3 Staging / web access + FTP failure
A reported situation combined:
- inability to access
staging.aimsight.ch; - FTP export not working.
This is a useful diagnostic case because the two symptoms may have:
- one common backend/network cause;
- independent causes;
- a station-side transport issue plus a separate UI issue.
The agent should not infer a common cause without evidence.
Minimum useful checks include:
- local acquisition freshness;
- last successful upload;
- FTP/SFTP status;
- backend ingestion freshness;
- direct export availability;
- UI/API reachability.
16.4 Lenzspitze
Relevant symptoms included:
- live plot error / 404;
- timestamps approximately two hours in the future;
- suspected timezone handling issue.
This is an important case because the measurement system may be producing data while the user-visible presentation is wrong.
Agent checks should include:
- actual CSV timestamps;
- station timestamp;
- timezone / UTC interpretation;
- backend timestamps;
- HTTP/UI resource failure;
- data freshness independent of plot rendering.
16.5 Monitoring / staging migration and CSV formats
AIMsight currently has or has had multiple paths/environments including:
monitoring.aimsight.ch;staging.aimsight.ch;- CSV outputs with differing formats;
- plans for more direct export from stations.
For the agent this means:
- source provenance is mandatory;
- CSV parsers should be schema-tolerant;
- identical-looking data from different sources may have different processing histories;
- a support conclusion should state which data path was analyzed.
16.6 Data Exploration / service modes
Relevant product context includes:
- Relative H/V [mm];
- Raw H/V [px];
- image quality / visibility / reliability signals;
- HDF5 as a more advanced source;
- simplified CSV export;
- optional reprocessing from stored images;
- local standalone operation;
- remote access through VPN/Nebula;
- cloud service mode.
These are natural support evidence sources.
The support agent should exploit the distinction between:
raw measurement
corrected measurement
diagnostic quality
processing provenance
rather than treating the exported displacement column as the only signal.
16.7 Installation / target visibility cases
Prior installation discussions provide useful point-health context:
- all prisms must remain visible;
- eye-level installation can be acceptable;
- vertical offsets can reduce prism-to-prism obstruction;
- larger prisms become useful at long range;
- short-range prisms can be unnecessarily bright;
- tunnel lighting conditions change reflector requirements;
- live view can be checked remotely or by direct Ethernet connection.
These are not estimator problems. They are installation/observation-quality problems and belong in the support knowledge base.
16.8 Crissier and future cases
Crissier is expected to become another regression/test case.
Do not invent its diagnosis before the data/context are added.
Each new resolved case should add:
- symptom;
- evidence available;
- expert diagnosis;
- intervention;
- deterministic features that should have detected/explained the issue;
- evidence that would have shortened the investigation;
- expected support-agent answer.
17. Regression-case philosophy
Historical cases should be used like software tests.
For each case define:
input evidence
support question
expected deterministic observations
expected diagnostic classification
important uncertainty / observability limitation
expected next-check recommendation
The regression suite should test not only correct diagnoses but also correct refusal to overclaim.
Examples:
Barcelona reference switching
Agent should detect:
- common-mode target response;
- reference-state/contribution changes;
- transform change;
- reversibility.
It should not diagnose “bad target D-300-A”.
UI 404
Agent should not diagnose station failure if source/export evidence shows current data.
Export gap with local acquisition
Agent should classify a pipeline/transport issue rather than NO_ACQUISITION.
Coherent reference movement
Agent should flag datum stability as unverified rather than claiming a healthy absolute datum.
18. POC implementation architecture
Keep the POC lightweight.
manual evidence
CSV / screenshots / notes
|
v
input adapters
|
v
normalized evidence model
|
+-------------------------+
| |
v v
deterministic analyzers visual/manual evidence extraction
| |
+-------------+-----------+
|
v
diagnostic facts
|
v
support-agent reasoning
|
+-----------+------------+
| |
v v
support explanation request next evidence
Suggested components:
A. Input adapters
- flexible CSV loader;
- screenshot/image input;
- manual context parser;
- optional log parser.
B. Deterministic analyzers
- cadence/gap analyzer;
- point-statistics analyzer;
- common-mode/spatial analyzer;
- reference-network diagnostics adapter;
- transform-change analyzer;
- provenance/timestamp checker.
C. Diagnostic fact store
Structured observations such as:
{
"fact": "COMMON_MODE_STEP",
"start": "...",
"magnitude": "...",
"affected_fraction": 0.86,
"source": ["csv_01"],
"method": "deterministic_v1"
}
D. Support reasoning layer
The LLM/agent:
- interprets the user question;
- selects relevant diagnostic facts;
- requests missing evidence;
- compares hypotheses;
- explains results in support language;
- links remediation/checks to evidence.
It should not silently replace deterministic validity rules.
19. Manual evidence-request strategy
The agent should minimize support friction.
Ask for the next most discriminating piece of evidence.
Example decision sequence for:
“Why is this point jumping?”
- Obtain CSV with Raw and Relative H/V for affected and neighboring points.
- If only corrected values jump, inspect reference/transform state.
- If raw value of one point jumps alone, inspect target image quality/capture.
- If many raw values jump together, inspect camera state / physical station movement.
- If local data are missing, distinguish acquisition from export using local image/measurement timestamps.
- If only web display is missing, inspect export/API/backend separately.
This should eventually become a formal evidence-request policy.
20. Support actions and remediation
The POC is primarily diagnostic/read-only.
Potential recommendations can include:
- inspect current image;
- inspect a specific point/pattern;
- export a narrower/broader CSV period;
- compare Raw and Relative coordinates;
- check known-good baseline;
- check reference residuals/weights;
- reprocess stored images;
- verify timezone configuration;
- verify transport backlog;
- connect locally through Ethernet;
- check VPN/modem;
- reconfigure a pattern/reference;
- consider reference exclusion/downweighting;
- inspect camera mounting;
- add/improve a reference.
However:
- do not automatically modify configuration;
- do not automatically exclude references;
- do not trigger reprocessing or station restart at POC;
- do not treat a support recommendation as a deterministic validity rule.
Future action-capable agents should require explicit authorization for consequential changes.
21. Interaction with automatic exclusion / alarms
The support agent and production quality logic serve different purposes.
Production quality logic should be:
- deterministic;
- transparent;
- estimator-independent;
- validated on regression datasets;
- configured by project policy.
The support agent can be more exploratory:
- correlate multiple evidence sources;
- compare periods;
- identify likely causes;
- request extra data;
- explain a deterministic alarm;
- explain why no alarm fired;
- identify missing observability.
A useful development loop is:
support incident
->
agent investigation
->
recurrent deterministic diagnostic discovered
->
add feature to event explainer / shadow alarms
->
validate across cases
->
possibly promote to production quality logic
Thus support work is also a discovery mechanism for missing production diagnostics.
22. Observability gaps the POC should expose
One objective of the POC is to determine which telemetry is worth implementing.
Likely high-value future signals include:
Station
- uptime / last reboot;
- application version;
- service state;
- storage remaining;
- last image acquisition;
- last successful processing epoch;
- camera connectivity;
- station clock/timezone.
Communication
- modem state;
- VPN state;
- last successful remote contact;
- last successful upload per channel;
- queued items / bytes;
- oldest queued item;
- retry count;
- transport error code.
Data pipeline
- locally acquired epochs;
- locally stored images;
- locally processed epochs;
- exported epochs;
- backend-ingested epochs;
- current backlog;
- re-upload/recovery state.
Point
- Error Value;
- visibility;
- brightness / flux;
- SNR;
- sharpness;
- saturation;
- raw coordinate;
- corrected coordinate;
- local measurement uncertainty;
- validity/rejection reason.
Reference / transform
- residual;
- normalized residual;
- robust weight;
- effective weight;
- leverage;
- influence;
- effective reference count;
- weighted geometry;
- transform;
- covariance;
- target-specific propagated uncertainty;
- model-mismatch indicators.
The POC should record every time an investigation is blocked because one of these signals is unavailable.
23. Priority POC use cases
Recommended initial set:
UC1 — Why is one point bad?
Focus:
- target image/tracking quality;
- raw vs corrected coordinates;
- local vs global behaviour.
UC2 — Why did many points jump?
Focus:
- common-mode response;
- transform;
- reference contribution state;
- camera movement.
UC3 — Why is there no recent data?
Focus:
- acquisition vs local processing vs transport vs backend/UI.
UC4 — Is a station healthy?
Return domain-by-domain health, not one Boolean.
UC5 — What changed around an event?
Before/after comparison.
UC6 — Can this corrected displacement be trusted?
Focus:
- target-local quality;
- reference-network quality;
- transform uncertainty;
- target-specific propagated uncertainty;
- provenance.
UC7 — Why do CSV and web UI disagree?
Focus:
- source;
- timestamps;
- backend/export path;
- processing/reprocessing provenance.
UC8 — What evidence should support request next?
Optimize the next diagnostic action.
24. Definition of POC success
The POC succeeds if, on a small set of real support cases, it can:
- correctly identify the affected diagnostic layer(s);
- avoid conflating point, reference, system and pipeline problems;
- compute useful features from manually supplied CSVs;
- use screenshots/context as supporting evidence;
- state what is known vs unknown;
- recommend the next discriminating check;
- reproduce expert diagnosis for resolved cases such as Barcelona;
- avoid overclaiming on incomplete cases;
- reveal missing telemetry worth adding;
- preserve compatibility with the evolving reference estimator.
The POC does not need to:
- autonomously connect to stations;
- autonomously access the backend;
- automatically repair systems;
- automatically change reference configuration;
- replace deterministic validity logic;
- solve every historical support case.
25. Near-term development sequence
Phase S0 — Context + regression cases
- establish this context document;
- collect Barcelona evidence;
- add Crissier when available;
- add Lenzspitze/UI-time case;
- add at least one data-export/network case;
- store expert diagnoses.
Phase S1 — Manual CSV diagnostic notebook/tool
Implement:
- schema inspection;
- timestamp normalization;
- cadence/gaps;
- per-point completeness;
- robust scatter / steps;
- Raw vs Relative comparison;
- common-mode correlation;
- before/after plots/tables;
- structured diagnostic output.
Phase S2 — Screenshot / manual context ingestion
Support:
- interface captures;
- current camera images;
- manually supplied configuration/status;
- evidence linkage to time windows.
Phase S3 — Reference diagnostics adapter
Consume the generic estimator-independent outputs from the roadmap:
- residual;
- weight;
- effective count;
- geometry;
- covariance;
- influence;
- transform.
Initially, the current RANSAC implementation can populate binary contribution weights.
Phase S4 — Support reasoning / evidence request
Given a user question:
- determine likely diagnostic layers;
- run available analyzers;
- identify missing discriminating evidence;
- provide support diagnosis.
Phase S5 — Observability specification
Use POC investigations to define the future station/backend diagnostic API.
Only after this should live automated querying become a priority.
26. Longer-term query/API direction
The manual POC should anticipate future calls such as:
get_station_health(station, time)
get_acquisition_status(station, window)
get_transport_status(station, window)
get_export_status(station, channel, window)
get_point_timeseries(station, points, window, fields)
get_point_quality(station, point, window)
get_reference_fit(station, window)
get_reference_network_quality(station, window)
get_transform_timeseries(station, window)
get_images(station, window)
get_logs(station, subsystem, window)
compare_periods(station, baseline, incident)
reprocess_preview(station, window, configuration) # future, controlled
The support agent should depend on semantic capabilities like these rather than on a specific current backend implementation.
27. Important non-goals / anti-patterns
Do not:
- build support logic around RANSAC inlier/outlier membership;
- use a single station-level
healthyflag as the primary representation; - use
sigma0as the only reference-quality metric; - assume “missing from UI” means “not acquired”;
- assume a bad corrected target means the target itself is badly tracked;
- assume a common-mode displacement is structural;
- assume a network outage necessarily causes data loss;
- assume internally consistent references are physically stable;
- let the LLM invent deterministic validity thresholds;
- hard-code Barcelona-specific rules;
- create automatic remediation before diagnosis is reliable;
- silently merge original and reprocessed data.
28. Guiding design principles
- Trace the full dependency chain.
- Keep operational health and measurement integrity separate.
- Keep point observation quality and corrected-point quality separate.
- Reuse the common reference/geometry diagnostic layer.
- Remain estimator-independent.
- Prefer continuous quality metrics over binary states.
- Use deterministic computations wherever practical.
- Use the agent for evidence orchestration, hypothesis comparison and explanation.
- Make provenance first-class.
- Represent unknowns explicitly.
- Request the next most discriminating evidence, not everything.
- Turn resolved support incidents into regression tests.
- Use blocked diagnoses to define future observability.
- Do not confuse data transport, backend and UI failures with acquisition failures.
- Preserve target-specific uncertainty and reference-network effects.
29. Working mental model
A concise mental model for future development is:
WHAT DOES THE USER SEE?
|
v
UI / export evidence
|
v
DID THE DATA REACH HERE?
|
+------------+-------------+
| |
no yes
| |
trace pipeline backward IS THE VALUE PLAUSIBLE?
| |
v v
transport -> local data target raw measurement
| |
v v
acquisition? target image quality
|
v
reference compensation
|
v
camera / model
|
v
physical interpretation
The support agent's job is not merely to answer questions. It should make this chain inspectable, repeatable and increasingly deterministic.
30. Context instruction for future agents/developers
When working on this project:
- use
AIMsight_reference_quality_event_explainer_roadmap.mdas the authoritative baseline for the reference-network, event-explainer and planning architecture; - continue from that roadmap rather than re-deriving it;
- treat the support agent as an extension that adds system, pipeline and point-level diagnosis;
- at POC stage assume evidence is manually supplied as CSVs, screenshots/images and operator context;
- do not prematurely design live data connectors;
- keep the architecture ready for future automated queries;
- do not introduce dependencies on RANSAC binary inlier/outlier logic;
- prefer residuals, continuous weights, influence/leverage, effective reference count, transform covariance and target-specific propagated uncertainty;
- use Barcelona as one important regression case;
- add Crissier and other real support cases as they become available;
- distinguish deterministic evidence from LLM synthesis;
- make uncertainty, provenance and observability limitations explicit;
- use support investigations to identify missing telemetry and candidate deterministic alarms.
The long-term objective is a support system able to move from:
“Something looks wrong.”
to:
“The station is acquiring normally; point observations remain measurable; the numerical export became stale after 11:42 while local acquisition continued; the evidence therefore localizes the issue to the transport/export path. Reference compensation was stable in the supplied interval. The next most useful check is the upload queue / last successful FTP or MQTT transaction.”
That level of layered, evidence-based diagnosis is the target.