Files
Support-Agent/docs/aimsight/AIMsight_agentic_support_project_context.md
maximeandClaude Opus 5.5 8a32f4d418 Organize the project by domain and name analyses by their rule
Energy code moves to src/energy/ and its docs to docs/energy/; shared code
(CSV loading, venv switch, browser, zoom script) goes to src/common/, and the
AIMsight originals to docs/aimsight/. Analyses are no longer numbered: two
Energy scripts remain, daily.py (delivered state and trend) and
all_analyses.py (comparison of the four). Calculations and pages unchanged (D-026).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 17:12:37 +02:00

51 KiB
Raw Permalink Blame History

AIMsight Agentic Support — Project Context

0. Purpose of this document

This document is the working context for development of an AIMsight support and diagnostic agent.

It extends, but does not replace, the existing roadmap:

AIMsight_reference_quality_event_explainer_roadmap.md

That roadmap remains the main reference for:

  1. planning / installation assessment for geometry, precision and stability;
  2. deterministic event explanation;
  3. reference observation, reference-network quality, automatic exclusion/downweighting and integrity alarms;
  4. estimator-independent diagnostics compatible with the current RANSAC similarity estimator and a future Danish-like robust weighted similarity adjustment.

The support-agent project adds a broader operational layer:

  • system health;
  • acquisition health;
  • communications / export health;
  • backend / interface health;
  • individual point / target health;
  • reference observation and reference-network health;
  • camera / transform / model-adequacy diagnostics;
  • data provenance and reprocessing state;
  • synthesis of these layers into a support-oriented diagnosis.

At the initial POC stage, data access is deliberately simple and manual. The user will provide or query evidence such as:

  • CSV exports;
  • screenshots / captures of AIMsight interfaces;
  • screenshots or saved images from the camera;
  • configuration extracts;
  • manually copied status information;
  • incident notes and timestamps;
  • later, selected logs when available.

The POC should therefore optimize the diagnostic reasoning model, evidence requirements, deterministic feature extraction and support workflow before building live station/backend integrations.


1. Core objective

The support agent should help answer support questions such as:

  • Is this AIMsight station healthy?
  • Why is there no data after a given time?
  • Was the image acquired but not uploaded?
  • Is the problem local to one monitored point?
  • Is a point poorly visible, saturated, occluded or badly tracked?
  • Is a corrected displacement trustworthy?
  • Did the camera or station move?
  • Did the reference compensation change?
  • Is the reference network weak or unstable?
  • Is the observed event probably physical motion, a compensation event, a point-quality problem, a model mismatch or a data-pipeline problem?
  • Why does the web interface disagree with a CSV export?
  • Why is a plot empty or returning an error although the station may still be operating?
  • Are timestamps consistent with the station, export and UI clocks?
  • Was data lost, delayed, buffered, re-uploaded or merely not displayed?
  • What changed relative to a known-good period?
  • What evidence is still required before a conclusion can be reached?
  • What is the next most informative support check?

The agent should reduce the number of ad-hoc manual investigations while making its reasoning traceable and evidence-based.


2. Relationship to the reference-quality / event-explainer roadmap

The existing roadmap establishes a shared geometric and diagnostic layer around:

  • target image quality;
  • per-reference residuals;
  • robust contribution weights;
  • leverage and influence;
  • effective reference count;
  • weighted reference geometry;
  • transform covariance;
  • target-specific propagated uncertainty;
  • transform / camera state;
  • temporal weight-state changes;
  • model mismatch / parallax diagnostics;
  • deterministic event classification;
  • project-specific alarms.

The support agent is a fourth consumer of the same diagnostic model:

                         common evidence / diagnostic layer
                                      |
          +---------------------------+---------------------------+
          |                           |                           |
 planning / commissioning       event explainer             runtime quality
          |                           |                           |
          +---------------------------+---------------------------+
                                      |
                                      v
                              support diagnostic agent
                                      |
          +---------------------------+---------------------------+
          |                |                |                    |
   system / pipeline    point health   reference health    event synthesis

The support agent must not create a separate, incompatible quality model.

In particular, it must preserve the roadmap principles:

  • do not confuse target image quality with reference-network quality;
  • do not confuse a small residual with a well-constrained transform;
  • do not use reference count without geometry and weight information;
  • do not build new architecture around binary RANSAC inlier/outlier logic;
  • make corrected measurement quality target-specific where possible;
  • preserve deterministic integrity logic;
  • use the LLM/agent for evidence collection, synthesis and support reasoning;
  • keep raw evidence and provenance;
  • keep the architecture estimator-independent.

3. Important estimator constraint

The current system uses a RANSAC-based similarity estimator, but work is in parallel to improve or replace it, potentially with a Danish-like robust weighted similarity adjustment.

Therefore, support logic must not be built around:

reference.inlier = true / false

Preferred concepts are:

  • image-quality eligibility;
  • image-quality weight;
  • geometric residual;
  • normalized residual;
  • robust contribution weight;
  • effective weight;
  • leverage;
  • influence;
  • effective reference count;
  • weight concentration / entropy;
  • weighted geometry;
  • transform covariance;
  • target-specific propagated uncertainty;
  • temporal contribution-state changes.

RANSAC may initially expose weights of 0 or 1 through the same generic interface. A future robust estimator may expose continuous weights between 0 and 1.

A binary accepted state may exist for compatibility or UI display, but it should be derived, not the architectural primitive.


4. Main diagnostic principle: traverse the dependency chain

A value shown to the user is the end of a chain. Support diagnosis must identify which layer failed or degraded.

A useful dependency chain is:

physical scene / structure
        |
        v
camera + illumination + target visibility
        |
        v
image acquisition
        |
        v
image storage / local availability
        |
        v
point / pattern measurement
        |
        v
reference similarity / compensation
        |
        v
corrected target displacement + quality
        |
        v
local persistence / queue / buffering
        |
        v
network / modem / VPN
        |
        v
FTP / SFTP / MQTT / future API transport
        |
        v
backend ingestion / processing
        |
        v
CSV export / API
        |
        v
web interface / plot / customer platform

This prevents a common support error:

“No value is visible” does not imply “the camera did not measure.”

It may instead mean:

  • no acquisition;
  • acquisition succeeded but processing failed;
  • processing succeeded but local persistence failed;
  • data exist locally but were not uploaded;
  • upload is delayed or queued;
  • backend ingestion failed;
  • export failed;
  • API/UI failed;
  • timestamps make the data appear absent;
  • a point was measured but rejected;
  • reference compensation was degraded or invalid;
  • the value exists in a reprocessed dataset rather than the original stream.

The agent should explicitly locate the failure/degradation in this chain.


5. Diagnostic domains

5.1 System / station health

Questions:

  • Is the station powered and running?
  • Is the SBC / processing service alive?
  • Is the camera reachable?
  • Is storage available?
  • Is local data still being created?
  • Is the station clock plausible?
  • Did the system reboot?
  • Are services repeatedly restarting?
  • Is there a hardware or thermal issue?
  • Is the modem alive?
  • Is the station accessible locally even if remote communication is unavailable?

Candidate evidence, depending on availability:

  • last acquisition timestamp;
  • uptime / boot timestamp;
  • service state;
  • disk usage;
  • CPU / memory / temperature;
  • camera connectivity;
  • local image count;
  • local measurement-file timestamps;
  • modem state;
  • VPN state;
  • power / battery status;
  • system log excerpts;
  • application log excerpts.

For the POC, much of this may be absent. The correct output in that situation is unknown / not observed, not an invented healthy state.


5.2 Acquisition health

This domain asks whether measurement opportunities are actually being generated.

Key checks:

  • expected cadence vs observed acquisition cadence;
  • missing epochs;
  • duplicated epochs;
  • irregular timing;
  • images present or absent;
  • acquisition gaps vs communication gaps;
  • timestamps monotonic and correctly spaced;
  • relationship between acquisition timestamp and export timestamp.

Important distinction:

NO_ACQUISITION
!=
NO_IMAGE_UPLOAD
!=
NO_CSV_EXPORT
!=
NO_UI_DISPLAY

The roadmap provenance vocabulary should be extended/reused where useful:

ACQUIRED_VALID
ACQUIRED_TARGET_REJECTED
ACQUIRED_REFERENCE_SOLUTION_DEGRADED
ACQUIRED_REFERENCE_SOLUTION_INVALID
NO_IMAGE
NO_ACQUISITION
REPROCESSED

Operational provenance may later add states such as:

ACQUIRED_LOCAL_ONLY
UPLOAD_PENDING
UPLOAD_FAILED
BACKEND_INGEST_FAILED
EXPORT_FAILED
UI_UNAVAILABLE

These should describe what is known to have happened, not speculate about root cause.


5.3 Communication / transport health

Relevant channels include current or planned:

  • FTP / SFTP;
  • internal MQTT;
  • future external MQTT;
  • VPN / Nebula;
  • direct Ethernet access;
  • future REST/API mechanisms.

Questions:

  • Is the station connected?
  • Is the transport service connected?
  • When was the last successful transmission?
  • Is there a queue/backlog?
  • Are older data being re-uploaded after reconnection?
  • Is transport FIFO behaviour causing undesirable latency or data loss?
  • Are images and numerical data handled consistently?
  • Do different export mechanisms have inconsistent recovery semantics?
  • Is the station locally healthy but remote data stale?

The desired long-term behaviour should be coherent across transport methods. Support diagnosis should not assume that FTP, MQTT and future APIs necessarily share identical current behaviour.


5.4 Backend / interface health

The web interface is one diagnostic surface, not the ground truth for the entire station.

Relevant symptoms include:

  • site not reachable;
  • HTTP 404 for a plot/resource;
  • plot not updating;
  • apparent data gap;
  • page available but export unavailable;
  • export available but UI stale;
  • staging and production behaving differently;
  • timestamp rendering/timezone discrepancies.

The support agent should distinguish:

station health
data production
transport health
backend ingestion
export generation
UI rendering

A UI problem must not automatically be classified as a station problem.


5.5 Point / target observation health

This is a major extension beyond the reference-network roadmap.

A target can be locally unhealthy even when the station and reference network are healthy.

Relevant signals include:

  • Error Value;
  • visibility / occlusion;
  • flux / brightness;
  • saturation;
  • SNR;
  • sharpness;
  • template consistency;
  • raw H/V coordinates;
  • local scatter / noise;
  • unexpected raw jumps;
  • missing measurements;
  • repeated rejection;
  • distance / mm-per-pixel;
  • image location;
  • prism/reflector brightness relative to distance;
  • obstruction by other prisms or tunnel geometry.

Point health must separate at least:

A. Can the image feature be measured?

Image / tracking quality.

B. Is the raw measurement temporally plausible?

Raw coordinate behaviour before reference correction.

C. Is the corrected measurement trustworthy?

Raw point uncertainty plus reference-transform uncertainty.

For target (i):

corrected quality
=
target measurement quality
+
reference compensation quality propagated to target i

Conceptually:

[ \Sigma_{corrected,i}

\Sigma_{measurement,i} + J_i \Sigma_T J_i^T ]

Therefore, two targets in the same image can have different corrected-measurement quality even when they share one global transform.


5.6 Reference observation health

Each reference is also an image observation.

Keep separate:

Image observability

  • Error Value;
  • flux;
  • saturation;
  • SNR;
  • sharpness;
  • visibility;
  • template consistency.

Geometric consistency

  • residual;
  • normalized residual;
  • robust weight;
  • leverage;
  • influence;
  • leave-one-reference-out effect.

A poor Error Value is not the same failure as a geometrically inconsistent but visually well-tracked reference.


5.7 Reference-network health

Use the existing roadmap directly.

Important quantities include:

  • configured reference count;
  • eligible reference count;
  • effective reference count;
  • per-reference contributions;
  • weighted horizontal / vertical span;
  • convex-hull coverage;
  • weighted coordinate covariance;
  • collinearity;
  • condition number;
  • transform covariance;
  • residual scale / sigma0;
  • weight concentration;
  • temporal weight-state stability;
  • leverage / influence;
  • leave-one-reference-out sensitivity;
  • depth distribution;
  • target-specific propagated uncertainty.

Avoid conclusions based solely on:

  • number of references;
  • RANSAC inlier count;
  • global sigma0.

5.8 Camera / compensation state

The agent should be able to relate point behaviour to camera/reference compensation.

Relevant signals:

  • translation;
  • rotation;
  • scale;
  • parameter covariance;
  • raw camera-state changes;
  • abrupt transform jumps;
  • persistent drift;
  • coincidence between transform changes and common-mode point motion;
  • changes in reference contribution state.

Useful support question:

Did the monitored object move, or did the compensation state change?

Evidence for a compensation event may include:

common-mode corrected-target jump
+ reference contribution change
+ transform jump
+ high reversibility

Evidence for probable object motion may include:

target displacement
+ stable reference contributions
+ stable transform
+ good target image quality
+ persistence

These remain deterministic evidence patterns, not semantic guesses.


5.9 Model adequacy / parallax

A statistically clean similarity transform can still be physically inadequate after significant 3D camera translation.

Potential evidence:

  • large camera translation;
  • residuals correlated with point depth;
  • residuals correlated with image position/radius;
  • target displacement correlated with target depth;
  • abnormal scale/rotation;
  • richer diagnostic model explains residual structure substantially better.

At first, model mismatch should be a diagnostic hypothesis / explainer feature, not an automatic rejection.

The support agent should be able to say:

The evidence is consistent with model mismatch / parallax, but this is not proven from the available data.


5.10 Datum / installation stability

A complete reference datum can move coherently and remain internally consistent.

The camera cannot prove physical stability of the entire datum using internal residuals alone.

This must be treated as an observability limitation.

Relevant external context can include:

  • known construction work;
  • station/support movement;
  • mounting changes;
  • geological motion;
  • GNSS or other external sensor observations;
  • manual survey;
  • site intervention.

The agent must not overclaim:

“References are internally consistent” is not equivalent to “references are physically fixed.”


5.11 Data provenance and reprocessing

Support conclusions depend strongly on provenance.

The agent should determine, where possible:

  • original online processing vs reprocessing;
  • configuration version;
  • pattern/reference configuration;
  • reset/reinitialization events;
  • whether motion compensation was applied;
  • source CSV format;
  • source station/backend;
  • staging vs production;
  • original acquisition time;
  • processing time;
  • export time;
  • timezone interpretation.

A corrected value without provenance can be misleading during incident analysis.


6. POC data-access model

6.1 Deliberate limitation

For the POC, there is no requirement for automatic station querying.

Evidence will be supplied manually by the support engineer/user.

Typical workflow:

support question
     |
     v
agent identifies minimum useful evidence
     |
     v
user exports / pastes / uploads CSV or screenshots
     |
     v
agent parses and normalizes evidence
     |
     v
deterministic computations / comparisons
     |
     v
diagnostic synthesis
     |
     v
next evidence request OR support conclusion

The POC should therefore make the evidence request precise and economical.

Bad:

Please send all logs.

Better:

Please export the measurement CSV from 00:30–09:30 on 19 Sep, including Raw H/V and Relative H/V for all targets and references, and send a screenshot of the reference status around the jump.


6.2 Expected manual inputs

CSV

Potential columns, depending on current export format:

  • timestamp;
  • point/pattern name;
  • role if available;
  • Relative H / V [mm];
  • Raw H / V [px];
  • validity;
  • Error Value;
  • distance;
  • image quality fields;
  • reference state;
  • sigma0;
  • transform parameters;
  • future residual / weight / influence / covariance fields.

The agent must tolerate different historical CSV schemas.

It should first identify:

available columns
missing expected columns
timestamp format
timezone
sampling cadence
point names
units
data source / export type if known

It should not silently assume a schema.

Interface captures

Useful examples:

  • station overview;
  • live plot;
  • point table;
  • reference state;
  • configuration;
  • current image;
  • image with pattern overlays;
  • system/network status;
  • export status;
  • error pages;
  • timestamps visible in UI.

The capture itself is evidence and should be associated with:

  • station/site;
  • approximate timestamp;
  • interface/environment;
  • screenshot creation time if known;
  • whether it represents live or historical state.

Camera / scene images

Can be used to inspect:

  • point visibility;
  • obstruction;
  • saturation;
  • illumination;
  • condensation/dirt if visible;
  • pattern placement;
  • major camera viewpoint changes;
  • possible scene changes.

Image interpretation remains qualitative unless deterministic image metrics are also provided.

Manual incident context

Examples:

  • construction activity;
  • known power interruption;
  • local network outage;
  • station physically touched;
  • camera mount adjustment;
  • pattern reset;
  • pattern configuration changed;
  • reference excluded;
  • software update;
  • server migration;
  • reprocessing performed.

This context should be stored as evidence with source operator_note, not mixed into measured facts.


7. Suggested POC evidence bundle

For repeatable investigations, organize each case approximately as:

case_name/
    context.md
    measurements.csv
    diagnostics.csv              # optional
    configuration.json           # optional
    notes.md
    screenshots/
        ...
    images/
        ...
    logs/
        ...                      # optional at POC

Example context.md:

site: Barcelona tunnel
station: <station-id>
question: Why did D-300-A jump?
period_start: 2026-09-19T00:00:00+02:00
period_end: 2026-09-19T12:00:00+02:00
known_actions:
  - local pattern reset later performed
files:
  - measurements.csv
  - screenshots/reference_state.png

This need not be mandatory for ad-hoc use, but it is useful for regression datasets.


8. Normalized evidence model

The agent should normalize heterogeneous manual inputs into a generic evidence structure.

Example:

{
  "case": {
    "site": "Barcelona",
    "station_id": null,
    "time_window": {
      "start": "2026-09-19T00:00:00+02:00",
      "end": "2026-09-19T12:00:00+02:00"
    },
    "question": "Why did D-300-A jump?"
  },
  "sources": [
    {
      "source_id": "csv_01",
      "type": "measurement_csv",
      "provenance": {
        "original_or_reprocessed": "unknown",
        "export_path": "SFTP",
        "timezone": "unknown"
      }
    },
    {
      "source_id": "ui_01",
      "type": "interface_capture",
      "timestamp": "2026-09-19T08:10:00+02:00"
    }
  ],
  "observations": {},
  "derived_features": {},
  "diagnostic_assessment": {}
}

The normalization layer matters because historical AIMsight data and future APIs will not necessarily expose the same fields.


9. Health-state representation

Do not collapse everything into one global Boolean healthy.

Each domain should expose:

  1. raw metrics / evidence;
  2. a derived state if a deterministic rule exists;
  3. evidence completeness;
  4. diagnostic notes.

Suggested operational state vocabulary:

HEALTHY
DEGRADED
FAILED
UNKNOWN
NOT_OBSERVED
NOT_APPLICABLE

Suggested measurement/reference states can retain the roadmap terminology:

VALID
DEGRADED
INVALID
MODEL_MISMATCH_SUSPECTED
DATUM_STABILITY_UNVERIFIED

These states should remain domain-specific.

Example:

{
  "station": {
    "state": "UNKNOWN",
    "reason": "No system status or local acquisition evidence supplied."
  },
  "transport": {
    "state": "DEGRADED",
    "reason": "15-minute export gap while local acquisition evidence continues."
  },
  "target:D-300-A": {
    "observation_state": "HEALTHY",
    "corrected_measurement_state": "DEGRADED"
  },
  "reference_network": {
    "state": "DEGRADED"
  }
}

This avoids contradictions such as “the station is bad” when only one point is poorly tracked.


10. Evidence completeness and epistemic rules

The support agent must explicitly distinguish:

  • observed fact;
  • deterministic derived feature;
  • known operator context;
  • hypothesis;
  • unresolved question.

Recommended confidence wording:

confirmed by supplied evidence
strongly supported
consistent with
possible
insufficient evidence
not observed

Avoid pseudo-probabilities unless a calibrated model exists.

The agent must not claim:

  • station downtime from UI downtime alone;
  • data loss from a missing plot alone;
  • physical reference stability from internal geometric consistency alone;
  • structural movement from corrected displacement without considering reference/camera state;
  • camera motion from a transform change alone if the evidence is ambiguous;
  • network outage from an export gap without a network signal/log.

11. Deterministic analysis available at POC

Even with manual CSV input, the support workflow can compute useful deterministic features.

11.1 Time / cadence

  • median cadence;
  • expected vs observed intervals;
  • missing epochs;
  • longest gap;
  • duplicated timestamps;
  • non-monotonic timestamps;
  • time offset between related files;
  • timezone-like constant offsets;
  • freshness / last sample.

11.2 Point behaviour

  • data completeness per point;
  • local median / MAD;
  • robust z-score;
  • step amplitude;
  • trend / rate;
  • short-term scatter;
  • persistence;
  • reversibility;
  • correlation between Raw H/V and Relative H/V;
  • image quality changes coincident with displacement.

11.3 Spatial behaviour

  • fraction of targets moving;
  • sign agreement;
  • common-mode component;
  • pairwise correlation;
  • clustering of affected points;
  • correlation with image position or distance.

11.4 Reference behaviour

As fields become available:

  • per-reference residual;
  • normalized residual;
  • robust weight;
  • effective reference count;
  • weight concentration;
  • reference-state changes;
  • geometry metrics;
  • sigma0 / robust scale;
  • transform change;
  • transform covariance;
  • influence / leave-one-out sensitivity.

11.5 Before/after comparison

Compare:

  • known-good baseline;
  • incident window;
  • post-intervention / reprocessed window.

This is particularly useful for regression cases.


12. Diagnostic hypothesis families

The agent should organize evidence around a limited set of support hypotheses.

12.1 Target-local observation degradation

Typical evidence:

one/few targets affected
+ Error Value / brightness / SNR / visibility degradation
+ healthy reference solution
+ no matching common-mode response

Potential causes to investigate:

  • occlusion;
  • obstruction by another target;
  • low or excessive reflected intensity;
  • illumination issue;
  • poor template;
  • scene change;
  • target physically altered.

12.2 Reference-compensation instability

Typical evidence:

common-mode corrected-target jump
+ reference contribution change
+ transform jump
+ possible reversibility

Barcelona is the primary regression case.


12.3 Weak reference-network geometry

Evidence can include:

  • low effective reference count;
  • concentrated weights;
  • poor span / collinearity;
  • high condition number;
  • high transform covariance;
  • high target-specific propagated uncertainty;
  • high influence of one reference.

This can occur even when residuals are small.


12.4 Camera/station movement

Possible evidence:

  • strong transform change;
  • many raw point coordinates move coherently;
  • reference diagnostics change;
  • target response common-mode;
  • known physical/site event.

A camera movement may still be compensated successfully. The agent should distinguish:

camera moved but compensation remained valid

from:

camera moved and compensation became unstable

and from:

camera moved enough for similarity-model mismatch / parallax

12.5 Model mismatch / parallax

Possible evidence:

  • large camera movement;
  • depth-dependent residuals;
  • target displacement related to target depth;
  • structured residual field;
  • similarity transform statistically insufficient.

Initially diagnostic only.


12.6 Acquisition failure

Evidence:

  • no new local images;
  • no local measurement epochs;
  • acquisition service failure;
  • simultaneous local and remote data stop.

Must be distinguished from transport failure.


12.7 Transport / export failure

Evidence:

  • local acquisition continues;
  • remote/export data stop or lag;
  • network/modem problem;
  • upload queue/backlog;
  • data later appear after recovery.

Important future diagnostic: whether data are actually recoverable and re-uploaded after outages.


12.8 Backend / UI failure

Evidence:

  • source data/export exists;
  • web page, plot or resource fails;
  • 404 or rendering issue;
  • staging/production discrepancy.

12.9 Timestamp / timezone issue

Evidence:

  • otherwise regular data consistently offset by a fixed amount;
  • UI timestamps differ from CSV/acquisition times;
  • future timestamps;
  • mismatch close to timezone/DST offset.

Do not modify timestamps silently. Identify the assumed time basis.


12.10 Coherent datum motion

Internal reference consistency may remain good.

Requires external/contextual evidence.

The correct diagnosis may remain:

reference datum internally consistent;
absolute datum stability cannot be verified from AIMsight image data alone.

13. Support query workflow

For each question:

Step 1 — Parse the question

Identify:

  • site / station;
  • point(s);
  • time window;
  • symptom;
  • expected behaviour;
  • comparison period if given.

Step 2 — Map symptom to diagnostic layers

Example:

“Point D-300-A jumped at 08:00.”

Relevant layers:

  • target observation quality;
  • Raw H/V;
  • corrected H/V;
  • other targets;
  • reference network;
  • transform;
  • acquisition timestamps;
  • intervention context.

Step 3 — Inventory available evidence

Report:

  • what is available;
  • what is missing;
  • which missing evidence materially limits diagnosis.

Step 4 — Run deterministic checks

Prefer calculations over semantic interpretation where possible.

Step 5 — Build candidate explanations

Rank only by supplied evidence.

Step 6 — Try to falsify the leading explanation

Example:

If “target tracking problem” is suspected:

  • did Raw H/V jump?
  • did Error Value degrade?
  • did neighboring/other targets jump?
  • did the transform change?
  • did the same target recover after reprocessing?

Step 7 — Produce support diagnosis

Include:

  • affected layer;
  • leading diagnosis;
  • evidence;
  • alternative explanations;
  • unknowns;
  • exact next check.

Step 8 — Preserve case for regression

Once resolved, store:

  • evidence bundle;
  • final expert diagnosis;
  • intervention;
  • expected agent result.

14. Recommended support-agent response format

Use a compact but traceable format:

Assessment
----------
Most likely issue: reference-compensation instability.
Measurement availability: data present.
Target-local tracking: no clear degradation.
Reference network: unstable around event.
System / communication: no evidence of failure in supplied files.

Evidence
--------
- D-300-A and multiple targets jump together.
- Raw target positions do not show an equivalent local-only tracking failure.
- Reference contribution/state changes at the same epoch.
- Similarity transform changes at the same epoch.
- Event is reversible.

What is not established
-----------------------
- Physical stability of the complete reference datum.
- Whether a network interruption also occurred; no network log supplied.

Next check
----------
Provide the reference residual/weight table for 07:30–08:30 if available.

For machine use, the same result can be represented as structured JSON.


15. Suggested structured output

{
  "query": {
    "site": "Barcelona",
    "station": null,
    "targets": ["D-300-A"],
    "time_window": {},
    "question": "Why did the point jump?"
  },
  "evidence_completeness": {
    "measurement_csv": true,
    "raw_coordinates": true,
    "target_quality": true,
    "reference_diagnostics": true,
    "system_status": false,
    "transport_logs": false
  },
  "domain_health": {
    "system": {"state": "UNKNOWN"},
    "acquisition": {"state": "HEALTHY"},
    "transport": {"state": "UNKNOWN"},
    "backend_ui": {"state": "NOT_OBSERVED"},
    "target_observation": {"state": "HEALTHY"},
    "reference_network": {"state": "DEGRADED"},
    "model_adequacy": {"state": "NOT_OBSERVED"}
  },
  "diagnosis": {
    "primary": "REFERENCE_COMPENSATION_INSTABILITY",
    "support": "strongly_supported",
    "alternatives": [
      {
        "type": "TARGET_LOCAL_DEGRADATION",
        "support": "weak"
      }
    ]
  },
  "evidence": [],
  "unknowns": [],
  "next_checks": []
}

Do not treat this schema as fixed. It should evolve with the POC.


16. Relevant previous AIMsight support contexts

16.1 Barcelona tunnel — core regression case

Barcelona remains the most important initial incident dataset.

Known context includes:

  • 1-minute SFTP CSV export;
  • images at a lower cadence (15 minutes in the incident workflow);
  • a quiet baseline around 12 Sep;
  • station/camera movement during the 19 Sep incident;
  • correlated movement later in the morning;
  • reference instability involving PR_284 / PR_291;
  • repeated switching between plausible compensation states;
  • monitored targets remaining individually measurable while compensation changed;
  • local pattern reset returning values to zero;
  • later pattern reconfiguration stabilizing the system;
  • discussion of excluding PR_284 as a fallback;
  • loss of references on the left in a later episode;
  • proposal for an additional reference;
  • questions about dedicated FTP export;
  • need to separate target tracking, reference quality, camera motion and export/acquisition behaviour.

Expected support-agent value:

  • diagnose a target jump without assuming target failure;
  • identify common-mode behaviour;
  • identify reference-state changes;
  • distinguish station/camera movement from structural motion;
  • distinguish original from reprocessed data;
  • explain missing data around network/export events;
  • request the minimum extra evidence needed.

The Barcelona incident should remain a regression suite, not a special-case rule.


16.2 Barcelona network-loss / MQTT / export behaviour

A separate but related support thread highlighted disappointing behaviour during loss of network connectivity:

  • holes in exported data;
  • holes in images;
  • suspicion around the newer MQTT chunk implementation;
  • questions about whether FTP data remained locally recoverable;
  • questions about missing re-upload/recovery logic;
  • undesirable FIFO behaviour;
  • desire to unify semantics across SFTP/FTP, internal MQTT and future external MQTT.

This led to the broader concept of full agentic observability.

Desired future observability includes:

  • application/system logs;
  • modem/network logs;
  • acquisition state;
  • local stored files;
  • upload queue/backlog;
  • last successful transport;
  • requested on-demand diagnostics;
  • local USB/Ethernet access for diagnosing a station when remote communication itself is unavailable.

This thread is the main context for the system-health and data-pipeline side of the support agent.


16.3 Staging / web access + FTP failure

A reported situation combined:

  • inability to access staging.aimsight.ch;
  • FTP export not working.

This is a useful diagnostic case because the two symptoms may have:

  • one common backend/network cause;
  • independent causes;
  • a station-side transport issue plus a separate UI issue.

The agent should not infer a common cause without evidence.

Minimum useful checks include:

  • local acquisition freshness;
  • last successful upload;
  • FTP/SFTP status;
  • backend ingestion freshness;
  • direct export availability;
  • UI/API reachability.

16.4 Lenzspitze

Relevant symptoms included:

  • live plot error / 404;
  • timestamps approximately two hours in the future;
  • suspected timezone handling issue.

This is an important case because the measurement system may be producing data while the user-visible presentation is wrong.

Agent checks should include:

  • actual CSV timestamps;
  • station timestamp;
  • timezone / UTC interpretation;
  • backend timestamps;
  • HTTP/UI resource failure;
  • data freshness independent of plot rendering.

16.5 Monitoring / staging migration and CSV formats

AIMsight currently has or has had multiple paths/environments including:

  • monitoring.aimsight.ch;
  • staging.aimsight.ch;
  • CSV outputs with differing formats;
  • plans for more direct export from stations.

For the agent this means:

  • source provenance is mandatory;
  • CSV parsers should be schema-tolerant;
  • identical-looking data from different sources may have different processing histories;
  • a support conclusion should state which data path was analyzed.

16.6 Data Exploration / service modes

Relevant product context includes:

  • Relative H/V [mm];
  • Raw H/V [px];
  • image quality / visibility / reliability signals;
  • HDF5 as a more advanced source;
  • simplified CSV export;
  • optional reprocessing from stored images;
  • local standalone operation;
  • remote access through VPN/Nebula;
  • cloud service mode.

These are natural support evidence sources.

The support agent should exploit the distinction between:

raw measurement
corrected measurement
diagnostic quality
processing provenance

rather than treating the exported displacement column as the only signal.


16.7 Installation / target visibility cases

Prior installation discussions provide useful point-health context:

  • all prisms must remain visible;
  • eye-level installation can be acceptable;
  • vertical offsets can reduce prism-to-prism obstruction;
  • larger prisms become useful at long range;
  • short-range prisms can be unnecessarily bright;
  • tunnel lighting conditions change reflector requirements;
  • live view can be checked remotely or by direct Ethernet connection.

These are not estimator problems. They are installation/observation-quality problems and belong in the support knowledge base.


16.8 Crissier and future cases

Crissier is expected to become another regression/test case.

Do not invent its diagnosis before the data/context are added.

Each new resolved case should add:

  • symptom;
  • evidence available;
  • expert diagnosis;
  • intervention;
  • deterministic features that should have detected/explained the issue;
  • evidence that would have shortened the investigation;
  • expected support-agent answer.

17. Regression-case philosophy

Historical cases should be used like software tests.

For each case define:

input evidence
support question
expected deterministic observations
expected diagnostic classification
important uncertainty / observability limitation
expected next-check recommendation

The regression suite should test not only correct diagnoses but also correct refusal to overclaim.

Examples:

Barcelona reference switching

Agent should detect:

  • common-mode target response;
  • reference-state/contribution changes;
  • transform change;
  • reversibility.

It should not diagnose “bad target D-300-A”.

UI 404

Agent should not diagnose station failure if source/export evidence shows current data.

Export gap with local acquisition

Agent should classify a pipeline/transport issue rather than NO_ACQUISITION.

Coherent reference movement

Agent should flag datum stability as unverified rather than claiming a healthy absolute datum.


18. POC implementation architecture

Keep the POC lightweight.

manual evidence
CSV / screenshots / notes
        |
        v
input adapters
        |
        v
normalized evidence model
        |
        +-------------------------+
        |                         |
        v                         v
deterministic analyzers      visual/manual evidence extraction
        |                         |
        +-------------+-----------+
                      |
                      v
               diagnostic facts
                      |
                      v
             support-agent reasoning
                      |
          +-----------+------------+
          |                        |
          v                        v
support explanation        request next evidence

Suggested components:

A. Input adapters

  • flexible CSV loader;
  • screenshot/image input;
  • manual context parser;
  • optional log parser.

B. Deterministic analyzers

  • cadence/gap analyzer;
  • point-statistics analyzer;
  • common-mode/spatial analyzer;
  • reference-network diagnostics adapter;
  • transform-change analyzer;
  • provenance/timestamp checker.

C. Diagnostic fact store

Structured observations such as:

{
  "fact": "COMMON_MODE_STEP",
  "start": "...",
  "magnitude": "...",
  "affected_fraction": 0.86,
  "source": ["csv_01"],
  "method": "deterministic_v1"
}

D. Support reasoning layer

The LLM/agent:

  • interprets the user question;
  • selects relevant diagnostic facts;
  • requests missing evidence;
  • compares hypotheses;
  • explains results in support language;
  • links remediation/checks to evidence.

It should not silently replace deterministic validity rules.


19. Manual evidence-request strategy

The agent should minimize support friction.

Ask for the next most discriminating piece of evidence.

Example decision sequence for:

“Why is this point jumping?”

  1. Obtain CSV with Raw and Relative H/V for affected and neighboring points.
  2. If only corrected values jump, inspect reference/transform state.
  3. If raw value of one point jumps alone, inspect target image quality/capture.
  4. If many raw values jump together, inspect camera state / physical station movement.
  5. If local data are missing, distinguish acquisition from export using local image/measurement timestamps.
  6. If only web display is missing, inspect export/API/backend separately.

This should eventually become a formal evidence-request policy.


20. Support actions and remediation

The POC is primarily diagnostic/read-only.

Potential recommendations can include:

  • inspect current image;
  • inspect a specific point/pattern;
  • export a narrower/broader CSV period;
  • compare Raw and Relative coordinates;
  • check known-good baseline;
  • check reference residuals/weights;
  • reprocess stored images;
  • verify timezone configuration;
  • verify transport backlog;
  • connect locally through Ethernet;
  • check VPN/modem;
  • reconfigure a pattern/reference;
  • consider reference exclusion/downweighting;
  • inspect camera mounting;
  • add/improve a reference.

However:

  • do not automatically modify configuration;
  • do not automatically exclude references;
  • do not trigger reprocessing or station restart at POC;
  • do not treat a support recommendation as a deterministic validity rule.

Future action-capable agents should require explicit authorization for consequential changes.


21. Interaction with automatic exclusion / alarms

The support agent and production quality logic serve different purposes.

Production quality logic should be:

  • deterministic;
  • transparent;
  • estimator-independent;
  • validated on regression datasets;
  • configured by project policy.

The support agent can be more exploratory:

  • correlate multiple evidence sources;
  • compare periods;
  • identify likely causes;
  • request extra data;
  • explain a deterministic alarm;
  • explain why no alarm fired;
  • identify missing observability.

A useful development loop is:

support incident
    ->
agent investigation
    ->
recurrent deterministic diagnostic discovered
    ->
add feature to event explainer / shadow alarms
    ->
validate across cases
    ->
possibly promote to production quality logic

Thus support work is also a discovery mechanism for missing production diagnostics.


22. Observability gaps the POC should expose

One objective of the POC is to determine which telemetry is worth implementing.

Likely high-value future signals include:

Station

  • uptime / last reboot;
  • application version;
  • service state;
  • storage remaining;
  • last image acquisition;
  • last successful processing epoch;
  • camera connectivity;
  • station clock/timezone.

Communication

  • modem state;
  • VPN state;
  • last successful remote contact;
  • last successful upload per channel;
  • queued items / bytes;
  • oldest queued item;
  • retry count;
  • transport error code.

Data pipeline

  • locally acquired epochs;
  • locally stored images;
  • locally processed epochs;
  • exported epochs;
  • backend-ingested epochs;
  • current backlog;
  • re-upload/recovery state.

Point

  • Error Value;
  • visibility;
  • brightness / flux;
  • SNR;
  • sharpness;
  • saturation;
  • raw coordinate;
  • corrected coordinate;
  • local measurement uncertainty;
  • validity/rejection reason.

Reference / transform

  • residual;
  • normalized residual;
  • robust weight;
  • effective weight;
  • leverage;
  • influence;
  • effective reference count;
  • weighted geometry;
  • transform;
  • covariance;
  • target-specific propagated uncertainty;
  • model-mismatch indicators.

The POC should record every time an investigation is blocked because one of these signals is unavailable.


23. Priority POC use cases

Recommended initial set:

UC1 — Why is one point bad?

Focus:

  • target image/tracking quality;
  • raw vs corrected coordinates;
  • local vs global behaviour.

UC2 — Why did many points jump?

Focus:

  • common-mode response;
  • transform;
  • reference contribution state;
  • camera movement.

UC3 — Why is there no recent data?

Focus:

  • acquisition vs local processing vs transport vs backend/UI.

UC4 — Is a station healthy?

Return domain-by-domain health, not one Boolean.

UC5 — What changed around an event?

Before/after comparison.

UC6 — Can this corrected displacement be trusted?

Focus:

  • target-local quality;
  • reference-network quality;
  • transform uncertainty;
  • target-specific propagated uncertainty;
  • provenance.

UC7 — Why do CSV and web UI disagree?

Focus:

  • source;
  • timestamps;
  • backend/export path;
  • processing/reprocessing provenance.

UC8 — What evidence should support request next?

Optimize the next diagnostic action.


24. Definition of POC success

The POC succeeds if, on a small set of real support cases, it can:

  1. correctly identify the affected diagnostic layer(s);
  2. avoid conflating point, reference, system and pipeline problems;
  3. compute useful features from manually supplied CSVs;
  4. use screenshots/context as supporting evidence;
  5. state what is known vs unknown;
  6. recommend the next discriminating check;
  7. reproduce expert diagnosis for resolved cases such as Barcelona;
  8. avoid overclaiming on incomplete cases;
  9. reveal missing telemetry worth adding;
  10. preserve compatibility with the evolving reference estimator.

The POC does not need to:

  • autonomously connect to stations;
  • autonomously access the backend;
  • automatically repair systems;
  • automatically change reference configuration;
  • replace deterministic validity logic;
  • solve every historical support case.

25. Near-term development sequence

Phase S0 — Context + regression cases

  • establish this context document;
  • collect Barcelona evidence;
  • add Crissier when available;
  • add Lenzspitze/UI-time case;
  • add at least one data-export/network case;
  • store expert diagnoses.

Phase S1 — Manual CSV diagnostic notebook/tool

Implement:

  • schema inspection;
  • timestamp normalization;
  • cadence/gaps;
  • per-point completeness;
  • robust scatter / steps;
  • Raw vs Relative comparison;
  • common-mode correlation;
  • before/after plots/tables;
  • structured diagnostic output.

Phase S2 — Screenshot / manual context ingestion

Support:

  • interface captures;
  • current camera images;
  • manually supplied configuration/status;
  • evidence linkage to time windows.

Phase S3 — Reference diagnostics adapter

Consume the generic estimator-independent outputs from the roadmap:

  • residual;
  • weight;
  • effective count;
  • geometry;
  • covariance;
  • influence;
  • transform.

Initially, the current RANSAC implementation can populate binary contribution weights.

Phase S4 — Support reasoning / evidence request

Given a user question:

  • determine likely diagnostic layers;
  • run available analyzers;
  • identify missing discriminating evidence;
  • provide support diagnosis.

Phase S5 — Observability specification

Use POC investigations to define the future station/backend diagnostic API.

Only after this should live automated querying become a priority.


26. Longer-term query/API direction

The manual POC should anticipate future calls such as:

get_station_health(station, time)
get_acquisition_status(station, window)
get_transport_status(station, window)
get_export_status(station, channel, window)
get_point_timeseries(station, points, window, fields)
get_point_quality(station, point, window)
get_reference_fit(station, window)
get_reference_network_quality(station, window)
get_transform_timeseries(station, window)
get_images(station, window)
get_logs(station, subsystem, window)
compare_periods(station, baseline, incident)
reprocess_preview(station, window, configuration)   # future, controlled

The support agent should depend on semantic capabilities like these rather than on a specific current backend implementation.


27. Important non-goals / anti-patterns

Do not:

  • build support logic around RANSAC inlier/outlier membership;
  • use a single station-level healthy flag as the primary representation;
  • use sigma0 as the only reference-quality metric;
  • assume “missing from UI” means “not acquired”;
  • assume a bad corrected target means the target itself is badly tracked;
  • assume a common-mode displacement is structural;
  • assume a network outage necessarily causes data loss;
  • assume internally consistent references are physically stable;
  • let the LLM invent deterministic validity thresholds;
  • hard-code Barcelona-specific rules;
  • create automatic remediation before diagnosis is reliable;
  • silently merge original and reprocessed data.

28. Guiding design principles

  1. Trace the full dependency chain.
  2. Keep operational health and measurement integrity separate.
  3. Keep point observation quality and corrected-point quality separate.
  4. Reuse the common reference/geometry diagnostic layer.
  5. Remain estimator-independent.
  6. Prefer continuous quality metrics over binary states.
  7. Use deterministic computations wherever practical.
  8. Use the agent for evidence orchestration, hypothesis comparison and explanation.
  9. Make provenance first-class.
  10. Represent unknowns explicitly.
  11. Request the next most discriminating evidence, not everything.
  12. Turn resolved support incidents into regression tests.
  13. Use blocked diagnoses to define future observability.
  14. Do not confuse data transport, backend and UI failures with acquisition failures.
  15. Preserve target-specific uncertainty and reference-network effects.

29. Working mental model

A concise mental model for future development is:

                  WHAT DOES THE USER SEE?
                           |
                           v
                   UI / export evidence
                           |
                           v
               DID THE DATA REACH HERE?
                           |
              +------------+-------------+
              |                          |
             no                         yes
              |                          |
     trace pipeline backward      IS THE VALUE PLAUSIBLE?
              |                          |
              v                          v
 transport -> local data       target raw measurement
              |                          |
              v                          v
        acquisition?          target image quality
                                         |
                                         v
                              reference compensation
                                         |
                                         v
                                 camera / model
                                         |
                                         v
                              physical interpretation

The support agent's job is not merely to answer questions. It should make this chain inspectable, repeatable and increasingly deterministic.


30. Context instruction for future agents/developers

When working on this project:

  • use AIMsight_reference_quality_event_explainer_roadmap.md as the authoritative baseline for the reference-network, event-explainer and planning architecture;
  • continue from that roadmap rather than re-deriving it;
  • treat the support agent as an extension that adds system, pipeline and point-level diagnosis;
  • at POC stage assume evidence is manually supplied as CSVs, screenshots/images and operator context;
  • do not prematurely design live data connectors;
  • keep the architecture ready for future automated queries;
  • do not introduce dependencies on RANSAC binary inlier/outlier logic;
  • prefer residuals, continuous weights, influence/leverage, effective reference count, transform covariance and target-specific propagated uncertainty;
  • use Barcelona as one important regression case;
  • add Crissier and other real support cases as they become available;
  • distinguish deterministic evidence from LLM synthesis;
  • make uncertainty, provenance and observability limitations explicit;
  • use support investigations to identify missing telemetry and candidate deterministic alarms.

The long-term objective is a support system able to move from:

“Something looks wrong.”

to:

“The station is acquiring normally; point observations remain measurable; the numerical export became stale after 11:42 while local acquisition continued; the evidence therefore localizes the issue to the transport/export path. Reference compensation was stable in the supplied interval. The next most useful check is the upload queue / last successful FTP or MQTT transaction.”

That level of layered, evidence-based diagnosis is the target.