29 lines
4.0 KiB
Markdown
29 lines
4.0 KiB
Markdown
# labeling
|
|
|
|
The AIMsight displacement labels and all the domain knowledge to place them, in a single copy on the workstation.
|
|
|
|
- `manual/<STATION>.jsonl`: the expert's labels, for him alone: no process and no agent uses them; `auto/<STATION>.jsonl`: those of the Support-Agent `etiqueteur` and those submitted by the aimsight-app `analyst`; `review` is the date the user validated a label in aimsight-app (quality over quantity: only the analyses discussed and validated there), and any change removes it; the agents' reviews of 2026-11-04 and 2026-11-05 were removed on 2026-11-17 (in the history); one label per line, `{"id", "start", "end", "family", "description", ...}`, UTC times, `family` = the family.
|
|
- `auto/analyses/<STATION>_<start>_<end>/`: the analyses launched from the Data page of aimsight-app (`work/` outside git).
|
|
- `auto/support/<YYYY-MM-DD>_<subject>/`: the folders of the `support` agent (agents repository): the request, the answer, the analyst's work when it was called (`work/` outside git).
|
|
- `auto/cases/<id>.json`: the cases, the phenomena, false friends and lessons drawn from the labels, one JSON file each (`docs/superpowers/specs/2026-11-18-cases-design.md`); built and refreshed by the `cases` skill (`/cases`, `.claude/skills/cases/`), which never writes `review` nor commits; `review` is the date the user validated a case, written by the user in its JSON, removed by the skill when it changes the case's text; `auto/cases/.read.json`, the skill's state (which case each label went to).
|
|
- The knowledge, one source per kind, maintained by hand by the user, but for the cases; the analyst reads the four files in full and searches the cases:
|
|
|
|
| Kind of knowledge | Source |
|
|
|---|---|
|
|
| The cases: phenomena, false friends, lessons | `auto/cases/`, through `cases.py` |
|
|
| The labels themselves, as precedents | `auto/<STATION>.jsonl`, through `precedents.py` |
|
|
| The families and their boundaries | `families.md` |
|
|
| How to place a label | `labeling-rules.md` |
|
|
| What the data mean: measurement chain, gaps, reading rules | `reading-rules.md` |
|
|
| The installation of each station | `stations.md` |
|
|
| What the data are: files, columns, image quality, images | `tools/DATA.md` (tools repository) |
|
|
- `scripts/`: `validate.py` (the labels and the cases against the rules; `check` for one label), `ids.py` (the stable id of each), `client.py` (what the client is told of a detectors' event).
|
|
- `scripts/cases.py`: the RAG of the cases. The cases closest in meaning to a text, each in full with its provenance (validated by the user or not) and its label sources dated; `--list` every case, one line each, without Ollama; `--pending` and `--record`, the bookkeeping of a refresh by the skill. bge-m3 model of the Ollama container `ollama-bge-m3` (started by the script itself if it does not answer; port 11435); the `.index/` index, outside git, updates itself at every call (only new or modified cases go through the model). The labels also feed the cases.
|
|
- `scripts/precedents.py`: the RAG of the labels. The labels of `auto/` closest in meaning to a text, with their provenance (validated by the user or not); `--exclude STATION START END` leaves out the analyzed range; its own index in `.index/`. An agent uses it, the case search, or both.
|
|
|
|
aimsight-app and Optimizer reach it through a link `labeling -> ~/aimsight-labeling`, ignored by git: aimsight-app reads and writes there (labels, analyses), Optimizer only reads; commits are made here. The scripts run from here (`python3 scripts/validate.py`) or through the link (`python3 labeling/scripts/validate.py`). Descriptions are written in English from now on.
|
|
|
|
History taken from Support-Agent (the `labeling/` folder, up to commit 48a3092), then the state of aimsight-app on 2026-11-13 (462ba9e): the archives of the etiqueteur and of the reviews (`auto/etiqueteur/`, `auto/revue/`, `auto/revue-croisee/`) and the old `labeling.py` tool remain in the history.
|
|
|
|
`docs/archive/`: the design and plan of the labeler (not built), the label map and `knowledge/`, archived on 2026-11-16.
|