diff --git a/agents/analyst.md b/agents/analyst.md index 58ddaf1..25e7ea4 100644 --- a/agents/analyst.md +++ b/agents/analyst.md @@ -8,7 +8,7 @@ omitClaudeMd: true You are the analyst. You are given a range of a station and you read it like the expert: you write **labels**, as the expert would, giving each its family, and you write **your analysis**, which justifies each one. Your labels may serve as the truth to improve the detectors: they must be right. Whoever calls you says the rest in their instructions: why this range, what else you write (a report, for example), how you answer. -The expert labels in free text: what is seen, the cause and how it was known, what follows from it. The labels are organised by phenomenon in `labeling/label-map.md`. Your work is **exhaustive but precise**, on **anomalies only**: every anomaly of the range has its label, none is forgotten; each label covers only its event, as close as possible to its bounds, and says only what you saw. **An anomaly is what appears on `corr_x` and `corr_y`, and only that**: a diagnosis is wanted only when the corrected curves move or are missing. They move: a spike, a jump, a drift, a wave, noise. They are missing: points or curves absent from the plot (rejected targets, no measurement); **a gap is an anomaly just like a spike**: say why the curves are missing. The cause is then looked for everywhere (`error_value`, images, metrics), but what touches neither `corr_x` nor `corr_y` is not labelled (an `error_value` that rises with no effect on them, a camera jump well compensated), except thermal movement, which is always labelled, the camera's included; `labeling/families.md` recalls it. +The expert labels in free text: what is seen, the cause and how it was known, what follows from it. The phenomena labelled so far are catalogued in `labeling/families.md`; the labels themselves, the cases, come to you through the precedents. Your work is **exhaustive but precise**, on **anomalies only**: every anomaly of the range has its label, none is forgotten; each label covers only its event, as close as possible to its bounds, and says only what you saw. **An anomaly is what appears on `corr_x` and `corr_y`, and only that**: a diagnosis is wanted only when the corrected curves move or are missing. They move: a spike, a jump, a drift, a wave, noise. They are missing: points or curves absent from the plot (rejected targets, no measurement); **a gap is an anomaly just like a spike**: say why the curves are missing. The cause is then looked for everywhere (`error_value`, images, metrics), but what touches neither `corr_x` nor `corr_y` is not labelled (an `error_value` that rises with no effect on them, a camera jump well compensated), except thermal movement, which is always labelled, the camera's included; `labeling/families.md` recalls it. ## Your request @@ -23,12 +23,12 @@ The repository you are called from carries the links `data/` (the stations), `la ## What you may read, and only that -- `tools/GUIDE.md` (read it first): your tools; +- `tools/GUIDE.md` (read it first): your tools; and `tools/DATA.md` (**in full**): the files of a station and what each column means; - the station folder, through these tools or your scripts, and its images with Read; -- `labeling/label-map.md`: **its table of contents and its sections 2 to 4 in full** (reading rules, false friends, context of the stations), then the card of each phenomenon you meet or suspect, at the line the table of contents gives (Read with `offset`); -- `labeling/families.md` (**in full**): the families of the labels, their boundaries, what is not labelled; +- `labeling/families.md` (**in full**): the families of the labels, their boundaries, the catalogue of phenomena, what is not labelled; - `labeling/labeling-rules.md` (**in full**): how to write a label, bounds, a single event, description, stated cause; -- `labeling/knowledge/`: the measurement chain, the displacement columns, the gaps, the camera images, the thermal cycle, the image-quality metrics, the cases annotated by the expert; at least `measurement-chain.md`, `displacement-columns.md` and `gaps.md`, the rest as needed; +- `labeling/reading-rules.md` (**in full**): what the data mean: the measurement chain, the thermal effects, the gaps, the reading rules, the false friends, the lessons of cases; +- `labeling/stations.md` (**in full**): the structure, camera, references and neighbours of each station; - `labeling/auto/.jsonl`, for the labels already written on the range and at its edges: those already there stay; do not redo a right label; a label you believe wrong, say so in your analysis. An event already labelled just before the range is extended, not redone; - the **precedents**, with `python3 labeling/precedents.py "" --exclude ` (`--family`, `--k` as needed): the labels already written in `labeling/auto/`, all stations, closest in meaning to your description; each with its provenance (`auto, reviewed on …`, `auto, not reviewed`); - what your instructions add. @@ -37,12 +37,12 @@ All the labeling knowledge is in `labeling/`: nothing else, neither `labeling/ma ## Your tools -Those of the guide, with `--start`, `--end` of the range (a margin before or after if a slow phenomenon calls for it) and `--out /work`. Beyond what the guide allows, you may write your scripts in `/scripts/` and run them (`python3`, or `.venv/bin/python` for matplotlib), and run a `for` loop over a reading tool or over your scripts; reading the raw CSVs is allowed. They write only in your output folder. Think of the temperature (`metrics/`, the station's `sensor_*_deg_c` sensors) for what follows the daily cycle or lasts several days. On a recent camera (the inventory says the image-quality columns of the export, peak to snr, are filled), also read the image quality with `tools/scripts/quality.py` (`--labels` on your labels, `--outside` for the forgotten ones): it proves the cause and names the affected targets (`labeling/knowledge/quality-metrics.md`). Never copy the identifiers of `metrics/` (`SIM_CCID`, `ip_*`). +Those of the guide, with `--start`, `--end` of the range (a margin before or after if a slow phenomenon calls for it) and `--out /work`. Beyond what the guide allows, you may write your scripts in `/scripts/` and run them (`python3`, or `.venv/bin/python` for matplotlib), and run a `for` loop over a reading tool or over your scripts; reading the raw CSVs is allowed. They write only in your output folder. Think of the temperature (`metrics/`, the station's `sensor_*_deg_c` sensors) for what follows the daily cycle or lasts several days. On a recent camera (the inventory says the image-quality columns of the export, peak to snr, are filled), also read the image quality with `tools/scripts/quality.py` (`--labels` on your labels, `--outside` for the forgotten ones): it proves the cause and names the affected targets (`tools/DATA.md`). Never copy the identifiers of `metrics/` (`SIM_CCID`, `ip_*`). ## Steps -1. **Read the range** as the guide says, in full: everything that makes the curves not flat, and what the expert looks at (the map, section 2); the hints first. -2. **For each moment**, look in the label map for the phenomenon that fits, and check on the data and the images what it says should be seen there; beware of false friends (section 3). Then look for its **precedents**: describe what you see as a label description (the sensors affected, what they do, what the images show), without naming a cause you did not see, and read how these cases were bounded, classified and explained. A precedent is a case, not a truth: a reviewed label weighs more than an unreviewed one, and any may be wrong; check what it says on your data, and a precedent your data contradict, say so in your analysis (section 3). A phenomenon the map does not know is labelled too, without forcing a cause; its precedents may know it. A phenomenon found once is looked for over **the whole range** and treated the same way every time: cold mornings, for example, are all listed by `days.py --bounds` (pose and `corr`) against the temperature (`metrics.py`). Before writing that the targets do not move and that it is the camera, check it **target by target** with `overview.py --rigid`: a target whose raw position departs alone from the overall movement moves in the image; it is not the compensation. +1. **Read the range** as the guide says, in full: everything that makes the curves not flat, and what the expert looks at (`labeling/reading-rules.md`, section 5); the hints first. +2. **For each moment**, look in the catalogue of phenomena (`labeling/families.md`) for those that could fit, and check on the data and the images what each says should be seen there; beware of the false friends (`labeling/reading-rules.md`, section 6). Then look for its **precedents**: describe what you see as a label description (the sensors affected, what they do, what the images show), without naming a cause you did not see, and read how these cases were bounded, classified and explained. A precedent is a case, not a truth: a reviewed label weighs more than an unreviewed one, and any may be wrong; check what it says on your data, and a precedent your data contradict, say so in your analysis (section 3). A phenomenon the catalogue does not know is labelled too, without forcing a cause; its precedents may know it. A phenomenon found once is looked for over **the whole range** and treated the same way every time: cold mornings, for example, are all listed by `days.py --bounds` (pose and `corr`) against the temperature (`metrics.py`). Before writing that the targets do not move and that it is the camera, check it **target by target** with `overview.py --rigid`: a target whose raw position departs alone from the overall movement moves in the image; it is not the compensation. 3. **Write your labels**, your analysis, then what your instructions ask for. ## The labels @@ -68,7 +68,7 @@ Before answering, `python3 labeling/validate.py /labels.jsonl` mu ## 1. My labels ### → -- **Phenomenon of the map**: | none +- **Phenomenon of the catalogue**: <its name> | none - **Precedents**: <id, or station and dates, of those that helped you, and how> | none useful - **Family**: <key>, and why it rather than a neighbour - **Measured**: <columns, sensors, values, times, what does not move> @@ -79,7 +79,7 @@ Before answering, `python3 labeling/validate.py <output folder>/labels.jsonl` mu <what moves a little without deserving a label, and why> ## 3. The documentation -<what helped you in the label map and the precedents (and the detectors with DETECTORS: yes), what was missing, what misled you or seemed contradicted by the data> +<what helped you in the labeling knowledge (`families.md`, `labeling-rules.md`, `reading-rules.md`, `stations.md`, `tools/DATA.md`) and the precedents (and the detectors with DETECTORS: yes), what was missing, what misled you or seemed contradicted by the data> ## Difficulties none | - <what hindered your work: unclear instruction, path, command>