Step 2: the evaluator (the measure of bench/score.py on predictions per instant; the rules reproduce it exactly)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b2d3734152
commit
c04cbc90bb
4 files changed
+153
-1
No files matched your search
@@ -21,6 +21,14 @@ uv venv .venv && uv pip install -p .venv -r requirements.txt
|
||||
|
||||
`dataset.py` (step 1, ~2 min): one `cache/<STATION>.npz` per labelled station, over its labelled range as the benchmark reads it: `xs` [instant, sensor, feature], `xc` [instant, feature], `y` [instant, family] with the conventions of `bench/score.py` (its labelled instants per family are the benchmark's, exactly). NaN where a value does not exist (a 1 h window at a 4 h step, no temperature, no image quality outside LENZSPITZE).
|
||||
|
||||
```bash
|
||||
.venv/bin/python test_evaluate.py
|
||||
.venv/bin/python evaluate.py rules --detectors ~/aimsight-detectors --data ~/aimsight-data/data --labels ~/aimsight-labeling/auto --cache cache
|
||||
.venv/bin/python evaluate.py score <predictions folder> --cache cache
|
||||
```
|
||||
|
||||
`evaluate.py rules` (step 2, ~20 min, once per change of the detectors): the rules' predictions per instant in `cache/rules/`, and the check that their counts are those of `bench/score.py`, station by station (exit 1 otherwise). `evaluate.py score`: a folder of `<STATION>.npz` with `yhat` [instant, family] against the labels, beside the rules, with the regressions of `detectors/CLAUDE.md` (more than 2 points of found or correct lost on a family of a station).
|
||||
|
||||
## Contracts
|
||||
|
||||
- Reads the measurements through `detectors/scripts/data.py`, the reading of the benchmark; the labels from `labeling/auto/`; the data from `data/` (read-only). Paths are passed as arguments.
|
||||
|
||||
Reference in new issue
Block a user