Step 3: M1, gradient boosting on station aggregates (leave one station out, and a time split)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
c04cbc90bb
commit
f93cdf3cd1
3 files changed
+126
No files matched your search
@@ -29,6 +29,23 @@ uv venv .venv && uv pip install -p .venv -r requirements.txt
|
||||
|
||||
`evaluate.py rules` (step 2, ~20 min, once per change of the detectors): the rules' predictions per instant in `cache/rules/`, and the check that their counts are those of `bench/score.py`, station by station (exit 1 otherwise). `evaluate.py score`: a folder of `<STATION>.npz` with `yhat` [instant, family] against the labels, beside the rules, with the regressions of `detectors/CLAUDE.md` (more than 2 points of found or correct lost on a family of a station).
|
||||
|
||||
```bash
|
||||
.venv/bin/python m1.py --cache cache --out runs/m1 --balance && .venv/bin/python evaluate.py score runs/m1 --cache cache
|
||||
.venv/bin/python m1.py --cache cache --out - --split-time --balance
|
||||
```
|
||||
|
||||
`m1.py` (step 3, a few minutes): gradient boosting on the station's features and the aggregates of its sensors', Visibility and Obstruction, leaving one station out; `--split-time`, the diagnosis on the last 30 % of every station, all stations seen in training. First results (2026-11-16, found / correct, total):
|
||||
|
||||
| | Visibility | Obstruction |
|
||||
|---|---|---|
|
||||
| rules, all stations | 47 / 77 % | 26 / 28 % |
|
||||
| M1, station left out | 44 / 62 % | 9 / 7 % |
|
||||
| M1, station left out, `--balance` | 46 / 70 % | 21 / 23 % |
|
||||
| rules, last 30 % | 49 / 77 % | 15 / 41 % |
|
||||
| M1, last 30 % (`--split-time`) | 58 / 87 % | 39 / 75 % |
|
||||
|
||||
The information is in the features: with every station seen, M1 beats the rules; moving to a new station is what fails (Obstruction is labelled on three stations only).
|
||||
|
||||
## Contracts
|
||||
|
||||
- Reads the measurements through `detectors/scripts/data.py`, the reading of the benchmark; the labels from `labeling/auto/`; the data from `data/` (read-only). Paths are passed as arguments.
|
||||
|
||||
Reference in new issue
Block a user