Regression computed by bench/compare.py (Total > 1 point; station > 5 points and > 20 instants), complexity in the benchmark, pruning after a fix

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
maximeandClaude Opus 5.5 committed 2026-11-16 17:53:09 +01:00
1 parent 63d954a126
commit a008ea8fcc
4 files changed
+77 -5

No files matched your search

+11 -2
View File
@@ -3,12 +3,13 @@
For each <STATION>.jsonl of --labels that has its folder <STATION>/ in --data, the score per family and per instant of
score.py, over the range of its labels: from the day of the first to the midnight after the last; with --from/--to, over
that range for every station, a station without a label on it included (several stations labelled the same days). Then the total,
family by family, which says whether the detectors do better overall. The block of a station is its score; the
others, the non-regression: a family that goes back on a station is a regression, even if the total rises.
family by family, which says whether the detectors do better overall, and the complexity of the detectors (code lines
and constants of scripts/). Two outputs are compared by compare.py (the regressions, CLAUDE.md).
Usage: python3 regression.py --labels <folder of <STATION>.jsonl> --data <folder of <STATION>/> [--stations A,B]
[--from YYYY-MM-DD --to YYYY-MM-DD]
"""
import argparse
import re
from datetime import timedelta
from pathlib import Path
@@ -17,6 +18,13 @@ import score # next to it
from data import iso, parse_time # on the path through score
def complexity(scripts=Path(__file__).resolve().parents[1] / "scripts"):
"""-> (code lines, constants) of the detectors: non-blank lines that are not a comment; NAME = lines at module level."""
lines = [x for f in sorted(scripts.glob("*.py")) for x in f.read_text().splitlines()]
code = [x for x in lines if x.strip() and not x.strip().startswith("#")]
return len(code), sum(bool(re.match(r"[A-Z][A-Z_0-9]* = ", x)) for x in lines)
def label_range(path):
"""Labels JSONL -> (start, end) iso, in whole UTC days, end excluded."""
labels = score.read_labels(path)
@@ -53,3 +61,4 @@ if __name__ == "__main__":
for k, v in n.items():
total[k] = [x + y for x, y in zip(total.get(k, [0, 0, 0]), v)]
score.show("Total", total)
print("\nComplexity: %d code lines, %d constants" % complexity())