diff --git a/README.md b/README.md index 574b405..2ddad84 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ -# River Annotation Tool +# Video Annotation Tool -A desktop GUI application for manually annotating river video clips as part of the [HydroScan](https://github.com/HydroScan) project. Annotators draw pixel-level water masks over river footage and answer structured survey questions about flow conditions, lighting, and scene quality. +A desktop GUI application for manually annotating video clips. Annotators draw pixel-level segmentation masks over footage and answer structured survey questions defined in a config file. ## Requirements @@ -24,7 +24,7 @@ cp config/clips.example.txt config/clips.txt # Edit config/questions.yaml to customise survey questions (optional) # 4. Run -uv run python -m river_annotation_tool.annotation_script +uv run python -m clip_annotator.annotation_script ``` ## Installation @@ -81,9 +81,9 @@ The `clips_file` (the list of clip filenames to annotate) is always read from th ## Usage ```sh -uv run python -m river_annotation_tool.annotation_script +uv run python -m clip_annotator.annotation_script # or, if you have the venv activated: -python -m river_annotation_tool.annotation_script +python -m clip_annotator.annotation_script ``` ### Arguments @@ -94,7 +94,7 @@ python -m river_annotation_tool.annotation_script | `--data` | *(from config)* | Override `data_dir` from config | | `--out` | *(from config)* | Override `out_dir` from config | | `--clips` | *(from config)* | Override `clips_file` from config | -| `--clip` | *(first unannotated in list)* | Open a specific clip by stem name | +| `--clip` | *(first unannotated in list)* | Open a specific clip by its stem name (filename without extension, e.g. `clip_20230501T120000`) | | `--extras` | off | Also save GIFs and extra PNGs (see Output section) | | `--no-skip` | off | Show already-annotated clips instead of skipping them | @@ -102,16 +102,16 @@ python -m river_annotation_tool.annotation_script ```sh # Annotate clips listed in config/clips.txt (default) -uv run python -m river_annotation_tool.annotation_script +uv run python -m clip_annotator.annotation_script # Use a different config file -uv run python -m river_annotation_tool.annotation_script --config config/my_config.yaml +uv run python -m clip_annotator.annotation_script --config config/my_config.yaml # Override paths from the command line -uv run python -m river_annotation_tool.annotation_script --data data/clips --out data/out +uv run python -m clip_annotator.annotation_script --data data/clips --out data/out # Annotate a single specific clip -uv run python -m river_annotation_tool.annotation_script --clip left_20230615T120000 +uv run python -m clip_annotator.annotation_script --clip clip_20230615T120000 ``` ## Configuration @@ -121,8 +121,8 @@ Main settings live in `config/config.yaml`. Copy `config/config.example.yaml` to ```yaml storage: local # required: 'local' or 's3' -data_dir: # required: directory containing ZIP archives (local path or bucket/prefix for S3) -out_dir: # required: where to write annotations +data_dir: # required: read-only source of ZIP archives (local path or bucket/prefix for S3) +out_dir: # required: write destination for annotations (can be same bucket as data_dir with a different prefix, or a separate location) clips_file: config/clips.txt optical_flow_config_file: config/optical_flow_config.yaml @@ -139,7 +139,7 @@ filenames: zip_extension: .zip ``` -Output filenames (`mask.png`, `metadata.json`, etc.) have sensible defaults and can be overridden in the `filenames:` block — see [`config.py`](src/river_annotation_tool/config.py) for the full list. +Output filenames (`mask.png`, `metadata.json`, etc.) have sensible defaults and can be overridden in the `filenames:` block — see [`config.py`](src/clip_annotator/config.py) for the full list. ### Survey questions @@ -147,9 +147,9 @@ Survey questions are defined in `config/questions.yaml` (committed to the repo). ### Optical flow segmentation -`config/optical_flow_config.yaml` controls the **Auto Segment** button. When pressed, the tool computes a river mask from the loaded frames and replaces the current mask (undoable). The segmentation combines two criteria: +`config/optical_flow_config.yaml` controls the **Auto Segment** button. When pressed, the tool computes a segmentation mask from the loaded frames and replaces the current mask (undoable). The segmentation combines two criteria: -- **Optical flow magnitude** — pixels where the temporal median of frame-to-frame flow (scaled by FPS) exceeds a fraction of the maximum are considered moving water. +- **Optical flow magnitude** — pixels where the temporal median of frame-to-frame flow (scaled by FPS) exceeds a fraction of the maximum are considered moving. - **Brightness** — pixels outside a brightness window are excluded (removes sky, saturated glare, etc.). ```yaml @@ -164,41 +164,25 @@ brightness_range: [2, 253] # [min, max] greyscale brightness to keep ## Clip list file -`config/clips.txt` lists the clip filenames to annotate, one per line. Lines starting with `#` are ignored. Clips are processed in order; already-annotated clips (those with an existing `mask.png`) are skipped automatically. Pass `--no-skip` to include them. When the last clip is reached, a dialog appears and the app exits. +`config/clips.txt` lists bare clip filenames (not full paths) to annotate, one per line — the tool prepends `data_dir` automatically. Each entry must be the exact filename of the ZIP archive as it appears in `data_dir` (e.g. `clip_20230501T120000.zip`). Lines starting with `#` are ignored. Clips are processed in order; a clip is considered already-annotated when `//mask.png` exists and is skipped automatically. Pass `--no-skip` to include already-annotated clips. When the last clip is reached, a dialog appears and the app exits. ``` # Example clips.txt -left_20230501T120000.zip -left_20230502T120000.zip +clip_20230501T120000.zip +clip_20230502T120000.zip ``` Copy `config/clips.example.txt` as a starting point. ## Multi-annotator setup -Pre-made clip lists for 7 annotators are included in `config/annotator_A.txt` through `config/annotator_G.txt`. Each annotator is assigned exactly 5 recording days (non-consecutive where possible), covering all 24 available days across the dataset. - -To run the tool for a specific annotator, pass their file via `--clips`: +To distribute clips across multiple annotators, create one clips file per annotator (e.g. `config/annotator_A.txt`) listing their assigned clip filenames, then pass it via `--clips`: ```sh -uv run python -m river_annotation_tool.annotation_script --clips config/annotator_A.txt +uv run python -m clip_annotator.annotation_script --clips config/annotator_A.txt ``` -### Assignment - -11 of the 24 days are reviewed by two annotators (the theoretical maximum given 7 × 5 = 35 slots and 24 days), giving 11 days with double coverage for inter-annotator agreement checks. - -| Annotator | Days | Clips | -|---|---|---| -| A | 2025-11-17 · 2025-12-03 · 2026-01-01 · 2026-01-09 · 2026-02-11 | 94 | -| B | 2025-11-18 · 2025-12-05 · 2026-01-06 · 2026-02-12 · 2026-03-02 | 128 | -| C | 2025-11-22 · 2025-12-12 · 2026-01-07 · 2026-02-16 · 2026-03-03 | 146 | -| D | 2025-11-18 · 2025-11-24 · 2025-12-16 · 2026-01-08 · 2026-03-02 | 102 | -| E | 2025-11-25 · 2025-12-03 · 2026-01-09 · 2026-01-12 · 2026-03-03 | 80 | -| F | 2025-11-25 · 2025-12-16 · 2026-01-10 · 2026-01-13 · 2026-03-11 | 93 | -| G | 2025-11-22 · 2025-12-05 · 2026-01-12 · 2026-02-11 · 2026-03-12 | 110 | - -Days covered by two annotators: 2025-11-18 (B, D) · 2025-11-22 (C, G) · 2025-11-25 (E, F) · 2025-12-03 (A, E) · 2025-12-05 (B, G) · 2025-12-16 (D, F) · 2026-01-09 (A, E) · 2026-01-12 (E, G) · 2026-02-11 (A, G) · 2026-03-02 (B, D) · 2026-03-03 (C, E) +Assigning non-overlapping clip lists lets each annotator work independently. Intentionally overlapping a subset of clips across annotators enables inter-annotator agreement checks. ## Controls @@ -218,7 +202,7 @@ Three drawing tools are available in the tool row. The active tool is highlighte | Action | How | |---|---| -| Draw water mask | Click and drag on the video | +| Draw mask | Click and drag on the video | | Erase mask | Toggle **Eraser** button (turns orange when active), then drag | | Brush preview | A white circle follows the cursor showing the current brush size | | Adjust brush size | **Brush size** slider (2–50 px, default 5); click **↺** to reset | @@ -261,7 +245,7 @@ Polygons are drawn as overlays and do not affect the mask until you use **Fill** | Action | How | |---|---| | Load mask from previous clip | **Load Prev Mask** — copies the saved mask of the previous clip onto the current one; undoable | -| Optical flow first guess | **Auto Segment** — replaces the current mask with an automatic river segmentation; undoable. Disabled when `enabled: false` in `config/optical_flow_config.yaml`. | +| Optical flow first guess | **Auto Segment** — replaces the current mask with an automatic segmentation based on motion and brightness; undoable. Disabled when `enabled: false` in `config/optical_flow_config.yaml`. | ### Image display adjustments @@ -288,7 +272,7 @@ Click **↺** below any slider to restore its default value. Each annotated clip produces a folder `//` with: ``` -mask.png # Binary water mask at full source resolution (always) +mask.png # Binary segmentation mask at full source resolution (always) metadata.json # Survey answers as JSON (always) frame.png # Middle frame of the clip (always) overlay.png # That frame with the mask blended in green (always) @@ -323,7 +307,7 @@ Keys and values are determined by `config/questions.yaml`. With the default ques ### Clip format -Each clip is a ZIP archive containing a video file (default `left.mp4`, configurable via `filenames.video_in_zip`). The filename encodes the recording timestamp (e.g. `left_20230615T120000.zip`). +Each clip is a ZIP archive containing a video file. The video inside the archive must be named `left.mp4` by default; if your archives use a different internal name, set `filenames.video_in_zip` in `config.yaml`. The ZIP filename becomes the output folder name (e.g. `clip_20230501T120000.zip` → `/clip_20230501T120000/`). ### Frame loading @@ -350,14 +334,14 @@ config/ clips.example.txt # Example clip list questions.yaml # Survey question definitions optical_flow_config.yaml # Optical flow parameters (set enabled: false to disable Auto Segment) -src/river_annotation_tool/ +src/clip_annotator/ annotation_script.py # Entry point — argument parsing and app launch annotator.py # Main QMainWindow — orchestrates all components clip_selector.py # Reads the clip list and picks the next clip filesystem.py # Storage backend — local passthrough or S3 via s3fs mask_canvas.py # Drawing widget — brush, undo, erase, mouse events video_loader.py # ZIP extraction and frame resizing - compute_optical_flow.py # Optical flow river segmentation (Auto Segment button) + compute_optical_flow.py # Optical flow segmentation (Auto Segment button) config.py # AppConfig dataclass and YAML loader __init__.py # Package version pyproject.toml # Project metadata and dependencies diff --git a/pyproject.toml b/pyproject.toml index f80a502..83d953d 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -3,7 +3,7 @@ requires = ["setuptools>=61.0.0"] build-backend = "setuptools.build_meta" [project] -name = "river_annotation_tool" +name = "clip_annotator" authors = [ # TODO configure authors # { name = "Jane Smith", email = "jane.smith@example.com" }, @@ -33,7 +33,7 @@ dev = [ ] [tool.setuptools.dynamic] -version = {attr = "river_annotation_tool.__version__"} +version = {attr = "clip_annotator.__version__"} [tool.ruff] target-version = "py312" diff --git a/requirements.txt b/requirements.txt index de3dc9b..014f5d8 100644 --- a/requirements.txt +++ b/requirements.txt @@ -433,14 +433,14 @@ matplotlib==3.10.9 \ --hash=sha256:d75d11c949914165976c621b2324f9ef162af7ebf4b057ddf95dd1dba7e5edcf \ --hash=sha256:f0c3c28d9fbcc1fe7a03be236d73430cf6409c41fb2383a7ac52fe932b072cb1 \ --hash=sha256:fd66508e8c6877d98e586654b608a0456db8d7e8a546eb1e2600efd957302358 - # via river-annotation-tool + # via clip-annotator matplotlib-inline==0.2.1 \ --hash=sha256:d56ce5156ba6085e00a9d54fead6ed29a9c47e215cd1bba2e976ef39f5710a76 \ --hash=sha256:e1ee949c340d771fc39e241ea75683deb94762c8fa5f2927ec57c83c4dffa9fe # via + # clip-annotator # ipykernel # ipython - # river-annotation-tool mistune==3.2.0 \ --hash=sha256:708487c8a8cdd99c9d90eb3ed4c3ed961246ff78ac82f03418f5183ab70e398a \ --hash=sha256:febdc629a3c78616b94393c6580551e0e34cc289987ec6c35ed3f4be42d0eee1 @@ -527,7 +527,7 @@ opencv-contrib-python-headless==4.12.0.88 \ --hash=sha256:b183e2322468c9d3bd9cac4ba44b272d828ec22842395bcfa51df31765224c0a \ --hash=sha256:c57e32812fea2a542bb220088fb3ce8a210fe114c9454d1c9e8cd162e1a1fde8 \ --hash=sha256:d60a12b915c55a50468c013fcd839e941b49ccc1f37b914b62543382c36bf81d - # via river-annotation-tool + # via clip-annotator packaging==26.2 \ --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 @@ -549,7 +549,7 @@ pandas==3.0.2 \ --hash=sha256:ef8b27695c3d3dc78403c9a7d5e59a62d5464a7e1123b4e0042763f7104dc74f \ --hash=sha256:f4753e73e34c8d83221ba58f232433fca2748be8b18dbca02d242ed153945043 \ --hash=sha256:f8d68083e49e16b84734eb1a4dcae4259a75c90fb6e2251ab9a00b61120c06ab - # via river-annotation-tool + # via clip-annotator pandocfilters==1.5.1 \ --hash=sha256:002b4a555ee4ebc03f8b66307e287fa492e4a77b4ea14d3f934328297bb4939e \ --hash=sha256:93be382804a9cdb0a7267585f157e5d1731bbe5545a85b268d6f5fe6232de2bc @@ -576,8 +576,8 @@ pillow==12.2.0 \ --hash=sha256:f3f40b3c5a968281fd507d519e444c35f0ff171237f4fdde090dd60699458421 \ --hash=sha256:fc3d34d4a8fbec3e88a79b92e5465e0f9b842b628675850d860b8bd300b159f5 # via + # clip-annotator # matplotlib - # river-annotation-tool platformdirs==4.9.6 \ --hash=sha256:3bfa75b0ad0db84096ae777218481852c0ebc6c727b3168c1b9e0118e458cf0a \ --hash=sha256:e61adb1d5e5cb3441b4b7710bea7e4c12250ca49439228cc1021c00dcfac0917 @@ -663,7 +663,7 @@ pyside6==6.11.0 \ --hash=sha256:9092cb002ca43c64006afb2e0d0f6f51aef17aa737c33a45e502326a081ddcbc \ --hash=sha256:b15f39acc2b8f46251a630acad0d97f9a0a0461f2baffcd66d7adfada8eb641e \ --hash=sha256:c642e2d25704ca746fd37f56feacf25c5aecc4cd40bef23d18eec81f87d9dc00 - # via river-annotation-tool + # via clip-annotator pyside6-addons==6.11.0 \ --hash=sha256:413e6121c24f5ffdce376298059eddecff74aa6d638e94e0f6015b33d29b889e \ --hash=sha256:8ffb40222456078930816ebcac2f2511716d2acbc11716dd5acc5c365179a753 \ @@ -697,7 +697,7 @@ python-discovery==1.2.2 \ python-dotenv==1.2.2 \ --hash=sha256:1d8214789a24de455a8b8bd8ae6fe3c6b69a5e3d64aa8a8e5d68e694bbcb285a \ --hash=sha256:2c371a91fbd7ba082c2c1dc1f8bf89ca22564a087c2c287cd9b662adde799cf3 - # via river-annotation-tool + # via clip-annotator python-json-logger==4.1.0 \ --hash=sha256:132994765cf75bf44554be9aa49b06ef2345d23661a96720262716438141b6b2 \ --hash=sha256:b396b9e3ed782b09ff9d6e4f1683d46c83ad0d35d2e407c09a9ebbf038f88195 @@ -723,9 +723,9 @@ pyyaml==6.0.3 \ --hash=sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f \ --hash=sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0 # via + # clip-annotator # jupyter-events # pre-commit - # river-annotation-tool pyzmq==27.1.0 \ --hash=sha256:0de3028d69d4cdc475bfe47a6128eb38d8bc0e8f4d69646adfbcd840facbac28 \ --hash=sha256:1c179799b118e554b66da67d88ed66cd37a169f1f23b5d9f0a231b4e8d44a113 \ @@ -811,7 +811,7 @@ ruff==0.15.0 \ s3fs==2026.4.0 \ --hash=sha256:5bdce0abb00b0435ee150807a45fea727451dbc22de4cbc116464f8504ab9d37 \ --hash=sha256:de0d2a1f33cdf03831fd2382d278c6e4e31fe57c3bf2f703c61f8aec6b703e2a - # via river-annotation-tool + # via clip-annotator send2trash==2.1.0 \ --hash=sha256:0da2f112e6d6bb22de6aa6daa7e144831a4febf2a87261451c4ad849fe9a873c \ --hash=sha256:1c72b39f09457db3c05ce1d19158c2cbef4c32b8bedd02c155e49282b7ea7459 diff --git a/src/river_annotation_tool/__init__.py b/src/clip_annotator/__init__.py similarity index 100% rename from src/river_annotation_tool/__init__.py rename to src/clip_annotator/__init__.py diff --git a/src/river_annotation_tool/annotation_script.py b/src/clip_annotator/annotation_script.py similarity index 100% rename from src/river_annotation_tool/annotation_script.py rename to src/clip_annotator/annotation_script.py diff --git a/src/river_annotation_tool/annotator.py b/src/clip_annotator/annotator.py similarity index 99% rename from src/river_annotation_tool/annotator.py rename to src/clip_annotator/annotator.py index 3dbadc7..be7e992 100644 --- a/src/river_annotation_tool/annotator.py +++ b/src/clip_annotator/annotator.py @@ -58,7 +58,7 @@ class Annotator(QMainWindow): self.history: list[str] = [] self.history_pos: int = -1 - self.setWindowTitle("River Annotator") + self.setWindowTitle("Clip Annotator") self._load_clip(specific=clip) self._history_push() self._init_ui() diff --git a/src/river_annotation_tool/clip_selector.py b/src/clip_annotator/clip_selector.py similarity index 100% rename from src/river_annotation_tool/clip_selector.py rename to src/clip_annotator/clip_selector.py diff --git a/src/river_annotation_tool/compute_optical_flow.py b/src/clip_annotator/compute_optical_flow.py similarity index 100% rename from src/river_annotation_tool/compute_optical_flow.py rename to src/clip_annotator/compute_optical_flow.py diff --git a/src/river_annotation_tool/config.py b/src/clip_annotator/config.py similarity index 100% rename from src/river_annotation_tool/config.py rename to src/clip_annotator/config.py diff --git a/src/river_annotation_tool/filesystem.py b/src/clip_annotator/filesystem.py similarity index 100% rename from src/river_annotation_tool/filesystem.py rename to src/clip_annotator/filesystem.py diff --git a/src/river_annotation_tool/mask_canvas.py b/src/clip_annotator/mask_canvas.py similarity index 100% rename from src/river_annotation_tool/mask_canvas.py rename to src/clip_annotator/mask_canvas.py diff --git a/src/river_annotation_tool/video_loader.py b/src/clip_annotator/video_loader.py similarity index 100% rename from src/river_annotation_tool/video_loader.py rename to src/clip_annotator/video_loader.py diff --git a/uv.lock b/uv.lock index bb4a696..ff8edcc 100644 --- a/uv.lock +++ b/uv.lock @@ -304,6 +304,48 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/db/8f/61959034484a4a7c527811f4721e75d02d653a35afb0b6054474d8185d4c/charset_normalizer-3.4.7-py3-none-any.whl", hash = "sha256:3dce51d0f5e7951f8bb4900c257dad282f49190fdbebecd4ba99bcc41fef404d", size = 61958, upload-time = "2026-04-02T09:28:37.794Z" }, ] +[[package]] +name = "clip-annotator" +source = { editable = "." } +dependencies = [ + { name = "matplotlib" }, + { name = "matplotlib-inline" }, + { name = "opencv-contrib-python-headless" }, + { name = "pandas" }, + { name = "pillow" }, + { name = "pyside6" }, + { name = "python-dotenv" }, + { name = "pyyaml" }, + { name = "s3fs" }, +] + +[package.dev-dependencies] +dev = [ + { name = "notebook" }, + { name = "pre-commit" }, + { name = "ruff" }, +] + +[package.metadata] +requires-dist = [ + { name = "matplotlib", specifier = ">=3.10.8" }, + { name = "matplotlib-inline", specifier = ">=0.2.1" }, + { name = "opencv-contrib-python-headless", specifier = "==4.12.0.88" }, + { name = "pandas", specifier = ">=2.3.3" }, + { name = "pillow", specifier = ">=12.2.0" }, + { name = "pyside6", specifier = ">=6.11.0" }, + { name = "python-dotenv", specifier = ">=1.0" }, + { name = "pyyaml", specifier = ">=6.0" }, + { name = "s3fs", specifier = ">=2024.0" }, +] + +[package.metadata.requires-dev] +dev = [ + { name = "notebook", specifier = "~=7.5" }, + { name = "pre-commit", specifier = "~=4.5" }, + { name = "ruff", specifier = "==0.15.0" }, +] + [[package]] name = "colorama" version = "0.4.6" @@ -1528,48 +1570,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/7e/71/44ce230e1b7fadd372515a97e32a83011f906ddded8d03e3c6aafbdedbb7/rfc3987_syntax-1.1.0-py3-none-any.whl", hash = "sha256:6c3d97604e4c5ce9f714898e05401a0445a641cfa276432b0a648c80856f6a3f", size = 8046, upload-time = "2025-07-18T01:05:03.843Z" }, ] -[[package]] -name = "river-annotation-tool" -source = { editable = "." } -dependencies = [ - { name = "matplotlib" }, - { name = "matplotlib-inline" }, - { name = "opencv-contrib-python-headless" }, - { name = "pandas" }, - { name = "pillow" }, - { name = "pyside6" }, - { name = "python-dotenv" }, - { name = "pyyaml" }, - { name = "s3fs" }, -] - -[package.dev-dependencies] -dev = [ - { name = "notebook" }, - { name = "pre-commit" }, - { name = "ruff" }, -] - -[package.metadata] -requires-dist = [ - { name = "matplotlib", specifier = ">=3.10.8" }, - { name = "matplotlib-inline", specifier = ">=0.2.1" }, - { name = "opencv-contrib-python-headless", specifier = "==4.12.0.88" }, - { name = "pandas", specifier = ">=2.3.3" }, - { name = "pillow", specifier = ">=12.2.0" }, - { name = "pyside6", specifier = ">=6.11.0" }, - { name = "python-dotenv", specifier = ">=1.0" }, - { name = "pyyaml", specifier = ">=6.0" }, - { name = "s3fs", specifier = ">=2024.0" }, -] - -[package.metadata.requires-dev] -dev = [ - { name = "notebook", specifier = "~=7.5" }, - { name = "pre-commit", specifier = "~=4.5" }, - { name = "ruff", specifier = "==0.15.0" }, -] - [[package]] name = "rpds-py" version = "0.30.0"