River Annotation Tool
A desktop application for manually annotating river video clips as part of the HydroScan project. Annotators draw pixel-level water masks over river footage and answer structured survey questions about flow conditions, lighting, and scene quality.
Requirements
- Python 3.12
- uv (recommended) or pip
Installation
# Clone the repository
git clone <repo-url>
cd river-annotation-tool
# Install with uv (creates the virtual environment automatically)
uv sync
# Or with pip
python -m venv .venv
.venv\Scripts\activate # Windows
# source .venv/bin/activate # macOS/Linux
pip install -e .
Setup
Before running, create your config and clip list from the provided examples:
cp config/config.example.yaml config/config.yaml
cp config/clips.example.txt config/clips.txt
Edit config/config.yaml to set your data_dir and out_dir, then edit config/clips.txt to list the clips you want to annotate.
Usage
python -m river_annotation_tool.annotation_script
Arguments
| Argument | Default | Description |
|---|---|---|
--config |
config/config.yaml |
Path to the config YAML file |
--data |
(from config) | Override data_dir from config |
--out |
(from config) | Override out_dir from config |
--clips |
(from config) | Override clips_file from config |
--clip |
(first unannotated in list) | Open a specific clip by stem name |
--extras |
off | Also save GIFs and extra PNGs (see Output section) |
--no-skip |
off | Show already-annotated clips instead of skipping them |
Typical workflows
# Annotate clips listed in config/clips.txt (default)
python -m river_annotation_tool.annotation_script
# Use a different config file
python -m river_annotation_tool.annotation_script --config config/my_config.yaml
# Override paths from the command line
python -m river_annotation_tool.annotation_script --data data/clips --out data/out
# Annotate a single specific clip
python -m river_annotation_tool.annotation_script --clip left_20230615T120000
Configuration
All settings live in config/config.yaml. Copy config/config.example.yaml to get started.
filenames:
video_in_zip: left.mp4 # video filename inside each ZIP archive
video_tmp_suffix: .mp4 # suffix for the extraction temp file
zip_extension: .zip # extension used when resolving clip names
mask: mask.png # saved water mask
metadata: metadata.json # saved survey answers
frame: frame.png # middle frame snapshot
overlay: overlay.png # frame with mask blended in green
mask_vis: mask_vis.png # greyscale mask PNG (--extras only)
gif_original_hires: video_original_hires.gif
gif_original_lowres: video_original_lowres.gif
gif_overlay_hires: video_overlay_hires.gif
gif_overlay_lowres: video_overlay_lowres.gif
display_max: 720 # longest side in pixels for display
fps_fallback: 25 # FPS to use if the video header is missing
max_frames: 100 # max frames to extract per clip
data_dir: data/clips # directory containing ZIP archives
out_dir: data/annotation_results
clips_file: config/clips.txt
# optical_flow_config_file: config/optical_flow_config.yaml # optional, enables Auto Segment
questions:
- section: River
items:
- key: flow
label: "Flow Regime"
options: [Turbulent, Laminar, Uncertain]
default: Laminar
# add more items or sections as needed
Add, remove, or reorder questions directly in the YAML — the UI rebuilds automatically. key is what gets saved in metadata.json; default selects the pre-checked option (omit or set to null to leave unselected).
Optical flow segmentation (optional)
Set optical_flow_config_file in config.yaml to point to a YAML file that enables the Auto Segment button. When pressed, the tool computes a river mask from the loaded frames and replaces the current mask (undoable). The segmentation combines two criteria:
- Optical flow magnitude — pixels where the temporal median of frame-to-frame flow (scaled by FPS) exceeds a fraction of the maximum are considered moving water.
- Brightness — pixels outside a brightness window are excluded (removes sky, saturated glare, etc.).
# config/optical_flow_config.yaml
enabled: true
norm_squared_threshold: 0.06 # fraction of max flow² that counts as moving
gaussian_kernel: [5, 5] # blur kernel applied to the reference frame before brightness check
brightness_range: [2, 253] # [min, max] greyscale brightness to keep
enabled: false disables the button without removing the config file.
Clip list file
config/clips.txt lists the clip filenames to annotate, one per line. Lines starting with # are ignored. Clips are processed in order; already-annotated clips (those with an existing mask.png) are skipped automatically. Pass --no-skip to include them. When the last clip is reached, a dialog appears and the app exits.
# Example clips.txt
left_20230501T120000.zip
left_20230502T120000.zip
Copy config/clips.example.txt as a starting point.
Controls
The window is split into two panels: the video canvas on the left (~70% of the width) and the survey panel on the right. The video auto-plays as a looping preview. Drawing tools and mask controls are arranged above and beside the canvas; navigation buttons (Previous / Next / Skip) sit at the top.
Tool modes
Three drawing tools are available in the tool row. The active tool is highlighted in blue.
| Tool | How to activate | Description |
|---|---|---|
| Brush | Click Brush | Click and drag to paint the mask with a circular brush (default) |
| Polygon | Click Polygon | Click to place vertices and build closed shapes; use Fill mode to commit them |
| Fill | Click Fill | Click inside a closed polygon to fill it onto the mask |
Brush tool
| Action | How |
|---|---|
| Draw water mask | Click and drag on the video |
| Erase mask | Toggle Eraser button (turns orange when active), then drag |
| Brush preview | A white circle follows the cursor showing the current brush size |
| Adjust brush size | Brush size slider (2–50 px, default 5); click ↺ to reset |
Polygon tool
Polygons are drawn as overlays and do not affect the mask until you use Fill mode.
| Action | How |
|---|---|
| Add vertex | Left-click on the canvas |
| Remove last vertex | Right-click |
| Close a shape | Left-click near the first vertex (red dot) when ≥ 3 vertices are placed; completed shapes turn bold cyan |
| Draw multiple shapes | Each closed shape is kept independently; draw as many as needed |
| Cancel in-progress polygon | Cancel Current Poly — discards the unfinished polygon, keeps completed shapes |
| Delete last completed shape | Del Shape |
Fill tool
| Action | How |
|---|---|
| Fill a shape | Left-click anywhere inside a closed polygon; that shape's interior is painted onto the mask |
| Nested shapes | If a closed polygon lies entirely inside the target, its interior is left unfilled (acts as a hole) |
| Innermost shape | Clicking inside nested shapes always fills the innermost (smallest) polygon containing the click |
| Undo fill | Undo — each fill is a single undoable step |
Mask editing
| Action | How |
|---|---|
| Undo last action | Undo |
| Undo 10 actions | Undo×10 |
| Redo | Redo |
| Clear entire mask | Clear |
| Toggle mask overlay | Hide Mask / Show Mask — button turns red when hidden; does not affect mask data |
| Mask transparency | Mask Alpha slider (0–100%, default 40%); click ↺ to reset |
Starting-point shortcuts
| Action | How |
|---|---|
| Load mask from previous clip | Load Prev Mask — copies the saved mask of the previous clip onto the current one; undoable |
| Optical flow first guess | Auto Segment — replaces the current mask with an automatic river segmentation; undoable. Only available when optical_flow_config_file is set in config.yaml. |
Image display adjustments
Three vertical sliders sit to the left of the video and affect display only — they do not change what is saved.
| Slider | Effect | Range |
|---|---|---|
| Brightness | Shifts all pixel values up or down | −100 to +100 |
| Contrast | Scales pixel values around the midpoint | −100 to +100 |
| Gamma | Applies a power-law correction (higher = brighter) | 0.1× to 3.0× |
Click ↺ below any slider to restore its default value.
Navigation
| Action | How |
|---|---|
| Save and continue | Next — saves current clip and loads the next one. If the clip already has a saved annotation a dialog asks whether to replace it or keep the existing save. |
| Go back | Previous — saves current clip and returns to the previously viewed clip. Disabled on the first clip. |
| Skip without saving | Skip — discards any unsaved changes and loads the next clip without writing anything to disk. |
Output
Each annotated clip produces a folder <out_dir>/<clip_stem>/ with:
mask.png # Binary water mask at full source resolution (always)
metadata.json # Survey answers as JSON (always)
frame.png # Middle frame of the clip (always)
overlay.png # That frame with the mask blended in green (always)
# Only with --extras:
mask_vis.png # Mask rendered as a greyscale PNG
video_original_hires.gif # All frames at display resolution
video_original_lowres.gif # All frames at 50% of display resolution
video_overlay_hires.gif # Overlay GIF at display resolution
video_overlay_lowres.gif # Overlay GIF at 50% of display resolution
All output filenames can be overridden via the filenames: section in config/config.yaml.
Survey answers (metadata.json)
Keys and values are determined by the questions section in config/config.yaml. With the default config:
{
"flow": "Turbulent | Laminar | Uncertain",
"shadows": "Yes | No | Uncertain",
"artifacts": "Yes | No | Uncertain",
"lighting": "Day | Night | Uncertain",
"exposure": "Overexposed | Underexposed | Both | Normal | Uncertain",
"snowing": "Yes | No | Uncertain",
"snow_on_ground":"Yes | No | Uncertain"
}
How it works
Clip format
Each clip is a ZIP archive containing a video file (default left.mp4, configurable via filenames.video_in_zip). The filename encodes the recording timestamp (e.g. left_20230615T120000.zip).
Frame loading
Up to max_frames frames are extracted from the video and scaled so the longest side is display_max px. This display-resolution copy is what the annotator works on; the full-resolution dimensions are remembered separately so the saved mask is upscaled back to the original size on export.
Mask drawing
The mask is a binary array at display resolution. Brush strokes stamp a filled circle (draw or erase). Polygon shapes are stored as overlays and don't touch the mask until a Fill click rasterises them — the innermost polygon containing the click is filled, and any polygon whose centroid falls inside it is punched out as a hole.
Every mask-changing operation is pushed onto an undo stack before it executes. On save, the mask is upscaled to the original video resolution and written as an 8-bit PNG (0 or 255).
Resuming
When a clip is loaded that already has a saved mask.png and metadata.json, the mask is restored at display resolution and the survey answers are pre-filled.
Repository structure
config/
config.yaml # Your local config (git-ignored, copy from example)
config.example.yaml # Example config to copy and edit
clips.txt # Your clip list (git-ignored, copy from example)
clips.example.txt # Example clip list
optical_flow_config.yaml # Optional optical flow parameters (enable via config.yaml)
src/river_annotation_tool/
annotation_script.py # Entry point — argument parsing and app launch
annotator.py # Main QMainWindow — orchestrates all components
clip_selector.py # Reads the clip list and picks the next clip
mask_canvas.py # Drawing widget — brush, undo, erase, mouse events
video_loader.py # ZIP extraction and frame resizing
compute_optical_flow.py # Optical flow river segmentation (Auto Segment button)
config.py # AppConfig dataclass and YAML loader
__init__.py # Package version
pyproject.toml # Project metadata and dependencies
Development
# Install pre-commit hooks
pre-commit install
pre-commit run --all-files # Run manually once
# Add a dependency
uv add <package>
uv add --dev <package> # Development-only