River Annotation Tool

A desktop application for manually annotating river video clips as part of the HydroScan project. Annotators draw pixel-level water masks over river footage and answer structured survey questions about flow conditions, lighting, and scene quality.

Requirements

• Python 3.12
• uv (recommended) or pip

Installation

# Clone the repository
git clone <repo-url>
cd river-annotation-tool

# Install with uv (creates the virtual environment automatically)
uv sync

# Or with pip
python -m venv .venv
.venv\Scripts\activate        # Windows
# source .venv/bin/activate   # macOS/Linux
pip install -e .

Setup

Before running, create your config and clip list from the provided examples:

cp config/config.example.yaml config/config.yaml
cp config/clips.example.txt config/clips.txt

Edit config/config.yaml to set your data_dir and out_dir, then edit config/clips.txt to list the clips you want to annotate.

Usage

python -m river_annotation_tool.annotation_script

Arguments

Argument	Default	Description
--config	config/config.yaml	Path to the config YAML file
--data	(from config)	Override data_dir from config
--out	(from config)	Override out_dir from config
--clips	(from config)	Override clips_file from config
--clip	(first unannotated in list)	Open a specific clip by stem name
--extras	off	Also save GIFs and extra PNGs (see Output section)
--no-skip	off	Show already-annotated clips instead of skipping them

Typical workflows

# Annotate clips listed in config/clips.txt (default)
python -m river_annotation_tool.annotation_script

# Use a different config file
python -m river_annotation_tool.annotation_script --config config/my_config.yaml

# Override paths from the command line
python -m river_annotation_tool.annotation_script --data data/clips --out data/out

# Annotate a single specific clip
python -m river_annotation_tool.annotation_script --clip left_20230615T120000

Configuration

All settings live in config/config.yaml. Copy config/config.example.yaml to get started.

filenames:
  video_in_zip: left.mp4           # video filename inside each ZIP archive
  video_tmp_suffix: .mp4           # suffix for the extraction temp file
  zip_extension: .zip              # extension used when resolving clip names
  mask: mask.png                   # saved water mask
  metadata: metadata.json          # saved survey answers
  frame: frame.png                 # middle frame snapshot
  overlay: overlay.png             # frame with mask blended in green
  mask_vis: mask_vis.png           # greyscale mask PNG (--extras only)
  gif_original_hires: video_original_hires.gif
  gif_original_lowres: video_original_lowres.gif
  gif_overlay_hires: video_overlay_hires.gif
  gif_overlay_lowres: video_overlay_lowres.gif

display_max: 720          # longest side in pixels for display
fps_fallback: 25          # FPS to use if the video header is missing
max_frames: 100           # max frames to extract per clip

data_dir: data/clips      # directory containing ZIP archives
out_dir: data/annotation_results
clips_file: config/clips.txt
# optical_flow_config_file: config/optical_flow_config.yaml   # optional, enables Auto Segment

questions:
  - section: River
    items:
      - key: flow
        label: "Flow Regime"
        options: [Turbulent, Laminar, Uncertain]
        default: Laminar
      # add more items or sections as needed

Add, remove, or reorder questions directly in the YAML — the UI rebuilds automatically. key is what gets saved in metadata.json; default selects the pre-checked option (omit or set to null to leave unselected).

Optical flow segmentation (optional)

Set optical_flow_config_file in config.yaml to point to a YAML file that enables the Auto Segment button. When pressed, the tool computes a river mask from the loaded frames and replaces the current mask (undoable). The segmentation combines two criteria:

• Optical flow magnitude — pixels where the temporal median of frame-to-frame flow (scaled by FPS) exceeds a fraction of the maximum are considered moving water.
• Brightness — pixels outside a brightness window are excluded (removes sky, saturated glare, etc.).

# config/optical_flow_config.yaml
enabled: true
norm_squared_threshold: 0.06   # fraction of max flow² that counts as moving
gaussian_kernel: [5, 5]        # blur kernel applied to the reference frame before brightness check
brightness_range: [2, 253]     # [min, max] greyscale brightness to keep

enabled: false disables the button without removing the config file.

Clip list file

config/clips.txt lists the clip filenames to annotate, one per line. Lines starting with # are ignored. Clips are processed in order; already-annotated clips (those with an existing mask.png) are skipped automatically. Pass --no-skip to include them. When the last clip is reached, a dialog appears and the app exits.

# Example clips.txt
left_20230501T120000.zip
left_20230502T120000.zip

Copy config/clips.example.txt as a starting point.

Controls

The window shows the video on the left (auto-playing) and the survey panel on the right.

Tool modes

Three drawing tools are available in the tool row. The active tool is highlighted in blue.

Tool	How to activate	Description
Brush	Click Brush	Click and drag to paint the mask with a circular brush (default)
Polygon	Click Polygon	Click to place vertices and build closed shapes; use Fill mode to commit them
Fill	Click Fill	Click inside a closed polygon to fill it onto the mask

Brush tool

Action	How
Draw water mask	Click and drag on the video
Erase mask	Toggle Eraser button (turns orange when active), then drag
Brush preview	A white circle follows the cursor showing the current brush size
Adjust brush size	Brush size slider; click ↺ to reset

Polygon tool

Polygons are drawn as overlays and do not affect the mask until you use Fill mode.

Action	How
Add vertex	Left-click on the canvas
Remove last vertex	Right-click
Close a shape	Left-click near the first vertex (red dot) when ≥ 3 vertices are placed; completed shapes turn bold cyan
Draw multiple shapes	Each closed shape is kept independently; draw as many as needed
Cancel in-progress polygon	Cancel Current Poly — discards the unfinished polygon, keeps completed shapes
Delete last completed shape	Del Shape

Fill tool

Action	How
Fill a shape	Left-click anywhere inside a closed polygon; that shape's interior is painted onto the mask
Nested shapes	If a closed polygon lies entirely inside the target, its interior is left unfilled (acts as a hole)
Innermost shape	Clicking inside nested shapes always fills the innermost (smallest) polygon containing the click
Undo fill	Undo — each fill is a single undoable step

Common mask actions

Action	How
Undo last action	Undo
Undo 10 actions	Undo×10
Redo	Redo
Clear entire mask	Clear
Toggle mask overlay	Hide Mask / Show Mask — button turns red when hidden; does not affect mask data
Mask transparency	Mask Alpha slider; click ↺ to reset
Load mask from previous clip	Load Prev Mask — copies the saved mask of the previous clip; undoable with Undo
Optical flow first guess	Auto Segment — runs automatic river segmentation and replaces the current mask; undoable with Undo. Only enabled when optical_flow_config_file is set in config.yaml.

Image display adjustments

Three vertical sliders sit to the left of the video and affect display only — they do not change what is saved.

Slider	Effect	Range
Brightness	Shifts all pixel values up or down	−100 to +100
Contrast	Scales pixel values around the midpoint	−100 to +100
Gamma	Applies a power-law correction (higher = brighter)	0.1× to 3.0×

Click ↺ below any slider to restore its default value.

Navigation

Action	How
Save and continue	Next — saves current clip and loads the next one. If the clip already has a saved annotation a dialog asks whether to replace it or keep the existing save.
Go back	Previous — saves current clip and returns to the previously viewed clip. Disabled on the first clip.
Skip without saving	Skip — discards any unsaved changes and loads the next clip without writing anything to disk.

Output

Each annotated clip produces a folder <out_dir>/<clip_stem>/ with:

mask.png          # Binary water mask at full source resolution (always)
metadata.json     # Survey answers as JSON (always)
frame.png         # Middle frame of the clip (always)
overlay.png       # That frame with the mask blended in green (always)

# Only with --extras:
mask_vis.png               # Mask rendered as a greyscale PNG
video_original_hires.gif   # All frames at display resolution
video_original_lowres.gif  # All frames at 50% of display resolution
video_overlay_hires.gif    # Overlay GIF at display resolution
video_overlay_lowres.gif   # Overlay GIF at 50% of display resolution

All output filenames can be overridden via the filenames: section in config/config.yaml.

Survey answers (metadata.json)

Keys and values are determined by the questions section in config/config.yaml. With the default config:

{
  "flow":          "Turbulent | Laminar | Uncertain",
  "shadows":       "Yes | No | Uncertain",
  "artifacts":     "Yes | No | Uncertain",
  "lighting":      "Day | Night | Uncertain",
  "exposure":      "Overexposed | Underexposed | Both | Normal | Uncertain",
  "snowing":       "Yes | No | Uncertain",
  "snow_on_ground":"Yes | No | Uncertain"
}

How it works

Clip format

Each clip is a ZIP archive containing a video file (default left.mp4, configurable via filenames.video_in_zip). The filename encodes the recording timestamp (e.g. left_20230615T120000.zip).

Frame loading

Up to max_frames frames are extracted from the video and scaled so the longest side is display_max px. This display-resolution copy is what the annotator works on; the full-resolution dimensions are remembered separately so the saved mask is upscaled back to the original size on export.

Mask drawing

The mask is a binary NumPy array matching the display frame size.

Brush: each stroke stamps a filled circle of the selected radius, setting pixels to 1 (draw) or 0 (erase).

Polygon: vertices are stored as a list of floating-point canvas coordinates. Multiple closed shapes can coexist. Completed shapes are rendered as cyan overlays on the canvas but do not touch the mask until a fill is applied.

Fill: clicking inside a closed polygon rasterises it with cv2.fillPoly and ORs the result into the mask. Among all shapes containing the click, the innermost (smallest area, determined by cv2.contourArea) is selected as the fill target. Any polygon whose centroid lies inside the target is then punched out as a hole.

Every mask-changing operation (brush stroke, fill) pushes the previous mask onto the undo stack before modifying it. On save the mask is resized to the original video resolution with nearest-neighbour interpolation and written as an 8-bit PNG (0 or 255).

Resuming

When a clip is loaded that already has a saved mask.png and metadata.json, the mask is restored at display resolution and the survey answers are pre-filled.

Repository structure

config/
    config.yaml                    # Your local config (git-ignored, copy from example)
    config.example.yaml            # Example config to copy and edit
    clips.txt                      # Your clip list (git-ignored, copy from example)
    clips.example.txt              # Example clip list
    optical_flow_config.yaml       # Optional optical flow parameters (enable via config.yaml)
src/river_annotation_tool/
    annotation_script.py           # Entry point — argument parsing and app launch
    annotator.py                   # Main QMainWindow — orchestrates all components
    clip_selector.py               # Reads the clip list and picks the next clip
    mask_canvas.py                 # Drawing widget — brush, undo, erase, mouse events
    video_loader.py                # ZIP extraction and frame resizing
    compute_optical_flow.py        # Optical flow river segmentation (Auto Segment button)
    config.py                      # AppConfig dataclass and YAML loader
    __init__.py                    # Package version
pyproject.toml              # Project metadata and dependencies

Development

# Install pre-commit hooks
pre-commit install
pre-commit run --all-files   # Run manually once

# Add a dependency
uv add <package>
uv add --dev <package>       # Development-only