A desktop application for manually annotating river video clips as part of the [HydroScan](https://github.com/HydroScan) project. Annotators draw pixel-level water masks over river footage and answer structured survey questions about flow conditions, lighting, and scene quality.
By default the tool reads clips from and writes annotations to the local filesystem (`storage: local`). To use an S3-compatible object store instead, set `storage: s3` in `config/config.yaml` and give `data_dir` / `out_dir` as `bucket/prefix` paths:
```yaml
storage: s3
data_dir: my-bucket/clips
out_dir: my-bucket/annotation_results
```
Copy `.env.example` to `.env` and fill in your credentials — the app loads this file automatically at startup:
```sh
cp .env.example .env
# edit .env with your credentials
```
| Variable | Description |
|---|---|
| `S3_ACCESS_KEY` | Access key ID |
| `S3_SECRET_ACCESS_KEY` | Secret access key |
| `S3_ENDPOINT_URL` | Endpoint URL (defaults to `https://os.zhdk.cloud.switch.ch` if not set) |
| `AWS_REQUEST_CHECKSUM_CALCULATION` | Set to `when_required` to avoid checksum errors on SwitchEngines/Ceph |
| `AWS_RESPONSE_CHECKSUM_VALIDATION` | Set to `when_required` to avoid checksum errors on SwitchEngines/Ceph |
The `clips_file` (the list of clip filenames to annotate) is always read from the local filesystem even when `storage: s3`.
Add, remove, or reorder questions directly in the YAML — the UI rebuilds automatically. `key` is what gets saved in `metadata.json`; `default` selects the pre-checked option (omit or set to `null` to leave unselected).
Set `optical_flow_config_file` in `config.yaml` to point to a YAML file that enables the **Auto Segment** button. When pressed, the tool computes a river mask from the loaded frames and replaces the current mask (undoable). The segmentation combines two criteria:
- **Optical flow magnitude** — pixels where the temporal median of frame-to-frame flow (scaled by FPS) exceeds a fraction of the maximum are considered moving water.
- **Brightness** — pixels outside a brightness window are excluded (removes sky, saturated glare, etc.).
```yaml
# config/optical_flow_config.yaml
enabled: true
norm_squared_threshold: 0.06 # fraction of max flow² that counts as moving
gaussian_kernel: [5, 5] # blur kernel applied to the reference frame before brightness check
brightness_range: [2, 253] # [min, max] greyscale brightness to keep
```
`enabled: false` disables the button without removing the config file.
`config/clips.txt` lists the clip filenames to annotate, one per line. Lines starting with `#` are ignored. Clips are processed in order; already-annotated clips (those with an existing `mask.png`) are skipped automatically. Pass `--no-skip` to include them. When the last clip is reached, a dialog appears and the app exits.
The window is split into two panels: the **video canvas** on the left (~70% of the width) and the **survey panel** on the right. The video auto-plays as a looping preview. Drawing tools and mask controls are arranged above and beside the canvas; navigation buttons (**Previous / Next / Skip**) sit at the top.
| Load mask from previous clip | **Load Prev Mask** — copies the saved mask of the previous clip onto the current one; undoable |
| Optical flow first guess | **Auto Segment** — replaces the current mask with an automatic river segmentation; undoable. Only available when `optical_flow_config_file` is set in `config.yaml`. |
| Save and continue | **Next** — saves current clip and loads the next one. If the clip already has a saved annotation a dialog asks whether to replace it or keep the existing save. |
| Go back | **Previous** — saves current clip and returns to the previously viewed clip. Disabled on the first clip. |
| Skip without saving | **Skip** — discards any unsaved changes and loads the next clip without writing anything to disk. |
Each clip is a ZIP archive containing a video file (default `left.mp4`, configurable via `filenames.video_in_zip`). The filename encodes the recording timestamp (e.g. `left_20230615T120000.zip`).
Up to `max_frames` frames are extracted from the video and scaled so the longest side is `display_max` px. This display-resolution copy is what the annotator works on; the full-resolution dimensions are remembered separately so the saved mask is upscaled back to the original size on export.
The mask is a binary array at display resolution. **Brush** strokes stamp a filled circle (draw or erase). **Polygon** shapes are stored as overlays and don't touch the mask until a **Fill** click rasterises them — the innermost polygon containing the click is filled, and any polygon whose centroid falls inside it is punched out as a hole.
Every mask-changing operation is pushed onto an undo stack before it executes. On save, the mask is upscaled to the original video resolution and written as an 8-bit PNG (0 or 255).
When a clip is loaded that already has a saved `mask.png` and `metadata.json`, the mask is restored at display resolution and the survey answers are pre-filled.
