Add S3 storage support via s3fs; make storage field required

- New filesystem.py: make_fs() factory (returns s3fs.S3FileSystem or None),
  plus fsjoin/fsstem/fsname path helpers
- config.py: storage field is now required ('local' or 's3'); load_config
  raises a clear ValueError when it is missing
- video_loader, clip_selector, annotator: thread fs through all file I/O;
  local paths unchanged, S3 paths use fs.open/fs.exists/fs.pipe
- annotation_script: load .env via python-dotenv at startup, create fs from
  config and pass to Annotator
- Add .env.example with SwitchEngines endpoint and AWS checksum env vars
- pyproject.toml: add s3fs and python-dotenv dependencies
- Reduce default mask alpha from 40% to 15%
- Update example clip names to colon-separated timestamps

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-05-20 16:15:38 +02:00
parent 8579bad2e2
commit dc59b8affb
15 changed files with 1539 additions and 106 deletions

View File

@@ -1,10 +1,10 @@
# List the clip filenames (without path) to annotate, one per line.
# Lines starting with # are ignored. Order is preserved.
GRAMMONT_2025-11-17T11_31_38.546953+00_00.zip
GRAMMONT_2025-11-17T12_31_39.650554+00_00.zip
GRAMMONT_2025-11-17T15_32_07.184007+00_00.zip
GRAMMONT_2025-11-17T15_32_07.184007+00_00.zip
GRAMMONT_2025-11-17T15_47_10.070449+00_00.zip
GRAMMONT_2025-11-22T10_47_02.705611+00_00.zip
GRAMMONT_2025-11-22T14_47_00.096714+00_00.zip
GRAMMONT_2025-11-22T15_32_01.015469+00_00.zip
GRAMMONT_2025-11-17T11:31:38.546953+00:00.zip
GRAMMONT_2025-11-17T12:31:39.650554+00:00.zip
GRAMMONT_2025-11-17T15:32:07.184007+00:00.zip
GRAMMONT_2025-11-17T15:32:07.184007+00:00.zip
GRAMMONT_2025-11-17T15:47:10.070449+00:00.zip
GRAMMONT_2025-11-22T10:47:02.705611+00:00.zip
GRAMMONT_2025-11-22T14:47:00.096714+00:00.zip
GRAMMONT_2025-11-22T15:32:01.015469+00:00.zip

View File

@@ -1,5 +1,13 @@
# For local storage, set data_dir and out_dir to file-system paths:
storage: local # 'local' (default) or 's3'
data_dir: data/filtered_data
out_dir: data/annotation_results
# For S3 storage, set storage: s3 and use bucket/prefix paths:
# storage: s3
# data_dir: my-bucket/clips
# out_dir: my-bucket/annotation_results
# Credentials are read from env vars (copy .env.example to .env):
# S3_ACCESS_KEY, S3_SECRET_ACCESS_KEY, S3_ENDPOINT_URL
clips_file: config/clips.txt
optical_flow_config_file: config/optical_flow_config.yaml