Add S3 storage support via s3fs; make storage field required

- New filesystem.py: make_fs() factory (returns s3fs.S3FileSystem or None),
  plus fsjoin/fsstem/fsname path helpers
- config.py: storage field is now required ('local' or 's3'); load_config
  raises a clear ValueError when it is missing
- video_loader, clip_selector, annotator: thread fs through all file I/O;
  local paths unchanged, S3 paths use fs.open/fs.exists/fs.pipe
- annotation_script: load .env via python-dotenv at startup, create fs from
  config and pass to Annotator
- Add .env.example with SwitchEngines endpoint and AWS checksum env vars
- pyproject.toml: add s3fs and python-dotenv dependencies
- Reduce default mask alpha from 40% to 15%
- Update example clip names to colon-separated timestamps

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-05-20 16:15:38 +02:00
parent 8579bad2e2
commit dc59b8affb
15 changed files with 1539 additions and 106 deletions

View File

@@ -1,5 +1,13 @@
# For local storage, set data_dir and out_dir to file-system paths:
storage: local # 'local' (default) or 's3'
data_dir: data/filtered_data
out_dir: data/annotation_results
# For S3 storage, set storage: s3 and use bucket/prefix paths:
# storage: s3
# data_dir: my-bucket/clips
# out_dir: my-bucket/annotation_results
# Credentials are read from env vars (copy .env.example to .env):
# S3_ACCESS_KEY, S3_SECRET_ACCESS_KEY, S3_ENDPOINT_URL
clips_file: config/clips.txt
optical_flow_config_file: config/optical_flow_config.yaml