Add S3 storage support via s3fs; make storage field required
- New filesystem.py: make_fs() factory (returns s3fs.S3FileSystem or None),
plus fsjoin/fsstem/fsname path helpers
- config.py: storage field is now required ('local' or 's3'); load_config
raises a clear ValueError when it is missing
- video_loader, clip_selector, annotator: thread fs through all file I/O;
local paths unchanged, S3 paths use fs.open/fs.exists/fs.pipe
- annotation_script: load .env via python-dotenv at startup, create fs from
config and pass to Annotator
- Add .env.example with SwitchEngines endpoint and AWS checksum env vars
- pyproject.toml: add s3fs and python-dotenv dependencies
- Reduce default mask alpha from 40% to 15%
- Update example clip names to colon-separated timestamps
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -1,10 +1,10 @@
|
||||
# List the clip filenames (without path) to annotate, one per line.
|
||||
# Lines starting with # are ignored. Order is preserved.
|
||||
GRAMMONT_2025-11-17T11_31_38.546953+00_00.zip
|
||||
GRAMMONT_2025-11-17T12_31_39.650554+00_00.zip
|
||||
GRAMMONT_2025-11-17T15_32_07.184007+00_00.zip
|
||||
GRAMMONT_2025-11-17T15_32_07.184007+00_00.zip
|
||||
GRAMMONT_2025-11-17T15_47_10.070449+00_00.zip
|
||||
GRAMMONT_2025-11-22T10_47_02.705611+00_00.zip
|
||||
GRAMMONT_2025-11-22T14_47_00.096714+00_00.zip
|
||||
GRAMMONT_2025-11-22T15_32_01.015469+00_00.zip
|
||||
GRAMMONT_2025-11-17T11:31:38.546953+00:00.zip
|
||||
GRAMMONT_2025-11-17T12:31:39.650554+00:00.zip
|
||||
GRAMMONT_2025-11-17T15:32:07.184007+00:00.zip
|
||||
GRAMMONT_2025-11-17T15:32:07.184007+00:00.zip
|
||||
GRAMMONT_2025-11-17T15:47:10.070449+00:00.zip
|
||||
GRAMMONT_2025-11-22T10:47:02.705611+00:00.zip
|
||||
GRAMMONT_2025-11-22T14:47:00.096714+00:00.zip
|
||||
GRAMMONT_2025-11-22T15:32:01.015469+00:00.zip
|
||||
|
||||
@@ -1,5 +1,13 @@
|
||||
# For local storage, set data_dir and out_dir to file-system paths:
|
||||
storage: local # 'local' (default) or 's3'
|
||||
data_dir: data/filtered_data
|
||||
out_dir: data/annotation_results
|
||||
# For S3 storage, set storage: s3 and use bucket/prefix paths:
|
||||
# storage: s3
|
||||
# data_dir: my-bucket/clips
|
||||
# out_dir: my-bucket/annotation_results
|
||||
# Credentials are read from env vars (copy .env.example to .env):
|
||||
# S3_ACCESS_KEY, S3_SECRET_ACCESS_KEY, S3_ENDPOINT_URL
|
||||
clips_file: config/clips.txt
|
||||
optical_flow_config_file: config/optical_flow_config.yaml
|
||||
|
||||
|
||||
Reference in New Issue
Block a user