Data processing pipeline: I3D/CLIP feature extraction + scalable dataset strategy

[jinmang2]

Goal

Complete the data-processing layer — feature extraction + an efficient,
scriptable dataset strategy — for current UCF-Crime and future AIHub / vision
benchmarks. (Model faithful-ports are done & merged in dev; see docs/REPRO_STATUS.md.)

Tasks

• I3D extraction: convert GowthamGottimukkala caffe2 weights → torch
  (pretrained/i3d/.pkl), reproduce RTFM-style 10-crop I3D extraction,
  cross-check feature stats vs UCF__ten_crop_i3d.
• open_clip ViT-B/16 CLIP feature extraction → reproduce UCFClipFeatures.
• Dataset strategy: WebDataset/tar shards vs zip-direct vs npy; manifest as the
  single source of truth (format-agnostic); throughput/IO benchmarks; scriptify.
• Stats matching: verify extracted features match published dataset statistics
  (scale, dim, segment counts).
• Future: AIHub anomaly video / vision data ingestion for I3D training +
  additional benchmarks.

Blockers / deps

• decord + pytorchvideo missing in the balaenoptera env (extraction blocked).
• GPU extraction over ~1900 videos (RTX 2070S) is multi-hour.

Out of scope (done)

• Faithful model ports — 10 models verified (7 output-verified vs official, 3
  paper-faithful), merged in dev. See docs/REPRO_STATUS.md §0.

Refs: docs/REPRO_STATUS.md §3 (I3D extraction plan), §4 (dataset strategy).