cam-sort

Sort surveillance camera image uploads into date-based folders, with optional hour-range bucketing, ownership fixes, and per-device retention. Designed to run unattended from cron.

Motivation

IP cameras typically dump every captured frame into a single SFTP staging directory, producing thousands of files per day. Browsing or backing up that pile is impractical, and old material accumulates indefinitely. cam-sort moves new uploads into a tidy YYYY-MM-DD/ layout on the archive volume and optionally trims folders past a configurable retention window — so the staging area stays empty, the archive stays navigable, and disk usage stays bounded.

Features

• Sorts each image into <target>/YYYY-MM-DD/, optionally further split by hour range (e.g. 08:00-18:59/).
• Extracts the timestamp from the filename (default) or EXIF (read_exif: true).
• Per-device configurable filename regex with named groups; a sensible default pattern is built in.
• Falls back to the file's mtime when the filename does not encode the year.
• Per-device retention (retention: 90d) deletes old date folders; unconfigured devices are never touched.
• Optional chown of the moved file and the target directory.
• Safe-by-default permissions: every moved file is chmoded (default 640) and every created directory (default 750); both modes are configurable globally and per device.
• Skips a device cleanly (with an error on stderr) when its source or target is missing or misconfigured; remaining devices continue.
• --dry-run previews every action without touching the filesystem.

Note: read_exif: true is only as accurate as the camera's clock. If the device clock is wrong (or never synced via NTP), images will be filed under the wrong date. Prefer the filename-based default unless you are sure the camera time is reliable.

Requirements

• Python 3.6+
• PyYAML
• Pillow (only when any device sets read_exif: true)

Configuration

Default path: ~/.config/cam-sort.yaml. Override with -c. A full annotated example ships next to the script as cam-sort.yaml.

config:
  file_mode: 640                  # global default for moved files
  path_mode: 750                  # global default for created directories

devices:
  cam-min:
    source_path: /data/ftp/cam-min
    target_path: /data/cloud/cam-min

  cam-max:
    source_path: /data/ftp/cam-max
    target_path: /data/cloud/cam-max
    read_exif: false
    filename_pattern: '(?P<year>\d{2})-(?P<month>\d{2})-(?P<day>\d{2})_(?P<hour>\d{2})-(?P<minute>\d{2})-(?P<second>\d{2})'
    owner: apache
    group: apache
    file_mode: 640                # per-device override of global file_mode
    path_mode: 750                # per-device override of global path_mode
    create_target: true
    retention: 90d
    group_hours:
      - 00:00-07:59
      - 08:00-18:59
      - 19:00-24:00

Allowed image extensions can be overridden globally via config.filenames. Permission modes are octal (digits 0-7, e.g. 640); per-device values win over config:, which in turn wins over the built-in defaults 640/750.

Usage

cam-sort.py [-c CONFIG] [--dry-run] [-v | -q]

• -c, --config — path to the YAML config (default: ~/.config/cam-sort.yaml)
• --dry-run — show planned actions, change nothing on disk
• -v, --verbose — log every individual file move
• -q, --quiet — suppress stdout (errors still go to stderr); intended for cron

Errors are always written to stderr; a non-zero exit code means the config itself was unusable.

Cron

Setup is left to the operator. A typical entry runs the script every few minutes in quiet mode:

*/5 * * * * /usr/local/bin/cam-sort.py -q