Skip to content

Support filename pattern grouping in cross-validation CLI#406

Merged
gbeane merged 1 commit into
mainfrom
feature/cv-cli-filename-pattern-grouping
Jun 23, 2026
Merged

Support filename pattern grouping in cross-validation CLI#406
gbeane merged 1 commit into
mainfrom
feature/cv-cli-filename-pattern-grouping

Conversation

@gbeane

@gbeane gbeane commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

The filename-pattern cross-validation grouping strategy (added in #398) was wired into the GUI, Project.get_labeled_features, and project settings, but the jabs-cli cross-validation command was left incomplete — there was no way to select or configure it from the CLI. This PR closes that gap.

What was missing

  1. --grouping-strategy only accepted video and individualFILENAME_PATTERN was unreachable.
  2. run_cross_validation had no grouping_regex parameter, so even if the strategy could be selected, no regex could reach the layer that requires it (compile_grouping_regex raises on an empty pattern).
  3. The training report read grouping strategy/regex directly from project settings, so a CLI override would not be reflected in the report.

Changes

scripts/cli/cli.py

  • Added filename to the --grouping-strategy choices.
  • Added a --grouping-pattern option for the regex (falls back to the project's saved pattern when omitted).
  • Replaced the if/elif chain with a name→enum mapping and forwarded the pattern as grouping_regex.

scripts/cli/cross_validation.py

  • Added a grouping_regex: str | None parameter, documented it, and passed it to get_labeled_features.
  • Resolved the effective strategy/regex (CLI override → project settings) so the training report records what was actually used. The regex is only emitted when the effective strategy is FILENAME_PATTERN (also fixes a pre-existing minor issue where a stale saved regex could appear in the report for non-filename strategies).

tests/scripts/test_cross_validation_cli.py (new)

  • 7 tests covering enum mapping (incl. case-insensitivity), regex forwarding, None defaults, and rejection of unknown strategies, using a run_cross_validation spy so they're fast and need no real project.

Example

uv run jabs-cli cross-validation /path/to/project --behavior Walk \
    --grouping-strategy filename --grouping-pattern '^(\w+?)_'

Testing

  • New tests: 7 passed. Related grouping suites (test_cv_grouping in core and project): 23 passed total.
  • ruff check clean; ruff format applied.
  • jabs-cli cross-validation --help renders both new options correctly.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR completes wiring of the filename-pattern cross-validation grouping strategy into the jabs-cli cross-validation command, allowing the strategy and its regex to be selected/configured via CLI and ensuring the training report reflects the effective settings actually used.

Changes:

  • Extended --grouping-strategy to include filename and added --grouping-pattern for the regex override.
  • Threaded an optional grouping_regex through run_cross_validation() into Project.get_labeled_features().
  • Updated report generation to record the effective grouping strategy/regex (CLI override → project settings), emitting the regex only for filename-pattern grouping.
  • Added CLI parsing tests using a run_cross_validation spy for fast option→argument verification.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
src/jabs/scripts/cli/cli.py Adds filename grouping option, --grouping-pattern, and maps CLI strings to the enum; forwards regex override.
src/jabs/scripts/cli/cross_validation.py Adds grouping_regex parameter, forwards to feature extraction, and records effective grouping settings in the report.
tests/scripts/test_cross_validation_cli.py New tests validating Click parsing/mapping and argument forwarding to run_cross_validation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@gbeane gbeane requested review from bergsalex and keithshep June 23, 2026 17:18
@gbeane gbeane self-assigned this Jun 23, 2026
@gbeane gbeane merged commit 14b0b6c into main Jun 23, 2026
6 checks passed
@gbeane gbeane deleted the feature/cv-cli-filename-pattern-grouping branch June 23, 2026 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants