Skip to content

feat: add datacompy compare CLI with multi-backend support#534

Open
fdosani wants to merge 21 commits into
capitalone:mainfrom
capitalone-contributions:datacompy-cli-feature
Open

feat: add datacompy compare CLI with multi-backend support#534
fdosani wants to merge 21 commits into
capitalone:mainfrom
capitalone-contributions:datacompy-cli-feature

Conversation

@fdosani

@fdosani fdosani commented Jun 25, 2026

Copy link
Copy Markdown
Member

Implements the datacompy compare CLI (closes #530), enabling dataset comparisons from the shell and CI/CD pipelines without writing Python.

  • Adds datacompy/cli/ subpackage: parser.py, compare.py, backends.py, loaders.py, sessions.py, output.py, errors.py
  • Supports all four backends: polars (default), pandas, spark, snowflake
  • Loads CSV, Parquet, and JSON files; Snowflake can compare native table refs (db.schema.table) or stage local files to a temp table
  • Exit codes: 0 match, 1 mismatch/threshold violated, 2 error
  • --max-unequal-rows N for threshold-based CI assertions
  • --json for machine-readable output; --quiet for exit-code-only pipelines
  • --debug re-raises unexpected exceptions for bug reporting
  • Argument validation split between argparse type= callables (value rules) and _validate_arg_combinations (cross-argument rules)
  • Session lifecycle managed via contextlib.ExitStack — Spark and Snowflake sessions close on both success and failure
  • All optional backend imports are deferred so datacompy --version does not pay the pyspark/snowflake import cost

fdosani added 21 commits June 22, 2026 16:24
…s and improve error handling for missing Snowflake config
@fdosani fdosani marked this pull request as ready for review June 25, 2026 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Command Line Interface

1 participant