Skip to content

feat: Command Line Interface #530

Description

@fdosani

Summary

DataComPy is currently a Python library only — users must write a script to compare two datasets. A command-line interface would make it immediately accessible for ad-hoc data validation, CI pipelines, and use by non-Python practitioners without any boilerplate.

Motivation

CICD pipelines often run as shell tasks (Airflow BashOperator, GitHub Actions, GitLab CI) where Python entry points are awkward. A native CLI that exits non-zero on mismatch and emits machine-readable JSON is the highest-leverage single CICD improvement DataComPy could add.

Other common friction points:

  • Ops / data engineers want to diff two CSV or Parquet files without opening a Python REPL
  • Analysts want a one-liner to spot-check a file diff without writing a throw-away script every time
  • Automation wants to shell out to datacompy compare ... and capture a structured result

Proposed interface

# Basic comparison
datacompy compare \
    --left s3://bucket/before.parquet \
    --right s3://bucket/after.parquet \
    --on id,date \
    --backend pandas \
    --max-unequal-rows 0 \
    --json
# exit 0 on match, 1 on mismatch, 2 on error

# Local files, HTML report written to file
datacompy compare \
    --left df1.csv \
    --right df2.csv \
    --on id \
    --output report.html

# Tolerances
datacompy compare \
    --left before.parquet \
    --right after.parquet \
    --on account_id \
    --abs-tol 0.01 \
    --rel-tol 0.001

# Ignore whitespace and case
datacompy compare \
    --left df1.csv \
    --right df2.csv \
    --on id \
    --ignore-spaces \
    --ignore-case

# Control report verbosity
datacompy compare \
    --left df1.csv \
    --right df2.csv \
    --on id \
    --sample-count 20 \
    --column-count 5

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Fields

    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions