Add dfbench statistics command

[gabotechs]

Which issue does this PR close?

• Part of Epic: Statistics improvements #8227

Rationale for this change

Provide a repeatable way to measure how closely planning cardinality statistics match runtime output for benchmark query suites.

What changes are included in this PR?

dfbench statistics accepts any Parquet data directory and SQL file or directory, so it can be used with TPC-DS, TPC-H, or another compatible query suite. It:

• prints an indented, per-operator estimate-versus-runtime report as each query completes;
• reports q-error per operator and finite q-error p50/p75/p95/p99 across the run;
• stores results by branch, comparing with the previous run by default or a named branch using --compare.

About q-error calculation: https://vldb.org/pvldb/vol9/p204-leis.pdf

Example: verify a TPC-DS Q21 improvement

1. Run TPC-DS Q21 for storing the baseline

cargo run --bin dfbench statistics \
  --path benchmarks/data/tpcds_sf1 \
  --query_path datafusion/core/tests/tpc-ds \
  --query 21

2. Perform this small change in https://github.com/apache/datafusion/blob/main/datafusion/physical-plan/src/sorts/sort.rs#L1427-L1427:

-        Ok(Arc::new(stats.with_fetch(self.fetch, 0, 1)?))
+        Ok(Arc::new(stats.with_fetch(
+            self.fetch,
+            0,
+            self.topk_emitter_count(),
+        )?))

3. Run again the command from step 1.

=== 21 ===
SortPreservingMergeExec: rows=Inexact(100) vs 100, q-error: previous=1.00x, current=1.00x, change=0.0%
  SortExec(TopK): rows=Inexact(1600) vs 1417, q-error: previous=14.40x, current=1.13x, change=✅ 92.2%
    ....

Compare against a baseline from main

# Run on main first; the result is stored under main
cargo run --bin dfbench statistics \
  --path benchmarks/data/tpcds_sf1 \
  --query_path datafusion/core/tests/tpc-ds \
  --query 21

# Then, on a feature branch
cargo run --bin dfbench statistics \
  --path benchmarks/data/tpcds_sf1 \
  --query_path datafusion/core/tests/tpc-ds \
  --query 21 \
  --compare main

TPC-H works with its partitioned data layout as well:

cargo run --bin dfbench statistics \
  --path benchmarks/data/tpch_sf1 \
  --query_path benchmarks/queries \
  --query q1

Are these changes tested?

No unit tests were added: this is a reporting CLI over planner statistics and runtime metrics, exercised by the benchmark-suite smoke runs above.

If reviewers think this type of code needs to be covered by unit tests let me know.

Are there any user-facing changes?

No, this is just for extending the dfbench command with a new dfbench statistics subcommand

[codecov-commenter]

Codecov Report

❌ Patch coverage is 0% with 434 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.65%. Comparing base (68d5874) to head (8b7b5ac).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
benchmarks/src/statistics.rs	0.00%	433 Missing ⚠️
benchmarks/src/bin/dfbench.rs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #23975      +/-   ##
==========================================
- Coverage   80.75%   80.65%   -0.10%     
==========================================
  Files        1096     1097       +1     
  Lines      373282   373847     +565     
  Branches   373282   373847     +565     
==========================================
+ Hits       301440   301526      +86     
- Misses      53869    54330     +461     
- Partials    17973    17991      +18     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.