Skip to content

Expand and harden the correspondence engine#132

Merged
jcushman merged 113 commits into
mainfrom
correspondence-hardening
Jul 10, 2026
Merged

Expand and harden the correspondence engine#132
jcushman merged 113 commits into
mainfrom
correspondence-hardening

Conversation

@jcushman

@jcushman jcushman commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

This PR advances binoc's correspondence-first engine with a coordinated set of dataset configuration, tabular analysis, rendering, performance, plugin catalog, and test-infrastructure changes.

  • Makes dataset.paths the sole per-path and per-table configuration surface, with descriptor-derived dispatch, content_type and forced-rule overrides, CSV dialect and shape facets, nested JSON records_path, row identity, structured-document node identity, and explicit validation for invalid facet/rule combinations.
  • Keeps dataset.defaults.row_identity as the global default and uses presence-aware paths[].row_identity patches so named and positional selectors and null/duplicate policies override correctly. Removed the obsolete dataset.tables model, flat row-identity aliases, selector merge/fallback code, and coexistence tests.
  • Expands tabular edit-list rewriting with column rename and reorder composition, type-only column changes, sorted row alignment, reduced-precision and suppression claims, row-addition consolidation, and a documented single-pass rule order.
  • Adds inference disclosure and conservative guardrails for auto-detected row keys, dialects, high-churn positional comparisons, and uncertain row correspondence.
  • Improves CSV parsing for declared and inferred delimiters, quoted fields, decimal-comma data, banner/header lines, extensionless inputs, and false-positive resistance.
  • Improves large-table performance with shared artifact decoding, a faster keyed writer, configurable in-memory thresholds, bounded streaming keyed comparisons, recorded performance baselines, and user-facing tuning documentation.
  • Adds bounded binary CDC range localization, deterministic source-materialized binary fixtures, UTF-8 boundary-safe content sniffing, and extensionless text/binary classification.
  • Streams archive and gzip expansion with per-entry and total decompression caps, incremental hashing, cleanup of partial outputs, and clear fallback diagnostics.
  • Tightens semantic writer ownership so parsed tabular or structured content is not also reported as raw text, while parser metadata and structural container facts still compose independently.
  • Makes projection and Markdown rendering more rule-driven through generic structured summaries, producer-backed content_summary facts for modified moves, bounded and round-robin examples, configurable verbosity/grouping, deduplicated details, humanized counts, and renderer-ignorance invariants. Removed the dead tabular_summary renderer channel.
  • Adds offline feedback-report bundles, reproducible snapshot capture/reference modes, and extract commands in configuration diagnostics.
  • Aligns the plugin catalog and Python packaging with the fat-binoc distribution. Removed the contradictory binoc[all] extra and the ignored registry= argument from binoc.diff.
  • Extends SDK and engine support for projection annotations, package identifiers, per-item dispatch hints, artifact decode caching, fallback writer composition, and source-aware extract hints without adding format knowledge to core.
  • Adds and strengthens behavioral test vectors, structural invariants, description-cost baselines, stdlib/plugin lints, generated gallery/replay documentation, and the native renderer ABI canary.
  • Updates active docs and ADR statuses to make the per-path and fat-wheel decisions authoritative while preserving historical ADR text.

Validation:

  • just fmt
  • just check
  • just test
  • just lint
  • UV_NO_CACHE=1 just docs
  • git diff --check main...correspondence-hardening

(Auto generated pull request)

jcushman added 30 commits July 9, 2026 16:08
Three accepted ADRs seed this round of foreman-organized work:
- Unified per-path dataset config model (selector + facets; dispatch
  override resolved before the tabular gate)
- Inference disclosed + overridable; high-churn guardrail as backstop
- Renderers are ecosystem-ignorant; ugly output from an unknown rule is
  an upstream bug
@jcushman
jcushman marked this pull request as ready for review July 10, 2026 15:09
@jcushman
jcushman merged commit c8206d7 into main Jul 10, 2026
10 checks passed
@jcushman
jcushman deleted the correspondence-hardening branch July 10, 2026 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant