feat[edition]: session-registered editions defitions by joseph-isaacs · Pull Request #8871 · vortex-data/vortex

joseph-isaacs · 2026-07-21T09:20:06Z

Rationale for this change

Any encoding that vortex will write using built-in layout strategies or compressors will come from an edition.

What changes are included in this PR?

Adds the vortex-editions crate and session variable.
Defines editions for core and unstable encodings from vortex 0.36.0 until today

github-actions · 2026-07-21T09:22:43Z

Polar Signals Profiling Results

Latest Run

Status	Commit	Job	Attempt	Link
🟢 Done	`4d00b07`		1	Explore Profiling Data

Previous Runs (11)

Status	Commit	Attempt	Link
🟢 Done	`44d1d4f`	1	Explore Profiling Data
🟢 Done	`3e8141e`	1	Explore Profiling Data
🟢 Done	`07e5746`	1	Explore Profiling Data
🟢 Done	`a4f07a8`	1	Explore Profiling Data
🟢 Done	`a07aa2e`	1	Explore Profiling Data
🟢 Done	`528cd3f`	1	Explore Profiling Data
🟢 Done	`89bcb4a`	1	Explore Profiling Data
🟢 Done	`e570448`	1	Explore Profiling Data
🟢 Done	`984cee4`	1	Explore Profiling Data
🟢 Done	`bb67435`	1	Explore Profiling Data
🟢 Done	`b653c99`	1	Explore Profiling Data

Powered by Polar Signals Cloud

github-actions · 2026-07-21T09:23:37Z

Benchmarks: Vortex queries 📖

Verdict: Likely regression (medium confidence)
Attributed Vortex impact: +12.6%
Engines: DataFusion Likely regression (+20.9%, high confidence) · DuckDB No clear signal (+4.8%, low confidence)
Vortex (geomean): 1.157x ❌
Parquet (geomean): 1.090x ➖
Shifts: Parquet (control) +9.0% · Median polish +11.6%

How to read Verdict and Engines

Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (1.332x ❌, 0↑ 2↓)

name	PR `4d00b07` (ns)	base `e767a30` (ns)	ratio (PR/base)
vortex_q00/datafusion:vortex-file-compressed 🚨	12721249	9825439	1.29
vortex_q01/datafusion:vortex-file-compressed 🚨	8741795	6382956	1.37

datafusion / parquet (1.102x ❌, 0↑ 1↓)

name	PR `4d00b07` (ns)	base `e767a30` (ns)	ratio (PR/base)
vortex_q00/datafusion:parquet	23011780	21147036	1.09
vortex_q01/datafusion:parquet 🚨	5676780	5088668	1.12

duckdb / vortex-file-compressed (1.131x ❌, 0↑ 2↓)

name	PR `4d00b07` (ns)	base `e767a30` (ns)	ratio (PR/base)
vortex_q00/duckdb:vortex-file-compressed 🚨	12440643	10812350	1.15
vortex_q01/duckdb:vortex-file-compressed 🚨	7171360	6455096	1.11

duckdb / parquet (1.079x ➖, 0↑ 1↓)

name	PR `4d00b07` (ns)	base `e767a30` (ns)	ratio (PR/base)
vortex_q00/duckdb:parquet 🚨	26449013	23704975	1.12
vortex_q01/duckdb:parquet	9968822	9561482	1.04

No file size changes detected.

codspeed-hq · 2026-07-21T09:24:33Z

Merging this PR will not alter performance

✅ 1796 untouched benchmarks
⏩ 46 skipped benchmarks¹

_{Comparing claude/editions-impl-fiicrg (4d00b07) with develop (e767a30)}

46 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

…tex facade vortex-edition provides the editions machinery: the types (EditionId, Edition, EditionInclusion, EditionDeclaration), the EditionSession session variable holding the per-session registry (starts empty; clones share the registry like other session registries), computation (editions(), encodings_in() with membership inherited by later editions of the family, current()), validation (identifier/version forms via EditionId::validate and EditionInclusion::validate, plus whole-registry checks: undeclared references, drafts-newest chronology, members requiring a release newer than their edition declares), and the validate!(session, EDITION) macro generating a per-edition unit test. Encoding ids use the session's intern pool (registry::Id, the same identifier space as ArrayId), and declaration blocks name encodings via the AsEncodingId trait — an id string today, an encoding vtable once vortex-array implements it. The actual declarations live in the public vortex crate (vortex::editions): two core editions declared block-wise with the encodings each adds — core2026.01.0 with the 12 canonical encodings and core2026.07.0 inheriting them plus the 20 container/compressed encodings the default writer emits. Both are drafts until their min_vortex_version is recorded, which is the act of freezing. register_default_editions seeds them into the default session, golden tests pin both sets, and transitivity/consistency tests cover inheritance and the one-inclusion-per-encoding invariant. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KdR42Svu74NcNzC7XJYwLr

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

robert3005

only had one question

robert3005 · 2026-07-21T13:33:18Z

+    fn encoding_id(&self) -> Id;
+}
+
+impl AsEncodingId for str {


Do we use this impl anywhere?

I think that is used for each definition since they are defined by strs

Adds a frozen, additive editions catalog mirroring the ground-truth data from vortex-data/vortex#8871 (the published spec's own registry section says "Coming soon", so the catalog was built directly from the upstream Rust source), a writer-side edition guard, and reader unknown-encoding UX naming which edition an id belongs to. core.model: - EditionId, Edition, EditionFamily (a closed CORE/UNSTABLE enum, not a sealed-interface-plus-Custom like EncodingId/LayoutId: a private edition family carries no real cross-implementation guarantee, so there's no legitimate custom-family use case) - Editions catalog: the 8 frozen/draft editions, cumulativeMembers(), owningEdition() writer: - WriteOptions gains an editions field, defaulting to the latest frozen core edition (core2026.07.0) - verified safe: the default cascade candidate list never includes the two unstable-family encoders vortex-java implements (Delta, Patched) - withEdition(Edition) / withoutEditionGuard() (the escape hatch) - VortexWriter's guard has two layers: it seeds the out-of-edition encoding ids into EncodeContext's existing exclusion set so CascadingCompressor gracefully steers away from them during cost-based selection (including nested competitions like a masked column's validity-bitmap cascade - reachable via ordinary cascading(depth) writes, not just an explicit custom encoder list), plus an after-the-fact backstop check for paths that don't consult the exclusion set at all reader: - ReadRegistry's unknown-encoding VortexException now names the edition an id belongs to and its minimum Vortex version (or that it's an unstable draft with no guarantee), or says it's unknown to every edition and points at allowUnknown() No wire-format changes - editions are a write-time/read-time policy, never persisted into the file (ADR 0023, which also records the mid-implementation course correction from a simple after-the-fact check to the two-layer filter+backstop design, and why EditionFamily ended up closed rather than mirroring EncodingId's Custom pattern). Mechanical fallout: WriteOptions' 8th record component required updating 29 direct-constructor call sites (Map.of(), preserving today's no-guard behavior) plus 3 existing test files that deliberately write unstable-family encoders through the explicit encoder-list overload (added .withoutEditionGuard()). Verified: full reactor build, full unit suite, full integration suite (including the real-world file-size comparison tests that exercise cascading writes extensively), core module javadoc build. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

joseph-isaacs force-pushed the claude/editions-impl-fiicrg branch from b653c99 to bb67435 Compare July 21, 2026 09:35

joseph-isaacs changed the title ~~feat(vortex-edition): session-registered editions with declaration macros~~ feat(vortex-edition): session-registered editions validated by unit tests Jul 21, 2026

joseph-isaacs force-pushed the claude/editions-impl-fiicrg branch 4 times, most recently from 89bcb4a to 528cd3f Compare July 21, 2026 09:50

joseph-isaacs marked this pull request as draft July 21, 2026 09:56

joseph-isaacs force-pushed the claude/editions-impl-fiicrg branch 2 times, most recently from a07aa2e to a4f07a8 Compare July 21, 2026 10:07

joseph-isaacs changed the title ~~feat(vortex-edition): session-registered editions validated by unit tests~~ feat(vortex-edition): session-registered editions declared in the vortex facade Jul 21, 2026

joseph-isaacs force-pushed the claude/editions-impl-fiicrg branch from a4f07a8 to 07e5746 Compare July 21, 2026 10:13

update

3e8141e

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

joseph-isaacs changed the title ~~feat(vortex-edition): session-registered editions declared in the vortex facade~~ feat[edition]: session-registered editions defitions Jul 21, 2026

joseph-isaacs requested a review from robert3005 July 21, 2026 10:50

joseph-isaacs marked this pull request as ready for review July 21, 2026 10:50

joseph-isaacs added the changelog/feature A new feature label Jul 21, 2026

joseph-isaacs added 2 commits July 21, 2026 11:54

update

44d1d4f

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

fix(vortex): qualify edition rustdoc links

4d00b07

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

joseph-isaacs mentioned this pull request Jul 21, 2026

feat(xtask): auto-generate the edition registry docs from the Rust declarations #8878

Draft

robert3005 approved these changes Jul 21, 2026

View reviewed changes

joseph-isaacs merged commit 49c4753 into develop Jul 21, 2026
86 checks passed

joseph-isaacs deleted the claude/editions-impl-fiicrg branch July 21, 2026 13:47

joseph-isaacs mentioned this pull request Jul 21, 2026

feat(vortex-file): gate the file writer behind editions #8877

Closed

This was referenced Jul 24, 2026

feat: adopt the Vortex "editions" model (frozen encoding sets + writer edition guard) dfa1/vortex-java#301

Closed

feat: adopt the Vortex editions model dfa1/vortex-java#320

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat[edition]: session-registered editions defitions - #8871

feat[edition]: session-registered editions defitions#8871
joseph-isaacs merged 4 commits into
developfrom
claude/editions-impl-fiicrg

joseph-isaacs commented Jul 21, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 21, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 21, 2026 •

edited

Loading

Uh oh!

codspeed-hq Bot commented Jul 21, 2026 •

edited

Loading

Uh oh!

robert3005 left a comment

Uh oh!

robert3005 Jul 21, 2026

Uh oh!

joseph-isaacs Jul 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

joseph-isaacs commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Polar Signals Profiling Results

Latest Run

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks: Vortex queries 📖

Uh oh!

codspeed-hq Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

robert3005 left a comment

Choose a reason for hiding this comment

Uh oh!

robert3005 Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

joseph-isaacs Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

joseph-isaacs commented Jul 21, 2026 •

edited

Loading

github-actions Bot commented Jul 21, 2026 •

edited

Loading

github-actions Bot commented Jul 21, 2026 •

edited

Loading

codspeed-hq Bot commented Jul 21, 2026 •

edited

Loading