feat: add built-in frozendict support for Python 3.15+#274
Open
XuehaiPan wants to merge 12 commits into
Open
Conversation
XuehaiPan
added
enhancement
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #274 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 15 15
Lines 1582 1614 +32
Branches 210 218 +8
=========================================
+ Hits 1582 1614 +32 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
XuehaiPan
force-pushed
the
frozendict
branch
4 times, most recently
from
3426921 to
12a3276
Compare
XuehaiPan
force-pushed
the
frozendict
branch
4 times, most recently
from
fe42e10 to
0faccef
Compare
XuehaiPan
force-pushed
the
frozendict
branch
5 times, most recently
from
19952e3 to
7656326
Compare
Register `frozendict` (Python 3.15.0a7+) as a built-in PyTree node type with key-sorted traversal, matching the behavior of `dict` and `collections.defaultdict`. The `dict_insertion_ordered` context manager also affects `frozendict`. Changes across 29 files: - C++ enum `PyTreeKind::FrozenDict` and `PyFrozenDictTypeObject` macro (guarded by `PY_VERSION_HEX >= 0x030F00A7`) - All treespec switch statements updated (flatten, unflatten, serialize, hash, compare, traverse, construct, paths, accessors, entries) - Python registry with sorted and insertion-ordered flatten/unflatten - `treespec_frozendict()` constructor, `STANDARD_DICT_TYPES` expansion, `PyTree[T]` type union update - `optree.treespec.frozendict` namespace alias - README, Sphinx docs, and spelling wordlist - Test data extended (3 frozendict entries in TREES fixture), subinterpreter test tree includes frozendict, registry size checks - Minor: empty deque repr improved (`deque()` instead of `deque([])`)
Sphinx autodoc/autosummary imports symbols during the reading phase regardless of ``.. only::`` directives, causing import errors on Python < 3.15 where ``treespec_frozendict`` does not exist. Remove the entries for now with TODO comments to add them back when building with Python 3.15+.
There was a problem hiding this comment.
Pull request overview
This PR adds first-class PyTree support for Python 3.15’s new built-in frozendict (PEP 814), aligning its traversal semantics with dict/defaultdict (key-sorted by default, insertion-ordered under dict_insertion_ordered). It also updates documentation, typing exports, CI matrices for Python 3.15, and tweaks the empty deque representation.
Changes:
- Add
PyTreeKind::FrozenDictacross the C++ treespec/registry pipeline and expose build-time capability via_C.OPTREE_HAS_FROZENDICT. - Register Python-level flatten/unflatten handlers for
frozendict, addtreespec_frozendict, and extend typing/aliases/STANDARD_DICT_TYPES. - Update docs/README/tests/CI to cover and validate
frozendictbehavior (including subinterpreter tests).
Reviewed changes
Copilot reviewed 33 out of 34 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_treespec.py | Adds constructor/from-collection coverage for frozendict treespecs under sorted/insertion-ordered modes. |
| tests/test_prefix_errors.py | Updates expected error-regex to optionally include frozendict in standard-dict type errors. |
| tests/test_ops.py | Extends flatten/unflatten metadata assertions to cover PyTreeKind.FROZENDICT. |
| tests/helpers.py | Adds OPTREE_HAS_FROZENDICT gating, bumps expected builtin registry size, and extends shared tree/path/accessor fixtures with frozendict. |
| tests/concurrent/test_subinterpreters.py | Adjusts subinterpreter registry-size checks and adds optional frozendict into the exercised tree/leaves expectations. |
| src/treespec/unflatten.cpp | Includes FrozenDict in unflatten switch paths. |
| src/treespec/treespec.cpp | Creates nodes/types for FrozenDict and extends multiple treespec operations to treat it as a dict-like node kind. |
| src/treespec/traversal.cpp | Extends iterator/walk traversal logic to include FrozenDict among dict-like node kinds. |
| src/treespec/serialization.cpp | Adds FrozenDict kind name and string rendering, and includes it in pickling/unpickling state handling. |
| src/treespec/richcomparison.cpp | Extends prefix comparison to consider FrozenDict compatible with dict-like kinds. |
| src/treespec/hashing.cpp | Extends hashing implementation to include FrozenDict in dict-like hashing. |
| src/treespec/flatten.cpp | Extends flatten/flatten-with-path/flatten-up-to code paths and type-mismatch errors to include FrozenDict. |
| src/treespec/constructors.cpp | Enables treespec constructors to recognize/construct FrozenDict kind. |
| src/registry.cpp | Registers built-in frozendict type in the C++ registry when available. |
| src/optree.cpp | Exposes OPTREE_HAS_FROZENDICT metadata and adds PyTreeKind.FROZENDICT to the Python enum. |
| requirements.txt | Adds a Python 3.15+ typing-extensions minimum constraint. |
| README.md | Documents frozendict as a built-in supported container and clarifies sorted vs insertion-ordered behavior. |
| pyproject.toml | Updates runtime/test dependency markers for typing-extensions on Python 3.15+. |
| optree/typing.py | Exports FrozenDict typing alias and extends PyTree[T] union to include frozendict when available. |
| optree/treespec.py | Adds optree.treespec.frozendict alias (gated by Python/version capability). |
| optree/registry.py | Registers frozendict node handlers (sorted and insertion-ordered variants) and integrates with insertion-order registry logic. |
| optree/ops.py | Adds treespec_frozendict, extends docs for dict-like ordering semantics, and updates STANDARD_DICT_TYPES. |
| optree/_C.pyi | Adds OPTREE_HAS_FROZENDICT and PyTreeKind.FROZENDICT to the stub. |
| optree/init.py | Conditionally re-exports treespec_frozendict at package top-level. |
| include/optree/treespec.h | Updates comments describing metadata/original-keys semantics to include FrozenDict. |
| include/optree/registry.h | Adds FrozenDict to PyTreeKind enum and constants. |
| include/optree/pytypes.h | Adds frozendict type-object macro and includes frozendict in “standard dict” validation/error messages. |
| include/optree/pymacros.h | Defines OPTREE_HAS_FROZENDICT based on PY_VERSION_HEX >= 0x030F00A7. |
| docs/source/treespec.rst | Notes pending doc autosummary integration for frozendict when built on Python 3.15+. |
| docs/source/spelling_wordlist.txt | Adds frozendict to spelling wordlist. |
| docs/source/ops.rst | Notes pending Sphinx function doc inclusion for treespec_frozendict when building on Python 3.15+. |
| .github/workflows/tests.yml | Adds Python 3.15 and 3.15t to the test matrix. |
| .github/workflows/tests-with-pydebug.yml | Adds Python 3.15 to the pydebug matrix. |
| .github/workflows/build.yml | Extends setup-python range to include 3.15 and leaves commented placeholders for 3.15 build targets. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Address Copilot PR review feedback (PR metaopt#274). The `PyTreeKind::FrozenDict` enum value exists unconditionally to keep C++ `switch` statements exhaustive (silences MSVC C4061), but on builds without `OPTREE_HAS_FROZENDICT` the downstream sites silently fell through, demoting `FrozenDict` to a mutable `dict` or hitting `INTERNAL_ERROR()`. A treespec deserialized from a Python 3.15+ build could therefore round-trip incorrectly on older interpreters. C++ hardening: - `src/treespec/serialization.cpp`: reject `FrozenDict` at the deserialization boundary with an actionable `py::value_error` so cross-version state fails loudly at the trust boundary rather than later in unrelated switches. - `src/treespec/treespec.cpp` (`MakeNode`, `GetType`): explicit `#else throw py::value_error` branches instead of silent fall-through / dict demotion. - `src/treespec/flatten.cpp` (`FlattenUpTo`): replace terminal `else` that mislabeled any unknown kind as "frozendict" with an explicit `FrozenDict` branch plus `INTERNAL_ERROR()` default. - `src/treespec/richcomparison.cpp` (`IsPrefix`): document why all dict-family kinds are treated as prefix-compatible with one another. Python/docs polish: - `optree/typing.py`: drop the redundant lowercase `frozendict` import; use the public `FrozenDict` alias internally for the `PyTree[T]` union too. - `optree/ops.py`: rewrite every `treespec_*` docstring opening line to the consistent form "Make a treespec representing a :class:`X` node from child treespecs." - `include/optree/treespec.h`: clarify that `FrozenDict` keys are sorted by default and insertion-ordered under `dict_insertion_ordered`. - `docs/source/{ops,treespec}.rst`: tighten the pending-autodoc TODO to note it activates when building docs with Python 3.15+. Tests (gated on Python 3.15+): - `tests/test_treespec.py`: explicit inequality of `frozendict` treespec vs `dict`/`OrderedDict`/`defaultdict` (incl. hash difference, empty case, and pickle round-trip kind preservation); `dict_insertion_ordered` behavior parity with `dict`. - `tests/test_prefix_errors.py`: convert the relaxed error-message regex into a positive assertion that "frozendict" appears in the type list and that a `dict` is accepted as a prefix of a `frozendict`.
XuehaiPan
added a commit
to XuehaiPan/optree
that referenced
this pull request
May 13, 2026
Address Copilot PR review feedback (PR metaopt#274). The `PyTreeKind::FrozenDict` enum value exists unconditionally to keep C++ `switch` statements exhaustive (silences MSVC C4061), but on builds without `OPTREE_HAS_FROZENDICT` the downstream sites silently fell through, demoting `FrozenDict` to a mutable `dict` or hitting `INTERNAL_ERROR()`. A treespec deserialized from a Python 3.15+ build could therefore round-trip incorrectly on older interpreters. C++ hardening: - `src/treespec/serialization.cpp`: reject `FrozenDict` at the deserialization boundary with an actionable `py::value_error` so cross-version state fails loudly at the trust boundary rather than later in unrelated switches. - `src/treespec/treespec.cpp` (`MakeNode`, `GetType`): explicit `#else throw py::value_error` branches instead of silent fall-through / dict demotion. - `src/treespec/flatten.cpp` (`FlattenUpTo`): replace terminal `else` that mislabeled any unknown kind as "frozendict" with an explicit `FrozenDict` branch plus `INTERNAL_ERROR()` default. - `src/treespec/richcomparison.cpp` (`IsPrefix`): document why all dict-family kinds are treated as prefix-compatible with one another. Python/docs polish: - `optree/typing.py`: drop the redundant lowercase `frozendict` import; use the public `FrozenDict` alias internally for the `PyTree[T]` union too. - `optree/ops.py`: rewrite every `treespec_*` docstring opening line to the consistent form "Make a treespec representing a :class:`X` node from child treespecs." - `include/optree/treespec.h`: clarify that `FrozenDict` keys are sorted by default and insertion-ordered under `dict_insertion_ordered`. - `docs/source/{ops,treespec}.rst`: tighten the pending-autodoc TODO to note it activates when building docs with Python 3.15+. Tests (gated on Python 3.15+): - `tests/test_treespec.py`: explicit inequality of `frozendict` treespec vs `dict`/`OrderedDict`/`defaultdict` (incl. hash difference, empty case, and pickle round-trip kind preservation); `dict_insertion_ordered` behavior parity with `dict`. - `tests/test_prefix_errors.py`: convert the relaxed error-message regex into a positive assertion that "frozendict" appears in the type list and that a `dict` is accepted as a prefix of a `frozendict`.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 33 out of 34 changed files in this pull request and generated 14 comments.
Comments suppressed due to low confidence (2)
tests/test_prefix_errors.py:1
- The dots in
collections.OrderedDictandcollections.defaultdictare unescaped regex metacharacters. They will match any character. Escape them ascollections\.OrderedDict/collections\.defaultdictto avoid accidental matches and for consistency with the escaping used elsewhere in this file.
tests/helpers.py:1 - Hard-coding
9 if … else 8and repeating the same expression intests/concurrent/test_subinterpreters.py(twice) couples three test sites to the same magic constant. Consider exposing the expected size fromhelpers.py(e.g.,EXPECTED_REGISTRY_SIZE) and importing it where needed, so future builtin additions only need to update one place.
…all time Address Copilot PR review on `docs/source/ops.rst` and `docs/source/treespec.rst`: the previous `TODO(frozendict): Add ... when building docs with Python 3.15+` comments meant the new API was perpetually undocumented even on toolchains that could support it, and conditional Sphinx directives (`only::`, `ifconfig::`) don't actually suppress the autodoc/autosummary import warnings emitted when the symbol is missing — those extensions resolve symbols at directive-parse / pre-scan time, before any rendering-side filter applies. Move the version gating from definition-time to call-time so the symbol always exists. `autodoc` and `autosummary` can introspect it on every supported Python; the docstring's `.. note::` block explicitly states the runtime requirement. Calling the function on a build without `frozendict` support raises a clear `RuntimeError` naming the version requirement, rather than the previous `AttributeError` users got when the symbol was conditionally defined. API changes: - `optree.treespec_frozendict` is now defined unconditionally (always callable, always in `optree.__all__`), with a runtime guard that raises `RuntimeError` if `_C.OPTREE_HAS_FROZENDICT` is unset. - `optree.treespec.frozendict` mirrors the same: unconditional re-export of `treespec_frozendict`, always in `optree.treespec.__all__`. - Dropped now-unused `import sys` / `import optree._C as _C` / `del sys` from `optree/__init__.py` and `optree/treespec.py`. - Moved `import builtins` from `TYPE_CHECKING` to module-top in `optree/ops.py` since it's accessed at runtime via `builtins.frozendict`. Docs: - `docs/source/ops.rst`: replace the TODO with the actual `.. autofunction:: treespec_frozendict` directive and add the symbol to the surrounding autosummary block. - `docs/source/treespec.rst`: replace the TODO with a `frozendict` entry in the autosummary list. - Result: `make docs` is warning-free on every supported Python — the symbol is always importable so `autodoc`/`autosummary` succeed. Docstring polish: rewrite the opening line of every `treespec_*` constructor's docstring to the consistent form `Make a treespec representing a :class:`X` node from child treespecs.` and use `:class:` markup uniformly in the `Returns:` block (tuple, list, dict, OrderedDict, defaultdict, deque). Tests: - New `tests/test_treespec.py::test_treespec_frozendict_runtime_error_on_unsupported_interpreter`: on Python < 3.15 (or wheels without `frozendict` support), both `optree.treespec_frozendict()` and `optree.treespec.frozendict(...)` raise a `RuntimeError` whose message contains the exact version requirement. Drive-by: add `# pylint: disable=redefined-builtin` to two `sentinel` references in `tree_replace_nones` (parameter) and `tree_broadcast_common` (local). Python 3.15 added `builtins.sentinel` (PEP 661), which makes pylint flag prior uses of the name; surfaced now that the lint venv runs on 3.15t.
XuehaiPan
force-pushed
the
frozendict
branch
3 times, most recently
from
0795f05 to
2792b40
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Register
frozendict(PEP 814, Python 3.15.0a7+) as a built-in PyTree node type with key-sorted traversal, matching the behavior ofdictandcollections.defaultdict. Thedict_insertion_orderedcontext manager also affectsfrozendict.Also improves the empty deque representation:
deque()instead ofdeque([]).Motivation and Context
Python 3.15 introduces
frozendictas a built-in immutable mapping type (PEP 814). As a fundamental container type, it should be supported as a built-in PyTree node — just likedict,OrderedDict, anddefaultdict.Types of changes
Implemented Tasks
PyTreeKind::FrozenDictenum andPyFrozenDictTypeObjectmacro (guarded byPY_VERSION_HEX >= 0x030F00A7)treespec_frozendict()constructor,STANDARD_DICT_TYPESexpansion,PyTree[T]type unionoptree.treespec.frozendictnamespace aliasChecklist
make format. (required)make lint. (required)make testpass. (required)