HOLD-filters: Custom filters PoC (TypeScript + Python) by SnowboardTechie · Pull Request #918 · HHS/simpler-grants-protocol

SnowboardTechie · 2026-06-16T15:37:14Z

Summary

Integration branch rolling up the custom-filters PoC work for both SDKs: Custom filters TS PoC (#868) #884 (TypeScript, issue [TS SDK] Draft custom filter PoC: TypeScript #868) and Custom filters Python PoC (#869) #885 (Python, issue [Py SDK] Draft custom filter PoC: Python #869).
Stacked on HOLD-transforms (PR Hold transforms #906). Stays in draft until Hold transforms #906 merges to main, then this PR re-targets main.
Closes [Py SDK] Draft custom filter PoC: Python #869.
Time to review: ~5 min. Both commits were already reviewed and merged via Custom filters TS PoC (#868) #884 and Custom filters Python PoC (#869) #885; this PR only stacks them onto the transforms bucket.

Changes proposed

Two commits, no new code beyond the already-merged sub-PRs.

TypeScript custom filters PoC (Custom filters TS PoC (#868) #884, issue [TS SDK] Draft custom filter PoC: TypeScript #868), in lib/ts-sdk.
Python custom filters PoC (Custom filters Python PoC (#869) #885, issue [Py SDK] Draft custom filter PoC: Python #869), in lib/python-sdk.

Diff against the HOLD-transforms base: 25 files, +2,959 / -21, confined to lib/ts-sdk and lib/python-sdk.

Context for reviewers

Why stacked on HOLD-transforms, not main: HOLD-filters builds directly on HOLD-transforms. Targeting HOLD-transforms keeps this diff scoped to the custom-filters work; the transforms changes are reviewed in Hold transforms #906. Targeting main would re-show every transforms commit (69 files, +9,370) and duplicate that review surface.
Merge order: do not merge until Hold transforms #906 lands on main. Once it does, re-target this PR's base to main; the three-dot diff narrows to exactly these two commits.
Verification: each sub-PR (Custom filters TS PoC (#868) #884, Custom filters Python PoC (#869) #885) carried its own checks and was reviewed before merging into HOLD-filters. No new code is introduced here.
HOLD-filters currently trails HOLD-transforms by 15 commits. The three-dot diff is unaffected, but I can sync the branch before re-targeting to main so CI runs against the latest base.

Additional information

The custom-filters API surface and three-bucket classification are exercised by the runnable examples added in #884 (TypeScript) and #885 (Python); see those PRs for demo output.

* Issue #799 Transform PoC * Fixing plugin in makefile * Adding custom handler example * Updating transform contract * Fixing make:checks errors * Reducing duplicate tests and updating makefile * Fixing pr comments. * Adding ADR fixes * PR comments rd 2 * Update website/src/content/docs/governance/adr/0022-plugin-framework.mdx Co-authored-by: Bryan Thompson <18094023+SnowboardTechie@users.noreply.github.com> --------- Co-authored-by: Bryan Thompson <18094023+SnowboardTechie@users.noreply.github.com>

* First pass on Transform completion * Fixing bugs and updating ADR * Fixing cspell * Fixing dependency * Removing stale comment * Refactoring * Fixing test * doc string chagnes * Updating examples to include more mapping types * Removing getClient and filters * Fixing Switch statement and adding by_alias flag * Putting custom fields under the same node as other per object properties * Updating to preserve mypy typing * Updating transforms.py from using Any to generics leveraging schemas to preserve type hinting. * Cleaning up __init__.py * Addressing missing branch * Adding transform custom field validation * Fix safety version conflict in poetry-audit-plugin CI steps Pin safety<3.3.0 via poetry's own venv pip to resolve import error introduced in safety 3.3.0. Remove unused audit plugin install from ci-lib-pysdk.yml which has no poetry audit step. * Updating docstring

* Initial ADR * Formatting ADR * Fixing spelling

Bumps the tooling group with 1 update in the / directory: [tsx](https://github.com/privatenumber/tsx). Updates `tsx` from 4.21.0 to 4.22.3 - [Release notes](https://github.com/privatenumber/tsx/releases) - [Changelog](https://github.com/privatenumber/tsx/blob/master/release.config.cjs) - [Commits](privatenumber/tsx@v4.21.0...v4.22.3) --- updated-dependencies: - dependency-name: tsx dependency-version: 4.21.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: tooling ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps the runtime group with 1 update: [wrangler](https://github.com/cloudflare/workers-sdk/tree/HEAD/packages/wrangler). Updates `wrangler` from 4.93.1 to 4.94.0 - [Release notes](https://github.com/cloudflare/workers-sdk/releases) - [Commits](https://github.com/cloudflare/workers-sdk/commits/wrangler@4.94.0/packages/wrangler) --- updated-dependencies: - dependency-name: wrangler dependency-version: 4.94.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: runtime ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

The lib/* CI workflows (ci-lib-ts-sdk, ci-lib-cli, ci-lib-core, ci-lib-changelog-emitter) ran the audit-deps step on every PR matching their path filters, regardless of base branch. When an advisory lives on main and can only be fixed there (e.g. brace-expansion in #842), PRs into HOLD-* batching branches failed audit with no path to action for the PR author — the fix has to land on main first, then HOLD rebases. Under the HOLD-* batching strategy, each HOLD branch eventually opens a single PR into main at the checkpoint. That PR re-runs audit, so any advisory live at merge time is gated at the actionable boundary. Auditing intermediate PRs into HOLD-* shifts noise earlier without shifting the fix earlier. Gate is base_ref == 'main' OR ref == 'refs/heads/main' so it also covers any future workflow_call invocation from a main-push context.

Bumps the website-framework group with 1 update in the / directory: [astro](https://github.com/withastro/astro/tree/HEAD/packages/astro). Updates `astro` from 6.3.6 to 6.3.8 - [Release notes](https://github.com/withastro/astro/releases) - [Changelog](https://github.com/withastro/astro/blob/main/packages/astro/CHANGELOG.md) - [Commits](https://github.com/withastro/astro/commits/astro@6.3.8/packages/astro) --- updated-dependencies: - dependency-name: astro dependency-version: 6.3.7 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: website-framework ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps the runtime group with 2 updates: [json-schema-faker](https://github.com/json-schema-faker/json-schema-faker) and [wrangler](https://github.com/cloudflare/workers-sdk/tree/HEAD/packages/wrangler). Updates `json-schema-faker` from 0.6.1 to 0.6.2 - [Release notes](https://github.com/json-schema-faker/json-schema-faker/releases) - [Commits](json-schema-faker/json-schema-faker@v0.6.1...v0.6.2) Updates `wrangler` from 4.94.0 to 4.95.0 - [Release notes](https://github.com/cloudflare/workers-sdk/releases) - [Commits](https://github.com/cloudflare/workers-sdk/commits/HEAD/packages/wrangler) --- updated-dependencies: - dependency-name: json-schema-faker dependency-version: 0.6.2 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: runtime - dependency-name: wrangler dependency-version: 4.95.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: runtime ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps the runtime group with 1 update in the / directory: [openapi-sampler](https://github.com/Redocly/openapi-sampler). Updates `openapi-sampler` from 1.7.3 to 1.7.4 - [Release notes](https://github.com/Redocly/openapi-sampler/releases) - [Changelog](https://github.com/Redocly/openapi-sampler/blob/main/CHANGELOG.md) - [Commits](Redocly/openapi-sampler@v1.7.3...v1.7.4) --- updated-dependencies: - dependency-name: openapi-sampler dependency-version: 1.7.4 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: runtime ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

… 2 updates (#867) Bumps the website-framework group with 2 updates in the / directory: [@astrojs/react](https://github.com/withastro/astro/tree/HEAD/packages/integrations/react) and [astro](https://github.com/withastro/astro/tree/HEAD/packages/astro). Updates `@astrojs/react` from 5.0.5 to 5.0.6 - [Release notes](https://github.com/withastro/astro/releases) - [Changelog](https://github.com/withastro/astro/blob/main/packages/integrations/react/CHANGELOG.md) - [Commits](https://github.com/withastro/astro/commits/@astrojs/react@5.0.6/packages/integrations/react) Updates `astro` from 6.3.8 to 6.4.2 - [Release notes](https://github.com/withastro/astro/releases) - [Changelog](https://github.com/withastro/astro/blob/main/packages/astro/CHANGELOG.md) - [Commits](https://github.com/withastro/astro/commits/astro@6.4.2/packages/astro) --- updated-dependencies: - dependency-name: "@astrojs/react" dependency-version: 5.0.6 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: website-framework - dependency-name: astro dependency-version: 6.4.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: website-framework ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

…dates (#859) Bumps the all-deps group with 2 updates in the /lib/python-sdk directory: [ruff](https://github.com/astral-sh/ruff) and [datamodel-code-generator](https://github.com/koxudaxi/datamodel-code-generator). Updates `ruff` from 0.15.13 to 0.15.15 - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](astral-sh/ruff@0.15.13...0.15.15) Updates `datamodel-code-generator` from 0.57.0 to 0.59.0 - [Release notes](https://github.com/koxudaxi/datamodel-code-generator/releases) - [Changelog](https://github.com/koxudaxi/datamodel-code-generator/blob/main/CHANGELOG.md) - [Commits](koxudaxi/datamodel-code-generator@0.57.0...0.59.0) --- updated-dependencies: - dependency-name: datamodel-code-generator dependency-version: 0.58.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: all-deps - dependency-name: ruff dependency-version: 0.15.14 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: all-deps ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

@throws

* [Issue #798] Transforms PoC: TypeScript Port the Python transforms PoC (PR #810, branch 799-transform-poc-fetch) to @common-grants/sdk so the ADR-0022 / ADR-0017 contract is validated in both SDKs before either is locked in for full implementation. Public additions under @common-grants/sdk/extensions: - buildTransforms() — compile a pair of ADR-0017 mapping objects into typed (toCommon, fromCommon) callables with call-time structural validation. Optional commonModel Zod schema turns parse failures into PluginError[] rather than thrown exceptions. - TransformResult<T> — unconditional { result, errors } return shape (ADR-0022 Decision #7). - PluginError — structured error class with path / handler / sourceValue / cause (ADR-0022 Decision #9). sourceValue and cause are stored non-enumerable; toJSON() omits them so JSON.stringify(err) is PII-safe. console.log(err) / util.inspect(err) still render [cause] via Node's default Error inspection — README PII callout calls this out and recommends a redacted projection. Zod-validation message is also data-bearing because Zod's default error map embeds rejected values; full-message sanitization tracked under #744. - transformFromMapping(), getFromPath(), DEFAULT_HANDLERS — mapping runtime; six built-in handlers (const, field, match, switch alias, numberToString, stringToNumber). - definePlugin() accepts optional meta and transformSchemas. Existing callers passing only `extensions` are unaffected. Security hardening (mapping JSON may be reconstituted from untrusted sources via mergeExtensions(), so the runtime must fail loud on hostile shapes): - buildTransforms() rejects custom handler names that collide with the default registry or shadow Object.prototype keys (constructor, toString, __proto__, etc.) at call time. - validateMapping() rejects `__proto__` as an output field name at build time; transformFromMapping() rejects it again at walk time so the JSON attack vector (own-enumerable __proto__ key from JSON.parse) fails fast in both places. Shared DEFAULT_MAX_TRANSFORM_DEPTH = 500 across both walkers so adversarial mapping JSON can't pass build-time validation only to blow the stack at runtime. - transformFromMapping() scrubs top-level own `__proto__` from plain-object handler returns (defense-in-depth: const / field / match can return JSON.parse-loaded objects with own __proto__ keys, and a downstream for-in deep-merge of the result would otherwise pollute Object.prototype). - getFromPath() uses Object.prototype.hasOwnProperty.call rather than `in` so attacker-controlled field paths cannot resolve to prototype-chain properties. - stringToNumber's error message does not embed the source value (would flow into PluginError.message and bypass the sourceValue PII guard). - Handler JSDoc documents two contracts custom-handler authors must respect: don't return objects with a `__proto__` key (the walker treats handler return values as opaque beyond the top-level scrub), and don't throw Errors with PII in .message (it flows verbatim into PluginError.message, which is rendered by Node Error inspection). Cross-SDK parity: - switchOnValue mirrors Python's `lookup.get(val, default)` exactly: only string source values are candidate keys; numeric / boolean / object source values short-circuit to default. Pinned by parity tests. - ClientConfig parity export added (Python `__init__.py` exports it as `dict[str, Any]`). - Six placeholder type exports (ObjectSchemas, ObjectMappings, PluginExtensions, PluginExtensionsObjectConfig, PluginCapability, PluginMeta) define the ADR-0022 contract surface and parallel the Python PoC's extensions/__init__.py exports. Documented divergences from ADR-0022's TS code blocks (justified in findings.md): - DefinePluginOptions.transformSchemas (not `schemas`) avoids collision with the existing Plugin.schemas field. Resolution deferred to #756. - BuildTransformsOptions.commonModel uses `z.ZodType<TCommon, z.ZodTypeDef, any>` to admit schemas with input/output asymmetry (e.g. .transform() producing Date from string). ZodType<TCommon> would reject the SDK's own OpportunityBaseSchema. Out of scope (matches Python PoC; deferred to full SDK): - Auto-generation of transforms from declarative extensions.schemas[obj].mappings inside definePlugin() (Decision #6 TODO). - Always-on commonModel validation inside definePlugin() — opt-in at buildTransforms() for now (Decision #7 TODO). - Sanitizing Zod's default error map output before it lands in PluginError.message — tracked in #744. - Defensive output-key rejection of __defineGetter__ / __defineSetter__ / constructor / toString — tracked in #744. - Recursive (nested) sanitization of __proto__ in handler return values — tracked in #744 alongside other Decision #8 hardening. Includes: - examples/transforms.ts round-trip (`pnpm example:transforms`) - README "Plugin transformations" section + API reference table - 14 define-plugin specs, 35 transformation-handler specs, 19 buildTransforms specs (439 tests across 24 suites, all passing) - Minor changeset bump for @common-grants/sdk Targets HOLD-transforms per the SDK Plugin Enhancements branching strategy. * review: stringToNumber rejects unsafe integers and empty strings Two foot-guns flagged in code review on PR #825: 1. Integer-shaped strings beyond Number.MAX_SAFE_INTEGER silently lost precision via Number(s) (e.g. "9999999999999999999" → 1e19). Plugin authors round-tripping 64-bit IDs through stringToNumber would have seen silent corruption. Guard the integer branch with Number.isSafeInteger and throw on overflow; callers needing arbitrary precision should declare the field as a string or write a custom handler returning BigInt. 2. Empty and whitespace-only strings coerced to 0 via Number("") / Number(" "), turning an implicit-absent CSV cell into a real zero on the transformed side. Explicit s === "" check after .trim() throws on both cases. Docstring rewritten to document both divergences from Python's int() alongside the existing "42.0" decimal-fallback note. Six new tests pin the boundaries (MAX/MIN_SAFE_INTEGER accepted, beyond-safe rejected on both signs, empty and whitespace-only rejected). pnpm run checks clean; pnpm test 443/443; example:transforms round-trip still verifies (legacy ID 12345 well within safe-integer range). * review: validateMapping rejects sibling keys at handler-dispatch nodes The runtime walker is first-key-wins, so `{ field: "x", const: "y" }` would silently drop `const`. Almost always an author bug. Match Python PoC `_validate_mapping` and fail loud at buildTransforms() call time. The low-level transformFromMapping walker keeps lenient behavior so programmatic users composing partial mappings aren't forced into the strict shape. * review: ADR-0022 alignment + parity-claim corrections + test gaps Two behavior changes: - PluginError: drop non-enumerable + toJSON() hardening to match ADR-0022 Decision #9 ("the SDK does not redact by default"). README + JSDoc document the adopter-supplied redacted projection as the supported PII-safety path. Existing PII test inverted. - switchOnValue: throw on non-object spec instead of silent undefined. Walker wraps as HandlerError → PluginError(handler: "match"). JSDoc / comment fixes: reframe "Mirrors Python PoC" claims that were factually wrong (Python's _validate_mapping doesn't reject siblings; Python has no __proto__ defenses; lookup.get accepts non-string val) as honest "TS-only hardening; cf. #810 for parallel Python proposal." Complete @throws lists on buildTransforms and transformFromMapping. Document stringToNumber null/undefined return. Correct misleading "re-exported" wording on DEFAULT_MAX_TRANSFORM_DEPTH. Correct "trimmed to match Python's __init__.py" comment on the barrel (TS exports a superset). Test gap coverage (ADR-0022 Decision #8): getFromPath prototype-chain safety; handler-shadow rejection for __proto__/toString/default-name collisions beyond \`field\`; build-time __proto__ rejection in validateMapping; fromCommonMapping depth-cap. README + CI: add missing ClientConfig and TransformFromMappingOptions rows to the API table; chain example:transforms into pnpm run ci as a smoke step. Simplify: extract deepMapping(levels) helper; PII test suite uses beforeEach so both tests assert on the same PluginError instance. Verification: pnpm run ci passes end-to-end (445 → 449 tests). Cross-SDK follow-ups (file against #810 / amend ADR-0022 separately): TS-only sibling-key rejection, __proto__ defenses, switchOnValue non-object throw, stringToNumber strictness, handler-shadow rejection; and the ADR-0022 Decision #9 default-redaction question itself. * review: revert sibling-key rejection (d52dd72) for cross-SDK parity Python's `_validate_mapping` accepts handler-dispatch nodes with sibling keys; the TS `validateMapping` was rejecting them at build time, creating the only genuine "TS adds mandatory validation Python doesn't" divergence in this PR. The first-key-wins walker behavior already matches Python, so both SDKs now share the same foot-gun: `{ field: "x", const: "fallback" }` silently drops `const` in both. Replaces the two rejection tests with one parity-pinning test (`accepts (does not reject) sibling keys ... cross-SDK parity`) that asserts `.not.toThrow()` — locks the decision so a future regression re-introducing build-time rejection (and re-creating the divergence) fails loudly. Updates `validateMapping` JSDoc to call out the parity decision and `transformFromMapping` JSDoc to drop the (now false) reference to a stricter build-time check. If we later want both SDKs to fail loud on this shape, the cleanest path is to add the check to Python first and then re-introduce TS — keeps divergence at zero. Verification: pnpm run ci passes end-to-end (449 → 448 tests; -2 removed rejection tests, +1 added parity-pinning test). * review: array-spec rejection, scrub clones source, README projection Three findings from a /review pass over the PoC. All three tighten existing contracts; none introduce new shapes. - switchOnValue: reject arrays as spec. Arrays pass `typeof === "object"` and non-null but lack the structural shape — a mapping like `{ match: ["posted", "archived"] }` would otherwise silently resolve to s.field/s.case/s.default all-undefined and return undefined. Same fail-loud direction as the existing non-object guards. - __proto__ scrub: shallow-clone on the pollution path rather than `delete`-in-place. `fieldValue` returns references plucked from caller input via `getFromPath`, so an in-place delete would silently mutate the caller's data — surprising for plugin authors caching parsed source records across `toCommon` calls (common in long-running adapter processes and multi-tenant deployments). Spread is the correct copy primitive here: CopyDataProperties -> CreateDataProperty bypasses the prototype setter. Object.assign would mutate the target's prototype chain instead — the source comment now warns against the regression. - README error-handling snippet: log a named redacted projection rather than building the message via template-string interpolation. Makes the "what gets logged" surface a single audit point and matches the projection in the prose PII callout and in the redacted-projection test verbatim. Verified: pnpm --filter @common-grants/sdk run ci passes (449 tests across 24 suites, +1 source-preservation test; example:transforms round-trip OK). * consolidate customFields onto ObjectSchemasInput (Python a156d31 parity) Mirror the Python PoC PR #838 commit a156d31 so plugin authors add a single per-object entry under transformSchemas[Object] when introducing a new object, rather than splitting customFields across a separate top-level extensions dict. - types.ts: add customFields?: Record<string, CustomFieldSpec> to ObjectSchemasInput; drop customFields from PluginExtensionsObjectConfig (now mappings-only, matching Python's PluginExtensionsSchema). - define-plugin.ts: make extensions optional in DefinePluginOptions and Plugin; definePlugin() sources customFields from transformSchemas[name].customFields first, falling back to the legacy extensions[name] surface so existing customFields-only plugins keep working. - examples/transforms.ts: consolidate to a single transformSchemas entry carrying customFields + toCommon + fromCommon together. - tests: cover the consolidated path (customFields on transformSchemas[obj]), no-extensions-arg case, and transformSchemas-wins-over-extensions priority. Open question — ADR-0022 as written places customFields inside PluginExtensions.schemas[obj] specifically so declarations can be combined across packages via mergeExtensions() (Decision driver line 23, Decision #4). This commit follows Jeff's Python move out of that serializable surface, which drops customFields from the cross-package merge contract. Pending an ADR-0022 amendment formalizing the trade-off; flagged inline on ObjectSchemasInput and PluginExtensionsObjectConfig. Type inference for plugin.schemas[obj] still flows through the legacy extensions parameter — the runtime applies customFields from either source, but the compiled-schema type only narrows when customFields are declared via extensions. Wiring the generic through transformSchemas is a follow-up (parallel to the transformSchemas → schemas rename deferred to #756). * align transforms-layer handlers with ADR-0024 three-state contract Preserve explicit `null` source values ("publisher asserts doesn't apply") across `toCommon` / `fromCommon` instead of collapsing them to `undefined` ("not provided"). Without this, ADR-0024's third state would be unobservable through the transforms layer — the ADR audited the Zod/Pydantic validation surface and explicitly left SDKs untouched, but the handler layer is downstream of that. Handler changes (lib/ts-sdk/src/extensions/transformation.ts): - numberToString / stringToNumber: null source -> null output (was undefined). Return types widen to string | null | undefined / number | null | undefined. - switchOnValue: null source passes through as null by default; mapping authors opt in to target-side translation via a `case: { "null": ... }` key. `default` is NOT consulted for null source (default is for unrecognized values, not publisher assertions). - field / getFromPath: unchanged — already preserves terminal null. JSDoc updated to document intermediate-null behavior as scoped-out ("parent N/A propagates to child paths"). - The walker places handler-returned null onto the output object as a real null (distinct from an absent key), so the three-state distinction survives end-to-end. Docs: - New "Null handling" subsection in src/extensions/README.md documenting the three-state contract for custom-handler authors. - API reference table updated to reflect the new return semantics. - Module-level JSDoc on transformation.ts documents the contract and the cross-SDK divergence (TS leads, Python PoC #810 parity tracked there). Example (examples/transforms.ts): - Source record now includes `source_url: null` demonstrating the publisher's "doesn't apply" assertion. - Round-trip spot-check fails if a future regression coerces it back to undefined. Tests: - Existing "returns undefined on null or missing" tests split into two for both coercing handlers (null -> null; absent -> undefined). - New switchOnValue coverage for null pass-through, opt-in "null" case-key precedence, and the no-default-for-null rule. - New walker test pinning that handler-returned null lands on the output. - New buildTransforms round-trip + commonModel Zod assertion using `source` (the .nullish() field on OpportunityBaseSchema). - fieldValue regression pin for terminal-null preservation and intermediate- null collapse. 466 tests pass (was 452, +14). pnpm run checks / build / example:transforms all clean. * review: fix doc drift surfaced by three-lens self-review Code/runtime behavior unchanged — these are JSDoc, README, and example-comment fixes for code-vs-docs drift that accumulated across review iterations. - `PluginError` JSDoc: rename the "Recommended redacted projection" snippet to "partial-redaction pattern" with an inline CAUTION on the data-bearing `message` field. The previous label invited copy-paste that contradicts the warning paragraph immediately below (Zod's default error map embeds runtime values into `issue.message`, which flows verbatim into `PluginError.message`). - `Handler` JSDoc: remove stale reference to `PluginError.toJSON()`. The toJSON()/non-enumerable hardening was removed earlier in this PR for ADR-0022 Decision #9 alignment; the JSDoc still described a protection mechanism that no longer exists. Replace with a pointer to the README PII warning and to `stringToNumber`'s generic-message pattern. - `Handler` JSDoc: note the cosmetic ADR-0022 parameter-name divergence (`(data, arg)` vs ADR's `(value, context)`) so a custom-handler author reading the ADR interface block doesn't trip on the rename. - `switchOnValue` and `DEFAULT_HANDLERS` JSDoc + README handler table: drop the phantom "ADR-0022 Decision #3" citation. Decision #3 is about per-object schema grouping, not handler naming. The `switch` alias is a convenience — no prior SDK API defined `switch`, so there is no backward-compatibility obligation. Cross-SDK parity rationale (if any) tracked separately. - README API reference table: update `ObjectSchemasInput` row to include `customFields?` (added in d9d9bb2) and update `PluginExtensionsObjectConfig` row to drop `customFields?` (moved out in the same commit). Both rows had drifted from the implementing interfaces. - `examples/transforms.ts`: add PII-warning comments on the two error-path `e.message` formatting blocks. Safe in this example (fixed PII-free source data) but adopters copy this shape into their own pipelines. Checks: - `pnpm run check:lint` — clean. - `pnpm run check:format` — clean (prettier --write applied to README after the table-row edit shifted column widths). - `pnpm run check:types` — clean. - `pnpm run test` — 466 / 466 pass across 24 suites. - `pnpm run example:transforms` — round-trip + ADR-0024 three-state spot-check both pass. * review: omit undefined output keys so in-memory shape carries three-state (ADR-0024) Resolves the walker/doc/test mismatch flagged in self-review (Important #1). Behavior change (transformFromMapping output-shape branch): a child that transforms to `undefined` (absent source — ADR-0024 "not provided") is now omitted from the output object instead of written as `out[k] = undefined`. `null` ("doesn't apply") is still written as a present key. So the in-memory object now distinguishes all three states the same way the wire does: absent → key omitted, `null` → present `null`, value → present value. Why this shape: - The wire output is unchanged — `JSON.stringify` already dropped `undefined`-valued keys, so absent→missing on the wire was already correct. This only fixes the in-memory representation to match. - Makes the in-memory object a faithful three-state carrier: consumers can use key presence (`hasOwnProperty` / `in`) to tell "not provided" from "doesn't apply", which a present-`undefined` key made impossible. - Aligns with the dict shape the Python SDK lands on once it adopts the three-state contract: Python has no `undefined`, so an absent field is a missing dict key there. (Python's PoC currently collapses absent+null into `None` — known non-conformant, per the cross-SDK note in the README.) Doc/test corrections that depended on the old behavior: - README "Null handling": the "distinct from an absent key" claim is now literally true; reworded to state the absent→omitted / null→present rule and that key presence is the probe. - transformFromMapping JSDoc: output-shape bullet documents undefined-omission. - transforms.spec.ts "distinguishes absent from null": the test comment claimed an absent output key but only asserted `toBeUndefined()` (satisfied by a present-undefined key too). Added `hasOwnProperty` assertions that pin the real contract — absent key for absent source, present key for null. - transformation.spec.ts: new low-level test asserting an undefined-returning handler omits the key while a sibling present key is still written. Checks: lint / prettier / tsc clean; test 467/467 (was 466, +1 walker test); example:transforms round-trip + source_url:null three-state spot-check pass. * review: correct stale cross-SDK parity note in transforms README The "Null handling" cross-SDK note claimed the Python ADR-0024 parity follow-up was "tracked there" and linked #810. Reading #810 directly (body + diff) confirms it is the base Python Transform PoC (closes #799, merged 2026-05-13), scoped "to the transform layer only" with no null/three-state content — its coercing handlers collapse absent and null into `None` (`str(val) if val is not None else None`; `if val is None: return None`). ADR-0024 (PR #855) was created and merged 2026-05-26, 13 days after #810, so #810 neither did the parity work nor tracks it. No dedicated Python-parity issue exists. Reword: drop the false "tracked there" claim, link ADR-0024 directly as the contract Python must meet, state plainly that Python predates the ADR and that handler parity is a pending follow-up. No tracker is asserted. Docs-only; prettier clean. * review: attribute scrub-time throws to the handler (move __proto__ scrub inside try) The walker's __proto__ scrub on a handler's return value ran *outside* the handler try/catch. A custom handler returning a trap-throwing exotic value (e.g. a Proxy whose getPrototypeOf/hasOwnProperty trap throws, or one that throws on spread) made the scrub throw, which escaped as an unattributed PluginError via runMapping's generic catch instead of HandlerError. Not a security issue — handler code is trusted (author-supplied, registered per buildTransforms call); ADR-0022 Decision #8's threat model is untrusted mapping JSON, not handler code. The failure was already contained (PluginError in errors[], no crash/pollution). This only restores handler attribution on that exotic path. Fix: move the scrub inside the existing handler try/catch. The scrub operates on the handler's return value, so a throw there is correctly attributed to the handler via HandlerError -> PluginError(handler). Added a comment marking the placement as deliberate ("don't hoist it out") and a test: a handler returning a Proxy with a throwing getPrototypeOf trap now yields PluginError(handler). Checks: lint / prettier / tsc clean; test 468/468 (+1); example:transforms passes. * fix: reject sibling keys at handler-dispatch nodes (restore d52dd72) The 3e557b9 revert removed TS's build-time sibling-key rejection on the premise that Python's `_validate_mapping` accepts the shape. It does not: `build_transforms` -> `_validate_mapping` (python-sdk extensions/transforms.py) raises on a handler key with sibling keys, and has since the Python PoC (#810). The revert conflated Python's lenient runtime walker (transform_from_mapping, first-key-wins) with its strict build-time validator, so it created the cross-SDK divergence it claimed to remove. Re-instates build-time rejection in validateMapping (mirroring Python), restores the two rejection tests, and corrects the now-false JSDoc in transforms.ts and transformation.ts. The low-level transformFromMapping walker stays lenient, matching Python's runtime walker. Verified: check:types + prettier + build + vitest (469 passed). * docs(ts-sdk/extensions): trim cross-reference cruft from transforms PoC Pointers cost per occurrence and rot at different rates; keep the inline reasoning, thin the citations. - Remove all `#838` / commit `a156d31` references (unmerged-branch pointers that describe Python code not on the base branch). Replaced with "pending ADR-0022 amendment" framing that states the trade-off in its own terms. - Collapse the ADR-0024 three-state explanation to one canonical block in transformation.ts + the README; handlers now note only their own departures instead of re-deriving the contract. - Remove the 4 duplicated per-handler "Python #810 parity" notes; keep the statement once in the README (#810 is merged, durable). - Thin repeated `(ADR-0022 Decision #8)` prototype-hardening tags in the walker; the security reasoning stays at every guard, one ADR anchor kept. Comment/doc-only; no behavior change. tsc/eslint/prettier clean, extension tests pass, example:transforms round-trips. * docs(ts-sdk/extensions): drop ungrounded "pending ADR-0022 amendment" framing The "pending amendment / open question" language asserted a governance artifact that exists nowhere: ADR-0022's doc never mentions it, no issue tracks it, and the only files in the repo referencing an "ADR-0022 amendment" were this PR's own. Same failure mode as the #838 pointer — a reference to something that isn't real. Replace with the verifiable behavior + trade-off: customFields can live on transformSchemas[Object] (preferred) or the legacy `extensions` surface (kept for mergeExtensions cross-package composition); transformSchemas wins when both are set. State the trade-off, drop the speculative future-ADR claim. Comment/doc-only. tsc/prettier clean, 159 extension tests pass. * docs(ts-sdk/extensions): second cruft pass — uniform across the whole PR The first pass cherry-picked the loudest offenders and left the broader "talks a lot about other issues/PRs/ADRs" problem (originally flagged on define-plugin.ts) largely intact. This applies one policy uniformly across all PR files: - ADR-0017: one canonical mention (README status line); elsewhere "mapping objects" stands alone. - ADR-0022 "Decision #N" tags: keep the reasoning; one anchor each for the two security decisions (#8 prototype guard in getFromPath, #9 PII in the README warning). Drop the rest, including in test names. - "full SDK will… (tracked under #756)": state PoC scope plainly; #756 kept once (define-plugin.ts). - Cross-SDK Python-parity pointers ("Mirrors the Python SDK's X", parity export, superset-of-__init__, .-join-matches-Python): consolidated to the one README cross-SDK note. Behavioral divergence docs (int(s) semantics, safe-integer rationale) kept — they describe this code. - #798/#810: one mention each (README/changeset opening). - Test names/comments: bare ADR/Decision numbers stripped, behavioral phrasing kept ("doesn't apply", "hardening", sibling-key "Python PoC parity"). De-referenced a rot-prone transforms.ts:218 line pointer. Comment/doc-only; no behavior change. tsc/eslint/prettier clean, 159 extension tests pass, example:transforms round-trips. * docs(ts-sdk/examples): explain the join/split custom handlers in transforms example Address review on PR #825: the Step 2 handler block had only a section banner. Expand it to cover the handler dispatch mechanism (name-as-key, the built-in set, the Step 4 handlers map, absolute data paths) and add a one-line behavior note above joinFields and splitField, framing them as an inverse pair for the derived compositeLabel round trip. * Flattening plugin contract to match ADR * Removing extraneous key checks * Updating documentation and fixing CI issue * Updating dependencies * Fixing CI * Fixing tests * Updating changeset * Code cleanup * Removing merge extensions artifacts --------- Co-authored-by: jcrichlake <jeff.crichlake@agile6.com>

* Expanding transform logic * Update transforms doctoring Co-authored-by: Bryan Thompson <18094023+SnowboardTechie@users.noreply.github.com> * Updating test file * Updating tests --------- Co-authored-by: Bryan Thompson <18094023+SnowboardTechie@users.noreply.github.com>

@ts-expect-error

* test(01-01): add failing tests for BooleanComparisonFilterSchema - Import BooleanComparisonFilterSchema from @/schemas - Add BooleanComparisonFilter Schema describe block with 4 cases: eq+true succeeds, neq+false succeeds, gt throws, string value throws * feat(01-01): add BooleanComparisonFilterSchema to filters Zod schemas (D-04) - Add BooleanComparisonFilterSchema using EquivalenceOperatorsEnum (eq|neq) and z.boolean() - Place after Money section with section-header comment style - No lib/core changes per D-04 (SDK/Zod level only) * feat(01-01): add CustomFilterType enum and filter declaration types (D-11) - Add 11-value CustomFilterType string-literal union (authoritative *Comparison naming) - Add CustomFilterSpec interface (filterType + optional description; no value field) - Add RouteDeclarations interface (optional filters record) - Add PluginRoutes type (resource → method → RouteDeclarations) - Placement: after CustomFieldSpec block, before HasCustomFields section * test(01-01): add failing routes pass-through tests for definePlugin - Import PluginRoutes type from @/extensions - Add describe("routes") block with 3 it() cases: routes pass through unchanged, routes undefined when omitted, routes accessible for later classification * feat(01-01): add routes key to definePlugin and barrel-export filter types (D-10) - Add PluginRoutes to import block in define-plugin.ts - Add routes?: PluginRoutes to DefinePluginOptions and Plugin interfaces - Destructure routes in definePlugin() and pass through in returned object - Add CustomFilterSpec, CustomFilterType, PluginRoutes, RouteDeclarations to extensions/index.ts barrel (types only; runtime exports in Plan 02) * feat(01-02): implement custom-filters classifier, validators, and F helpers - classifyFilters: three-bucket consumer→OppFilters wire body (default→top-level, registered/ad-hoc→customFilters) - validateRoutes: registration-time validation (unknown filterType, duplicate names, default-name collisions) → PluginError - validateFilterCall: call-time validation (operator/schema for registered; shape-only for ad-hoc) → PluginError - F namespace: 12 helpers (eq, neq, gt, gte, lt, lte, in, notIn, like, notLike, between, outside) → {operator, value} - FILTER_TYPE_SCHEMAS map covers all 11 CustomFilterType values; integerComparison reuses NumberComparisonFilterSchema (finding candidate) - gov.<system>@<filterName> namespaced keys pass through to customFilters verbatim (D-16) * test(01-02): add Vitest coverage for classifyFilters, validators, and F helpers - describe("classifyFilters") with sub-blocks: three-bucket classification (7 tests including exact wire-body deep-equal), registration-time validation (5 tests for unknown filterType, default-name collisions, valid routes), call-time validation (8 tests for op/type mismatch, shape mismatch, ad-hoc passthrough) - describe("F helpers") with 14 tests covering all 12 helpers + DefaultFilterSchema compatibility - Uses @/ path alias; file-scope fixtures for grantsGovRoutes and mixedConsumerFilters - Runtime coverage only; compile-time as const narrowing tests deferred to Plan 03 * feat(01-02): add runtime barrel exports for custom-filters classifier and F helpers - Export classifyFilters, validateRoutes, validateFilterCall, F from ./custom-filters - Update comment from placeholder "Runtime exports added in Plan 02" to active exports - Type exports for CustomFilterSpec, CustomFilterType, PluginRoutes, RouteDeclarations retained from Plan 01 * feat(01-03): add compile-time as const narrowing type-tests (D-13) - Add custom-filters-types.spec.ts: four @ts-expect-error assertions proving unknown key, operator/filterType mismatch, value-shape, and array-shape mismatches are compile errors WITH as const (check:types exit 0 = proof) - Add WITHOUT-as-const widening trap demonstration via Plugin type annotation - Extend Plugin<T> with second generic TRoutes to preserve literal routes type; backward-compatible (default = PluginRoutes); required for TypedConsumerFilters narrowing to propagate from definePlugin() return type (D-13) - Add FilterValueShape, FilterOperator, TypedFilter, TypedConsumerFilters, RouteFilterSpecs utility types as the compile-time narrowing layer * feat(01-03): add canonical grants.gov custom-filters example + wire ci script - Add examples/custom-filters.ts: runnable grants.gov demo with agency (stringArray), fundingProgram (stringComparison), ad-hoc legacyTag, and default status/closeDateRange; prints classified OppFilters wire body; assertions verify three-bucket classification; no network I/O - Wire example:custom-filters script in package.json and append to ci chain - Non-null assertion on plugin.routes for classifyFilters call (routes? is optional on Plugin interface; example declares it so the assertion is safe) * docs(01-04): add Plugin custom filters (PoC) section to extensions README - Add "Plugin custom filters (PoC)" section mirroring the transforms-PoC precedent - Document routes-vs-schemas critical distinction (filters on methods, fields on schemas) - Include filter-type catalog (11 values), F.* helpers table, classifyFilters usage - Add validation and as-const trap subsections with code examples - Add "Custom filters (PoC)" API reference table for all four new exports - Update table of contents with new section and API ref entry - Prettier-clean (check:format passes on README) * style(01): apply Prettier to phase 01 custom-filters files Plan 01-03 executors passed check:types and tests but never ran prettier --write; the whole-package check:format gate (part of pnpm ci) was failing on 6 files. Formatting only — no behavioral change. * fix(01-03): place @ts-expect-error inside literals so the build catches them Prettier reflow (commit e6a21c9) pushed the directives above the `const bad = {` line, but tsc reports object-literal excess-property/assignability errors on the PROPERTY line — so the directives went unused (TS2578) and the real errors surfaced, failing `pnpm build` (and any fresh consumer install). Moving each directive directly above its offending property is prettier-stable and matches where tsc reports, restoring the as-const narrowing proof under the real build. * docs(01): drop internal planning references from custom-filters comments Comments carried planning-process artifacts a reader of the published SDK cannot resolve: decision/task IDs, internal plan numbers, point-in-time "PoC" framing, a tech-spec reference, and one stray tooling-file reference. Rewrite to durable, self-contained comments — every ADR reference and the technical substance (e.g. integerComparison reusing NumberComparisonFilterSchema) is preserved. Comment-only; no behavior change. * fix(01): satisfy eslint — reference type-only fixture, drop stale disable CI check:lint (the one check not run before pushing) failed on: - custom-filters-types.spec.ts: `grantsGovPlugin` is consumed only via `typeof`; the eslint config sets no varsIgnorePattern, so reference it with `void` to satisfy no-unused-vars. - custom-filters.ts: the `in` helper takes `unknown[]`, not `any` — its no-explicit-any disable directive was unused; removed. * docs(01): fix validateFilterCall signature and range filter shape in extensions README The call-time validation example documented a five-argument validateFilterCall(routes, resource, method, name, value) that does not exist; the real signature is validateFilterCall(spec, filterName, filterValue). Correct the example to look the spec up from the route-method and pass the three real arguments. Also fix the F.between examples: range filters serialize their value as { min, max } (matching NumberRangeFilterSchema / DateRangeFilterSchema / MoneyRangeFilterSchema and the catalog table), not as a [min, max] array. * docs(01): correct classifyFilters default-bucket validation comment The inline comment claimed default fields are validated against their own schemas from OppDefaultFiltersSchema. They are not: default keys get shape-only validation via DefaultFilterSchema, the same treatment as ad-hoc keys. Update the comment to describe the actual behavior. * fix(ts-sdk): enforce int values for integerComparison; drop unused operator metadata and unreachable duplicate check - Add IntegerComparisonFilterSchema (.int()) so integerComparison no longer reuses the looser NumberComparisonFilterSchema - Flatten FILTER_TYPE_SCHEMAS to a plain CustomFilterType -> Zod schema map: the operatorEnum entries were never read (each filter schema already constrains its operator enum, so one parse covers operator + value) - Remove the duplicate-name check in validateRoutes: filter names are object keys, and JS object literals cannot represent duplicate keys * fix(ts-sdk): correct compile-time filter map for money types; lock with new compile-error tests The conditional-type chains diverged from the runtime schemas where no @ts-expect-error test exercised them: - moneyComparison operators were between|outside; MoneyComparisonFilterSchema accepts gt|gte|lt|lte (ComparisonOperatorsEnum) - moneyComparison/moneyRange values used amount: number; MoneySchema.amount is a decimal string Replace the two parallel operator/value conditional chains with a single FilterTypeMap lookup interface (one row per filterType, directly comparable to its Zod schema; the TypedFilter indexed access doubles as a completeness guard) and add two moneyComparison @ts-expect-error assertions so the row can't drift silently again. * docs(ts-sdk): correct operator catalog, clarify classification entry points, use request-body wording - Catalog table: dateComparison and moneyComparison support only gt|gte|lt|lte (eq|neq rows were wrong); integerComparison value is int - Explain that definePlugin registers declarations while classifyFilters is the search-time step the full SDK client will call internally (standalone here because client wiring is out of scope, tracked under #645), and that the PoC's definePlugin passes routes through unvalidated - Merge the routes-vs-schemas bullets into one combined definePlugin example - Cross-reference the transforms PII warning for PluginError.sourceValue - Describe the classifier output as the request body sent to the API rather than 'wire body' (README, example, test names) * ci(ts-sdk): drop custom-filters example from the ci script The example is slated to be reworked against a real endpoint, so the ci script should not gate on it. It remains runnable manually via `pnpm example:custom-filters`. * refactor(ts-sdk): reduce custom-filters type tests to pure compile-time assertions The vitest wrapper around these assertions could never fail at runtime — the real gate is `tsc --noEmit`, where each @ts-expect-error directive guards a line that only errors while `as const` narrowing is in effect (removing the narrowing makes the directive unused, ts2578, failing the type check). Drop the runtime test scaffolding and the widening-demo block (which compiled clean by design and asserted nothing), and rename .spec.ts -> .ts so vitest no longer collects the file. The as-const trap remains documented in the extensions README and the example's comment block. * fix(ts-sdk): revert integerComparison to NumberComparisonFilterSchema The spec defines no integer filter model (lib/core numeric filters are NumberComparisonFilter / NumberRangeFilter / NumberArrayFilter only), so the SDK-side IntegerComparisonFilterSchema added in 21127a9 had no spec counterpart. The spec is the source of truth and changes flow from the spec to the SDK; an int-enforcing schema can follow a spec change that introduces one. integerComparison validates as a plain number again. * docs(ts-sdk): drop PoC status banners from transforms and custom-filters sections The extensions README sections should read as ready-to-use once published; remove the two status blockquotes and the (PoC) heading markers along with their TOC entries.

* feat(sdk): accept a Date as input on date schemas UTCDateTime, ISODate, and OffsetDateTime schemas now accept either an ISO string or a Date, normalizing a Date to a string before the existing strict validation. Output is still a Date, so hand-written transforms can return real Date fields and have them validate. * feat(sdk): model-based ToCommon/FromCommon helper types Take a named { model, sourceSchema, customFields } argument instead of positional <TSource, TCommon>. The common type is resolved from the model's base schema via EXTENSIBLE_SCHEMA_MAP, so authors pass their field specs rather than a prebuilt common schema, and one common type covers both directions. * docs(sdk): rework transforms example and refresh extensions docs Switch transforms.ts from calling buildTransforms() directly to using definePlugin(), which compiles the mappings internally and is the path plugin authors actually use. The example covers the mappings, hand-written-functions, no-custom-fields, and mappings-vs-functions cases with a runnable round-trip. In the README, move the "when you need as const" guidance into Best practices, restore the Important callout, and refresh the transform-helper API reference entries. Adds a test that a schema-only entry's toCommon is a compile error. * docs(ts-sdk): Removes unnecessary code comments * feat(sdk): require sourceSchema for transforms; always validate fromCommon Split SchemaInput into three variants — mappings (sourceSchema + mappings), functions (sourceSchema + both toCommon and fromCommon), and schema-only (customFields, no transforms). Providing a transform without a sourceSchema, or only one direction, is now a compile error. Since a transform entry always carries a sourceSchema, definePlugin wraps the hand-written fromCommon with safeParse validation against it unconditionally, mirroring toCommon against the common schema. * docs(ts-sdk): cleans up code comments

@overload

* test(01-01): add failing tests for BooleanComparisonFilter - Import BooleanComparisonFilter from filters.boolean (RED — module does not exist yet) - Tests: constructs with eq/neq + bool, coerces string operators, rejects invalid operators, rejects non-bool values * feat(01-01): implement BooleanComparisonFilter Pydantic model - BooleanComparisonFilter(operator, value: bool) with EquivalenceOperator (eq|neq) - @field_validator coerces string operators to EquivalenceOperator enum - Rejects operators outside eq|neq (ValidationError); rejects non-bool values - SDK-level only — no lib/core TypeSpec change (DP-14, CFP-02) * feat(01-01): export BooleanComparisonFilter from filters barrel - Add '# Boolean Filters' section to __all__ (between Numeric and Opportunity) - Add 'from .boolean import BooleanComparisonFilter' to import block - BooleanComparisonFilter now importable from common_grants_sdk.schemas.pydantic.filters * feat(01-02): add CustomFilterType enum and CustomFilterSpec dataclass - 11-value StrEnum (stringComparison / stringArray / *Range / *Array) per DP-14 - CustomFilterSpec(filter_type, description=None) mirrors CustomFieldSpec but intentionally omits `value` — operators derived at classify time (DP-07) * feat(01-02): add PluginRoutes / RouteDeclarations / RouteMethodFilters aliases Route-keyed type aliases for custom-filter declaration (DP-06). CustomFilterSpec referenced under TYPE_CHECKING to prevent circular imports at runtime; existing runtime import from .specs is unchanged. * feat(01-02): add routes slot to PluginConfig and define_plugin - Import PluginRoutes from .types - Add routes: PluginRoutes | None = None as last field on frozen PluginConfig - Add routes kwarg to both @overload signatures (2-overload approach; A1 verified green — mypy accepts routes independently of schemas narrowing) - Add routes to implementation signature and pass routes=routes to PluginConfig() - inject_transforms is byte-for-byte unchanged (iterates config.schemas only) - Positional-construction safety confirmed: only one PluginConfig( call site, using keyword args * test(01-03): add failing pytest suite for filter helpers, classification, and validation - Covers f.* helpers (eq, neq, gt, like, in_, not_in, between, outside) - Covers three-bucket classify_filters (default snake/camel, registered custom, ad-hoc) - Includes camelCase-alias landmine test (closeDateRange -> named field, not customFilters) - Includes test_oppfilters_mixed_case_roundtrip (Strategy A proof, RESEARCH A4) - Covers validate_routes raising on unknown filter_type and DEFAULT_FILTER_NAMES collision - Covers validate_filter_call raising on registered mismatch and ad-hoc invalid shape - No custom-vs-custom duplicate-name test (vacuous with dict semantics, DP-15 note) * feat(01-03): implement extensions/filters.py — f helpers, classifier, validators - _FHelpers class with all 12 methods (eq/neq/gt/gte/lt/lte/in_/not_in/like/not_like/between/outside) - FILTER_TYPE_SCHEMAS maps all 11 CustomFilterType members (integerComparison reuses NumberComparisonFilter) - DEFAULT_FILTER_NAMES frozenset includes both snake_case and camelCase aliases (landmine fix) - Strategy A (RESEARCH A4): normalize snake_case keys to alias form for OppFilters construction (OppFilters requires alias keys without populate_by_name; passing snake_case silently yields None) - validate_routes raises PluginError on unknown filter_type and DEFAULT_FILTER_NAMES collision - validate_filter_call raises PluginError on registered mismatch and ad-hoc invalid shape - classify_filters routes default keys (snake+camel) to named fields, custom/ad-hoc to customFilters - All 23 pytest tests pass including test_oppfilters_mixed_case_roundtrip (Strategy A proof) * refactor(01-03): wire filters public API + fix enum types for mypy - extensions/__init__.py: export classify_filters, f, validate_routes, CustomFilterSpec, CustomFilterType, PluginRoutes (six new public symbols) - filters.py: use typed enum values in _FHelpers (EquivalenceOperator, ComparisonOperator, ArrayOperator, StringOperator, RangeOperator) instead of string literals to satisfy mypy arg-type checks - black-format both modified files; make checks + make test pass (pre-existing test_generate_models_typecheck_with_pyright_strict failure excluded per success criteria) * feat(01-04): add codegen-free custom_filters.py example - Demonstrates route-keyed plugin declaration with STRING_ARRAY and STRING_COMPARISON custom filter specs - Classifies five consumer filters (2 default, 2 registered, 1 ad-hoc) into the ADR-0012 three-bucket wire body - Includes camelCase alias case (closeDateRange) to exercise the DEFAULT_FILTER_NAMES landmine path - Demonstrates PluginError raised and specifically caught on bad operator - Uses mode='json' on model_dump to produce JSON-serializable output - SDK-direct imports only, no codegen dependency * fix(01): unify PluginError contract across all three filter buckets Wrong-shaped default-bucket filters (e.g. an equivalence filter for the array-typed "status" field) previously leaked a raw pydantic ValidationError from the terminal OppFilters construction, while the registered and ad-hoc buckets wrapped failures in PluginError — so a consumer following the documented `except PluginError` pattern missed default-bucket errors. - Wrap the OppFilters construction in the same PluginError translation, making it the single validation point for default values (validated against each named field's real type, stricter than the permissive DefaultFilter check the dropped per-key call performed). - Narrow the ad-hoc branch's `except (ValidationError, Exception)` to `except ValidationError` so unexpected non-validation bugs propagate instead of being mislabeled as invalid filter shapes. - Document the uniform contract in the classify_filters docstring and cover both default-key forms (snake_case and camelCase alias) with regression tests. * docs(01): scrub internal planning shorthand from committed source Replace planning-doc IDs and working labels in comments/docstrings with the technical reasoning they pointed at, matching the TS custom-filters code style. Real references (ADR-0012, issues #646/#869, TS SDK parallels) are kept. No code changes. * feat: enforce int values for integerComparison via dedicated filter model Add IntegerComparisonFilter (StrictInt value) and point the integerComparison filter type at it instead of reusing the looser NumberComparisonFilter, mirroring the TS SDK's IntegerComparisonFilterSchema (z.number().int()). Fractional values and numeric strings now fail call-time validation; the strict annotation disables pydantic's lax str -> int coercion to keep cross-SDK behavior aligned. * refactor: drop unreachable duplicate-name guard from validate_routes The seen-set check could never fire: routes is dict-keyed, so duplicate filter names cannot exist by construction. Removing it also drops the noqa: C901 it forced. Parallel to the TS SDK dropping the same unreachable check from validateRoutes. * docs: use request-body framing for the classifier output Rename wire-body wording to request body in docstrings, comments, test names, and the example output label — the classifier result is the JSON body of the ADR-0012 search request sent to the receiver. Operator serialization notes (wire operator values for in/notIn) keep their wording. Matches the TS SDK's request-body wording. * test: cover money filter types in validate_filter_call moneyComparison and moneyRange had no extensions-level coverage — the money types are where the TS compile-time filter map drifted (range operators and a numeric amount against a schema taking gt/gte/lt/lte and a decimal-string Money). Lock the runtime layer here: valid Money shapes pass, array/comparison operator mismatches and numeric amounts raise PluginError. * style: apply black to integer filter and money tests * fix: use StrictBool in BooleanComparisonFilter for TS parity Lax pydantic bool coerces 1/0 and "true"/"false", which the TS SDK's z.boolean() rejects. Matches the StrictInt treatment already applied to IntegerComparisonFilter. * docs: correct routes validation claim in define_plugin docstring validate_routes() is an explicit registration-time check, not invoked by classify_filters() — matching the TS SDK where validateRoutes is a standalone function. * refactor: align PluginError paths with the TS filters.<name> convention Buckets 2/3 now raise path="filters.<name>" (TS uses filters.${filterName}); the bucket-1 OppFilters-construction wrap names the failing field(s) instead of the misleading routes.<resource>.<method> path. Also document mode="json" in the classify_filters return docstring — the python-mode dump is not JSON-serializable (dates, operator enums). * test: construct bogus filter spec directly CustomFilterSpec is not a frozen dataclass, so the __new__ + object.__setattr__ dance is unnecessary — dataclasses don't validate field types at runtime. * fix: conform DefaultFilter.value to the core spec's `unknown` The narrowed union (str | int | float | list | dict) diverged from filters/base.tsp (`value: unknown`) and the TS SDK's z.unknown(), and lax union coercion mutated inputs: True -> 1, so boolean custom filters serialized as integers on the wire. `Any` preserves inputs as-is; strict per-type checking stays with the typed filter models. * fix: accept eq/neq on numeric comparison filters per core spec NumberComparisonFilter.operator in filters/numeric.tsp widened to ComparisonOperators | EquivalenceOperators in protocol v0.3; the Pydantic model never tracked the change (the TS SDK did). IntegerComparisonFilter inherits the same surface so the numeric family stays consistent across SDKs. * fix: ship validated filter values and unify the PluginError contract validate_filter_call now returns the validated filter as a DefaultFilter and classify_filters places that in the wire body, so the payload is exactly what passed validation (lax coercion could previously diverge: a value validated as 42.0 shipped as "42"). - ad-hoc DefaultFilter instances are re-validated via model_dump instead of trusted; mutated instances now raise instead of serializing invalid operators with only a stderr warning - unknown spec.filter_type raises PluginError (uniform contract) instead of a bare KeyError when validate_routes was skipped - OppFilters construction errors name the failing filter from nested customFilters locs instead of misattributing them as default-filter failures * docs: correct filter-engine docstrings and demonstrate alias normalization - classify_filters bucket-1 description said keys normalize to snake_case; construction requires the camelCase alias form (the code and the docstring's own construction paragraph already said so) - mode="json" rationale named operator enums as non-serializable; StrEnum members serialize fine — only coerced date objects need it - the alias-detection map's comment described a lookup that never happens (only key membership is used) - DEFAULT_FILTER_NAMES comment now names both consumers of the dual-form set (validate_routes collision check AND bucket-1 routing) - the example passed closeDateRange (the form that needs no normalization) while calling it the landmine case; it now passes close_date_range and shows the snake->alias rewrite in the wire body - drop the hardcoded enum count, diff-relative "new" framing, and the codegen-only framing on define_plugin's docstring * test: lock wire-body integrity, normalization, and error-contract behavior - boolean true survives to the wire as JSON true (not 1) - registered filters ship the validated/coerced value, not raw input - mutated ad-hoc instances are re-validated and raise - unknown filter_type raises PluginError, not KeyError - snake_case form of an aliased default key normalizes to the alias (the _SNAKE_TO_ALIAS hit branch had no coverage) - model_dump(mode="json") round-trips through json.dumps with ISO dates - FILTER_TYPE_SCHEMAS covers every CustomFilterType member (catalog drift now fails CI instead of raising at runtime) - PluginError path is filters.<name> across all three buckets, and the multi-failure collective path is "filters" - eq/neq accepted on Number/Integer comparison per filters/numeric.tsp; integer filters reject True (bool subclasses int) and 100.0 (recorded divergence: JS cannot represent the distinction, StrictInt rejects) - all 12 f helpers parametrized over their wire operator/value shapes - the camelCase-alias landmine test's conditional assert could never fail on the path it guarded; assert the empty bucket outright * fix(py-sdk): reject bool values in numeric filter models bool subclasses int, so the int|float unions in NumberComparisonFilter, NumberArrayFilter, and NumberRange lax-coerced True -> 1 and shipped a number for a boolean — the same wire corruption the DefaultFilter.value widening fixed, surviving one layer down in the typed models. Also an internal inconsistency (IntegerComparisonFilter's StrictInt already rejects bool) and a divergence from z.number(), which rejects booleans. Pinned at both layers: schema-level rejection tests and a classify-time wire-integrity test (f.eq(True) on a numberComparison filter raises instead of shipping value 1). * fix(py-sdk): raise on duplicate snake/camel forms of a default filter Both forms normalize to the same OppFilters key, so plain dict assignment silently dropped whichever value the consumer's dict ordered first — a quiet wrong-results bug when merging filter dicts that use different naming conventions. Now raises PluginError, keeping the uniform error contract. Also collapses the bucket-1 normalization to a single _SNAKE_TO_ALIAS lookup (alias-form and unaliased keys fall through .get(key, key) unchanged, so the _ALIAS_TO_SNAKE membership branch was redundant) and documents that registered-spec lookup is exact on (resource, method): a non-matching pair downgrades every non-default filter to permissive ad-hoc validation with no signal. Route-scoping now pinned by tests at both lookup levels. * feat(py-sdk): surface first validation failure in PluginError messages str(PluginError) carried an error count but no error content, so the logged message a consumer sees ('2 error(s)') said nothing actionable; the detail lived only on .cause. Each wrapping site now appends a one-line summary of the first pydantic error (message + loc), and the example demonstrates the structured fields (.path, .cause) rather than teaching the lossy print(exc)-only pattern. Also collapses the customFilters arm of the bucket-1 field extractor: custom-bucket values are validated before OppFilters construction, so only default fields can fail there — the nested-loc branch was unreachable defensive code. * refactor(py-sdk): enforce filter-engine invariants at the type level Convert documented invariants into enforced ones: - CustomFilterSpec is frozen: a spec is an immutable declaration, and mutating one after registration would bypass validate_routes. - FILTER_TYPE_SCHEMAS is a MappingProxyType: the catalog is closed, so runtime mutation is not an extension point. - classify_filters takes Mapping[str, DefaultFilter | dict[str, Any]] instead of dict[str, Any], matching its documented contract so type checkers can catch e.g. a bare string value at the call site (Mapping keeps narrower caller dicts assignable where dict's invariance would reject them). - CustomFilterSpec joins the existing module-level specs import in types.py; the TYPE_CHECKING guard and forward-string reference were workarounds for a cycle that doesn't exist (specs.py imports .types only lazily). * docs(py-sdk): fix three inaccurate filter comments; ground TS parity claims Accuracy fixes, each verified by execution: - DEFAULT_FILTER_NAMES comment had the failure mode inverted: the aliases ARE the camelCase forms, so an alias-only set would drop snake_case keys into customFilters, not camelCase ones. - The example's alias-landmine note claimed an unnormalized snake key would land in customFilters; pydantic actually ignores the kwarg (extra='ignore', no populate_by_name) so the filter is silently dropped from the wire body entirely — now consistent with filters.py. - validate_filter_call's coercion example claimed "42" -> 42.0; the int|float smart-union resolves int first, so it coerces to 42. Grounding fixes: - TS parity notes cited TS SDK symbols (F object, validateRoutes, *FilterSchema names) that exist only on the unmerged TS PoC branch; rephrased as the planned/PoC surface so the claims can't silently rot if the TS shape changes before merge. - validate_routes' gov.<system>@<filterName> wording read as a settled protocol mandate; no ADR defines it yet, so it now describes the escape-hatch mechanism instead. - 'validate_routes catches this at registration time' overstated an opt-in check; reworded as guidance to call it. - 'three-bucket ADR-0012 classifier' implied bucket language in ADR-0012; the buckets are this implementation's framing, the ADR defines the request-body shape. - CustomFilterSpec docstring said operators are 'derived' from filter_type; nothing derives — the filter_type's validation model enforces the allowed set at call time. * test(py-sdk): dedupe filter tests, close coverage gaps Deletions (tests must discriminate — no test that a parametrized table already covers): - test_f_eq..test_f_outside were an exact subset of the test_f_helper_wire_values table; the table moves up into the f.* section and keeps the reserved-word cases. - test_boolean_comparison_filter_coerces_string_operator was a strict subset of test_boolean_comparison_filter_constructs. Additions: - Invalid-operator rejection parametrized across NumberComparisonFilter AND IntegerComparisonFilter: both share the rewritten two-enum validate_operator dispatch whose fallthrough relies on the enum-union field to reject unknown strings, but only the integer model was pinned. - define_plugin(routes=...) -> PluginConfig.routes passthrough. Tightening: - test_oppfilters_mixed_case_roundtrip's disjunctive customFilters assertion could not fail on the path it guards; now asserts the bucket is absent outright, matching the sibling test's stated standard. - test_registered_filter_ships_validated_value_not_raw_input now asserts the coerced type (int), pinning the documented "42" -> 42 behavior. * fix(py-sdk): revert integerComparison to NumberComparisonFilter The spec defines no integer filter model (lib/core numeric filters are NumberComparisonFilter / NumberRangeFilter / NumberArrayFilter only), so the SDK-side IntegerComparisonFilter added in 70b1f97 had no spec counterpart. Mirrors the TS-side revert on #884: the spec is the source of truth and changes flow from the spec to the SDK; an int-enforcing model can follow a spec change that introduces one. integerComparison validates as a plain number again. * refactor(py-sdk): align PluginRoutes shape with the TypeScript SDK The TS SDK nests per-method filter declarations under a RouteDeclarations wrapper ({resource: {method: {filters: {name: spec}}}}); the Python SDK mapped filter names directly at the method level. Cross-SDK parity testing showed the asymmetry is hazardous for consumers porting between SDKs: a Python-shaped routes object passed to the TS classifier silently downgrades every registered filter to ad-hoc shape-only validation. PluginRoutes is now {resource: {method: RouteDeclarations}} where RouteDeclarations is a TypedDict with an optional filters key, mirroring the TS interface (optional so a method can be declared without filters and so future declaration kinds can ride alongside). validate_routes skips methods with no filters key and its error path gains the filters segment to match the TS path string; classify_filters resolves registered specs through the filters key. Tests and the runnable example are updated to the new shape. * Address review feedback on custom-filters PoC - Export validate_filter_call from extensions __init__ (parity with TS SDK) - Note classify_filters' opportunity-bound shape; link resource-agnostic follow-up (#896) - Rename opp_key to alias_key in classify_filters - Drop the future-declaration-kinds clause from RouteDeclarations docstring * docs(py-sdk): keep user-facing docstrings free of ADR refs and cross-SDK notes Docstrings rendered to SDK users should describe behavior in plain language; ADR numbers and TS-SDK implementation details belong in contributor-facing comments. The tracking link for the classifier's Opportunity binding moves from the docstring to a code comment above classify_filters.

Brings the latest HOLD-transforms: #924 (codegen-free plugin framework, now on HOLD-transforms as a squash) plus dependency bumps (#919 audit remediation, #922 runtime, #914 catalog, #926 all-deps, #927 astro). HOLD-filters already carried #924's content (via the earlier 888-python-sdk-refactor merge), so the squash arrived via a different lineage. Resolution: - types.py: keep the filters CustomFilterSpec import; drop the duplicate PassthroughModel the two #924 lineages produced. - __init__.py: union of the #924 public surface and the filters exports; dedupe SchemaWithTransforms. - pnpm-lock.yaml: regenerated to match the merged catalog (the merge took the HOLD-transforms catalog in pnpm-workspace.yaml but not the matching lock; reconciled via pnpm install --no-frozen-lockfile; frozen install now passes). Verified: py-sdk mypy + 365 pytest; ts-sdk check:types + 515 vitest; pnpm --frozen-lockfile consistent.

github-actions · 2026-06-18T19:46:19Z

🚀 Website Preview Deployed!

Preview your changes at: https://cg-pr-918.billy-daly.workers.dev

This preview will be automatically deleted when the PR is closed.

# Conflicts: # lib/python-sdk/common_grants_sdk/extensions/__init__.py # lib/python-sdk/common_grants_sdk/extensions/specs.py # lib/python-sdk/common_grants_sdk/extensions/types.py # lib/ts-sdk/__tests__/extensions/define-plugin.spec.ts # lib/ts-sdk/package.json # lib/ts-sdk/src/extensions/README.md # lib/ts-sdk/src/extensions/define-plugin.ts # lib/ts-sdk/src/extensions/index.ts # lib/ts-sdk/src/extensions/types.ts # pnpm-lock.yaml # website/src/content/docs/governance/adr/0022-plugin-framework.mdx

* feat(py-sdk): typed-declaration custom-filter authoring DX Add a static-typing layer over the throw-based custom-filter runtime so a plugin author registers a route's filters by declaring a TypedDict and a consumer gets per-key narrowing on the registered filters with no call-site annotations. - extensions/routes.py (new): RouteFilters[TF] phantom carrier; frozen, covariant ResourceRoutes / Routes carriers (PEP 696 defaults). Static-typing only; the runtime classifier does not consume them. - filters/opportunity.py: OpportunityFilters TypedDict bag keyed to the standard opportunity filters; clean value-model aliases (StringArray, NumberComparison, ...) for author declarations. - extensions/plugin.py: define_plugin gains routes= (default Routes()); Plugin carries it and exposes search_filters_type(), a constrained-self projection that recovers the registered filter TypedDict. Transport is out-of-scope (ADR-0022), so this returns the typed filter slot, not a client. - examples/typed_filters.py: author + consumer DX with assert_type narrowing. - test_plugin.py: route threading, default carrier, frozen carriers. Narrows cleanly under both mypy and pyright. The covariant dataclass fields need type: ignore[misc] for mypy (covariant-as-parameter) but not pyright; the constrained-self projection narrows identically under both. make checks + pytest green (369 passed). * docs(py-sdk): end-to-end consumer example searching with custom filters Add examples/consumer_search_with_filters.py: the full downstream flow an adopter writes — author registers opportunities.search custom filters (typed OppSearchFilters declaration + runtime PluginRoutes), a stubbed client runs classify_filters to build the three-bucket OppFilters request body and returns typed OpportunityBase rows, and the consumer builds filters with f.* and iterates the results. Transport is stubbed per ADR-0022 (the SDK client is a typed slot; real transport is downstream). The production client/opportunities.py:search() is NOT modified — it currently takes only search+status; the filters= parameter shown is the proposed signature being finalized (PR #934 open question 1), kept local to the example. Runnable: poetry run python examples/consumer_search_with_filters.py make checks + pytest green (369 passed); mypy + pyright clean. * docs(py-sdk): drop redundant filters annotation in consumer example * feat(py-sdk): overload f.* builders to precise filter models Overload each f.* builder so its static return type is the precise filter model the value implies — f.eq("x") -> StringComparisonFilter, f.eq(5) -> NumberComparisonFilter, f.eq(True) -> BooleanComparisonFilter, f.in_([str]) -> StringArrayFilter, f.gte(3) -> NumberComparisonFilter, f.between(0,1) -> NumberRangeFilter. Value types that don't map to a precise model (Money dicts, ISODate strings, date/Money ranges) fall through to a DefaultFilter overload, so ad-hoc and Money/date usage still works. This closes the cross-SDK parity gap flagged on the consumer example: the runtime-ergonomic f.* call site now composes with the typed OppSearchFilters bag with no annotation and full per-key narrowing. Runtime dispatches on value type (bool before int/float, since bool is an int subclass) to construct the matching model; model_dump() wire shape stays {operator, value}, identical to before — classify_filters and the wire body are unchanged. Fallout fixed: - classify_filters consumer_filters param widened from Mapping[str, DefaultFilter | dict] to Mapping[str, BaseModel | dict] (the precise models are siblings of DefaultFilter, not subclasses). - consumer_search_with_filters.py: search(filters=) typed to OppSearchFilters; the consumer call composes annotation-free with assert_type proof; "doesn't compose" caveat removed. - test_f_helper_wire_values asserts the JSON wire dump (the real contract) rather than .value, which is now a sub-model for range filters. Type-checker note: pyright flags the precise/Any overload overlap (and the bool/int overlap) which mypy does not; suppressed per-builder on the first overload. Both checkers clean on the changed files. make checks + pyright + pytest green (369 passed); all three examples run. * chore(py-sdk): tidy typed-filter authoring DX - test_plugin fixture: declare agency with the StringArray alias (the canonical authoring shape) rather than list[str] - plugin: import the TF TypeVar from routes instead of redeclaring it - plugin: note why the shared module-level routes default is safe to reuse - consumer example: use list[]/type[] builtins to match SDK style * test(py-sdk): pin numberRange round-trip; tidy filter comment and example import - add a NUMBER_RANGE registered-filter round-trip test: f.between(int, int) validates and model_dump recurses the NumberRange sub-model to {min, max} (the moneyRange analog was covered; the numeric path was not) - drop the inline bool-before-int comment in _equivalence; the block comment above the eq/neq overloads already explains the bool ⊂ int ordering - drop the redundant StringArrayFilter import in typed_filters.py and use the StringArray alias in the runtime guards (StringArray is StringArrayFilter), matching the NumberComparison alias usage alongside it * fix(py-sdk): type range fields with explicit *RangeFilter to remove a name collision DateRange / MoneyRange / NumberRange are the range value sub-models exported from the filters package root, so reusing those names as filter aliases in .opportunity made one public name resolve to two different types by import path: a consumer extending OpportunityFilters who imported a *Range name from the package root got the value sub-model, not the filter — silently breaking per-key narrowing for any custom range filter. - drop the three colliding *Range filter aliases (the *Comparison / *Array aliases have no clash and stay) - type the standard range fields (closeDateRange, *AwardAmountRange) with the explicit *RangeFilter names, which are unambiguous everywhere - example imports the *RangeFilter names from the package root * docs(py-sdk): drop PR-ref and point-in-time framing from consumer example * docs(py-sdk): move ADR-0022 rationale out of comments; fix runtime-unchanged note Comments carry brief intent only; the transport-scope rationale lives in the PR description. Also corrects the typed_filters docstring, which wrongly listed f.* among the unchanged runtime (f.* now returns precise filter models). * docs(py-sdk): trim wire-body narration from filter docstrings Keep docstrings to quirks and intent. Remove the runtime/wire-shape paragraph from _FHelpers (the bool-ordering quirk it restated is already documented on the bool overload and the runtime guard), and drop the request-body / wire-shape clauses from the test_f_helper_wire_values docstring, leaving the assertion intent. * address review: drop search_filters_type carriers, wire client custom filters - Remove search_filters_type() + Routes/ResourceRoutes/RouteFilters carriers; reshape Plugin.routes to PluginRoutes (ADR-0022 Decision #10) - Opportunities.search() takes filters=/routes=, building the body via classify_filters (mode="json" so date filters serialize to ISO strings) - Rewrite consumer example to call the real Client + plugin.routes - Rewrite OpportunityFilters docstring; drop the redundant typed_filters example

…filters design (#941) * docs(adr): #898 use operator (not operation) in ADR-0012 filter schema * docs(adr): #898 record route-keyed custom-filter declaration in ADR-0022 * docs(adr): #898 align ADR-0012 value-field wording with TypeSpec

@ts-expect-error

…both SDKs (#928) * spike(ts-sdk): classify custom filters behind opportunities.search() Move custom-filter classification off the consumer and into the API client: `search({ filters, routes })` classifies a flat bag via `classifyFilters` inside `buildSearchBody` — matching the existing `statuses` shorthand — so consumers no longer call `classifyFilters` by hand. `routes` arrives per-call, mirroring the `schema` option. Also exports `FILTER_TYPE_SCHEMAS` read-only in both SDKs for consumer introspection. Spike / exploratory — see SPIKE-FINDINGS.md. * style(ts-sdk): apply prettier formatting for check:format * chore(spike): drop SPIKE-FINDINGS.md; findings live in the PR description * spike(python-sdk): classify custom filters behind opportunities.search() Mirror the TS change in the Python client: search() gains `filters` (flat bag) + `routes`, classifies via `classify_filters` to build the OppFilters body — the same role the `status` shorthand plays — and `status` becomes an optional shorthand. Brings the search-endpoint integration to full TS/Python parity (the catalog export was already in both SDKs). * fix(sdk): validate default filters against their real type; drop speculative export classifyFilters now validates default filters (status, closeDateRange, ...) against their real field schema in OppDefaultFiltersSchema and raises FilterError, matching the Python SDK. Default fields previously got a shape-only check and were silently deferred to the server, so a malformed default filter shipped without complaint. Also removes the FILTER_TYPE_SCHEMAS public export from both SDKs: it has no in-repo consumer and the TS copy was a plain mutable const despite the read-only claim. It stays internal until a real consumer need lands (a public export is a one-way door). * fix(sdk): raise on status shorthand/filters-bag collision search() now raises FilterError when status is supplied via both the statuses/status shorthand and a status key in the filters bag, instead of silently letting the shorthand win (last-write). Symmetric across both SDKs. Adds error-path tests in both SDKs that a malformed registered filter surfaces FilterError out of search(), and clarifies the CustomFilterBag typing doc (envelope check at the type level; per-filterType value validation is runtime). * docs(ts-sdk): fix filter-narrowing wording and @ts-expect-error count The compile-time assertions file claimed two @ts-expect-error directives but ships one; corrected. Describes the static guarantee as filter-envelope checking ({ operator, value } shape), with per-filterType value validation noted as runtime (classifyFilters). * refactor(sdk): client-bound routes and surfaced filter feedback - Bind `routes` at client construction instead of passing it per call to search() in both SDKs. routes is fixed plugin config, not per-call data; in TS the client is generic over the routes type so the filter-name narrowing on search({ filters }) still resolves from the bound routes. - Python: surface the server's filterInfo/sortInfo (including filterInfo.errors) instead of fabricating empty placeholders. client.search validates into Filtered and the pagination layer preserves the first page's response envelope, matching the TS SDK's behavior. - Add a Python classifier-level test for MoneyRangeFilter default-field validation, the one default filter type not previously covered. * fix(sdk): omit empty filters from search wire body; keep R generic in buildSearchBody - Python search() omits the `filters` key when neither a filters bag nor the status shorthand is given, matching the TS SDK (which only sets the key when present) — identical wire body on the no-filter path. - TS buildSearchBody takes SearchOptions<OppSchema, R> so the private helper preserves the routes generic instead of widening to the default and casting. * feat(sdk): fail-soft custom-filter validation in search() (collect, don't throw) Custom-filter validation inside opportunities.search() now collects errors instead of throwing on the first bad filter, mirroring the ADR-0022 transform { result, errors } contract. Both SDKs, identical wire/response behavior. - classifyFilters / classify_filters return { result, errors }: a key that fails validation is dropped from the request body and its FilterError collected, rather than aborting the whole call. - search() merges the collected errors into the response's filterInfo.errors (flattened "path: message", client errors ahead of server) and never throws on a call-time filter problem. - status collision (statuses shorthand + filters.status) resolves filters-wins with a collected warning, instead of throwing. - validateRoutes / validate_routes (registration-time) still throw — a malformed plugin declaration has no result to return. * chore(python-sdk): drop ADR-0012 refs from filter comments Strip the ADR citation from the Filtered-validation comment and the filter/sort-info test docstring. Keeps the intent and the Filtered IS-A Paginated quirk; ADR rationale belongs in the PR description, not inline. * fix(sdk): null-safe filterInfo merge + validate routes at construction - TS: guard mergeFilterErrors against a missing filterInfo on the auto-paginate path so a dropped filter stays fail-soft (never throws) - Both SDKs: validate routes at client construction (validateRoutes was exported but never wired in); README updated to match - Strip stale comment framing (a "today", a review "FINDING" label) and a tooling prefix from published source - Tests: cut a wording-coupled assertion, rename a mislabeled test, add regression tests for the two behavior changes * test(ts-sdk): pin client-before-server order of filterInfo.errors Mirror the Python ordering assertion: a search whose dropped-filter error must precede the server-provided error. Closes the cross-SDK coverage gap where TS asserted only error membership, not order. * style(ts-sdk): prettier-format the routes-validation test * feat(sdk): reject custom filters on unsupported routes; address review feedback Route-support validation: validate_routes / validateRoutes now raise FilterError when custom filters are declared on a route that does not support them (only opportunities.search is filter-capable), in both SDKs, each with a regression test. Also from review: - ClassifyResult is generic (Generic[T] / ClassifyResult<T>) so the result container can extend past opportunities; classify returns ClassifyResult[OppFilters]. - validateFilterCall / validate_filter_call are now internal — dropped from the public extensions surface; classifyFilters runs call-time validation. Docs updated. - Rename SearchFilterNames -> CustomFilterNames (TS) for vocabulary consistency. - Client.search: note that request_data already carries the assembled filters. - De-emphasize a shouted code comment.

* feat(core): add boolean and integer comparison filter models Add BooleanComparisonFilter (eq/neq against a boolean) and IntegerComparisonFilter (comparison + equivalence operators against an integer), exported via filters/index.tsp at protocol v0.3.0. Refs #895. * docs: correct changeset — comparison filters emit to JSON Schema only Filter models aren't route-referenced, so the openapi3 emitter omits them; only the json-schema emitter outputs them. The changeset claimed both. * fix(core): version new comparison filters at v0.4 v0.3 is already published, so BooleanComparisonFilter and IntegerComparisonFilter first ship in v0.4. Add v0_4 to the Versions enum and retag both models; correct the changeset to match. * chore: remove accidentally committed .venv and gitignore it The lib/python-sdk/.venv virtualenv (1,039 files) was swept in by an overly broad git add. Untrack it and add .venv/ to the python-sdk .gitignore so it can't recur.

#970) * chore(python-sdk): switch type gate from mypy to pyright (basic) * feat(python-sdk): make OpportunityFilters an open TypedDict (extra_items) * feat(python-sdk): add SearchResult/ListResult with per-row parse error partitioning * feat(python-sdk): typed custom-filters authoring surface and client * feat(python-sdk): add typed custom-filters example, type fixtures, and test * fix(python-sdk): export parse_batch from the client package * fix(python-sdk): address self-review findings (sort_info, collision guard, pytest pythonpath) * ci: use full clone in bump-version workflow to prevent changeset hang actions/checkout defaulted to a shallow depth-1 clone. When a PR's changesets were added in commits before HEAD, `changeset version` deepens the clone via `git fetch --deepen` to find each adding commit, and that fetch hangs the job until the 6h runner cap. fetch-depth: 0 gives full history so the deepen path is never entered; timeout-minutes bounds any future hang. * docs(py-sdk): fix stale review comments; normalize status shorthand echo - Correct results.py/opportunities.py comments: page-aggregation preserves the first page's sort/filter info via model_copy (the getattr fallbacks only fire on the empty-result path that returns a plain Paginated), not "aggregation drops them". - Update three _FHelpers comments that referenced the deleted filter_type machinery (CustomFilterSpec / FILTER_TYPE_SCHEMAS) to the current route-TypedDict value-model mechanism. - status shorthand now stores plain str values so filter_info.filters echoes the same shape as the classified path; the wire request is unchanged. * feat(py-sdk): raise on invalid standard/registered filter values client.opportunities.search() now raises FilterError on an invalid value for a standard or registered filter (and on a conflicting status shorthand) before sending the request, instead of collecting it into filter_info.errors. Ad-hoc (unregistered) keys stay best-effort and are dropped if unusable; filter_info.errors now carries server-returned errors only. classify_filters stays fail-soft (still collects, never raises) and tags standard/registered failures as strict so the resource client can act on them. Adds a runnable runtime unhappy-path demo to the typed-custom-filters example. Part of #966. * ci(py-sdk): conform Node setup to repo standard (setup-node@v6 + .nvmrc) The pyright type gate's Node step used actions/setup-node@v4 with a hardcoded node-version "20", diverging from every other workflow (setup-node@v6 + node-version-file: .nvmrc, which pins 22) and undershooting the repo's node engines floor (>=22). Match the standard. * docs(py-sdk): note deprecation of the status shorthand on search() The status param duplicates the status key in filters; flag it as deprecated so consumers get a heads-up before it is removed.

) * fix(py-sdk): use get_client() in the custom-filters consumer example Client(config, routes=...) leaves the Client generics unbound, so search(filters=...) fell back to OpportunityFilters and rejected the OppSearchFilters filters input. This switches the example to use plugin.get_client(), which binds FiltersT/ItemT, and adds /examples to pyright to catch similar type issues earlier. * refactor(py-sdk): remove routes and schemas param from Client This makes `Plugin.get_client()` the single correct path for accessing a Client that has been scoped with custom filters and custom fields from a Plugin, and avoids the type inference errors that stemmed from a plugin consumer directly calling `Client(routes=..., schemas=...)` * fix(py-sdk): type hints in examples/custom_filters.py Applies pyright checks to examples (excluding intentional failures file) and fixes a lingering IDE-level error in custom_filters.py by slotting OppSearchFilters in PluginRoutes * docs(py-sdk): fix inconsistent client docstrings

#975) * feat(ts-sdk): close PluginRoutes keys to registered resource and method names * feat(ts-sdk): type registered custom-filter values by their declared filterType * feat(ts-sdk): add parseBatch row partitioning with ParseFailure detail * feat(ts-sdk): add fail-fast categorizeFilters that throws before the request * feat(ts-sdk): partition fetch helpers via parseBatch and add put/patch transport verbs * feat(ts-sdk)!: registry-driven resources, fail-fast filters, partitioned results * feat(ts-sdk)!: remove fail-soft classifyFilters in favor of categorizeFilters * feat(ts-sdk): plugin.getClient() builds a client typed by the plugin's schemas and routes * docs(ts-sdk): getClient consumer examples, fail-fast filter docs, changeset * refactor(ts-sdk): review polish and statuses deprecation - Drop unused write-verb transport surface (Client.put/patch, mutate) and the one-entry resource registry; Opportunities constructed directly - Shrink the public barrel (RESOURCE_NAMES, FILTER_TYPE_SCHEMAS removed) - encodeURIComponent the id path segment; auth-passthrough test for getClient - PII caveats on ParseFailure.raw / FilterError.sourceValue; redacted log patterns in README and example - Deprecate the statuses search shorthand in favor of filters.status - Honest optional filterInfo typing, undefined-spec guard in validateRoutes, example output-ordering fix, doc/comment accuracy * refactor(ts-sdk): keep the classifyFilters name for the fail-fast classifier Nothing consumes this surface yet and the Python SDK will converge on the same erroring pattern under classify_filters, so keep the name consistent across SDKs instead of renaming to categorizeFilters alongside the fail-soft removal.

* feat(ts-sdk): enforce operator/value structure for ad-hoc filters Ad-hoc (unregistered) filters were validated shape-only against DefaultFilterSchema, which accepted any value for a known operator. They now validate against FILTER_TYPE_SCHEMAS, the same map registered filters use: an ad-hoc filter is valid when its {operator, value} is well-formed for some known filterType. This reuses the single source of truth (and picks up new filter types automatically) rather than a parallel schema, and rejects combinations no filter family supports (e.g. `in` with a non-array value, `between` with a scalar). On a mismatch, FilterError carries a targeted, operator-aware message instead of the value passing through. * docs(ts-sdk): rework custom-filters example and teach the mock server custom filters Restructure examples/custom-filters.ts into three scenarios: happy path (mock server, mixed default/registered/ad-hoc filters with a typed custom field), authoring errors (default-name collision, unknown filterType, misspelled resource/method caught at compile time and at runtime), and consumer errors (registered wrong value family, ad-hoc operator/value mismatch). Upgrade the mock server to apply the standard status filter, echo received filters (including customFilters) into filterInfo, and carry a programCode custom field. Export createMockServer() with a guarded auto-start so `pnpm example:server` still runs it standalone.

# Conflicts: # lib/python-sdk/poetry.lock # lib/python-sdk/pyproject.toml # lib/ts-sdk/package.json # lib/ts-sdk/src/extensions/README.md

The grants-gov transforms example landed on main under the mypy gate; HOLD-filters' pyright gate flags the top-level import as private (cg_grants_gov.__all__ only exports grants_gov). Import from the models module, which declares it.

* feat(py-sdk): add integer comparison filter Plugin authors can register an integer comparison filter, and consumers can filter by it. It accepts the comparison and equivalence operators against an integer value, and rejects a value that is not an integer, such as 3.5 or a boolean. * feat(py-sdk): validate custom filters in define_plugin() An invalid filter registration is now caught when the plugin author defines the plugin, instead of later when a consumer builds a client. The author sees the mistake right away rather than passing it downstream to whoever uses the plugin. * feat(py-sdk)!: fail fast on invalid custom filters An invalid filter value now raises a FilterError before the request is sent, whether it is a standard, registered, or ad-hoc filter. An invalid ad-hoc filter used to be dropped without a word, which quietly widened the consumer's search instead of telling them they made a mistake; now an ad-hoc filter whose value does not fit its operator, for example "in" with a plain string instead of a list, is rejected too. Only real filter types are accepted as custom filters. BREAKING CHANGE: classify_filters returns the request body directly and raises on the first invalid filter. ClassifyResult and FilterError.strict, which only supported the old drop-and-collect behavior, are removed. * docs(py-sdk): update custom filters example Show that an invalid filter raises a FilterError instead of being collected and dropped, so the example matches how the client behaves. * fix(py-sdk): reword the invalid filter error Say "valid filter model" in the message a plugin author sees when a route registers a filter with the wrong type, matching the rest of the wording. * docs(py-sdk): custom filters example end to end Rework the custom filters example into one runnable walk through the happy path and the unhappy paths: a search mixing standard, registered, and ad-hoc filters against a small mock server, an invalid registration caught at define_plugin, a misspelled resource or method caught when the route is built, and an invalid filter value raising before any request. Add the mock server the happy path talks to. The mistakes an IDE also flags are marked so a reader can see what the type checker catches on top of the runtime guard. * ci(py-sdk): ignore grants.gov transforms example The grants.gov example includes a reference to the published grants.gov plugin that's not imported in this project. * docs(py-sdk): fix inconsistent changeset

* fix(py-sdk): ship PEP 561 py.typed marker The package is fully type-annotated but shipped no py.typed marker, so downstream type checkers treated every common_grants_sdk import as untyped. Adding the marker lets consumers pick up the SDK's inline annotations, matching the TypeScript SDK which already ships its .d.ts. * fix(ci): write multiline py_changelog via heredoc in version job The version job greps all changesets for "common-grants-sdk" and wrote the result to $GITHUB_OUTPUT as key=value. When more than one changeset bumps the package (a feature + this py.typed patch), the grep result is multiline and GitHub Actions rejects the key=value form. Use the heredoc delimiter form so any number of changesets is handled.

# Conflicts: # lib/python-sdk/poetry.lock # lib/python-sdk/pyproject.toml

…953)" This reverts commit c861b40. We want to group these changes with other changes to the core library that will be released with v0.4.0 rather than releasing them with the next SDK version.

Resolves the integer-vs-number divergence from the PR #988 review thread: Python's IntegerComparisonFilter rejected floats while the TS SDK aliased integerComparison to NumberComparisonFilterSchema and accepted them, so the same registered filter behaved differently per SDK. With the core integer filter model moved off this release, neither SDK ships a filter type the spec does not define: integerComparison is removed from the Python models/exports/catalog and the TS CustomFilterType union and schema map. NumberComparisonFilter already accepts integer values (matching JSON Schema's number type), so integer filtering needs no dedicated type. The catalog is now 10 types, identical across SDKs.

jcrichlake and others added 24 commits May 13, 2026 15:39

Merge remote-tracking branch 'origin/main' into HOLD-transforms

b3cc578

Merge main into HOLD-transforms (2026-05-21)

211ecec

[Issue #736] ADR: Nullable Optional Fields (#855)

3a78941

* Initial ADR * Formatting ADR * Fixing spelling

Update code.json metadata (#874)

fd14d4a

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Merge branch 'main' into HOLD-transforms

60c31a3

Merging main in

49d38a0

Update examples file

debd55b

Addressing PR comments

0f85b79

Fixing pnpm merge conflict

a60ff42

SnowboardTechie added sdk Issue or PR related to our SDKs typescript Issue or PR related to TypeScript tooling python Issue or PR related to Python tooling ts-sdk Related to TypeScript SDK py-sdk Related to Python SDK enhancement New feature or request labels Jun 16, 2026

Base automatically changed from HOLD-transforms to main June 22, 2026 17:38

SnowboardTechie and others added 3 commits June 22, 2026 15:37

github-actions Bot added the website Issues related to the website label Jun 25, 2026

SnowboardTechie added 2 commits June 26, 2026 12:32

github-actions Bot added the core Issues related to @common-grants/core library label Jun 26, 2026

SnowboardTechie and others added 6 commits July 6, 2026 09:52

Merge main into HOLD-filters (2026-07-07)

1f3f98b

# Conflicts: # lib/python-sdk/poetry.lock # lib/python-sdk/pyproject.toml # lib/ts-sdk/package.json # lib/ts-sdk/src/extensions/README.md

SnowboardTechie mentioned this pull request Jul 8, 2026

CommonGrants: SDK Plugin Enhancements * HHS/simpler-grants-gov#8765

Open

6 tasks

This was referenced Jul 10, 2026

fix(py-sdk): ship PEP 561 py.typed marker #1002

Merged

docs(sdk): add grants.gov custom-filters search examples (Py + TS) #1003

Closed

SnowboardTechie and others added 2 commits July 13, 2026 15:23

Merge main into HOLD-filters (2026-07-13)

9689815

# Conflicts: # lib/python-sdk/poetry.lock # lib/python-sdk/pyproject.toml

SnowboardTechie requested a review from widal001 July 13, 2026 23:24

SnowboardTechie removed the DO NOT MERGE label Jul 13, 2026

SnowboardTechie marked this pull request as ready for review July 13, 2026 23:24

Revert "feat(core): add boolean and integer comparison filter models (#…

af1f759

…953)" This reverts commit c861b40. We want to group these changes with other changes to the core library that will be released with v0.4.0 rather than releasing them with the next SDK version.

widal001 removed the core Issues related to @common-grants/core library label Jul 15, 2026

This was referenced Jul 15, 2026

fix(sdk): drop integerComparison from both SDK filter catalogs #1011

Merged

chore: delete leftover transforms changeset (already released in 0.5.0) #1012

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HOLD-filters: Custom filters PoC (TypeScript + Python)#918

HOLD-filters: Custom filters PoC (TypeScript + Python)#918
SnowboardTechie wants to merge 62 commits into
mainfrom
HOLD-filters

SnowboardTechie commented Jun 16, 2026

Uh oh!

github-actions Bot commented Jun 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

SnowboardTechie commented Jun 16, 2026

Summary

Changes proposed

Context for reviewers

Additional information

Uh oh!

github-actions Bot commented Jun 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants