Ryan capnweb compat#83
Open
ryanrasti wants to merge 12 commits into
Open
Conversation
Adds toRpc/fromRpc adapters bridging the @expose vocabulary to capnweb RPC: - toRpc wraps a class instance in a Proxy over a capnweb RpcTarget that exposes exactly the @expose surface (via exoeval's getTool), presented as prototype properties the way capnweb requires. Wrapping propagates recursively through return values, promises, arrays and plain objects, so builders compose over the wire with no per-class glue. Outgoing functions are adapted so their args are unwrapped and results wrapped. - fromRpc is the dual: it unwraps shim proxies back to the raw objects (relying on capnweb's stub identity preservation for round-tripped capabilities) and adapts incoming functions -- notably capnweb record-replay closures -- so typegres can call them with raw values and get raw values back, synchronously when the replay is fully local. capnweb is vendored as a file dependency on packages/capnweb (gitignored); clone ryanrasti/capnweb#generalize_record_replay there and build it to work on this branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The capnweb twin of src/rpc.test.ts: the landing-page query shape -- users().where(...).select(...).execute(conn) -- authored client-side inside stub.map(), shipped as capnweb record-replay closures, and replayed server-side against the real @expose-shimmed query builder on Postgres. Covers select/insert/update/delete + returning, by-value closure captures, the @expose deserializeRows gate, non-exposed column access failing RowType validation, and hydrated class instances crossing the wire as opaque @expose-gated capabilities rather than data. Driving the real builder surfaced two capnweb-side needs now in packages/capnweb (zero-argument closures for .set(() => ...), and identity preservation for path-form references so execute(api.conn) receives the raw Connection). Also extracts the in-memory capnweb harness into src/exoeval/capnweb-harness.ts, shared with the shim unit tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…lTarget() capnweb no longer unwraps round-tripped stubs automatically at deserialization -- stubs always arrive as stubs, and capnweb just exposes getLocalTarget() (a CapabilityServerSet-style explicit recovery helper). The unwrap policy now lives here: fromRpc() recovers the object behind any stub that points back at our side and, when it's a shim proxy, maps it to the raw builder object. Genuinely remote stubs pass through unchanged. Externally identical shim behavior; all shim and e2e tests unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
exoeval and capnweb are separate RPC approaches that share only the @expose vocabulary (exoeval/tool.ts). The shim, harness, and unit tests now live in src/capnweb/ (shim.ts, harness.ts, shim.test.ts), importing the shared tooling from ../exoeval/tool. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Names the transport like its capnweb twin (rpc-capnweb.test.ts), now that there are two e2e RPC suites side by side. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
hydrateRows builds instances with Object.create(shape's prototype), which drops the instance-level toolFieldsSymbol marker that @expose field decorators register at construction -- so getTool/exposedFieldsOf saw no exposed fields on hydrated rows and their capability surface was empty over RPC. Copy the marker descriptor from the shape onto each hydrated instance, the same way reAlias already does. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e call
A replayed closure's captures are owned by the delivering call's args
payload, but typegres retains callbacks past the call -- a .select()
callback replays at execute() time. wrapIncomingFunction now dup()s the
incoming function (closures and stubs both support dup) and releases the
dup via FinalizationRegistry when the adapted wrapper is collected.
Surfaced by extending the hydrated-rows e2e test to prove returned row
capabilities are live: the client stubs a hydrated row's columns into a
follow-up query (.select(() => ({ idAgain: row0.id, ... }))) -- the first
closure in the suite that captures an outer stub.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ord_replay) packages/capnweb un-gitignored and registered as a submodule pinned at efebddf (getLocalTarget tip of the record-replay closure branch); package-lock picks up the file: dependency. NOTE: the fork's generalize_record_replay is pushed only through 43dc308 — the top 4 commits (sync replay, zero-arg closures, RpcStub import fix, getLocalTarget) need a `git push` from packages/capnweb before anyone else can clone this pin. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Un-ignoring packages/ for the capnweb submodule surfaced it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds end-to-end RPC coverage for typegres query-builder usage over two RPC mechanisms (exoeval-evaluated closures and capnweb record/replay closures), plus a capnweb shim that enforces the existing @expose capability surface across the RPC boundary.
Changes:
- Added comprehensive integration tests for “typegres over exoeval RPC” and “typegres over capnweb RPC”, including
@expose-gating scenarios and scalar/hydrate behaviors. - Introduced a capnweb
toRpc/fromRpcshim + in-memory harness to adapt@exposecapability objects to capnweb’s RPC model. - Updated query hydration to preserve the
@exposefield-marker (toolFieldsSymbol) on hydrated instances.
Reviewed changes
Copilot reviewed 8 out of 11 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/rpc-exoeval.test.ts | New exoeval RPC end-to-end tests for query-builder composition and @expose gating. |
| src/rpc-capnweb.test.ts | New capnweb RPC end-to-end tests mirroring the exoeval scenarios. |
| src/capnweb/shim.ts | New capnweb shim to wrap/unwrap @expose objects/functions across RPC. |
| src/capnweb/shim.test.ts | Unit + end-to-end tests validating shim behavior and RPC closure replay. |
| src/capnweb/harness.ts | Test-only in-memory capnweb wiring for client/server session pairs. |
| src/builder/query.ts | Preserve toolFieldsSymbol marker during hydrateRows instance construction. |
| package.json | Adds local capnweb dependency via file:packages/capnweb. |
| .gitmodules | Adds packages/capnweb as a git submodule. |
| .gitignore | Stops ignoring all packages/, only ignores packages/exoagent/. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+74
to
+102
| export const toRpc = (value: unknown): unknown => { | ||
| if (value === null || (typeof value !== "object" && typeof value !== "function")) { | ||
| return value; | ||
| } | ||
| if (typeof value === "function") { | ||
| return wrapOutgoingFunction(value as (...args: unknown[]) => unknown); | ||
| } | ||
| if (proxyTargets.has(value)) { | ||
| // Already a wrapper (e.g. a value that round-tripped through fromRpc and back). | ||
| return value; | ||
| } | ||
| if (value instanceof RpcTarget) { | ||
| // Already a native capnweb capability (stub, promise, or hand-written RpcTarget). | ||
| return value; | ||
| } | ||
| if (isThenable(value)) { | ||
| return value.then(toRpc); | ||
| } | ||
| if (Array.isArray(value)) { | ||
| return value.map(toRpc); | ||
| } | ||
| if (isPlainObject(value)) { | ||
| return Object.fromEntries(Object.entries(value).map(([k, v]) => [k, toRpc(v)])); | ||
| } | ||
| if (isRpcSerializableBuiltin(value)) { | ||
| return value; | ||
| } | ||
| return wrapInstance(value); | ||
| }; |
Comment on lines
89
to
92
| "dependencies": { | ||
| "camelcase": "^9.0.0" | ||
| "camelcase": "^9.0.0", | ||
| "capnweb": "file:packages/capnweb" | ||
| } |
Comment on lines
+1
to
+3
| [submodule "packages/capnweb"] | ||
| path = packages/capnweb | ||
| url = https://github.com/ryanrasti/capnweb.git |
The hydration fix poked toolFieldsSymbol directly from the builder; the registry format is tool.ts's business, so it exports the one-call helper instead and query.ts says what it means. Also drops the formatting-only churn the cherry-pick brought into query.ts — its diff vs main is now just the import + the marker copy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
capnweb's own types can't describe the shimmed world: RpcStub's RpcCompatible constraint rejects the raw classes the shim adapts, and map()'s MapCallbackValue/Result<Array<V>> mangling describes plain capnweb — not record-replay against the @expose shim, where closures see raw objects and call synchronously. shim.ts now owns the client-side types: ShimStub<T> (members as async methods/properties; capability returns Disposable, distributing into arrays) and doRpc(stub, (api: T) => R) — the typed record-replay entry point whose callback sees the UNMANGLED T. The harness is generic (CapnwebHarness<T> carries ShimStub<T>), and both test suites drop their (api: any)/(row: any) annotations — callbacks and result rows are now fully inferred. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rospection API isPlainObject had three identical definitions (util, exoeval/expr, capnweb/shim) and isThenable two (shim + an inline check in tool.ts) — one exported copy of each in util.ts now serves all of them. rpcShimTargetSymbol answered a Proxy get nothing ever performed (the WeakMap, not the symbol, backs unwrapping), and rpcShimTarget() duplicated what fromRpc() already does for wrappers — both deleted; the shim's API is toRpc/fromRpc/doRpc/ShimStub. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.