feat: decode binary/protobuf payloads with messageType (LOE)#3356
Draft
rossnelson wants to merge 31 commits into
Draft
feat: decode binary/protobuf payloads with messageType (LOE)#3356rossnelson wants to merge 31 commits into
rossnelson wants to merge 31 commits into
Conversation
…load component Consolidates two divergent payload display components into a single `<Payload />` component at `src/lib/components/payload.svelte`. Previously, `payload-decoder.svelte` (Svelte 5 runes, required a `children` snippet, always decoded full attribute trees) and `metadata-decoder.svelte` (Svelte 4 options API with slot syntax, decoded single payloads to truncated summary strings) served different display purposes but used inconsistent APIs and decoding paths across 65+ call sites. The new component unifies both under a single `mode` prop: - `code-block` (default): renders a CodeBlock with the decoded value, replacing all PayloadDecoder + CodeBlock snippet patterns - `summary`: truncates to 120 chars and renders a Badge, replacing all MetadataDecoder usages - `inline-truncated`: renders the compact .payload pre block used in event detail rows - `children` snippet: escape hatch for callers that need custom rendering (schedule input, timeline SVG text elements) Both old components used a single decoding path internally (cloneAllPotentialPayloadsWithCodec -> decodePayloadAttributes -> stringifyWithBigInt). MetadataDecoder previously used a separate decodeSingleReadablePayloadWithCodec path; the new component uses the same unified path for consistency. The component is written in Svelte 5 runes throughout and imports from $app/state instead of $app/stores, matching the newer direction of the codebase. Migrated files: - src/lib/components/event/event-card.svelte - src/lib/components/event/event-details-row.svelte (+ moved .payload CSS) - src/lib/components/event/event-metadata-expanded.svelte - src/lib/components/event/event-summary-row.svelte - src/lib/components/lines-and-dots/svg/group-details-text.svelte - src/lib/components/lines-and-dots/svg/timeline-graph-row.svelte - src/lib/components/schedule/schedule-form/schedule-input-payload.svelte - src/lib/components/standalone-activities/activity-input-and-outcome.svelte - src/lib/components/workflow/client-actions/update-confirmation-modal.svelte - src/lib/components/workflow/input-and-results-payload.svelte - src/lib/components/workflow/metadata/metadata-events.svelte - src/lib/components/workflow/pending-activity/pending-activity-card.svelte - src/lib/components/workflow/workflow-json-navigator.svelte - src/lib/pages/standalone-activity-details.svelte - src/lib/pages/standalone-activity-search-attributes.svelte - src/lib/pages/workflow-memo.svelte - src/lib/pages/workflow-metadata.svelte - src/lib/pages/workflow-search-attributes.svelte All 1730 tests pass.
Moves payload.svelte into src/lib/components/payload/ and adds a USAGE.md report documenting all 31 call sites grouped by feature area. Updates all 17 import paths accordingly.
… components Replace the 14-prop multi-mode payload.svelte with four single-purpose components: PayloadCodeBlock, PayloadSummary, PayloadDecoder, and PayloadInline. Each component carries only the props relevant to its render mode, eliminating dead props at call sites. Also updates the decode paths to use the renamed decode-payload.ts API (decodeEventAttributes, parsePayloadAttributes) following the #3302 cleanup.
Replace decodeEventAttributes with decodeUserMetadata — the correct decode path for a single raw Payload (Phase 2 codec only, no attribute tree walking). Also add onDecode callback prop, called after a successful decode, consistent with PayloadCodeBlock.
…lue utility Remove 3-way duplication of getInitialValue/onMount decode logic from payload-code-block, payload-decoder, and payload-inline by extracting getInitialPayloadValue and decodePayloadValue into src/lib/utilities/decode-payload-value.ts.
Components now re-decode when value or fieldName props change after mount, fixing the reactivity gap that caused stale decoded content when parent components updated their payload bindings.
Add a Phase 1.5 dispatch in parseRawPayloadToJSON: when a Payload's
metadata says encoding=binary/protobuf and carries a messageType, look
the type up by full name in @temporalio/proto's static namespace and
return the typed JSON. After the outer decode, walk the result tree
and recursively decode any nested {metadata, data} Payload nodes
through the same pipeline so user-side payloads (json/plain, json/
protobuf, json/external-storage-reference, etc.) come out fully
parsed. Unknown messageTypes and decode errors fall through to the
existing path so behavior for non-binary/protobuf payloads is
unchanged.
This is the proposal for handling System Nexus operations like
SignalWithStartWorkflowExecutionRequest, where the gRPC envelope
arrives as binary protobuf and the inner Payloads carry user data.
Tests cover round-tripping a real SignalWithStart fixture and the
graceful fallback for an unknown messageType. 27/27 passing.
Caveat: protobufjs builds its decoders with Function() and so needs
'unsafe-eval' in the script-src CSP. That's relaxed in a separate
commit so reviewers can drop it before merge if we want a different
proto runtime (@bufbuild/protobuf has no eval).
@temporalio/proto uses protobufjs, which compiles per-message encoders/decoders via the Function() constructor. With the existing 'strict-dynamic' CSP and no 'unsafe-eval', binary/protobuf payload decode silently fails in the browser. Relax the dev/auto CSP to permit unsafe-eval so the new decode path works end-to-end during the LOE evaluation. If the team decides to ship browser-side proto decoding, the long-term option is to swap @temporalio/proto for @bufbuild/protobuf (which doesn't use eval) and drop this commit. Open question for review: do we keep this, or migrate the runtime?
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…-binary-protobuf Moves lookupTemporalProtoType, base64ToUint8Array, looksLikeRawPayload, recursivelyDecodeNestedPayloads, and the full decode path into a new decode-binary-protobuf utility. The recursive walker now accepts a recurse callback rather than calling parseRawPayloadToJSON directly, eliminating the circular dependency. Adds unit tests covering successful decode, silent null on unknown type, warn-on-throw, looksLikeRawPayload edge cases, and the callback injection path.
…quests
Converts payload-decoder.svelte from {#await} to AbortController + $effect
so stale in-flight decodes cannot overwrite a newer result. Rewrites
codeServerRequest in data-encoder.ts to accept an optional AbortSignal and
retry transient errors (network/5xx) up to 3 attempts with 500ms/1000ms
delays, without retrying 4xx. Deletes the now-unused decode-payload-value
module (zero importers confirmed).
Adds a nexus-operation-registry that recognises the 4 system-level Nexus operation types (StartWorkflow, SignalWorkflow, SignalWithStartWorkflow, QueryWorkflow) and returns a human-readable descriptor with the embedded input payloads. NexusOperationRenderer uses the registry to show the embedded operation directly — users see "Signal With Start Workflow: EchoWorkflow" and the decoded input, not the raw protobuf wrapper. input-and-results-payload routes Nexus payloads through the renderer automatically; all other content paths are unchanged. Adds 6 registry tests covering the happy path and null returns for unknown types.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
LOE prototype for browser-side decode of System Nexus payloads, where the gRPC envelope (
SignalWithStartWorkflowExecutionRequest, futureWaitForExternalWorkflow…,StandaloneActivity…, etc.) arrives in workflow history as a singlePayloadwithmetadata.encoding = binary/protobufandmetadata.messageTypenaming a fully-qualifiedtemporal.api.*type.Adds a Phase 1.5 dispatch in
parseRawPayloadToJSON:Every existing renderer (event card, JSON view, compact row, copy/export) inherits the typed view for free. Adding the next system Nexus operation costs zero code —
@temporalio/protoalready ships every workflowservice message.Demo
Repro harness lives in
temporalio/nexus-endpoint(sibling repo): pinnedtemporalserver +sdk-pythonspk/signal-with-starttest that produces a realSignalWithStartWorkflowExecutionRequestin workflow history.Before — what every renderer saw before this patch for
nexusOperationScheduledEventAttributes.input:{ "metadata": { "encoding": "binary/protobuf", "messageType": "temporal.api.workflowservice.v1.SignalWithStartWorkflowExecutionRequest" }, "data": "CgdkZWZhdWx0EhhzeXN0ZW0tbmV4… (1068 base64 chars)" }After — same render path, no other changes:
{ "namespace": "default", "workflowId": "system-nexus-workflow-id", "workflowType": { "name": "EchoWorkflow" }, "taskQueue": { "name": "969f0ffd-…" }, "input": { "payloads": [ { "driver_claim": { … }, "driver_name": "test-driver" } ] }, "signalName": "test-signal", "signalInput": { "payloads": [ { "driver_claim": { … }, "driver_name": "test-driver" } ] }, "memo": { "fields": { "memo-key": { "driver_claim": { … }, "driver_name": "test-driver" } } }, "userMetadata": { "summary": { … }, "details": { … } }, … }Note the inner Payloads (the user's signal args, memo, etc.) are also decoded — the recursive pass in this PR feeds nested
{metadata, data}nodes back throughparseRawPayloadToJSON, so the existing JSON-decode path handles them transparently.Visible UI surfaces (verified live)
Input/Resulttyped JSONOpen questions for review
script-srcto add'unsafe-eval', because protobufjs builds per-message decoders viaFunction(). Two paths:unsafe-evalin production)@bufbuild/protobuf(no eval, smaller, ESM-native; ~2 days)@temporalio/protois already a dep but only used as types today. Worth apnpm buildbefore/after to see the runtime delta. If meaningful, switch to dynamicawait import('@temporalio/proto')inside the decode helper../atoblandmine — the localatobdoes UTF-8 decoding viadecodeURIComponent, which corrupts raw protobuf bytes. The patch usesglobalThis.atobdirectly. Worth either documenting that on./atob.tsor extracting arawBase64ToUint8Arrayhelper for future binary work.What this isn't
binary/protobufpayloads. Other encodings unchanged.Test plan
pnpm test src/lib/utilities/decode-payload.test.ts -- --run— 27/27 passing (added 2)pnpm check— 0 errors, 84 pre-existing warningspnpm lint— 0 errors, 210 pre-existing warningsSignalWithStartWorkflowExecutionRequest+ nested external-storage payloads decoded in browser, screenshots in conversationpnpm buildsize comparison — TODO before promoting beyond LOENotion
System Nexus Endpoint — the page's outstanding "UI" question is whether the browser can decode without a proto library. Answer: yes, it already can; this PR shows how.