fix(spec): reject undeliverable IOA webhooks - #386
Conversation
|
RED evidence · commit 2549a37 The regressions fail because both IOA webhook forms currently parse successfully and materialize $ env CARGO_TARGET_DIR=/tmp/temper-arn227-target cargo test -p temper-spec rejected_until_delivery_is_durable -- --nocapture
running 2 tests
... webhook_action_trigger_is_rejected_until_delivery_is_durable ... FAILED
... legacy_webhook_integration_is_rejected_until_delivery_is_durable ... FAILED
test result: FAILED. 0 passed; 2 failed; 268 filtered out$ env CARGO_TARGET_DIR=/tmp/temper-arn227-target cargo test -p temper-spec defaulted_legacy_webhook_integration_is_also_rejected -- --nocapture
running 1 test
... defaulted_legacy_webhook_integration_is_also_rejected ... FAILED
test result: FAILED. 0 passed; 1 failed; 269 filtered out |
|
Fresh GPT-5.6 review of the GitHub PR metadata and complete diff. No actionable findings. Evidence:
Verdict: PASS |
|
@greptile review |
|
Fresh follow-up review at HEAD 9c54061. No actionable findings. Evidence:
Verdict: PASS |
|
Live local E2E evidence · final HEAD The pre-fix RED behavior is captured in the isolated RED comment: both outbound webhook forms parsed successfully and materialized an undeliverable The final build was started with an app containing a real action-owned webhook target ( $ env TURSO_URL=file:/tmp/temper-arn227-e2e.db /tmp/temper-arn227-target/debug/temper serve --port 39126 --no-observe --specs-dir .progress/arn-227-e2e --tenant arn227-e2e
OTEL export configured: endpoint=https://logfire-us.pydantic.dev source=LOGFIRE_TOKEN logfire_auth=true
Storage: turso (file:/tmp/temper-arn227-e2e.db)
Loading app: arn227-e2e from .progress/arn-227-e2e
Error: failed to parse IOA spec for Order
Caused by:
validation error: trigger 'notify_fulfillment' on action 'ConfirmOrder': outbound IOA webhooks are unsupported until durable delivery is available
[exit code: 1]The process exits during app registration, before the server binds and before any source action can commit: $ curl --silent --show-error --max-time 2 http://127.0.0.1:39126/health
curl: (7) Failed to connect to 127.0.0.1 port 39126 after 0 ms: Couldn't connect to server
[exit code: 7]Result: the previously accepted-and-dropped webhook can no longer install as healthy; rejection is stable, explicit, and occurs before runtime delivery is possible. |
Summary
webhooks.toml, and the directly configured platform engine APIRegression strategy
mainand open PRs2549a376contains only failing behavioral/durability regressions for action-trigger, explicit legacy, and defaulted legacy webhook formsee14b001is the reviewed GREEN boundary and compatibility cleanupValidation status
temper-spec: 269/269 passedtemper-actor-runtimetest target, where all five tests timed out creating Postgres containers because Docker was unresponsive (CreateContainer(RequestTimeoutError)); final Docker-dependent rerun remains required before shippable statusLinear: https://linear.app/arni-build/issue/ARN-227/bug-ioa-webhook-integrations-are-accepted-by-specs-but-never-execute
Greptile Summary
This PR eliminates a silent false promise: IOA webhook declarations (
[[action.triggers]] kind = \"webhook\"and legacy[[integration]] type = \"webhook\") were accepted, verified, and installed but never dispatched — the production runtime only executeswasmandadapterintegration types. ADR-0171 formalizes the rejection and defines the six-gate acceptance contract for a future durable delivery runtime.validatenow rejects both declaration forms before synthesis, verification, JIT construction, or registration; the omitted-typedefault (which historically resolved to\"webhook\") is also caught, andexpand_external_action_triggersadds a redundant defensive check as a second layer.heartbeat_run.ioa.toml,subscription.ioa.toml, and thetemper-platformintegration test fixtures remove or replace all webhook declarations;AGENT_GUIDE.md,PAPER.md,SKILL.md,protocol.rs, and both superseded ADRs are updated consistently.webhooks.tomltrajectory subscriptions, the directly configuredtemper-platformintegration engine (with its retry/dead-letter API), entity triggers, WASM triggers, and adapter triggers are all unchanged.Confidence Score: 5/5
Safe to merge: removes a false promise (silent accept + no dispatch) with no working capability dropped, and all three affected declaration paths are covered by targeted regression tests.
The change is a pure narrowing of what the parser accepts. Every previously working runtime path (entity triggers, WASM, adapters, operator webhooks, and the standalone platform integration engine) is explicitly preserved and tested. The two new rejection sites fire before any state-mutating step, and the six test cases covering explicit webhook type, omitted type, inline action trigger, field-presence ordering, and expansion prevention leave no gap in the enforcement contract.
No files require special attention. The defensive webhook check inside expand_external_action_triggers is unreachable (validate fires first), but it is harmless and serves as a guard against future refactoring that might reorder the two calls.
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart TD A[parse_toml_to_automaton] --> B[validate] B --> C{integration_type == webhook?} C -- Yes --> ERR1[AutomatonParseError OUTBOUND_IOA_WEBHOOK_UNSUPPORTED] C -- No --> D[validate_action_triggers] D --> E{trigger.kind == Webhook?} E -- Yes --> ERR2[AutomatonParseError OUTBOUND_IOA_WEBHOOK_UNSUPPORTED] E -- No --> F[wire_state_timeout_from_states] F --> G[expand_external_action_triggers] G --> H{trigger.kind == Webhook? defensive} H -- Yes --> ERR3[AutomatonParseError dead path validate fired first] H -- No --> I{kind == Wasm or Adapter?} I -- Yes --> J[Synthesize Integration and Trigger Effect] I -- No --> K[Skip Entity or other] J --> L[check_liveness_coverage] K --> L L --> M[Automaton accepted no webhook declarations possible]%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% flowchart TD A[parse_toml_to_automaton] --> B[validate] B --> C{integration_type == webhook?} C -- Yes --> ERR1[AutomatonParseError OUTBOUND_IOA_WEBHOOK_UNSUPPORTED] C -- No --> D[validate_action_triggers] D --> E{trigger.kind == Webhook?} E -- Yes --> ERR2[AutomatonParseError OUTBOUND_IOA_WEBHOOK_UNSUPPORTED] E -- No --> F[wire_state_timeout_from_states] F --> G[expand_external_action_triggers] G --> H{trigger.kind == Webhook? defensive} H -- Yes --> ERR3[AutomatonParseError dead path validate fired first] H -- No --> I{kind == Wasm or Adapter?} I -- Yes --> J[Synthesize Integration and Trigger Effect] I -- No --> K[Skip Entity or other] J --> L[check_liveness_coverage] K --> L L --> M[Automaton accepted no webhook declarations possible]Reviews (2): Last reviewed commit: "docs: accept ADR 0171" | Re-trigger Greptile