Skip to content

Fix intermittent Lambda 409 when adding Function URL permissions#6933

Open
AppleDannyClegg wants to merge 1 commit into
anomalyco:devfrom
AppleDannyClegg:fix/lambda-url-permission-race
Open

Fix intermittent Lambda 409 when adding Function URL permissions#6933
AppleDannyClegg wants to merge 1 commit into
anomalyco:devfrom
AppleDannyClegg:fix/lambda-url-permission-race

Conversation

@AppleDannyClegg

Copy link
Copy Markdown

Fix intermittent Lambda 409 when adding Function URL permissions

Problem

Deploying an SSR site (Nextjs, Astro, SolidStart, …) intermittently fails with:

operation error Lambda: AddPermission, https response error StatusCode: 409,
ResourceConflictException: The function could not be updated due to a concurrent
update operation.

It's non-deterministic and hits fresh/ephemeral stacks (e.g. per-PR preview
environments) far more often than re-deploys. Retrying the deploy succeeds.

Cause

AWS Lambda serialises mutating control-plane operations per function: after any
update the function is briefly LastUpdateStatus: InProgress, and a second
mutating call in that window is rejected with a 409.

During a deploy two resources mutate the same function and, with no ordering
between them, Pulumi runs them concurrently:

  • aws.lambda.FunctionUrl — created in Function.createUrl()
    (CreateFunctionUrlConfig), and
  • aws.lambda.Permission with action: "lambda:InvokeFunctionUrl".

For the OAC path the permission is created in SsrSite and depends only on the
function — not on the function URL. It can't depend on the URL because the
FunctionUrl resource isn't exposed on Function.nodes (only role,
function, logGroup, eventInvokeConfig). So AddPermission races
CreateFunctionUrlConfig and lands mid-update. Fresh stacks hit it more often
because the function and its URL are created together in a single run.

Fix

Serialise the permission after the function URL:

  • expose the FunctionUrl resource on Function.nodes.url, and
  • add dependsOn on it to every lambda.Permission that grants URL access —
    both the ones created inside Function.createUrl() (public + router modes)
    and the OAC/CloudFront ones created in SsrSite (server functions and the
    image optimizer).

dependsOn only changes the Pulumi dependency graph, so existing permission
resources are updated in place — no replacement, no downtime.

Testing

  • tsc --noEmit passes.
  • platform test suite: all component tests pass (the only failures are a
    pre-existing Node-version incompatibility in unrelated suites — alb,
    bucket, service-alb — that error on @types/node resolution /
    ERR_TRACE_EVENTS_UNAVAILABLE, unaffected by this change).
  • Recommended before merge: a deploy of an SSR site in oac /
    oac-with-edge-signing and none protection modes (and a multi-region
    deploy) to confirm the 409 no longer occurs and that no permission resources
    are replaced on an in-place update.

Notes / possible follow-ups

The same class of concurrent-mutation race can in principle affect other
per-function operations that run alongside URL/permission creation (e.g.
provisioned-concurrency config, or an environment update). Those are out of
scope here; happy to look at them separately if maintainers think it's worth it.

Deploying an SSR site (Nextjs/Astro/etc.) intermittently fails with:

    operation error Lambda: AddPermission, StatusCode: 409,
    ResourceConflictException: The function could not be updated due to a
    concurrent update operation.

Lambda serialises mutating control-plane operations per function. During a
deploy, two resources mutate the same function but have no ordering between
them, so Pulumi runs them concurrently:

  - the `aws.lambda.FunctionUrl` created in `Function.createUrl()`
    (CreateFunctionUrlConfig), and
  - the `aws.lambda.Permission` granting `lambda:InvokeFunctionUrl`.

For the OAC path the permission is created in `SsrSite` and depends only on
the function, not on the function URL — and the URL resource is not reachable
because it is not exposed on `Function.nodes`. The permission's AddPermission
call therefore lands while CreateFunctionUrlConfig still has the function in
`LastUpdateStatus: InProgress`, yielding the 409. Fresh stacks hit it far more
often because the function and its URL are created together in one run.

Serialise the permission after the function URL:

  - expose the `FunctionUrl` resource on `Function.nodes.url`, and
  - add `dependsOn` on it to every `lambda.Permission` that grants URL access,
    both the ones created inside `Function.createUrl()` and the OAC/CloudFront
    ones created in `SsrSite`.

`dependsOn` is metadata-only, so existing permissions are not replaced.
@AppleDannyClegg AppleDannyClegg force-pushed the fix/lambda-url-permission-race branch from f2912b6 to ce8177c Compare July 9, 2026 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant