[BUG] SMUS Tooling KMS key missing Bedrock authorization

[marcinc]

Summary

Tooling KMS key missing Bedrock authorization:

• missing bedrock.amazonaws.com principal
• missing provisioning role kms:CreateGrant, and
• incomplete ToolingKmsCreateGrant statement

When creating a SageMaker Unified Studio project with a profile that enables AmazonBedrockGuardrail (or any other blueprint that creates a AWS::Bedrock::* resource encrypted with a customer KMS key), the environment deployment fails immediately with AccessDenied. No Bedrock resource is created.

The failure stems from three gaps in the MDAA-generated tooling KMS key setup. AWS KMS enforces each independently, so omitting any one is sufficient to deny the operation -- in this case, all three were present simultaneously.

Affected version

@aws-mdaa/datazone-l3-construct v1.5.0

Component

packages/constructs/L3/governance/datazone-l3-construct/lib/private/sagemaker-domain-helper.ts

Error message

Project environment AmazonBedrockGuardrail (<env-id>) failed to deploy in the account <account> and region <region> due to the failure:
  Access denied for operation 'AWS::Bedrock::Guardrail'.
  HandlerErrorCode: AccessDenied
    CloudFormation stack ID: DataZone-Env-<env-id>

KMS authorization for CMK-encrypted Bedrock resources

The KMS authorization sequence that runs when CloudFormation creates a CMK-encrypted Bedrock resource:

1. CloudFormation calls kms:CreateGrant on behalf of the provisioning role. This creates a service grant that allows bedrock.amazonaws.com to use the key for subsequent encrypt/decrypt operations during the resource's lifetime. kms:CreateGrant is evaluated against both the caller's IAM policy and the key resource policy - both must allow it. When the resource is later deleted, CloudFormation calls kms:RetireGrant to clean up the grant; the provisioning role must have this permission as well.

2. Bedrock calls kms:GenerateDataKey / kms:Decrypt as the bedrock.amazonaws.com service principal, using the grant from step 1 to perform the actual encryption when creating the resource (Guardrail, KnowledgeBase, etc.).

MDAA is missing the wiring for all three steps:

• The tooling KMS key resource policy did not list bedrock.amazonaws.com as an allowed principal → step 2 fails entirely regardless of the provisioning role's IAM policy.
• The provisioning role's IAM policy did not include kms:CreateGrant → step 1 denied at IAM evaluation stage.
• The ToolingKmsCreateGrant statement in domainKmsAdminPolicy was also incomplete: it lacked kms:RetireGrant (preventing grant cleanup on resource deletion), used StringLike instead of StringEquals for kms:CallerAccount, and was missing the kms:GrantIsForAWSResource condition (which restricts the grant to AWS service invocations only).

Either gap alone causes the deployment to fail with the same AccessDenied surface error. All three must be closed for Bedrock blueprint environments to deploy and clean up successfully.

Confirmed via CloudTrail

CloudTrail showed two sequential failures for the same project environment creation:

2026-04-24T10:23:53Z  kms:CreateGrant  AccessDenied
  principal: test-dev-governance-sagemaker-main-provisioning-role/
             AmazonDataZoneEnvironmentDeployer-<account>
  key: <tooling-key-arn>

2026-04-24T10:23:53Z  bedrock:CreateGuardrail  AccessDenied
  errorMessage: You don't have sufficient permissions for this KMS key.
  principal: test-dev-governance-sagemaker-main-provisioning-role/
             AmazonDataZoneEnvironmentDeployer-<account>

Note: When diagnosing KMS-related AccessDenied errors in SMUS environments, always check CloudTrail
directly -- it seems as IAM simulation does not accurately model key resource policy evaluation or service principal grants.

Proposed Fix

Three code changes in sagemaker-domain-helper.ts, each closing one gap:

1. Add bedrock.amazonaws.com to the tooling KMS key policy

In createToolingResources(), add a BedrockEncryption statement alongside the existing
CloudWatch Logs and account root statements:

const bedrockStatement = new PolicyStatement({
  sid: 'BedrockEncryption',
  effect: Effect.ALLOW,
  actions: [...DECRYPT_ACTIONS, ...ENCRYPT_ACTIONS, 'kms:DescribeKey'],
  principals: [new ServicePrincipal('bedrock.amazonaws.com')],
  resources: ['*'],
  conditions: {
    StringEquals: {
      'aws:SourceAccount': account,
    },
  },
});
kmsKey.addToResourcePolicy(bedrockStatement);

kms:DescribeKey is included so that the Bedrock service principal can inspect the key metadata before attempting encrypt/decrypt operations. The aws:SourceAccount condition scopes the grant to the domain's own account, preventing cross-account misuse.

2. Harden the ToolingKmsCreateGrant statement

In createToolingResources(), update the ToolingKmsCreateGrant statement inside domainKmsAdminPolicy to add kms:RetireGrant, correct the condition operator, and add the kms:GrantIsForAWSResource guard:

  domainKmsAdminPolicy.addStatements(
    new PolicyStatement({
      sid: 'ToolingKmsCreateGrant',
      effect: Effect.ALLOW,
      resources: [kmsKey.keyArn],
-     actions: ['kms:CreateGrant'],
+     actions: ['kms:CreateGrant', 'kms:RetireGrant'],
      conditions: {
-       StringLike: {
+       StringEquals: {
          'kms:CallerAccount': account,
        },
+       Bool: {
+         'kms:GrantIsForAWSResource': 'true',
+       },
      },
    }),
  );

kms:RetireGrant allows the provisioning role to retire service grants when Bedrock resources are deleted. kms:GrantIsForAWSResource: true restricts grant creation to AWS service invocations, preventing the provisioning role from creating arbitrary grants on the tooling key.

3. Attach kms-admin policy to the primary provisioning role

In createPrimaryDomainResources(), add the missing policy attachment alongside the existing ones:

  domainDefaultBlueprintProvisioningRole.addManagedPolicy(policies.domainKmsUsagePolicy);
+ domainDefaultBlueprintProvisioningRole.addManagedPolicy(policies.domainKmsAdminPolicy);
  domainDefaultBlueprintProvisioningRole.addManagedPolicy(policies.domainBucketUsagePolicy);

The domainKmsAdminPolicy (now with the hardened ToolingKmsCreateGrant statement from change 2) gives the primary account provisioning role kms:CreateGrant and kms:RetireGrant on the tooling key, matching the permissions already present on the cross-account provisioning role.

Workaround

The two most impactful gaps (missing Bedrock key policy statement and missing policy attachment) can be closed manually. Both steps are required. The ToolingKmsCreateGrant statement improvements (change 2) require a full MDAA redeploy to take effect on existing deployments.

Step 1 — add bedrock.amazonaws.com to the tooling KMS key policy:

# Fetch the current policy
aws kms get-key-policy \
  --region <region> \
  --key-id <tooling-key-id> \
  --policy-name default \
  --query Policy --output text > /tmp/key-policy.json

# Add the following statement to the Statement array in /tmp/key-policy.json:
# {
#   "Sid": "BedrockEncryption",
#   "Effect": "Allow",
#   "Principal": { "Service": "bedrock.amazonaws.com" },
#   "Action": ["kms:Decrypt","kms:DescribeKey","kms:Encrypt","kms:ReEncryptFrom","kms:ReEncryptTo",
#               "kms:GenerateDataKey","kms:GenerateDataKeyWithoutPlaintext",
#               "kms:GenerateDataKeyPair","kms:GenerateDataKeyPairWithoutPlaintext"],
#   "Resource": "*",
#   "Condition": { "StringEquals": { "aws:SourceAccount": "<account-id>" } }
# }

aws kms put-key-policy \
  --region <region> \
  --key-id <tooling-key-id> \
  --policy-name default \
  --policy file:///tmp/key-policy.json

Step 2 — attach the kms-admin policy to the provisioning role:

aws iam attach-role-policy \
  --role-name <domain-name>-provisioning-role \
  --policy-arn arn:aws:iam::<account>:policy/<domain-name>-domain-kms-admin

The domain-kms-admin-<domain> policy already exists and contains ToolingKmsCreateGrant -- it just needs to be attached to the provisioning role. Note that the existing deployed policy will have the old statement (without kms:RetireGrant and kms:GrantIsForAWSResource); these improvements only take effect after a full MDAA redeploy.

After both steps, delete the failed project environment and recreate it. CloudFormation does not retry a failed environment stack automatically.

Impact

Any SageMaker Unified Studio project environment that creates a AWS::Bedrock::* resource encrypted with the domain's tooling KMS key will fail. Confirmed for AmazonBedrockGuardrail blueprint. Expected to also affect AmazonBedrockKnowledgeBase, AmazonBedrockAgent, AmazonBedrockFlow, AmazonBedrockPrompt, and any other blueprint that provisions CMK-encrypted Bedrock resources. Environments that do not create Bedrock resources (e.g. Tooling, LakehouseDatabase) are unaffected.

[suddash24]

Hi @marcinc,

Thank you for the detailed bug report. The root cause analysis, CloudTrail evidence, and proposed fix are all very thorough and well-documented.

We're actively investigating this issue and evaluating the associated pull request (#47). Your breakdown of the three independent gaps in the tooling KMS key setup (missing bedrock.amazonaws.com principal, missing kms:CreateGrant on the provisioning role, and the incomplete ToolingKmsCreateGrant statement) is clear and aligns with what we're seeing on our end.

We'll review the proposed changes and follow up here once we have an update on the PR status and timeline for including this in an upcoming release.

In the meantime, thank you for including the manual workaround steps. That should help others who encounter this before the fix lands.

We'll keep you posted on progress.

[suddash24]

Hello @marcinc,

We have attempted to replicate the issue you reported; however, we were unable to reproduce it in our environment. To assist us in investigating further, could you please share your MDAA configuration along with the detailed steps to reproduce the problem?

Based on our initial assessment, it is possible that certain permissions may have been modified since your deployment attempt. For reference, we enabled the AmazonBedrockGuardrail blueprint and used it to create an environment via a profile, and the deployment completed without issue.

Please let us know if there is any additional context that might help us identify the root cause.