Cross-account SMUS deployment fails: ListDomainUnitsForParent AccessDeniedException for cfn-exec-role user profile stuck in ASSIGNED status

[cantoraa]

Bug Description

When deploying the smus_data_mesh starter kit in a multi-account setup, the cross-account CloudFormation stacks (mesh-lab-ibk-dev-ent-data-smus-dom-<target-account-id>) fail with an AccessDeniedException on the Custom::DomainConfig resource:

Received response status [FAILED] from custom resource.
Message returned: Error: An error occurred (AccessDeniedException) when calling the
ListDomainUnitsForParent operation: User is not permitted to perform operation: ListDomainUnitsForParent

The main domain stack (smus-dom) deploys successfully in the governance account, but both cross-account stacks (team1 and team2) fail identically.

Root Cause Analysis

The issue is a chicken-and-egg problem with DataZone V2 user profile activation in cross-account deployments:

1. MDAA creates a CfnUserProfile for each target account's cdk-hnb659fds-cfn-exec-role-<account>-<region> in the main domain stack. These profiles are created with status: ASSIGNED by DataZone (regardless of what CloudFormation specifies).

2. DataZone V2 restricts operations (including ListDomainUnitsForParent) for user profiles in ASSIGNED status. Only ACTIVATED profiles can perform domain operations.

3. The DomainConfig custom resource Lambda (domainconfig-handler) in the governance account calls list_domain_units_for_parent (line 69 in src/lambda/domain_config/domain_config.py). Despite this Lambda having an ACTIVATED profile, DataZone's authorization policy evaluation appears to check the context of the cross-account stack deployment, which maps back to the cfn-exec-role's ASSIGNED profile.

4. The status cannot be changed via API: UpdateUserProfile from ASSIGNED → ACTIVATED returns ValidationException: Error while updating user profile due to invalid status update. This is a DataZone service limitation — profiles can only be activated through portal login or initial service interaction by the principal itself.

5. The cfn-exec-role cannot "self-activate": Since this role only executes within CloudFormation, there's no way to make it interact with the DataZone domain before the cross-account stack deployment.

Environment

• MDAA Version: 1.6.0 (via npx @aws-mdaa/cli)
• Starter Kit: smus_data_mesh
• Region: us-east-1
• Account Setup: 3 accounts in the same AWS Organization
  • Governance/Enterprise (management account): XXXXXXX2477
  • Team1/Producer: XXXXXXX8556
  • Team2/Consumer: XXXXXXX7037
• AWS Organizations: RAM sharing enabled (ram.amazonaws.com and sso.amazonaws.com service access)
• IAM Identity Center: Enabled in governance account, us-east-1
• Authentication: SSO credentials (as required by SMUS documentation)
• CDK Bootstrap: Multi-account with --trust from governance to target accounts
• Docker: Running (required for Lambda asset builds)

Steps to Reproduce

1. Clone MDAA v1.6.0: git clone https://github.com/aws/modern-data-architecture-accelerator.git
2. Copy starter_kits/smus_data_mesh to a working directory
3. Configure mdaa.yaml with 3 accounts in the same org, IAM Identity Center enabled, SSO users/groups created, VPCs with private subnets
4. Set region: us-east-1 in mdaa.yaml
5. CDK bootstrap all accounts with cross-account trust
6. Authenticate with SSO credentials
7. Run: AWS_REGION=us-east-1 npx @aws-mdaa/cli deploy -c mdaa.yaml

Result:

• ✅ ent-com (bootstrap, glue-cat, roles, lf-set) — all 3 accounts
• ✅ team1-com (bootstrap, glue-cat, roles, lf-set)
• ✅ team2-com (bootstrap, glue-cat, roles, lf-set)
• ✅ ent-data/caef-bootstrap
• ✅ ent-data/smus-dom (main domain stack — domain created successfully)
• ❌ ent-data/smus-dom-XXXXXXX8556 (cross-account stack for team1)
• ❌ ent-data/smus-dom-XXXXXXX7037 (cross-account stack for team2)

Workarounds Attempted (All Failed)

Approach	Result
Re-run deploy after waiting for propagation (5+ minutes)	Same error — not a timing issue
UpdateUserProfile via API (ASSIGNED → ACTIVATED)	ValidationException: invalid status update
UpdateUserProfile as domain custom-resource role (ACTIVATED profile, domain owner)	Same ValidationException — service-level restriction
Import resource into separate CFN stack and update status	Cannot import — resource already belongs to MDAA stack
Create new user profile with ACTIVATED status	AlreadyExists — profile was auto-created as ASSIGNED
Call GetDomain from producer account to trigger activation	Profile remains ASSIGNED — only the specific principal's interaction activates its own profile
Assume cfn-exec-role and call DataZone	Cannot assume cfn-exec-role from SSO credentials (trust only allows CloudFormation service)

Expected Behavior

The cross-account stacks should deploy successfully. The DomainConfig custom resource Lambda should be able to call ListDomainUnitsForParent in the context of cross-account deployments.

Relevant Code Paths

• User profile creation: packages/constructs/L3/governance/datazone-l3-construct/lib/private/common-domain-helper.ts (lines 743-757) — Creates CfnUserProfile for associated account CDK roles
• DomainConfig Lambda: packages/constructs/L2/datazone-constructs/src/lambda/domain_config/domain_config.py (line 69) — Calls list_domain_units_for_parent
• Cross-account stack creation: packages/constructs/L3/governance/datazone-l3-construct/lib/private/sagemaker-domain-helper.ts (lines 744+) — Creates cross-account resources referencing DomainConfig custom resource

Suggested Fixes

1. Have the domainconfig-handler Lambda assume a role that IS activated before making DataZone API calls in cross-account context (e.g., the domain1-custom-resource role which is ACTIVATED)

2. Create a pre-activation step in the main domain stack that forces activation of cross-account user profiles before the cross-account stacks deploy (e.g., a custom resource that uses the DataZone admin API to activate profiles)

3. Use the domain execution role for the ListDomainUnitsForParent call instead of relying on the Lambda's own profile authorization

4. Add an authorization policy for the domainconfig-handler that explicitly grants ListDomainUnitsForParent on all domain units, bypassing the user-profile-based authorization check

Additional Notes

• The AWS_REGION environment variable must be set to the deployment region. If not set correctly, MDAA's bootstrap module may synthesize stacks targeting us-west-2 even when region: us-east-1 is specified in mdaa.yaml. This is because MdaaCdkApp.deployRegion reads from CDK_DEFAULT_REGION (line 189 in packages/apps/core/app/lib/app.ts) but the CLI only exports CDK_DEPLOY_REGION and AWS_DEFAULT_REGION (line 977 in packages/cli/lib/mdaa-cli.ts), not CDK_DEFAULT_REGION. Setting AWS_REGION=us-east-1 in the shell works around this.

• The associated accounts show as "Associated" status in the DataZone console, confirming the org-level association is working correctly.

• The SSO user is correctly ACTIVATED in the domain and can access the SMUS portal.

[suddash24]

Thank you for reporting this issue and for providing detailed reproduction steps. We appreciate your engagement with the project.

We wanted to let you know that this issue has been identified and is already being tracked internally. A fix has been developed and will be included in the upcoming v1.7.0 release.

In the meantime, if you need an immediate workaround for your multi-account deployment, please don't hesitate to reach out and we can provide guidance.

Thank you for your patience and for helping us improve MDAA. We'll update this issue once v1.7.0 is released.