(2026.03 and earlier) IAM policies not appearing in "add policies" dropdown when account has more than 1000 policies

[junbo75]

Bug description

When listing available host policies for VDI projects, the Cluster Manager fails to retrieve all project policies. This is caused by two issues: (1) the invoke_aws_listing pagination helper passes response incorrectly when extracting the next page token, causing pagination to silently stop after the first page, and (2) the list_available_host_policies method uses iam:ListPolicies which only returns customer-managed policies and does not filter by the RES VDI resource tag. This results in missing policies in the project configuration UI.

Affected versions

2026.03 and earlier

Mitigation

Prerequisite (all versions): Before applying the patch, you must add the tag:GetResources IAM permission to the Cluster Manager IAM role. This permission is required for the fix to query the Resource Groups Tagging API.

1. Open the IAM Console and find the role named <environment-name>-cluster-manager-role (or the role attached to your Cluster Manager instance).
2. Add the following permission to the role's policy:

{
    "Effect": "Allow",
    "Action": "tag:GetResources",
    "Resource": "*"
}

For 2025.06 and 2025.09:

1. Download patch.py and fix-retrieve-all-project-policies.patch for version 2025.06, 2025.06.01, or 2025.09 by replacing <output-directory> with the directory to download the patch script and patch file, <res-version> with 2025.06, 2025.06.01, or 2025.09, and <environment-name> with the name of your RES environment in the command below:
   1. The patch applies to RES 2025.06, 2025.06.01, or 2025.09
   2. The patch script requires AWS CLI v2, Python 3.9.16, and Boto3.
   3. Configure the AWS CLI for the account / region where RES is deployed, and make sure that you have S3 permissions to write to the bucket created by RES.

OUTPUT_DIRECTORY=<output-directory>
ENVIRONMENT_NAME=<environment-name>
RES_VERSION=<res-version> # e.g. 2025.06, 2025.06.01, or 2025.09
mkdir -p ${*OUTPUT_DIRECTORY*}
curl https://research-engineering-studio-us-east-1.s3.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patch.py --output ${OUTPUT_DIRECTORY}/patch.py
curl https://research-engineering-studio-us-east-1.s3.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patches/fix-retrieve-all-project-policies.patch --output ${OUTPUT_DIRECTORY}/fix-retrieve-all-project-policies.patch

2. Navigate to the directory where the patch script and patch file are downloaded. Run the following patch command:

python3 ${OUTPUT_DIRECTORY}/patch.py --environment-name ${ENVIRONMENT_NAME} --res-version ${RES_VERSION} --module cluster-manager --patch ${OUTPUT_DIRECTORY}/fix-retrieve-all-project-policies.patch

3. Restart the Cluster Manager instance for your environment. You may also terminate the instance from the Amazon EC2 Management Console.

INSTANCE_ID=$(aws ec2 describe-instances \
            --filters \
            Name=tag:Name,Values=${ENVIRONMENT_NAME}-cluster-manager \
            Name=tag:res:EnvironmentName,Values=${ENVIRONMENT_NAME}\
            --query "Reservations[0].Instances[0].InstanceId" \
            --output text)

aws ec2 terminate-instances --instance-ids ${INSTANCE_ID}

4. Verify the new Cluster Manager instance status by checking the activity of the auto scaling group starting with the name <environment-name>-cluster-manager-asg. Wait until the new instance is launched successfully.
5. Navigate to the project configuration page → Click on Policies dropdown → confirm all expected policies appear.

For 2025.12 and 2026.03:

1. Create an S3 bucket with ACLs disabled.
2. Download patch_host.py and idea-cluster-manager-{res-version}-<HASH>.tar.gz by replacing <output-directory> with the directory to download the patch script, <environment-name> with the name of your RES environment, <bucket-name> with the name of a ACLs disabled S3 bucket under the account/region where RES is deployed, RES_VERSION with 2025.12, 2025.12.01, or 2026.03 in the command below, and <partition> with Classic or GovCloud:
   1. The patch applies to 2025.12, 2025.12.01, 2026.03
   2. The patch script requires AWS CLI v2, Python 3.9.16 or above, and Boto3.
   3. Configure the AWS CLI for the account / region where RES is deployed, and make sure that you have S3 permissions to write to the bucket provided through <bucket-name>.

OUTPUT_DIRECTORY=<output-directory>
ENVIRONMENT_NAME=<environment-name>
RES_VERSION=<RES_VERSION>
BUCKET_NAME=<bucket-name>
PARTITION=<partition>

mkdir -p ${OUTPUT_DIRECTORY}
HASH=$(case $RES_VERSION in "2026.03") echo 6ecb935e;; "2025.12.01") echo 07394829;; "2025.12") echo 9b9412dd;; esac)
curl https://research-engineering-studio-us-east-1.s3.us-east-1.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patch_host.py --output ${OUTPUT_DIRECTORY}/patch_host.py
curl https://research-engineering-studio-us-east-1.s3.us-east-1.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patches/idea-cluster-manager-${RES_VERSION}-${HASH}.tar.gz --output ${OUTPUT_DIRECTORY}/idea-cluster-manager-${RES_VERSION}-${HASH}.tar.gz

3. Run the following patch command:

python3 ${OUTPUT_DIRECTORY}/patch_host.py --environment-name ${ENVIRONMENT_NAME} --module cluster-manager --zip-file ${OUTPUT_DIRECTORY}/idea-cluster-manager-${RES_VERSION}-${HASH}.tar.gz --s3-bucket ${BUCKET_NAME} --partition ${PARTITION}

4. Restart the Cluster Manager instance for your environment. You may also terminate the instance from the Amazon EC2 Management Console.

INSTANCE_ID=$(aws ec2 describe-instances \
            --filters \
            Name=tag:Name,Values=${ENVIRONMENT_NAME}-cluster-manager \
            Name=tag:res:EnvironmentName,Values=${ENVIRONMENT_NAME}\
            --query "Reservations[0].Instances[0].InstanceId" \
            --output text)

aws ec2 terminate-instances --instance-ids ${INSTANCE_ID}

5. Verify the new Cluster Manager instance status by checking the activity of the auto scaling group starting with the name <environment-name>-cluster-manager-asg. Wait until the new instance is launched successfully.
6. Navigate to the project configuration page → Click on Policies dropdown → confirm all expected policies appear.

Rollback

Re-running patch.py with the same arguments will unpatch the module, restoring the original code.