(2025.09 and earlier) Fail to mount EFS file systems on RHEL, Rocky and Ubuntu VDIs due to incompatible Rust version

[junbo75]

Bug description

RHEL, Rocky and Ubuntu VDIs fail to mount EFS systems during the bootstrap process, which can lead to users not being able to log in to their Ubuntu VDIs. This is caused by the incompatible Rust version.

During instance bootstrap, the amazon-efs-utils package fails to build due to an incompatibility between the Rust version provided by OS specifc package repositories and the newer Cargo.lock format required by efs-utils. As a result, EFS filesystems, including the shared /home directory, are not mounted, and desktop sessions terminate immediately upon login.

As shown in the VDI installation logs at /root/bootstrap/logs/install.log.{timestamp}:

• For Ubuntu,

+ cargo build --release --manifest-path /root/bootstrap/latest/efs-utils/src/proxy/Cargo.toml
error: failed to parse lock file at: /root/bootstrap/latest/efs-utils/src/proxy/Cargo.lock
+ apt install -y './build/amazon-efs-utils*deb'
E: Unsupported file ./build/amazon-efs-utils*deb given on commandline

• For RHEL/Rocky,

Either upgrade rustc or select compatible dependency versions with
`cargo update <name>@<current-ver> --precise <compatible-ver>`
where `<compatible-ver>` is the latest version supporting rustc 1.88.0

error: Bad exit status from /var/tmp/rpm-tmp.Fvv8Il (%build)


RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.Fvv8Il (%build)
make: *** [Makefile:56: rpm-only] Error 1
+ yum -y install 'build/amazon-efs-utils*rpm'

Affected versions

2025.09 and earlier

Mitigation

RES environment with public internet access

1. Download the patch script and efs_rhel_ubuntu_login_fix.patch using the commands below:
   1. The patch script requires AWS CLI v2, Python 3.9.16 or above, and Boto3.
   2. Configure the AWS CLI for the account / region where RES is deployed, and make sure that you have S3 permissions to write to the bucket for storing the patch script and add tags to S3 objects.
   3. Replace <res_version>, <environment-name>, <output-directory> with your RES version, RES environment name and local directory for storing the patch script.

RES_VERSION=<res_version>
ENVIRONMENT_NAME=<environment-name>
OUTPUT_DIRECTORY=<output-directory>

mkdir -p ${OUTPUT_DIRECTORY}
curl https://research-engineering-studio-us-east-1.s3.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patch.py --output ${OUTPUT_DIRECTORY}/patch.py
curl https://research-engineering-studio-us-east-1.s3.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patches/efs_rhel_ubuntu_login_fix.patch --output ${OUTPUT_DIRECTORY}/efs_rhel_ubuntu_login_fix.patch

2. Navigate to the directory where the patch script and patch file are downloaded. Run the following patch command:

python3 ${OUTPUT_DIRECTORY}/patch.py --environment-name ${ENVIRONMENT_NAME} --res-version ${RES_VERSION} --module installation --patch ${OUTPUT_DIRECTORY}/efs_rhel_ubuntu_login_fix.patch

3. Launch a new VDI from the RES project to verify the fix.

RES environment without public internet access

1. Download the patch script and efs_rhel_ubuntu_login_fix.patch by running the following commands:
   1. The patch script requires AWS CLI v2, Python 3.9.16 or above, and Boto3.
   2. Configure the AWS CLI for the account / region where RES is deployed, and make sure that you have S3 permissions to write to the bucket created by RES.
   3. Replace <res_version>, <environment-name>, and <output-directory> with your RES version, RES environment name, local directory for storing the patch script and your S3 bucket for storing the patch script.

RES_VERSION=<res_version>
ENVIRONMENT_NAME=<environment-name>
OUTPUT_DIRECTORY=<output-directory>

mkdir -p ${OUTPUT_DIRECTORY}
curl https://research-engineering-studio-us-east-1.s3.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patch.py --output ${OUTPUT_DIRECTORY}/patch.py
curl https://research-engineering-studio-us-east-1.s3.amazonaws.com/releases/${RES_VERSION}/patch_scripts/patches/efs_rhel_ubuntu_login_fix.patch --output ${OUTPUT_DIRECTORY}/efs_rhel_ubuntu_login_fix.patch

2. Navigate to the directory where the patch script and patch file are downloaded. Run the following patch command:

python3 ${OUTPUT_DIRECTORY}/patch.py --environment-name ${ENVIRONMENT_NAME} --res-version ${RES_VERSION} --module installation --patch ${OUTPUT_DIRECTORY}/efs_rhel_ubuntu_login_fix.patch

3. Bake new RES ready VDI AMIs following https://docs.aws.amazon.com/res/latest/ug/res-ready-ami.html and replace the source of DownloadRESLinuxInstallPackage step with s3://{env-name}-cluster-{region}-{account-id}/idea/releases/res-installation-scripts.tar.gz in the image builder component content:
   1. Replace <res_version> with your RES version, <env-name> with your Environment name, <region> with your Region, and <account-id> with your Account Id.

#  Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
#
#  Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
#  with the License. A copy of the License is located at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
#  or in the 'license' file accompanying this file. This file is distributed on an 'AS IS' BASIS, WITHOUT WARRANTIES
#  OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions
#  and limitations under the License.
name: research-and-engineering-studio-vdi-linux
description: An RES EC2 Image Builder component to install required RES software dependencies for Linux VDI.
schemaVersion: 1.0
parameters:
  - GPUFamily:
      type: string
      description: GPU family (NONE, NVIDIA, or AMD)
      default: NONE
phases:
  - name: build
    steps:
      - name: PrepareRESBootstrap
        action: ExecuteBash
        onFailure: Abort
        maxAttempts: 3
        inputs:
          commands:
            - "mkdir -p /root/bootstrap/logs"
            - "mkdir -p /root/bootstrap/latest"
      - name: DownloadRESLinuxInstallPackage
        action: S3Download
        onFailure: Abort
        maxAttempts: 3
        inputs:
          - source: "s3://<env-name>-cluster-<region>-<account-id>/idea/releases/res-installation-scripts.tar.gz"
            destination: "/root/bootstrap/res-installation-scripts/res-installation-scripts.tar.gz"
      - name: RunInstallScript
        action: ExecuteBash
        onFailure: Abort
        maxAttempts: 3
        inputs:
          commands:
            - "cd /root/bootstrap/res-installation-scripts"
            - "tar -xf res-installation-scripts.tar.gz"
            - "cd scripts/virtual-desktop-host/linux"
            - "/bin/bash install.sh -g {{ GPUFamily }}"
      - name: RunInstallPostRebootScript
        action: ExecuteBash
        onFailure: Abort
        maxAttempts: 3
        inputs:
          commands:
            - "cd /root/bootstrap/res-installation-scripts/scripts/virtual-desktop-host/linux"
            - 'sed -i ''/^export AWS_DEFAULT_PROFILE="bootstrap_profile"$/d'' install_post_reboot.sh'
            - "/bin/bash install_post_reboot.sh -g {{ GPUFamily }}"
      - name: PreventAL2023FromUninstallingCronie
        action: ExecuteBash
        onFailure: Abort
        maxAttempts: 3
        inputs:
          commands:
            - "rm -f /tmp/imagebuilder_service/crontab_installed" 

4. Launch a new VDI using the new RES ready AMI to verify the fix.