5-app-infra

5-app-infra

This repo is part of a multi-part guide that shows how to configure and deploy the example.com reference architecture described in Google Cloud security foundations guide. The following table lists the parts of the guide.

0-bootstrap	Bootstraps a Google Cloud organization, creating all the required resources and permissions to start using the Cloud Foundation Toolkit (CFT). This step also configures a CI/CD Pipeline for foundations code in subsequent stages.
1-org	Sets up top-level shared folders, networking projects, organization-level logging, and baseline security settings through organizational policies.
2-environments	Sets up development, nonproduction, and production environments within the Google Cloud organization that you've created.
3-networks-svpc	Sets up shared VPCs with default DNS, NAT (optional), Private Service networking, VPC service controls, on-premises Dedicated Interconnect, and baseline firewall rules for each environment. It also sets up the global DNS hub.
3-networks-hub-and-spoke	Sets up shared VPCs with all the default configuration found on step 3-networks-svpc, but here the architecture will be based on the Hub and Spoke network model. It also sets up the global DNS hub
4-projects	Sets up a folder structure, projects, and an application infrastructure pipeline for applications, which are connected as service projects to the shared VPC created in the previous stage.
5-app-infra (this file)	Deploy a simple [Compute Engine](https://cloud.google.com/compute/) instance in one of the business unit projects using the infra pipeline set up in 4-projects.

For an overview of the architecture and the parts, see the terraform-example-foundation README file.

Purpose

The purpose of this step is to deploy a simple Compute Engine instance in one of the business unit projects using the infra pipeline set up in 4-projects. The infra pipeline is created in step 4-projects within the shared env and has a Cloud Build pipeline configured to manage infrastructure within projects.

As part of this deployment, the provided Terraform code automates the provisioning of a secure infrastructure in Google Cloud to run workloads in Confidential Space — based on Confidential VMs with integrity verification, data isolation, trusted image execution, and integration with KMS for encryption and GCS for storage. In addition to this documentation, you can also follow Google's official tutorial to test each step directly: Create your first Confidential Space environment.

To better understand the structure and content of the tokens used in Confidential Space, refer to the reference documentation on Token Claims. This link details the information contained in the tokens and how they are used to ensure the security and integrity of the environment.

This Compute Engine instance is created using the base network from step 3-networks and is used to access private services. For the Confidential Space, a Docker image built in step 4-projects serves as the base for the confidential instance. There is also a Source Repository configured with build triggers similar to the CI/CD Pipeline setup in 0-bootstrap.

Prerequisites

1. 0-bootstrap executed successfully.
2. 1-org executed successfully.
3. 2-environments executed successfully.
4. 3-networks executed successfully.
5. 4-projects executed successfully.

Troubleshooting

Please refer to troubleshooting if you run into issues during this step.

Usage

Notes:

• For Confidential space, additional firewall rules and directional perimeter rules may be required, depending on the additional workloads to be deployed.
• If you are using MacOS, replace cp -RT with cp -R in the relevant commands. The -T flag is needed for Linux, but causes problems for MacOS.

Deploying with Cloud Build

1. Clone the gcp-policies repo based on the Terraform output from the 0-bootstrap step. Clone the repo at the same level of the terraform-example-foundation folder, the following instructions assume this layout. Run terraform output cloudbuild_project_id in the 0-bootstrap folder to get the Cloud Build Project ID.

   export INFRA_PIPELINE_PROJECT_ID=$(terraform -chdir="gcp-projects/business_unit_1/shared/" output -raw cloudbuild_project_id)
   echo ${INFRA_PIPELINE_PROJECT_ID}
   
   gcloud source repos clone gcp-policies gcp-policies-app-infra --project=${INFRA_PIPELINE_PROJECT_ID}

   Note: gcp-policies repo has the same name as the repo created in step 1-org. In order to prevent a collision, the previous command will clone this repo in the folder gcp-policies-app-infra.

2. Navigate into the repo and copy contents of policy-library to new repo. All subsequent steps assume you are running them from the gcp-policies-app-infra directory. If you run them from another directory, adjust your copy paths accordingly.

   cd gcp-policies-app-infra
   git checkout -b main
   
   cp -RT ../terraform-example-foundation/policy-library/ .

3. Commit changes and push your main branch to the new repo.

   git add .
   git commit -m 'Initialize policy library repo'
   
   git push --set-upstream origin main

4. Navigate out of the repo.

   cd ..

5. Clone the bu1-example-app repo.

   gcloud source repos clone bu1-example-app --project=${INFRA_PIPELINE_PROJECT_ID}

6. Navigate into the repo, change to non-main branch and copy contents of foundation to new repo. All subsequent steps assume you are running them from the bu1-example-app directory. If you run them from another directory, adjust your copy paths accordingly.

   cd bu1-example-app
   git checkout -b plan
   
   cp -RT ../terraform-example-foundation/5-app-infra/ .
   cp ../terraform-example-foundation/build/cloudbuild-tf-* .
   cp ../terraform-example-foundation/build/tf-wrapper.sh .
   chmod 755 ./tf-wrapper.sh

7. Rename common.auto.example.tfvars to common.auto.tfvars.

   mv common.auto.example.tfvars common.auto.tfvars

8. Update the file with values from your environment and 0-bootstrap. See any of the business unit 1 envs folders README.md files for additional information on the values in the common.auto.tfvars file.

   export remote_state_bucket=$(terraform -chdir="../terraform-example-foundation/0-bootstrap/" output -raw projects_gcs_bucket_tfstate)
   echo "remote_state_bucket = ${remote_state_bucket}"
   sed -i'' -e "s/REMOTE_STATE_BUCKET/${remote_state_bucket}/" ./common.auto.tfvars

9. Get the confidential_image_digest value from the Docker image created in gcp-projects

export CLOUD_BUILD_PROJECT_ID=$(terraform -chdir="terraform-example-foundation/0-bootstrap/" output -raw cloudbuild_project_id)
echo ${CLOUD_BUILD_PROJECT_ID}

export DEFAULT_REGION=$(terraform -chdir="../gcp-projects/business_unit_1/shared" output -raw default_region)
echo ${DEFAULT_REGION}

export confidential_image_digest=$(gcloud artifacts docker images describe ${DEFAULT_REGION}-docker.pkg.dev/${CLOUD_BUILD_PROJECT_ID}/tf-runners/confidential_space_image:latest --project=${CLOUD_BUILD_PROJECT_ID})
echo "confidential_image_digest = ${confidential_image_digest}"

1. Update IMAGE_DIGEST value in the file common.auto.tfvars.

sed -i'' -e "s/IMAGE_DIGEST/${confidential_image_digest}/" ./common.auto.tfvars

1. Commit changes.

   git add .
   git commit -m 'Initialize bu1 example app repo'

2. Push your plan branch to trigger a plan for all environments. Because the plan branch is not a named environment branch, pushing your plan branch triggers terraform plan but not terraform apply. Review the plan output in your Cloud Build project https://console.cloud.google.com/cloud-build/builds;region=DEFAULT_REGION?project=YOUR_INFRA_PIPELINE_PROJECT_ID

   git push --set-upstream origin plan

3. Merge changes to development. Because this is a named environment branch, pushing to this branch triggers both terraform plan and terraform apply. Review the apply output in your Cloud Build project https://console.cloud.google.com/cloud-build/builds;region=DEFAULT_REGION?project=YOUR_INFRA_PIPELINE_PROJECT_ID

   git checkout -b development
   git push origin development

4. Merge changes to nonproduction. Because this is a named environment branch, pushing to this branch triggers both terraform plan and terraform apply. Review the apply output in your Cloud Build project https://console.cloud.google.com/cloud-build/builds;region=DEFAULT_REGION?project=YOUR_INFRA_PIPELINE_PROJECT_ID

   git checkout -b nonproduction
   git push origin nonproduction

5. Merge changes to production branch. Because this is a named environment branch, pushing to this branch triggers both terraform plan and terraform apply. Review the apply output in your Cloud Build project https://console.cloud.google.com/cloud-build/builds;region=DEFAULT_REGION?project=YOUR_INFRA_PIPELINE_PROJECT_ID

   git checkout -b production
   git push origin production

Run Terraform locally

1. The next instructions assume that you are at the same level of the terraform-example-foundation folder. Change into 5-app-infra folder, copy the Terraform wrapper script and ensure it can be executed.

   cd terraform-example-foundation/5-app-infra
   cp ../build/tf-wrapper.sh .
   chmod 755 ./tf-wrapper.sh

2. Rename common.auto.example.tfvars files to common.auto.tfvars.

   mv common.auto.example.tfvars common.auto.tfvars

3. Update common.auto.tfvars file with values from your environment.

4. Use terraform output to get the project backend bucket value from 0-bootstrap.

   export remote_state_bucket=$(terraform -chdir="../0-bootstrap/" output -raw projects_gcs_bucket_tfstate)
   echo "remote_state_bucket = ${remote_state_bucket}"
   sed -i'' -e "s/REMOTE_STATE_BUCKET/${remote_state_bucket}/" ./common.auto.tfvars

5. Provide the user that will be running ./tf-wrapper.sh the Service Account Token Creator role to the bu1 Terraform service account.

6. Provide the user permissions to run the terraform locally with the serviceAccountTokenCreator permission.

   member="user:$(gcloud auth list --filter="status=ACTIVE" --format="value(account)")"
   echo ${member}
   
   project_id=$(terraform -chdir="../4-projects/business_unit_1/shared/" output -raw cloudbuild_project_id)
   echo ${project_id}
   
   terraform_sa=$(terraform -chdir="../4-projects/business_unit_1/shared/" output -json terraform_service_accounts | jq '."bu1-example-app"' --raw-output)
   echo ${terraform_sa}
   
   gcloud iam service-accounts add-iam-policy-binding ${terraform_sa} --project ${project_id} --member="${member}" --role="roles/iam.serviceAccountTokenCreator"

7. Update backend.tf with your bucket from the infra pipeline output.

   export backend_bucket=$(terraform -chdir="../4-projects/business_unit_1/shared/" output -json state_buckets | jq '."bu1-example-app"' --raw-output)
   echo "backend_bucket = ${backend_bucket}"
   
   for i in `find . -name 'backend.tf'`; do sed -i'' -e "s/UPDATE_APP_INFRA_BUCKET/${backend_bucket}/" $i; done

We will now deploy each of our environments (development/production/nonproduction) using this script. When using Cloud Build or Jenkins as your CI/CD tool, each environment corresponds to a branch in the repository for the 5-app-infra step. Only the corresponding environment is applied.

To use the validate option of the tf-wrapper.sh script, please follow the instructions to install the terraform-tools component.

1. Use terraform output to get the Infra Pipeline Project ID from 4-projects output.

   export INFRA_PIPELINE_PROJECT_ID=$(terraform -chdir="../4-projects/business_unit_1/shared/" output -raw cloudbuild_project_id)
   echo ${INFRA_PIPELINE_PROJECT_ID}
   
   export GOOGLE_IMPERSONATE_SERVICE_ACCOUNT=$(terraform -chdir="../4-projects/business_unit_1/shared/" output -json terraform_service_accounts | jq '."bu1-example-app"' --raw-output)
   echo ${GOOGLE_IMPERSONATE_SERVICE_ACCOUNT}

2. Run init and plan and review output for environment production.

   ./tf-wrapper.sh init production
   ./tf-wrapper.sh plan production

3. Run validate and check for violations.

   ./tf-wrapper.sh validate production $(pwd)/../policy-library ${INFRA_PIPELINE_PROJECT_ID}

4. Run apply production.

   ./tf-wrapper.sh apply production

5. Run init and plan and review output for environment nonproduction.

   ./tf-wrapper.sh init nonproduction
   ./tf-wrapper.sh plan nonproduction

6. Run validate and check for violations.

   ./tf-wrapper.sh validate nonproduction $(pwd)/../policy-library ${INFRA_PIPELINE_PROJECT_ID}

7. Run apply nonproduction.

   ./tf-wrapper.sh apply nonproduction

8. Run init and plan and review output for environment development.

   ./tf-wrapper.sh init development
   ./tf-wrapper.sh plan development

9. Run validate and check for violations.

   ./tf-wrapper.sh validate development $(pwd)/../policy-library ${INFRA_PIPELINE_PROJECT_ID}

10. Run apply development.

    ./tf-wrapper.sh apply development

If you received any errors or made any changes to the Terraform config or common.auto.tfvars you must re-run ./tf-wrapper.sh plan <env> before running ./tf-wrapper.sh apply <env>.

After executing this stage, unset the GOOGLE_IMPERSONATE_SERVICE_ACCOUNT environment variable.

unset GOOGLE_IMPERSONATE_SERVICE_ACCOUNT