From 6c5bc441dc8e516f3bb6fcfcf2b088f558b56744 Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Sun, 17 May 2026 07:48:19 -0700 Subject: [PATCH 01/23] Add production refactor design spec Documents full improvement plan: Datadog observability, Flux GitOps wiring, S3/DynamoDB remote state, GitHub Actions CI/CD, module restructure, and bug fixes. Co-Authored-By: Claude Sonnet 4.6 --- ...-eks-gitops-platform-improvement-design.md | 235 ++++++++++++++++++ 1 file changed, 235 insertions(+) create mode 100644 docs/superpowers/specs/2026-05-17-eks-gitops-platform-improvement-design.md diff --git a/docs/superpowers/specs/2026-05-17-eks-gitops-platform-improvement-design.md b/docs/superpowers/specs/2026-05-17-eks-gitops-platform-improvement-design.md new file mode 100644 index 0000000..a78067c --- /dev/null +++ b/docs/superpowers/specs/2026-05-17-eks-gitops-platform-improvement-design.md @@ -0,0 +1,235 @@ +# EKS GitOps Platform — Production Refactor Design + +**Date:** 2026-05-17 +**Status:** Approved + +--- + +## Summary + +Full production-grade refactor of the EKS GitOps Platform. Replaces Komodor with Datadog (full-stack observability), wires Flux GitOps to a parameterized GitHub repo, provisions remote Terraform state via S3 + DynamoDB, adds a real GitHub Actions CI/CD pipeline, fixes all current bugs and structural debt, and makes the repo forkable and runnable by anyone. + +Single environment, production-grade. + +--- + +## 1. Repository Structure + +``` +eks-gitops-platform/ +├── bootstrap/ # Run once — provisions S3 + DynamoDB for remote state +│ ├── main.tf +│ ├── variables.tf +│ └── outputs.tf +├── modules/ +│ ├── cert_manager/ # Improved: add variables.tf + outputs.tf +│ ├── datadog/ # New: replaces komodor/ +│ ├── flux/ # Improved: add GitRepository + Kustomization wiring +│ ├── istio/ # Improved: add variables.tf + outputs.tf +│ ├── postgres/ # Improved: add variables.tf + outputs.tf +│ └── vault/ # Fixed: correct manifest paths +├── flux-kustomize/test-app/ # Unchanged +├── manifests/ # Unchanged +├── scripts/ +│ ├── bootstrap.sh # New: runs bootstrap/ then writes backend.tf +│ └── one-shot-deploy.sh # Fixed: correct paths, idempotent +├── .github/workflows/ +│ ├── terraform-plan.yml # New: runs on PR +│ └── terraform-apply.yml # New: runs on merge to main +├── backend.tf # New: S3 remote state config +├── main.tf # Cleaned: remove duplicate vars, wire all modules +├── variables.tf # Cleaned: single source of truth +├── outputs.tf # Unchanged +├── providers.tf # New: explicit provider versions + IRSA +└── terraform.tfvars.example # New: documents all required inputs +``` + +`komodor/` is removed entirely. `bootstrap/` is a one-time run with local state before the main root module is initialized. + +--- + +## 2. State Management & Providers + +### Bootstrap Module (`bootstrap/`) + +Standalone Terraform root with its own local state (intentional — it bootstraps the remote state for everything else). Provisions: + +- **S3 bucket** — versioning enabled, SSE-S3 encryption, public access fully blocked +- **DynamoDB table** — `LockID` hash key for state locking + +`scripts/bootstrap.sh` runs `bootstrap/` and writes the resulting bucket name and table name into `backend.tf` automatically — no manual copy-paste. + +### `backend.tf` + +```hcl +terraform { + backend "s3" { + bucket = "" + key = "eks-gitops-platform/terraform.tfstate" + region = "us-west-2" + dynamodb_table = "" + encrypt = true + } +} +``` + +### `providers.tf` + +Explicit version constraints for all providers: + +```hcl +terraform { + required_providers { + aws = { source = "hashicorp/aws", version = "~> 5.0" } + helm = { source = "hashicorp/helm", version = "~> 2.13" } + kubernetes = { source = "hashicorp/kubernetes", version = "~> 2.29" } + } +} +``` + +IRSA configured via the `aws` provider and `eks` data sources — enables Datadog, cert-manager, and Vault to assume IAM roles without static credentials. + +--- + +## 3. Datadog Module (`modules/datadog/`) + +Replaces `modules/komodor/` entirely. Installs the Datadog Agent via the official `datadog/datadog` Helm chart. + +### What gets deployed + +| Component | Purpose | +|---|---| +| DaemonSet Agent | Node-level metrics, host network stats | +| Cluster Agent | Kubernetes state metrics (deployments, pods, nodes, HPA) | +| APM | Trace collection, port 8126 on all nodes | +| Log Collection | All container logs with Kubernetes metadata tagging | +| NPM | Off by default, enabled via variable | + +### Module inputs (`variables.tf`) + +| Variable | Type | Default | Notes | +|---|---|---|---| +| `datadog_api_key` | string | — | Sensitive, required | +| `datadog_app_key` | string | — | Sensitive, required | +| `datadog_site` | string | `"datadoghq.com"` | Override for EU: `datadoghq.eu` | +| `cluster_name` | string | — | Passed from root, tags all telemetry | +| `enable_npm` | bool | `false` | Network Performance Monitoring | + +### IRSA + +Datadog agent gets an IAM role with read-only EC2/EKS permissions to enrich metrics with AWS metadata (instance type, AZ, etc.) without static credentials. + +--- + +## 4. Flux GitOps Wiring (`modules/flux/`) + +### Current problem + +The Flux module installs the controller but never creates a `GitRepository` or `Kustomization` — GitOps is not active. + +### Fix + +The module creates two additional `kubernetes_manifest` resources: + +1. **`GitRepository`** — points at the user's GitHub repo, watches the configured branch, polls every 1 minute +2. **`Kustomization`** — syncs `flux-kustomize/` path from that repo, health checks enabled, pruning enabled (orphaned resources deleted automatically) + +### Module inputs added + +| Variable | Default | Notes | +|---|---|---| +| `github_repo_url` | — | Required. e.g. `https://github.com/yourname/eks-gitops-platform` | +| `github_branch` | `"main"` | Branch to watch | +| `flux_sync_path` | `"./flux-kustomize"` | Path within repo to sync | + +### Authentication + +Public repos require no credentials. A commented-out `flux_secret` resource block is included for private repos (GitHub PAT in a Kubernetes secret), clearly documented in `terraform.tfvars.example`. + +### Result + +Anyone who forks the repo sets `github_repo_url` in their tfvars and Flux immediately syncs their fork. The `test-app` in `flux-kustomize/test-app/` deploys automatically as a GitOps smoke test. + +--- + +## 5. GitHub Actions CI/CD Pipeline + +### `terraform-plan.yml` — triggers on Pull Request + +1. Checkout + setup Terraform +2. `terraform fmt -check` — fails PR if formatting is off +3. `terraform validate` +4. `tfsec` — static security scan, fails on HIGH/CRITICAL findings +5. `terraform plan` — output posted as PR comment via `actions/github-script` + +### `terraform-apply.yml` — triggers on push to `main` + +1. Checkout + setup Terraform +2. `terraform init` with S3 backend +3. `terraform plan` — re-runs plan to ensure state is current at apply time +4. `terraform apply -auto-approve` + +### GitHub Secrets required + +| Secret | Purpose | +|---|---| +| `AWS_ACCESS_KEY_ID` | Terraform AWS access | +| `AWS_SECRET_ACCESS_KEY` | Terraform AWS access | +| `DATADOG_API_KEY` | Passed as `TF_VAR_datadog_api_key` | +| `DATADOG_APP_KEY` | Passed as `TF_VAR_datadog_app_key` | +| `TF_VAR_github_repo_url` | Flux GitRepository target | + +### Concurrency & ordering + +- Apply workflow requires plan workflow to pass (`needs:` dependency) +- `concurrency: group: terraform` prevents simultaneous applies corrupting state + +--- + +## 6. Fixes & Cleanup + +### Structural fixes + +| Issue | Fix | +|---|---| +| Variables duplicated between `main.tf` and `variables.tf` | Remove duplicates from `main.tf`; `variables.tf` is sole source of truth | +| `cluster_version = "1.29"` hardcoded | Promoted to variable, default `"1.30"` | +| Modules lack `variables.tf` / `outputs.tf` | Added to all modules | +| Vault manifest path incorrect | Files moved to `modules/vault/` to match `${path.module}/` references | +| VPC block ignores `var.vpc_cidr` etc. | Updated to use all variables from `variables.tf` | +| Inline `provider "aws"` in `main.tf` | Moved to `providers.tf` | + +### `one-shot-deploy.sh` rewrite + +- Paths corrected to `modules/` subdirectory structure +- Bootstrap step added at top +- References to non-existent `app-deploy.yaml` removed (Flux wiring handled by Terraform) +- Idempotent checks added + +### `terraform.tfvars.example` + +Documents every required and optional variable with descriptions and example values. Sensitive variables (API keys) clearly marked with instructions never to commit actual values. + +--- + +## 7. Onboarding Flow (for anyone who forks) + +``` +1. Clone the repo +2. Configure AWS credentials (AWS_PROFILE or env vars) +3. Copy terraform.tfvars.example → terraform.tfvars and fill in values +4. Run: bash scripts/bootstrap.sh # creates S3 + DynamoDB, writes backend.tf +5. Run: terraform init # initializes with remote state +6. Run: terraform apply # provisions EKS + all platform services +# OR: push to GitHub and let Actions handle apply +``` + +--- + +## 8. What Is Not Changing + +- `flux-kustomize/test-app/` manifests — unchanged +- `manifests/cert-manager/`, `manifests/istio/`, `manifests/vault/` — unchanged +- `outputs.tf` — unchanged +- Overall EKS cluster topology (VPC CIDR, subnet layout, node group sizing) +- Istio, cert-manager, Postgres, Vault Helm chart versions (bumped only if a newer stable version exists) From 3f89b140f7b635ed7ff4c7bdf77b9257982636e6 Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Sun, 17 May 2026 07:58:44 -0700 Subject: [PATCH 02/23] Add production refactor implementation plan 16-task plan covering: S3/DynamoDB bootstrap, providers, module restructure, Datadog IRSA, Flux GitOps wiring, GitHub Actions CI/CD, and full onboarding documentation. Co-Authored-By: Claude Sonnet 4.6 --- ...eks-gitops-platform-production-refactor.md | 1759 +++++++++++++++++ 1 file changed, 1759 insertions(+) create mode 100644 docs/superpowers/plans/2026-05-17-eks-gitops-platform-production-refactor.md diff --git a/docs/superpowers/plans/2026-05-17-eks-gitops-platform-production-refactor.md b/docs/superpowers/plans/2026-05-17-eks-gitops-platform-production-refactor.md new file mode 100644 index 0000000..d47c233 --- /dev/null +++ b/docs/superpowers/plans/2026-05-17-eks-gitops-platform-production-refactor.md @@ -0,0 +1,1759 @@ +# EKS GitOps Platform — Production Refactor Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Refactor the EKS GitOps Platform into a production-grade, forkable Terraform monorepo with remote state, full Datadog observability, active Flux GitOps, and a real GitHub Actions CI/CD pipeline. + +**Architecture:** A single Terraform root (`eks-gitops-platform/main.tf`) calls all child modules in `modules/`. A separate `bootstrap/` root (local state only) provisions the S3 + DynamoDB backend before the main root is initialized. The `providers.tf` configures AWS, Helm, and Kubernetes providers using EKS cluster outputs. Komodor is removed; Datadog replaces it with full APM + logs + metrics via IRSA. + +**Tech Stack:** Terraform ≥ 1.7, AWS provider ~> 5.0, Helm provider ~> 2.13, Kubernetes provider ~> 2.29, terraform-aws-modules/vpc v5.8.1, terraform-aws-modules/eks v20.8.4, Datadog Helm chart v3.67.0, fluxcd-community/flux2 v2.10.0, GitHub Actions. + +--- + +## File Map + +All paths below are relative to the Terraform root: `eks-gitops-platform/eks-gitops-platform/` + +| Action | File | Purpose | +|---|---|---| +| Create | `bootstrap/main.tf` | S3 bucket + DynamoDB table for remote state | +| Create | `bootstrap/variables.tf` | Bootstrap inputs | +| Create | `bootstrap/outputs.tf` | Bucket + table name outputs | +| Create | `providers.tf` | All provider version constraints + Helm/K8s config | +| Create | `backend.tf` | S3 backend stub (overwritten by bootstrap.sh) | +| Rewrite | `variables.tf` | Single source of truth — add cluster_version, datadog, flux vars | +| Rewrite | `main.tf` | Remove duplicate vars + inline provider; fix EKS args; wire all modules | +| Create | `modules/cert_manager/variables.tf` | Namespace + chart version inputs | +| Create | `modules/cert_manager/outputs.tf` | Separate outputs file | +| Create | `modules/istio/variables.tf` | Namespace + chart version inputs | +| Create | `modules/istio/outputs.tf` | Separate outputs file | +| Create | `modules/postgres/variables.tf` | Namespace + chart version inputs | +| Create | `modules/postgres/outputs.tf` | Separate outputs file | +| Rewrite | `modules/vault/main.tf` | Replace broken kubernetes_manifest with native k8s resources | +| Create | `modules/vault/variables.tf` | Namespace + chart version inputs | +| Create | `modules/vault/outputs.tf` | Separate outputs file | +| Rewrite | `modules/flux/main.tf` | Add GitRepository + Kustomization resources | +| Create | `modules/flux/variables.tf` | github_repo_url, github_branch, flux_sync_path | +| Create | `modules/flux/outputs.tf` | Separate outputs file | +| Create | `modules/datadog/main.tf` | Datadog Helm release + IRSA IAM role | +| Create | `modules/datadog/variables.tf` | API keys, site, cluster_name, OIDC inputs | +| Create | `modules/datadog/outputs.tf` | Namespace + IAM role ARN | +| Create | `modules/datadog/values.yaml` | Datadog Helm values (APM, logs, cluster agent) | +| Delete | `modules/komodor/` | Entire directory removed | +| Rewrite | `scripts/bootstrap.sh` | Runs bootstrap/, writes backend.tf | +| Rewrite | `scripts/one-shot-deploy.sh` | Fixed paths, idempotent, single root apply | +| Create | `.github/workflows/terraform-plan.yml` | PR: fmt + validate + tfsec + plan + comment | +| Create | `.github/workflows/terraform-apply.yml` | Push to main: plan + apply | +| Create | `terraform.tfvars.example` | Documents all required inputs | +| Update | `.gitignore` | Add terraform.tfvars, *.tfstate, terraform.lock.hcl to ignore | +| Update | `README.md` | New onboarding flow + GitHub Secrets table | + +--- + +## Task 1: Bootstrap Module — S3 + DynamoDB State Backend + +**Files:** +- Create: `eks-gitops-platform/bootstrap/main.tf` +- Create: `eks-gitops-platform/bootstrap/variables.tf` +- Create: `eks-gitops-platform/bootstrap/outputs.tf` + +- [ ] **Step 1: Create `bootstrap/main.tf`** + +```hcl +terraform { + required_providers { + aws = { + source = "hashicorp/aws" + version = "~> 5.0" + } + } + required_version = ">= 1.7" +} + +provider "aws" { + region = var.aws_region +} + +resource "aws_s3_bucket" "tfstate" { + bucket = var.state_bucket_name + + lifecycle { + prevent_destroy = true + } +} + +resource "aws_s3_bucket_versioning" "tfstate" { + bucket = aws_s3_bucket.tfstate.id + versioning_configuration { + status = "Enabled" + } +} + +resource "aws_s3_bucket_server_side_encryption_configuration" "tfstate" { + bucket = aws_s3_bucket.tfstate.id + rule { + apply_server_side_encryption_by_default { + sse_algorithm = "AES256" + } + } +} + +resource "aws_s3_bucket_public_access_block" "tfstate" { + bucket = aws_s3_bucket.tfstate.id + block_public_acls = true + block_public_policy = true + ignore_public_acls = true + restrict_public_buckets = true +} + +resource "aws_dynamodb_table" "tfstate_lock" { + name = var.lock_table_name + billing_mode = "PAY_PER_REQUEST" + hash_key = "LockID" + + attribute { + name = "LockID" + type = "S" + } +} +``` + +- [ ] **Step 2: Create `bootstrap/variables.tf`** + +```hcl +variable "aws_region" { + description = "AWS region for state infrastructure" + type = string + default = "us-west-2" +} + +variable "state_bucket_name" { + description = "S3 bucket name for Terraform state — must be globally unique" + type = string + default = "eks-gitops-tfstate" +} + +variable "lock_table_name" { + description = "DynamoDB table name for state locking" + type = string + default = "eks-gitops-tfstate-lock" +} +``` + +- [ ] **Step 3: Create `bootstrap/outputs.tf`** + +```hcl +output "state_bucket_name" { + description = "S3 bucket name for Terraform state" + value = aws_s3_bucket.tfstate.bucket +} + +output "lock_table_name" { + description = "DynamoDB table name for state locking" + value = aws_dynamodb_table.tfstate_lock.name +} + +output "aws_region" { + description = "AWS region where state infrastructure was created" + value = var.aws_region +} +``` + +- [ ] **Step 4: Validate the bootstrap module** + +```bash +cd eks-gitops-platform/eks-gitops-platform/bootstrap +terraform init -input=false +terraform validate +terraform fmt -check +``` + +Expected: `Success! The configuration is valid.` + +- [ ] **Step 5: Commit** + +```bash +git add eks-gitops-platform/bootstrap/ +git commit -m "feat: add bootstrap module for S3 + DynamoDB remote state" +``` + +--- + +## Task 2: providers.tf and backend.tf + +**Files:** +- Create: `eks-gitops-platform/providers.tf` +- Create: `eks-gitops-platform/backend.tf` + +- [ ] **Step 1: Create `providers.tf`** + +```hcl +terraform { + required_version = ">= 1.7" + + required_providers { + aws = { + source = "hashicorp/aws" + version = "~> 5.0" + } + helm = { + source = "hashicorp/helm" + version = "~> 2.13" + } + kubernetes = { + source = "hashicorp/kubernetes" + version = "~> 2.29" + } + } +} + +provider "aws" { + region = var.aws_region +} + +provider "helm" { + kubernetes { + host = module.eks.cluster_endpoint + cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data) + exec { + api_version = "client.authentication.k8s.io/v1beta1" + command = "aws" + args = ["eks", "get-token", "--cluster-name", var.cluster_name, "--region", var.aws_region] + } + } +} + +provider "kubernetes" { + host = module.eks.cluster_endpoint + cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data) + exec { + api_version = "client.authentication.k8s.io/v1beta1" + command = "aws" + args = ["eks", "get-token", "--cluster-name", var.cluster_name, "--region", var.aws_region] + } +} +``` + +- [ ] **Step 2: Create `backend.tf`** + +```hcl +# Generated by scripts/bootstrap.sh — do not edit manually. +# Run scripts/bootstrap.sh to populate the bucket and table names. +terraform { + backend "s3" { + bucket = "REPLACE_WITH_BUCKET_NAME" + key = "eks-gitops-platform/terraform.tfstate" + region = "us-west-2" + dynamodb_table = "REPLACE_WITH_TABLE_NAME" + encrypt = true + } +} +``` + +- [ ] **Step 3: Commit** + +```bash +git add eks-gitops-platform/providers.tf eks-gitops-platform/backend.tf +git commit -m "feat: add providers.tf with version constraints and Helm/K8s config" +``` + +--- + +## Task 3: Rewrite variables.tf — Single Source of Truth + +**Files:** +- Rewrite: `eks-gitops-platform/variables.tf` + +- [ ] **Step 1: Overwrite `variables.tf` with complete variable set** + +```hcl +variable "aws_region" { + description = "AWS region to deploy all resources" + type = string + default = "us-west-2" +} + +variable "cluster_name" { + description = "EKS cluster name" + type = string + default = "eks-gitops-cluster" +} + +variable "cluster_version" { + description = "Kubernetes version for the EKS cluster" + type = string + default = "1.30" +} + +variable "environment" { + description = "Environment tag applied to all resources" + type = string + default = "prod" +} + +variable "vpc_cidr" { + description = "CIDR block for the VPC" + type = string + default = "10.0.0.0/16" +} + +variable "private_subnets" { + description = "List of private subnet CIDRs" + type = list(string) + default = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"] +} + +variable "public_subnets" { + description = "List of public subnet CIDRs" + type = list(string) + default = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"] +} + +variable "datadog_api_key" { + description = "Datadog API key — store in GitHub Secrets as DATADOG_API_KEY" + type = string + sensitive = true +} + +variable "datadog_app_key" { + description = "Datadog application key — store in GitHub Secrets as DATADOG_APP_KEY" + type = string + sensitive = true +} + +variable "datadog_site" { + description = "Datadog intake site (datadoghq.com for US, datadoghq.eu for EU)" + type = string + default = "datadoghq.com" +} + +variable "enable_npm" { + description = "Enable Datadog Network Performance Monitoring (requires kernel headers on nodes)" + type = bool + default = false +} + +variable "github_repo_url" { + description = "GitHub repo URL for Flux to sync — e.g. https://github.com/yourname/eks-gitops-platform" + type = string +} + +variable "github_branch" { + description = "Git branch Flux watches for changes" + type = string + default = "main" +} + +variable "flux_sync_path" { + description = "Path within the repo that Flux syncs to the cluster" + type = string + default = "./flux-kustomize" +} +``` + +- [ ] **Step 2: Commit** + +```bash +git add eks-gitops-platform/variables.tf +git commit -m "refactor: variables.tf is now single source of truth; add cluster_version, datadog, flux vars" +``` + +--- + +## Task 4: Rewrite main.tf — Fix EKS Args and Wire All Modules + +**Files:** +- Rewrite: `eks-gitops-platform/main.tf` + +Note: The current `main.tf` has three bugs: (1) `subnets` should be `subnet_ids`, (2) `desired_capacity`/`max_capacity`/`min_capacity` should be `desired_size`/`max_size`/`min_size`, (3) inline `provider "aws"` and duplicate variable blocks must be removed. + +- [ ] **Step 1: Overwrite `main.tf`** + +```hcl +data "aws_availability_zones" "available" {} + +module "vpc" { + source = "terraform-aws-modules/vpc/aws" + version = "5.8.1" + + name = "eks-vpc" + cidr = var.vpc_cidr + + azs = slice(data.aws_availability_zones.available.names, 0, 3) + private_subnets = var.private_subnets + public_subnets = var.public_subnets + + enable_nat_gateway = true + single_nat_gateway = true + + tags = { + Terraform = "true" + Environment = var.environment + } +} + +module "eks" { + source = "terraform-aws-modules/eks/aws" + version = "20.8.4" + + cluster_name = var.cluster_name + cluster_version = var.cluster_version + subnet_ids = module.vpc.private_subnets + vpc_id = module.vpc.vpc_id + cluster_endpoint_public_access = true + + enable_irsa = true + + eks_managed_node_group_defaults = { + instance_types = ["t3.medium"] + } + + eks_managed_node_groups = { + default = { + desired_size = 3 + max_size = 4 + min_size = 2 + instance_types = ["t3.medium"] + capacity_type = "ON_DEMAND" + } + } + + tags = { + Environment = var.environment + Terraform = "true" + } +} + +module "cert_manager" { + source = "./modules/cert_manager" + + depends_on = [module.eks] +} + +module "istio" { + source = "./modules/istio" + + depends_on = [module.eks] +} + +module "vault" { + source = "./modules/vault" + + depends_on = [module.eks] +} + +module "postgres" { + source = "./modules/postgres" + + depends_on = [module.eks] +} + +module "flux" { + source = "./modules/flux" + + github_repo_url = var.github_repo_url + github_branch = var.github_branch + flux_sync_path = var.flux_sync_path + + depends_on = [module.eks] +} + +module "datadog" { + source = "./modules/datadog" + + datadog_api_key = var.datadog_api_key + datadog_app_key = var.datadog_app_key + datadog_site = var.datadog_site + cluster_name = var.cluster_name + enable_npm = var.enable_npm + oidc_provider_arn = module.eks.oidc_provider_arn + oidc_provider_url = module.eks.cluster_oidc_issuer_url + + depends_on = [module.eks] +} +``` + +- [ ] **Step 2: Validate (skip backend check since backend.tf has placeholder)** + +```bash +cd eks-gitops-platform/eks-gitops-platform +terraform init -backend=false -input=false +terraform validate +``` + +Expected: `Success! The configuration is valid.` + +- [ ] **Step 3: Commit** + +```bash +git add eks-gitops-platform/main.tf +git commit -m "fix: correct EKS arg names (subnet_ids, desired_size); wire all child modules; remove duplicate vars" +``` + +--- + +## Task 5: cert_manager Module — Add variables.tf and outputs.tf + +**Files:** +- Create: `eks-gitops-platform/modules/cert_manager/variables.tf` +- Rewrite: `eks-gitops-platform/modules/cert_manager/outputs.tf` (currently inline in main.tf) +- Modify: `eks-gitops-platform/modules/cert_manager/main.tf` + +- [ ] **Step 1: Create `modules/cert_manager/variables.tf`** + +```hcl +variable "namespace" { + description = "Namespace for cert-manager" + type = string + default = "cert-manager" +} + +variable "chart_version" { + description = "cert-manager Helm chart version" + type = string + default = "v1.14.3" +} +``` + +- [ ] **Step 2: Create `modules/cert_manager/outputs.tf`** + +```hcl +output "namespace" { + description = "Namespace where cert-manager is deployed" + value = kubernetes_namespace.cert_manager.metadata[0].name +} +``` + +- [ ] **Step 3: Update `modules/cert_manager/main.tf` to use variables** + +Replace the existing content of `modules/cert_manager/main.tf` with: + +```hcl +resource "kubernetes_namespace" "cert_manager" { + metadata { + name = var.namespace + labels = { + "cert-manager.io/disable-validation" = "true" + } + } +} + +resource "helm_release" "cert_manager" { + name = "cert-manager" + repository = "https://charts.jetstack.io" + chart = "cert-manager" + version = var.chart_version + namespace = kubernetes_namespace.cert_manager.metadata[0].name + + values = [file("${path.module}/values.yaml")] + + set { + name = "installCRDs" + value = "true" + } + + depends_on = [kubernetes_namespace.cert_manager] +} + +resource "kubernetes_manifest" "letsencrypt_issuer" { + manifest = yamldecode(file("${path.module}/letsencrypt-clusterissuer.yaml")) + depends_on = [helm_release.cert_manager] +} +``` + +- [ ] **Step 4: Commit** + +```bash +git add eks-gitops-platform/modules/cert_manager/ +git commit -m "refactor: cert_manager module — add variables.tf and outputs.tf" +``` + +--- + +## Task 6: istio Module — Add variables.tf and outputs.tf + +**Files:** +- Create: `eks-gitops-platform/modules/istio/variables.tf` +- Create: `eks-gitops-platform/modules/istio/outputs.tf` +- Modify: `eks-gitops-platform/modules/istio/main.tf` + +- [ ] **Step 1: Create `modules/istio/variables.tf`** + +```hcl +variable "namespace" { + description = "Namespace for Istio control plane" + type = string + default = "istio-system" +} + +variable "chart_version" { + description = "Istio Helm chart version (applies to base, istiod, and gateway)" + type = string + default = "1.21.0" +} +``` + +- [ ] **Step 2: Create `modules/istio/outputs.tf`** + +```hcl +output "namespace" { + description = "Namespace where Istio is deployed" + value = kubernetes_namespace.istio_system.metadata[0].name +} + +output "ingress_gateway_service" { + description = "Istio ingress gateway service name" + value = "istio-ingressgateway" +} +``` + +- [ ] **Step 3: Update `modules/istio/main.tf` to use variables** + +Replace the existing content with: + +```hcl +resource "kubernetes_namespace" "istio_system" { + metadata { + name = var.namespace + } +} + +resource "helm_release" "istio_base" { + name = "istio-base" + repository = "https://istio-release.storage.googleapis.com/charts" + chart = "base" + version = var.chart_version + namespace = kubernetes_namespace.istio_system.metadata[0].name + + depends_on = [kubernetes_namespace.istio_system] +} + +resource "helm_release" "istiod" { + name = "istiod" + repository = "https://istio-release.storage.googleapis.com/charts" + chart = "istiod" + version = var.chart_version + namespace = kubernetes_namespace.istio_system.metadata[0].name + + values = [file("${path.module}/istiod-values.yaml")] + + depends_on = [helm_release.istio_base] +} + +resource "helm_release" "istio_ingress" { + name = "istio-ingressgateway" + repository = "https://istio-release.storage.googleapis.com/charts" + chart = "gateway" + version = var.chart_version + namespace = kubernetes_namespace.istio_system.metadata[0].name + + values = [file("${path.module}/ingress-values.yaml")] + + depends_on = [helm_release.istiod] +} +``` + +- [ ] **Step 4: Commit** + +```bash +git add eks-gitops-platform/modules/istio/ +git commit -m "refactor: istio module — add variables.tf and outputs.tf" +``` + +--- + +## Task 7: postgres Module — Add variables.tf and outputs.tf + +**Files:** +- Create: `eks-gitops-platform/modules/postgres/variables.tf` +- Create: `eks-gitops-platform/modules/postgres/outputs.tf` +- Modify: `eks-gitops-platform/modules/postgres/main.tf` + +- [ ] **Step 1: Create `modules/postgres/variables.tf`** + +```hcl +variable "namespace" { + description = "Namespace for PostgreSQL" + type = string + default = "app-namespace" +} + +variable "chart_version" { + description = "PostgreSQL Bitnami Helm chart version" + type = string + default = "12.5.2" +} +``` + +- [ ] **Step 2: Create `modules/postgres/outputs.tf`** + +```hcl +output "namespace" { + description = "Namespace where PostgreSQL is deployed" + value = kubernetes_namespace.app.metadata[0].name +} + +output "service_name" { + description = "PostgreSQL service name" + value = "postgresql" +} +``` + +- [ ] **Step 3: Update `modules/postgres/main.tf` to use variables** + +Replace the existing content with: + +```hcl +resource "kubernetes_namespace" "app" { + metadata { + name = var.namespace + } +} + +resource "helm_release" "postgres" { + name = "postgresql" + repository = "https://charts.bitnami.com/bitnami" + chart = "postgresql" + version = var.chart_version + namespace = kubernetes_namespace.app.metadata[0].name + + values = [file("${path.module}/values.yaml")] + + depends_on = [kubernetes_namespace.app] +} +``` + +- [ ] **Step 4: Commit** + +```bash +git add eks-gitops-platform/modules/postgres/ +git commit -m "refactor: postgres module — add variables.tf and outputs.tf" +``` + +--- + +## Task 8: vault Module — Fix Broken Manifest Reference and Restructure + +**Files:** +- Rewrite: `eks-gitops-platform/modules/vault/main.tf` +- Create: `eks-gitops-platform/modules/vault/variables.tf` +- Create: `eks-gitops-platform/modules/vault/outputs.tf` + +The current `modules/vault/main.tf` loads `vault-k8s-auth.yaml` via `yamldecode(file(...))` but that file is at `manifests/vault/vault-k8s-auth.yaml` (wrong path) AND the YAML contains multiple documents which `yamldecode` cannot handle. Fix: replace `kubernetes_manifest` with native `kubernetes_service_account` and `kubernetes_cluster_role_binding` resources. + +Note: `vault/values.yaml` currently runs Vault in dev mode (`server.dev.enabled: true`). This is intentionally left as-is — production Vault HA config is a separate workstream. + +- [ ] **Step 1: Create `modules/vault/variables.tf`** + +```hcl +variable "namespace" { + description = "Namespace for HashiCorp Vault" + type = string + default = "vault" +} + +variable "chart_version" { + description = "HashiCorp Vault Helm chart version" + type = string + default = "0.27.0" +} +``` + +- [ ] **Step 2: Create `modules/vault/outputs.tf`** + +```hcl +output "namespace" { + description = "Namespace where Vault is deployed" + value = kubernetes_namespace.vault.metadata[0].name +} + +output "service_name" { + description = "Vault service name" + value = "vault" +} +``` + +- [ ] **Step 3: Rewrite `modules/vault/main.tf`** + +Replace the entire file with: + +```hcl +resource "kubernetes_namespace" "vault" { + metadata { + name = var.namespace + } +} + +resource "helm_release" "vault" { + name = "vault" + repository = "https://helm.releases.hashicorp.com" + chart = "vault" + version = var.chart_version + namespace = kubernetes_namespace.vault.metadata[0].name + + values = [file("${path.module}/values.yaml")] + + depends_on = [kubernetes_namespace.vault] +} + +resource "kubernetes_service_account" "vault_auth" { + metadata { + name = "vault-auth" + namespace = kubernetes_namespace.vault.metadata[0].name + } + + depends_on = [kubernetes_namespace.vault] +} + +resource "kubernetes_cluster_role_binding" "vault_auth" { + metadata { + name = "role-tokenreview-binding" + } + + role_ref { + api_group = "rbac.authorization.k8s.io" + kind = "ClusterRole" + name = "system:auth-delegator" + } + + subject { + kind = "ServiceAccount" + name = kubernetes_service_account.vault_auth.metadata[0].name + namespace = kubernetes_namespace.vault.metadata[0].name + } + + depends_on = [helm_release.vault] +} +``` + +- [ ] **Step 4: Commit** + +```bash +git add eks-gitops-platform/modules/vault/ +git commit -m "fix: vault module — replace broken kubernetes_manifest with native k8s resources; add variables.tf + outputs.tf" +``` + +--- + +## Task 9: flux Module — Add GitRepository + Kustomization Wiring + +**Files:** +- Rewrite: `eks-gitops-platform/modules/flux/main.tf` +- Create: `eks-gitops-platform/modules/flux/variables.tf` +- Create: `eks-gitops-platform/modules/flux/outputs.tf` + +- [ ] **Step 1: Create `modules/flux/variables.tf`** + +```hcl +variable "namespace" { + description = "Namespace for Flux system components" + type = string + default = "flux-system" +} + +variable "chart_version" { + description = "fluxcd-community/flux2 Helm chart version" + type = string + default = "2.10.0" +} + +variable "github_repo_url" { + description = "GitHub repository URL for Flux to sync (e.g. https://github.com/yourname/eks-gitops-platform)" + type = string +} + +variable "github_branch" { + description = "Git branch Flux watches for changes" + type = string + default = "main" +} + +variable "flux_sync_path" { + description = "Path within the repo that Flux syncs to the cluster" + type = string + default = "./flux-kustomize" +} +``` + +- [ ] **Step 2: Create `modules/flux/outputs.tf`** + +```hcl +output "namespace" { + description = "Namespace where Flux is deployed" + value = kubernetes_namespace.flux_system.metadata[0].name +} + +output "git_repository_name" { + description = "Name of the Flux GitRepository resource" + value = "eks-gitops-platform" +} +``` + +- [ ] **Step 3: Rewrite `modules/flux/main.tf`** + +Replace the entire file with: + +```hcl +resource "kubernetes_namespace" "flux_system" { + metadata { + name = var.namespace + } +} + +resource "helm_release" "flux" { + name = "flux2" + namespace = kubernetes_namespace.flux_system.metadata[0].name + repository = "https://fluxcd-community.github.io/helm-charts" + chart = "flux2" + version = var.chart_version + + values = [file("${path.module}/values.yaml")] + + depends_on = [kubernetes_namespace.flux_system] +} + +resource "kubernetes_manifest" "git_repository" { + manifest = { + apiVersion = "source.toolkit.fluxcd.io/v1" + kind = "GitRepository" + metadata = { + name = "eks-gitops-platform" + namespace = var.namespace + } + spec = { + interval = "1m" + url = var.github_repo_url + ref = { + branch = var.github_branch + } + } + } + + depends_on = [helm_release.flux] +} + +resource "kubernetes_manifest" "kustomization" { + manifest = { + apiVersion = "kustomize.toolkit.fluxcd.io/v1" + kind = "Kustomization" + metadata = { + name = "eks-gitops-platform" + namespace = var.namespace + } + spec = { + interval = "5m" + path = var.flux_sync_path + prune = true + sourceRef = { + kind = "GitRepository" + name = "eks-gitops-platform" + } + } + } + + depends_on = [kubernetes_manifest.git_repository] +} + +# For private repos: uncomment and populate a GitHub PAT in a Kubernetes Secret. +# resource "kubernetes_secret" "flux_git_auth" { +# metadata { +# name = "flux-git-auth" +# namespace = var.namespace +# } +# data = { +# username = "git" +# password = var.github_pat # add var.github_pat to variables.tf +# } +# depends_on = [kubernetes_namespace.flux_system] +# } +``` + +- [ ] **Step 4: Commit** + +```bash +git add eks-gitops-platform/modules/flux/ +git commit -m "feat: flux module — wire GitRepository + Kustomization; add variables.tf + outputs.tf" +``` + +--- + +## Task 10: Create Datadog Module — Full Stack with IRSA + +**Files:** +- Create: `eks-gitops-platform/modules/datadog/main.tf` +- Create: `eks-gitops-platform/modules/datadog/variables.tf` +- Create: `eks-gitops-platform/modules/datadog/outputs.tf` +- Create: `eks-gitops-platform/modules/datadog/values.yaml` + +- [ ] **Step 1: Create `modules/datadog/variables.tf`** + +```hcl +variable "namespace" { + description = "Namespace for Datadog agent" + type = string + default = "datadog" +} + +variable "chart_version" { + description = "Datadog Helm chart version" + type = string + default = "3.67.0" +} + +variable "datadog_api_key" { + description = "Datadog API key" + type = string + sensitive = true +} + +variable "datadog_app_key" { + description = "Datadog application key" + type = string + sensitive = true +} + +variable "datadog_site" { + description = "Datadog intake site (datadoghq.com or datadoghq.eu)" + type = string + default = "datadoghq.com" +} + +variable "cluster_name" { + description = "EKS cluster name — used to tag all Datadog telemetry" + type = string +} + +variable "enable_npm" { + description = "Enable Datadog Network Performance Monitoring" + type = bool + default = false +} + +variable "oidc_provider_arn" { + description = "ARN of the EKS OIDC provider (from module.eks.oidc_provider_arn)" + type = string +} + +variable "oidc_provider_url" { + description = "URL of the EKS OIDC provider (from module.eks.cluster_oidc_issuer_url)" + type = string +} +``` + +- [ ] **Step 2: Create `modules/datadog/outputs.tf`** + +```hcl +output "namespace" { + description = "Namespace where Datadog is deployed" + value = kubernetes_namespace.datadog.metadata[0].name +} + +output "irsa_role_arn" { + description = "IAM role ARN assigned to the Datadog agent via IRSA" + value = aws_iam_role.datadog_agent.arn +} +``` + +- [ ] **Step 3: Create `modules/datadog/values.yaml`** + +```yaml +datadog: + logs: + enabled: true + containerCollectAll: true + apm: + portEnabled: true + kubeStateMetricsEnabled: true + kubeStateMetricsCore: + enabled: true + +clusterAgent: + enabled: true + replicas: 2 + +datadog-crds: + crds: + datadogMetrics: true +``` + +- [ ] **Step 4: Create `modules/datadog/main.tf`** + +```hcl +resource "kubernetes_namespace" "datadog" { + metadata { + name = var.namespace + } +} + +data "aws_iam_policy_document" "datadog_assume_role" { + statement { + effect = "Allow" + actions = ["sts:AssumeRoleWithWebIdentity"] + + principals { + type = "Federated" + identifiers = [var.oidc_provider_arn] + } + + condition { + test = "StringEquals" + variable = "${trimprefix(var.oidc_provider_url, "https://")}:sub" + values = ["system:serviceaccount:${var.namespace}:datadog-agent"] + } + + condition { + test = "StringEquals" + variable = "${trimprefix(var.oidc_provider_url, "https://")}:aud" + values = ["sts.amazonaws.com"] + } + } +} + +resource "aws_iam_role" "datadog_agent" { + name = "datadog-agent-irsa" + assume_role_policy = data.aws_iam_policy_document.datadog_assume_role.json +} + +resource "aws_iam_role_policy_attachment" "datadog_ec2_readonly" { + role = aws_iam_role.datadog_agent.name + policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess" +} + +resource "helm_release" "datadog" { + name = "datadog" + repository = "https://helm.datadoghq.com" + chart = "datadog" + version = var.chart_version + namespace = kubernetes_namespace.datadog.metadata[0].name + + values = [file("${path.module}/values.yaml")] + + set_sensitive { + name = "datadog.apiKey" + value = var.datadog_api_key + } + + set_sensitive { + name = "datadog.appKey" + value = var.datadog_app_key + } + + set { + name = "datadog.site" + value = var.datadog_site + } + + set { + name = "datadog.clusterName" + value = var.cluster_name + } + + set { + name = "datadog.networkMonitoring.enabled" + value = tostring(var.enable_npm) + } + + set { + name = "agents.serviceAccount.annotations.eks\\.amazonaws\\.com/role-arn" + value = aws_iam_role.datadog_agent.arn + } + + depends_on = [kubernetes_namespace.datadog] +} +``` + +- [ ] **Step 5: Validate root with backend skipped** + +```bash +cd eks-gitops-platform/eks-gitops-platform +terraform init -backend=false -input=false +terraform validate +``` + +Expected: `Success! The configuration is valid.` + +- [ ] **Step 6: Commit** + +```bash +git add eks-gitops-platform/modules/datadog/ +git commit -m "feat: add Datadog module — APM, logs, cluster metrics, IRSA" +``` + +--- + +## Task 11: Remove Komodor Module + +**Files:** +- Delete: `eks-gitops-platform/modules/komodor/` (entire directory) + +- [ ] **Step 1: Delete the komodor module directory** + +```bash +rm -rf eks-gitops-platform/eks-gitops-platform/modules/komodor +``` + +- [ ] **Step 2: Verify it's gone** + +```bash +ls eks-gitops-platform/eks-gitops-platform/modules/ +``` + +Expected: `cert_manager datadog flux istio komodor` is NOT in the list. + +- [ ] **Step 3: Commit** + +```bash +git add -A +git commit -m "chore: remove komodor module — replaced by Datadog" +``` + +--- + +## Task 12: Rewrite bootstrap.sh Script + +**Files:** +- Rewrite: `eks-gitops-platform/scripts/bootstrap.sh` + +- [ ] **Step 1: Overwrite `scripts/bootstrap.sh`** + +```bash +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +TF_ROOT="$(dirname "$SCRIPT_DIR")" +BOOTSTRAP_DIR="$TF_ROOT/bootstrap" + +echo "==> Initializing bootstrap module (S3 + DynamoDB)..." + +cd "$BOOTSTRAP_DIR" +terraform init -input=false +terraform apply -auto-approve -input=false + +BUCKET=$(terraform output -raw state_bucket_name) +TABLE=$(terraform output -raw lock_table_name) +REGION=$(terraform output -raw aws_region) + +echo "==> Writing $TF_ROOT/backend.tf..." + +cat > "$TF_ROOT/backend.tf" < EKS GitOps Platform — One-Shot Deploy" +echo "" + +# Step 1: Bootstrap remote state if not already done +BOOTSTRAP_OUTPUT="$TF_ROOT/bootstrap/.terraform" +if [ ! -d "$BOOTSTRAP_OUTPUT" ]; then + echo "==> Running bootstrap (first time only)..." + bash "$SCRIPT_DIR/bootstrap.sh" +else + echo "==> Bootstrap already initialized. Checking outputs..." + cd "$TF_ROOT/bootstrap" + if ! terraform output -raw state_bucket_name &>/dev/null 2>&1; then + echo "==> Bootstrap state missing — re-running bootstrap..." + bash "$SCRIPT_DIR/bootstrap.sh" + else + echo "==> Bootstrap already complete, skipping." + fi +fi + +# Step 2: Initialize root module +cd "$TF_ROOT" +echo "" +echo "==> Initializing Terraform root module..." +terraform init -input=false + +# Step 3: Apply everything +echo "" +echo "==> Applying platform configuration..." +terraform apply -input=false + +# Step 4: Show cluster status +echo "" +echo "==> Platform status:" +aws eks update-kubeconfig --name "$(terraform output -raw cluster_name)" --region "$(terraform output -raw aws_region 2>/dev/null || echo us-west-2)" 2>/dev/null || true +kubectl get pods -A 2>/dev/null || echo " (kubectl not configured yet — run: aws eks update-kubeconfig --name )" +kubectl get kustomizations -A 2>/dev/null || true +kubectl get gitrepositories -A 2>/dev/null || true + +echo "" +echo "✅ One-shot deployment complete!" +``` + +- [ ] **Step 2: Make executable** + +```bash +chmod +x eks-gitops-platform/eks-gitops-platform/scripts/one-shot-deploy.sh +``` + +- [ ] **Step 3: Commit** + +```bash +git add eks-gitops-platform/scripts/one-shot-deploy.sh +git commit -m "fix: rewrite one-shot-deploy.sh — correct paths, single root apply, idempotent bootstrap" +``` + +--- + +## Task 14: GitHub Actions — terraform-plan.yml + +**Files:** +- Create: `eks-gitops-platform/.github/workflows/terraform-plan.yml` + +- [ ] **Step 1: Create the `.github/workflows/` directory and `terraform-plan.yml`** + +```bash +mkdir -p eks-gitops-platform/.github/workflows +``` + +Create `.github/workflows/terraform-plan.yml`: + +```yaml +name: Terraform Plan + +on: + pull_request: + branches: + - main + +permissions: + contents: read + pull-requests: write + +concurrency: + group: terraform-plan-${{ github.ref }} + cancel-in-progress: true + +jobs: + plan: + name: Terraform Plan + runs-on: ubuntu-latest + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Setup Terraform + uses: hashicorp/setup-terraform@v3 + with: + terraform_version: "~1.7" + + - name: Configure AWS credentials + uses: aws-actions/configure-aws-credentials@v4 + with: + aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} + aws-region: us-west-2 + + - name: Terraform Format Check + id: fmt + run: terraform fmt -check -recursive + working-directory: eks-gitops-platform + + - name: Terraform Init + id: init + run: terraform init -input=false + working-directory: eks-gitops-platform + + - name: Terraform Validate + id: validate + run: terraform validate + working-directory: eks-gitops-platform + + - name: Run tfsec + uses: aquasecurity/tfsec-action@v1.0.3 + with: + working_directory: eks-gitops-platform + minimum_severity: HIGH + soft_fail: false + + - name: Terraform Plan + id: plan + run: | + terraform plan -no-color -input=false \ + -var="datadog_api_key=${{ secrets.DATADOG_API_KEY }}" \ + -var="datadog_app_key=${{ secrets.DATADOG_APP_KEY }}" \ + -var="github_repo_url=${{ secrets.TF_VAR_GITHUB_REPO_URL }}" \ + 2>&1 | tee /tmp/plan.txt + echo "exit_code=${PIPESTATUS[0]}" >> "$GITHUB_OUTPUT" + working-directory: eks-gitops-platform + + - name: Post Plan as PR Comment + uses: actions/github-script@v7 + if: always() + env: + PLAN_PATH: /tmp/plan.txt + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + const fs = require('fs'); + const plan = fs.readFileSync(process.env.PLAN_PATH, 'utf8'); + const truncated = plan.length > 60000 + ? plan.substring(0, 60000) + '\n\n...[truncated — see Actions logs for full output]' + : plan; + const header = `## Terraform Plan — ${{ github.event.pull_request.head.sha }}`; + const body = `${header}\n\n\`\`\`\n${truncated}\n\`\`\``; + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body, + }); +``` + +- [ ] **Step 2: Commit** + +```bash +git add eks-gitops-platform/.github/workflows/terraform-plan.yml +git commit -m "feat: add GitHub Actions terraform-plan workflow (PR: fmt + validate + tfsec + plan comment)" +``` + +--- + +## Task 15: GitHub Actions — terraform-apply.yml + +**Files:** +- Create: `eks-gitops-platform/.github/workflows/terraform-apply.yml` + +- [ ] **Step 1: Create `.github/workflows/terraform-apply.yml`** + +```yaml +name: Terraform Apply + +on: + push: + branches: + - main + +concurrency: + group: terraform-apply + cancel-in-progress: false + +jobs: + apply: + name: Terraform Apply + runs-on: ubuntu-latest + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Setup Terraform + uses: hashicorp/setup-terraform@v3 + with: + terraform_version: "~1.7" + + - name: Configure AWS credentials + uses: aws-actions/configure-aws-credentials@v4 + with: + aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} + aws-region: us-west-2 + + - name: Terraform Init + run: terraform init -input=false + working-directory: eks-gitops-platform + + - name: Terraform Plan + run: | + terraform plan -no-color -input=false \ + -var="datadog_api_key=${{ secrets.DATADOG_API_KEY }}" \ + -var="datadog_app_key=${{ secrets.DATADOG_APP_KEY }}" \ + -var="github_repo_url=${{ secrets.TF_VAR_GITHUB_REPO_URL }}" + working-directory: eks-gitops-platform + + - name: Terraform Apply + run: | + terraform apply -auto-approve -input=false \ + -var="datadog_api_key=${{ secrets.DATADOG_API_KEY }}" \ + -var="datadog_app_key=${{ secrets.DATADOG_APP_KEY }}" \ + -var="github_repo_url=${{ secrets.TF_VAR_GITHUB_REPO_URL }}" + working-directory: eks-gitops-platform +``` + +- [ ] **Step 2: Commit** + +```bash +git add eks-gitops-platform/.github/workflows/terraform-apply.yml +git commit -m "feat: add GitHub Actions terraform-apply workflow (merge to main: plan + auto-apply)" +``` + +--- + +## Task 16: terraform.tfvars.example, .gitignore, and README + +**Files:** +- Create: `eks-gitops-platform/terraform.tfvars.example` +- Modify: `eks-gitops-platform/.gitignore` +- Rewrite: `eks-gitops-platform/README.md` + +- [ ] **Step 1: Create `terraform.tfvars.example`** + +```hcl +# Copy this file to terraform.tfvars and fill in your values. +# IMPORTANT: Never commit terraform.tfvars — it is gitignored. + +# ── AWS ───────────────────────────────────────────────────────────────────── +aws_region = "us-west-2" +cluster_name = "eks-gitops-cluster" +cluster_version = "1.30" +environment = "prod" + +# ── Networking ─────────────────────────────────────────────────────────────── +vpc_cidr = "10.0.0.0/16" +private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"] +public_subnets = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"] + +# ── Datadog (sensitive — store in GitHub Secrets for CI) ───────────────────── +# Get these from: https://app.datadoghq.com/organization-settings/api-keys +datadog_api_key = "your-datadog-api-key" +datadog_app_key = "your-datadog-app-key" +datadog_site = "datadoghq.com" # datadoghq.eu for EU region +enable_npm = false # set true to enable Network Performance Monitoring + +# ── Flux GitOps ────────────────────────────────────────────────────────────── +# Set this to YOUR fork of the repo — Flux will sync ./flux-kustomize from it +github_repo_url = "https://github.com/YOUR_USERNAME/eks-gitops-platform" +github_branch = "main" +flux_sync_path = "./flux-kustomize" +``` + +- [ ] **Step 2: Update `.gitignore`** + +Append to the existing `.gitignore` file at `eks-gitops-platform/eks-gitops-platform/.gitignore`: + +``` +# Existing +.terraform/ + +# Added +terraform.tfvars +*.tfstate +*.tfstate.backup +.terraform.lock.hcl +/backend.tf +``` + +- [ ] **Step 3: Rewrite `README.md`** + +Replace the contents of `eks-gitops-platform/README.md` with: + +```markdown +# EKS GitOps Platform + +Production-grade Kubernetes platform on AWS. Provisions EKS with Flux GitOps, Istio service mesh, cert-manager TLS, HashiCorp Vault, PostgreSQL, and Datadog full-stack observability — all via Terraform. + +--- + +## Platform Components + +| Component | Purpose | Namespace | +|---|---|---| +| EKS 1.30 | Managed Kubernetes cluster | — | +| Flux v2 | GitOps — syncs this repo to the cluster | `flux-system` | +| Istio 1.21 | Service mesh + ingress gateway | `istio-system` | +| cert-manager | Automatic TLS via Let's Encrypt | `cert-manager` | +| HashiCorp Vault | Secrets management | `vault` | +| PostgreSQL | Relational database | `app-namespace` | +| Datadog | Metrics + APM + logs + k8s state | `datadog` | + +--- + +## Quickstart — Fork and Deploy + +### 1. Prerequisites + +- AWS CLI configured (`aws configure` or `AWS_PROFILE`) +- Terraform ≥ 1.7 (`brew install terraform`) +- kubectl (`brew install kubectl`) +- A Datadog account (free trial works) + +### 2. Fork and clone + +```bash +# Fork https://github.com/daringanitch/eks-gitops-platform on GitHub, then: +git clone https://github.com/YOUR_USERNAME/eks-gitops-platform.git +cd eks-gitops-platform/eks-gitops-platform +``` + +### 3. Configure your variables + +```bash +cp terraform.tfvars.example terraform.tfvars +# Edit terraform.tfvars — fill in datadog_api_key, datadog_app_key, and your github_repo_url +``` + +### 4. Bootstrap remote state (first time only) + +```bash +bash scripts/bootstrap.sh +``` + +This creates an S3 bucket + DynamoDB table and writes `backend.tf` automatically. + +### 5. Deploy everything + +```bash +terraform init +terraform apply +``` + +Or use the one-shot helper: + +```bash +bash scripts/one-shot-deploy.sh +``` + +--- + +## GitHub Actions CI/CD + +On every **pull request** → `terraform fmt` + `terraform validate` + `tfsec` + plan posted as PR comment. + +On every **merge to main** → `terraform plan` + `terraform apply` (auto). + +### Required GitHub Secrets + +Go to `Settings → Secrets → Actions` in your fork and add: + +| Secret | Value | +|---|---| +| `AWS_ACCESS_KEY_ID` | IAM access key with EKS/VPC/IAM permissions | +| `AWS_SECRET_ACCESS_KEY` | Corresponding secret key | +| `DATADOG_API_KEY` | From Datadog → Organization Settings → API Keys | +| `DATADOG_APP_KEY` | From Datadog → Organization Settings → Application Keys | +| `TF_VAR_GITHUB_REPO_URL` | Your fork URL: `https://github.com/YOUR_USERNAME/eks-gitops-platform` | + +--- + +## Architecture + +``` +GitHub (your fork) + │ + │ Flux polls every 1 min + ▼ + flux-system (Flux v2) + │ syncs ./flux-kustomize/ + ▼ + app-namespace (test-app) + + istio-system (Istio + ingress gateway) + cert-manager (Let's Encrypt TLS) + vault (HashiCorp Vault) + app-namespace (PostgreSQL) + datadog (Agent + Cluster Agent + APM) +``` + +--- + +## Repository Structure + +``` +eks-gitops-platform/ ← Terraform root +├── bootstrap/ ← Run once: provisions S3 + DynamoDB +├── modules/ +│ ├── cert_manager/ +│ ├── datadog/ ← Replaces Komodor +│ ├── flux/ ← Includes GitRepository + Kustomization +│ ├── istio/ +│ ├── postgres/ +│ └── vault/ +├── flux-kustomize/test-app/ ← Deployed by Flux as smoke test +├── manifests/ ← Reference manifests +├── scripts/ +│ ├── bootstrap.sh +│ └── one-shot-deploy.sh +├── .github/workflows/ +│ ├── terraform-plan.yml +│ └── terraform-apply.yml +├── main.tf +├── variables.tf +├── outputs.tf +├── providers.tf +├── backend.tf ← Generated by bootstrap.sh +└── terraform.tfvars.example +``` +``` + +- [ ] **Step 4: Commit everything** + +```bash +git add eks-gitops-platform/terraform.tfvars.example \ + eks-gitops-platform/.gitignore \ + eks-gitops-platform/README.md +git commit -m "docs: add terraform.tfvars.example, update .gitignore and README with full onboarding guide" +``` + +--- + +## Self-Review Checklist + +- [x] Bootstrap module (S3 + DynamoDB) — Task 1 +- [x] providers.tf + backend.tf — Task 2 +- [x] variables.tf single source of truth — Task 3 +- [x] main.tf: fixed EKS args, wired all modules, removed duplicates — Task 4 +- [x] cert_manager module: variables.tf + outputs.tf — Task 5 +- [x] istio module: variables.tf + outputs.tf — Task 6 +- [x] postgres module: variables.tf + outputs.tf — Task 7 +- [x] vault module: fixed manifest path, native k8s resources — Task 8 +- [x] flux module: GitRepository + Kustomization wired — Task 9 +- [x] Datadog module: full stack + IRSA — Task 10 +- [x] Komodor removed — Task 11 +- [x] bootstrap.sh rewritten — Task 12 +- [x] one-shot-deploy.sh rewritten — Task 13 +- [x] terraform-plan.yml — Task 14 +- [x] terraform-apply.yml — Task 15 +- [x] terraform.tfvars.example + .gitignore + README — Task 16 From e3a6eb0657ff915a25e405c7085c0f9c15babf7a Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Sun, 17 May 2026 20:05:42 -0700 Subject: [PATCH 03/23] chore: add .gitignore with .worktrees/ and .DS_Store --- .gitignore | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 .gitignore diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..ea85cdc --- /dev/null +++ b/.gitignore @@ -0,0 +1,2 @@ +.worktrees/ +.DS_Store From 58383be1dffd766a22174d74bd1c19726b0553ef Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Sun, 17 May 2026 20:14:05 -0700 Subject: [PATCH 04/23] feat: add bootstrap module for S3 + DynamoDB remote state --- .../bootstrap/.terraform.lock.hcl | 25 +++++++++ eks-gitops-platform/bootstrap/main.tf | 56 +++++++++++++++++++ eks-gitops-platform/bootstrap/outputs.tf | 14 +++++ eks-gitops-platform/bootstrap/variables.tf | 17 ++++++ 4 files changed, 112 insertions(+) create mode 100644 eks-gitops-platform/bootstrap/.terraform.lock.hcl create mode 100644 eks-gitops-platform/bootstrap/main.tf create mode 100644 eks-gitops-platform/bootstrap/outputs.tf create mode 100644 eks-gitops-platform/bootstrap/variables.tf diff --git a/eks-gitops-platform/bootstrap/.terraform.lock.hcl b/eks-gitops-platform/bootstrap/.terraform.lock.hcl new file mode 100644 index 0000000..cdc1668 --- /dev/null +++ b/eks-gitops-platform/bootstrap/.terraform.lock.hcl @@ -0,0 +1,25 @@ +# This file is maintained automatically by "terraform init". +# Manual edits may be lost in future updates. + +provider "registry.terraform.io/hashicorp/aws" { + version = "5.100.0" + constraints = "~> 5.0" + hashes = [ + "h1:Ijt7pOlB7Tr7maGQIqtsLFbl7pSMIj06TVdkoSBcYOw=", + "zh:054b8dd49f0549c9a7cc27d159e45327b7b65cf404da5e5a20da154b90b8a644", + "zh:0b97bf8d5e03d15d83cc40b0530a1f84b459354939ba6f135a0086c20ebbe6b2", + "zh:1589a2266af699cbd5d80737a0fe02e54ec9cf2ca54e7e00ac51c7359056f274", + "zh:6330766f1d85f01ae6ea90d1b214b8b74cc8c1badc4696b165b36ddd4cc15f7b", + "zh:7c8c2e30d8e55291b86fcb64bdf6c25489d538688545eb48fd74ad622e5d3862", + "zh:99b1003bd9bd32ee323544da897148f46a527f622dc3971af63ea3e251596342", + "zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425", + "zh:9f8b909d3ec50ade83c8062290378b1ec553edef6a447c56dadc01a99f4eaa93", + "zh:aaef921ff9aabaf8b1869a86d692ebd24fbd4e12c21205034bb679b9caf883a2", + "zh:ac882313207aba00dd5a76dbd572a0ddc818bb9cbf5c9d61b28fe30efaec951e", + "zh:bb64e8aff37becab373a1a0cc1080990785304141af42ed6aa3dd4913b000421", + "zh:dfe495f6621df5540d9c92ad40b8067376350b005c637ea6efac5dc15028add4", + "zh:f0ddf0eaf052766cfe09dea8200a946519f653c384ab4336e2a4a64fdd6310e9", + "zh:f1b7e684f4c7ae1eed272b6de7d2049bb87a0275cb04dbb7cda6636f600699c9", + "zh:ff461571e3f233699bf690db319dfe46aec75e58726636a0d97dd9ac6e32fb70", + ] +} diff --git a/eks-gitops-platform/bootstrap/main.tf b/eks-gitops-platform/bootstrap/main.tf new file mode 100644 index 0000000..17d7a83 --- /dev/null +++ b/eks-gitops-platform/bootstrap/main.tf @@ -0,0 +1,56 @@ +terraform { + required_providers { + aws = { + source = "hashicorp/aws" + version = "~> 5.0" + } + } + required_version = ">= 1.7" +} + +provider "aws" { + region = var.aws_region +} + +resource "aws_s3_bucket" "tfstate" { + bucket = var.state_bucket_name + + lifecycle { + prevent_destroy = true + } +} + +resource "aws_s3_bucket_versioning" "tfstate" { + bucket = aws_s3_bucket.tfstate.id + versioning_configuration { + status = "Enabled" + } +} + +resource "aws_s3_bucket_server_side_encryption_configuration" "tfstate" { + bucket = aws_s3_bucket.tfstate.id + rule { + apply_server_side_encryption_by_default { + sse_algorithm = "AES256" + } + } +} + +resource "aws_s3_bucket_public_access_block" "tfstate" { + bucket = aws_s3_bucket.tfstate.id + block_public_acls = true + block_public_policy = true + ignore_public_acls = true + restrict_public_buckets = true +} + +resource "aws_dynamodb_table" "tfstate_lock" { + name = var.lock_table_name + billing_mode = "PAY_PER_REQUEST" + hash_key = "LockID" + + attribute { + name = "LockID" + type = "S" + } +} diff --git a/eks-gitops-platform/bootstrap/outputs.tf b/eks-gitops-platform/bootstrap/outputs.tf new file mode 100644 index 0000000..babc5da --- /dev/null +++ b/eks-gitops-platform/bootstrap/outputs.tf @@ -0,0 +1,14 @@ +output "state_bucket_name" { + description = "S3 bucket name for Terraform state" + value = aws_s3_bucket.tfstate.bucket +} + +output "lock_table_name" { + description = "DynamoDB table name for state locking" + value = aws_dynamodb_table.tfstate_lock.name +} + +output "aws_region" { + description = "AWS region where state infrastructure was created" + value = var.aws_region +} diff --git a/eks-gitops-platform/bootstrap/variables.tf b/eks-gitops-platform/bootstrap/variables.tf new file mode 100644 index 0000000..cab585b --- /dev/null +++ b/eks-gitops-platform/bootstrap/variables.tf @@ -0,0 +1,17 @@ +variable "aws_region" { + description = "AWS region for state infrastructure" + type = string + default = "us-west-2" +} + +variable "state_bucket_name" { + description = "S3 bucket name for Terraform state — must be globally unique" + type = string + default = "eks-gitops-tfstate" +} + +variable "lock_table_name" { + description = "DynamoDB table name for state locking" + type = string + default = "eks-gitops-tfstate-lock" +} From e9550dbe2609b02ed00f336ddccab071074b14c1 Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 05:22:22 -0700 Subject: [PATCH 05/23] =?UTF-8?q?fix:=20bootstrap=20module=20=E2=80=94=20a?= =?UTF-8?q?dd=20S3=20sub-resource=20ordering=20and=20DynamoDB=20prevent=5F?= =?UTF-8?q?destroy?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- eks-gitops-platform/bootstrap/main.tf | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/eks-gitops-platform/bootstrap/main.tf b/eks-gitops-platform/bootstrap/main.tf index 17d7a83..b061934 100644 --- a/eks-gitops-platform/bootstrap/main.tf +++ b/eks-gitops-platform/bootstrap/main.tf @@ -25,6 +25,8 @@ resource "aws_s3_bucket_versioning" "tfstate" { versioning_configuration { status = "Enabled" } + + depends_on = [aws_s3_bucket_public_access_block.tfstate] } resource "aws_s3_bucket_server_side_encryption_configuration" "tfstate" { @@ -34,6 +36,8 @@ resource "aws_s3_bucket_server_side_encryption_configuration" "tfstate" { sse_algorithm = "AES256" } } + + depends_on = [aws_s3_bucket_public_access_block.tfstate] } resource "aws_s3_bucket_public_access_block" "tfstate" { @@ -53,4 +57,8 @@ resource "aws_dynamodb_table" "tfstate_lock" { name = "LockID" type = "S" } + + lifecycle { + prevent_destroy = true + } } From 5edb77c585dcfe31b6ea6571fd5797663caae3b3 Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 05:35:53 -0700 Subject: [PATCH 06/23] feat: add providers.tf with version constraints and Helm/K8s config --- eks-gitops-platform/backend.tf | 11 ++++++++ eks-gitops-platform/main.tf | 4 --- eks-gitops-platform/providers.tf | 44 ++++++++++++++++++++++++++++++++ 3 files changed, 55 insertions(+), 4 deletions(-) create mode 100644 eks-gitops-platform/backend.tf create mode 100644 eks-gitops-platform/providers.tf diff --git a/eks-gitops-platform/backend.tf b/eks-gitops-platform/backend.tf new file mode 100644 index 0000000..6967dd8 --- /dev/null +++ b/eks-gitops-platform/backend.tf @@ -0,0 +1,11 @@ +# Generated by scripts/bootstrap.sh — do not edit manually. +# Run scripts/bootstrap.sh to populate the bucket and table names. +terraform { + backend "s3" { + bucket = "REPLACE_WITH_BUCKET_NAME" + key = "eks-gitops-platform/terraform.tfstate" + region = "us-west-2" + dynamodb_table = "REPLACE_WITH_TABLE_NAME" + encrypt = true + } +} diff --git a/eks-gitops-platform/main.tf b/eks-gitops-platform/main.tf index e8d4387..848245c 100644 --- a/eks-gitops-platform/main.tf +++ b/eks-gitops-platform/main.tf @@ -1,7 +1,3 @@ -provider "aws" { - region = var.aws_region -} - module "vpc" { source = "terraform-aws-modules/vpc/aws" version = "4.0.2" diff --git a/eks-gitops-platform/providers.tf b/eks-gitops-platform/providers.tf new file mode 100644 index 0000000..ad7da51 --- /dev/null +++ b/eks-gitops-platform/providers.tf @@ -0,0 +1,44 @@ +terraform { + required_version = ">= 1.7" + + required_providers { + aws = { + source = "hashicorp/aws" + version = "~> 5.0" + } + helm = { + source = "hashicorp/helm" + version = "~> 2.13" + } + kubernetes = { + source = "hashicorp/kubernetes" + version = "~> 2.29" + } + } +} + +provider "aws" { + region = var.aws_region +} + +provider "helm" { + kubernetes { + host = module.eks.cluster_endpoint + cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data) + exec { + api_version = "client.authentication.k8s.io/v1beta1" + command = "aws" + args = ["eks", "get-token", "--cluster-name", var.cluster_name, "--region", var.aws_region] + } + } +} + +provider "kubernetes" { + host = module.eks.cluster_endpoint + cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data) + exec { + api_version = "client.authentication.k8s.io/v1beta1" + command = "aws" + args = ["eks", "get-token", "--cluster-name", var.cluster_name, "--region", var.aws_region] + } +} From 00a41eafe277d96da365c466075876df84f619a9 Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 05:37:22 -0700 Subject: [PATCH 07/23] refactor: variables.tf is now single source of truth; add cluster_version, datadog, flux vars --- eks-gitops-platform/variables.tf | 57 ++++++++++++++++++++++++++++---- 1 file changed, 51 insertions(+), 6 deletions(-) diff --git a/eks-gitops-platform/variables.tf b/eks-gitops-platform/variables.tf index ab2f6e4..6f26aee 100644 --- a/eks-gitops-platform/variables.tf +++ b/eks-gitops-platform/variables.tf @@ -1,20 +1,25 @@ -# variables.tf variable "aws_region" { - description = "The AWS region to deploy EKS" + description = "AWS region to deploy all resources" type = string default = "us-west-2" } variable "cluster_name" { - description = "EKS Cluster name" + description = "EKS cluster name" type = string - default = "eks-cluster" + default = "eks-gitops-cluster" +} + +variable "cluster_version" { + description = "Kubernetes version for the EKS cluster" + type = string + default = "1.30" } variable "environment" { - description = "Environment tag (e.g., dev, prod)" + description = "Environment tag applied to all resources" type = string - default = "dev" + default = "prod" } variable "vpc_cidr" { @@ -35,3 +40,43 @@ variable "public_subnets" { default = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"] } +variable "datadog_api_key" { + description = "Datadog API key — store in GitHub Secrets as DATADOG_API_KEY" + type = string + sensitive = true +} + +variable "datadog_app_key" { + description = "Datadog application key — store in GitHub Secrets as DATADOG_APP_KEY" + type = string + sensitive = true +} + +variable "datadog_site" { + description = "Datadog intake site (datadoghq.com for US, datadoghq.eu for EU)" + type = string + default = "datadoghq.com" +} + +variable "enable_npm" { + description = "Enable Datadog Network Performance Monitoring (requires kernel headers on nodes)" + type = bool + default = false +} + +variable "github_repo_url" { + description = "GitHub repo URL for Flux to sync — e.g. https://github.com/yourname/eks-gitops-platform" + type = string +} + +variable "github_branch" { + description = "Git branch Flux watches for changes" + type = string + default = "main" +} + +variable "flux_sync_path" { + description = "Path within the repo that Flux syncs to the cluster" + type = string + default = "./flux-kustomize" +} From a4b369571a7c815598b4913ef616daddbc608a05 Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 05:37:34 -0700 Subject: [PATCH 08/23] fix: correct EKS arg names (subnet_ids, desired_size); wire all child modules; remove duplicate vars --- eks-gitops-platform/main.tf | 87 ++++++++++++++++++++++--------------- 1 file changed, 52 insertions(+), 35 deletions(-) diff --git a/eks-gitops-platform/main.tf b/eks-gitops-platform/main.tf index 848245c..9b7e757 100644 --- a/eks-gitops-platform/main.tf +++ b/eks-gitops-platform/main.tf @@ -1,33 +1,36 @@ +data "aws_availability_zones" "available" {} + module "vpc" { source = "terraform-aws-modules/vpc/aws" - version = "4.0.2" + version = "5.8.1" name = "eks-vpc" - cidr = "10.0.0.0/16" + cidr = var.vpc_cidr azs = slice(data.aws_availability_zones.available.names, 0, 3) - private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"] - public_subnets = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"] + private_subnets = var.private_subnets + public_subnets = var.public_subnets enable_nat_gateway = true single_nat_gateway = true tags = { - Terraform = "true" + Terraform = "true" Environment = var.environment } } module "eks" { - source = "terraform-aws-modules/eks/aws" - version = "20.8.4" + source = "terraform-aws-modules/eks/aws" + version = "20.8.4" - cluster_name = var.cluster_name - cluster_version = "1.29" - subnets = module.vpc.private_subnets - vpc_id = module.vpc.vpc_id + cluster_name = var.cluster_name + cluster_version = var.cluster_version + subnet_ids = module.vpc.private_subnets + vpc_id = module.vpc.vpc_id + cluster_endpoint_public_access = true - enable_irsa = true + enable_irsa = true eks_managed_node_group_defaults = { instance_types = ["t3.medium"] @@ -35,10 +38,9 @@ module "eks" { eks_managed_node_groups = { default = { - desired_capacity = 3 - max_capacity = 4 - min_capacity = 2 - + desired_size = 3 + max_size = 4 + min_size = 2 instance_types = ["t3.medium"] capacity_type = "ON_DEMAND" } @@ -50,35 +52,50 @@ module "eks" { } } -data "aws_availability_zones" "available" {} +module "cert_manager" { + source = "./modules/cert_manager" -output "cluster_name" { - value = module.eks.cluster_name + depends_on = [module.eks] } -output "cluster_endpoint" { - value = module.eks.cluster_endpoint -} +module "istio" { + source = "./modules/istio" -output "cluster_security_group_id" { - value = module.eks.cluster_security_group_id + depends_on = [module.eks] } -output "node_group_role_arn" { - value = module.eks.eks_managed_node_groups["default"].iam_role_arn +module "vault" { + source = "./modules/vault" + + depends_on = [module.eks] } -variable "aws_region" { - description = "AWS region" - default = "us-west-2" +module "postgres" { + source = "./modules/postgres" + + depends_on = [module.eks] } -variable "environment" { - description = "Environment tag" - default = "dev" +module "flux" { + source = "./modules/flux" + + github_repo_url = var.github_repo_url + github_branch = var.github_branch + flux_sync_path = var.flux_sync_path + + depends_on = [module.eks] } -variable "cluster_name" { - description = "EKS Cluster name" - default = "eks-cluster" +module "datadog" { + source = "./modules/datadog" + + datadog_api_key = var.datadog_api_key + datadog_app_key = var.datadog_app_key + datadog_site = var.datadog_site + cluster_name = var.cluster_name + enable_npm = var.enable_npm + oidc_provider_arn = module.eks.oidc_provider_arn + oidc_provider_url = module.eks.cluster_oidc_issuer_url + + depends_on = [module.eks] } From 89dc46c6d158db82f471860c2e6140cd207af80d Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 05:39:36 -0700 Subject: [PATCH 09/23] =?UTF-8?q?refactor:=20cert=5Fmanager=20module=20?= =?UTF-8?q?=E2=80=94=20add=20variables.tf=20and=20outputs.tf?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- eks-gitops-platform/modules/cert_manager/main.tf | 15 +++------------ .../modules/cert_manager/outputs.tf | 4 ++++ .../modules/cert_manager/variables.tf | 11 +++++++++++ 3 files changed, 18 insertions(+), 12 deletions(-) create mode 100644 eks-gitops-platform/modules/cert_manager/outputs.tf create mode 100644 eks-gitops-platform/modules/cert_manager/variables.tf diff --git a/eks-gitops-platform/modules/cert_manager/main.tf b/eks-gitops-platform/modules/cert_manager/main.tf index a3256ae..d90c93b 100644 --- a/eks-gitops-platform/modules/cert_manager/main.tf +++ b/eks-gitops-platform/modules/cert_manager/main.tf @@ -1,7 +1,6 @@ -# cert_manager/main.tf resource "kubernetes_namespace" "cert_manager" { metadata { - name = "cert-manager" + name = var.namespace labels = { "cert-manager.io/disable-validation" = "true" } @@ -12,7 +11,7 @@ resource "helm_release" "cert_manager" { name = "cert-manager" repository = "https://charts.jetstack.io" chart = "cert-manager" - version = "v1.14.3" + version = var.chart_version namespace = kubernetes_namespace.cert_manager.metadata[0].name values = [file("${path.module}/values.yaml")] @@ -26,14 +25,6 @@ resource "helm_release" "cert_manager" { } resource "kubernetes_manifest" "letsencrypt_issuer" { - manifest = yamldecode(file("${path.module}/letsencrypt-clusterissuer.yaml")) + manifest = yamldecode(file("${path.module}/letsencrypt-clusterissuer.yaml")) depends_on = [helm_release.cert_manager] } - -# cert_manager/outputs.tf -output "cert_manager_namespace" { - value = kubernetes_namespace.cert_manager.metadata[0].name -} - -# cert_manager/variables.tf -# (optional inputs for custom email or staging/production selection) diff --git a/eks-gitops-platform/modules/cert_manager/outputs.tf b/eks-gitops-platform/modules/cert_manager/outputs.tf new file mode 100644 index 0000000..9743914 --- /dev/null +++ b/eks-gitops-platform/modules/cert_manager/outputs.tf @@ -0,0 +1,4 @@ +output "namespace" { + description = "Namespace where cert-manager is deployed" + value = kubernetes_namespace.cert_manager.metadata[0].name +} diff --git a/eks-gitops-platform/modules/cert_manager/variables.tf b/eks-gitops-platform/modules/cert_manager/variables.tf new file mode 100644 index 0000000..902f10f --- /dev/null +++ b/eks-gitops-platform/modules/cert_manager/variables.tf @@ -0,0 +1,11 @@ +variable "namespace" { + description = "Namespace for cert-manager" + type = string + default = "cert-manager" +} + +variable "chart_version" { + description = "cert-manager Helm chart version" + type = string + default = "v1.14.3" +} From 69d0e03284768316433993e0c6866d37e8276297 Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 05:40:00 -0700 Subject: [PATCH 10/23] =?UTF-8?q?refactor:=20istio=20module=20=E2=80=94=20?= =?UTF-8?q?add=20variables.tf=20and=20outputs.tf?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- eks-gitops-platform/modules/istio/main.tf | 17 ++++------------- eks-gitops-platform/modules/istio/outputs.tf | 9 +++++++++ eks-gitops-platform/modules/istio/variables.tf | 11 +++++++++++ 3 files changed, 24 insertions(+), 13 deletions(-) create mode 100644 eks-gitops-platform/modules/istio/outputs.tf create mode 100644 eks-gitops-platform/modules/istio/variables.tf diff --git a/eks-gitops-platform/modules/istio/main.tf b/eks-gitops-platform/modules/istio/main.tf index 8de83f7..73baade 100644 --- a/eks-gitops-platform/modules/istio/main.tf +++ b/eks-gitops-platform/modules/istio/main.tf @@ -1,7 +1,6 @@ -# istio/main.tf resource "kubernetes_namespace" "istio_system" { metadata { - name = "istio-system" + name = var.namespace } } @@ -9,7 +8,7 @@ resource "helm_release" "istio_base" { name = "istio-base" repository = "https://istio-release.storage.googleapis.com/charts" chart = "base" - version = "1.21.0" + version = var.chart_version namespace = kubernetes_namespace.istio_system.metadata[0].name depends_on = [kubernetes_namespace.istio_system] @@ -19,7 +18,7 @@ resource "helm_release" "istiod" { name = "istiod" repository = "https://istio-release.storage.googleapis.com/charts" chart = "istiod" - version = "1.21.0" + version = var.chart_version namespace = kubernetes_namespace.istio_system.metadata[0].name values = [file("${path.module}/istiod-values.yaml")] @@ -31,18 +30,10 @@ resource "helm_release" "istio_ingress" { name = "istio-ingressgateway" repository = "https://istio-release.storage.googleapis.com/charts" chart = "gateway" - version = "1.21.0" + version = var.chart_version namespace = kubernetes_namespace.istio_system.metadata[0].name values = [file("${path.module}/ingress-values.yaml")] depends_on = [helm_release.istiod] } - -# istio/outputs.tf -output "istio_namespace" { - value = kubernetes_namespace.istio_system.metadata[0].name -} - -# istio/variables.tf -# (optional: can define version, values, or namespace name here) diff --git a/eks-gitops-platform/modules/istio/outputs.tf b/eks-gitops-platform/modules/istio/outputs.tf new file mode 100644 index 0000000..a5474eb --- /dev/null +++ b/eks-gitops-platform/modules/istio/outputs.tf @@ -0,0 +1,9 @@ +output "namespace" { + description = "Namespace where Istio is deployed" + value = kubernetes_namespace.istio_system.metadata[0].name +} + +output "ingress_gateway_service" { + description = "Istio ingress gateway service name" + value = "istio-ingressgateway" +} diff --git a/eks-gitops-platform/modules/istio/variables.tf b/eks-gitops-platform/modules/istio/variables.tf new file mode 100644 index 0000000..027a1a1 --- /dev/null +++ b/eks-gitops-platform/modules/istio/variables.tf @@ -0,0 +1,11 @@ +variable "namespace" { + description = "Namespace for Istio control plane" + type = string + default = "istio-system" +} + +variable "chart_version" { + description = "Istio Helm chart version (applies to base, istiod, and gateway)" + type = string + default = "1.21.0" +} From fa30f38b78846dcfa4957987a3c4da7c7d26f479 Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 05:40:17 -0700 Subject: [PATCH 11/23] =?UTF-8?q?refactor:=20postgres=20module=20=E2=80=94?= =?UTF-8?q?=20add=20variables.tf=20and=20outputs.tf?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- eks-gitops-platform/modules/postgres/main.tf | 13 ++----------- eks-gitops-platform/modules/postgres/outputs.tf | 9 +++++++++ eks-gitops-platform/modules/postgres/variables.tf | 11 +++++++++++ 3 files changed, 22 insertions(+), 11 deletions(-) create mode 100644 eks-gitops-platform/modules/postgres/outputs.tf create mode 100644 eks-gitops-platform/modules/postgres/variables.tf diff --git a/eks-gitops-platform/modules/postgres/main.tf b/eks-gitops-platform/modules/postgres/main.tf index 7768ddb..4a19e5e 100644 --- a/eks-gitops-platform/modules/postgres/main.tf +++ b/eks-gitops-platform/modules/postgres/main.tf @@ -1,7 +1,6 @@ -# postgres/main.tf resource "kubernetes_namespace" "app" { metadata { - name = "app-namespace" + name = var.namespace } } @@ -9,18 +8,10 @@ resource "helm_release" "postgres" { name = "postgresql" repository = "https://charts.bitnami.com/bitnami" chart = "postgresql" - version = "12.5.2" + version = var.chart_version namespace = kubernetes_namespace.app.metadata[0].name values = [file("${path.module}/values.yaml")] depends_on = [kubernetes_namespace.app] } - -# postgres/outputs.tf -output "postgres_namespace" { - value = kubernetes_namespace.app.metadata[0].name -} - -# postgres/variables.tf -# (optional: db name, user, password, storage size) diff --git a/eks-gitops-platform/modules/postgres/outputs.tf b/eks-gitops-platform/modules/postgres/outputs.tf new file mode 100644 index 0000000..33d2ed1 --- /dev/null +++ b/eks-gitops-platform/modules/postgres/outputs.tf @@ -0,0 +1,9 @@ +output "namespace" { + description = "Namespace where PostgreSQL is deployed" + value = kubernetes_namespace.app.metadata[0].name +} + +output "service_name" { + description = "PostgreSQL service name" + value = "postgresql" +} diff --git a/eks-gitops-platform/modules/postgres/variables.tf b/eks-gitops-platform/modules/postgres/variables.tf new file mode 100644 index 0000000..a46b478 --- /dev/null +++ b/eks-gitops-platform/modules/postgres/variables.tf @@ -0,0 +1,11 @@ +variable "namespace" { + description = "Namespace for PostgreSQL" + type = string + default = "app-namespace" +} + +variable "chart_version" { + description = "PostgreSQL Bitnami Helm chart version" + type = string + default = "12.5.2" +} From 44dc8ea9cbdd5ade14fc116c806003807765fdde Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 05:44:20 -0700 Subject: [PATCH 12/23] =?UTF-8?q?fix:=20vault=20module=20=E2=80=94=20repla?= =?UTF-8?q?ce=20broken=20kubernetes=5Fmanifest=20with=20native=20k8s=20res?= =?UTF-8?q?ources;=20add=20variables.tf=20+=20outputs.tf?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- eks-gitops-platform/modules/vault/main.tf | 36 ++++++++++++++----- eks-gitops-platform/modules/vault/outputs.tf | 9 +++++ .../modules/vault/variables.tf | 11 ++++++ 3 files changed, 47 insertions(+), 9 deletions(-) create mode 100644 eks-gitops-platform/modules/vault/outputs.tf create mode 100644 eks-gitops-platform/modules/vault/variables.tf diff --git a/eks-gitops-platform/modules/vault/main.tf b/eks-gitops-platform/modules/vault/main.tf index 42be690..9afc2ec 100644 --- a/eks-gitops-platform/modules/vault/main.tf +++ b/eks-gitops-platform/modules/vault/main.tf @@ -1,7 +1,6 @@ -# vault/main.tf resource "kubernetes_namespace" "vault" { metadata { - name = "vault" + name = var.namespace } } @@ -9,7 +8,7 @@ resource "helm_release" "vault" { name = "vault" repository = "https://helm.releases.hashicorp.com" chart = "vault" - version = "0.27.0" + version = var.chart_version namespace = kubernetes_namespace.vault.metadata[0].name values = [file("${path.module}/values.yaml")] @@ -17,12 +16,31 @@ resource "helm_release" "vault" { depends_on = [kubernetes_namespace.vault] } -resource "kubernetes_manifest" "vault_auth_config" { - manifest = yamldecode(file("${path.module}/vault-k8s-auth.yaml")) - depends_on = [helm_release.vault] +resource "kubernetes_service_account" "vault_auth" { + metadata { + name = "vault-auth" + namespace = kubernetes_namespace.vault.metadata[0].name + } + + depends_on = [kubernetes_namespace.vault] } -# vault/outputs.tf -output "vault_namespace" { - value = kubernetes_namespace.vault.metadata[0].name +resource "kubernetes_cluster_role_binding" "vault_auth" { + metadata { + name = "role-tokenreview-binding" + } + + role_ref { + api_group = "rbac.authorization.k8s.io" + kind = "ClusterRole" + name = "system:auth-delegator" + } + + subject { + kind = "ServiceAccount" + name = kubernetes_service_account.vault_auth.metadata[0].name + namespace = kubernetes_namespace.vault.metadata[0].name + } + + depends_on = [helm_release.vault] } diff --git a/eks-gitops-platform/modules/vault/outputs.tf b/eks-gitops-platform/modules/vault/outputs.tf new file mode 100644 index 0000000..3d9d439 --- /dev/null +++ b/eks-gitops-platform/modules/vault/outputs.tf @@ -0,0 +1,9 @@ +output "namespace" { + description = "Namespace where Vault is deployed" + value = kubernetes_namespace.vault.metadata[0].name +} + +output "service_name" { + description = "Vault service name" + value = "vault" +} diff --git a/eks-gitops-platform/modules/vault/variables.tf b/eks-gitops-platform/modules/vault/variables.tf new file mode 100644 index 0000000..5173830 --- /dev/null +++ b/eks-gitops-platform/modules/vault/variables.tf @@ -0,0 +1,11 @@ +variable "namespace" { + description = "Namespace for HashiCorp Vault" + type = string + default = "vault" +} + +variable "chart_version" { + description = "HashiCorp Vault Helm chart version" + type = string + default = "0.27.0" +} From 153aaae03d40d55bae8da6b2d5245d6f1647f13c Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 05:44:50 -0700 Subject: [PATCH 13/23] =?UTF-8?q?feat:=20flux=20module=20=E2=80=94=20wire?= =?UTF-8?q?=20GitRepository=20+=20Kustomization;=20add=20variables.tf=20+?= =?UTF-8?q?=20outputs.tf?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- eks-gitops-platform/modules/flux/main.tf | 66 +++++++++++++++---- eks-gitops-platform/modules/flux/outputs.tf | 9 +++ eks-gitops-platform/modules/flux/variables.tf | 28 ++++++++ 3 files changed, 92 insertions(+), 11 deletions(-) create mode 100644 eks-gitops-platform/modules/flux/outputs.tf create mode 100644 eks-gitops-platform/modules/flux/variables.tf diff --git a/eks-gitops-platform/modules/flux/main.tf b/eks-gitops-platform/modules/flux/main.tf index 4bfe6f4..0542338 100644 --- a/eks-gitops-platform/modules/flux/main.tf +++ b/eks-gitops-platform/modules/flux/main.tf @@ -1,7 +1,6 @@ -# flux/main.tf resource "kubernetes_namespace" "flux_system" { metadata { - name = "flux-system" + name = var.namespace } } @@ -10,19 +9,64 @@ resource "helm_release" "flux" { namespace = kubernetes_namespace.flux_system.metadata[0].name repository = "https://fluxcd-community.github.io/helm-charts" chart = "flux2" - version = "2.10.0" + version = var.chart_version values = [file("${path.module}/values.yaml")] - depends_on = [ - kubernetes_namespace.flux_system - ] + depends_on = [kubernetes_namespace.flux_system] } -# flux/outputs.tf -output "flux_namespace" { - value = kubernetes_namespace.flux_system.metadata[0].name +resource "kubernetes_manifest" "git_repository" { + manifest = { + apiVersion = "source.toolkit.fluxcd.io/v1" + kind = "GitRepository" + metadata = { + name = "eks-gitops-platform" + namespace = var.namespace + } + spec = { + interval = "1m" + url = var.github_repo_url + ref = { + branch = var.github_branch + } + } + } + + depends_on = [helm_release.flux] +} + +resource "kubernetes_manifest" "kustomization" { + manifest = { + apiVersion = "kustomize.toolkit.fluxcd.io/v1" + kind = "Kustomization" + metadata = { + name = "eks-gitops-platform" + namespace = var.namespace + } + spec = { + interval = "5m" + path = var.flux_sync_path + prune = true + sourceRef = { + kind = "GitRepository" + name = "eks-gitops-platform" + } + } + } + + depends_on = [kubernetes_manifest.git_repository] } -# flux/variables.tf -# (optional — define custom values or cluster name if needed) +# For private repos: uncomment and populate a GitHub PAT in a Kubernetes Secret. +# resource "kubernetes_secret" "flux_git_auth" { +# metadata { +# name = "flux-git-auth" +# namespace = var.namespace +# } +# data = { +# username = "git" +# password = var.github_pat # add var.github_pat to variables.tf +# } +# depends_on = [kubernetes_namespace.flux_system] +# } diff --git a/eks-gitops-platform/modules/flux/outputs.tf b/eks-gitops-platform/modules/flux/outputs.tf new file mode 100644 index 0000000..dcac545 --- /dev/null +++ b/eks-gitops-platform/modules/flux/outputs.tf @@ -0,0 +1,9 @@ +output "namespace" { + description = "Namespace where Flux is deployed" + value = kubernetes_namespace.flux_system.metadata[0].name +} + +output "git_repository_name" { + description = "Name of the Flux GitRepository resource" + value = "eks-gitops-platform" +} diff --git a/eks-gitops-platform/modules/flux/variables.tf b/eks-gitops-platform/modules/flux/variables.tf new file mode 100644 index 0000000..a2f058e --- /dev/null +++ b/eks-gitops-platform/modules/flux/variables.tf @@ -0,0 +1,28 @@ +variable "namespace" { + description = "Namespace for Flux system components" + type = string + default = "flux-system" +} + +variable "chart_version" { + description = "fluxcd-community/flux2 Helm chart version" + type = string + default = "2.10.0" +} + +variable "github_repo_url" { + description = "GitHub repository URL for Flux to sync (e.g. https://github.com/yourname/eks-gitops-platform)" + type = string +} + +variable "github_branch" { + description = "Git branch Flux watches for changes" + type = string + default = "main" +} + +variable "flux_sync_path" { + description = "Path within the repo that Flux syncs to the cluster" + type = string + default = "./flux-kustomize" +} From b089e64ccd7e76f7dd6dd93b6824e1ee4fc20f77 Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 05:50:56 -0700 Subject: [PATCH 14/23] =?UTF-8?q?feat:=20add=20Datadog=20module=20?= =?UTF-8?q?=E2=80=94=20APM,=20logs,=20cluster=20metrics,=20IRSA?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- eks-gitops-platform/modules/datadog/main.tf | 81 +++++++++++++++++++ .../modules/datadog/outputs.tf | 9 +++ .../modules/datadog/values.yaml | 17 ++++ .../modules/datadog/variables.tf | 50 ++++++++++++ 4 files changed, 157 insertions(+) create mode 100644 eks-gitops-platform/modules/datadog/main.tf create mode 100644 eks-gitops-platform/modules/datadog/outputs.tf create mode 100644 eks-gitops-platform/modules/datadog/values.yaml create mode 100644 eks-gitops-platform/modules/datadog/variables.tf diff --git a/eks-gitops-platform/modules/datadog/main.tf b/eks-gitops-platform/modules/datadog/main.tf new file mode 100644 index 0000000..28d3369 --- /dev/null +++ b/eks-gitops-platform/modules/datadog/main.tf @@ -0,0 +1,81 @@ +resource "kubernetes_namespace" "datadog" { + metadata { + name = var.namespace + } +} + +data "aws_iam_policy_document" "datadog_assume_role" { + statement { + effect = "Allow" + actions = ["sts:AssumeRoleWithWebIdentity"] + + principals { + type = "Federated" + identifiers = [var.oidc_provider_arn] + } + + condition { + test = "StringEquals" + variable = "${trimprefix(var.oidc_provider_url, "https://")}:sub" + values = ["system:serviceaccount:${var.namespace}:datadog-agent"] + } + + condition { + test = "StringEquals" + variable = "${trimprefix(var.oidc_provider_url, "https://")}:aud" + values = ["sts.amazonaws.com"] + } + } +} + +resource "aws_iam_role" "datadog_agent" { + name = "datadog-agent-irsa" + assume_role_policy = data.aws_iam_policy_document.datadog_assume_role.json +} + +resource "aws_iam_role_policy_attachment" "datadog_ec2_readonly" { + role = aws_iam_role.datadog_agent.name + policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess" +} + +resource "helm_release" "datadog" { + name = "datadog" + repository = "https://helm.datadoghq.com" + chart = "datadog" + version = var.chart_version + namespace = kubernetes_namespace.datadog.metadata[0].name + + values = [file("${path.module}/values.yaml")] + + set_sensitive { + name = "datadog.apiKey" + value = var.datadog_api_key + } + + set_sensitive { + name = "datadog.appKey" + value = var.datadog_app_key + } + + set { + name = "datadog.site" + value = var.datadog_site + } + + set { + name = "datadog.clusterName" + value = var.cluster_name + } + + set { + name = "datadog.networkMonitoring.enabled" + value = tostring(var.enable_npm) + } + + set { + name = "agents.serviceAccount.annotations.eks\\.amazonaws\\.com/role-arn" + value = aws_iam_role.datadog_agent.arn + } + + depends_on = [kubernetes_namespace.datadog] +} diff --git a/eks-gitops-platform/modules/datadog/outputs.tf b/eks-gitops-platform/modules/datadog/outputs.tf new file mode 100644 index 0000000..0b2f3a9 --- /dev/null +++ b/eks-gitops-platform/modules/datadog/outputs.tf @@ -0,0 +1,9 @@ +output "namespace" { + description = "Namespace where Datadog is deployed" + value = kubernetes_namespace.datadog.metadata[0].name +} + +output "irsa_role_arn" { + description = "IAM role ARN assigned to the Datadog agent via IRSA" + value = aws_iam_role.datadog_agent.arn +} diff --git a/eks-gitops-platform/modules/datadog/values.yaml b/eks-gitops-platform/modules/datadog/values.yaml new file mode 100644 index 0000000..9b28316 --- /dev/null +++ b/eks-gitops-platform/modules/datadog/values.yaml @@ -0,0 +1,17 @@ +datadog: + logs: + enabled: true + containerCollectAll: true + apm: + portEnabled: true + kubeStateMetricsEnabled: true + kubeStateMetricsCore: + enabled: true + +clusterAgent: + enabled: true + replicas: 2 + +datadog-crds: + crds: + datadogMetrics: true diff --git a/eks-gitops-platform/modules/datadog/variables.tf b/eks-gitops-platform/modules/datadog/variables.tf new file mode 100644 index 0000000..b8164ea --- /dev/null +++ b/eks-gitops-platform/modules/datadog/variables.tf @@ -0,0 +1,50 @@ +variable "namespace" { + description = "Namespace for Datadog agent" + type = string + default = "datadog" +} + +variable "chart_version" { + description = "Datadog Helm chart version" + type = string + default = "3.67.0" +} + +variable "datadog_api_key" { + description = "Datadog API key" + type = string + sensitive = true +} + +variable "datadog_app_key" { + description = "Datadog application key" + type = string + sensitive = true +} + +variable "datadog_site" { + description = "Datadog intake site (datadoghq.com or datadoghq.eu)" + type = string + default = "datadoghq.com" +} + +variable "cluster_name" { + description = "EKS cluster name — used to tag all Datadog telemetry" + type = string +} + +variable "enable_npm" { + description = "Enable Datadog Network Performance Monitoring" + type = bool + default = false +} + +variable "oidc_provider_arn" { + description = "ARN of the EKS OIDC provider (from module.eks.oidc_provider_arn)" + type = string +} + +variable "oidc_provider_url" { + description = "URL of the EKS OIDC provider (from module.eks.cluster_oidc_issuer_url)" + type = string +} From d8ed0630da887777ea138315c040359163ea374f Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 05:52:38 -0700 Subject: [PATCH 15/23] =?UTF-8?q?chore:=20remove=20komodor=20module=20?= =?UTF-8?q?=E2=80=94=20replaced=20by=20Datadog?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- eks-gitops-platform/modules/komodor/main.tf | 26 ------------------- .../modules/komodor/values.yaml | 20 -------------- 2 files changed, 46 deletions(-) delete mode 100644 eks-gitops-platform/modules/komodor/main.tf delete mode 100644 eks-gitops-platform/modules/komodor/values.yaml diff --git a/eks-gitops-platform/modules/komodor/main.tf b/eks-gitops-platform/modules/komodor/main.tf deleted file mode 100644 index e36560f..0000000 --- a/eks-gitops-platform/modules/komodor/main.tf +++ /dev/null @@ -1,26 +0,0 @@ -# komodor/main.tf -resource "kubernetes_namespace" "komodor" { - metadata { - name = "komodor" - } -} - -resource "helm_release" "komodor_agent" { - name = "komodor-agent" - repository = "https://helm-charts.komodor.io" - chart = "komodor-agent" - version = "1.5.1" - namespace = kubernetes_namespace.komodor.metadata[0].name - - values = [file("${path.module}/values.yaml")] - - depends_on = [kubernetes_namespace.komodor] -} - -# komodor/outputs.tf -output "komodor_namespace" { - value = kubernetes_namespace.komodor.metadata[0].name -} - -# komodor/variables.tf -# (optional: apiKey or secret reference) diff --git a/eks-gitops-platform/modules/komodor/values.yaml b/eks-gitops-platform/modules/komodor/values.yaml deleted file mode 100644 index c2d929e..0000000 --- a/eks-gitops-platform/modules/komodor/values.yaml +++ /dev/null @@ -1,20 +0,0 @@ -# komodor/values.yaml -agent: - apiKey: "YOUR_KOMODOR_API_KEY" - clusterName: "eks-cluster" - logLevel: "info" - - filters: - includeNamespaces: - - app-namespace - excludeNamespaces: - - kube-system - - kube-public - - resources: - requests: - cpu: 100m - memory: 128Mi - limits: - cpu: 200m - memory: 256Mi From 4acbe9966254afcc0df0bc2233e29d050d07d1f2 Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 05:52:46 -0700 Subject: [PATCH 16/23] =?UTF-8?q?feat:=20add=20bootstrap.sh=20=E2=80=94=20?= =?UTF-8?q?runs=20bootstrap/=20and=20writes=20backend.tf=20automatically?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- eks-gitops-platform/scripts/bootstrap.sh | 42 ++++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100755 eks-gitops-platform/scripts/bootstrap.sh diff --git a/eks-gitops-platform/scripts/bootstrap.sh b/eks-gitops-platform/scripts/bootstrap.sh new file mode 100755 index 0000000..6c11ded --- /dev/null +++ b/eks-gitops-platform/scripts/bootstrap.sh @@ -0,0 +1,42 @@ +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +TF_ROOT="$(dirname "$SCRIPT_DIR")" +BOOTSTRAP_DIR="$TF_ROOT/bootstrap" + +echo "==> Initializing bootstrap module (S3 + DynamoDB)..." + +cd "$BOOTSTRAP_DIR" +terraform init -input=false +terraform apply -auto-approve -input=false + +BUCKET=$(terraform output -raw state_bucket_name) +TABLE=$(terraform output -raw lock_table_name) +REGION=$(terraform output -raw aws_region) + +echo "==> Writing $TF_ROOT/backend.tf..." + +cat > "$TF_ROOT/backend.tf" < Date: Mon, 18 May 2026 05:53:04 -0700 Subject: [PATCH 17/23] =?UTF-8?q?fix:=20rewrite=20one-shot-deploy.sh=20?= =?UTF-8?q?=E2=80=94=20correct=20paths,=20single=20root=20apply,=20idempot?= =?UTF-8?q?ent=20bootstrap?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../scripts/one-shot-deploy.sh | 78 ++++++++++--------- 1 file changed, 42 insertions(+), 36 deletions(-) mode change 100644 => 100755 eks-gitops-platform/scripts/one-shot-deploy.sh diff --git a/eks-gitops-platform/scripts/one-shot-deploy.sh b/eks-gitops-platform/scripts/one-shot-deploy.sh old mode 100644 new mode 100755 index 4900777..b5ce498 --- a/eks-gitops-platform/scripts/one-shot-deploy.sh +++ b/eks-gitops-platform/scripts/one-shot-deploy.sh @@ -1,40 +1,46 @@ -#!/bin/bash - +#!/usr/bin/env bash set -euo pipefail -CLUSTER_NAME="eks-cluster" -REGION="us-west-2" - -# Step 1: Set AWS credentials and region -export AWS_REGION=$REGION - -# Step 2: Initialize and apply all Terraform modules -for module in eks flux istio postgres cert_manager vault komodor; do - echo "\n>>> Deploying module: $module" - pushd $module - terraform init -input=false - terraform apply -auto-approve - popd -done - -# Step 3: Apply Flux app deployment -pushd flux -if kubectl get namespace flux-system &>/dev/null; then - echo "\n>>> Applying Flux GitRepository + Kustomization" - kubectl apply -f app-deploy.yaml +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +TF_ROOT="$(dirname "$SCRIPT_DIR")" + +echo "==> EKS GitOps Platform — One-Shot Deploy" +echo "" + +# Step 1: Bootstrap remote state if not already done +BOOTSTRAP_OUTPUT="$TF_ROOT/bootstrap/.terraform" +if [ ! -d "$BOOTSTRAP_OUTPUT" ]; then + echo "==> Running bootstrap (first time only)..." + bash "$SCRIPT_DIR/bootstrap.sh" +else + echo "==> Bootstrap already initialized. Checking outputs..." + cd "$TF_ROOT/bootstrap" + if ! terraform output -raw state_bucket_name &>/dev/null 2>&1; then + echo "==> Bootstrap state missing — re-running bootstrap..." + bash "$SCRIPT_DIR/bootstrap.sh" + else + echo "==> Bootstrap already complete, skipping." + fi fi -popd - -# Step 4: Apply cert-manager TLS certificate -kubectl apply -f cert_manager/certificate.yaml - -# Step 5: Apply Istio mTLS + AuthZ policies -kubectl apply -f istio/mtls-policy.yaml - -# Step 6: Confirm status -kubectl get pods -A -kubectl get certificate -A -kubectl get gateways -A -kubectl get kustomizations -A -echo "\n✅ One-shot deployment complete!" +# Step 2: Initialize root module +cd "$TF_ROOT" +echo "" +echo "==> Initializing Terraform root module..." +terraform init -input=false + +# Step 3: Apply everything +echo "" +echo "==> Applying platform configuration..." +terraform apply -input=false + +# Step 4: Show cluster status +echo "" +echo "==> Platform status:" +aws eks update-kubeconfig --name "$(terraform output -raw cluster_name)" --region "$(terraform output -raw aws_region 2>/dev/null || echo us-west-2)" 2>/dev/null || true +kubectl get pods -A 2>/dev/null || echo " (kubectl not configured yet — run: aws eks update-kubeconfig --name )" +kubectl get kustomizations -A 2>/dev/null || true +kubectl get gitrepositories -A 2>/dev/null || true + +echo "" +echo "✅ One-shot deployment complete!" From 37a869d938e1f1f0d5f6b5f9250aa923e5dbaca0 Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 05:54:22 -0700 Subject: [PATCH 18/23] feat: add GitHub Actions terraform-plan workflow (PR: fmt + validate + tfsec + plan comment) --- .github/workflows/terraform-plan.yml | 90 ++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 .github/workflows/terraform-plan.yml diff --git a/.github/workflows/terraform-plan.yml b/.github/workflows/terraform-plan.yml new file mode 100644 index 0000000..a67daf9 --- /dev/null +++ b/.github/workflows/terraform-plan.yml @@ -0,0 +1,90 @@ +name: Terraform Plan + +on: + pull_request: + branches: + - main + +permissions: + contents: read + pull-requests: write + +concurrency: + group: terraform-plan-${{ github.ref }} + cancel-in-progress: true + +jobs: + plan: + name: Terraform Plan + runs-on: ubuntu-latest + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Setup Terraform + uses: hashicorp/setup-terraform@v3 + with: + terraform_version: "~1.7" + + - name: Configure AWS credentials + uses: aws-actions/configure-aws-credentials@v4 + with: + aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} + aws-region: us-west-2 + + - name: Terraform Format Check + id: fmt + run: terraform fmt -check -recursive + working-directory: eks-gitops-platform + + - name: Terraform Init + id: init + run: terraform init -input=false + working-directory: eks-gitops-platform + + - name: Terraform Validate + id: validate + run: terraform validate + working-directory: eks-gitops-platform + + - name: Run tfsec + uses: aquasecurity/tfsec-action@v1.0.3 + with: + working_directory: eks-gitops-platform + minimum_severity: HIGH + soft_fail: false + + - name: Terraform Plan + id: plan + run: | + terraform plan -no-color -input=false \ + -var="datadog_api_key=${{ secrets.DATADOG_API_KEY }}" \ + -var="datadog_app_key=${{ secrets.DATADOG_APP_KEY }}" \ + -var="github_repo_url=${{ secrets.TF_VAR_GITHUB_REPO_URL }}" \ + 2>&1 | tee /tmp/plan.txt + echo "exit_code=${PIPESTATUS[0]}" >> "$GITHUB_OUTPUT" + working-directory: eks-gitops-platform + + - name: Post Plan as PR Comment + uses: actions/github-script@v7 + if: always() + env: + PLAN_PATH: /tmp/plan.txt + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + const fs = require('fs'); + const plan = fs.readFileSync(process.env.PLAN_PATH, 'utf8'); + const truncated = plan.length > 60000 + ? plan.substring(0, 60000) + '\n\n...[truncated — see Actions logs for full output]' + : plan; + const header = `## Terraform Plan — ${{ github.event.pull_request.head.sha }}`; + const body = `${header}\n\n\`\`\`\n${truncated}\n\`\`\``; + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body, + }); From 967b372b985ff4b1e1de215afd81405b227cdb8c Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 05:54:31 -0700 Subject: [PATCH 19/23] feat: add GitHub Actions terraform-apply workflow (merge to main: plan + auto-apply) --- .github/workflows/terraform-apply.yml | 51 +++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 .github/workflows/terraform-apply.yml diff --git a/.github/workflows/terraform-apply.yml b/.github/workflows/terraform-apply.yml new file mode 100644 index 0000000..1b6e1d1 --- /dev/null +++ b/.github/workflows/terraform-apply.yml @@ -0,0 +1,51 @@ +name: Terraform Apply + +on: + push: + branches: + - main + +concurrency: + group: terraform-apply + cancel-in-progress: false + +jobs: + apply: + name: Terraform Apply + runs-on: ubuntu-latest + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Setup Terraform + uses: hashicorp/setup-terraform@v3 + with: + terraform_version: "~1.7" + + - name: Configure AWS credentials + uses: aws-actions/configure-aws-credentials@v4 + with: + aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} + aws-region: us-west-2 + + - name: Terraform Init + run: terraform init -input=false + working-directory: eks-gitops-platform + + - name: Terraform Plan + run: | + terraform plan -no-color -input=false \ + -var="datadog_api_key=${{ secrets.DATADOG_API_KEY }}" \ + -var="datadog_app_key=${{ secrets.DATADOG_APP_KEY }}" \ + -var="github_repo_url=${{ secrets.TF_VAR_GITHUB_REPO_URL }}" + working-directory: eks-gitops-platform + + - name: Terraform Apply + run: | + terraform apply -auto-approve -input=false \ + -var="datadog_api_key=${{ secrets.DATADOG_API_KEY }}" \ + -var="datadog_app_key=${{ secrets.DATADOG_APP_KEY }}" \ + -var="github_repo_url=${{ secrets.TF_VAR_GITHUB_REPO_URL }}" + working-directory: eks-gitops-platform From 89bd075954a20598f0ca3c57ee1365363b449227 Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 05:56:30 -0700 Subject: [PATCH 20/23] docs: add terraform.tfvars.example, update .gitignore and README with full onboarding guide --- eks-gitops-platform/.gitignore | 5 + eks-gitops-platform/README.md | 233 ++++++++----------- eks-gitops-platform/terraform.tfvars.example | 26 +++ 3 files changed, 128 insertions(+), 136 deletions(-) create mode 100644 eks-gitops-platform/terraform.tfvars.example diff --git a/eks-gitops-platform/.gitignore b/eks-gitops-platform/.gitignore index 1c99dc1..0c74832 100644 --- a/eks-gitops-platform/.gitignore +++ b/eks-gitops-platform/.gitignore @@ -1 +1,6 @@ .terraform/ +terraform.tfvars +*.tfstate +*.tfstate.backup +.terraform.lock.hcl +/backend.tf diff --git a/eks-gitops-platform/README.md b/eks-gitops-platform/README.md index f0ab9fc..2d216f1 100644 --- a/eks-gitops-platform/README.md +++ b/eks-gitops-platform/README.md @@ -1,174 +1,135 @@ -# EKS GitOps Platform with Flux, Istio, PostgreSQL & Komodor +# EKS GitOps Platform -This project bootstraps a production-ready Kubernetes platform on AWS using Terraform and Helm, with GitOps and service mesh integration. +Production-grade Kubernetes platform on AWS. Provisions EKS with Flux GitOps, Istio service mesh, cert-manager TLS, HashiCorp Vault, PostgreSQL, and Datadog full-stack observability — all via Terraform. --- -## 🚀 Modules Overview +## Platform Components -### ✅ `eks/` -- Provisions an AWS EKS Cluster -- Sets up managed node groups +| Component | Purpose | Namespace | +|---|---|---| +| EKS 1.30 | Managed Kubernetes cluster | — | +| Flux v2 | GitOps — syncs this repo to the cluster | `flux-system` | +| Istio 1.21 | Service mesh + ingress gateway | `istio-system` | +| cert-manager | Automatic TLS via Let's Encrypt | `cert-manager` | +| HashiCorp Vault | Secrets management | `vault` | +| PostgreSQL | Relational database | `app-namespace` | +| Datadog | Metrics + APM + logs + k8s state | `datadog` | -### ✅ `flux/` -- Installs Flux v2 GitOps tooling -- Enables automated deployments from Git +--- -### ✅ `istio/` -- Installs Istio base, control plane, and ingress gateway via Helm +## Quickstart — Fork and Deploy -### ✅ `postgres/` -- Deploys a PostgreSQL instance using Bitnami's Helm chart -- Includes persistent storage with PVC +### 1. Prerequisites -### ✅ `komodor/` -- Deploys Komodor agent to monitor application behavior and surface changes +- AWS CLI configured (`aws configure` or `AWS_PROFILE`) +- Terraform ≥ 1.7 (`brew install terraform`) +- kubectl (`brew install kubectl`) +- A Datadog account (free trial works) ---- +### 2. Fork and clone -## 📦 Optional App Setup (Istio Ingress) +```bash +# Fork https://github.com/daringanitch/eks-gitops-platform on GitHub, then: +git clone https://github.com/YOUR_USERNAME/eks-gitops-platform.git +cd eks-gitops-platform/eks-gitops-platform +``` -Add this app manifest under `kubernetes/test-app.yaml` (or Flux GitOps source): +### 3. Configure your variables -```yaml -apiVersion: v1 -kind: Namespace -metadata: - name: app-namespace ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - name: test-app - namespace: app-namespace -spec: - replicas: 1 - selector: - matchLabels: - app: test-app - template: - metadata: - labels: - app: test-app - spec: - containers: - - name: test-app - image: hashicorp/http-echo - args: - - "-text=Hello from Istio" - ports: - - containerPort: 5678 ---- -apiVersion: v1 -kind: Service -metadata: - name: test-app - namespace: app-namespace -spec: - ports: - - port: 80 - targetPort: 5678 - selector: - app: test-app ---- -apiVersion: networking.istio.io/v1beta1 -kind: Gateway -metadata: - name: test-gateway - namespace: app-namespace -spec: - selector: - istio: ingressgateway - servers: - - port: - number: 443 - name: https - protocol: HTTPS - tls: - mode: SIMPLE - credentialName: tls-secret - hosts: - - "example.com" ---- -apiVersion: networking.istio.io/v1beta1 -kind: VirtualService -metadata: - name: test-app - namespace: app-namespace -spec: - hosts: - - "example.com" - gateways: - - test-gateway - http: - - match: - - uri: - prefix: "/" - route: - - destination: - host: test-app - port: - number: 80 +```bash +cp terraform.tfvars.example terraform.tfvars +# Edit terraform.tfvars — fill in datadog_api_key, datadog_app_key, and your github_repo_url ``` ---- - -## ⚙️ Terraform Bootstrap Steps +### 4. Bootstrap remote state (first time only) -### 1. Configure AWS credentials: ```bash -export AWS_PROFILE=your-profile -export AWS_REGION=us-west-2 +bash scripts/bootstrap.sh ``` -### 2. Initialize and apply +This creates an S3 bucket + DynamoDB table and writes `backend.tf` automatically. + +### 5. Deploy everything + ```bash terraform init -terraform apply -auto-approve +terraform apply ``` -> Repeat for each module (`eks/`, `flux/`, `istio/`, etc.) +Or use the one-shot helper: ---- +```bash +bash scripts/one-shot-deploy.sh +``` -## 🧪 CI/CD Pipeline Stub (GitHub Actions) +--- -```yaml -name: Terraform Apply +## GitHub Actions CI/CD -on: - push: - branches: - - main +On every **pull request** → `terraform fmt` + `terraform validate` + `tfsec` + plan posted as PR comment. -jobs: - terraform: - runs-on: ubuntu-latest +On every **merge to main** → `terraform plan` + `terraform apply` (auto). - steps: - - uses: actions/checkout@v3 +### Required GitHub Secrets - - name: Setup Terraform - uses: hashicorp/setup-terraform@v2 +Go to `Settings → Secrets → Actions` in your fork and add: - - name: Terraform Init and Apply - run: | - cd eks - terraform init - terraform apply -auto-approve -``` +| Secret | Value | +|---|---| +| `AWS_ACCESS_KEY_ID` | IAM access key with EKS/VPC/IAM permissions | +| `AWS_SECRET_ACCESS_KEY` | Corresponding secret key | +| `DATADOG_API_KEY` | From Datadog → Organization Settings → API Keys | +| `DATADOG_APP_KEY` | From Datadog → Organization Settings → Application Keys | +| `TF_VAR_GITHUB_REPO_URL` | Your fork URL: `https://github.com/YOUR_USERNAME/eks-gitops-platform` | --- -## ✅ Roadmap / Extras +## Architecture -- [ ] Add Cert-Manager & Let's Encrypt -- [ ] Enable mTLS and advanced Istio policies -- [ ] Include sealed-secrets or Vault for secret management -- [ ] Add ArgoCD dashboard (optional) -- [ ] Automate app deployment via Flux GitRepository & Kustomization +``` +GitHub (your fork) + │ + │ Flux polls every 1 min + ▼ + flux-system (Flux v2) + │ syncs ./flux-kustomize/ + ▼ + app-namespace (test-app) + + istio-system (Istio + ingress gateway) + cert-manager (Let's Encrypt TLS) + vault (HashiCorp Vault) + app-namespace (PostgreSQL) + datadog (Agent + Cluster Agent + APM) +``` --- -## 📬 Contact & Contributions -Feel free to open PRs or issues to expand this platform! +## Repository Structure + +``` +eks-gitops-platform/ ← Terraform root +├── bootstrap/ ← Run once: provisions S3 + DynamoDB +├── modules/ +│ ├── cert_manager/ +│ ├── datadog/ ← Full-stack observability (replaces Komodor) +│ ├── flux/ ← Includes GitRepository + Kustomization +│ ├── istio/ +│ ├── postgres/ +│ └── vault/ +├── flux-kustomize/test-app/ ← Deployed by Flux as smoke test +├── manifests/ ← Reference manifests +├── scripts/ +│ ├── bootstrap.sh ← First-time setup +│ └── one-shot-deploy.sh ← Full deploy helper +├── .github/workflows/ +│ ├── terraform-plan.yml ← Runs on PR +│ └── terraform-apply.yml ← Runs on merge to main +├── main.tf +├── variables.tf +├── outputs.tf +├── providers.tf +├── backend.tf ← Generated by bootstrap.sh +└── terraform.tfvars.example ← Copy to terraform.tfvars +``` diff --git a/eks-gitops-platform/terraform.tfvars.example b/eks-gitops-platform/terraform.tfvars.example new file mode 100644 index 0000000..888c3c0 --- /dev/null +++ b/eks-gitops-platform/terraform.tfvars.example @@ -0,0 +1,26 @@ +# Copy this file to terraform.tfvars and fill in your values. +# IMPORTANT: Never commit terraform.tfvars — it is gitignored. + +# ── AWS ───────────────────────────────────────────────────────────────────── +aws_region = "us-west-2" +cluster_name = "eks-gitops-cluster" +cluster_version = "1.30" +environment = "prod" + +# ── Networking ─────────────────────────────────────────────────────────────── +vpc_cidr = "10.0.0.0/16" +private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"] +public_subnets = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"] + +# ── Datadog (sensitive — store in GitHub Secrets for CI) ───────────────────── +# Get these from: https://app.datadoghq.com/organization-settings/api-keys +datadog_api_key = "your-datadog-api-key" +datadog_app_key = "your-datadog-app-key" +datadog_site = "datadoghq.com" # datadoghq.eu for EU region +enable_npm = false # set true to enable Network Performance Monitoring + +# ── Flux GitOps ────────────────────────────────────────────────────────────── +# Set this to YOUR fork of the repo — Flux will sync ./flux-kustomize from it +github_repo_url = "https://github.com/YOUR_USERNAME/eks-gitops-platform" +github_branch = "main" +flux_sync_path = "./flux-kustomize" From e8ab455951cbdb506d8c3a07fed0d89fc93699b3 Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 06:02:56 -0700 Subject: [PATCH 21/23] fix: use OIDC auth, -backend-config for state, and plan artifact in CI workflows Co-Authored-By: Claude Sonnet 4.6 --- .github/workflows/terraform-apply.yml | 23 ++++++++++++++--------- .github/workflows/terraform-plan.yml | 12 +++++++++--- eks-gitops-platform/.gitignore | 2 +- eks-gitops-platform/README.md | 7 +++++-- eks-gitops-platform/backend.tf | 11 ----------- eks-gitops-platform/backend.tf.example | 19 +++++++++++++++++++ 6 files changed, 48 insertions(+), 26 deletions(-) delete mode 100644 eks-gitops-platform/backend.tf create mode 100644 eks-gitops-platform/backend.tf.example diff --git a/.github/workflows/terraform-apply.yml b/.github/workflows/terraform-apply.yml index 1b6e1d1..78c1b18 100644 --- a/.github/workflows/terraform-apply.yml +++ b/.github/workflows/terraform-apply.yml @@ -5,6 +5,10 @@ on: branches: - main +permissions: + contents: read + id-token: write + concurrency: group: terraform-apply cancel-in-progress: false @@ -26,26 +30,27 @@ jobs: - name: Configure AWS credentials uses: aws-actions/configure-aws-credentials@v4 with: - aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} - aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} + role-to-assume: ${{ secrets.AWS_ROLE_ARN }} aws-region: us-west-2 - name: Terraform Init - run: terraform init -input=false + run: | + terraform init -input=false \ + -backend-config="bucket=${{ secrets.TF_STATE_BUCKET }}" \ + -backend-config="key=eks-gitops-platform/terraform.tfstate" \ + -backend-config="region=us-west-2" \ + -backend-config="dynamodb_table=${{ secrets.TF_STATE_TABLE }}" \ + -backend-config="encrypt=true" working-directory: eks-gitops-platform - name: Terraform Plan run: | - terraform plan -no-color -input=false \ + terraform plan -no-color -input=false -out=/tmp/tfplan \ -var="datadog_api_key=${{ secrets.DATADOG_API_KEY }}" \ -var="datadog_app_key=${{ secrets.DATADOG_APP_KEY }}" \ -var="github_repo_url=${{ secrets.TF_VAR_GITHUB_REPO_URL }}" working-directory: eks-gitops-platform - name: Terraform Apply - run: | - terraform apply -auto-approve -input=false \ - -var="datadog_api_key=${{ secrets.DATADOG_API_KEY }}" \ - -var="datadog_app_key=${{ secrets.DATADOG_APP_KEY }}" \ - -var="github_repo_url=${{ secrets.TF_VAR_GITHUB_REPO_URL }}" + run: terraform apply -auto-approve /tmp/tfplan working-directory: eks-gitops-platform diff --git a/.github/workflows/terraform-plan.yml b/.github/workflows/terraform-plan.yml index a67daf9..29b1afa 100644 --- a/.github/workflows/terraform-plan.yml +++ b/.github/workflows/terraform-plan.yml @@ -8,6 +8,7 @@ on: permissions: contents: read pull-requests: write + id-token: write concurrency: group: terraform-plan-${{ github.ref }} @@ -30,8 +31,7 @@ jobs: - name: Configure AWS credentials uses: aws-actions/configure-aws-credentials@v4 with: - aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} - aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} + role-to-assume: ${{ secrets.AWS_ROLE_ARN }} aws-region: us-west-2 - name: Terraform Format Check @@ -41,7 +41,13 @@ jobs: - name: Terraform Init id: init - run: terraform init -input=false + run: | + terraform init -input=false \ + -backend-config="bucket=${{ secrets.TF_STATE_BUCKET }}" \ + -backend-config="key=eks-gitops-platform/terraform.tfstate" \ + -backend-config="region=us-west-2" \ + -backend-config="dynamodb_table=${{ secrets.TF_STATE_TABLE }}" \ + -backend-config="encrypt=true" working-directory: eks-gitops-platform - name: Terraform Validate diff --git a/eks-gitops-platform/.gitignore b/eks-gitops-platform/.gitignore index 0c74832..849b7dc 100644 --- a/eks-gitops-platform/.gitignore +++ b/eks-gitops-platform/.gitignore @@ -3,4 +3,4 @@ terraform.tfvars *.tfstate *.tfstate.backup .terraform.lock.hcl -/backend.tf +backend.tf diff --git a/eks-gitops-platform/README.md b/eks-gitops-platform/README.md index 2d216f1..12d33c5 100644 --- a/eks-gitops-platform/README.md +++ b/eks-gitops-platform/README.md @@ -77,12 +77,15 @@ Go to `Settings → Secrets → Actions` in your fork and add: | Secret | Value | |---|---| -| `AWS_ACCESS_KEY_ID` | IAM access key with EKS/VPC/IAM permissions | -| `AWS_SECRET_ACCESS_KEY` | Corresponding secret key | +| `AWS_ROLE_ARN` | IAM role ARN with EKS/VPC/IAM permissions + GitHub OIDC trust policy | +| `TF_STATE_BUCKET` | S3 bucket name created by bootstrap.sh | +| `TF_STATE_TABLE` | DynamoDB table name created by bootstrap.sh | | `DATADOG_API_KEY` | From Datadog → Organization Settings → API Keys | | `DATADOG_APP_KEY` | From Datadog → Organization Settings → Application Keys | | `TF_VAR_GITHUB_REPO_URL` | Your fork URL: `https://github.com/YOUR_USERNAME/eks-gitops-platform` | +> **Note:** `AWS_ROLE_ARN` requires an IAM role with a GitHub OIDC trust policy. Static `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` credentials are no longer used. + --- ## Architecture diff --git a/eks-gitops-platform/backend.tf b/eks-gitops-platform/backend.tf deleted file mode 100644 index 6967dd8..0000000 --- a/eks-gitops-platform/backend.tf +++ /dev/null @@ -1,11 +0,0 @@ -# Generated by scripts/bootstrap.sh — do not edit manually. -# Run scripts/bootstrap.sh to populate the bucket and table names. -terraform { - backend "s3" { - bucket = "REPLACE_WITH_BUCKET_NAME" - key = "eks-gitops-platform/terraform.tfstate" - region = "us-west-2" - dynamodb_table = "REPLACE_WITH_TABLE_NAME" - encrypt = true - } -} diff --git a/eks-gitops-platform/backend.tf.example b/eks-gitops-platform/backend.tf.example new file mode 100644 index 0000000..c3ac2d1 --- /dev/null +++ b/eks-gitops-platform/backend.tf.example @@ -0,0 +1,19 @@ +# backend.tf.example +# This file is generated by scripts/bootstrap.sh at first-time setup. +# Copy to backend.tf and replace the placeholder values, OR +# let bootstrap.sh generate it automatically: +# +# bash scripts/bootstrap.sh +# +# For CI/CD: backend config is passed via -backend-config flags using +# TF_STATE_BUCKET and TF_STATE_TABLE GitHub Secrets. +# +# terraform { +# backend "s3" { +# bucket = "your-eks-gitops-tfstate-bucket" +# key = "eks-gitops-platform/terraform.tfstate" +# region = "us-west-2" +# dynamodb_table = "your-eks-gitops-tfstate-lock" +# encrypt = true +# } +# } From 74f3158aa4d5c1f6da3df8404742ef5ca0112514 Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Mon, 18 May 2026 07:41:13 -0700 Subject: [PATCH 22/23] fix: parameterize postgres credentials, cert-manager email, and fix bootstrap idempotency Co-Authored-By: Claude Sonnet 4.6 --- eks-gitops-platform/main.tf | 5 +++++ .../cert_manager/letsencrypt-clusterissuer.yaml | 2 +- .../modules/cert_manager/main.tf | 4 +++- .../modules/cert_manager/variables.tf | 5 +++++ eks-gitops-platform/modules/postgres/main.tf | 10 ++++++++++ .../modules/postgres/values.yaml | 2 -- .../modules/postgres/variables.tf | 12 ++++++++++++ eks-gitops-platform/scripts/one-shot-deploy.sh | 16 ++++++---------- eks-gitops-platform/terraform.tfvars.example | 7 +++++++ eks-gitops-platform/variables.tf | 17 +++++++++++++++++ 10 files changed, 66 insertions(+), 14 deletions(-) diff --git a/eks-gitops-platform/main.tf b/eks-gitops-platform/main.tf index 9b7e757..0f1eee4 100644 --- a/eks-gitops-platform/main.tf +++ b/eks-gitops-platform/main.tf @@ -55,6 +55,8 @@ module "eks" { module "cert_manager" { source = "./modules/cert_manager" + letsencrypt_email = var.letsencrypt_email + depends_on = [module.eks] } @@ -73,6 +75,9 @@ module "vault" { module "postgres" { source = "./modules/postgres" + postgres_password = var.postgres_password + postgres_user_password = var.postgres_user_password + depends_on = [module.eks] } diff --git a/eks-gitops-platform/modules/cert_manager/letsencrypt-clusterissuer.yaml b/eks-gitops-platform/modules/cert_manager/letsencrypt-clusterissuer.yaml index 3e96104..995f51e 100644 --- a/eks-gitops-platform/modules/cert_manager/letsencrypt-clusterissuer.yaml +++ b/eks-gitops-platform/modules/cert_manager/letsencrypt-clusterissuer.yaml @@ -5,7 +5,7 @@ metadata: name: letsencrypt-prod spec: acme: - email: your-email@example.com + email: ${letsencrypt_email} server: https://acme-v02.api.letsencrypt.org/directory privateKeySecretRef: name: letsencrypt-prod diff --git a/eks-gitops-platform/modules/cert_manager/main.tf b/eks-gitops-platform/modules/cert_manager/main.tf index d90c93b..02b2466 100644 --- a/eks-gitops-platform/modules/cert_manager/main.tf +++ b/eks-gitops-platform/modules/cert_manager/main.tf @@ -25,6 +25,8 @@ resource "helm_release" "cert_manager" { } resource "kubernetes_manifest" "letsencrypt_issuer" { - manifest = yamldecode(file("${path.module}/letsencrypt-clusterissuer.yaml")) + manifest = yamldecode(templatefile("${path.module}/letsencrypt-clusterissuer.yaml", { + letsencrypt_email = var.letsencrypt_email + })) depends_on = [helm_release.cert_manager] } diff --git a/eks-gitops-platform/modules/cert_manager/variables.tf b/eks-gitops-platform/modules/cert_manager/variables.tf index 902f10f..88a9a12 100644 --- a/eks-gitops-platform/modules/cert_manager/variables.tf +++ b/eks-gitops-platform/modules/cert_manager/variables.tf @@ -9,3 +9,8 @@ variable "chart_version" { type = string default = "v1.14.3" } + +variable "letsencrypt_email" { + description = "Email address for Let's Encrypt ACME registration and expiry notifications" + type = string +} diff --git a/eks-gitops-platform/modules/postgres/main.tf b/eks-gitops-platform/modules/postgres/main.tf index 4a19e5e..8b0f476 100644 --- a/eks-gitops-platform/modules/postgres/main.tf +++ b/eks-gitops-platform/modules/postgres/main.tf @@ -13,5 +13,15 @@ resource "helm_release" "postgres" { values = [file("${path.module}/values.yaml")] + set_sensitive { + name = "auth.postgresPassword" + value = var.postgres_password + } + + set_sensitive { + name = "auth.password" + value = var.postgres_user_password + } + depends_on = [kubernetes_namespace.app] } diff --git a/eks-gitops-platform/modules/postgres/values.yaml b/eks-gitops-platform/modules/postgres/values.yaml index d10920e..99860d0 100644 --- a/eks-gitops-platform/modules/postgres/values.yaml +++ b/eks-gitops-platform/modules/postgres/values.yaml @@ -5,9 +5,7 @@ architecture: standalone auth: enablePostgresUser: true - postgresPassword: "changeme" username: "testuser" - password: "testpassword" database: "testdb" primary: diff --git a/eks-gitops-platform/modules/postgres/variables.tf b/eks-gitops-platform/modules/postgres/variables.tf index a46b478..1ae2ca6 100644 --- a/eks-gitops-platform/modules/postgres/variables.tf +++ b/eks-gitops-platform/modules/postgres/variables.tf @@ -9,3 +9,15 @@ variable "chart_version" { type = string default = "12.5.2" } + +variable "postgres_password" { + description = "PostgreSQL superuser password" + type = string + sensitive = true +} + +variable "postgres_user_password" { + description = "PostgreSQL application user password" + type = string + sensitive = true +} diff --git a/eks-gitops-platform/scripts/one-shot-deploy.sh b/eks-gitops-platform/scripts/one-shot-deploy.sh index b5ce498..697c4a7 100755 --- a/eks-gitops-platform/scripts/one-shot-deploy.sh +++ b/eks-gitops-platform/scripts/one-shot-deploy.sh @@ -8,20 +8,16 @@ echo "==> EKS GitOps Platform — One-Shot Deploy" echo "" # Step 1: Bootstrap remote state if not already done -BOOTSTRAP_OUTPUT="$TF_ROOT/bootstrap/.terraform" -if [ ! -d "$BOOTSTRAP_OUTPUT" ]; then +echo "==> Checking bootstrap state..." +cd "$TF_ROOT/bootstrap" +if ! terraform output -raw state_bucket_name &>/dev/null 2>&1; then echo "==> Running bootstrap (first time only)..." + cd "$TF_ROOT" bash "$SCRIPT_DIR/bootstrap.sh" else - echo "==> Bootstrap already initialized. Checking outputs..." - cd "$TF_ROOT/bootstrap" - if ! terraform output -raw state_bucket_name &>/dev/null 2>&1; then - echo "==> Bootstrap state missing — re-running bootstrap..." - bash "$SCRIPT_DIR/bootstrap.sh" - else - echo "==> Bootstrap already complete, skipping." - fi + echo "==> Bootstrap already complete, skipping." fi +cd "$TF_ROOT" # Step 2: Initialize root module cd "$TF_ROOT" diff --git a/eks-gitops-platform/terraform.tfvars.example b/eks-gitops-platform/terraform.tfvars.example index 888c3c0..a2aa89c 100644 --- a/eks-gitops-platform/terraform.tfvars.example +++ b/eks-gitops-platform/terraform.tfvars.example @@ -12,6 +12,13 @@ vpc_cidr = "10.0.0.0/16" private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"] public_subnets = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"] +# ── PostgreSQL (sensitive — store in GitHub Secrets for CI) ────────────────── +postgres_password = "REPLACE_ME_strong_password_here" +postgres_user_password = "REPLACE_ME_app_user_password_here" + +# ── cert-manager / Let's Encrypt ───────────────────────────────────────────── +letsencrypt_email = "your@email.com" + # ── Datadog (sensitive — store in GitHub Secrets for CI) ───────────────────── # Get these from: https://app.datadoghq.com/organization-settings/api-keys datadog_api_key = "your-datadog-api-key" diff --git a/eks-gitops-platform/variables.tf b/eks-gitops-platform/variables.tf index 6f26aee..37b0091 100644 --- a/eks-gitops-platform/variables.tf +++ b/eks-gitops-platform/variables.tf @@ -40,6 +40,23 @@ variable "public_subnets" { default = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"] } +variable "postgres_password" { + description = "PostgreSQL superuser password — store in GitHub Secrets" + type = string + sensitive = true +} + +variable "postgres_user_password" { + description = "PostgreSQL application user password — store in GitHub Secrets" + type = string + sensitive = true +} + +variable "letsencrypt_email" { + description = "Email for Let's Encrypt ACME registration — must be a real address" + type = string +} + variable "datadog_api_key" { description = "Datadog API key — store in GitHub Secrets as DATADOG_API_KEY" type = string From 2d351ac4cd8435a5868e885535c13cf328afd298 Mon Sep 17 00:00:00 2001 From: Darin Ganitch Date: Tue, 19 May 2026 18:20:27 -0700 Subject: [PATCH 23/23] fix(ci): switch AWS auth to access key secrets; guard PR comment against missing plan file --- .github/workflows/terraform-apply.yml | 4 ++-- .github/workflows/terraform-plan.yml | 8 ++++++-- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/.github/workflows/terraform-apply.yml b/.github/workflows/terraform-apply.yml index 78c1b18..5d5be43 100644 --- a/.github/workflows/terraform-apply.yml +++ b/.github/workflows/terraform-apply.yml @@ -7,7 +7,6 @@ on: permissions: contents: read - id-token: write concurrency: group: terraform-apply @@ -30,7 +29,8 @@ jobs: - name: Configure AWS credentials uses: aws-actions/configure-aws-credentials@v4 with: - role-to-assume: ${{ secrets.AWS_ROLE_ARN }} + aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: us-west-2 - name: Terraform Init diff --git a/.github/workflows/terraform-plan.yml b/.github/workflows/terraform-plan.yml index 29b1afa..6d82bc2 100644 --- a/.github/workflows/terraform-plan.yml +++ b/.github/workflows/terraform-plan.yml @@ -8,7 +8,6 @@ on: permissions: contents: read pull-requests: write - id-token: write concurrency: group: terraform-plan-${{ github.ref }} @@ -31,7 +30,8 @@ jobs: - name: Configure AWS credentials uses: aws-actions/configure-aws-credentials@v4 with: - role-to-assume: ${{ secrets.AWS_ROLE_ARN }} + aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: us-west-2 - name: Terraform Format Check @@ -82,6 +82,10 @@ jobs: github-token: ${{ secrets.GITHUB_TOKEN }} script: | const fs = require('fs'); + if (!fs.existsSync(process.env.PLAN_PATH)) { + console.log('No plan output — skipping comment (plan did not run)'); + return; + } const plan = fs.readFileSync(process.env.PLAN_PATH, 'utf8'); const truncated = plan.length > 60000 ? plan.substring(0, 60000) + '\n\n...[truncated — see Actions logs for full output]'