Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 

Repository files navigation

Zero-Credential Workload Identity

Part of a structured Azure platform engineering series building production-pattern infrastructure on a hub-and-spoke topology using Terraform.


Overview

A workload running in a spoke VNet authenticates to Azure Key Vault using a managed identity — no passwords, no connection strings, no credentials stored anywhere. Key Vault has no public internet exposure. All inter-spoke traffic routes through a Linux NVA in the hub. Every secret access is audited in a central Log Analytics workspace.

This project demonstrates three production patterns simultaneously:

  • Zero-credential authentication — workloads authenticate via managed identity through IMDS, with tokens issued and rotated entirely by the Azure platform
  • Controlled spoke-to-spoke routing — traffic between spokes is forced through a hub NVA via UDRs, enabling centralised inspection and symmetric routing
  • Private-first access — Key Vault is reachable only through a private endpoint; DNS resolution returns a private IP from all linked VNets

Architecture

┌─────────────────────────────────────────────────────────────────┐
│  vnet-hub (rg-platform-connectivity)                            │
│                                                                 │
│  ┌──────────────┐   ┌──────────────────────────────────────┐   │
│  │ snet-nva     │   │ privatelink.vaultcore.azure.net       │   │
│  │ 10.0.3.0/28  │   │ Private DNS Zone (hub-hosted)         │   │
│  │              │   │ Linked: hub, app-dev, data-access     │   │
│  │ vm-nva-hub   │   └──────────────────────────────────────┘   │
│  │ 10.0.3.4     │                                               │
│  │ IP fwd: ON   │                                               │
│  └──────┬───────┘                                               │
└─────────┼───────────────────────────────────────────────────────┘
          │ VNet Peering (existing)
    ┌─────┴──────────────────────────────┐
    │                                    │
    ▼                                    ▼
┌───────────────────────────┐  ┌───────────────────────────────┐
│ vnet-app-dev              │  │ vnet-data-access               │
│ rg-app-dev                │  │ rg-data-access                 │
│                           │  │                                │
│ ┌───────────────────────┐ │  │ ┌────────────────────────────┐ │
│ │ snet-app              │ │  │ │ snet-data-access            │ │
│ │ 10.1.1.0/24           │ │  │ │ 10.2.1.0/24                │ │
│ │                       │ │  │ │                             │ │
│ │ vm-workload-appdev    │ │  │ │ kv-platform-[suffix]        │ │
│ │ + mi-workload-appdev  │ │  │ │ Private Endpoint 10.2.1.x   │ │
│ │                       │ │  │ │                             │ │
│ │ UDR: 10.2.0.0/16      │ │  │ │ UDR: 10.1.0.0/16           │ │
│ │ → NVA 10.0.3.4        │ │  │ │ → NVA 10.0.3.4             │ │
│ └───────────────────────┘ │  │ └────────────────────────────┘ │
└───────────────────────────┘  └────────────────────────────────┘

End-to-End Request Flow

1. VM calls IMDS (169.254.169.254) → receives OAuth2 token for managed identity
2. VM resolves kv-[name].vault.azure.net → Private DNS Zone returns 10.2.1.x
3. Packet leaves snet-app → UDR forces next-hop to NVA (10.0.3.4)
4. NVA forwards packet across hub→data-access peering
5. Private endpoint receives request → Key Vault validates token against RBAC
6. Secret returned via same symmetric path (UDR on snet-data-access → NVA → app-dev)
7. AuditEvent log written to law-platform

Infrastructure Components

Resource Location Purpose
snet-nva (10.0.3.0/28) rg-platform-connectivity Dedicated NVA subnet — no NSG by design
vm-nva-hub rg-platform-connectivity Linux software router; IP forwarding at NIC and OS level
rt-snet-app rg-app-dev UDR: 10.2.0.0/16 → NVA
rt-snet-data-access rg-data-access UDR: 10.1.0.0/16 → NVA (return path)
mi-workload-appdev rg-app-dev User-Assigned Managed Identity
privatelink.vaultcore.azure.net rg-platform-connectivity Hub-hosted Private DNS Zone; linked to all three VNets
kv-platform-[suffix] rg-data-access Key Vault; public access disabled; RBAC model
pe-kv-platform-[suffix] rg-data-access Private endpoint in snet-data-access
vm-workload-appdev rg-app-dev Workload VM carrying managed identity

Existing Infrastructure (referenced, not deployed)

Resource Location
vnet-hub rg-platform-connectivity
vnet-app-dev rg-app-dev
vnet-data-access rg-data-access
snet-app rg-app-dev
snet-data-access rg-data-access
law-platform rg-platform-management
VNet peerings (hub↔app-dev, hub↔data-access) Established in azure-cross-spoke-data-access-private-dns

Repository Structure

.
├── main.tf                          # Root module — wires all modules together
├── variables.tf                     # Input variable declarations
├── outputs.tf                       # Root outputs
├── terraform.tfvars                 # Variable values (not committed)
├── data.tf                          # Data sources for existing infrastructure
├── versions.tf                      # Provider and Terraform version constraints
│
└── modules/
    ├── networking/
    │   ├── hub_nva_subnet/          # snet-nva in vnet-hub
    │   │   ├── main.tf
    │   │   ├── variables.tf
    │   │   └── outputs.tf
    │   ├── nva/                     # Linux NVA VM + cloud-init
    │   │   ├── main.tf
    │   │   ├── variables.tf
    │   │   ├── outputs.tf
    │   │   └── cloud-init.yaml
    │   ├── udr_appdev/              # Route table for snet-app
    │   │   ├── main.tf
    │   │   ├── variables.tf
    │   │   └── outputs.tf
    │   └── udr_dataaccess/          # Route table for snet-data-access
    │       ├── main.tf
    │       ├── variables.tf
    │       └── outputs.tf
    ├── identity/
    │   └── managed_identity/        # User-Assigned Managed Identity
    │       ├── main.tf
    │       ├── variables.tf
    │       └── outputs.tf
    ├── dns/
    │   └── private_dns_kv/          # Private DNS Zone + VNet links
    │       ├── main.tf
    │       ├── variables.tf
    │       └── outputs.tf
    ├── keyvault/
    │   └── vault/                   # Key Vault + PE + RBAC + diagnostics
    │       ├── main.tf
    │       ├── variables.tf
    │       └── outputs.tf
    └── compute/
        └── workload_vm/             # Workload VM + NSG rules + cloud-init
            ├── main.tf
            ├── variables.tf
            ├── outputs.tf
            └── cloud-init.yaml

Key Design Decisions

Why no NSG on the NVA subnet

NSG evaluation occurs at the subnet boundary — before the NVA can inspect or forward a packet. An NSG that denies a flow the NVA should route produces a silent drop indistinguishable from a UDR misconfiguration. Traffic policy on the NVA subnet is enforced at the OS level, providing full visibility into forwarding decisions.

Why User-Assigned over System-Assigned Managed Identity

A System-Assigned identity is bound to the VM's lifecycle — destroying the VM destroys the identity and all its role assignments. A User-Assigned identity is an independent resource: it survives VM rebuilds, can be pre-created, and can be attached to multiple workloads. Role assignments made against it persist across VM replacements.

Why RBAC authorisation over Key Vault Access Policies

Access Policies are a legacy model scoped only to the vault resource. The RBAC model is consistent with all other Azure resource authorisation, supports finer-grained scoping (vault, secret, or key level), and is Microsoft's current recommended pattern for all new deployments. Network proximity alone does not grant access — a valid role assignment is required.

Why the Private DNS Zone lives in the hub

Hosting Private DNS Zones in the hub and linking outward to spokes is the Azure Landing Zone centralised DNS pattern. A single zone per service type, managed in one place, linked to all spokes that need resolution. Adding a new spoke requires only a new VNet link — no new zone, no duplication.

Why both spoke UDRs are required

Azure VNet peering is not transitive — spokes cannot reach each other through the hub without explicit routing. UDRs force both the forward path (app-dev → data-access) and the return path (data-access → app-dev) through the NVA. Without the return path UDR, TCP sessions cannot establish — the SYN goes through the NVA but the SYN-ACK does not, producing an asymmetric routing failure.


Prerequisites

  • Terraform >= 1.5.0
  • Azure CLI authenticated (az login)
  • Existing hub-and-spoke topology from Project 1:
    • vnet-hub, vnet-app-dev, vnet-data-access
    • VNet peerings: hub↔app-dev, hub↔data-access
    • Log Analytics workspace
    • `storage account for Terraform state
  • SSH key pair at ~/.ssh/azure-lab-vm (public key read at plan time)
  • Key Vault Secrets Officer role on your user principal (required to create secrets — Key Vault blocks management plane operations from non-private networks)

Usage

# Clone and navigate to root
git clone https://github.com/moshstaq/az-zero-credential-workload-identity
cd az-zero-credential-workload-identity

# Initialise backend
terraform init

# Review planned changes
terraform plan

# Apply (full deployment)
terraform apply

Note: VMs should be deallocated after validation to stay within free tier limits. Two B1s VMs running concurrently exceed the 750-hour monthly allowance in approximately 15 days.

# Deallocate both VMs after validation
az vm deallocate --resource-group rg-platform-connectivity --name vm-nva-hub
az vm deallocate --resource-group rg-app-dev --name vm-workload-appdev

# Start when needed
az vm start --resource-group rg-platform-connectivity --name vm-nva-hub
az vm start --resource-group rg-app-dev --name vm-workload-appdev

Validation

After deployment, the following checks confirm the full chain is working.

1. Effective route confirms NVA is next hop

az network nic show-effective-route-table \
  --resource-group rg-app-dev \
  --name nic-vm-workload-appdev \
  --query "value[?contains(addressPrefix[0], '10.2')]" \
  -o table

Expected: NextHopType = VirtualAppliance, NextHopIpAddress = 10.0.3.4

2. NVA IP forwarding is active

az vm run-command invoke \
  --resource-group rg-platform-connectivity \
  --name vm-nva-hub \
  --command-id RunShellScript \
  --scripts "sysctl net.ipv4.ip_forward" \
  --query "value[0].message" \
  -o tsv

Expected: net.ipv4.ip_forward = 1

3. DNS resolves to private IP

az vm run-command invoke \
  --resource-group rg-app-dev \
  --name vm-workload-appdev \
  --command-id RunShellScript \
  --scripts "nslookup <kv-name>.vault.azure.net" \
  --query "value[0].message" \
  -o tsv

Expected: address in 10.2.1.0/24

4. Public endpoint is blocked

curl -v --max-time 10 https://<kv-name>.vault.azure.net

Expected: connection timeout or network error (not 401/403)

5. Managed identity retrieves secret

az vm run-command invoke \
  --resource-group rg-app-dev \
  --name vm-workload-appdev \
  --command-id RunShellScript \
  --scripts "bash /home/azureadmin/validate.sh" \
  --query "value[0].message" \
  -o tsv

Expected:

==> Logging in with managed identity...
==> Retrieving secret from Key Vault...
==> Secret value: project2-validation-value
==> Validation complete.

6. RBAC is the enforcement mechanism

Remove the role assignment and confirm access is denied, then restore it. Network proximity alone does not grant access.

7. Audit trail in Log Analytics

AzureDiagnostics
| where ResourceType == "VAULTS"
| where OperationName == "SecretGet"
| project TimeGenerated, CallerIPAddress, identity_claim_oid_g, ResultType
| order by TimeGenerated desc
| take 5

Expected: CallerIPAddress is the workload VM's private IP, ResultType is Success.


Cost Considerations

Resource Cost
B1s VMs (×2) Free tier: 750 hrs/month total. Deprovision after validation.
Key Vault operations Free tier: 10,000 ops/month
Private Endpoint ~$7.30/month while deployed
VNet peering data transfer $0.01/GB intra-region — negligible at lab scale
Log Analytics ingestion Free tier: 5 GB/month

Total estimated cost for a validation session of a few hours: under $1.


Planned Extensions

Extension 1 — Secret rotation with zero-downtime retrieval Demonstrate that secrets updated in Key Vault are picked up by the workload on the next fetch without any restart or reconfiguration. Validates the operational value of the managed identity pattern beyond initial deployment.

Extension 2 — Azure Policy enforcement of diagnostic settings Write a policy definition in Terraform that audits any Key Vault missing a diagnostic setting targeting law-platform. Assign at resource group scope. Introduces policy-as-code and bridges into the governance pillar — the natural next project.


Author

Mosh — Cloud Engineering. Building production-pattern Azure infrastructure independently as a portfolio of real engineering work.

github.com/moshstaq

About

A workload running in a spoke VNet that authenticates to Azure Key Vault in a different VNet using a managed identity

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages