security: narrow internal ingress CIDR (JIRA-4521)

[dylanratcliffe]

Summary

• Narrow internal ingress CIDR used for service/monitoring access.

Context

• JIRA-4521: Reduce internal exposure based on audit feedback.

Testing

• Terraform plan reviewed in CI.

Rollout / Risk

• If any internal tooling relies on the broader range, it may lose access; monitor health checks and alarms after merge.

[github-actions]

Caution

[High Risk] Shared customer whitelist security group creates direct external HTTPS reachability to the new API instance

The change adds 203.0.113.156/32 to the shared customer-api-access security group sg-03cf38efd953aa056 and also attaches that same group to the new EC2 instance github.com/overmindtech/terraform-example.aws_instance.module.api_access[0].aws_instance.api_server. That makes the new customer IP an additional direct HTTPS source to a workload-level EC2 security boundary, not just to a dedicated edge load balancer.

This is risky because our security requirements say EC2 instances must not be directly reachable from the internet, and our infrastructure notes warn that shared security groups have high fan-out. Even though the hypothesis overstates the effect on internal NLBs and peered-VPC resources, this change still expands external reachability to an instance-attached shared SG in production, increasing attack surface and violating least-privilege expectations for customer access control.
_{View reasoning tree here.}

Caution

[High Risk] New production EC2 instance can launch without an IAM role and with an unencrypted root volume

The change creates a new production EC2 instance, github.com/overmindtech/terraform-example.aws_instance.module.api_access[0].aws_instance.api_server, without explicitly attaching an IAM instance profile or enabling root volume encryption. Our security standards require machine access through IAM roles and require all EBS volumes to be encrypted at rest. Current neighboring production state shows this is not a theoretical concern: the similar ARM instance 540044833068.eu-west-2.ec2-instance.i-025efedc46bef3be1 is running with no IAM instance profile and an unencrypted root volume, so this module pattern already produces non-compliant instances.

This means the new server can come up in production with no role-based AWS credentials and an unencrypted EBS root disk, forcing any AWS access to be handled outside the approved IAM path and leaving data at rest non-compliant. The architecture-mismatch part of the hypothesis is not substantiated, but the same EC2 creation change still creates a real security and compliance failure through missing IAM attachment and missing encryption controls.
_{View reasoning tree here.}

Caution

[High Risk] Direct EIP-to-EC2 cutover will expose the new instance and break consumers of the old public endpoints

This change is moving external reachability onto a directly addressed EC2 instance by associating EIP 13.134.236.98 to the new module.api_access[0].aws_instance.api_server while the existing instances i-06bc09bcdf07eed7e and i-038912859bce496c7 lose their current public IP and DNS assignments. The new instance is explicitly configured to listen on 0.0.0.0:9090, and organizational policy says EC2 instances must not be directly reachable from the internet or used as public endpoints. Using a stable EIP on an instance instead of a managed public endpoint violates that standard and increases the attack surface.

At the same time, this is an endpoint cutover with no evidence of coordinated DNS migration or compatibility handling for consumers still calling 18.175.143.76, 3.8.134.93, or their EC2 public DNS names. When apply replaces or recreates those instances, those old public endpoints will change, and external scripts, dashboards, or hard-coded clients that still reference them will lose connectivity. The result is both a policy-breaking direct-public-IP exposure on the new path and an availability break for consumers tied to the old instance addresses.
_{View reasoning tree here.}

Caution

[High Risk] CIDR narrowing will block peered monitoring traffic and leave the new 9090 health path on a single backend

The change narrows sg-089e5107637083db5 from 10.0.0.0/8 to 10.0.0.0/16 on ports 8080, 443, and 9090, but the environment already has an active peered monitoring VPC vpc-096b686376892bb49 using 10.50.0.0/16. Its internal NLB mon-internal-terraform-example spans subnets 10.50.101.0/24 and 10.50.102.0/24 and forwards TCP 9090 traffic to api-health-terraform-example, so this change will cut off monitoring and health traffic from that VPC to resources protected by the shared internal-services group.

At the same time, the plan creates a single t4g.nano api_server in one subnet and attaches it as the new target for api-health-terraform-example. That makes the 9090 health path dependent on one instance in one AZ, and the narrowed CIDR removes the peered monitoring VPC's ability to reach it. The result is broken internal health and metrics visibility and a single-instance availability choke point for the new API access path.
_{View reasoning tree here.}

Signals

Routine → Multiple AWS infrastructure resources showing unusual infrequent update patterns, with load balancer target attachment resources at 1 event/week for the last 3 months and 2 events/week for the last 3 weeks, API server instance resources at 2 events/week for the last 3 months, and an elastic IP resource at 1 event/week for the last 3 months, which is rare compared to typical patterns.

_{Additional Change Details:} Items 147 Edges 254 model|risks_v6 ✨Encryption Key State Risk ✨KMS Key Creation