Phase 4 — Kubernetes and DNS

[jenniferoko]

Please review

Full Documentation Design:

Phase_4_Kubernetes_DNS_Full_Design_Guide.docx

Excel Document:

Phase_4_Kubernetes_DNS_Acceptance_Gate.xlsx

Phase 4 Acceptance Gate – Engineer Sign-off

Overview

Phase 4 introduces two services with unique implementation considerations:

• Kubernetes provisioning is asynchronous and requires polling until the cluster reaches the Running state.
• DNS tests require a platform-approved internal test domain before implementation.

The engineer may proceed to Phase 5 only after all acceptance criteria below have been successfully completed.

---

Jennifer's Engineering Review

Before implementation begins, Jennifer should review the following design considerations with the engineer.

Kubernetes

Asynchronous Resource Provisioning

Unlike previous services, Kubernetes cluster creation is not immediate.

After a successful POST /kubernetes request, the API returns success while the cluster is still provisioning.

The test must not continue immediately.

Instead:

1. Create the cluster.
2. Poll the cluster endpoint.
3. Wait until

data.state == "Running"

4. Continue with remaining tests.

---

Required Timeout

Tests 45–47 must use

test.setTimeout(300_000);

which equals

5 minutes

This prevents failures caused by legitimate provisioning delays.

---

Polling Strategy

Implement a helper similar to:

waitForClusterRunning()

Retry:
GET /kubernetes/{slug}

Until:

state == Running

or

Timeout after 5 minutes

The helper should:

• retry every 10–15 seconds
• stop immediately when Running
• fail gracefully after timeout
• include useful timeout messages

---

DNS

Before implementation, confirm the test domain with the Platform Team.

Use an internal testing domain only.

Example:

smoke.test.internal

Do not use

• .com
• .net
• .org
• any public production domain

The smoke suite must never modify public DNS.

---

Test Matrix

Kubernetes (06-kubernetes.spec.ts)

Test	Description	Method	Endpoint	Expected Result
44	List clusters	GET	/kubernetes	HTTP 200, array
45	Create cluster	POST	/kubernetes	HTTP 200/201, data.slug present
46	Wait until cluster is Running	GET (polling)	/kubernetes/smoke-k8s-01	data.state == Running within 5 minutes
47	Stop cluster	PUT	/kubernetes/smoke-k8s-01/stop	HTTP 200, status=Success

---

DNS (07-dns.spec.ts)

Test	Description	Method	Endpoint	Expected Result
48	List domains	GET	/dns	HTTP 200, array
49	Create test domain	POST	/dns	HTTP 200/201, domain slug present
50	List DNS records	GET	/dns/smoke-test-domain/records	HTTP 200, array

---

Resource Lifecycle

Kubernetes

List Clusters
      │
      ▼
Create Cluster
      │
      ▼
Poll Until Running
      │
      ▼
Stop Cluster

---

DNS

List Domains
      │
      ▼
Create Test Domain
      │
      ▼
List DNS Records

---

Dependency Rules

Kubernetes

Tests execute in the following order.

44
 │
 ▼
45 Create Cluster
 │
 ▼
46 Wait Until Running
 │
 ▼
47 Stop Cluster

Test 47 must never execute until Test 46 confirms the cluster is Running.

---

DNS

48
 │
 ▼
49 Create Domain
 │
 ▼
50 List Records

Test 50 depends on successful domain creation.

---

Cleanup Requirements

Verify the following resources are removed after execution.

• smoke-k8s-*
• smoke-test-domain
• Temporary Kubernetes resources
• Temporary DNS records

Cleanup must execute even if earlier tests fail.

---

Acceptance Criteria

Category	Acceptance Criteria	Verification
Kubernetes Spec	06-kubernetes.spec.ts contains Tests 44–47	Code Review
DNS Spec	07-dns.spec.ts contains Tests 48–50	Code Review
Timeout	Tests 45–47 use test.setTimeout(300_000)	Code Review
Polling Helper	Waits until cluster state equals Running	Code Review
Cluster Lifecycle	Cluster is created, reaches Running state, then stops successfully	Tests 45–47
Polling Timeout	Polling fails cleanly after 5 minutes if cluster never reaches Running	Test Review
DNS Domain	Internal test domain approved by Platform Team	Platform Confirmation
DNS Lifecycle	Domain is created and records retrieved successfully	Tests 48–50
HTTP Responses	Expected status codes returned	Assertion Review
Cleanup	Temporary Kubernetes and DNS resources removed	Manual/API Verification
Regression	Full smoke suite (Tests 1–50) passes	CI Pipeline

---

Engineer Checklist

Required Before Phase 4 Sign-off

• 06-kubernetes.spec.ts completed
• 07-dns.spec.ts completed
• Tests 44–50 pass
• Kubernetes tests use 5-minute timeout
• Polling helper implemented
• Cluster reaches Running state before Stop test
• DNS test domain approved by Platform Team
• Internal domain used
• Cleanup removes temporary Kubernetes resources
• Cleanup removes temporary DNS resources
• Regression suite (Tests 1–50) passes
• Jennifer reviewed nightly execution report
• Code review approved

---

Resource Verification

Resource	Verified
smoke-k8s-* removed	- [ ]
Test DNS domain removed	- [ ]
Temporary DNS records removed	- [ ]

---

Engineer Sign-off

Field	Value
Engineer
Reviewer (Jennifer)
Date
Build / Release Version
Git Commit / PR
CI Pipeline Run

---

Final Result

• PASS – Approved for Phase 5
• FAIL – Rework Required

---

Reviewer Comments

______________________________________________________________

______________________________________________________________

______________________________________________________________

______________________________________________________________

---

Exit Rule

Phase 4 is complete only when:

• All Tests 44–50 pass.
• Kubernetes provisioning completes successfully.
• Polling confirms the cluster reaches Running.
• DNS tests use a Platform Team–approved internal domain.
• Temporary Kubernetes and DNS resources are removed.
• The complete smoke suite (Tests 1–50) passes.
• Jennifer reviews the nightly smoke test report and approves the implementation.

Only then may the engineer begin Phase 5.

[godsonten]

You correctly identified the two highest-risk points in this phase: Kubernetes is asynchronous, and DNS must not touch public infrastructure. The polling requirement, 5-minute timeout, and Platform Team DNS approval are the right controls.

I would still ask you to revise Phase 4 before QA sign-off.

The biggest gap is Kubernetes cleanup.

Your lifecycle ends at Stop Cluster, but stopping a cluster is not cleanup. A stopped cluster can still exist, consume resources, or block future runs. You need to explicitly require deletion of smoke-k8s-* clusters in teardown using the Swagger-confirmed DELETE endpoint.

Your cleanup section says temporary Kubernetes resources must be removed, but the test matrix does not include delete behavior, and the acceptance criteria do not explicitly say the cluster must be deleted. Add that clearly:

• cluster is registered for cleanup immediately after data.slug is returned
• teardown deletes the cluster even if polling or stop fails
• teardown tolerates 404 if the resource is already gone
• post-run verification confirms no smoke-k8s-* clusters remain

Serial execution

You also need to make serial execution explicit. These tests are dependency-based. Test 46 depends on Test 45, Test 47 depends on Test 46, and DNS Test 50 depends on Test 49. Tell the engineer to configure lifecycle groups as serial:

test.describe.configure({ mode: 'serial' });

Kubernetes polling

For Kubernetes polling, your helper design is good, but add one more requirement: the timeout failure should include the last observed state. If the cluster ends in Error, Failed, Provisioning, or any other unexpected state, the report should show that. Otherwise debugging the nightly report will be harder.

You should also define what happens if the cluster never reaches Running. Test 47 must not attempt to stop a cluster that never reached the required state. The failure should be clean and attributed to provisioning readiness, not a misleading stop failure.

DNS

For DNS, your domain approval rule is good, but it should be a hard precondition. Engineering should not start DNS implementation until the approved internal domain is documented. Do not leave smoke.test.internal as an assumed value unless Platform approves it.

I recommend adding an environment variable for this:

ZCP_SMOKE_DNS_DOMAIN=smoke.test.internal

The DNS cleanup also needs more precision. If Test 49 creates a domain, teardown must delete that domain. If records are created automatically or during setup, confirm whether deleting the domain removes the records. If not, the records must be deleted separately before deleting the domain.

There is also a naming mismatch to clean up. Test 49 says the example domain is smoke.test.internal, but Test 50 uses /dns/smoke-test-domain/records. Please clarify whether smoke-test-domain is the API slug generated from the approved domain, or whether to use the actual returned slug from Test 49. The engineer should not hardcode a guessed slug.

Payload details

You should also define the payload details the engineer needs for:

• Kubernetes cluster name
• project
• region
• Kubernetes version
• network
• any DNS cleanup details
• approved domain/payload

Final QA recommendation

Phase 4 is close, but it still needs revision before engineering. Add explicit Kubernetes deletion, strengthen teardown requirements, require serial lifecycle execution, improve polling failure reporting, clarify the DNS slug and cleanup behavior, and define the Kubernetes and DNS payload details. After those changes, I would approve Phase 4 for engineering.