Local environment for testing IPAM against a real Milo control plane

[scotwells]

What & why

Adds a reusable local environment that runs IPAM wired to a real,
in-cluster Milo control plane, so we can exercise the full IPAM↔Milo path
locally — delegated authn/authz, quota enforcement, and entitlement → grant →
claim — which the standalone e2e (--enable-quota=false, no Milo) cannot
cover.

It proves, on a single kind cluster, that a project-scoped IPClaim:

1. binds synchronously with status.allocatedCIDR set, and
2. is actually quota-enforced — a Milo ResourceClaim is granted (not
   bypassed) and the project's AllowanceBucket decrements.

What you get

• config/overlays/milo-integration/ — additive IPAM overlay: --enable-quota=true,
  all three delegation kubeconfig flags pointed at the in-cluster milo-apiserver
  Service (via a milo-kubeconfig Secret), NetworkPolicy egress to
  milo-system:6443. dev/test-infra overlays are untouched.
• config/overlays/milo-integration/quota/ — quota.miloapis.com primitives
  applied to Milo: ResourceRegistration + ClaimCreationPolicy +
  GrantCreationPolicy that register the IPAM quotable type and auto-create the
  per-project grant / bucket / claim.
• rbac-tenant-user.yaml — Milo RBAC for the impersonated tenant user (IPAM
  delegates authz to Milo).
• task milo-integration:up (+ deploy-milo / deploy-ipam / provision-quota).
• docs/milo-integration.md — how to run it, the full-claim walkthrough, a
  coverage table vs the standalone e2e, and known issues.

End-to-end result (verified locally)

IPClaim status:  {"allocatedCIDR":"10.200.0.0/24","phase":"Bound", ...}
ResourceClaim:   ipclaim-int-claim-1  GRANTED=True  reason=QuotaAvailable
AllowanceBucket: limit=100 allocated=1 available=99

Key findings

• Grant controllers ship in milo-controller-manager. The full quota grant
  pipeline (resource-registration, resource-grant, resource-claim,
  allowance-bucket, claim/grant-creation policies) runs inside the single
  controller-manager via its multicluster manager — there is no separate
  services-controller-manager as in staging, so task dev:deploy brings up
  the whole pipeline.
• This Milo build ships only raw quota.miloapis.com primitives, not the
  higher-level services.miloapis.com catalog API. The existing
  config/components/service-catalog/ (Service / ServiceConfiguration) cannot
  be applied here; the overlay reproduces its effect with ResourceRegistration +
  policies directly.
• readyz passes with quota ON — the quota + APF (FlowSchema /
  PriorityLevelConfiguration) informers sync from Milo, the staging readyz
  dependency.

Known issues surfaced (IPAM-side, documented, not fixed here)

• ClaimCreationPolicy name templates must use requestInfo.name, not
  trigger.metadata.name — the IPClaim object handed to the quota plugin lacks
  a usable metadata key for CEL (related [SHOULD NOT HAPPEN] ... no type matching IPClaimSpec managedFields logs). origin/main includes a
  quota-object-convertor fix (Fix quota enforcement for IPAM claims #47) that may address this; the overlay uses the
  robust requestInfo.name regardless.
• Intermittent fatal error: concurrent map writes on the claim path (the
  unsynchronised-map failure mode noted around the MaxConns=10 cap in
  serve.go); the pod restarts and the retried claim succeeds.

Scope / safety

• No Go changes; internal/allocation/ stays zero-dep; forbidden-import rules intact.
• No .github/workflows/* changes — CI wiring is a follow-up.
• Draft: env/config only; needs a review of the dev-token Secret approach before
  reuse against any shared Milo.

🤖 Generated with Claude Code