validateExclusiveAuthSettings treats Arc and bootstrap-token as mutually exclusive

[nnamuhcs]

Hi team, second of three related observations on the Arc-mode kubelet auth path. Cross-links to the others at the bottom.

What we observed

validateExclusiveAuthSettings in pkg/config/validate.go treats arc.enabled and bootstrapToken as mutually exclusive. Picking either appears to disable the other across the whole agent.

In our scenario these two felt like they were doing different jobs:

• arc.enabled controls the agent's own Azure machine identity (used for enrich-cluster-config, assignRBACRoles, AKS RP calls, etc).
• bootstrapToken controls how kubelet authenticates to the API server.

When we picked bootstrapToken to work around the MI exec issue (see related issue below), Arc was also disabled, which disabled enrich-cluster-config. The agent then no longer refreshed the cluster CA on its own, and when the API-server CA rotated (this happened after we enabled AAD on the cluster), the bootstrap kubeconfig went stale and kubelet bootstrap failed until we manually re-fetched the CA.

Impact for our scenario

The Arc-mode MI happy path doesn't yet work end-to-end for us (separate issue), so the only path that actually completes is "bootstrap-token for kubelet". But that path also disables Arc for the agent, which gives up enrich-cluster-config and leaves us with a stale CA on rotation. Feels like we're being forced to pick one cost or the other.

One possible direction

Would it be reasonable to treat these two as orthogonal? Roughly:

arc.enabled	bootstrapToken	node.kubelet.auth.mode	Resulting behavior
true	unset	mi	Arc-driven everything; kubelet uses MI exec
true	set	bootstraptoken	Arc for the agent's Azure-side ops (incl. enrich-cluster-config), bootstrap-token brings kubelet up
false	set	bootstraptoken	"Bring-your-own-everything" path (existing VM demos)

The middle row would unblock us today without depending on the MI exec fix landing first. Totally happy to be told we're holding it wrong if there's a reason these are exclusive that we missed.

Version tested: v0.1.0 (git 65d8d38).

---

Related on this auth path (tightly coupled)

• Kubelet MI exec credential does not propagate Arc HIMDS env vars #185 — Kubelet MI exec credential does not propagate Arc HIMDS env vars
• Agent could auto-mint kubelet bootstrap-token when Arc/MI/SP credentials are available #187 — Agent could auto-mint kubelet bootstrap-token when Arc/MI/SP credentials are available

---

Other observations from the same Pi-rack edge demo

• Arc machine RG and AKS cluster RG must match (hard-coded in v0.1.0) #182 — Arc machine RG and AKS cluster RG must match (hard-coded in v0.1.0)
• Agent should preflight RBAC permissions before assignRBACRoles #183 — Agent should preflight RBAC permissions before assignRBACRoles
• Hostname casing not normalized; PascalCase machine names fail node validation #184 — Hostname casing not normalized; PascalCase machine names fail node validation
• Kubelet MI exec credential does not propagate Arc HIMDS env vars #185 — Kubelet MI exec credential does not propagate Arc HIMDS env vars
• Agent could auto-mint kubelet bootstrap-token when Arc/MI/SP credentials are available #187 — Agent could auto-mint kubelet bootstrap-token when Arc/MI/SP credentials are available
• kube1 systemd-nspawn launcher passes --network-veth but actually shares host netns #188 — kube1 systemd-nspawn launcher passes --network-veth but actually shares host netns