feat: evaluate Dynamic Resource Allocation (DRA) for GPU/TPU scheduling

[mpeterson]

Background

Kubernetes 1.34 graduated Dynamic Resource Allocation (DRA) to GA. DRA enables Kubernetes to natively understand, allocate, and share resources like GPUs, TPUs, NICs, and FPGAs with finer granularity than the current device plugin model.

Current State

• GPU scheduling uses nvidia-device-plugin DaemonSet (kubernetes/infra/nvidia-device-plugin/)
• TPU (Coral Edge) uses udev rules in talconfig.yaml
• Workers talos-01 and talos-02 have NVIDIA GPUs, talos-01/02 have Coral Edge TPUs

Opportunity

DRA could replace nvidia-device-plugin with native K8s resource allocation:

• More flexible GPU sharing and partitioning
• Better scheduling decisions (allocate GPU + memory together)
• Simpler stack (no separate device plugin DaemonSet)

Prerequisites

• Cluster running K8s ≥ 1.34 (in progress via sequential upgrade)
• NVIDIA DRA driver (check nvidia/k8s-dra-driver for Talos compatibility)

Action

Evaluate whether DRA is mature enough and supported on Talos to replace nvidia-device-plugin. Consider impact on existing GPU workloads (Ollama, Plex HW transcoding).