feature(cluster): Add Nebius as an inference cluster provider#340
Open
haarchri wants to merge 3 commits into
Open
feature(cluster): Add Nebius as an inference cluster provider#340haarchri wants to merge 3 commits into
haarchri wants to merge 3 commits into
Conversation
Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
Contributor
Author
|
full crossplane traces: |
Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of your changes
Modelplane could provision inference clusters on GKE and EKS only. This adds Nebius: a new NebiusCluster XR and compose-nebius-cluster function provision an mk8s cluster with a VPC network, a system node group, GPU node groups, and GPU clusters for InfiniBand fabrics.
InferenceCluster gains
source: Nebius. Nebius has no server-side way to mint kubeconfig credentials, so the composed ProviderConfigs authenticate with the service account of the Nebius ClusterProviderConfig nameddefault, the same identity that provisions the cluster.ModelCache RWX storage mirrors the EFS integration on EKS. Nebius has no managed RWX storage, so the composition assembles it: a shared filesystem attached to every node group and mounted by cloud-init, Nebius's csi-mounted-fs-path CSI driver installed as a Helm release, and a modelplane-rwx-fs StorageClass pinned to that driver. The filesystem is a fixed 1024 GiB because Nebius filesystems are fixed size; there is no elastic option like EFS. The class is published on
status.cache.storageClassName.One fix reach beyond Nebius. The cluster compositions taint GPU nodes with
nvidia.com/gpu, but two DaemonSets the serving stack installs tolerate nothing by default: the NFD worker. Without the NFD worker on a GPU node, the node never gets thepci-10delabel the NVIDIA DRA driver's kubelet plugin schedules on, no ResourceSlices are published, and every GPU ResourceClaim stays unallocatable while all components look healthy.Breaking change:
nodePools[].fabricon InferenceCluster is now an object. It was a string enum (None, EFA):before:
with this PR:
Existing manifests using
fabric: EFAmust move tofabric: {type: EFA}. The infrastructure XRs (EKSCluster, NebiusCluster) are unchanged.Open questions:
crossplane project runinstalls provider-helm before prerequisites.yaml can be applied, so the ImageConfig's runtime config never attaches and the DeploymentRuntimeConfig has to be patched onto the Provider by hand... something i doing wrong ?! #:dancers:Validation:
Validated end to end on a real Nebius project: mk8s cluster and H100 node group up, serving stack installed, a 0.5B model served through the gateway, then a 72B AWQ model loaded from a ModelCache staged on the composed shared filesystem, created after the cluster existed.
Create credentials JSON
Create Secret + ClusterProviderConfig:
Fixes #
I have:
nix flake check(or./nix.sh flake check) and made sure it passes.git commit -s.