Skip to content

feat(auth): Authenticate existing clusters via their declared cloud identity#324

Merged
negz merged 2 commits into
modelplaneai:mainfrom
haarchri:feature/auth-existing-clusters
Jul 7, 2026
Merged

feat(auth): Authenticate existing clusters via their declared cloud identity#324
negz merged 2 commits into
modelplaneai:mainfrom
haarchri:feature/auth-existing-clusters

Conversation

@haarchri

@haarchri haarchri commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Description of your changes

Bringing your own cluster with an identitySecretRef always authenticated as a Google credential, whatever the cloud. So a Nebius or AWS credential got the wrong type, the providers couldn't log in, and the cluster never went Ready, with no upfront error.

identitySecretRef now takes a type for the cloud identity (default GoogleApplicationCredentials, so existing setups are unchanged). It's threaded through to both ProviderConfigs, and an unsupported type is rejected at apply time. GKE stays Google as before.

Nebius needs provider builds carrying the NebiusServiceAccountCredentials identity (crossplane-contrib/provider-kubernetes#499), so both providers are pinned to pre-release builds for now. Tested end-to-end against a real Nebius cluster.

Depends on crossplane-contrib/provider-kubernetes#499.

Fixes #321

I have:

  • Read and followed Modelplane's contribution process.
  • Run nix flake check (or ./nix.sh flake check) and made sure it passes.
  • Added or updated tests covering any composition function changes.
  • Signed off every commit with git commit -s.

How this PR gets tested

tested servingStack only

kubectl get objects.kubernetes.m.crossplane.io -A
NAMESPACE           NAME                                    KIND                       PROVIDERCONFIG                           SYNCED   READY   AGE
modelplane-system   byo-nebius-serving-stack-0e926c5b737d   EnvoyProxy                 byo-nebius-serving-stack-cluster-88e24   True     True    7m46s
modelplane-system   byo-nebius-serving-stack-2e7018ecb766   CustomResourceDefinition   byo-nebius-serving-stack-cluster-88e24   True     True    7m45s
modelplane-system   byo-nebius-serving-stack-36360f7ba6a3   CustomResourceDefinition   byo-nebius-serving-stack-cluster-88e24   True     True    7m46s
modelplane-system   byo-nebius-serving-stack-802a019b1c68   ResourceQuota              byo-nebius-serving-stack-cluster-88e24   True     True    7m46s
modelplane-system   byo-nebius-serving-stack-817ff8965857   Namespace                  byo-nebius-serving-stack-cluster-88e24   True     True    7m47s
modelplane-system   byo-nebius-serving-stack-a3e7e7459bca   GatewayClass               byo-nebius-serving-stack-cluster-88e24   True     True    7m45s
modelplane-system   byo-nebius-serving-stack-add372c2a494   CustomResourceDefinition   byo-nebius-serving-stack-cluster-88e24   True     True    7m46s
...
NAMESPACE           NAME                                    CHART                    VERSION   SYNCED   READY   STATE      REVISION   DESCRIPTION                                                                           AGE
modelplane-system   byo-nebius-serving-stack-0142ef6eea41   lws                      v0.8.0    True     True    deployed   1          Install complete                                                                      26m
...

on the nebius side:

PROJECT_ID=<your-project-id>
CLUSTER_ID=<your-cluster-id> 

SA_ID=$(nebius iam service-account create --parent-id "$PROJECT_ID" \
          --name modelplane-sa --format json | jq -r '.metadata.id')

nebius iam auth-public-key generate \
  --parent-id "$PROJECT_ID" --service-account-id "$SA_ID" \
  --output ~/.nebius/nebius-raw.json

jq '{
  private_key:   ."subject-credentials"."private-key",
  public_key_id: ."subject-credentials".kid,
  account_id:    ."subject-credentials".sub
}' ~/.nebius/nebius-raw.json > credentials.final.json

nebius mk8s cluster get-credentials \
  --id "$CLUSTER_ID" --external --kubeconfig ./nebius-kubeconfig.yaml --force

inside the controlplane where modelplane is installed:

kubectl create namespace modelplane-system --dry-run=client -o yaml | kubectl apply -f -

kubectl create secret generic byo-nebius-kubeconfig -n modelplane-system \
  --from-file=kubeconfig=nebius-kubeconfig.yaml \
  --dry-run=client -o yaml | kubectl apply -f -

kubectl create secret generic byo-nebius-creds -n modelplane-system \
  --from-file=credentials.json=credentials.final.json \
  --dry-run=client -o yaml | kubectl apply -f -

inside nebius kubernetes cluster:

kubectl --kubeconfig ./nebius-kubeconfig.yaml apply -f - <<'EOF'
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: modelplane-provider-admin
subjects:
- kind: User
  name: serviceaccount-e00f33e17xwqtc03xc
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io
EOF

setup serving stack:

kubectl apply -f - <<'EOF'
apiVersion: infrastructure.modelplane.ai/v1alpha1
kind: ServingStack
metadata:
  name: byo-nebius-serving-stack
  namespace: modelplane-system
spec:
  secrets:
  - type: Kubeconfig
    name: byo-nebius-kubeconfig
    key: kubeconfig
  - type: NebiusServiceAccountCredentials
    name: byo-nebius-creds
    key: credentials.json
EOF

@negz negz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @haarchri, looks good overall but a few suggestions.

Comment thread apis/inferenceclusters/definition.yaml Outdated
Comment thread apis/servingstacks/definition.yaml Outdated
Comment thread functions/compose-inference-cluster/function/fn.py Outdated
Comment thread functions/compose-inference-cluster/function/fn.py Outdated
Comment thread functions/compose-inference-cluster/function/fn.py Outdated
…dentity

Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
@negz negz merged commit 8cdf805 into modelplaneai:main Jul 7, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Identity credentials for existing clusters are always typed as GCP

2 participants