Skip to content

feat: default model to Qwen3.6-35B-A3B + GPU Operator v26.3 + H100 - #23

Closed
Hadar301 wants to merge 1 commit into
rh-ai-quickstart:mainfrom
Hadar301:feat/defaults
Closed

feat: default model to Qwen3.6-35B-A3B + GPU Operator v26.3 + H100#23
Hadar301 wants to merge 1 commit into
rh-ai-quickstart:mainfrom
Hadar301:feat/defaults

Conversation

@Hadar301

Copy link
Copy Markdown
Collaborator

Summary

  • Upgrade NVIDIA GPU Operator default from v25.3 → v26.3 (v26.3.3 validated on a fresh ROSA HCP cluster).
  • Switch default GPU target from L40S (g6e.2xlarge) to H100 (p5.4xlarge) in Terraform and Helm defaults; strip redundant ROSA/ARO value overrides that now match the base, and document g6e.12xlarge (4× L40S, tensor-parallel-size=4) as an alternative.
  • Use the custom vLLM runtime (v0.19.0) as the default for all clouds, replacing the RHOAI-bundled runtime. Promotes the validated ARO stack to the base.

Test plan

  • helm template charts/pca-ai-serving -f charts/pca-ai-serving/values-rosa.yaml — confirmed model ID, served alias, max-model-len=262144, vllm-openai:v0.19.0, H100 resource sizing, extraEnv, tensor-parallel-size=1 all render correctly
  • helm template charts/pca-ai-serving -f charts/pca-ai-serving/values-aro.yaml — confirmed ARO overlay still renders correctly with its gpuProduct override (NVIDIA-H100-NVL)
  • GPU Operator v26.3.3 installed successfully on a fresh ROSA HCP 4.21.7 cluster (CSV in Succeeded phase)
  • ArgoCD App-of-Apps from feat/defaults branch resolved and synced all child apps on the test cluster
  • pca-operators Synced/Healthy — GPU Operator, RHOAI, DevSpaces, cert-manager all installed
  • pca-devspaces / pca-platform-config OutOfSync is controller-drift only (normal), not caused by these changes
  • Live smoke test against running model — blocked by p5.4xlarge provisioning issue on ROSA HCP sandbox (ROSA's management cluster does not issue EC2 RunInstances for p5 instances; ARO cluster with H100 already validates the same model/runtime stack in production)

- Promote Qwen/Qwen3.6-35B-A3B-FP8 as the base default model; keep old
  model ID as servedName alias for backward compatibility with IDEs/smoke tests
- Upgrade NVIDIA GPU Operator default channel v25.3 -> v26.3
- Switch default GPU target to H100 (p5.4xlarge) in Terraform and Helm;
  add gpu_pool_name variable; document g6e.12xlarge (4x L40S) as alternative
- Strip redundant ROSA/ARO value overrides that now match base defaults
- Use custom vLLM runtime (v0.19.0) as the default for all clouds;
  set maxModelLen=262144 (256k context window)
- Update observability cost references L40S/$1.86 -> H100/$4.50
@Hadar301
Hadar301 requested review from itay1551 and manujoy7 July 30, 2026 12:06
@Hadar301 Hadar301 self-assigned this Jul 30, 2026
@Hadar301 Hadar301 closed this Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant