docs: restructure operator guide index into navigation hub (#1036)

## Summary

The current `operator-guides/index.md` is the full platform
configuration reference, not a navigation hub. Operators arriving at the
guide have no entry point — they land in a wall of YAML with no context
for where to start.

Changes:
- Rename `index.md` → `platform-setup.md` (content unchanged, just
renamed)
- Create new `index.md` as a navigation hub with:
- A getting-started reading path for fresh deployments (platform setup →
register cluster → serving → auth)
- Organized tables linking to every operator guide by category (platform
config, jobs, serving, UI, integrations, operations, ingester)

Part of the operator/contributor guide improvements proposed in #1033.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Austin Greco <austingreco@gmail.com>

Author: 3 people

docs/operator-guides/index.md

@@ -1,260 +1,62 @@
-# Platform Setup Guide
+# Operator Guides
 
-This guide describes how to configure **Michelangelo server components** in Kubernetes cluster. It focuses on the **configuration surfaces** (ConfigMaps, fields, and key parameters).
+These guides cover deploying, configuring, and integrating Michelangelo in a Kubernetes environment. They target platform engineers and infrastructure operators who are responsible for running Michelangelo in production and for connecting it to the broader ML infrastructure their teams already use — experiment tracking, model registries, compute clusters, schedulers, and serving frameworks.
 
-# Overview
+## Getting Started
 
-Michelangelo consists of four core server components:
+For a fresh deployment, follow this recommended reading order:
 
-1. **API Server** – Central gRPC API
-2. **Controller Manager** – Kubernetes controllers
-3. **Worker** – Workflow execution (Temporal workers + compute integration)
-4. **UI + Envoy** – Frontend and proxy
+1. **[Platform Setup](platform-setup.md)** — configure each component (API server, controller manager, worker, UI/Envoy) via ConfigMaps and Kustomize overlays
+2. **[Register a Compute Cluster](jobs/register-a-compute-cluster-to-michelangelo-control-plane.md)** — connect an existing Kubernetes cluster so Michelangelo can dispatch Ray and Spark jobs to it
+3. **[Cluster Setup for Serving](serving/cluster-setup.md)** — enable model inference on a local or remote cluster
+4. **[Authentication](authentication.md)** — connect an identity provider and configure RBAC before opening to users
 
-Each component exposes server-side configuration through ConfigMaps and overlays.
+## Platform Configuration
 
-This document explains:
+| Guide | Description |
+|-------|-------------|
+| [Platform Setup](platform-setup.md) | ConfigMaps and key fields for API server, controller manager, worker, and UI/Envoy |
+| [API Framework](api-framework.md) | Architecture overview of the Michelangelo API and control plane |
 
-* Where each component's configuration lives
-* What fields can be customized
-* What each field means
-* How to apply changes using Kustomize overlays
+## Jobs & Compute
 
-# Michelangelo Service architecture diagram
+| Guide | Description |
+|-------|-------------|
+| [Jobs Overview](jobs/index.md) | Ray and Spark job lifecycle, compute selection, and observability |
+| [Register a Compute Cluster](jobs/register-a-compute-cluster-to-michelangelo-control-plane.md) | Connect an existing Kubernetes cluster to the Michelangelo control plane |
+| [Run a Pipeline on a Compute Cluster](jobs/run-uniflow-pipeline-on-compute-cluster.md) | Submit and monitor a Uniflow pipeline on a registered cluster |
+| [Extend the Job Scheduler](jobs/extend-michelangelo-batch-job-scheduler-system.md) | Custom scheduling backends (Kueue, Volcano) and assignment strategies |
 
-The following diagram shows the relationship between each of the services in Michelangelo eco-system.
+## Model Serving
 
-![Michelangelo Service Architecture](./images/ma-service-architecture.png)
+| Guide | Description |
+|-------|-------------|
+| [Serving Overview](serving/index.md) | InferenceServer and Deployment lifecycle, architecture |
+| [Cluster Setup for Serving](serving/cluster-setup.md) | Configure a cluster for inference |
+| [Integrate a Custom Backend](serving/integrate-custom-backend.md) | Plugin interfaces for Triton, vLLM, TensorRT-LLM, and custom frameworks |
 
-# Server Configuration
+## UI
 
-## API Server Configuration
+| Guide | Description |
+|-------|-------------|
+| [Deploying the UI](ui/deploying-michelangelo-ui.md) | Deploy the Michelangelo web UI to Kubernetes |
+| [Local UI Development](ui/local-development-setup.md) | Run the UI locally for development |
 
-### **Key Fields**
+## Integrating with Your ML Stack
 
-```yaml
-apiserver:
-  yarpc:
-    host: 0.0.0.0
-    port: 15566
-  k8s:
-    qps: 300
-    burst: 600
-  metadataStorage:
-    enableMetadataStorage: false
-  crdSync:
-    enableCRDUpdate: true
-    enableIncompatibleUpdate: false
-```
+Michelangelo is designed to be adopted alongside existing ML infrastructure. These guides cover how to connect Michelangelo to the systems your teams already use.
 
-### **Field Explanations**
+| Guide | Description |
+|-------|-------------|
+| [Custom Serving Backend](serving/integrate-custom-backend.md) | Add support for any inference framework — Triton, vLLM, TensorRT-LLM, or your own |
+| [Custom Job Scheduler](jobs/extend-michelangelo-batch-job-scheduler-system.md) | Replace or extend the job scheduler — integrate Kueue, Volcano, or a custom assignment strategy |
+| [Register a Compute Cluster](jobs/register-a-compute-cluster-to-michelangelo-control-plane.md) | Connect an existing Kubernetes cluster so Michelangelo can dispatch jobs to it |
 
-| Field | Description |
-| ----- | ----- |
-| `yarpc.host/port` | gRPC bind address + port |
-| `k8s.qps/burst` | Throttling limits for Kubernetes API calls |
-| `enableMetadataStorage` | Enables metadata persistence |
-| `enableCRDUpdate` | Controls whether CRDs can be sync'd |
-| `enableIncompatibleUpdate` | Allows breaking CRD changes (use only during major migrations) |
+## Operations
 
-## Controller Manager Configuration
+| Guide | Description |
+|-------|-------------|
+| [Authentication](authentication.md) | OIDC identity provider setup, RBAC, session configuration, multi-tenant isolation |
+| [Compliance](compliance.md) | SOC 2, GDPR, and HIPAA configuration |
+| [Troubleshooting](troubleshooting.md) | Common failure modes and `kubectl` diagnostic commands |
 
-### **Key Fields**
-
-```yaml
-controllermgr:
-  metricsBindAddress: 8091
-  healthProbeBindAddress: 8081
-  leaderElection: false
-  leaderElectionID: michelangelo.your-organization.com
-  port: 9443
-
-controllers:
-  rayCluster:
-    k8sQps: 300
-    k8sBurst: 600
-
-minio:
-  awsRegion: ap-southeast-1
-  awsEndpointUrl: s3.ap-southeast-1.amazonaws.com
-  useIam: true
-
-workflowClient:
-  service: temporal-frontend
-  host: temporal.your-domain.com:7233
-  transport: grpc
-  domain: uniflow
-```
-
-### **Field Explanations**
-
-| Field | Description |
-| ----- | ----- |
-| `metricsBindAddress` | Controller metrics port |
-| `healthProbeBindAddress` | Health check port |
-| `leaderElection` | Enable for production HA |
-| `minio.*` | S3 / MinIO backend configuration |
-| `workflowClient.*` | Temporal client configuration |
-| `controllers.*` | Each controller components' configuration |
-
-## Worker Configuration
-
-### **Key Fields**
-
-```yaml
-worker:
-  address: michelangelo-apiserver.your-domain.com:443
-  maApiServiceName: ma-apiserver
-  useTLS: true
-
-logging:
-  level: info
-  development: true
-  encoding: console
-
-workflow-engine:
-  host: temporal.your-domain.com:7233
-  transport: grpc
-  provider: temporal
-  workers:
-    - domain: default
-      taskList: production-uniflow
-  client:
-    domain: uniflow
-```
-
-### **Field Explanations**
-
-| Field | Description |
-| ----- | ----- |
-| `worker.address` | API server endpoint used by workers |
-| `workflow-engine.host` | Temporal endpoint |
-| `workers[].taskList` | Worker task list to poll |
-| `client.domain` | Temporal workflow domain |
-
-## UI & Envoy Configuration
-
-### **Envoy Proxy**
-
-**ConfigMap:**
-
-```yaml
-static_resources:
-  listeners:
-    - address:
-        socket_address: { address: 0.0.0.0, port_value: 8081 }
-
-  clusters:
-    - name: michelangelo-apiserver
-      load_assignment:
-        endpoints:
-          - lb_endpoints:
-              - endpoint:
-                  address:
-                    socket_address:
-                      address: michelangelo-apiserver
-                      port_value: 15566
-```
-
-### **Public UI Config**
-
-**ConfigMap:**
-
-```json
-config.json: |
-  {
-    "apiBaseUrl": "https://michelangelo-envoy.<your-domain>"
-  }
-```
-
-## Environment Overrides / Domain Settings
-
-You must customize domain-specific values in overlays:
-
-| Location | Fields to Update |
-| ----- | ----- |
-| Worker ConfigMap | API server domain, compute domain, Temporal host |
-| UI Public Config | `"apiBaseUrl"` |
-| Envoy Config | CORS allowed origins, API cluster hostname |
-| Ingress | Hostnames for API server & UI |
-| Controller Manager | S3 region, endpoint, Temporal host |
-
-# Object Store Configuration
-
-Object storage (MinIO / S3) is used by Michelangelo for artifacts and metadata.
-
-## Controller Manager Object Store Settings
-
-These live in the controller manager ConfigMap:
-
-```yaml
-minio:
-  awsRegion: sample   # AWS region 
-  awsEndpointUrl: sample.amazonaws.com
-  useIam: true                # Use IAM roles for authentication
-```
-
-### **Fields**
-
-* `awsRegion` – The AWS region of your S3 bucket.
-* `awsEndpointUrl` – S3 endpoint (`s3.amazonaws.com` or regional endpoint).
-* `useIam` – Set to `true` in production (do not hardcode keys in config).
-
-## **Storage Setup Checklist (from original guide)**
-
-* Configure **AWS credentials/IAM roles** for pods that need S3 access.
-* Verify **region and endpoint** in the ConfigMap match your S3 setup.
-* Test connectivity from worker/controller pods to the bucket.
-
-# Workflow Engine Configuration (Temporal/Cadence)
-
-Michelangelo uses a workflow engine (Temporal or Cadence) for orchestrating workflows. Most of your current guide examples use **Temporal**, and Cadence is used in sandbox/dev.
-
-## Controller Manager Workflow Client
-
-From `controllermgr-configmap.yaml`:
-
-```yaml
-workflowClient:
-  service: temporal-frontend    # Temporal service name
-  host: temporal.your-domain.com:7233  # Temporal endpoint
-  transport: grpc               # Transport protocol
-  domain: uniflow               # Temporal domain
-```
-
-### **Fields**
-
-* `service` – Workflow engine frontend service name (`temporal-frontend` / `cadence-frontend`).
-* `host` – Full endpoint (host:port).
-* `transport` – Typically `grpc`.
-* `domain` – Temporal domain (or Cadence domain) to target.
-
-## Worker Workflow Engine Settings
-
-From `worker-configmap.yaml`:
-
-```yaml
-workflow-engine:
-  host: temporal(/cadence).your-domain.com:7233
-  transport: grpc
-  provider: temporal/cadence
-  workers:
-    - domain: default
-      taskList: production-uniflow
-  client:
-    domain: uniflow
-```
-
-### Fields
-
-* `provider` – `temporal` (or **cadence**); can be extended to `cadence` if needed.
-* `host` – Temporal/Cadence endpoint.
-* `workers[].domain` – Domain where worker polls for tasks.
-* `workers[].taskList` – Task list (queue) used for workflow tasks.
-* `client.domain` – Client domain for starting workflows.
-
-## Temporal Setup (from original external dependencies)
-
-* Ensure Temporal is accessible at the configured endpoint.
-* Create required domains (`uniflow`, `default`, `production-uniflow`).
-* Configure task lists such as `production-uniflow`.