feat: a new workflow for epp+keda baseline vs FMA with keda (warm/hot)

[aavarghese]

Adds a new OpenShift nightly (ci-nightly-benchmark-ocp-fma-keda.yaml) that benchmarks the direct EPP+KEDA saturation autoscaling path (no WVA controller):

1. Baseline (EPP+KEDA, no FMA) — cicd/ocp-keda
2. FMA warm-start with EPP+KEDA — cicd/ocp-keda-fma-warmstart
3. FMA hot-start with EPP+KEDA — cicd/ocp-keda-fma-hotstart

KEDA scales the decode/FMA-requester Deployment on EPP pool metrics (inference_pool_average_kv_cache_utilization > 0.7, inference_pool_average_queue_size > 2) via a ScaledObject → HPA, enabled purely through eppKedaSaturation.enabled: true — no VariantAutoscaling, no prometheus-adapter.

Shared observability:

• New T_scale_up metric — autoscaler scale-up latency = HPA status.lastScaleTime → newest replica Ready. Replaces the old WVA-controller-log-derived "cold-start" row (which didn't exist on the EPP-KEDA path and is deprecated now that WVA scales via KEDA). Scaler-agnostic (reads resources.yaml), so it's identical for both nightlies. Sits alongside T_actuation (container start → Ready).
• step_09a capture step:
  • Captures KEDA CRs (scaledobjects.keda.sh, triggerauthentications.keda.sh) in resources.yaml, plus scaledobject.txt and scaledobject-describe.txt
  • Adds EPP pool trigger-metric Thanos queries (kv-cache utilization, queue size)
  • Removes deprecated prometheus-adapter-era captures (external-metric query/discovery, prometheus-adapter logs) and the VariantAutoscaling CR
• Display dumps: per-arm Dump ScaledObject + Dump ScaledObject describe (both nightlies); KEDA workflow drops the dead WVA-controller-log dump, WVA workflow keeps it.

🤖 Generated with Claude Code (https://claude.com/claude-code)

[Vezio]

@aavarghese what is the status of this - tested etc?

[aavarghese]

@Vezio this one's testing is in-progress. We are not seeing the numbers we expect for hot start with FMA so investigation ongoing with @dumb0002 - kinda related to PR #1657

[aavarghese]

/cc @Vezio @dumb0002 ready for review please