Flink bg merged

docs/content/docs/custom-resource/reference.md

@@ -89,6 +89,7 @@ This serves as a full reference for FlinkDeployment and FlinkSessionJob custom r
 | Parameter | Type | Docs |
 | ----------| ---- | ---- |
 | configuration | java.util.Map<java.lang.String,java.lang.String> |  |
+| ingress | org.apache.flink.kubernetes.operator.api.spec.IngressSpec |  |
 | template | org.apache.flink.kubernetes.operator.api.spec.FlinkDeploymentTemplateSpec |  |
 
 ### FlinkDeploymentSpec

docs/content/docs/operations/metrics-logging.md

@@ -35,15 +35,20 @@ Different operator metrics can be turned on/off individually using the configura
 ### Flink Resource Metrics
 The Operator gathers aggregates metrics about managed resources.
 
-| Scope              | Metrics                                                                                   | Description                                                                                                                                                                                        | Type      |
-|--------------------|-------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------|
+| Scope              | Metrics                                                                                   | Description                                                                                                                                                                                       | Type      |
+|--------------------|-------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------|
 | Namespace          | FlinkDeployment/FlinkSessionJob.Count                                                     | Number of managed resources per namespace                                                                                                                               | Gauge     |
-| Namespace          | FlinkDeployment.ResourceUsage.Cpu/Memory                                            | Total resources used per namespace                                                                                                                                                                | Gauge     |
-| Namespace          | FlinkDeployment.JmDeploymentStatus.<Status>.Count                                   | Number of managed FlinkDeployment resources per <Status> per namespace. <Status> can take values from: READY, DEPLOYED_NOT_READY, DEPLOYING, MISSING, ERROR                            | Gauge     |
-| Namespace          | FlinkDeployment.FlinkVersion.<FlinkVersion>.Count                               | Number of managed FlinkDeployment resources per <FlinkVersion> per namespace. <FlinkVersion> is retrieved via REST API from Flink JM.                                                  | Gauge     |
+| Namespace          | FlinkDeployment.ResourceUsage.Cpu/Memory                                            | Total resources used per namespace                                                                                                                                                               | Gauge     |
+| Namespace          | FlinkDeployment.JmDeploymentStatus.<Status>.Count                                   | Number of managed FlinkDeployment resources per <Status> per namespace. <Status> can take values from: READY, DEPLOYED_NOT_READY, DEPLOYING, MISSING, ERROR                           | Gauge     |
+| Namespace          | FlinkDeployment.FlinkVersion.<FlinkVersion>.Count                               | Number of managed FlinkDeployment resources per <FlinkVersion> per namespace. <FlinkVersion> is retrieved via REST API from Flink JM.                                                 | Gauge     |
 | Namespace          | FlinkDeployment/FlinkSessionJob.Lifecycle.State.<State>.Count                       | Number of managed resources currently in state <State> per namespace. <State> can take values from: CREATED, SUSPENDED, UPGRADING, DEPLOYED, STABLE, ROLLING_BACK, ROLLED_BACK, FAILED | Gauge     |
-| System/Namespace   | FlinkDeployment/FlinkSessionJob.Lifecycle.State.<State>.TimeSeconds                 | Time spent in state <State> for a given resource. <State> can take values from: CREATED, SUSPENDED, UPGRADING, DEPLOYED, STABLE, ROLLING_BACK, ROLLED_BACK, FAILED                      | Histogram |
-| System/Namespace   | FlinkDeployment/FlinkSessionJob.Lifecycle.Transition.<Transition>.TimeSeconds       | Time statistics for selected lifecycle state transitions. <Transition> can take values from: Resume, Upgrade, Suspend, Stabilization, Rollback, Submission                                   | Histogram |
+| System/Namespace   | FlinkDeployment/FlinkSessionJob.Lifecycle.State.<State>.TimeSeconds                 | Time spent in state <State> for a given resource. <State> can take values from: CREATED, SUSPENDED, UPGRADING, DEPLOYED, STABLE, ROLLING_BACK, ROLLED_BACK, FAILED                     | Histogram |
+| System/Namespace   | FlinkDeployment/FlinkSessionJob.Lifecycle.Transition.<Transition>.TimeSeconds       | Time statistics for selected lifecycle state transitions. <Transition> can take values from: Resume, Upgrade, Suspend, Stabilization, Rollback, Submission                                  | Histogram |
+| Namespace          | FlinkBlueGreenDeployment.BlueGreenState.<State>.Count                              | Number of managed FlinkBlueGreenDeployment resources currently in state <State> per namespace. <State> can take values from: INITIALIZING_BLUE, ACTIVE_BLUE, SAVEPOINTING_BLUE, TRANSITIONING_TO_GREEN, ACTIVE_GREEN, SAVEPOINTING_GREEN, TRANSITIONING_TO_BLUE | Gauge     |
+| Namespace          | FlinkBlueGreenDeployment.JobStatus.<Status>.Count                                  | Number of managed FlinkBlueGreenDeployment resources currently in JobStatus <Status> per namespace. <Status> can take values from: RUNNING, FAILING, SUSPENDED, FAILED, RECONCILING | Gauge     |
+| Namespace          | FlinkBlueGreenDeployment.Failures                                                        | Historical count of failure events (transitions to FAILING state) for all FlinkBlueGreenDeployment resources in the namespace. Counter increments on each transition to FAILING and never decrements. | Counter   |
+| System/Namespace   | FlinkBlueGreenDeployment.Lifecycle.State.<State>.TimeSeconds                        | Time spent in state <State> for a given FlinkBlueGreenDeployment resource. <State> values same as above.                                                                               | Histogram |
+| System/Namespace   | FlinkBlueGreenDeployment.Lifecycle.Transition.<Transition>.TimeSeconds              | Time statistics for blue-green lifecycle state transitions. <Transition> can take values from: InitialDeployment, BlueToGreen, GreenToBlue                                                  | Histogram |
 
 #### Lifecycle metrics
 
@@ -60,6 +65,32 @@ In addition to the simple counts we further track a few selected state transitio
  - Rollback : Time from deployed to rolled_back state if the resource was rolled back
  - Submission: Flink resource submission time
 
+#### FlinkBlueGreenDeployment Lifecycle metrics
+
+FlinkBlueGreenDeployment resources have their own lifecycle states that track the blue-green deployment process. The operator monitors the following transitions:
+
+ - InitialDeployment : Time from leaving INITIALIZING_BLUE to reaching ACTIVE_BLUE (first deployment)
+ - BlueToGreen : Time from leaving ACTIVE_BLUE to reaching ACTIVE_GREEN (actual transition duration)
+ - GreenToBlue : Time from leaving ACTIVE_GREEN to reaching ACTIVE_BLUE (actual transition duration)
+
+Transition metrics measure the actual transition time - from when the deployment leaves the source stable state until it reaches the target stable state. This excludes time spent running stably before the transition was initiated.
+
+State time metrics track how long a resource spends in each state (ACTIVE_BLUE, SAVEPOINTING_BLUE, TRANSITIONING_TO_GREEN, etc.), which helps identify bottlenecks in the deployment pipeline.
+
+#### FlinkBlueGreenDeployment JobStatus Tracking
+
+In addition to BlueGreenState tracking, FlinkBlueGreenDeployment resources also expose JobStatus metrics that track the Flink job state:
+
+**JobStatus Gauges**: Current count of deployments per JobStatus (RUNNING, FAILING, SUSPENDED, etc.) - these gauges go up and down as deployments transition between states.
+
+**Failures Counter**: Historical count that increments each time a deployment transitions TO the FAILING state. This counter:
+ - Never decrements (accumulates total failures since operator start)
+ - Increments on each new transition to FAILING (even if the same deployment fails multiple times)
+ - Persists across deployment recoveries (provides historical failure tracking)
+ - Useful for calculating failure rates and setting up alerts
+
+Example: A deployment goes RUNNING → FAILING → RUNNING → FAILING. The FAILING gauge shows 0 or 1 (current state), while the Failures counter shows 2 (historical events).
+
 ### Kubernetes Client Metrics
 
 The Operator gathers various metrics related to Kubernetes API server access.

e2e-tests/data/bluegreen-ingress.yaml

@@ -0,0 +1,58 @@
+################################################################################
+#  Licensed to the Apache Software Foundation (ASF) under one
+#  or more contributor license agreements.  See the NOTICE file
+#  distributed with this work for additional information
+#  regarding copyright ownership.  The ASF licenses this file
+#  to you under the Apache License, Version 2.0 (the
+#  "License"); you may not use this file except in compliance
+#  with the License.  You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+apiVersion: flink.apache.org/v1beta1
+kind: FlinkBlueGreenDeployment
+metadata:
+  name: bg-ingress-test
+spec:
+  configuration:
+    kubernetes.operator.bluegreen.deployment-deletion.delay: "2s"
+  # Parent-level ingress configuration
+  ingress:
+    template: "{{name}}.{{namespace}}.example.com"
+    className: "nginx"
+    annotations:
+      nginx.ingress.kubernetes.io/rewrite-target: "/"
+  template:
+    spec:
+      image: flink:1.20
+      flinkVersion: v1_20
+      flinkConfiguration:
+        rest.port: "8081"
+        taskmanager.numberOfTaskSlots: "1"
+      serviceAccount: flink
+      jobManager:
+        resource:
+          memory: 1G
+          cpu: 1
+      taskManager:
+        resource:
+          memory: 2G
+          cpu: 1
+      job:
+        jarURI: local:///opt/flink/examples/streaming/StateMachineExample.jar
+        parallelism: 1
+        entryClass: org.apache.flink.streaming.examples.statemachine.StateMachineExample
+        args:
+          - "--error-rate"
+          - "0.15"
+          - "--sleep"
+          - "30"
+        upgradeMode: stateless
+      mode: native

e2e-tests/data/bluegreen-savepoint-redeploy.yaml

@@ -0,0 +1,100 @@
+################################################################################
+#  Licensed to the Apache Software Foundation (ASF) under one
+#  or more contributor license agreements.  See the NOTICE file
+#  distributed with this work for additional information
+#  regarding copyright ownership.  The ASF licenses this file
+#  to you under the Apache License, Version 2.0 (the
+#  "License"); you may not use this file except in compliance
+#  with the License.  You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+apiVersion: flink.apache.org/v1beta1
+kind: FlinkBlueGreenDeployment
+metadata:
+  name: bg-savepoint-redeploy-example
+spec:
+
+  configuration:
+    kubernetes.operator.bluegreen.deployment-deletion.delay: "1s"
+  template:
+    spec:
+      image: flink:1.20
+      flinkVersion: v1_20
+      flinkConfiguration:
+        rest.port: "8081"
+        execution.checkpointing.interval: "10s"
+        execution.checkpointing.storage: "filesystem"
+        state.backend.incremental: "true"
+        state.checkpoints.dir: file:///opt/flink/volume/flink-cp
+        state.savepoints.dir: file:///opt/flink/volume/flink-sp
+        state.checkpoints.num-retained: "5"
+        taskmanager.numberOfTaskSlots: "1"
+      serviceAccount: flink
+      jobManager:
+        resource:
+          memory: 1G
+          cpu: 1
+        podTemplate:
+          spec:
+            containers:
+              - name: flink-main-container
+                resources:
+                  requests:
+                    ephemeral-storage: 2048Mi
+                  limits:
+                    ephemeral-storage: 2048Mi
+                volumeMounts:
+                  - mountPath: /opt/flink/volume
+                    name: flink-volume
+            volumes:
+              - name: flink-volume
+                persistentVolumeClaim:
+                  claimName: flink-bg-sp-redeploy
+      taskManager:
+        resource:
+          memory: 2G
+          cpu: 1
+      job:
+        jarURI: local:///opt/flink/examples/streaming/StateMachineExample.jar
+        parallelism: 1
+        entryClass: org.apache.flink.streaming.examples.statemachine.StateMachineExample
+        args:
+          - "--error-rate"
+          - "0.15"
+          - "--sleep"
+          - "30"
+        upgradeMode: savepoint
+      mode: native
+
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: flink-bg-sp-redeploy
+spec:
+  accessModes:
+    - ReadWriteOnce
+  volumeMode: Filesystem
+  resources:
+    requests:
+      storage: 1Gi
+
+---
+apiVersion: networking.k8s.io/v1
+kind: IngressClass
+metadata:
+  annotations:
+    ingressclass.kubernetes.io/is-default-class: "true"
+  labels:
+    app.kubernetes.io/component: controller
+  name: nginx
+spec:
+  controller: k8s.io/ingress-nginx