docs: announce Kafka record headers (article + comparison updates)#266
Conversation
…re bullets - new canonical article for the KZM-3.4 header feature, wired into nav and the articles overview (syndication targets: dev.to/Hashnode/Medium) - upstream comparison tables in README and upstream-vs-kzm gain a Kafka record headers row and reflect KZM-3.4 - docs home What-you-get list gains the headers bullet with guide link
WalkthroughThe release documentation is updated to describe Kafka record header support and identify 3.4.0-KZM-3.4 as the current release. A separate article documents upgrading the Flink Kubernetes Operator from 1.11 to 1.15 for Flink 2.2. ChangesKafka headers release documentation
Flink operator upgrade article
Estimated code review effort: 2 (Simple) | ~15 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…ry, tune article voice
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/articles/flink-operator-1.15-upgrade.md`:
- Around line 104-110: Update the migration success criterion in the
FlinkDeployment example to inspect the job-level JobManager condition and job
lifecycle status, including the Running condition, rather than declaring success
from status.jobManagerDeploymentStatus=READY alone. Revise the kubectl query and
surrounding explanation so migration is confirmed only when the job is running
successfully and restored correctly.
- Around line 43-53: Update the Helm commands in the “The mechanical part is
small” section to document upgrading the existing release: first apply the Flink
Operator 1.15 CRD updates for FlinkDeployment, FlinkSessionJob, and
FlinkStateSnapshot, then replace helm install with helm upgrade while retaining
the 1.15 chart version and image tag.
- Around line 77-81: Revise the migration guidance to remove the claim that
changing flinkVersion leaves effective configuration unchanged. Explain that
operator validation and version-specific defaults under
kubernetes.operator.default-configuration.flink-version.<version> can alter
rendered configuration when switching from v2_0 to v2_2, and instruct readers to
compare effective/rendered configuration against those defaults before treating
the change as cosmetic.
In `@docs/articles/kafka-record-headers.md`:
- Line 140: Remove the unused Markdown link reference definition `[our-repo]`
from the document, or update repository links to use it if they are intended to
share that reference; ensure no unused reference remains.
- Around line 60-62: Update docs/articles/kafka-record-headers.md around the
null-header behavior to distinguish Kafka client rejection of null keys from the
SDK’s effective contract, and state that contract explicitly. Update the
corresponding release bullet in CHANGELOG.md to use the same null-key behavior
and terminology; both documentation sites must remain consistent.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 063f9535-1bde-4822-9a28-18946dc7cd62
📒 Files selected for processing (12)
CHANGELOG.mdREADME.mddocs/articles/flink-operator-1.15-upgrade.mddocs/articles/index.mddocs/articles/kafka-record-headers.mddocs/guides/k8s-deployment.mddocs/index.mddocs/install.mddocs/quickstart.mddocs/snapshots.mddocs/upstream-vs-kzm.mdmkdocs.yml
| The mechanical part is small. Bump the Helm release: | ||
|
|
||
| ```bash | ||
| helm repo add --force-update flink-operator-repo \ | ||
| "https://archive.apache.org/dist/flink/flink-kubernetes-operator-1.15.0/" | ||
| helm repo update flink-operator-repo | ||
| helm install flink-kubernetes-operator flink-operator-repo/flink-kubernetes-operator \ | ||
| --namespace flink-operator --create-namespace --wait --timeout 5m \ | ||
| --version "1.15.0" \ | ||
| --set image.tag="1.15.0" | ||
| ``` |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== file excerpt =="
sed -n '1,120p' docs/articles/flink-operator-1.15-upgrade.md || true
echo "== search for helm install/upgrade/crds in file =="
rg -n "helm (install|upgrade|repo)|CRD|CRDs|FlinkDeployment|FlinkSessionJob|FlinkStateSnapshot|apiVersion|api-version|namespace|flinkVersion" docs/articles/flink-operator-1.15-upgrade.md || true
echo "== relevant docs articles =="
fd -a 'flink-operator.*upgrade|upgrade.*flink|flink.*kubernetes.*operator' docs . 2>/dev/null | sed 's#^\./##' | head -50Repository: kzmlabs/flink-statefun
Length of output: 13060
🌐 Web query:
Apache Flink Kubernetes Operator 1.15 upgrade documentation helm upgrade CRDs FlinkDeployment FlinkSessionJob FlinkStateSnapshot
💡 Result:
Upgrading to Apache Flink Kubernetes Operator 1.15 involves specific steps for CRD management and Helm deployment [1]. Because CRDs are not automatically removed or upgraded by Helm, manual intervention is required [1][2]. CRD Upgrade Process The first step is to manually upgrade the CRDs for FlinkDeployment, FlinkSessionJob, and FlinkStateSnapshot [1]. This ensures the cluster is prepared for the new operator version [1]. You can perform this update using kubectl: kubectl delete crd flinkdeployments.flink.apache.org (Repeat for FlinkSessionJobs and FlinkStateSnapshots as necessary, then apply the new manifests from the 1.15 release.) Helm Upgrade Once the CRDs are updated, proceed to upgrade the Helm deployment [1]. The standard command is: helm upgrade flink-kubernetes-operator /flink-kubernetes-operator --version 1.15.0 [1][3] Key 1.15 Changes The 1.15.0 release introduced several significant changes: - Kubernetes-native Conditions: FlinkDeployment, FlinkSessionJob, and FlinkStateSnapshot now include standardized Kubernetes-style status conditions [4][5]. - FlinkStateSnapshot Integration: This resource is now the primary mechanism for managing snapshots, deprecating older status fields like savepointInfo and checkpointInfo within the deployment CRs [6]. - Flink Compatibility: The operator is fully validated against Flink 2.2, with continued support for versions 2.1, 2.0, 1.20, and 1.19 [4]. Backward Compatibility Starting from v1beta1 (introduced in 1.0.0), the operator aims to provide backward compatibility for existing Flink resources [7]. While you can safely upgrade the operator and CRD without disrupting running applications, be aware that backward compatibility is not guaranteed for Helm chart configuration or Java APIs (e.g., Validators), which may require updates to your custom configurations [7].
Citations:
- 1: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.15/docs/operations/upgrade/
- 2: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.15/docs/development/guide/
- 3: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.15/docs/operations/helm/
- 4: https://flink.apache.org/2026/05/26/apache-flink-kubernetes-operator-1.15.0-release-announcement/
- 5: apache/flink-kubernetes-operator@eeb63fb
- 6: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.15/docs/custom-resource/snapshots/
- 7: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.15/docs/operations/compatibility/
Document the Helm upgrade instead of a new install.
This article is for users upgrading an existing 1.11 release, but the current helm install command fails when that release already exists. Replace this with the Flink Operator 1.15 CRD update for FlinkDeployment, FlinkSessionJob, and FlinkStateSnapshot, followed by helm upgrade with the 1.15 chart version/app image tag.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/articles/flink-operator-1.15-upgrade.md` around lines 43 - 53, Update
the Helm commands in the “The mechanical part is small” section to document
upgrading the existing release: first apply the Flink Operator 1.15 CRD updates
for FlinkDeployment, FlinkSessionJob, and FlinkStateSnapshot, then replace helm
install with helm upgrade while retaining the 1.15 chart version and image tag.
| `flinkVersion` drives the operator's reconciler behaviour, not the runtime's config parsing. Your `flinkConfiguration` map is handed to a Flink 2.2.0 process whether the operator label says `v2_0` or `v2_2`. If the job already ran on a 2.2.0 image, those keys were already being parsed by Flink 2.2, and they worked. Bumping `flinkVersion` doesn't change how the runtime reads them. | ||
|
|
||
| That gives a low-risk rule: only remove or rename a key when you can prove it was removed, renamed, or rejected on the target Flink version. Keys that merely look redundant with defaults stay. Anything ambiguous stays and gets a note instead of a deletion. | ||
|
|
||
| The asymmetry is the argument. Flink warns on unknown config keys and moves on, so a stale key is cosmetic — but a wrongly deleted key is a behaviour change. In our migration the audit changed exactly one thing of substance: `flinkVersion`. The checkpointing, state-backend, HA, and restart-strategy blocks already ran clean on 2.2.0, so they stayed — verified, not assumed. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
Apache Flink Kubernetes Operator 1.15 configuration flinkVersion version-specific defaults
💡 Result:
In the Apache Flink Kubernetes Operator 1.15, you can configure Flink version-specific defaults by using the kubernetes.operator.default-configuration.flink-version.<version> prefix in your operator configuration [1]. These settings allow you to define default configuration values that apply only to specific Flink versions, which is useful when behavior changes between versions [1]. Key rules for these configuration defaults include: Precedence: Flink version-specific defaults have a higher precedence than namespace-specific defaults; therefore, version-specific settings will override namespace-specific defaults if they share the same configuration key [1][2]. Version Targeting: You can target a specific Flink version (e.g., v1_17) or use the + suffix to target that version and all subsequent versions (e.g., v1_16+) [1]. Example Syntax: To set version-specific defaults, you define them in your operator configuration as follows [1]: kubernetes.operator.default-configuration.flink-version.v1_16+.k4: v4 kubernetes.operator.default-configuration.flink-version.v1_17.k5: v5.1 In this example, the key k4 is set to v4 for Flink 1.16 and any newer versions, while k5 is set to v5.1 specifically for Flink 1.17 [1]. Note that when configuring these via Helm, you should follow the operator's configuration guidelines and avoid overriding env.java.default-opts.all in your FlinkDeployment spec, preferring env.java.opts.all for custom Java options [1].
Citations:
- 1: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.15/docs/operations/configuration/
- 2: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.10/docs/operations/configuration/
🌐 Web query:
Apache Flink Kubernetes Operator 1.12 configuration validation Flink 2.x configuration keys
💡 Result:
The Apache Flink Kubernetes Operator 1.12.0 introduced dedicated support for Flink 2.x, which includes enhanced configuration validation capabilities to ensure compatibility with Flink 2.x configuration standards [1]. Key aspects regarding Flink 2.x configuration and the 1.12 operator include: 1. Upfront YAML Validation: The operator 1.12.0 introduced upfront YAML configuration validation specifically for Flink 2.x, allowing for the detection of misconfigurations before they are deployed to the cluster [1]. This system builds upon the operator's existing resource validation framework, which checks FlinkDeployment and FlinkSessionJob specifications for correctness, including forbidden configuration keys and required parameters [2]. 2. Configuration File Format: Flink 2.x mandates the use of a new configuration file format, config.yaml [3][4]. The 1.12.0 operator added full compatibility with this format to support seamless integration with Flink 2.x deployments [1][3]. While legacy flink-conf.yaml files were previously common, the operator's Helm chart now supports defaultConfiguration.config.yaml to align with the Flink 2.x requirement [3][4]. 3. Upgrade Path: The 1.12.0 operator addressed specific issues related to Flink 2.0 upgrades, such as resolving a defect where the "last-state" upgrade mode was broken for Flink 2.0 deployments, thereby restoring correct behavior [1]. 4. Operational Usage: Users managing Flink 2.x clusters via the operator should ensure their configurations are provided in the config.yaml format [3][5]. The operator utilizes its DefaultValidator and FlinkConfigManager to perform these validations [2]. If users encounter validation errors, the operator will report them, typically identifying specific issues such as unsupported keys or invalid parameter formats [2]. For operational guidance, it is recommended to use the Helm chart's defaultConfiguration.config.yaml parameter rather than flink-conf.yaml to ensure compliance with the target Flink 2.x environment [3][4].
Citations:
- 1: https://flink.apache.org/2025/06/03/apache-flink-kubernetes-operator-1.12.0-release-announcement/
- 2: https://deepwiki.com/apache/flink-kubernetes-operator/4.3-resource-validation
- 3: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.12/docs/operations/helm/
- 4: [FLINK-39791][helm] Honor defaultConfiguration.config.yaml and always mount config.yaml apache/flink-kubernetes-operator#1126
- 5: https://nightlies.apache.org/flink/flink-docs-release-2.0/docs/deployment/resource-providers/standalone/kubernetes/
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'Repository files matching target:\n'
git ls-files | rg '(^|/)flink-operator-1\.15-upgrade\.md$|docs/articles/flink-operator-1\.15-upgrade\.md' || true
file="docs/articles/flink-operator-1.15-upgrade.md"
if [ -f "$file" ]; then
echo
echo "Relevant doc excerpt:"
nl -ba "$file" | sed -n '1,140p'
echo
echo "References to flinkVersion/flinkConfiguration/defaultConfiguration around excerpt:"
nl -ba "$file" | sed -n '55,95p'
fiRepository: kzmlabs/flink-statefun
Length of output: 303
🏁 Script executed:
#!/bin/bash
set -euo pipefail
file="docs/articles/flink-operator-1.15-upgrade.md"
echo "Line count:"
wc -l "$file"
echo
echo "Relevant doc excerpt (lines 1-140):"
python3 - <<'PY'
from pathlib import Path
p = Path("docs/articles/flink-operator-1.15-upgrade.md")
for n, line in enumerate(p.read_text().splitlines(), 1):
if n <= 140 or 70 <= n <= 90:
print(f"{n:4d}: {line}")
PY
echo
echo "Occurrences of relevant terms in file:"
python3 - <<'PY'
from pathlib import Path
p = Path("docs/articles/flink-operator-1.15-upgrade.md")
terms = ["flinkVersion", "flinkConfiguration", "flink-configuration", "default-configuration", "v2_0", "v2_2", "2.2.0"]
for n, line in enumerate(p.read_text().splitlines(), 1):
if any(t in line for t in terms):
print(f"{n:4d}: {line}")
PYRepository: kzmlabs/flink-statefun
Length of output: 13978
🌐 Web query:
"Operator Flink 2.x Configuration Validation" "flinkVersion" "v2_0" "kubernetes.operator.default-configuration"
💡 Result:
In the Apache Flink Kubernetes Operator, configuration validation for Flink 2.x was significantly enhanced to ensure compatibility with Flink 2.0's standardized YAML runtime configuration format [1][2]. Key aspects of this configuration handling include: Upfront YAML Validation: Starting with the Flink Kubernetes Operator 1.12.0, the operator introduced upfront validation for Flink 2.x configurations, allowing for the early detection of misconfigurations before they are deployed to the cluster [1]. Flink 2.0 Configuration Format: Flink 2.0 exclusively supports the standardized YAML runtime configuration format [2]. To accommodate this, the operator includes logic to automatically handle configuration translation and management [2]. In the Helm chart, the default configuration is managed via the defaultConfiguration.config.yaml parameter (which replaces the legacy flink-conf.yaml approach for Flink 2.x) [3][4][5]. Operator Default Configuration: You can define default configurations that apply to specific Flink versions using the syntax: kubernetes.operator.default-configuration.flink-version.v2_0.: [6][7][8] These version-specific defaults take precedence over general or namespace-specific defaults [6][7]. When configuring your environment, it is recommended to review these settings in the Helm values.yaml file to ensure they align with the requirements of your Flink 2.0 deployments [6]. Technical note: The operator correctly identifies Flink 2.0 deployments when flinkVersion: v2_0 is specified in the FlinkDeployment custom resource [8][9]. This triggers the use of the appropriate configuration mounting logic to ensure the operator provides a compliant config.yaml to the Flink 2.x job managers and task managers [4].
Citations:
- 1: https://flink.apache.org/2025/06/03/apache-flink-kubernetes-operator-1.12.0-release-announcement/
- 2: [FLINK-37236] Flink 2.0 preview support apache/flink-kubernetes-operator#938
- 3: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.13/docs/operations/helm/
- 4: [FLINK-39791][helm] Honor defaultConfiguration.config.yaml and always mount config.yaml apache/flink-kubernetes-operator#1126
- 5: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/operations/helm/
- 6: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/operations/configuration/
- 7: https://github.com/apache/flink-kubernetes-operator/blob/main/docs/content/docs/operations/configuration.md
- 8: https://flink.apache.org/2025/03/03/apache-flink-kubernetes-operator-1.11.0-release-announcement/
- 9: apache/flink-kubernetes-operator@d6e20e2
🌐 Web query:
site:nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.15 "Bumping flinkVersion" "effective configuration"
💡 Result:
In the context of the Apache Flink Kubernetes Operator, flinkVersion is a top-level field within the FlinkDeployment Custom Resource specification that dictates the Flink version to be used for the deployment [1][2]. To update or "bump" the flinkVersion effectively, follow these steps: 1. Update the Specification: Modify the flinkVersion field in your FlinkDeployment YAML manifest to the desired version (e.g., from v1_19 to v1_20) [1][3]. 2. Reconcile: Apply the updated manifest to your Kubernetes cluster using kubectl apply -f <your-file>.yaml. The Flink Kubernetes Operator will detect the change in the specification and automatically trigger an upgrade process [1]. 3. Upgrade Mechanism: The operator handles the transition by managing the lifecycle of the Flink cluster. Depending on your configuration and the state of your job, the operator will use the defined upgradeMode (such as savepoint or last-state) to safely transition the job to the new Flink version [4][3]. 4. Version Compatibility: Ensure that the image field is also updated to a compatible Docker image corresponding to the new flinkVersion, as the operator uses this image to run the Flink JobManager and TaskManager processes [1][2]. Important Considerations: - Operator Defaults: The operator supports version-specific configuration overrides via kubernetes.operator.default-configuration.flink-version.<version>+ keys [5]. When bumping a major version, verify if any previous version-specific defaults need adjustment, as these take precedence over general defaults [5]. - State Management: Always review your upgradeMode. For stateful applications, using savepoint mode provides the highest level of safety during version upgrades [3]. - Operator Upgrades: Ensure your Flink Kubernetes Operator version itself is compatible with the target Flink version [2]. If the operator version is outdated, you may need to upgrade the operator via Helm before bumping the flinkVersion in your deployments [6].
Citations:
- 1: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.15/docs/custom-resource/overview/
- 2: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.15/docs/custom-resource/reference/
- 3: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.15/docs/custom-resource/job-management/
- 4: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.15/docs/concepts/controller-flow/
- 5: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.15/docs/operations/configuration/
- 6: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.15/docs/operations/upgrade/
Do not claim that flinkVersion changes leave effective configuration unchanged.
flinkVersion is used by the operator for Flink 2.x config validation in 1.12+, and 1.15 adds version-specific operator defaults under kubernetes.operator.default-configuration.flink-version.<version>. Switching from v2_0 to v2_2 can change rendered/default config or validation results even with the same Flink 2.2.0 image, so compare the effective/rendered configuration against version-specific defaults before treating the change as cosmetic.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/articles/flink-operator-1.15-upgrade.md` around lines 77 - 81, Revise
the migration guidance to remove the claim that changing flinkVersion leaves
effective configuration unchanged. Explain that operator validation and
version-specific defaults under
kubernetes.operator.default-configuration.flink-version.<version> can alter
rendered configuration when switching from v2_0 to v2_2, and instruct readers to
compare effective/rendered configuration against those defaults before treating
the change as cosmetic.
| The migration is real when a `FlinkDeployment` on `flinkVersion: v2_2` reaches `READY`: | ||
|
|
||
| ```bash | ||
| kubectl get flinkdeployment -n my-app \ | ||
| -o jsonpath='{.items[*].status.jobManagerDeploymentStatus}' | ||
| # READY | ||
| ``` |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== file excerpt =="
sed -n '1,160p' docs/articles/flink-operator-1.15-upgrade.md | cat -n
echo
echo "== search related mentions =="
rg -n "jobManagerDeploymentStatus|migration is real|Running|flinkVersion|FlinkDeployment|READY" docs/articles/flink-operator-1.15-upgrade.md docs articles . || true
echo
echo "== locate flink operator docs/api references =="
git ls-files | rg -i 'flink|kubernetes|operator|migration|deployment' | head -200Repository: kzmlabs/flink-statefun
Length of output: 50380
🌐 Web query:
Apache Flink Kubernetes Operator JobManagerDeploymentStatus READY means JobManager ready REST application job status fails Running condition
💡 Result:
In the Apache Flink Kubernetes Operator, a JobManagerDeploymentStatus of READY indicates specifically that the JobManager is running and has successfully initialized its REST API, making it capable of receiving REST API calls [1][2][3]. This status pertains only to the health and readiness of the JobManager component itself [1][4]. It does not inherently guarantee that the Flink job inside the cluster is currently in a "Running" state [5]. The status of the job itself is tracked separately via the JobStatus field [2][5]. If your JobManager deployment is READY but the job fails to reach the "Running" condition, consider the following technical points: 1. Distinct Status Layers: The operator manages the JobManager deployment as a prerequisite for job execution [5]. Once the JobManager is READY, the operator proceeds to observe and manage the job status [5]. A job may be failing for reasons independent of the JobManager's readiness, such as deployment configuration errors, missing resources, job code exceptions, or failing to restore from a checkpoint/savepoint [5][6]. 2. Job Lifecycle and Errors: If the job state cannot be determined or is failing (e.g., due to an unsuccessful restoration), the operator may transition the job to a reconciling, failing, or error state, even while the JobManager pod remains healthy and READY [5]. 3. Debugging Steps: - Check the logs of the Flink main container to identify job-level exceptions [7]. - Inspect the FlinkDeployment or FlinkSessionJob status field for detailed error messages, which are often populated when the job fails to run [2]. - Ensure the job specification is valid, as terminal errors in the deployment spec can prevent the job from starting even if the infrastructure is healthy [2][4]. Recent updates to the operator (via FLIP-493) have introduced status conditions to the Custom Resource Definition (CRD) to better distinguish between JobManager deployment health and actual job execution status [8][9]. Check your CR status conditions for a type of Running to see if it provides more specific information regarding why the job is not progressing [8][10].
Citations:
- 1: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/api/java/org/apache/flink/kubernetes/operator/api/status/JobManagerDeploymentStatus.html
- 2: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.8/docs/custom-resource/reference/
- 3: https://github.com/apache/flink-kubernetes-operator/blob/main/docs/content/docs/custom-resource/reference.md
- 4: https://apache.googlesource.com/flink-kubernetes-operator/+/HEAD/docs/content/docs/custom-resource/reference.md
- 5: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/concepts/controller-flow/
- 6: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.12/docs/custom-resource/job-management/
- 7: https://issues.apache.org/jira/browse/FLINK-26804
- 8: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=334760351
- 9: https://cwiki.apache.org/confluence/display/FLINK/FLIP-493%3A+Add+K8S+conditions+to+CRD%27s+Status
- 10: [FLINK-33634] Add Conditions to Flink CRD's Status field apache/flink-kubernetes-operator#957
Use the job-level job manager condition/job status, not READY, to declare migration success.
status.jobManagerDeploymentStatus=READY only means the JobManager deployment is ready for REST calls; it does not prove the Flink job ran successfully or restored correctly. Also check the Running condition/job lifecycle status before saying the migration worked.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/articles/flink-operator-1.15-upgrade.md` around lines 104 - 110, Update
the migration success criterion in the FlinkDeployment example to inspect the
job-level JobManager condition and job lifecycle status, including the Running
condition, rather than declaring success from
status.jobManagerDeploymentStatus=READY alone. Revise the kubectl query and
surrounding explanation so migration is confirmed only when the job is running
successfully and restored correctly.
| They stay null. This detail is easy to get wrong: Kafka's wire format distinguishes a header whose value is **null** from one whose value is **empty** (a null value encodes as length `-1`), and `Header#value()` in the Kafka client legitimately returns `null`. Protobuf — which carries StateFun's remote-function protocol — cannot represent null bytes. | ||
|
|
||
| Our first cut coerced null to empty bytes. It worked, and it was wrong — we only caught it after actually checking what `kafka-clients` does (null header values are legal and preserved; null *keys* are rejected). The fix borrows the idiom StateFun's protocol already uses for exactly this problem: an explicit `has_value` flag, same as `TypedValue.has_value`. A null-valued header now goes in as null and comes out as null, which also means echoing headers is lossless: |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Null-key behavior is documented inconsistently.
docs/articles/kafka-record-headers.md#L60-L62: distinguish Kafka-client rejection from SDK behavior and state the effective contract.CHANGELOG.md#L28-L33: update the release bullet to match that contract.
📍 Affects 2 files
docs/articles/kafka-record-headers.md#L60-L62(this comment)CHANGELOG.md#L28-L33
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/articles/kafka-record-headers.md` around lines 60 - 62, Update
docs/articles/kafka-record-headers.md around the null-header behavior to
distinguish Kafka client rejection of null keys from the SDK’s effective
contract, and state that contract explicitly. Update the corresponding release
bullet in CHANGELOG.md to use the same null-key behavior and terminology; both
documentation sites must remain consistent.
|
|
||
| Full reference: **[the Kafka record headers guide](https://kzmlabs.github.io/flink-statefun/guides/kafka-headers/)**. Background on the fork itself: **[We forked Apache Stateful Functions for Flink 2.x — here's why](forking-statefun.md)**. | ||
|
|
||
| [our-repo]: https://github.com/kzmlabs/flink-statefun |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Remove the unused our-repo reference.
Nothing uses [our-repo]; markdownlint reports MD053. Delete the definition or use it for the repository links.
🧰 Tools
🪛 markdownlint-cli2 (0.23.0)
[warning] 140-140: Link and image reference definitions should be needed
Unused link or image reference definition: "our-repo"
(MD053, link-image-reference-definitions)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/articles/kafka-record-headers.md` at line 140, Remove the unused
Markdown link reference definition `[our-repo]` from the document, or update
repository links to use it if they are intended to share that reference; ensure
no unused reference remains.
Source: Linters/SAST tools
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
1 participant
Tier-1 announcement for the KZM-3.4 headers feature: canonical article for syndication, upstream-comparison rows in README and docs, feature bullet on the docs home. Validated with mkdocs build --strict.
Summary by CodeRabbit
New Features
Documentation