Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Normalize line endings: LF in the repository and working tree.
* text=auto eol=lf

# Binary assets
*.png binary
*.jpg binary
*.jpeg binary
*.gif binary
*.webp binary
*.ico binary
*.woff binary
*.woff2 binary
*.ttf binary
*.eot binary
*.pdf binary

# Shell scripts must use LF
*.sh text eol=lf
Comment on lines +17 to +18

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Windows batch files (.bat and .cmd) require CRLF line endings to execute correctly in Windows environments. Since the global rule * text=auto eol=lf forces LF line endings on all text files, any batch files added to the repository will be checked out with LF line endings, which can cause syntax and execution errors on Windows. It is recommended to explicitly configure CRLF for these file extensions.

# Shell scripts must use LF
*.sh text eol=lf

# Windows batch files must use CRLF
*.bat text eol=crlf
*.cmd text eol=crlf

152 changes: 76 additions & 76 deletions docs/administrator/compatibility/compatibility.md
Original file line number Diff line number Diff line change
@@ -1,76 +1,76 @@
---
title: Kubernetes Compatibility
---
This document provides a comprehensive guide to Karmada's compatibility with Kubernetes versions, focusing on two
primary dimensions:
- Karmada control plane APIs support: How many APIs are supported?
- Kubernetes versions support: How many Kubernetes versions are supported?
This document helps administrators understand how Karmada manages resources across clusters with mixed Kubernetes versions.
It ensures safe propagation by respecting each cluster’s API capabilities and provides guidelines for maintaining
compatibility during upgrades or migrations.
## Karmada control plane APIs support
Karmada supports all Kubernetes APIs available in the version of kube-apiserver it uses. For example, if Karmada APIServer
uses kube-apiserver@v1.32, it inherits support for every API (e.g., Deployment, Service, Ingress) and API version
(e.g., apps/v1, networking.k8s.io/v1) available in Kubernetes v1.32.
Beyond native Kubernetes APIs, Karmada extends its capabilities through Custom Resource Definitions (CRDs) and
Aggregated APIServer, the same extension mechanisms used in Kubernetes itself:
- **CRD Extensions**: Most Karmada-specific APIs (e.g., PropagationPolicy, OverridePolicy) are implemented as CRDs.
Users can also define custom CRDs in Karmada and propagate them to member clusters, following the same workflow as
native Kubernetes APIs.
- **Aggregated APIServer**: Karmada’s Cluster API is extended by karmada-aggregated-apiserver, thus users can directly
query member cluster resources (e.g., Nodes, Pods) through Karmada’s unified API endpoint.
Users can extend Karmada’s API surface in the same way they extend Kubernetes -- whether for custom resources or
third-party integrations.
To view all supported APIs, run the `karmadactl api-resources` command against Karmada, this lists all supported API
including Kubernetes API and extended APIs, the output like:
```
# karmadactl api-resources
NAME SHORTNAMES APIVERSION NAMESPACED KIND
pods po v1 true Pod
deployments deploy apps/v1 true Deployment
clusters cluster.karmada.io/v1alpha1 false Cluster
overridepolicies op policy.karmada.io/v1alpha1 true OverridePolicy
propagationpolicies pp policy.karmada.io/v1alpha1 true PropagationPolicy
...
```
## Kubernetes versions support
Karmada manages Kubernetes clusters using a minimal set of stable APIs and endpoints, ensuring broad compatibility
across Kubernetes versions. Each Karmada release integrates the latest Kubernetes client libraries and tests
compatibility with the latest 10 Kubernetes versions. For example, Karmada v1.14 uses Kubernetes client v1.32 and is
validated against Kubernetes versions from v1.23 to v1.32.
A resource can be propagated to member clusters only if both Karmada and the target cluster support its API.
- If Karmada does not recognize the API (e.g., unsupported CRDs or deprecated APIs), the resource cannot be created in Karmada.
- If a member cluster lacks the API (e.g., an older cluster missing networking.k8s.io/v1/Ingress), Karmada skips propagation to that cluster.
## FAQ
### How to choose the version of Karmada APIServer?
The Karmada project automatically updates the default kube-apiserver version in its installation tools (e.g., Karmada Operator)
to support the latest Kubernetes APIs. In most cases, users do not need to change it, even in case of upgrading Karmada.
You may need to upgrade Karmada APIServer in cases such as:
- Adopting New Kubernetes APIs: To leverage features introduced in newer Kubernetes versions.
- Addressing CVEs: To resolve security vulnerabilities present in older versions.
Note that upgrading should be taken very carefully:
- Review API Deprecations: Ensure resource types used in your workloads are not deprecated in the target Kubernetes version.
- Incremental Upgrades: Always upgrade one minor version at a time (e.g., kube-apiserver@v1.32 → v1.33).
Avoid skipping versions (e.g., v1.32 → v1.34), as Kubernetes relies on incremental API deprecation and automatic
conversion mechanisms to safely migrate deprecated APIs to newer versions.
### How to handle member clusters with large version gaps?
For scenarios where member clusters span incompatible API versions (e.g., some support networking.k8s.io/v1, others only v1beta1),
Currently, Karmada does not automatically convert API versions according to the capacity of member clusters, instead, it is
recommended to upgrade the outdated clusters to a newer Kubernetes version.
---
title: Kubernetes Compatibility
---

This document provides a comprehensive guide to Karmada's compatibility with Kubernetes versions, focusing on two
primary dimensions:
- Karmada control plane APIs support: How many APIs are supported?
- Kubernetes versions support: How many Kubernetes versions are supported?

This document helps administrators understand how Karmada manages resources across clusters with mixed Kubernetes versions.
It ensures safe propagation by respecting each cluster’s API capabilities and provides guidelines for maintaining
compatibility during upgrades or migrations.

## Karmada control plane APIs support

Karmada supports all Kubernetes APIs available in the version of kube-apiserver it uses. For example, if Karmada APIServer
uses kube-apiserver@v1.32, it inherits support for every API (e.g., Deployment, Service, Ingress) and API version
(e.g., apps/v1, networking.k8s.io/v1) available in Kubernetes v1.32.

Beyond native Kubernetes APIs, Karmada extends its capabilities through Custom Resource Definitions (CRDs) and
Aggregated APIServer, the same extension mechanisms used in Kubernetes itself:
- **CRD Extensions**: Most Karmada-specific APIs (e.g., PropagationPolicy, OverridePolicy) are implemented as CRDs.
Users can also define custom CRDs in Karmada and propagate them to member clusters, following the same workflow as
native Kubernetes APIs.
- **Aggregated APIServer**: Karmada’s Cluster API is extended by karmada-aggregated-apiserver, thus users can directly
query member cluster resources (e.g., Nodes, Pods) through Karmada’s unified API endpoint.
Users can extend Karmada’s API surface in the same way they extend Kubernetes -- whether for custom resources or
third-party integrations.

To view all supported APIs, run the `karmadactl api-resources` command against Karmada, this lists all supported API
including Kubernetes API and extended APIs, the output like:
```
# karmadactl api-resources
NAME SHORTNAMES APIVERSION NAMESPACED KIND
pods po v1 true Pod
deployments deploy apps/v1 true Deployment
clusters cluster.karmada.io/v1alpha1 false Cluster
overridepolicies op policy.karmada.io/v1alpha1 true OverridePolicy
propagationpolicies pp policy.karmada.io/v1alpha1 true PropagationPolicy
...
```

## Kubernetes versions support

Karmada manages Kubernetes clusters using a minimal set of stable APIs and endpoints, ensuring broad compatibility
across Kubernetes versions. Each Karmada release integrates the latest Kubernetes client libraries and tests
compatibility with the latest 10 Kubernetes versions. For example, Karmada v1.14 uses Kubernetes client v1.32 and is
validated against Kubernetes versions from v1.23 to v1.32.

A resource can be propagated to member clusters only if both Karmada and the target cluster support its API.
- If Karmada does not recognize the API (e.g., unsupported CRDs or deprecated APIs), the resource cannot be created in Karmada.
- If a member cluster lacks the API (e.g., an older cluster missing networking.k8s.io/v1/Ingress), Karmada skips propagation to that cluster.

## FAQ

### How to choose the version of Karmada APIServer?

The Karmada project automatically updates the default kube-apiserver version in its installation tools (e.g., Karmada Operator)
to support the latest Kubernetes APIs. In most cases, users do not need to change it, even in case of upgrading Karmada.

You may need to upgrade Karmada APIServer in cases such as:
- Adopting New Kubernetes APIs: To leverage features introduced in newer Kubernetes versions.
- Addressing CVEs: To resolve security vulnerabilities present in older versions.

Note that upgrading should be taken very carefully:

- Review API Deprecations: Ensure resource types used in your workloads are not deprecated in the target Kubernetes version.
- Incremental Upgrades: Always upgrade one minor version at a time (e.g., kube-apiserver@v1.32 → v1.33).
Avoid skipping versions (e.g., v1.32 → v1.34), as Kubernetes relies on incremental API deprecation and automatic
conversion mechanisms to safely migrate deprecated APIs to newer versions.

### How to handle member clusters with large version gaps?

For scenarios where member clusters span incompatible API versions (e.g., some support networking.k8s.io/v1, others only v1beta1),
Currently, Karmada does not automatically convert API versions according to the capacity of member clusters, instead, it is
recommended to upgrade the outdated clusters to a newer Kubernetes version.
88 changes: 44 additions & 44 deletions docs/administrator/upgrading/v1.11-v1.12.md
Original file line number Diff line number Diff line change
@@ -1,45 +1,45 @@
---
title: v1.11 to v1.12
---
Follow the [Regular Upgrading Process](./README.md).
## Upgrading Notable Changes
### API Changes
- Introduced `SecretRef` to `Karmada` API as part of the configuration for connecting to an external etcd cluster can be used to reference a secret that contains credentials for connecting to an external etcd cluster.
- Introduced `extraVolumes` and `extraVolumemounts` to the `Karmada` API to optionally specify extra volumes and volume mounts for the Karmada API server component.
- Introduced `ApiServerService` field to `Karmada` API as part of the Karmada instance status can be used to reference the API Server service for that instance. This is useful for scenarios where higher level operators need to discover the API Server service of a Karmada instance for tasks like setting up ingress traffic.
- Introduced `CustomCertificate.ApiServerCACert` field to `Karmada` API as part of the `Karmada` spec to specify the reference to a secret that contains a custom CA certificate for the Karmada API Server.
- API change: The ServiceType of the Karmada API server in `Karmada` API now has been restricted to `ClusterIP`, `NodePort` and `LoadBalancer`.
- Introduced a new condition `CompleteAPIEnablements` to `Cluster` API to represent the API collection status.
- Introduced `PreserveResourcesOnDeletion` field to both `PropagationPolicy` and `ClusterPropagationPolicy` API, which provides the ability to roll back migration safely.
- API Change: Introduced `FieldOverrider` to both `OverridePolicy` and `ClusterOverridePolicy`, which provides the ability to override structured data nested in manifest like ConfigMap or Secret.
- Introduced `PurgeMode` to `GracefulEvictionTask` in `ResourceBinding` and `ClusterResourceBinding` API.
- Introduced `StatePreservation` to `PropagationPolicy`, which will be used to preserve status in case of application failover.
### Deprecation
- `ExternalEtcd.CAData`, `ExternalEtcd.CertData` and `ExternalEtcd.KeyData` in `Karmada` API are deprecated and will be removed in a future version. Use SecretRef for providing client connection credentials.
- The following flags have been deprecated in release `v1.11.0` and now have been removed:
* `karmada-agent`:
* --bind-address
* --secure-port
* `karmada-controller-manager`:
* --bind-address
* --secure-port
* `karmada-scheduler-estimator`:
* --bind-address
* --secure-port
* `karmada-scheduler`:
* --bind-address
* --secure-port
* `karmada-descheduler`:
* --bind-address
* --secure-port
### karmada-controller-manager
- Introduced feature gate `StatefulFailoverInjection` to control whether Karmada collects and injects state information during a failover event for stateful application.
---
title: v1.11 to v1.12
---

Follow the [Regular Upgrading Process](./README.md).

## Upgrading Notable Changes

### API Changes

- Introduced `SecretRef` to `Karmada` API as part of the configuration for connecting to an external etcd cluster can be used to reference a secret that contains credentials for connecting to an external etcd cluster.
- Introduced `extraVolumes` and `extraVolumemounts` to the `Karmada` API to optionally specify extra volumes and volume mounts for the Karmada API server component.
- Introduced `ApiServerService` field to `Karmada` API as part of the Karmada instance status can be used to reference the API Server service for that instance. This is useful for scenarios where higher level operators need to discover the API Server service of a Karmada instance for tasks like setting up ingress traffic.
- Introduced `CustomCertificate.ApiServerCACert` field to `Karmada` API as part of the `Karmada` spec to specify the reference to a secret that contains a custom CA certificate for the Karmada API Server.
- API change: The ServiceType of the Karmada API server in `Karmada` API now has been restricted to `ClusterIP`, `NodePort` and `LoadBalancer`.
- Introduced a new condition `CompleteAPIEnablements` to `Cluster` API to represent the API collection status.
- Introduced `PreserveResourcesOnDeletion` field to both `PropagationPolicy` and `ClusterPropagationPolicy` API, which provides the ability to roll back migration safely.
- API Change: Introduced `FieldOverrider` to both `OverridePolicy` and `ClusterOverridePolicy`, which provides the ability to override structured data nested in manifest like ConfigMap or Secret.
- Introduced `PurgeMode` to `GracefulEvictionTask` in `ResourceBinding` and `ClusterResourceBinding` API.
- Introduced `StatePreservation` to `PropagationPolicy`, which will be used to preserve status in case of application failover.

### Deprecation

- `ExternalEtcd.CAData`, `ExternalEtcd.CertData` and `ExternalEtcd.KeyData` in `Karmada` API are deprecated and will be removed in a future version. Use SecretRef for providing client connection credentials.
- The following flags have been deprecated in release `v1.11.0` and now have been removed:
* `karmada-agent`:
* --bind-address
* --secure-port
* `karmada-controller-manager`:
* --bind-address
* --secure-port
* `karmada-scheduler-estimator`:
* --bind-address
* --secure-port
* `karmada-scheduler`:
* --bind-address
* --secure-port
* `karmada-descheduler`:
* --bind-address
* --secure-port

### karmada-controller-manager

- Introduced feature gate `StatefulFailoverInjection` to control whether Karmada collects and injects state information during a failover event for stateful application.
- The feature `Failover` now has been disabled by default, which should be explicitly enabled to avoid unexpected incidents.
Loading
Loading