WorkloadRebalancer does not fail back to the primary cluster after recovery when using clusterAffinities

[surenthar-rajamohan-work]

What happened:

I believe I have encountered unexpected behavior with clusterAffinities when using it for a disaster recovery (primary/backup) deployment.

It was originally raised in #4990 after seeing that the issue had been closed with the note that this feature was implemented in v1.18.

I am following the disaster recovery example documented here:

https://karmada.io/docs/userguide/scheduling/propagation-policy/#use-case-2-disaster-recovery-primary-and-backup-clusters

My PropagationPolicy uses two affinity groups:

clusterAffinities:
  - affinityName: primary-cluster
    clusterNames:
      - member-cluster-1

  - affinityName: backup-cluster
    clusterNames:
      - member-cluster-2

The workload initially runs on member-cluster-1 as expected.

When I make member-cluster-1 unavailable, the workload correctly fails over to member-cluster-2.

After restoring member-cluster-1 to a healthy state, I manually created a WorkloadRebalancer for the Deployment.

The WorkloadRebalancer completed successfully:

Status:
  Finish Time:          2026-07-03T02:46:31Z
  Observed Generation:  1
  Observed Workloads:
    Result:  Successful
    Workload:
      API Version:  apps/v1
      Kind:         Deployment
      Name:         example-deployment
      Namespace:    example-namespace

The ResourceBinding also confirms that rescheduling was triggered:

Reschedule Triggered At:  2026-07-03T02:46:31Z
Last Scheduled Time:     2026-07-03T02:46:31Z

However, the scheduler continues selecting the backup cluster instead of moving the workload back to the primary cluster.

The resulting ResourceBinding shows:

selected=[{"name":"member-cluster-2","replicas":1}]
observingAffinity=backup-cluster

The scheduling event is:

Normal  ScheduleBindingSucceed  default-scheduler
Binding has been scheduled successfully.
Result: {member-cluster-2:1}

The final ResourceBinding also reports:

Scheduler Observing Affinity Name: backup-cluster

even though member-cluster-1 is healthy and satisfies the placement policy.

---

What you expected to happen:

Based on the documentation, I expected the scheduler to evaluate the affinity groups again after WorkloadRebalancer triggered rescheduling.

Since member-cluster-1 had recovered and belongs to the first affinity group (primary-cluster), I expected the workload to be moved back from member-cluster-2 to member-cluster-1.

In other words, I expected WorkloadRebalancer to perform a failback to the highest-priority matching affinity group.

---

How to reproduce it (as minimally and precisely as possible):

1. Create two member clusters:

   • member-cluster-1
   • member-cluster-2

2. Create a PropagationPolicy similar to:

placement:
  clusterAffinities:
    - affinityName: primary-cluster
      clusterNames:
        - member-cluster-1

    - affinityName: backup-cluster
      clusterNames:
        - member-cluster-2

  spreadConstraints:
    - spreadByField: cluster
      minGroups: 1
      maxGroups: 1

  replicaScheduling:
    replicaSchedulingType: Duplicated

failover:
  cluster:
    purgeMode: Directly

3. Deploy a workload.

4. Verify the workload is scheduled onto member-cluster-1.

5. Make member-cluster-1 unavailable (or otherwise unschedulable).

6. Verify the workload fails over to member-cluster-2.

7. Restore member-cluster-1 until it reports:

Ready=True

and ensure both clusters have no taints.

8. Create a WorkloadRebalancer:

apiVersion: apps.karmada.io/v1alpha1
kind: WorkloadRebalancer
metadata:
  name: example-failback-to-primary
spec:
  ttlSecondsAfterFinished: 600
  workloads:
    - apiVersion: apps/v1
      kind: Deployment
      name: example-deployment
      namespace: example-namespace

9. Observe that:

• WorkloadRebalancer completes successfully.
• ResourceBinding is rescheduled.
• The scheduler still selects member-cluster-2.
• Scheduler Observing Affinity Name remains backup-cluster.

---

Anything else we need to know?:

The prerequisite resources (Namespace, ServiceAccount, ConfigMap, SealedSecret) are propagated to both clusters using a separate PropagationPolicy, so the primary cluster already contains all required dependent resources.

The recovered cluster reports:

member-cluster-2 taints=
member-cluster-1 taints=

CompleteAPIEnablements=True
Ready=True

The ResourceBinding after rebalance shows:

Spec:
  Clusters:
    Name: member-cluster-2
    Replicas: 1

Placement:
  Cluster Affinities:
    primary-cluster
      member-cluster-1

    backup-cluster
      member-cluster-2

Reschedule Triggered At:
  2026-07-03T02:46:31Z

Status:
  Scheduler Observing Affinity Name: backup-cluster

The scheduler event is:

Normal ScheduleBindingSucceed

Binding has been scheduled successfully.
Result: {member-cluster-2:1}

I also verified that this is not caused by an older API version.

The running components are newer than the v1.18.0 release, and the API exposes the new overflowAffinities field:

kubectl explain propagationpolicy.spec.placement.clusterAffinities.overflowAffinities

Therefore, it appears that the scheduler supports the newer affinity API.

My main question is whether this behavior is expected.

Should clusterAffinities only be used for failover, or should WorkloadRebalancer also trigger failback to the highest-priority affinity group once the preferred cluster becomes available again?

If additional configuration is required for failback, could you please clarify what is missing from my configuration?

---

Environment:

• Karmada version:

Scheduler:

GitVersion: v1.18.0-40-g7edc6e691
GitCommit: 7edc6e6919cc0207007383ebd5078ca3a4fbc13d

Controller Manager:

GitVersion: v1.18.0-66-gffbade988
GitCommit: ffbade988fc3d6652daa504ff6cac37141e9f755

Webhook:

GitVersion: v1.18.0-66-gffbade988
GitCommit: ffbade988fc3d6652daa504ff6cac37141e9f755

• kubectl-karmada or karmadactl version:

Not installed (using kubectl directly).

• Others:

• WorkloadRebalancer completes successfully.

• ResourceBinding is rescheduled successfully.

• Scheduler continues selecting the backup affinity group after the primary cluster has recovered.

• overflowAffinities is available in the CRD/API.

[zhzhuang-zju]

This is expected. If you want the workload to return to the primary cluster after workloadRebalancer, you should define overflowAffinities instead of defining multiple ClusterAffinity.
More info refer to https://karmada.io/docs/userguide/scheduling/propagation-policy/#use-case-3-overflow-from-idc-gpu-clusters-to-cloud-gpu-clusters