Skip to content

Plan voting silently degenerates to "first plan wins" when plan_weight is absent #4

Description

@Rasputin02

plan_vote_and_update is described as a weighted vote across plan runs. It is — but the default weight is zero, and a vote where every ballot weighs zero is not a vote.

        try:
            w = float(pr.get("plan_weight", 0.0) or 0.0)
        except Exception:
            w = 0.0
        weights.append(w)

core/voting.py:72-76

        win_key = max(score.items(), key=lambda kv: kv[1])[0]

core/voting.py:84

Failure scenario

  • Trigger: call it with three plan runs that do not carry plan_weight — which the docstring at core/voting.py:32 explicitly permits ("optional, default 0.0"):

    plan_vote_and_update(
        plan_runs=[{"final_text": "42"}, {"final_text": "17"}, {"final_text": "17"}],
        selector=None, policy_name="linucb",
    )
  • Observed: score == {"42": 0.0, "17": 0.0}. max over equal values returns the first item in insertion order, so win_key == "42" — the minority answer. Two of three runs agreed on 17 and the vote ignored them.

  • Expected: with no weights supplied the function should fall back to plurality (count of runs), giving 17.

Note that the truthiness guard makes it worse in a subtler way: pr.get("plan_weight", 0.0) or 0.0 maps an explicit plan_weight=0.0 and an explicit plan_weight=None and a missing key to the same value, so a caller cannot distinguish "this plan is worthless" from "I have no opinion".

Knock-on effect on the bandit

This is not just a reporting problem. The winner drives the reward update:

        for pr, k in zip(plan_runs or [], keys):
            if k != win_key:
                continue
            for rec in pr.get("step_records", []) or []:

core/voting.py:90-93

So an arbitrary insertion-order winner hands win_bonus to the agents that served the first plan run, on every call, forever. That is a systematic positive bias toward whichever plan the orchestrator happens to generate first — precisely the "always selecting the same agent" pathology the routing layer's docstring says the design is trying to avoid (core/routing.py:15).

Suggested fix

if not any(w > 0.0 for w in weights):
    # no weights supplied -> plurality vote
    for k in keys:
        score[k] = score.get(k, 0.0) + 1.0

and break ties deterministically and explicitly — max(score.items(), key=lambda kv: (kv[1], -keys.index(kv[0]))) or simply document that ties go to the earliest plan. Right now the tie-break is an accident of dict ordering.

One dead branch while you're in here

                    try:
                        selector.update(x, float(win_bonus))
                    except TypeError:
                        # in case selector.update signature differs
                        selector.update(x, win_bonus)

core/voting.py:95-100

The fallback calls the same method with the same two positional arguments; float(win_bonus) and win_bonus are both floats by then. If update raises TypeError the first time it will raise it again, and the second raise is uncaught. Either delete the handler or make the fallback actually different (e.g. selector.update(x=x, reward=win_bonus)).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions