[Bug]: reasoning_effort is inverted below "high" — minimal/low/medium all render "Reasoning Effort: Max"

[ThefloorMiner]

Summary

render_chat() maps every accepted reasoning_effort value except "high" onto Max:

if enable_thinking:
    effort = "High" if reasoning_effort == "high" else "Max"
    prompt.append(f"<|system|>Reasoning Effort: {effort}")

The endpoint accepts none, minimal, low, medium, high, xhigh. Of the five that enable thinking, four render Max — including minimal and low. A client asking for the least reasoning gets the most.

The ordering is not merely coarse, it is non-monotonic:

requested	rendered
minimal	Max
low	Max
medium	Max
high	High
xhigh	Max

Measured, so this is not a reading of the source alone

GLM-5.2 744B on 4× RTX A6000, same prompt with a tool declaration, temperature 0, measuring the length of the returned reasoning block:

condition	reasoning emitted
default (no reasoning_effort, no enable_thinking)	none
reasoning_effort=high	120 chars
reasoning_effort=xhigh → renders Max	178 chars
COLI_THINK=1	120 chars

So the two levels are real and distinguishable — Max genuinely produces more reasoning than High. Which makes the inversion a behaviour change rather than a cosmetic one: a client throttling to minimal to save tokens gets the most expensive setting available.

Second, related: COLI_THINK=1 cannot reach Max

if (reasoning_effort is None and "enable_thinking" not in body
        and os.environ.get("COLI_THINK", "0") == "1"):
    reasoning_effort = "high"

COLI_THINK is the operator-side switch for clients that send neither field — which is the common case; Open WebUI, for one, sends neither. It hard-codes "high", so an operator can turn thinking on but cannot select the level, and specifically cannot select the one level the code goes out of its way to support.

Why this is easy to miss

The default path is enable_thinking=False, and then no Reasoning Effort line is emitted at all. So on a stock deployment the mapping is unreachable: you get no reasoning, which looks like "the model does not think" rather than "the effort ladder is inverted". I only found it because I went looking for why a deployment had never once emitted a reasoning block.

Suggested shape

An ordered mapping rather than a two-way branch — minimal/low → Low, medium → Medium, high → High, xhigh → Max, or whatever ladder the template actually supports; I have only verified that High and Max are both accepted and behave differently. And letting COLI_THINK carry a level (COLI_THINK=max) rather than being boolean, with 1 keeping today's meaning.

Happy to send that as a PR if the ladder above is the one you want. I did not open with a patch because the set of levels GLM-5.2 accepts is a question about the model, not the code, and you will know it better than my two measured points do.

---

Observed on dev @ 7fb1159, Linux, CUDA build sm_86, 4× RTX A6000, model mastouri/GLM-5.2-colibri-int4-g64-with-int8-mtp.

[JustVugg]

Confirmed at openai_server.py:806, and it is exactly as you describe:

if enable_thinking:
    effort = "High" if reasoning_effort == "high" else "Max"

Four of the five thinking levels render Max. A client asking for minimal — the cheapest setting the endpoint accepts — receives the most expensive one the model has.

What makes this a behaviour bug rather than a rounding of a coarse scale is your measurement: 120 chars at High, 178 at Max, same prompt at temperature 0. The two levels are genuinely distinguishable, so the inversion is spending real tokens against the client's explicit instruction. A throttling client is the one that gets hurt most, which is the opposite of what throttling is for.

And the ordering is not just coarse, it is non-monotonic — minimal and xhigh land on the same string while high sits below both. Any client reasoning about the scale monotonically is wrong about this endpoint.

On the second half — COLI_THINK=1 cannot reach Max

Also right, and worth keeping in the same fix. COLI_THINK exists so a deployment can make thinking the default without every client asking; capping that path at one level while an explicit reasoning_effort=xhigh reaches the other means the env var is not "thinking on by default", it is "thinking on, at a level you cannot choose".

What the fix should be

The mapping has to be monotone and cover the whole scale the endpoint already accepts. GLM's template takes a string, so this is a table, not a computation:

none     -> no Reasoning Effort line at all (thinking off)
minimal  -> Minimal
low      -> Low
medium   -> Medium
high     -> High
xhigh    -> Max

Two things I would want in the PR:

Whatever GLM's template actually accepts. High and Max are known good because you measured them. If Minimal/Low/Medium are not strings the model was trained on, they will be out of distribution and the fix would trade an inversion for something worse. If only two levels are real, then the honest mapping is minimal|low|medium -> High and high|xhigh -> Max — still monotone, still never inverted, and no invented tokens.

COLI_THINK reaching the top of whatever scale survives, so the env default is not permanently second-class.

Ask

Do you want to take this one? You have the 4× A6000 host, the measurement harness, and you already know which strings produce distinguishable reasoning — which is the part I cannot check from here and the part that decides between the two mappings above. A short A/B on Minimal/Low/Medium against High would settle it.

If you would rather not, say so and I will take it, but I would be guessing at the template's vocabulary where you can measure it.

Labelled bug + quality. Thank you — four reports today and every one of them had the measurement attached.