Per-agent model selection is enabled by default without its precheck, allowlist and provider-pinning safeguards

[lfnothias]

Observation

orchestrator_choose_model defaults to True (config.py:72). With it enabled, three safety mechanisms that guard model routing are switched off at once:

• main.py:351 (and 354, 357): PreCheck(config).run(check_provider=not config.orchestrator_choose_model), so the OpenRouter provider probe — which filters unsafe quantizations and checks reachability — does not run.
• workflow_factory.py:338: providers = self.config.openrouter_provider_for(default_model) if self.config.orchestrator_choose_model == False else None, so provider pinning and the quantization filter are dropped for every agent.
• smolagent_factory.py:109: self.model_id = model or MODEL_ID — the model the workflow-generation LLM emitted is used verbatim, with no check against the configured candidate list.

Only the first candidate in smolagent_model_id is ever prechecked (precheck.py:372), so models the orchestrator may assign to agents are neither validated nor pinned.

Why it matters

These are the checks that stop a run from silently routing to an fp8/int4 endpoint that passes a capability probe but corrupts escape sequences in generated code, which is the failure mode precheck exists to prevent. And because the chosen model is unvalidated, a hallucinated or retired model identifier reaches LiteLLM directly. The three symptoms are one design gap rather than three defects: per-agent model selection was added without moving the surrounding safety plumbing along with it.

Pathway

1. Validate the orchestrator's choice: in smolagent_factory, resolve model against the configured candidate list and fall back to the default (with a warning) when it does not match. This alone closes the hallucinated-model path.
2. Precheck every distinct candidate rather than smolagent_model_id[0], so each model an agent may receive has been probed.
3. Pin providers per resolved model instead of disabling pinning wholesale: keep openrouter_provider_for(model) on the per-agent model rather than gating it on orchestrator_choose_model.
4. Decide whether orchestrator_choose_model should default to True before 1–3 are in place; defaulting it off until then would restore the previous safety posture without removing the feature.

Steps 1 and 3 are small and independent. Step 2 costs one probe per candidate at startup.