Cookbook pip installs on a server without pip show "running" forever — no preflight, failure never surfaced

[catagris]

Prerequisites

• I searched open issues and discussions and did not find an existing report of this bug.
• This is not a security vulnerability. (Vulnerabilities go to GitHub Security Advisories — see SECURITY.md.)
• I am running the latest code from the dev branch (the default branch you get on clone, where fixes land first) and the bug still reproduces there. Please git pull the latest dev before filing.

Install Method

Docker (docker compose up)

Operating System

Linux

Steps to Reproduce

1. Add a remote server whose python has no pip module (stock Arch/CachyOS: python-pip is a separate package) and leave the server profile's env as None.
2. Cookbook → Dependencies → install any pip package (e.g. hf_transfer) on that server.
3. Watch the Running tab.

Expected Behaviour

The install either runs, or fails fast with an actionable error telling the user what's wrong with the target's Python environment (no pip / PEP 668) and how to fix it.

Actual Behaviour

The runner dies in under a second with /usr/bin/python3: No module named pip and prints its === Process exited with code 1 === sentinel — but then (by design) drops into an interactive shell so the tmux session stays inspectable. The task card shows "running" indefinitely (an hour, in my case): tmux-session-alive is treated as process-alive, the user's shell greeting overwrites the pane, and "Copy last 50 lines" reports no log content. Nothing anywhere tells the user the install already failed or why.

Two layers are missing:

1. No preflight — /api/model/serve happily creates a pip-install task on a target whose python cannot run pip at all.
2. No diagnosis — _diagnose_serve_output has no patterns for No module named pip or PEP 668's externally-managed-environment, so even when the output is captured there's no remediation shown.

Logs / Screenshots

/tmp/odysseus-tmux/serve-<id>.log on the target after the "stuck" install:

[odysseus] HF token: NOT SET — gated/private models will be denied. Add one in Odysseus Cookbook -> Settings -> HuggingFace Token.
/usr/bin/python3: No module named pip

=== Process exited with code 1 ===

(task card meanwhile: status "running", elapsed climbing, no output)

Model / Backend (if relevant)

No response

Are you willing to submit a fix?

Yes — I can open a PR

Additional Information

Fix ready as a PR: /api/model/serve preflights <python> -m pip --version on the target (honouring the profile's venv/conda env prefix, via sh -lc so it also works on non-POSIX login shells) before creating any pip-install task, and returns an immediate actionable error (venv recommended; distro package named). Plus _diagnose_serve_output patterns for No module named pip and externally-managed-environment for pip failures reaching a runner via other paths. Verified on a real Arch server: env=None now errors instantly with the guidance; with a venv configured the same install succeeds.

Related: #5689 (the probe/launch chain on non-POSIX login shells), and the "runner keeps the tmux session open after exit" design that turns any instant failure into a phantom running task.