CLI: add a model catalogue and model-selection UX (`praisonai models` + model-string validation)

[MervinPraison]

Summary

The CLI has no way to discover which models are available, their capabilities (tool-calling, vision, reasoning), context-window limits, or pricing — and it does not validate a --model / YAML llm: string before use. New and daily users must guess provider/model identifiers, and a typo or unsupported model only surfaces as a late, opaque runtime error from the underlying client. This is a core "few lines to first success" gap on the CLI-first path.

Current behaviour

• There is no models command. The 50+ command tree (src/praisonai/praisonai/cli/app.py) exposes config, auth, setup, run, etc., but nothing to list or describe models.
• Model strings are passed straight through to the endpoint resolver (cli/commands/run.py → llm/credentials.py:resolve_llm_endpoint_with_credentials → llm/env.py:resolve_llm_endpoint) with no validation, no capability lookup, and no "did you mean …" suggestion.
• The no-argument TUI hardcodes a default (AsyncTUIConfig(model="gpt-4o-mini") in cli/app.py) with no way for the user to see alternatives or their limits.
• Result: users cannot answer basic questions from the terminal — "which models can I use?", "does this model support tools/vision?", "what is its context limit and cost?" — and a wrong model id fails late.

Desired behaviour

• praisonai models lists available models grouped by provider, showing capability flags (tool-calling, vision, reasoning), context/output limits and (where known) cost. Support --provider <name>, --json, and a filter argument.
• praisonai models describe <provider/model> prints full metadata for one model.
• Model-string resolution validates the requested id against the catalogue and, on a miss, raises a clear error with close-match suggestions (reuse the existing difflib.get_close_matches pattern already used for field typos in agents_generator.py).
• Catalogue data is sourced from metadata already available via the optional litellm dependency (model cost/context maps), cached locally with a short TTL, and degrades gracefully when offline or when litellm is not installed.

Layer placement

• Primary layer: wrapper (src/praisonai/praisonai/)
• Why not core: catalogue fetching, caching, Rich-table presentation and CLI wiring are heavy I/O + UI concerns; the protocol-driven core must stay lightweight. Core already holds the model router; it should not own a user-facing catalogue command.
• Why not tools: this is not an agent-callable integration — agents do not call "list models" as a task tool; it is operator/developer tooling.
• Why not plugins: it is not a lifecycle hook, guardrail or policy.
• Secondary touch (optional): a tiny ModelInfoProtocol in core could standardise the metadata shape, with the wrapper providing the litellm-backed adapter.
• 3-way surface (CLI + YAML + Python): yes — CLI praisonai models; YAML llm:/model: values validated against the catalogue at load; Python helper (e.g. praisonai.models.list_models() / describe_model()).

Proposed approach

1. Add cli/commands/models.py (Typer/Click group) registered in the lazy command table in cli/app.py.
2. Add a wrapper-side catalogue module that reads litellm's cost/context metadata, normalises it, and caches to ~/.praison/cache/models.json with a TTL; offline/missing-dep fallback returns a curated static subset.
3. Add a validate_model(model: str) helper used by run/TUI/agents_generator resolution that raises a clear error with suggestions on miss.
4. Surface capability flags so YAML validation (see related validation work) can warn when, e.g., tools are configured for a non-tool-calling model.

Resolution sketch

cli/commands/models.py        # `models list|describe` Typer group
llm/catalogue.py (wrapper)    # litellm-backed, cached, offline fallback
llm/env.py / credentials.py   # call validate_model() before resolving
cli/app.py                    # register "models" in _LAZY_COMMANDS

Severity

Medium — not blocking, but a daily-use and first-success gap that makes model selection guesswork and pushes avoidable failures to runtime.

Validation

• praisonai models lists models with capabilities/limits; --json machine-readable.
• praisonai run "hi" --model gpt-5o-typo fails fast with a suggestion, not a late client error.
• Real agentic test: agent.start("real task") on a catalogue-validated model still calls the LLM and succeeds; an unknown model id is rejected before any network call.
• Works with and without litellm installed (graceful fallback), and offline (cached).

[praisonai-triage-agent]

Claude finished @MervinPraison's task —— View job • claude/issue-2123-20260621-0841 • [Create PR ➔](https://github.com/MervinPraison/PraisonAI/compare/main...claude/issue-2123-20260621-0841?quick_pull=1&title=Issue%20%232123%3A%20Changes%20from%20Claude&body=This%20PR%20addresses%20issue%20%232123%0A%0AGenerated%20with%20%5BClaude%20Code%5D(https%3A%2F%2Fclaude.ai%2Fcode)

---

✅ Task completed successfully!

Implemented Features

New praisonai models Command

• praisonai models list - List available models with capabilities, limits, and costs
• praisonai models describe <model> - Show detailed information for a specific model
• praisonai models validate <model> - Check if a model ID is valid

Model Catalogue System

• Dynamic discovery via litellm when available
• Graceful fallback to curated list when litellm not installed
• Local caching with 1-hour TTL to reduce lookups
• Smart suggestions using difflib for typo correction

Model Validation Integration

• Model strings validated during LLM resolution
• YAML agent configs checked for invalid models
• Warnings when tools configured for non-tool-calling models
• "Did you mean..." suggestions for misspelled models

Implementation Details

✅ Architecture compliant: All code in wrapper layer (src/praisonai/praisonai/)
✅ No core SDK changes: Maintains protocol-driven design
✅ Backward compatible: Validation can be disabled via flag
✅ Dependency optional: Works with or without litellm

Files Changed

• cli/commands/models.py - New models command implementation
• llm/catalogue.py - Model catalogue with caching and validation
• llm/env.py - Added model validation to resolution
• llm/credentials.py - Updated to pass validation flag
• agents_generator.py - Added YAML model validation
• cli/app.py - Registered models command

Testing

All functionality tested successfully:

• ✅ Model listing and filtering
• ✅ Model description retrieval
• ✅ Invalid model rejection with suggestions
• ✅ YAML configuration validation
• ✅ Graceful litellm fallback

Pull Request

Created PR #2127: feat: add model catalogue and model-selection UX
|