Skip to content

feat(providers): Support DeepInfra as an LLM provider#5521

Open
ats3v wants to merge 1 commit into
NousResearch:mainfrom
ats3v:feat/deepinfra-integration
Open

feat(providers): Support DeepInfra as an LLM provider#5521
ats3v wants to merge 1 commit into
NousResearch:mainfrom
ats3v:feat/deepinfra-integration

Conversation

@ats3v

@ats3v ats3v commented Apr 6, 2026

Copy link
Copy Markdown

What does this PR do?

Adds DeepInfra as a first-class LLM provider, giving users access to 100+ top open source and frontier models (GLM-5, DeepSeek-V3+, Qwen3+, Kimi-K2+, Nemotron, etc.) via a single API key.

DeepInfra is one of the largest open-model inference platforms and a top provider on OpenRouter. This integration lets users connect directly — lower latency, simpler billing, no middleman.

The implementation follows the exact same pattern as HuggingFace, KiloCode, and other API-key providers. Purely additive — no changes to existing provider behavior.

Related Issue

N/A — new provider integration.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

Source:

  • hermes_cli/models.py — Add _fetch_deepinfra_catalog, _fetch_deepinfra_models_by_tag, _fetch_deepinfra_pricing; switch to ?filter=true&sort_by=hermes; empty out the static fallback list.
  • agent/model_metadata.py — Extend _extract_pricing() to recognize DeepInfra's $/MTok pricing keys.
  • tools/tts_tool.py — Register deepinfra in BUILTIN_TTS_PROVIDERS; parametrize _generate_openai_tts with optional creds; thin _generate_deepinfra_tts shim.
  • tools/transcription_tools.py — Parametrize _transcribe_openai with optional creds + label; thin _transcribe_deepinfra shim; gate + auto-detect chain.
  • hermes_cli/tools_config.py — Add the "DeepInfra TTS" picker row.
  • hermes_cli/doctor.py — Allowlist deepinfra in providers_accepting_vendor_slugs (kills a false-positive warning).
  • hermes_cli/config.py — Mention deepinfra in the stt.provider doc-string.
  • plugins/model-providers/deepinfra/__init__.py (new)ProviderProfile registration; default_aux_model="stepfun-ai/Step-3.5-Flash".
  • plugins/model-providers/deepinfra/plugin.yaml (new)kind: model-provider manifest.
  • plugins/image_gen/deepinfra/__init__.py (new)DeepInfraImageGenProvider; live catalog discovery via the image-gen tag; SDK-based generate().
  • plugins/image_gen/deepinfra/plugin.yaml (new)kind: backend manifest.

Docs:

  • cli-config.yaml.example — Document optional tts.deepinfra / stt.deepinfra / image_gen.deepinfra blocks.

Tests (invariants only — no model-id snapshots):

  • tests/hermes_cli/test_api_key_providers.py — Add tag-filter, pricing, profile classes; reshape existing tests for the catalog-as-source-of-truth model.
  • tests/plugins/image_gen/test_deepinfra_provider.py (new) — 6 tests covering availability, catalog filter, no-model error, SDK delegation, SDK-exception handling.
  • tests/tools/test_tts_deepinfra.py (new) — 5 tests covering registration, offline contract, no-model raise, delegation, dispatch.
  • tests/tools/test_transcription_deepinfra.py (new) — 5 tests covering offline contract, gating, no-model error, delegation, dispatch.

How to Test

  1. Setup:
    # Get a key at https://deepinfra.com/dash/api_keys
    echo "DEEPINFRA_API_KEY=<your-key>" >> ~/.hermes/.env
    
  2. Provider selection:

hermes model # Select "DeepInfra" from the menu

  1. Direct chat:

hermes chat --provider deepinfra -m "deepseek-ai/DeepSeek-V3-0324" -q "Hello"

  1. Tool calling (agent loop):

hermes chat --provider deepinfra -m "deepseek-ai/DeepSeek-V3-0324" -t terminal -q "List files in the current directory"

  1. Auto-detection:

With only DEEPINFRA_API_KEY set (no other provider keys), hermes chat auto-selects DeepInfra.

  1. Unit tests:

pytest tests/test_api_key_providers.py -k deepinfra -v

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: Ubuntu 24.04 on WSL2.0

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

image image image

@ats3v ats3v force-pushed the feat/deepinfra-integration branch 3 times, most recently from 15cc954 to e3cc103 Compare April 8, 2026 10:51
@ats3v ats3v force-pushed the feat/deepinfra-integration branch 3 times, most recently from 9b24f0c to c7fe34f Compare April 18, 2026 04:15
@ats3v ats3v force-pushed the feat/deepinfra-integration branch from c7fe34f to 2da92e5 Compare April 23, 2026 06:22
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/cli CLI entry point, hermes_cli/, setup wizard comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint area/auth Authentication, OAuth, credential pools labels Apr 23, 2026
@ats3v ats3v force-pushed the feat/deepinfra-integration branch from 2da92e5 to 48e309d Compare April 23, 2026 11:16
@ats3v ats3v force-pushed the feat/deepinfra-integration branch 3 times, most recently from ebd0a12 to 65845f4 Compare May 4, 2026 04:41
@ats3v ats3v force-pushed the feat/deepinfra-integration branch 2 times, most recently from 6f97129 to 5b5b611 Compare May 13, 2026 11:48
@ats3v ats3v force-pushed the feat/deepinfra-integration branch 7 times, most recently from 20421de to b06245e Compare May 21, 2026 04:48
@ats3v ats3v force-pushed the feat/deepinfra-integration branch 4 times, most recently from e96934e to 629bcf1 Compare June 1, 2026 07:46
@ats3v ats3v force-pushed the feat/deepinfra-integration branch 4 times, most recently from adf57cd to 5af7a3a Compare June 8, 2026 04:21
@ats3v ats3v force-pushed the feat/deepinfra-integration branch 3 times, most recently from 65218f2 to 970459d Compare June 18, 2026 13:24
@ats3v ats3v force-pushed the feat/deepinfra-integration branch 3 times, most recently from 3fe13ad to 05a10c2 Compare June 24, 2026 07:22
@ats3v ats3v force-pushed the feat/deepinfra-integration branch 3 times, most recently from c3f9b6a to 4190e9b Compare July 6, 2026 08:41
@ats3v ats3v force-pushed the feat/deepinfra-integration branch from 4190e9b to 5a425fb Compare July 9, 2026 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants