Skip to content

[feature]: generic OpenAI-compatible gateway quota provider (configurable base URL) #108

@rice-as681

Description

@rice-as681

Pre-flight checks

  • I searched existing issues and did not find a duplicate request (no open issue mentions openrouter / base url / custom / generic / compatible / gateway).
  • I reviewed current behavior against the published OpenCode config schema + plugin API and the current provider architecture in this repo.

OpenCode version reviewed: 1.15.12 (config schema opencode.ai/config.json + plugin/provider docs)

Problem statement

Every provider hardcodes its vendor's balance/quota URL in src/lib/<vendor>.ts, and createProviderApiKeyResolver reads only an API key — never a base URL. So self-hosted / OpenAI-compatible gateways (LiteLLM proxies, in-house or university gateways, Vertex/Apigee fronts, OpenRouter-compatible endpoints) can't be supported, even though users already configure them as ordinary OpenAI-compatible providers for chat. There's no standard remaining-quota endpoint, so each such gateway publishes its own small JSON — but the plugin currently has no way to point at one.

Proposed change

A single, config-driven generic provider (openai-compatible) that polls a configurable quota endpoint and maps a small JSON shape into the normalized entries. One provider covers any number of gateways:

"experimental": { "quotaToast": { "openaiCompatibleGateways": [
  { "providerId": "my-gateway", "quotaPath": "/quota" }
  // baseURL inherited from provider.<id>.options.baseURL; apiKey via the existing resolver
] } }
  • Key resolved via the existing createProviderApiKeyResolver (env / trusted global config / auth.json), keyed on providerId.
  • GET <baseURL><quotaPath> (Bearer) → a default vendor-neutral shape
    { key, tokens:{limit,used,remaining,resets_at}, cost:{currency,limit,used,remaining} },
    with a built-in openrouter preset mapping (data:{usage,limit,limit_remaining}) so OpenRouter-style endpoints work too. Token bucket → percent entry; cost → value entries. Bound to no product.

I'd start from contributing/provider-template/ and follow the existing resolver/result-helper patterns.

Alternatives considered

  • Hardcode OpenRouter (add an openrouter.ts): narrower (one product) and binds to a product API rather than the open OpenAI-compatible class.
  • Read a marker off provider.<id>.options instead of plugin config: avoids new plugin-config surface, but risks OpenCode rejecting unknown options keys and couples to the chat provider block.

Design questions for maintainer input

  1. Config location — is experimental.quotaToast.openaiCompatibleGateways the right home, or do you prefer reading markers off the OpenCode provider.* block?
  2. matchesCurrentModel — its signature (model, context) can't see config; for a config-driven provider, is resolving matches via the configured providerIds (cached at fetch time) acceptable, or would you rather extend the signature/context?
  3. enabledProviders — should it accept user-defined gateway ids, or stay limited to the canonical openai-compatible id?

Acceptance criteria

  • openai-compatible provider registered; config-driven gateways list validated like other settings.
  • Default neutral shape + openrouter preset mapping, both unit-tested (env/config/auth key sources covered).
  • pnpm run typecheck && pnpm test && pnpm run build green; started from contributing/provider-template/.
  • README documents setup; /quota_status reports the provider's key source.

Happy to open the PR against this design — wanted your steer on questions 1–3 first. Context: built for a university LLM gateway that exposes the neutral /quota contract above; I have an implementation in progress on a fork.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions