Add provider-declared OAuth bootstrap; run provider setup before registration

[BillJr99]

Summary

Two follow-ups from running the gmail-filters code provider:

1. Provider-declared OAuth bootstrap — a provider YAML can now declare a one-time Google browser-consent flow, and mcpproxy runs it end-to-end: surfaces the consent URL in the UI, handles the redirect callback, exchanges the code, and writes a token file that google.oauth2.credentials.Credentials.from_authorized_user_file() loads. No more running InstalledAppFlow on a laptop and copying gmail_token.json into the container.
2. Setup-before-register — provider requirements/setup now run before registration, eliminating the noisy failed-first-attempt ModuleNotFoundError tracebacks for code providers that import their declared requirements at module level.

OAuth bootstrap (oauth: block)

oauth:
  type: google            # only supported type today; dispatch tables allow more later
  client_secret_file: /app/tools/secrets/client_secret.json
  token_file: /app/tools/secrets/gmail_token.json
  scopes:
    - https://www.googleapis.com/auth/gmail.settings.basic
  # optional: prompt (default "consent"), login_hint

How it works

• New oauth_bootstrap.py module reuses the existing authorization_code machinery: flows register in AuthCodeTokenStore._pending_flows tagged kind: "google", consent URLs publish into pending_rest_auth (so the existing yellow banner shows them with zero polling changes), and the shared GET /oauth/callback dispatches non-REST flows back via a two-line hook in rest_provider.py.
• The consent URL is built with PKCE, access_type=offline, and prompt=consent (what makes Google issue a refresh_token). client_secret.json is parsed for both installed and web client types.
• The exchanged token is written in Credentials.to_json() format, so provider code keeps calling Credentials.from_authorized_user_file() unchanged and the google libs refresh at call time. If Google omits the refresh_token (prior silent consent), an existing one is salvaged from the old token file, else a clear error explains the revoke-and-reconsent fix.
• A startup oauth-warmup thread surfaces missing tokens in the banner before the first failed tool call (gated by MCPPROXY_WARM_REMOTE like the other warm-ups).

UI / API

• POST /api/oauth-bootstrap {name} begins/restarts the flow on demand.
• Provider editor gets a read-only OAuth summary (paths, scopes, token status, redirect URI) with a 🔐 Authorize button; the provider list shows a 🔐 badge while no usable token exists; /api/tools exposes per-provider token status.
• The oauth: block round-trips through the structured editor and is validated (type/paths/scopes, plus a client_secret_file existence check that points at the Files manager).
• README documents the block, the installed-vs-web redirect-URI rules, and the prompt=consent caveat.

Setup-before-register

bootstrap_provider() now runs run_provider_setup() first, then register_provider() — both steps already ran sequentially before the MCP server starts, so startup time is unchanged. A setup failure still doesn't block registration (tools advertise and surface errors at call time), and the previous install-then-retry path is no longer needed.

Testing

• 41 new tests (466 total passing): client-secret parsing, consent-URL contents and flow registration, token exchange and written file format (loaded back through real google-auth), refresh-token salvage and the consent-hint error, callback dispatch with a REST-flow regression test, warm-up behavior, the new endpoint, structured round-trip, and validation.
• Live end-to-end smoke against a fake Google token endpoint: startup warm-up → banner URL → /api/oauth-bootstrap → /oauth/callback → token file loadable by Credentials.from_authorized_user_file() → banner cleared → list badge flips.
• Embedded UI JavaScript passes node --check; the Dockerfile guard test caught and verified the new module's COPY line.

https://claude.ai/code/session_01WnK1rtXGHDCNpsycAvxFqC

---

Generated by Claude Code