feat(ai): route Anthropic/OpenAI LLM spend through chittyclaw AI Gateway#103
feat(ai): route Anthropic/OpenAI LLM spend through chittyclaw AI Gateway#103chitcommit wants to merge 8 commits into
Conversation
Adds standard ChittyOS branding header with tier badge, org badge, and accurate service description to README.md. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
myCh1tty (mychitty.com) — "Your Ch1tty." CHARTER defines: - Per-user customization layer (custom connections, unbounded learning) - Commercial trigger: first custom connection (need-based, not time-gated) - Innovation flowback to Ch1tty core with Foundation attribution CHITTY defines: - Architecture: Ollama model forked from Ch1tty base - Lifecycle: starts when user needs first custom connection - Three-step resume: entity → topic → go Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ChittyComptroller captures provider LLM spend from the chittyclaw CF AI Gateway logs (-> chittyops.cost_ledger). Previously the chittycan AI plugins defaulted to DIRECT provider URLs (api.anthropic.com / api.openai.com), so all Anthropic/OpenAI calls bypassed the gateway and were invisible to the Comptroller. Repoint each plugin's DEFAULT baseUrl at its per-provider chittyclaw gateway URL (design (a)), keeping the optional override so callers can still point elsewhere (proxy / direct). When the resolved baseUrl is the gateway, attach `cf-aig-authorization: Bearer <token>`, sourced from options.cfAigToken -> env CF_AIG_TOKEN / CF_ISSUED_AIGATEWAY_TOKEN (never hardcoded). The provider key stays in its native header (x-api-key / Authorization) and is forwarded upstream by the gateway. Design: (a) per-provider gateway URLs, not the OpenAI-compat endpoint. Each client already appends its provider-native path (/messages, /chat/completions); compat would force rewriting every request body into OpenAI chat format. Note the suffix is NOT uniform — anthropic keeps /v1 (.../anthropic/v1 + /messages), openai drops it (.../openai + /chat/completions). Verified against CF AI Gateway docs. Shared helper src/plugins/ai/gateway.ts: per-provider base URLs, gateway-detection (only inject cf-aig header for gateway URLs, so a non-gateway override never leaks the token), and token resolution. Both the request() and streaming fetch sites are patched in each plugin. Scope: Anthropic + OpenAI (the providers with existing chittycan plugins). Google has no chittycan plugin today — follow-up. groq/cohere/ together/replicate/huggingface/ollama are out of the stated scope. Verification: live call routed through the gateway to provider=anthropic (model claude-3-5-haiku), captured in chittyclaw gateway logs as the first non-workers-AI entry (status 401 = reached Anthropic, gateway auth + routing proven). A 200 cost-bearing call needs a provider key, which was not reachable this session (credential-gated). Typecheck (tsc --noEmit) passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Warning Review limit reached
More reviews will be available in 28 minutes and 7 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (15)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| /** True when the resolved baseUrl points at the Cloudflare AI Gateway. */ | ||
| export function isGatewayUrl(baseUrl: string): boolean { | ||
| return baseUrl.includes(GATEWAY_HOST); |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 60c7ce588a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -1,12 +1,14 @@ | |||
| import type { ChittyPlugin, CommandDefinition, RemoteTypeDefinition } from "@/lib/plugin"; | |||
| import type { Config } from "@/lib/config"; | |||
| import { CHITTYCLAW_PROVIDER_BASE_URLS, gatewayAuthHeaders } from "./gateway"; | |||
There was a problem hiding this comment.
This repo publishes native ESM (package.json has type: "module" and tsconfig uses module: "ESNext"), so TypeScript preserves this new runtime import as import ... from "./gateway" in dist. Node ESM does not append .js for relative specifiers, so loading dist/plugins/ai/index.js will fail with ERR_MODULE_NOT_FOUND before any OpenAI/Anthropic command can run; the neighboring plugin imports already use .js. Please import ./gateway.js here and in the analogous Anthropic import.
Useful? React with 👍 / 👎.
| export function isGatewayUrl(baseUrl: string): boolean { | ||
| return baseUrl.includes(GATEWAY_HOST); | ||
| } |
There was a problem hiding this comment.
When a caller configures a custom baseUrl that merely contains this hostname string, such as https://gateway.ai.cloudflare.com.evil.example, this check treats it as the Cloudflare gateway and gatewayAuthHeaders() sends the cf-aig-authorization bearer token to that non-gateway origin. Since baseUrl is explicitly user-configurable for proxies/direct endpoints, parse the URL and compare the hostname exactly before attaching the gateway credential.
Useful? React with 👍 / 👎.
|
|
||
| // Custom agent to bypass SSL SNI mismatch for IP-based gateway routing | ||
| const httpsAgent = new https.Agent({ | ||
| rejectUnauthorized: false |
| rejectUnauthorized: false | ||
| }); | ||
|
|
||
| process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0"; |
|
|
||
| let response; | ||
| try { | ||
| response = await fetch(`${baseUrl}/v1/chat/completions`, { |
| headers: { | ||
| "Content-Type": "application/json", | ||
| "x-chitty-client": "chittycan", | ||
| "Authorization": remote.apiKey ? `Bearer ${remote.apiKey}` : "" | ||
| }, |
| body: JSON.stringify({ | ||
| model: remote.defaultModel || "claude-3-5-sonnet-20241022", | ||
| messages: [ | ||
| { role: "system", content: systemPrompt }, | ||
| { role: "user", content: userPrompt } | ||
| ], | ||
| temperature: 0.1, | ||
| max_tokens: 200, | ||
| }), |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5bbab313b3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // Remember this fix as the new correct mapping | ||
| recordExecution(detectedCLI, naturalLanguage, suggestedFix, true); | ||
| } catch (healError: any) { | ||
| console.log(chalk.red("Healing failed: " + healError.message)); |
There was a problem hiding this comment.
When the original command fails and an AI remote is configured, this catch logs a failed healing attempt but then falls through, so chittyCommand never throws and the CLI exits successfully even though both the original command and the fix failed; the same false-success path happens if the user declines runFix. Please rethrow the original error or otherwise set a non-zero exit unless the suggested fix actually succeeds.
Useful? React with 👍 / 👎.
| rejectUnauthorized: false | ||
| }); | ||
|
|
||
| process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0"; |
There was a problem hiding this comment.
When a chittyclaw remote is used, this assignment disables certificate verification globally for the rest of the Node process, so any later HTTPS request made by this CLI can accept an invalid certificate for unrelated hosts. If the IP gateway needs special TLS handling, keep it scoped to that request with a proper dispatcher/agent/server name instead of changing NODE_TLS_REJECT_UNAUTHORIZED process-wide.
Useful? React with 👍 / 👎.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
@claude resolve conflicts |
2 participants
Why
ChittyComptroller captures provider LLM spend by polling the chittyclaw Cloudflare AI Gateway logs (→
chittyops.cost_ledger). The chittycan AI plugins defaulted to DIRECT provider URLs (api.anthropic.com/api.openai.com), so every Anthropic/OpenAI call bypassed the gateway and was invisible to the Comptroller. Onlyworkers-aispend was captured.What changed
src/plugins/ai/gateway.ts(new) — shared helper: per-provider gateway base URLs;gatewayAuthHeaders()injectscf-aig-authorization: Bearer <token>only when the resolved baseUrl is the CF gateway (a non-gateway override never leaks the token);resolveCfAigToken()resolvesoptions.cfAigToken → env CF_AIG_TOKEN → env CF_ISSUED_AIGATEWAY_TOKEN(never hardcoded).src/plugins/ai/anthropic.ts— default baseUrl →…/chittyclaw/anthropic/v1; patched bothrequest()andmessageStream();cfAigTokenadded to remote type + config fields + all handlers.src/plugins/ai/openai.ts— default baseUrl →…/chittyclaw/openai; patchedrequest()andchatStream(); samecfAigTokenwiring.Provider key stays in its native header (
x-api-key/Authorization), forwarded upstream by the gateway. OptionalbaseUrloverride preserved.Design — (a) per-provider gateway URLs (not OpenAI-compat)
Each client already appends its provider-native path, so swapping host+prefix drops in cleanly; compat would force rewriting every request body into OpenAI chat format. Suffix is NOT uniform (per CF docs): anthropic keeps
/v1(…/anthropic/v1+/messages), openai drops it (…/openai+/chat/completions).Scope
Anthropic + OpenAI (the providers with existing chittycan plugins). Google has no chittycan plugin today → follow-up (net-new plugin, not shipped unvalidated per no-mocks). groq/cohere/together/replicate/huggingface/ollama out of scope.
Verification
Typecheck
tsc --noEmitpasses. Live call routed toprovider: anthropic, captured in chittyclaw logs as the first non-workers-aientry:The
401is from Anthropic (Anthropicrequest_id+ "x-api-key header is required"), not the gateway — proving end-to-end: URL resolves → gateway acceptedcf-aig-authorization→ forwarded to Anthropic. Lands inchittyops.cost_ledgeron the Comptroller's next poll.Credential gap: a
200cost-bearing call needs a provider API key; none reachable this session (AIG token resolved via 1Password, but no provider key in env / probeable op ref in Connect mode, no BYOK default key, and user is never asked for creds). 401-from-provider is strictly stronger than a bare URL probe.🤖 Generated with Claude Code