Skip to content

feat: complete Ai interface compatibility in jack-ai proxy wrapper #12

@hellno

Description

@hellno

Context

Our createJackAI() wrapper (jack-ai.ts) currently only implements .run() from the full Cloudflare Ai interface. This works for all current templates since workers-ai-provider only calls .run(), but users calling env.AI methods directly will hit missing functionality.

Current State

// What we return
{ run: (model, inputs, options?) => Promise<T | ReadableStream> }

// Full Ai interface (from @cloudflare/workers-types)
abstract class Ai {
  aiGatewayLogId: string | null;
  gateway(gatewayId: string): AiGateway;
  autorag(autoragId: string): AutoRAG;
  run(model, inputs, options?): Promise<...>;
  models(params?): Promise<AiModelsSearchObject[]>;
  toMarkdown(): ToMarkdownService;
  toMarkdown(files, options?): Promise<ConversionResponse[]>;
}

Missing Methods

Method Priority Notes
gateway(id) P1 AI Gateway for caching, rate limiting, analytics. Power users need this for production
toMarkdown() P2 Document-to-markdown conversion (PDFs, images). Useful for document processing templates
models(params?) P3 List available models. Nice-to-have for model pickers in agent UIs
autorag(id) P3 AutoRAG binding. Newer API, lower demand
aiGatewayLogId P3 Only relevant once gateway() is implemented

Implementation Approach

Each method needs a corresponding route in the binding-proxy worker (apps/binding-proxy-worker/):

  • /ai/gateway → forward to env.AI.gateway(id) with metering
  • /ai/models → forward to env.AI.models(params) with metering
  • /ai/toMarkdown → forward to env.AI.toMarkdown(files, options) with metering

The jack-ai.ts wrapper would add methods that call these proxy routes, same pattern as .run().

Why Not Just Use env.AI Directly?

Jack cloud is multi-tenant. The proxy provides:

  • Per-project quota enforcement (1000 req/day default)
  • Usage metering via Analytics Engine
  • Rate limiting (100 req/10s burst protection)

Direct env.AI is not available in jack cloud deployments — only the __AI_PROXY service binding is injected.

Acceptance Criteria

  • createJackAI() return type passes satisfies Ai (or close to it)
  • Each proxied method has metering + quota enforcement
  • Existing .run() behavior unchanged
  • Templates can use gateway() for caching in production

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions