Run Claude Code directly on a Databricks cluster. No laptop. No external Anthropic key. All inference inside your workspace.
Overview · Quick Start · Routing · Safety · How It Works · Examples · Troubleshooting
⚠️ Use at your own risk. This notebook runs Claude Code with--dangerously-skip-permissionsso the agent can complete multi-step tasks without prompting. The Claude Code permission UI is bypassed. The only safety layer is the destructive-action guardrails embedded in the notebook'sCLAUDE.mdcell. Read that cell before running any prompt. Review every command Claude proposes. You are responsible for the actions Claude takes against your workspace.
- Overview
- Why This Exists
- Quick Start
- Routing Modes
- Safety
- How It Works
- Included Skills
- Examples
- Requirements
- Troubleshooting
- License
ClaudeBook is a single Databricks notebook that lets you run Claude Code directly inside your cluster. You point it at a workspace, give it a prompt, and it does the rest using only your workspace's own Anthropic-compatible serving endpoints.
The notebook handles:
- Installing Node.js and the Claude Code npm package into a per-run temp directory (no sudo, no global install).
- Detecting which Anthropic-compatible route is available on your workspace: the AI Gateway (preferred, May 2026 rollout) or the classic Foundation Model API with a compatibility patch (fallback).
- Writing a
CLAUDE.mdthat gives Claude full Databricks context — CLI patterns, REST API examples, app structure, and hard safety rules for destructive operations. - Writing 12 Databricks skills (apps, lakebase, lakeview, lineage, query, warehouse-selector, data-generation, demo, lakeview-analyzer, resource-deployment, workspace-files, authentication) into a workspace directory Claude can read.
- Running your prompt non-interactively and streaming Claude's tool calls, reasoning, and final answer back into the notebook output.
All Anthropic API traffic stays inside your Databricks workspace.
Most Databricks customers want the productivity of Claude Code (a coding-aware agent that can read files, run shell commands, edit code, and chain multi-step work) but cannot run it from a laptop because:
- They cannot install Node.js or npm packages on corporate laptops.
- They cannot send code, prompts, or telemetry to a third-party API.
- Their data and credentials never leave the workspace boundary.
ClaudeBook solves this by running Claude Code inside the cluster, with the upstream LLM call going through your workspace's own AI Gateway or FMAPI endpoint. The agent uses Databricks-issued tokens, runs in a workspace-scoped temp directory, and operates against your own resources — no laptop dependency, no external API key.
- Clone or download this repo. Open
agent/claudebook.ipynbin your Databricks workspace. - Attach the notebook to any DBR 13.0+ cluster (shared, single-node, or serverless).
- Run cells 1–4 in order. The Setup cell auto-detects the routing mode and prints which one it picked.
- Edit cell 5 with your prompt and run. Claude will stream its work back into the notebook.
prompt = "List all clusters in this workspace and show their state."
claude_code(prompt)Subsequent prompt cells continue the same conversation (Claude has full memory of prior cells in the run).
The notebook auto-detects which route is enabled on your workspace:
| Mode | When | How |
|---|---|---|
| AI Gateway (preferred) | Workspace has the AI Gateway Anthropic endpoint enabled — for example, after the Genie Code for AI Gateway rollout (May 2026). | Native /ai-gateway/anthropic route. Adds x-databricks-use-coding-agent-mode: true header and sets CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1. No patching of Claude Code. |
| FMAPI compatibility (fallback) | Workspace has classic Foundation Model APIs but no AI Gateway Anthropic route. | /serving-endpoints/anthropic with a Node.js compatibility patch loaded via NODE_OPTIONS=--require. The patch strips anthropic-beta header and context_management / metadata / service_tier body fields before each upstream call. |
The setup cell prints which mode it chose. If your workspace gets the AI Gateway enabled later, just re-run the Setup cell — it will switch automatically.
Because the notebook runs claude --dangerously-skip-permissions, the Claude Code permission UI is bypassed. The only safety layer is the guardrail policy embedded in CLAUDE.md (cell 2). Claude reads it on every run and applies it to its own behaviour.
The guardrails currently forbid the following without an explicit yes destroy <target> confirmation from the user:
- Data destruction:
DROP TABLE/VIEW/CATALOG/SCHEMA/DATABASE/FUNCTION/VOLUME/MODEL,DELETE FROM,TRUNCATE,CREATE OR REPLACE TABLEon existing tables,rm -rf/ recursivedbutils.fs.rm. - Compute destruction:
databricks {clusters|warehouses|jobs|pipelines|apps|serving-endpoints} delete. - Governance / identity destruction: delete groups, users, service principals, secret scopes, revoke UC privileges.
- Storage destruction: delete storage credentials, external locations, recursive workspace deletes.
The full policy lives in cell 2. Edit it to match your organisation's risk posture before running prompts. Every destructive action Claude does take (after approval) is appended to /tmp/claudebook_audit.log with timestamp, user, target, and command.
This is a model-level guardrail. It depends on Claude following the instructions in CLAUDE.md. It is not a hardened permission system. Use only on workspaces where you accept that risk model.
┌──────────────────────────────────────────────────────────────┐
│ Databricks Cluster (DBR 13+) │
│ │
│ ┌────────────────────────────────────────────────────────┐ │
│ │ Notebook Python (driver) │ │
│ │ ┌──────────────────────────────────────────────────┐ │ │
│ │ │ claude_code(prompt) │ │ │
│ │ │ ↓ subprocess │ │ │
│ │ │ npx claude (Node.js in /tmp) │ │ │
│ │ │ ↓ Anthropic API HTTP call │ │ │
│ │ │ AI Gateway OR FMAPI + JS patch │ │ │
│ │ └──────────────────────────────────────────────────┘ │ │
│ └────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ Workspace /serving-endpoints/anthropic (or /ai-gateway/…) │
│ │ │
└────────────────────────────┼─────────────────────────────────┘
│
▼
Foundation Model serving:
databricks-claude-{opus|sonnet|haiku}-4-*
Components:
- Setup cell installs Node.js v20 into
/tmp/claude_code_*andnpm install -g @anthropic-ai/claude-codeinto the same temp prefix. - Routing detection probes
/ai-gateway/anthropic/v1/messages. If the route exists, AI Gateway mode is selected. Otherwise the FMAPI fallback is used. - CLAUDE.md is rendered with workspace context, user identity, the selected routing mode, and the safety guardrails. It is written to a workspace directory.
- Skills are written as a single
DATABRICKS_SKILLS.mdreference in the workspace, with model placeholders replaced by the actual latest Claude models available on the workspace. - Tools are written as standalone Python files (lakeview_builder, lakebase_client, query formatter, lineage scripts, etc.) to a
tools/subdirectory. - claude_code() spawns
claude --print --verbose --output-format stream-jsonwith the assembled env vars. It parses the streamed JSON and renders tool calls, reasoning, and final answer back into the notebook output.
| Skill | Capability |
|---|---|
| databricks-apps | Build & deploy full-stack Databricks Apps with Lakebase + Foundation Model APIs |
| databricks-authentication | CLI authentication and profile management |
| databricks-data-generation | Generate realistic synthetic data across 8 verticals |
| databricks-demo | Create, deploy, and run end-to-end Databricks demos |
| databricks-lakebase | Create, configure, and query Lakebase Postgres databases |
| databricks-lakeview-dashboard | Programmatically create and manage AI/BI Lakeview dashboards |
| databricks-lakeview-dashboard-analyzer | Analyze existing Lakeview dashboards |
| databricks-lineage | Explore Unity Catalog data lineage |
| databricks-query | Execute SQL queries on warehouses with formatted output |
| databricks-resource-deployment | Deploy notebooks, jobs, clusters, warehouses, apps |
| databricks-warehouse-selector | Discover and select SQL warehouses |
| databricks-workspace-files | Browse, pull, and manage workspace files and notebooks |
List clusters:
prompt = "List all clusters in this workspace and show their state."
claude_code(prompt)Build a Databricks App:
prompt = """
Build a Databricks App with a Streamlit frontend that shows the top 10 tables by
DBU spend last month. Use a Lakeview dashboard for the chart. Deploy it.
"""
claude_code(prompt)Continue a previous task:
prompt = "Where is the app? Continue from where you left off."
claude_code(prompt)- A Databricks workspace with either:
- AI Gateway with the Anthropic route enabled (preferred), or
- Foundation Model APIs with at least one
databricks-claude-*endpoint inREADYstate.
- A cluster running DBR 13.0+ (shared, single-node, or serverless all work).
- Outbound internet access from the cluster (the Setup cell downloads Node.js from
nodejs.organd Claude Code from npm).
No sudo, no laptop installation, no external Anthropic API key.
"Routing: FMAPI compatibility (JS patch — fallback)" — Your workspace does not have the AI Gateway Anthropic route enabled. Either ask your account admin to enable it, or continue using the fallback (works fine, just less clean).
"No claude- endpoints found"* — Your workspace has neither AI Gateway nor any Foundation Model claude-* endpoint. Open Account Console → AI Gateway, or Workspace → Serving → New endpoint with a claude-* model.
Cluster has no internet — The setup cell downloads Node.js and the Claude Code npm package. If your workspace blocks outbound traffic, pre-install Node.js into a workspace-scoped path and edit the Setup cell to skip the download.
Token expired mid-run — claude_code() refreshes the notebook token from dbutils on every call. If you see 401 errors, just re-run the prompt cell.
Claude attempts something destructive — The guardrails in CLAUDE.md should catch this, but if Claude proposes a destructive command it should not have run, kill the cell. Then strengthen the relevant rule in CLAUDE.md.
Apache 2.0. See LICENSE.