feat(mcp): optional built-in MCP server — list/get/search docs over Streamable HTTP

.github/workflows/ci.yml

@@ -36,3 +36,28 @@ jobs:
 
       - name: Run the suite + lint
         run: bundle exec rake
+
+  # The optional-dependency gate: docs-kit's MCP server is built on the `mcp`
+  # gem, which is NOT a runtime dependency (a consuming site adds it itself). This
+  # leg installs WITHOUT the `mcp` group and runs the suite — proving the feature
+  # no-ops cleanly (the MCP specs self-skip) when the gem is absent, so a site
+  # that never bundles `mcp` is byte-identical to before this feature.
+  without-mcp:
+    name: gate (no mcp gem)
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Ruby
+        uses: ruby/setup-ruby@v1
+        with:
+          ruby-version: "3.4"
+          bundler-cache: true
+
+      - name: Install without the optional mcp gem
+        run: |
+          bundle config set --local without mcp
+          bundle install
+
+      - name: Run the suite (must stay green with mcp absent)
+        run: bundle exec rspec

Gemfile

@@ -10,6 +10,16 @@ gemspec
 # `daisyui >= 1.2` dependency.
 gem "daisyui", path: "../daisyui" if File.directory?(File.expand_path("../daisyui", __dir__))
 
+# The official MCP Ruby SDK. docs-kit's MCP server (DocsKit::McpServer /
+# DocsKit::McpController) is an OPTIONAL, runtime-detected feature — the gem is
+# NOT a runtime dependency (a consuming site adds it itself). It lives in its own
+# group so the optional-dependency GATE can be exercised by excluding it:
+# `bundle config set --local without mcp && bundle install` (a CI leg does exactly
+# this). Without the gem the MCP specs self-skip and the feature must no-op.
+group :mcp do
+  gem "mcp"
+end
+
 group :development, :test do
   gem "rake"
   gem "rspec"

README.md

@@ -409,6 +409,60 @@ get "/llms-full.txt" => "docs_kit/llms#full", as: :llms_full
 **Existing sites:** re-run `bin/rails g docs_kit:install` (it adds the two routes
 idempotently), or paste the two lines above into `config/routes.rb`.
 
+## Add your docs to an agent (MCP)
+
+`llms.txt` covers fetch-style consumption; **MCP** (the Model Context Protocol) is
+the native one — a reader adds one URL and your docs become first-class agent
+tools instead of scraped text. docs-kit ships a **read-only, stateless** MCP
+server that any site can turn on with one gem + one route. It exposes three tools
+over the SAME registry the docs render from (so an agent queries live docs, never
+a stale copy):
+
+| Tool | Returns |
+|------|---------|
+| `list_pages` | every authored page — `slug`, `title`, `group`, `url` |
+| `get_page(slug:)` | one page's Markdown twin (the same `.md` twin `/llms.txt` links) |
+| `search_docs(query:)` | ranked hits — `page_title`, `section_title`, `url`, `snippet` |
+
+The `mcp` gem is **optional** — docs-kit depends on it in no gemspec list, and the
+endpoint stays off (byte-identical to before) unless you opt in. Two steps:
+
+```ruby
+# Gemfile
+gem "mcp"
+```
+
+```ruby
+# config/routes.rb — the install generator scaffolds these COMMENTED; uncomment.
+post  "/mcp" => "docs_kit/mcp#create", as: :mcp
+match "/mcp" => "docs_kit/mcp#method_not_allowed", via: %i[get delete]
+```
+
+Then a reader connects — for Claude Code:
+
+```bash
+claude mcp add --transport http docs https://your-docs.example/mcp
+```
+
+and can ask Claude to search or read your docs, which now appear as tools. The
+JSON-RPC is stateless (each `POST` is independent — no SSE session), so it works
+behind the existing Kamal/Cloudflare deploy unchanged; `GET`/`DELETE` return
+`405`. When enabled, `/llms.txt` grows a final `## MCP` line advertising the
+endpoint so agents discover it.
+
+`c.mcp` defaults to `true`, so once the gem + route are present the endpoint is
+live. Set it `false` to keep it off even on a site that bundles the gem:
+
+```ruby
+DocsKit.configure { |c| c.mcp = false }
+```
+
+The endpoint is read-only over already-public content — writing docs is still
+git, and private-docs auth is a host concern (the route is yours to wrap). Rate
+limiting is the host's responsibility too (e.g. `rate_limit` in your base
+controller). The server ships in the gem (`DocsKit::McpServer` /
+`DocsKit::McpController`); the **route lives in your app**, like `llms.txt`.
+
 ## API docs — one request, every client tab
 
 An endpoint example is a request shown in several clients (curl, JavaScript,

app/controllers/docs_kit/mcp_controller.rb

@@ -0,0 +1,60 @@
+# frozen_string_literal: true
+
+module DocsKit
+  # The built-in read-only MCP endpoint — one gem controller, host-drawn route
+  # (same shape as DocsKit::LlmsController/SearchController; the engine adds no
+  # routes):
+  #
+  #   # config/routes.rb
+  #   post  "/mcp" => "docs_kit/mcp#create"
+  #   match "/mcp" => "docs_kit/mcp#method_not_allowed", via: %i[get delete]
+  #
+  # A user adds `https://docs.example.com/mcp` to Claude Code / Claude.ai / Cursor
+  # once and the docs become first-class agent tools (list_pages / get_page /
+  # search_docs) over the SAME registry the site renders from. See
+  # DocsKit::McpServer / DocsKit::McpTools.
+  #
+  # Stateless JSON-RPC: each POST is independent (no SSE session), so it works
+  # behind the existing Kamal/Cloudflare deploy unchanged. #create delegates the
+  # whole protocol to DocsKit::McpServer#handle_json — the SDK parses the request,
+  # dispatches the tool, and serializes the response (including JSON-RPC errors),
+  # so the controller never hand-rolls the protocol.
+  #
+  # OFF unless BOTH the optional `mcp` gem is present AND the site left c.mcp on
+  # (DocsKit.configuration#mcp_enabled?). A site without the gem, or with
+  # c.mcp = false, gets a 404 here and is byte-identical to before this feature.
+  class McpController < ActionController::Base
+    # A JSON-RPC POST carries no CSRF token to verify (there's no form, no
+    # session — an agent posts a raw JSON body). Unlike the GET-only text
+    # endpoints (which use :null_session so csrf_meta_tags resolves in a rendered
+    # <head>), this action renders JSON only and never a Shell, so drop forgery
+    # protection outright.
+    skip_forgery_protection
+
+    def create
+      return head(:not_found) unless docs_config.mcp_enabled?
+
+      server = DocsKit::McpServer.build(docs_config, base_url: request.base_url, view_context:)
+      return head(:not_found) unless server
+
+      # #handle_json returns an already-serialized JSON string, so render it as the
+      # raw body with the JSON content type — `render json:` would re-encode the
+      # string (wrapping it in quotes), corrupting the JSON-RPC envelope.
+      render body: server.handle_json(request.body.read), content_type: "application/json"
+    end
+
+    # Read-only + stateless: the endpoint speaks JSON-RPC over POST only. There is
+    # no standalone SSE stream (GET) and no session to terminate (DELETE), so both
+    # are 405 rather than the SDK's session machinery.
+    def method_not_allowed
+      head :method_not_allowed
+    end
+
+    private
+
+    # NOT named #config — ActionController::Base#config is the Rails config object
+    # and RequestForgeryProtection delegates to it; shadowing it breaks forgery
+    # handling (see LlmsController). The DocsKit config reader is #docs_config.
+    def docs_config = DocsKit.configuration
+  end
+end

docs/Gemfile

@@ -7,6 +7,11 @@ gem "phlex-rails"
 gem "rails_icons", "~> 1.1"
 gem "rouge"
 
+# Dogfood docs-kit's OPTIONAL MCP endpoint — the /mcp route below serves these
+# docs to AI agents (search_docs / get_page / list_pages). Optional for a
+# consuming site; enabled here so the gem's own docs prove the feature.
+gem "mcp"
+
 # Bundle edge Rails instead: gem "rails", github: "rails/rails", branch: "main"
 gem "rails", "~> 8.1.3"
 # The modern asset pipeline for Rails [https://github.com/rails/propshaft]

docs/config/routes.rb

@@ -12,6 +12,12 @@
   # concatenates every page's Markdown twin.
   get "/llms.txt" => "docs_kit/llms#index", as: :llms
   get "/llms-full.txt" => "docs_kit/llms#full", as: :llms_full
+
+  # Read-only MCP endpoint (DocsKit::McpController) — dogfooding docs-kit's own
+  # optional MCP server. POST speaks JSON-RPC (search_docs / get_page /
+  # list_pages); GET/DELETE are 405 (read-only, stateless — no SSE session).
+  post "/mcp" => "docs_kit/mcp#create", as: :mcp
+  match "/mcp" => "docs_kit/mcp#method_not_allowed", via: %i[get delete]
   # Define your application routes per the DSL in https://guides.rubyonrails.org/routing.html
 
   # Reveal health status on /up that returns 200 if the app boots with no exceptions, otherwise 500.

lib/docs_kit/configuration.rb

@@ -128,6 +128,14 @@ class Configuration
     # works). See DocsKit::MarkdownExport / DocsKit::Controller#render_page.
     attr_accessor :page_markdown_action
 
+    # Whether the built-in read-only MCP endpoint (DocsKit::McpController, a
+    # POST /mcp JSON-RPC server exposing list_pages / get_page / search_docs over
+    # the same registry the docs render from) is active. Defaults to true, but the
+    # endpoint only turns on when the optional `mcp` gem is ALSO present and the
+    # host draws the route — #mcp_enabled? gates on both. Set false to keep the
+    # endpoint off even on a site that bundles the gem. See DocsKit::McpServer.
+    attr_accessor :mcp
+
     # Whether the topbar renders the docs-search form (and the docs-nav palette
     # markup). Defaults to true. Set false to hide search site-wide — the route
     # can stay drawn, but no affordance points at it. Gated together with a
@@ -219,6 +227,7 @@ def initialize
       @code_lexer_fallback = "plaintext"
       @code_language_labels = {}
       @page_markdown_action = true
+      @mcp = true
       @search = true
       @search_path = "/docs/search"
       @search_shortcuts = DEFAULT_SEARCH_SHORTCUTS
@@ -270,6 +279,24 @@ def normalize_on_page(value)
 
     private
 
+    # True when the optional `mcp` gem can be loaded. Memoized across both
+    # outcomes so a site without the gem doesn't pay a failed require per request.
+    # We attempt the require lazily (rather than only checking defined?(MCP)) so a
+    # bundled-but-not-yet-required gem still counts as present — the same
+    # degrade-gracefully-on-a-missing-optional-gem posture as DocsUI::Icon's
+    # rails_icons guard.
+    def mcp_gem_present?
+      return @mcp_gem_present if defined?(@mcp_gem_present)
+
+      @mcp_gem_present =
+        begin
+          require "mcp"
+          defined?(::MCP::Server) ? true : false
+        rescue LoadError
+          false
+        end
+    end
+
     # { heading => registry.nav_items }, dropping headings with no authored
     # pages so the sidebar never shows an empty group.
     def nav_groups_from_registries
@@ -299,6 +326,15 @@ def title_suffix
       @title_suffix || @brand
     end
 
+    # Whether the built-in MCP endpoint is active: the #mcp toggle is on AND the
+    # optional `mcp` gem is loadable — the same "toggle AND capability" shape as
+    # #search_enabled?. Read by DocsKit::LlmsText (to advertise /mcp in llms.txt)
+    # and DocsKit::McpController (to 404 when off), so a site without the gem, or
+    # one that set c.mcp = false, is byte-identical to before this feature.
+    def mcp_enabled?
+      !!@mcp && mcp_gem_present?
+    end
+
     # Whether the Shell renders the search affordance: search is on AND a path is
     # set to submit to. A site with @search_path blanked (or nil) gets no form
     # even if @search is true — there'd be nothing to submit to.

lib/docs_kit/llms_text.rb

@@ -36,15 +36,24 @@ def index(config, base_url: nil)
       blocks = ["# #{config.brand}"]
       tagline = config.tagline
       blocks << "> #{tagline}" if tagline && !tagline.to_s.empty?
+      blocks.concat(section_blocks(config, base_url))
 
-      groups(config).each do |group, links|
-        section = ["## #{group}", *links.map { |link| link_line(link, base_url) }]
-        blocks << section.join("\n")
-      end
+      # Advertise the built-in MCP endpoint last, so an agent that reads llms.txt
+      # discovers it can also connect over the protocol (native tools vs fetching
+      # text). Only when the endpoint is actually live (gem present + c.mcp on).
+      blocks << mcp_block(base_url) if config.mcp_enabled?
 
       blocks.join("\n\n")
     end
 
+    # One `## {group}` block per nav group, each a tight bullet list of its
+    # authored pages' `.md` links, in registry order.
+    def section_blocks(config, base_url)
+      groups(config).map do |group, links|
+        ["## #{group}", *links.map { |link| link_line(link, base_url) }].join("\n")
+      end
+    end
+
     # The authored pages across every registry, in config/registry order — each
     # responds to #title / #href / #view_class. The controller renders these to
     # Markdown for .full.
@@ -67,6 +76,17 @@ def groups(config)
       end
     end
 
+    # The `## MCP` section pointing an agent at the read-only MCP endpoint. The
+    # `/mcp` URL is absolutized against base_url when available (agents connect to
+    # a portable URL); relative otherwise.
+    def mcp_block(base_url)
+      url = base_url ? "#{base_url.chomp('/')}/mcp" : "/mcp"
+      "## MCP\n" \
+        "This documentation is also available over the Model Context Protocol " \
+        "(search, page retrieval) at #{url} — add it to an MCP client " \
+        "(Claude Code, Claude.ai, Cursor) to query these docs as tools."
+    end
+
     # `- [label](absolute .md url)`. The `.md` suffix targets the page's Markdown
     # twin (DocsKit::Controller#render_page).
     def link_line(link, base_url)