Support inbound file/image transfer from Discord and Slack to agents

[edspencer]

Summary

Users should be able to send images (and other files) to agents via Discord or Slack, just as they can paste images into Claude Code's CLI with Ctrl+V. Currently, both platforms silently drop all attachments — only the text content of messages reaches the agent.

Goal

When a user uploads an image in a Discord or Slack message to an agent, the image should be forwarded to the agent as a multimodal content block, equivalent to pasting an image directly into Claude Code.

Non-image files (PDFs, code files, etc.) should be saved to the agent's working directory and referenced in the text prompt, since the Claude API only accepts images as visual content blocks.

How Claude Code Handles Images

The Claude Agent SDK's query() function accepts either a plain string or an AsyncIterable<SDKUserMessage>:

function query(params: {
  prompt: string | AsyncIterable<SDKUserMessage>;
  options?: Options;
}): Query;

To pass images, the prompt must be an AsyncIterable<SDKUserMessage> where the message contains multimodal content blocks:

async function* createMultimodalPrompt(
  text: string,
  imageBase64: string,
  mediaType: string,
  sessionId: string
) {
  yield {
    type: 'user' as const,
    message: {
      role: 'user' as const,
      content: [
        {
          type: 'image',
          source: {
            type: 'base64',
            media_type: mediaType,
            data: imageBase64,
          },
        },
        {
          type: 'text',
          text: text,
        },
      ],
    },
    parent_tool_use_id: null,
    session_id: sessionId,
  };
}

const result = query({
  prompt: createMultimodalPrompt(text, base64Data, 'image/png', sessionId),
  options: sdkOptions,
});

Supported image formats

The Claude API accepts exactly four image MIME types:

• image/jpeg
• image/png
• image/gif
• image/webp

Limits: 5MB per image, 8000x8000px max resolution, up to 100 images per request.

What Needs to Change

1. Extract attachments from incoming messages

Discord (mention-handler.ts): Read message.attachments collection. Each attachment provides:

• url — publicly accessible CDN URL
• filename, contentType, size, width, height

Slack (slack-connector.ts): Read event.files array. Each file provides:

• url_private — requires Authorization: Bearer <bot-token> to download
• name, mimetype, size, filetype

2. Download and classify attachments

• Download the file content (Discord: plain HTTP GET; Slack: authenticated GET with bot token)
• Classify by MIME type:
  • Image (image/jpeg, image/png, image/gif, image/webp): Base64-encode for inclusion as an image content block
  • Other files: Save to the agent's working directory and mention the file path in the text prompt

3. Widen the prompt type through the pipeline

The entire execution pipeline is currently prompt: string:

Chat Manager → fleetManager.trigger(prompt: string) → JobExecutor → Runtime → SDK query()

This needs to support multimodal content:

Chat Manager → fleetManager.trigger(prompt: PromptContent) → JobExecutor → Runtime → SDK query()

Where PromptContent is something like:

type PromptContentBlock =
  | { type: "text"; text: string }
  | { type: "image"; mediaType: string; data: string };  // base64

type PromptContent = string | PromptContentBlock[];

Files that need the type change:

• FleetManager.trigger() — accept PromptContent instead of string
• RunnerOptions.prompt (packages/core/src/runner/types.ts)
• RuntimeExecuteOptions.prompt (packages/core/src/runner/runtime/interface.ts)
• SDKRuntime.execute() — construct AsyncIterable<SDKUserMessage> with content blocks when prompt contains images; pass plain string when text-only
• ContainerRuntime — equivalent support for Docker agents (pass content blocks via the SDK inside the container, or write images to mounted volume)
• Job output logging — decide how to represent image content in stored job output (e.g. store as [image: filename.png] placeholder)

4. Docker runtime considerations

For Docker-containerized agents, images need to reach the agent inside the container. Options:

• Base64 in prompt — works but increases payload size; the SDK inside the container handles multimodal natively
• Mounted volume — write images to a shared volume, reference by path in the prompt
• The right approach depends on image size limits and how the container runtime invokes the SDK

Configurability

File transfer should be configurable per-agent per-platform in herd.yaml:

agents:
  my-agent:
    chat:
      discord:
        channels:
          - id: "123456789"
        files:
          send: true      # Agent can send files to Discord (default: true)
          receive: true   # Agent receives files from Discord (default: true)
      slack:
        channels:
          - id: C0123456789
            mode: auto
        files:
          send: true      # Agent can send files to Slack (default: true)
          receive: true   # Agent receives files from Slack (default: true)

When files.receive is false, attachments are silently ignored (current behavior). When files.send is false, the herdctl_send_file MCP tool is not injected.

Scope

This is a medium-to-large change. The main complexity is widening prompt: string to support content blocks across the execution pipeline, and correctly constructing the AsyncIterable<SDKUserMessage> that the SDK expects. The per-platform attachment extraction and download logic is straightforward.

Related

• Add file sending support for Discord agents (agent -> Discord) #55 — Outbound file sending for Discord (agent → Discord)
• MCP HTTP bridge should use per-request bearer token authentication #54 — MCP HTTP bridge authentication