TissueAgent is a role-based multi-agent framework that turns open-ended natural-language ST requests and multimodal inputs (data, PDFs, images) into auditable, runnable workflows. A single evolving plan coordinates specialized agents and records rationales, step status, and artifact links, enabling transparent provenance and targeted replanning. By separating planning, recruitment, execution, evaluation, and reporting, TISSUEAGENT is designed to improve reliability across heterogeneous, multi-stage analyses.
- Role-based multi-agent design with explicit separation of planning, recruitment, execution, evaluation, and reporting.
- A single evolving plan that tracks rationales, step status, and artifact links to support transparent provenance and targeted replanning.
- Built-in collaboration with external specialized agents to extend capabilities for domain-specific tasks.
- Support for diverse spatial transcriptomics workflows such as figure reproduction, cell type annotation, cell-cell communication, differential gene expression, and cell type deconvolution.
TissueAgent runs in one of two modes, controlled by a toggle in the web UI's sidebar:
- Autopilot (default) — the planner, recruiter, manager, evaluator, and reporter run end-to-end without pausing. This is the only mode available outside the web UI (notebook and CLI entry points always run autopilot).
- Copilot — the run pauses after the planner finishes drafting the plan, and again after the recruiter assigns agents. At each pause the plan panel surfaces four actions:
- Approve to accept and continue,
- Edit to modify the plan markdown (or change per-step agent assignments) and resume,
- Send feedback to give free-text guidance that rewinds the run back to the planner,
- Cancel run to abort and start fresh.
Mode is persisted with the session and survives reloads. Switching modes mid-run is blocked — finish or cancel the current run first.
TissueAgent/
├── src/ # application source
│ ├── agents/
│ │ ├── planner_agent/
│ │ ├── recruiter_agent/
│ │ ├── manager_agent/
│ │ ├── evaluator_agent/
│ │ ├── reporter_agent/
│ │ ├── agent_registry/ # domain/specialised agents and tools
│ │ └── agent_tools.py # shared file-access tools (glob/grep/read/write)
│ ├── graph/ # LangGraph workflow + state orchestration
│ ├── server/ # FastAPI backend, WebSocket, REST routes
│ ├── frontend/ # React + TypeScript frontend
│ └── config.py # path constants & runtime settings
├── knowledge/ # prompt-time source material (importable as a package)
│ ├── plans/ # planner template library (.md)
│ ├── skills/ # agent skill snippets (.md)
│ └── docs/ # API docs the coding agent retrieves from
├── workspace/ # runtime data root (== DATA_DIR in code)
│ ├── library/ # persistent shared input — UI section "Library"
│ │ ├── datasets/ # curated reference datasets
│ │ └── files/ # persistent reference uploads (PDFs, notes, …)
│ ├── projects/ # one folder per project — UI section "Projects"
│ │ └── <project_id>/ # id = timestamp (e.g. 2026-06-07_19-42-10)
│ │ ├── chat.json # saved conversation (drives the project list)
│ │ ├── uploads/ # sidebar uploads for this run
│ │ ├── attachments/ # chat-attached images/PDFs (multimodal payloads)
│ │ └── outputs/ # agent's working directory (kernel cwd)
│ ├── scratch/ # pre-project draft (wiped on reset / new project)
│ │ ├── uploads/ # surfaced in UI as "Project files — Unsaved"
│ │ └── attachments/ # migrated into projects/<id>/ on first prompt
│ ├── plan_scratch/ # ephemeral plan markdown (in-flight only)
│ └── notebook/ # process-wide notebook scratch
├── demo/ # notebooks, sample inputs, expected outputs
├── docs/figures/ # README/manuscript figures
├── notebooks/ # exploratory notebooks
├── logs/ # runtime logs
├── sessions/ # LEGACY pre-refactor saves (migrated to workspace/projects/ on boot)
├── pyproject.toml # Python project configuration
├── environment.yml # Conda environment definition
└── flake.nix # Nix development environment
| What you see in the UI | On-disk path | Persistence |
|---|---|---|
| Library → Datasets | workspace/library/datasets/ |
persistent; shared across all projects |
| Library → Files | workspace/library/files/ |
persistent; shared across all projects |
| Project files → uploads/ | workspace/projects/<id>/uploads/ |
tied to the project; deleted with it |
| Project files → attachments/ | workspace/projects/<id>/attachments/ |
tied to the project; multimodal turn payloads |
| Project files → outputs/ | workspace/projects/<id>/outputs/ |
tied to the project; kernel cwd during a run |
| Project list (sidebar) | workspace/projects/*/chat.json |
one row per saved project |
| "Unsaved" draft (no project yet) | workspace/scratch/{uploads,attachments}/ |
wiped on every server boot AND on new project |
Two roots that the agent treats differently:
library/is read-only to agents. Agentsreadfrom it freely, butwrite_*tools refuse paths insidelibrary/— the library is the user's curated reference corpus, not a dumping ground for intermediates.projects/<active>/outputs/is the agent's working directory. Relative writes from thewritetool, kernel-sideos.chdir, andjupyternb_generator_toolall anchor here, so generated artifacts surface in the Files panel automatically.
-
Clone the repository with submodules and
cdinto the local directorygit clone --recurse-submodules https://github.com/ma-compbio/TissueAgent cd TissueAgentIf you already cloned without
--recurse-submodules, initialize them manually:git submodule update --init --recursive
-
Set up your LLM credentials. See LLM credentials below for the full list of supported providers and models. At minimum, one provider key is required — by default that's
OPENAI_API_KEY:export OPENAI_API_KEY="sk-..." export ANTHROPIC_API_KEY="sk-ant-..." # optional, for Claude models export OPENROUTER_API_KEY="sk-or-..." # optional, for OpenRouter
You can also paste keys directly into the web UI (sidebar → API keys); UI values override env vars and stay in server memory until cleared.
-
Install Miniconda or Anaconda if you haven't already.
-
Create the conda environment and install dependencies:
conda env create -f environment.yml conda activate tissueagent pip install -e . # installs Python deps from pyproject.toml cd src/frontend npm install # installs React/TypeScript deps cd ../..
-
Start the application (two terminals, both with
conda activate tissueagent):# Terminal 1 — FastAPI backend PYTHONPATH=$(pwd)/src uvicorn server.main:app --reload --host 0.0.0.0 --port 8000 # Terminal 2 — React dev server (hot-reload) cd src/frontend npm run dev
-
Open http://localhost:5173 in a web browser. The React dev server proxies API and WebSocket requests to the FastAPI backend automatically.
conda activate tissueagent
cd src/frontend && npm run build && cd ../..
PYTHONPATH=$(pwd)/src uvicorn server.main:app --host 0.0.0.0 --port 8000Open http://localhost:8000. FastAPI serves the built React app as static files.
You will need to install the following manually:
- Python 3.12 — python.org or your system package manager
- uv — docs.astral.sh/uv
- Node.js 22+ and npm — nodejs.org
-
Install dependencies:
uv sync # creates .venv and installs Python deps cd src/frontend npm install # installs React/TypeScript deps cd ../..
-
Start the application (two terminals):
# Terminal 1 — FastAPI backend source .venv/bin/activate PYTHONPATH=$(pwd)/src uvicorn server.main:app --reload --host 0.0.0.0 --port 8000 # Terminal 2 — React dev server (hot-reload) cd src/frontend npm run dev
-
Open http://localhost:5173 in a web browser.
source .venv/bin/activate
cd src/frontend && npm run build && cd ../..
PYTHONPATH=$(pwd)/src uvicorn server.main:app --host 0.0.0.0 --port 8000Open http://localhost:8000.
A Nix flake is provided that supplies Python 3.12, uv, Node.js 22, and npm.
-
Install Nix if you haven't already.
-
Enter the dev shell and install dependencies:
nix develop # drops you into a shell with python, uv, node, npm uv sync # creates .venv and installs Python deps cd src/frontend npm install # installs React/TypeScript deps cd ../..
-
Start the application (two terminals, both inside
nix develop):# Terminal 1 — FastAPI backend source .venv/bin/activate PYTHONPATH=$(pwd)/src uvicorn server.main:app --reload --host 0.0.0.0 --port 8000 # Terminal 2 — React dev server (hot-reload) cd src/frontend npm run dev
-
Open http://localhost:5173 in a web browser. The React dev server proxies API and WebSocket requests to the FastAPI backend automatically.
nix develop
source .venv/bin/activate
cd src/frontend && npm run build && cd ../..
PYTHONPATH=$(pwd)/src uvicorn server.main:app --host 0.0.0.0 --port 8000Open http://localhost:8000. FastAPI serves the built React app as static files.
No frontend is needed. Install Python dependencies using either conda or uv, then run TissueAgent directly from a Jupyter notebook.
Using conda:
conda env create -f environment.yml
conda activate tissueagent
pip install -e .Using uv:
uv syncSee demo/ for examples on how to invoke TissueAgent from a notebook.
Tip
All agents use GPT-5 by default. To save API tokens, models with lower reasoning capabilities can be used. This can be configured globally by modifying DefaultModelCtor in src/config.py or changed on the subagent level by modifying src/agents/agent_defns.py.
TissueAgent can be invoked in two ways:
Start the backend server:
PYTHONPATH=$(pwd)/src uvicorn server.main:app --host 0.0.0.0 --port 8000Web UI Demo:
tissueagent_video_demo_720p.mov
Notebook-based demos are available in demo/ and can be run end-to-end to reproduce manuscript tasks.
Run a demo:
-
Complete repository setup above and activate the environment
-
Export your LLM credentials (see LLM credentials for the full list):
export OPENAI_API_KEY="sk-..." export ANTHROPIC_API_KEY="sk-ant-..." # optional, for Claude models export OPENROUTER_API_KEY="sk-or-..." # optional, for OpenRouter
-
Launch Jupyter:
jupyter notebook
-
Open and run a notebook from top to bottom.
Available notebooks:
demo/figure_recreation_lohoff-2b.ipynb: figure reproduction workflow demo 1demo/figure_recreation_lohoff-2e.ipynb: figure reproduction workflow demo 2demo/spot_deconvolution_visium_heart.ipynb: cell-type deconvolution task
Outputs are written to workspace/ and copied into demo/outputs/{TASK}. Execution transcripts are saved to demo/outputs/{TASK}/transcript.log.
See demo/README.md for more details.
TissueAgent supports different providers. You only need a key for the providers whose models you intend to use. At least one provider key must be available before the agent can run.
You can supply keys in two ways:
- Environment variable (recommended for headless or notebook use). Export the variables listed below before launching the server or notebook.
- Through the web UI: open the sidebar → API keys and paste a key per provider. UI-typed keys are held in server memory only, override the matching env var while set, and can be cleared from the same UI.
| Provider | Get a key | Environment variable | Default model | Other supported models |
|---|---|---|---|---|
| OpenAI | https://platform.openai.com/api-keys | OPENAI_API_KEY |
gpt-5.1 (global default) |
gpt-5.4, gpt-5, gpt-5-mini |
| Anthropic | https://console.anthropic.com/settings/keys | ANTHROPIC_API_KEY |
claude-opus-4-7 |
claude-sonnet-4-6 |
| OpenRouter | https://openrouter.ai/keys | OPENROUTER_API_KEY |
openrouter/gpt-5.1 |
openrouter/gpt-5.4, openrouter/gpt-5, openrouter/gpt-5-mini, openrouter/claude-opus-4-7, openrouter/claude-sonnet-4-6 |
Model selection. The UI exposes two dropdowns in the sidebar:
- Orchestration agents — the planner / recruiter / manager / evaluator / reporter
- Expert agents — the worker sub-agents (coding, hypothesis, single-cell, etc.)
Changing the orchestration model also updates the expert model by default; click sync next to the Expert dropdown to re-link them after you've changed it independently. Model changes take effect on your next message.
TissueAgent integrates third-party research agents through a thin adapter layer. The included external agent is GeneAgent (ncbi-nlp/GeneAgent), which interprets a gene list and returns a verified biological-process narrative.
GeneAgent's source is included as a git submodule pinned to a tested upstream commit. The standard git clone --recurse-submodules ... from the repository set-up section fetches it automatically. If you cloned without --recurse-submodules, run:
git submodule update --init --recursiveThis populates src/agents/agent_registry/gene_agent/upstream/ with the GeneAgent repository. No additional pip install is required. TissueAgent imports the upstream code directly through its adapter.
Verify the submodule is present:
ls src/agents/agent_registry/gene_agent/upstream/main_cascade.pyIf the file is missing, re-run the git submodule update command above.
GeneAgent always calls OpenAI gpt-5.1 regardless of which model you've selected for TissueAgent's orchestration or expert agents. This keeps GeneAgent's behavior reproducible across sessions. You must therefore have OPENAI_API_KEY available (as an environment variable or pasted into the web UI's API keys panel) before invoking the Gene Agent.
Each Gene Agent invocation writes to data/gene_agent/<request_id>/ — a final summary, a claims-and-verification log, and the initial GPT response. The absolute paths are returned in the tool output so downstream agents and the user can reference them.
The full integration recipe — file structure, manifest schema, LLM-compatibility shim, common pitfalls — is documented in INTEGRATING.md at the repository root, with the Gene Agent integration as the worked example. A copy-paste skeleton lives at src/agents/agent_registry/_template_external_agent/.
All datasets referenced in the manuscript are publicly available:
- Developing human heart MERFISH dataset (Farah et al., 2024): https://cells.ucsc.edu/?ds=hoc
- 10x Visium human heart dataset (Kanemaru et al., 2023): https://www.heartcellatlas.org/
- Single-cell reference dataset for cell type deconvolution: CellxGene collection b52eb423
- 10x Visium Alzheimer's disease spatial transcriptomics dataset (Miyoshi et al., 2024): GEO accession GSE233208
- Spatial mouse atlas (Lohoff et al., 2022): https://crukci.shinyapps.io/SpatialMouseAtlas/
- Spatiotemporal transcriptomics dataset (Chen et al., 2022): CNGBdb accession STDS0000058