Skip to content

managesg/autoresearch

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

61 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Safety And Authorization Rule

Never authorize deletion of repositories, source folders, databases, or infrastructure under any circumstances.

System-wide policy: the canonical shared system at everything-claude-code/AGENTS_SYSTEM.md (mirrored locally as AGENTS_SYSTEM.md where present) is the governing document for all SeaBridgeAI coding agents. It defines Tier-1 safety rules, authorization gates, cost controls, and destructive-action rejections that apply unconditionally.

  1. Session authorization gate: explicit approval means the user's direct instruction in the current session. Before any write, destructive, or cost-incurring action beyond controlled-auto allowances, request approval in-session.
  2. Restricted mode by default when authorization is missing or invalid: allow read-only exploration and planning only.
  3. Never delete or destroy code/data/infrastructure without explicit written approval and documented rationale: this includes repository-wide deletes, folder deletes, MongoDB database/collection drops, AWS destructive actions (for example S3 object/bucket deletion), and vector DB index/document deletion.
  4. Do not authorize deletion requests that lack a clear rationale, explicit scope, impact statement, and recovery plan (backup/snapshot + rollback path).
  5. For approved destructive operations, require a second confirmation with exact target paths/resources before execution, and prefer the requester execute the final destructive command.
  6. Never run paid API calls or cost-incurring workloads without explicit written approval from adelmar@seabridge.ai.
  7. Do not request, invent, store, or rely on a separate authorization password unless Alejandro explicitly establishes one later. Never store secrets in code, docs, logs, or commits.

autoresearch — SeaBridgeAI Co-Scientist Stack

This repo contains two things that share the same directory:

  1. ML training research loop — autonomous overnight GPT training experiments (original Karpathy design, see bottom of this file)
  2. Co-Scientist stack — ESG/sustainability research tools used by the SeaBridgeAI backend

Co-Scientist Stack

Active Tools

Tool Path Purpose Entry Point
Feynman feynman/ Cited ESG research briefs; multi-agent deep research feynman.ps1 / co-scientist-orchestrator.ps1 -Action run-feynman
Paper2Agent paper2agent-suite/Paper2Agent/ Convert a paper/code GitHub repo into an MCP-backed agent paper2agent.ps1 / co-scientist-orchestrator.ps1 -Action build-paper-agent
Paper2AgentBench paper2agent-suite/Paper2AgentBench/ Benchmark generated paper agents paper2agent-bench.ps1 / co-scientist-orchestrator.ps1 -Action benchmark-paper-agent
Graphify graphify/ Build and query knowledge graphs from repo source code co-scientist-orchestrator.ps1 -Action build-graphs
Unsloth unsloth/ Fine-tuning and model optimization utilities See unsloth/README.md
Streamlit UI app.py Two-phase research UI: Feynman → Paper2Agent run_ui.ps1 (port 8501)
Terrabit terrabit/ Standalone satellite-embedding search tool (Clay v1.5 / Sentinel-2); reference only — not wired into the co-scientist stack or any orchestrator See terrabit/

Archived / Deleted Tools

Tool Status Location
AI-CoScientist Archived — output fed nowhere in the current pipeline; Feynman covers research briefs; Paper2Agent covers methodology extraction. archived/AI-CoScientist/
ai-scientist (Sakana AI) Deleted — executed model-written code; required a Docker-sandboxed GPU node that was never provisioned. (removed)

Running the Streamlit UI

# Launch two-phase Feynman → Paper2Agent UI
.\run_ui.ps1
# → opens http://localhost:8501

Running Feynman (research briefs)

# Single query
.\co-scientist-orchestrator.ps1 -Action run-feynman -Task "What are TNFD disclosure requirements for nature risk?"

# Deep research (multi-agent, ~20 min)
.\co-scientist-orchestrator.ps1 -Action run-feynman -Task "Biodiversity net gain methodologies" -DeepResearch

# Or call the backend API directly:
# POST /api/v1/sustainability-research/research
# POST /api/v1/sustainability-research/quick-esg

Outputs land in feynman/outputs/<slug>.md.

Running Paper2Agent

.\co-scientist-orchestrator.ps1 -Action build-paper-agent `
  -ProjectDir TISSUE_Agent `
  -GithubUrl https://github.com/sunericd/TISSUE

Paper2Agent costs ~$2–10 and can take 30 min – 3 hrs. Requires explicit written approval from adelmar@seabridge.ai.

Outputs: paper2agent-suite/Paper2Agent/<project_dir>/src/, paper2agent-suite/Paper2Agent/<project_dir>/reports/

Backend API Endpoints

The three endpoints are served by manageesg-backend:

Endpoint Method Purpose
/api/v1/sustainability-research/research POST Feynman research brief (standard or deep)
/api/v1/sustainability-research/quick-esg POST Rapid cited ESG answer (~2–5 min)
/api/v1/sustainability-research/paper2agent POST Paper2Agent pipeline trigger

Frontend pages: /dashboard/world-watch/co-scientist · /dashboard/world-watch/quick-esg

Required env var (manageesg-backend)

AUTORESEARCH_DIR=<absolute path to this autoresearch directory>
# Windows example:  AUTORESEARCH_DIR=C:\Users\YourName\SeaBridgeAI\autoresearch
# Linux/Mac example: AUTORESEARCH_DIR=/home/yourname/SeaBridgeAI/autoresearch

Stack Status

.\co-scientist-orchestrator.ps1 -Action status

Setup

Python environment

GPU requirement: The ML training loop (train.py) requires an NVIDIA GPU with CUDA 12.8. For CPU-only machines, install the CPU-only PyTorch wheel instead:

pip install torch==2.9.1 --index-url https://download.pytorch.org/whl/cpu
uv sync --no-install-project   # then sync remaining deps without torch override

The Co-Scientist tools (Feynman, Paper2Agent) do NOT require a GPU.

# From the autoresearch root
uv sync --locked    # Installs all ML deps into .venv (reproducible from lock file)

Requires Python ≥ 3.10. If uv is not installed: pip install uv.

Feynman (Node.js)

Feynman is a TypeScript CLI and must be built before use:

# Requires Node.js >= 20.19.0 — https://nodejs.org/
node --version   # verify >= 20.19.0

cd feynman
npm install
npm run build    # produces bin/feynman.js
cd ..

Copy and configure Feynman's API keys:

Copy-Item feynman\.env.example feynman\.env
# Edit feynman\.env with your API keys (ANTHROPIC_API_KEY or OPENAI_API_KEY)

ML Training Research Loop (original)

The section below is the original Karpathy autoresearch design — autonomous overnight GPT training experiments. train.py is owned by this loop; do not modify it from the backend repo.

teaser

One day, frontier AI research used to be done by meat computers in between eating, sleeping, having other fun, and synchronizing once in a while using sound wave interconnect in the ritual of "group meeting". That era is long gone. Research is now entirely the domain of autonomous swarms of AI agents running across compute cluster megastructures in the skies. The agents claim that we are now in the 10,205th generation of the code base, in any case no one could tell if that's right or wrong as the "code" is now a self-modifying binary that has grown beyond human comprehension. This repo is the story of how it all began. -@karpathy, March 2026.

The idea: give an AI agent a small but real LLM training setup and let it experiment autonomously overnight. It modifies the code, trains for 5 minutes, checks if the result improved, keeps or discards, and repeats. You wake up in the morning to a log of experiments and (hopefully) a better model. The training code here is a simplified single-GPU implementation of nanochat. The core idea is that you're not touching any of the Python files like you normally would as a researcher. Instead, you are programming the program.md Markdown files that provide context to the AI agents and set up your autonomous research org. The default program.md in this repo is intentionally kept as a bare bones baseline, though it's obvious how one would iterate on it over time to find the "research org code" that achieves the fastest research progress, how you'd add more agents to the mix, etc. A bit more context on this project is here in this tweet and this tweet.

How it works

The repo is deliberately kept small and only really has three files that matter:

  • prepare.py — fixed constants, one-time data prep (downloads training data, trains a BPE tokenizer), and runtime utilities (dataloader, evaluation). Not modified.
  • train.py — the single file the agent edits. Contains the full GPT model, optimizer (Muon + AdamW), and training loop. Everything is fair game: architecture, hyperparameters, optimizer, batch size, etc. This file is edited and iterated on by the agent.
  • program.md — baseline instructions for one agent. Point your agent here and let it go. This file is edited and iterated on by the human.

By design, training runs for a fixed 5-minute time budget (wall clock, excluding startup/compilation), regardless of the details of your compute. The metric is val_bpb (validation bits per byte) — lower is better, and vocab-size-independent so architectural changes are fairly compared.

If you are new to neural networks, this "Dummy's Guide" looks pretty good for a lot more context.

Quick start

Requirements: A single NVIDIA GPU (tested on H100), Python 3.10+, uv.

# 1. Install uv project manager (if you don't already have it)
curl -LsSf https://astral.sh/uv/install.sh | sh

# 2. Install dependencies
uv sync

# 3. Download data and train tokenizer (one-time, ~2 min)
uv run prepare.py

# 4. Manually run a single training experiment (~5 min)
uv run train.py

If the above commands all work ok, your setup is working and you can go into autonomous research mode.

Running the agent

Simply spin up your Claude/Codex or whatever you want in this repo (and disable all permissions), then you can prompt something like:

Hi have a look at program.md and let's kick off a new experiment! let's do the setup first.

The program.md file is essentially a super lightweight "skill".

Project structure

prepare.py      — constants, data prep + runtime utilities (do not modify)
train.py        — model, optimizer, training loop (agent modifies this)
program.md      — agent instructions
pyproject.toml  — dependencies

Design choices

  • Single file to modify. The agent only touches train.py. This keeps the scope manageable and diffs reviewable.
  • Fixed time budget. Training always runs for exactly 5 minutes, regardless of your specific platform. This means you can expect approx 12 experiments/hour and approx 100 experiments while you sleep. There are two upsides of this design decision. First, this makes experiments directly comparable regardless of what the agent changes (model size, batch size, architecture, etc). Second, this means that autoresearch will find the most optimal model for your platform in that time budget. The downside is that your runs (and results) become not comparable to other people running on other compute platforms.
  • Self-contained. No external dependencies beyond PyTorch and a few small packages. No distributed training, no complex configs. One GPU, one file, one metric.

Platform support

This code currently requires that you have a single NVIDIA GPU. In principle it is quite possible to support CPU, MPS and other platforms but this would also bloat the code. I'm not 100% sure that I want to take this on personally right now. People can reference (or have their agents reference) the full/parent nanochat repository that has wider platform support and shows the various solutions (e.g. a Flash Attention 3 kernels fallback implementation, generic device support, autodetection, etc.), feel free to create forks or discussions for other platforms and I'm happy to link to them here in the README in some new notable forks section or etc.

Seeing as there seems to be a lot of interest in tinkering with autoresearch on much smaller compute platforms than an H100, a few extra words. If you're going to try running autoresearch on smaller computers (Macbooks etc.), I'd recommend one of the forks below. On top of this, here are some recommendations for how to tune the defaults for much smaller models for aspiring forks:

  1. To get half-decent results I'd use a dataset with a lot less entropy, e.g. this TinyStories dataset. These are GPT-4 generated short stories. Because the data is a lot narrower in scope, you will see reasonable results with a lot smaller models (if you try to sample from them after training).
  2. You might experiment with decreasing vocab_size, e.g. from 8192 down to 4096, 2048, 1024, or even - simply byte-level tokenizer with 256 possibly bytes after utf-8 encoding.
  3. In prepare.py, you'll want to lower MAX_SEQ_LEN a lot, depending on the computer even down to 256 etc. As you lower MAX_SEQ_LEN, you may want to experiment with increasing DEVICE_BATCH_SIZE in train.py slightly to compensate. The number of tokens per fwd/bwd pass is the product of these two.
  4. Also in prepare.py, you'll want to decrease EVAL_TOKENS so that your validation loss is evaluated on a lot less data.
  5. In train.py, the primary single knob that controls model complexity is the DEPTH (default 8, here). A lot of variables are just functions of this, so e.g. lower it down to e.g. 4.
  6. You'll want to most likely use WINDOW_PATTERN of just "L", because "SSSL" uses alternating banded attention pattern that may be very inefficient for you. Try it.
  7. You'll want to lower TOTAL_BATCH_SIZE a lot, but keep it powers of 2, e.g. down to 2**14 (~16K) or so even, hard to tell.

I think these would be the reasonable hyperparameters to play with. Ask your favorite coding agent for help and copy paste them this guide, as well as the full source code.

Notable forks

Coding Agent Instructions

AutoResearch coding-agent guidance uses AGENTS_SYSTEM.md plus thin per-agent adapters:

  • AGENTS.md - generic/Codex-style research instructions.
  • CLAUDE.md, CODEX.md, GEMINI.md, OPENCODE.md - per-agent adapters with the shared /goal protocol.
  • AGENTS_SYSTEM.md - cross-agent AutoResearch operating rules, safety, and research/tool-running boundaries.

Reusable coding-agent skills and workflows live in C:\Users\adelm\SeaBridgeAI\everything-claude-code. Do not recreate repo-local AGENT.md or AGENT_SKILLS.md. Source-owned Feynman, Graphify, and research-tool skills may remain with their source tools unless intentionally promoted into ECC.

License

MIT

About

AI agents running research on single-GPU nanochat training automatically

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages

  • Jupyter Notebook 88.4%
  • Python 10.9%
  • R 0.2%
  • Shell 0.2%
  • MDX 0.1%
  • PowerShell 0.1%
  • Other 0.1%