aicrawl is a local-first archive for personal AI conversation history. The v0.1 CLI imports official Claude and ChatGPT export ZIP/JSON files into a private SQLite archive, preserves raw JSON payloads, indexes searchable message and extracted attachment/file text with FTS5, and exports conversations as Markdown.
The project follows the OpenClaw pattern: provider-specific parsing lives in aicrawl, while reusable local archive mechanics use github.com/openclaw/crawlkit where it fits.
This repository builds with Go 1.26.2 or newer because the current crawlkit module requires that toolchain.
GOWORK=off go build ./cmd/aicrawlaicrawl init
aicrawl import ./claude-export.zip --provider claude
aicrawl import ./chatgpt-export.zip --provider chatgpt
aicrawl conversations --limit 25
aicrawl messages --conversation <conversation-id> --path current
aicrawl search "known phrase"
aicrawl search "finance" --group conversations --sort recent --scope visible
aicrawl messages --conversation <conversation-id> --around <message-id> --context 5
aicrawl sql "select count(*) from messages"
aicrawl export markdown --out ./exported-md
aicrawl export markdown --conversation <conversation-id> --out ./conversation-md
aicrawl crawlbar manifestAutomation and CrawlBar control surfaces:
aicrawl metadata --json
aicrawl status --json
aicrawl doctor --jsonv0.1 supports local files only:
- Claude official export ZIPs or extracted JSON containing conversation data.
- ChatGPT official export ZIPs or extracted JSON containing
conversations.json, export batch JSON, or top-level conversations withmapping.
aicrawl does not call Claude, ChatGPT, Anthropic, or OpenAI network APIs during import, search, SQL, Markdown export, or CrawlBar manifest generation.
- Session-token scraping.
- Browser automation to click export buttons.
- Private product APIs.
- Enterprise compliance API ingestion.
- Embeddings or semantic search.
- TUI, watch daemon, scheduler, cloud sync, mirror, publish, or backups.
- Importing archived conversations back into Claude or ChatGPT.
Exports and archive databases contain private conversation data. Keep real export files and generated archives in private ignored paths such as imports/private/ during local testing.
The CLI creates runtime directories, config files, Markdown exports, and CrawlBar manifest files with private permissions where it creates them. It rejects unsafe ZIP paths and opens SQL inspection paths read-only.
aicrawl import does not delete source exports. Delete or archive them manually when you no longer need them.
Expected verification before handoff:
GOWORK=off go mod tidy
git diff --exit-code -- go.mod go.sum
GOWORK=off go vet ./...
GOWORK=off go test -count=1 ./...Synthetic fixtures live under testdata/redacted/. Do not commit real exports, local archives, logs, caches, credentials, or private planning docs.