Solana-Yellowstone-Indexer

A real-time Solana indexer written in Rust. It connects to a Yellowstone gRPC stream, subscribes to account / slot / transaction / block updates, and persists them to a local SQLite database.

This is a Rust port of a TypeScript indexer. The Node/Prisma stack was replaced with an idiomatic async Rust stack:

Concern	Implementation
gRPC stream	yellowstone-grpc-client + yellowstone-grpc-proto
Async runtime	tokio + futures
Storage	sqlx with SQLite + SQL migrations
Logging	tracing / tracing-subscriber
Config	dotenvy

---

What it does

1. Connects to a Yellowstone gRPC endpoint (with an optional x-token).
2. Subscribes using one of three built-in modes (see below).
3. Processes each SubscribeUpdate, matching on the update kind and writing a row to the matching table. Server pings are answered to keep the stream alive.
4. Stores the data in SQLite. The database file and schema are created automatically on first run via embedded migrations.

Indexed update kinds: accounts, transactions, slot updates, and block updates.

---

Architecture

Each module has a single, testable responsibility:

src/
├── main.rs          # entry point: wire everything together
├── settings.rs      # environment config + SUBSCRIPTION_MODE parsing
├── connection.rs    # build the Yellowstone gRPC client (TLS, x-token)
├── subscriptions.rs # the three SubscribeRequest builders
├── pipeline.rs      # drive the bidirectional stream, answer pings
├── processor.rs     # decode each update and route it to the store
└── store.rs         # SQLite pool + one typed insert per update kind
migrations/
└── 0001_initial_schema.sql   # accounts, transactions, slot_updates, block_updates
tests/
└── end_to_end.rs    # synthetic updates -> processor -> SQLite -> assertions

Data flow: main → settings → store → subscriptions → connection → pipeline → processor → store.

---

Subscription modes

Select the mode with the SUBSCRIPTION_MODE environment variable (default main):

Mode	What it subscribes to
main	USDC/USDT account updates, slot updates, USDC/USDT transactions, USDC blocks
accounts	USDC/USDT/WSOL account updates only
token	All accounts owned by the SPL Token program, filtered to 165-byte accounts

---

Setup

1. Prerequisites

• Rust 1.80+ (rustup)
• Access to a Yellowstone gRPC endpoint (and an x-token if it is authenticated)

2. Configure

Copy the example environment file and fill it in:

cp .env.example .env

ENDPOINT=https://your-endpoint.example.com:443
TOKEN=your_x_token          # leave blank if the endpoint is unauthenticated
DATABASE_URL=sqlite://indexer.db
SUBSCRIPTION_MODE=main      # main | accounts | token

3. Run

cargo run --release

The SQLite file at DATABASE_URL is created and migrated automatically. Use RUST_LOG to control log verbosity, e.g.:

RUST_LOG=info,solana_yellowstone_indexer=debug cargo run --release

---

Testing

cargo test          # 9 unit tests + 6 integration tests
cargo clippy --all-targets -- -D warnings
cargo fmt --check

The integration test (tests/end_to_end.rs) feeds synthetic account, transaction, slot, and block updates through the real processing pipeline into a temporary SQLite database and asserts the exact rows that land — exercising the full decode-and-persist path without needing a live endpoint.

---

Data model

Large u64 chain values (lamports, slot, rent epoch, block height, counts) are stored as TEXT to avoid signed 64-bit overflow.

• accounts — pubkey, owner, lamports, executable, rent_epoch, slot, data_length
• transactions — signature, slot, success
• slot_updates — slot, parent
• block_updates — blockhash, block_height, block_time, parent_slot, parent_blockhash, executed_transactions, updated_accounts, entries

---

License

MIT