Quant Engine v2 — full polyglot implementation, runner & CI - #1
Open
ertanuj96 wants to merge 8 commits into
Open
Quant Engine v2 — full polyglot implementation, runner & CI#1ertanuj96 wants to merge 8 commits into
ertanuj96 wants to merge 8 commits into
Conversation
Rust quant-engine workspace (7 crates, all compiling, 8 tests passing): - common: Canonical event types, NATS subjects, config for all engines - regime: HMM (Baum-Welch + Viterbi), Hurst exponent (R/S), GARCH(1,1) - trend: Kalman filter, OLS regression + R², Mann-Kendall significance - reversal: Bayesian changepoint detection, CUSUM, divergence detection - movement: Realized vol, ATR, option-implied expected move - signal: Bayesian fusion, Brier score calibration, confidence filter - executor: Capped Kelly sizing, position-level risk, SEBI OPS throttle Go ingestion service (structural scaffold): - Module with Dhan/Fyers/INDstocks WebSocket consumer stubs - Canonical tick normalization matching Rust event types Architecture docs: - PTS-004: Architecture v2 with full Mermaid diagrams (system overview, v1→v2 integration map, signal pipeline, ERD, NATS topology, deployment) Infrastructure: - Top-level Makefile for polyglot build/test/dev - Updated .gitignore for Rust target/ directory
- broker-supervisor/: Erlang/OTP supervisor tree for multi-broker connections (Dhan, Fyers, INDstocks) with kill switch, rate limiter (SEBI <10 OPS), and auto-reconnect via broker_session gen_server - persistence.py: adds pts_quant DB + 7 new ClickHouse tables for v2 (candle_data, pcr_history, vix_history, instrument_master, regime_history, signal_scores, commodity_data) per PTS-003 + PTS-004 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…s all subsystems Rust (quant-engine): - proptest + approx added as workspace dev-deps - regime: 24 unit tests (HMM construction/convergence/stability, Hurst bounds, GARCH stationarity/forecast/variance), + regime_pipeline integration tests - trend: Kalman (uptrend/downtrend/flat/noisy), Mann-Kendall (significance/bounds) - signal: Bayesian fusion (confidence bounds, regime/trend/reversal sensitivity), CalibrationTracker (Brier score: perfect/random/always-wrong/windowing) - executor: Kelly sizing (edge/no-edge/caps) + proptest property coverage, RiskCheck (daily loss / position size / drawdown guards), executor_pipeline integration tests (Kelly → risk approval pipeline) Go (ingestion): - normalize/tick_test.go: table-driven tests for TickEvent JSON round-trip, omitempty field handling, LTP bounds, exchange field, multiple scenarios Erlang (broker-supervisor): - rate_limiter.erl: EUnit (known broker, unknown broker, 9-OPS SEBI enforcement) - kill_switch.erl: EUnit (initially inactive, activates once, idempotent second call) - test/broker_supervisor_SUITE.erl: Common Test suite (supervisor lifecycle, kill switch semantics, rate limiter SEBI compliance, 1-second refill) Python (dashboard-api + trading-core): - pyproject.toml: pytest + pytest-asyncio + httpx dev deps - conftest.py: Redis + NATS mocked fixtures (no infrastructure required) - test_positions.py: /api/positions, /api/pnl, /api/exposure, /api/prices, /api/portfolio — empty state + populated state coverage - test_system.py: ConnectionManager unit tests - trading-core/tests/test_events.py: EventEnvelope serialization, TickEvent, SignalGeneratedEvent, RiskApprovedEvent, LatencyTracker, Subjects, Enums React (dashboard-ui): - vitest + @testing-library/react + jsdom added - vitest.config.js with jsdom environment + coverage config - StatsRow.test.jsx: 11 tests (profit/loss classes, position count, Indian locale formatting, price placeholders, live price display) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…eline Rust (quant-engine): - reversal/changepoint.rs: Full BOCD (Adams & MacKay 2007) with NIG conjugate prior, log-space belief propagation, MAX_RUN=500 cap, Student-t predictive - reversal/engine.rs: ReversalEngine — BOCD + CUSUM(5.0) on normalized returns, OI/volume divergence, 200-bar rolling buffers, 20-obs warmup gate - movement/engine.rs: MovementEngine — EWMA(λ=0.94) vol, ATR(14), realized vol, optional implied move from ATM straddle premiums - signal/engine.rs: SignalEngine — async NATS subscriber over 4 event streams, 30s staleness gate, Bayesian fusion → ScoredSignal on pts.quant.signal - executor/engine.rs: ExecutorEngine — Kelly sizing → risk check → OPS throttle → OrderIntent published to pts.order.submitted - executor/state_machine.rs: OrderStateMachine FSM (New → Submitted → Acknowledged → PartiallyFilled/Filled/Cancelled/Rejected) with transition guards - common/events.rs: impl Display for Timeframe (needed as HashMap key) - Cargo.toml: added futures workspace dep for StreamExt Go (ingestion): - internal/config/config.go: env-var config with sensible defaults - internal/broker/types.go: RawTick + Adapter interface - internal/broker/dhan.go: Dhan MARKETFEED WebSocket adapter - internal/broker/fyers.go: Fyers WebSocket v3 adapter - internal/normalize/normalizer.go: broker.RawTick → canonical TickEvent - internal/candle/materializer.go: OHLCV candle builder with IST bar alignment for 1m/5m/15m/1h/4h; emits on boundary cross - internal/candle/materializer_test.go: 5 table-driven tests - internal/publisher/nats.go: NATS publisher for ticks and candles - internal/publisher/clickhouse.go: batched CH writer with flush loop - cmd/ingest/main.go: full wiring — adapters → normalizer → materializer → NATS + ClickHouse, graceful shutdown, auto-reconnect All 111 Rust tests pass; Go tests pass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
quant-engine:
- New crate engine-runner with `quant-engine` binary — the orchestrator that
boots the full pipeline on NATS:
pts.market.candle -> Regime/Trend/Reversal/Movement engines (per symbol+tf)
-> pts.quant.{regime,trend,reversal,movement} -> SignalEngine (Bayesian fusion)
-> pts.quant.signal -> ExecutorEngine (Kelly->risk->OPS) -> pts.order.submitted
Config via env vars; SignalEngine + ExecutorEngine run as concurrent tasks,
analysis engines driven off the candle stream.
- executor/lib.rs: export ExecutorConfig + OrderIntent
infra:
- infra/clickhouse/init/01-schema.sql — full schema (5 databases, 15 tables),
column order matches the Go writer's batch.Append() calls; mounted into
ClickHouse /docker-entrypoint-initdb.d so tables exist at container startup
- docker-compose.yml: bind-mount the init dir into clickhouse
- Makefile: cargo run --bin quant-engine
All 111 Rust tests still green; workspace + runner binary build clean.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
CI (.github/workflows/ci.yml) — 6 jobs on push/PR to main: - rust: cargo fmt --check, clippy -D warnings, test --workspace, build --release - go: go vet, build, test -race (ingestion) - python-api: pip install -e .[test] + pytest (dashboard-api) - python-core: pip install -e . + pytest (trading-core) - ui: npm ci + vitest (dashboard-ui) - erlang: rebar3 compile + eunit (broker-supervisor) To make the gates pass: - Ran cargo fmt across the whole workspace - Fixed all clippy lints (manual RangeInclusive::contains → .contains(), redundant lots.min/max, missing Default impls) so clippy -D warnings is clean - Added [build-system] + setuptools package config to dashboard-api and trading-core pyproject.toml so `pip install -e .` works in CI - gitignore .claude/ (agent worktrees) All 111 Rust tests, Go tests pass locally; fmt + clippy clean. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ertanuj96
changed the title
- dashboard-ui: Vitest 2 couldn't drive Vite 8's plugin pipeline, so the
React JSX transform fell back to the classic runtime ("React is not
defined"). Bump vitest + coverage-v8 to ^3.2.4 and transform JSX via
esbuild's automatic runtime in vitest.config (drops the plugin-react ↔ Vite
version coupling). 11/11 StatsRow tests pass.
- dashboard-api: inject the Redis/NATS mock as the cached module global
(deps._redis / deps._nc) so routes that call get_redis() directly never hit a
real connection. 14/14 pass.
- broker-supervisor: drop the unresolvable `nats`/`jsx` hex deps (unused in
source) so `rebar3 compile`/`eunit` run hermetically; remove `nats` from the
app's applications list.
Local: dashboard-ui 11, dashboard-api 14, trading-core 32 — all green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
npm ci on the Linux runner failed because the macOS-generated lockfile omits Linux-only optional deps (@emnapi/* wasm fallbacks). Use npm install so they resolve on the runner. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Quant Engine v2 — full polyglot implementation, runner, and CI
This PR builds out the complete v2 quant pipeline across the polyglot stack, wires it into a runnable orchestrator, and adds CI.
Rust quant-engine (7 crates)
ScoredSignal, Brier-score calibrationOrderIntent, with a full order-state-machine FSMquant-enginebinary that boots the whole pipeline:Go ingestion
Broker WebSocket adapters (Dhan, Fyers) → normalizer → IST-aligned OHLCV candle materializer (1m/5m/15m/1h/4h) → NATS + batched ClickHouse writer, with graceful shutdown and auto-reconnect.
Erlang broker-supervisor
OTP supervision tree with rate limiter (≤9 OPS, SEBI) and kill switch.
Python + React
dashboard-api(FastAPI) andtrading-coreevent/persistence layerdashboard-ui(React) dashboardInfra
docker-compose.yml: NATS (JetStream) + Redis + ClickHouse + Prometheus + Grafanainfra/clickhouse/init/01-schema.sql: full schema (5 databases, 15 tables), auto-applied at container startup; column order matches the Go writer's batch insertsTests & CI
.github/workflows/ci.ymlwith 6 jobs: rust (fmt + clippy-D warnings+ test + build), go (vet + race tests), python-api, python-core, ui (vitest), erlang (rebar3 eunit)Run locally
🤖 Generated with Claude Code