Skip to content

Add paper trading mode with Finnhub data fallback#1

Open
smshapiro85 wants to merge 21 commits into
mainfrom
fix/multi-ai-review-issues
Open

Add paper trading mode with Finnhub data fallback#1
smshapiro85 wants to merge 21 commits into
mainfrom
fix/multi-ai-review-issues

Conversation

@smshapiro85
Copy link
Copy Markdown
Owner

@smshapiro85 smshapiro85 commented Jan 25, 2026

Summary

  • Add paper trading mode for experiments that don't require Schwab connectivity
  • Implement Finnhub API integration for real-time stock quotes as data source fallback
  • Create unified market data service with automatic fallback chain: Schwab → Finnhub → Mock
  • Fix "Generate Today's Experiments" button by ensuring experiments table exists in seed

Changes

  • New: finnhubQuotes.ts - Real-time stock quotes from Finnhub API with caching
  • New: marketDataService.ts - Unified market data interface with fallback chain
  • Updated: Database schema adds paper_trading and data_source columns to experiments
  • Updated: Strategy executor uses unified market data service
  • Updated: Status API reports available data sources and their connection status

How Paper Trading Works

  1. Experiments default to paperTrading: true and dataSource: 'finnhub'
  2. Configurable starting balance (default: $10,000)
  3. Real market data from Finnhub tracks virtual P&L
  4. No Schwab connection required

Test plan

  • Run npm run db:seed to ensure experiments table is created
  • Set FINNHUB_API_KEY in .env and verify quotes are fetched
  • Click "Generate Today's Experiments" button and verify it works
  • Check /api/status shows data source availability
  • Generate experiments with custom startBalance and dataSource params

🤖 Generated with Claude Code

smshapiro85 and others added 21 commits January 24, 2026 22:49
@smshapiro85
Fix multi-AI review issues
4604ef2
@smshapiro85 @claude
Add paper trading mode with Finnhub data fallback
a885b80 
Implement test account / paper trading mode for experiments that don't
require Schwab connectivity. Experiments can now run on real market
data from Finnhub API as a fallback.

Changes:
- Add finnhubQuotes.ts for real-time stock quotes from Finnhub API
- Create unified marketDataService.ts with fallback chain: Schwab → Finnhub → Mock
- Add paper_trading and data_source columns to experiments table
- Update seed.ts to ensure experiments table exists
- Strategy executor now uses unified market data service
- Status API reports available data sources

API accepts paperTrading, dataSource, and startBalance params when
generating experiments.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85 @claude
Add API and E2E test suites
a99693e 
Set up comprehensive testing infrastructure:

API Tests (Vitest):
- Configure Vitest with test utilities and mock factories
- Add market data service tests (14 tests) covering fallback chain
- Add experiments API tests (11 tests) for paper trading flows
- Create test helpers for mocking database and Finnhub API

E2E Tests (Playwright):
- Configure Playwright with auto-start for frontend/backend
- Add experiment flow tests (generate, display, navigation)
- Add dashboard and strategies page tests

Test commands: npm run test, npm run test:e2e

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85 @claude
Fix E2E test selector for multiple generate buttons
81da334 
Use .first() to handle pages with multiple "Generate Experiments" buttons.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85 @claude
Fix Express route order bug in strategies API
225cc36 
Move all /experiments routes before /:id routes to prevent
'experiments' from being matched as a strategy ID parameter.

Express matches routes in order, so the wildcard /:id on line 108
was intercepting requests to /experiments before they could reach
the actual experiment endpoints on line 308.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85 @claude
Add experiment deletion and show all today's experiments
232028c 
UI fixes:
- Remove .slice(0, 5) limit to show ALL today's experiments
- Add delete button to individual experiments in historical list
- Add "Clear Date" button to delete all experiments for selected date
- Show experiment count in "Today's Experiments" header

Backend changes:
- Add deleteExperiment and deleteExperimentsByDate to strategyLibrary
- Add DELETE /experiments/:id endpoint for single experiment
- Add DELETE /experiments?date=YYYY-MM-DD endpoint for bulk delete

Frontend API:
- Add experimentsApi.delete() method
- Add experimentsApi.deleteByDate() method

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85 @claude
Add automated experiment execution with shared trading logic
162411a 
New architecture ensures experiments use IDENTICAL execution logic as live trading:

Shared Components:
- tradeExecutionContext.ts: Abstraction layer for position/trade tracking
  - Can route to live positions table or experiment-specific tracking
  - Provides consistent interface for balance, positions, and trade recording

- signalProcessor.ts: Shared signal execution logic
  - processBuySignal(): Same VIX adjustment, position sizing, risk checks
  - processSellSignal(): Same P&L calculation, position closing
  - closePositionAtMarket(): Same EOD close logic
  - updatePositionsAndCheckStopLoss(): Same stop-loss monitoring

Experiment Runner:
- Uses executeStrategy() from strategyExecutor (same YAML parsing)
- Uses signalProcessor for trade execution (same execution logic)
- Only differences are:
  - Virtual balance per experiment (paper trading)
  - Trades stored in experiment_trades table
  - Positions tracked per-experiment

Scheduler Integration:
- 9:35 AM ET: Start PENDING experiments -> RUNNING
- Every 30 sec (market hours): Run experiment analysis cycles
- 3:45 PM ET: Complete experiments -> COMPLETED with P&L
- Startup recovery: Loads running experiments if app was restarted

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85
Make wake command instructions more explicit
b31321a
@smshapiro85 @claude
Fix Codex review issues: ensure identical execution logic
ab7bc66 
Addresses all 3 error-level issues from Codex review:

1. Quote source mismatch (issue-3):
   - Changed from marketDataService.getQuote to mockBroker.getQuote
   - Now uses SAME quote source as live trading engine

2. Double VIX adjustment / position sizing (issue-1):
   - Removed signal.quantity-based sizing
   - Now uses tradingSettings.maxPositionSize * portfolioValue (SAME as live)
   - VIX adjustment applied ONCE via getVIXAdjustedPositionSize()

3. Stop-loss logic mismatch (issue-2):
   - Removed fixed percent stopLossPct parameter
   - Now uses calculateStopLossPrice() from riskManager (SAME as live)
   - Config-driven stop-loss from trading settings

Also addresses warning-level issues:
- Added concurrency guard (cycleInProgress flag) to prevent overlapping cycles
- Fixed position resume logic to properly net buy/sell quantities per symbol
- Removed unused dataSource field from TradeExecutionContext

The only differences between experiments and live trading are now:
- Virtual balance tracking per experiment
- Trades stored in experiment_trades table (not trades table)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85 @claude
Add manual trigger endpoints for experiment testing
5e04f1a 
New endpoints:
- POST /api/strategies/experiments/start-all
  Manually triggers startTodaysExperiments() to start all PENDING
  experiments for today. Useful for testing on weekends/off-hours.

- POST /api/strategies/experiments/run-cycle
  Manually runs one experiment analysis cycle. Useful for testing
  signal processing and trade execution.

- GET /api/strategies/experiments/running-status
  Returns the count and IDs of currently running experiments.

All endpoints are placed before /:id routes to avoid route matching issues.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85 @claude
Add comprehensive E2E test framework for experiments
641b457 
- Create e2eTestUtils.ts with 10 test scenarios per experiment:
  1. Winning Trade - verify profit calculation
  2. Losing Trade (Stop Loss) - verify stop loss execution
  3. Max Daily Trades - verify trade limit enforcement
  4. Max Daily Loss - verify loss limit enforcement
  5. Position Sizing - verify position size limits
  6. Schedule End - verify time-based termination
  7. Manual Disable - verify manual stop functionality
  8. Negative News Threshold - placeholder for sentiment-based pausing
  9. Global Risk Limit - verify global loss limit
  10. Experiment Completion - verify proper status transitions

- Add test API endpoints:
  - POST /experiments/test/force-trade - force a trade for testing
  - POST /experiments/test/simulate-cycle - simulate complete trade cycle
  - POST /experiments/test/run-e2e-suite - run full test suite
  - GET /experiments/test/termination-paths - list all termination paths
  - POST /experiments/test/cleanup - identify test trades for cleanup

- Add gap analysis for test coverage of termination paths
- Add comprehensive documentation in docs/E2E-TESTING.md

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85 @claude
Fix E2E test symbol length and idempotency issues
4e636af 
- Shorten test symbols to fit VARCHAR(10) constraint:
  TEST_WIN → T_WIN, TEST_LOSS → T_LOSS, etc.
- Make max_daily_trades test idempotent with unique run IDs
- Update cleanup function to match new symbol prefixes

Test results: 220/220 scenarios passing (100%)
- 22 experiments tested
- 10 scenarios per experiment
- 66.7% termination path coverage (10/15 paths)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85
Simplify alerting - cron monitor is now primary
29d1ff5
@smshapiro85 @claude
Document E2E test audit findings
e614915 
Audit confirms E2E tests work as designed:
- Playwright tests call real services (OpenRouter, Finnhub) when configured
- Backend test utilities use synthetic data for offline/weekend testing
- Market data fallback to mocks is intentional for after-hours testing

No code changes needed - architecture is sound.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85 @claude
Verify E2E tests pass - all 7 tests green
16b2a33 
Ran full E2E test suite confirming all tests pass:
- Experiments page display and generation
- Paper trading mode status
- Dashboard navigation
- Strategies page display

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85 @claude
Remove build artifacts from git tracking
19f861f 
Add dist/, playwright-report/, and test-results/ to .gitignore.
These are build/test artifacts that should not be version controlled.

All E2E tests verified passing (7/7).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85 @claude
Enable 2 previously skipped experiment tests
f11df45 
The route resolution issue was already fixed. Removed TODOs and
it.skip() wrappers - all 27 backend tests now pass.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85 @claude
Fix activity log API port (3001 -> 3005)
ac7ac5c 
The ActivityLog page was hardcoded to port 3001 but the backend
runs on port 3005, causing "Error loading activity log" message.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85 @claude
Fix ActivityLog to use Vite proxy instead of direct URL
77f69e8 
Use relative '/api' path to go through Vite's proxy configuration,
avoiding CORS issues with direct cross-origin requests.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85 @claude
fix: auto-complete orphaned RUNNING experiments on startup
47211dd 
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@smshapiro85 @claude
fix: add defense-in-depth for historical experiments status
45c3361 
Historical experiments should NEVER be in RUNNING state. This adds
multiple layers of protection:

1. updateExperimentStatus() - Blocks attempts to set past-day
   experiments to RUNNING status

2. getExperiments() - Auto-completes orphaned RUNNING experiments
   when querying by past date

3. getExperimentsByDate() - Same auto-completion for date queries

4. getExperimentById() - Auto-completes if fetching an orphaned
   RUNNING experiment from a past day

Combined with the startup auto-complete from the previous commit,
this ensures orphaned experiments are cleaned up at multiple points:
- On application startup
- When querying experiments for display
- When attempting status updates

Note: An ARCHIVED status was considered but deemed unnecessary - the
existing COMPLETED status adequately represents finished experiments
regardless of age.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@smshapiro85