Performance: App re-parses entire JSONL files on every session click instead of using database

[ZeroSumQuant]

Problem

The current working extractor (502K, SHA256: 9ae3e44f534671e7ed648dc4b4f83e64feccf0fc1e53c16bc2e065b477d80cab) has a critical performance issue:

Every time a user clicks on a session, the entire JSONL file is re-parsed from scratch, even though we run build_index on startup which should populate a database.

Current Behavior

1. On app startup: build_index runs but doesn't actually populate the SQLite database
2. On session click: extract command re-parses the entire JSONL file directly
3. Performance impact:
   • Small sessions (1KB): ~instant
   • Medium sessions (177KB): ~200ms
   • Large sessions (64MB): ~1-2 seconds
   • Huge sessions (530MB): ~5-10 seconds

Expected Behavior

1. On app startup: build_index should parse all JSONL files ONCE and populate SQLite database
2. On session click: extract should run a simple SQL query:

   SELECT * FROM messages WHERE conversation_id = ? LIMIT 50 OFFSET ?

3. Performance: Should be <1ms regardless of session size

Technical Details

The newer extractor.zig (539K) attempted to fix this but has bugs:

• Transaction handling issues (rollback called incorrectly)
• BlockIndex updates before import completes
• Foreign key constraint failures
• Results in 0 messages being imported

Files Involved

• extractor.zig - IncrementalImporter.importFile() function
• claude_ui/lib/features/conversation/conversation_screen.dart - _loadInitialMessages()
• Database schema with messages, conversations, and FTS5 tables

Reproduction

1. Open the app
2. Click on a large session (e.g., session_12 which is 530MB)
3. Notice the long delay before messages appear
4. Click back and click the same session again
5. Still slow (no caching benefit)

Impact

Users with large conversation histories experience 5-10 second delays when opening conversations, making the app feel sluggish and unresponsive despite having loading skeletons.

Proposed Solution

Fix the database population in build_index so that:

1. All messages are properly inserted into SQLite on first run
2. BlockIndex correctly tracks processed lines
3. extract uses SQL queries instead of file parsing
4. Response time becomes constant (~1ms) regardless of file size