Rag engine by IsaacBurns7 · Pull Request #18 · Firelight-Innovations/FiApply

IsaacBurns7 · 2025-10-11T04:24:47Z

OVERARCHING GOAL:

Support scholarship matching based upon user documents, attributes, and scholarship documents and attributes.

To this end, the following must be created:

Test suite for the below
Vector store and accompanying scripts
Embedding Service
Retrieval Engine and accompanying scripts
Context Builder and accompanying scripts
LLM Client
Response processor to verify model fits deterministic parameters
Query classifier to route user queries (may be separated from current rag engine)
Rag engine (to bundle together)

- Updated electron package-lock.json - Removed frontend build output files (out/main/index.js, out/preload/index.js) - Removed frontend/src/features/health/HealthDashboard.tsx - Added backend/services/llm_manager/TODO.md

- Added PostgreSQL 16 Alpine Dockerfile with pgvector, pg_trgm, and pg_stat_statements - Created comprehensive init.sql with extension initialization and configuration - Consolidated schema into fiapply.sql with all tables, indexes, and views - Updated docker-compose.yml with PostgreSQL and pgAdmin services - Created database utility script (db_utils.sh) for common operations - Added comprehensive database documentation (README.md) - Updated main README to reflect PostgreSQL usage - Updated Makefile to launch pgAdmin instead of SQLite Web - Updated architecture and tech stack documentation - Removed SQLite references from documentation Database features: - pgvector for 384-dimensional embeddings and semantic search - pg_trgm for fuzzy text search with trigram similarity - pg_stat_statements for query performance monitoring - HNSW indexes for fast approximate nearest neighbor search - GIN indexes for efficient trigram-based text search - Materialized views for statistics - Auto-updating triggers for updated_at timestamps

- Replace SQLite database paths with PostgreSQL connection parameters - Update DATABASE_URL to use postgresql:// connection string format - Update RAG Engine database settings to use PostgreSQL environment variables

- Switch to PostgreSQL 17-alpine3.22 (from 16) - Update pgvector to v0.8.0 for compatibility - Skip LLVM bitcode generation (with_llvm=no) to avoid clang-19 dependency - All extensions (vector, pg_stat_statements, pg_trgm) working correctly - Successfully tested with full schema initialization

- Migrated all custom components from Archive/frontend to frontend/ - Installed and configured ShadCN UI with Tailwind CSS v3 - Migrated UI components: button, card, badge, separator, hover-card - Migrated pages: Dashboard, HealthDashboard, Landing - Added Redux Toolkit store with healthSlice - Configured React Router for navigation - Updated TypeScript paths and build configuration - Created Dockerfile for Electron frontend - Updated docker-compose.yml to reflect new frontend structure - Consolidated electron directory into frontend root - Removed old /electron directory in favor of /frontend as the Electron app

…being used solely for reference. Created test file, basic document upload. will add more sophisticated upload later, probably using minilm for now. we should probably use resume as initial doc score (run on up to 10000), then use cv / essays / career goals with colbert(heay) for doc score 2 (run on up to 100)

FOOincognita

RAG Engine is missing a core library, LangChain, which it should be built on; this will greatly lower development time & increase functionality. It must also receive & handle requests from AI Orchestrator, as well as send requests to LLM Manager for all LLM Interactions.

FOOincognita · 2025-10-12T17:20:12Z

+import datetime
+from typing import List, Dict, Any, Optional, Tuple
+from dataclasses import dataclass
+from enum import Enum
+import psycopg2
+
+import ollama


Noticed you don't import gRPC nor LangChain anywhere; RAGEngine is meant to be built using LangChain, & also must invoke gRPC to make calls to LLM Manager. It must also receive requests from ai orchestrator.

LangChain will provide every possible operation & performance enhancement we'd need for RAG Engine. The challenge with this service is to use LangChain to handle requests from ai orchestrator & apply the correct LangChain functions to the request to generate a valuable response, as well as optimizing that process using LangChain's optimizations + supportive caching libraries.

No other service aside from LLM Manager should ever call OpenRouter or Ollama; RAG Engine must make all requests to LLMs by calling LLM Manager.

I'll add that arrow to the architecture diagram.

FOOincognita · 2025-10-12T17:23:27Z

 ------------
-
-CREATE EXTENSION vector;
+CREATE EXTENSION IF NOT EXISTS vector;


Remove; pg_vector is already initialized in init.sql along with the other plugins

FOOincognita · 2025-10-12T17:23:34Z

 ------------
-
-CREATE EXTENSION vector;
+CREATE EXTENSION IF NOT EXISTS vector;


Remove; pg_vector is already initialized in init.sql along with the other plugins

FOOincognita · 2025-10-12T17:28:14Z

    updated_at TIMESTAMP DEFAULT NOW()
 );

+CREATE TABLE IF NOT EXISTS user_documents (


User documents should be placed in a separate model file, & user-document relationships should use a 1-to-many relationship to a general document table. This means users have a column which contains some int or string (hash) identifier corresponding to a document which is located in a separate table in a separate file.

Technically for now since this will be 100% local tho, a document can only ever belong to 1 user; the host user. However since we'll eventually add a cloud hosted option, setting it up to be distributed now will save us time down the line.

…nd matching of scholarships, along with tests

FOOincognita

Left comments in new rag.py file, which shows significant improvement compared to previous versions.

FOOincognita · 2025-10-13T18:30:52Z

-            password = config["password"],
-            host = config["host"],
-            port = config["port"]
+    def embed_pdf(self, pdfFilePath):


This comment addresses the entire file:

This looks significantly better. Here are a few candidates for improvements:

Lazy loading & batch processing - Don't load entire PDFs into memory, & add docs in batches.

This implementation's missing error handling; there are multiple failure points without recovery methods.

Implement token-aware chunking rather than character-based.

Currently can't efficiently filter by user, document type, etc.

This monolithic class does too many things; refactor for separation of concerns.

Implement caching, maybe LRU policy.

Leverage async for non-blocking operations.

Leverage structured logger in /backend/shared/utils/logger.py

I've included an example.py implementation which makes use of each of these improvements for reference (excluding the shared logger). Note that it's not meant to replace yours; it's just a handy reference.

…hing is async now :), will have to centralize from core/rag.py to server.py later once other smaller auxiliary services are made, after vectordbservice is finished, and then subsequently cut into several pieces

FOOincognita

Looking good hoss. For those merge conflicts, many can be resolved by running a git pull origin main, which will update your .gitignore, which resolves the conflicts with **/generated directories + the swagger docs in docs/openapi.

FOOincognita · 2025-10-16T01:53:44Z

+    #document embedding services
+    async def EmbedScholarshipBatch(self, request: rag_engine_pb2.EmbedScholarshipBatchRequest,
+                                      context: grpc.aio.ServicerContext) -> rag_engine_pb2.EmbedScholarshipBatchResponse:
+        print("Handler called!!")


Change print() usage to logs.debug() unless you want output in cout rather than cerr; both will appear in terminal.

FOOincognita · 2025-10-16T02:03:05Z

+service VectorDBService {
+  // Initialize vector DB connection or collection
+  rpc Init(InitRequest) returns (InitResponse);
+
+  // Embed a batch of scholarships and store them
+  rpc EmbedScholarshipBatch(EmbedScholarshipBatchRequest) returns (EmbedScholarshipBatchResponse);
+}
+
+// =========================================
+// VectorDBService - Define RPC method requests and response schemas
+// =========================================
+
+message InitRequest {
+  string connection_string = 1;  // e.g. "postgresql://user:pass@localhost/db"
+  string collection_name = 2;
+}
+
+message InitResponse {
+  bool success = 1;
+  fi.common.v1.Error error = 2;
+}
+
+message EmbedScholarshipBatchRequest {
+  repeated Scholarship scholarships = 1; // Example dataclass converted below
+}
+
+message EmbedScholarshipBatchResponse {
+  int32 total_embedded = 1;
+  fi.common.v1.Error error = 2;
+}
+
+//may change
+message Document {
+  string document_id = 1;
+  string content = 2;
+  google.protobuf.Struct metadata = 3;
+}
+
+message Scholarship {
+  // Unique scholarship identifier
+  int32 id = 1;
+
+  // Scholarship name
+  string name = 2;
+
+  // Scholarship description or summary
+  string description = 3;
+
+  // Minimum GPA required (e.g. 3.5)
+  double min_gpa = 4;
+
+  // Eligible majors (e.g. "Computer Science", "Engineering")
+  repeated string eligible_majors = 5;
+
+  // Citizenship requirement (optional, can be null)
+  string citizenship_required = 6;
+
+  // Location requirement or region (optional)
+  string location = 7;
+
+  // Award amount or range, kept as a string for flexibility
+  string award_amount = 8;
+
+  // Keywords for semantic search / classification
+  repeated string keywords = 9;
+}


Given these message types & service are meant to service database interactions rather than service-to-service communication, we should move it to its own proto file, maybe postgres.proto or something similar. Any other services which speak with the database will share this service to use when communicating with the DB.

FOOincognita · 2025-10-16T02:06:09Z

+class DocumentType:
+    SCHOLARSHIP = "scholarship"
+    RESUME = "resume"
+    ESSAY = "essay"
+    TRANSCRIPT = "transcript"
+    FORM_CONTEXT = "form_context"
+    USER_PROFILE = "user_profile"


May benefit from inheriting from the base enum class:

class DocumentType(Enum):

iirc, you should be able to maintain the associated string values, tho they'd also map to an int, making case matching significantly safer/easier (if you plan on using the definitions for case matching later on)

FOOincognita · 2025-10-16T02:18:30Z

+import signal
 import datetime
+from backend.shared.utils import logs
 from typing import List, Dict, Any, Optional, Tuple


While still valid, much of the Typing library is deprecated since Python 3.9+ excluding type Typing.Any. However, the use of the native type hint object is a valid native replacement for Typing.Any, tho contextually more ambiguous.

Conversions:

Typing.List $\rightarrow$ list

Typing.Dict $\rightarrow$ dict

Typing.Tuple $\rightarrow$ tuple

Typing.Optional $\rightarrow$ <type> | None

Example: Optional[str] $\rightarrow$ str | None

FOOincognita · 2025-10-16T02:19:48Z

+            return rag_engine_pb2.InitResponse(error = e)
+
+    async def __getattr__(self, name):
+        print("gRPC called:", name)


Change print() usage to logs.debug() unless you want output in cout rather than cerr; both will appear in terminal.

IsaacBurns7 and others added 8 commits October 9, 2025 21:51

basic rag, test, docs on rag

3414423

Initial commit for psql-electron-migrations branch

9cfd397

- Updated electron package-lock.json - Removed frontend build output files (out/main/index.js, out/preload/index.js) - Removed frontend/src/features/health/HealthDashboard.tsx - Added backend/services/llm_manager/TODO.md

Update environment variables documentation for PostgreSQL

fc784c2

- Replace SQLite database paths with PostgreSQL connection parameters - Update DATABASE_URL to use postgresql:// connection string format - Update RAG Engine database settings to use PostgreSQL environment variables

PostgreSQL 17-alpine3.22 implemented

cb6be95

IsaacBurns7 requested a review from FOOincognita October 11, 2025 04:24

IsaacBurns7 linked an issue Oct 11, 2025 that may be closed by this pull request

RAG engine scholarship, resume, essay storage #17

Open

IsaacBurns7 self-assigned this Oct 11, 2025

FOOincognita suggested changes Oct 12, 2025

View reviewed changes

RAG now supports embedding scholarships, retrieval of scholarships, a…

a1fe422

…nd matching of scholarships, along with tests

FOOincognita suggested changes Oct 13, 2025

View reviewed changes

EmbedScholarshipBatch done, EmbedPDFLazy next, pytest working, everyt…

db612cd

…hing is async now :), will have to centralize from core/rag.py to server.py later once other smaller auxiliary services are made, after vectordbservice is finished, and then subsequently cut into several pieces

FOOincognita suggested changes Oct 16, 2025

View reviewed changes

FOOincognita added the enhancement New feature or request label Oct 16, 2025

FOOincognita added this to the Sprint 2 milestone Oct 16, 2025

braden-seaborn unassigned IsaacBurns7 Jan 7, 2026

Uh oh!

Conversation

IsaacBurns7 commented Oct 11, 2025

Uh oh!

FOOincognita left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FOOincognita Oct 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FOOincognita Oct 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FOOincognita Oct 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FOOincognita Oct 12, 2025

Choose a reason for hiding this comment

Uh oh!

FOOincognita left a comment

Choose a reason for hiding this comment

Uh oh!

FOOincognita Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

This comment addresses the entire file:

Uh oh!

FOOincognita Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FOOincognita left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FOOincognita Oct 16, 2025

Choose a reason for hiding this comment

Uh oh!

FOOincognita Oct 16, 2025

Choose a reason for hiding this comment

Uh oh!

FOOincognita Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FOOincognita Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

FOOincognita Oct 16, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

FOOincognita left a comment •

edited

Loading

FOOincognita Oct 12, 2025 •

edited

Loading

FOOincognita Oct 12, 2025 •

edited

Loading

FOOincognita Oct 12, 2025 •

edited

Loading

FOOincognita Oct 13, 2025 •

edited

Loading

FOOincognita Oct 13, 2025 •

edited

Loading

FOOincognita left a comment •

edited

Loading

FOOincognita Oct 16, 2025 •

edited

Loading

FOOincognita Oct 16, 2025 •

edited

Loading