A fully custom RAG pipeline with a source-highlighting proxy, built from scratch. No frameworks, no wrappers, no vibe-coded shortcuts.
Insight Engine lets you ingest any public web page and ask questions about it. The system retrieves the most relevant content, generates a grounded answer with citations, and lets you verify every answer against the original source with one click.
https://leelinkoff.com/mvps/rag/
- Fetches any public URL and extracts readable text using Axios and Cheerio
- Cleans and normalizes content, then splits it into semantically meaningful chunks
- Each chunk is embedded using OpenAI's text-embedding-3-small model and stored in an in-memory vector store
- User questions are embedded and compared against all stored chunks using cosine similarity
- The top matching chunks are assembled into a grounded context block
- GPT-4o-mini generates a concise answer strictly from that context, with bracket citations
- If the answer is not supported by the retrieved content, the system says so explicitly rather than hallucinating
RAG systems fail in specific, known ways beyond simple hallucination. Insight Engine checks for the main ones explicitly rather than trusting the model's output by default:
- Every generated answer is run through OpenAI's moderation endpoint before being returned; flagged answers are withheld rather than shown
- Responses are flagged when an answer is grounded in a single source only, since one skewed or outdated document could otherwise appear as broadly confirmed
- A standalone eval harness checks refusal correctness, citation correctness, and retrieval precision against a fixed test set, and can gate CI
This is the most innovative feature and solves a real trust problem with RAG systems.
Standard RAG products return an answer and a source URL. The user has no way to verify whether the answer is grounded without manually searching the original page.
Insight Engine solves this with a server-side highlight proxy. After receiving an answer, the user can open any cited source in a proxy view that fetches the original HTML, injects a client-side token-matching script, and scrolls the browser directly to the relevant passage on the page. Verification becomes a one-click operation instead of a manual ctrl+F exercise.
The React frontend is decomposed into focused, single-responsibility components rather than one large file. App.jsx owns all shared state (the target URL, query state, answer/source/safety results) and the three functions that talk to the backend (ingestUrl, getHealth, runQuery); every visual piece is a component that receives only the state and callbacks it needs as props, with no duplicated logic and no component reaching back up into a parent.
App.jsx
├── AboutCard.jsx
├── PageUrlCard.jsx
├── RoutingErrorNotice.jsx (shown if VITE_API_BASE is misconfigured, catching a whole class of broken-URL bugs)
├── HighlighterTab.jsx
├── AskQuestionTab.jsx
│ ├── AnswerCard.jsx (renders the safety and source-diversity warnings)
│ ├── SourcesCard.jsx
│ └── HighlightedPreviewCard.jsx
├── SystemCheckDialog.jsx
└── HowItWorksDialog.jsx
├── InstructionSection.jsx
└── InstructionStep.jsx
Tabs and dialogs are built with Radix UI primitives (@radix-ui/react-tabs, @radix-ui/react-dialog) for accessible keyboard navigation and focus handling, with the visual styling layered on top in App.css.
An interactive Swagger UI, generated from a static OpenAPI 3.0 spec (swagger-spec.js), is served at /api/docs. It documents every endpoint's request/response shape with realistic examples, and its top-level description surfaces the safety/bias guardrails up front rather than leaving them buried in a single endpoint's details.
- Full request lifecycle logging with timestamps
- 120-second watchdog with clean error response on timeout
- Token-safe chunk truncation to stay within OpenAI embedding limits
- Restart-safe Docker containerization with environment variable isolation
- Apache reverse proxy for clean routing, no CORS issues, and no public port exposure
See DEPLOYMENT_AND_ARCHITECTURE.md for full directory layout and VPS path details.
Request flow:
- User submits a URL. Backend fetches, parses, chunks, and embeds the content. Vectors stored in memory.
- User submits a question. Backend embeds the question, runs cosine similarity against stored vectors, selects top chunks.
- Top chunks plus the question are sent to GPT-4o-mini. Answer returned with bracket citations.
- User clicks a citation. Backend proxy fetches the source page, injects highlight script, scrolls to the relevant passage.
The OpenAI chat completion API is stateless. It has no knowledge of your content. Every request starts from zero. The only option the raw API gives you is to paste content directly into the prompt on every request. That breaks down immediately: pages exceed the context window, costs scale with content size, and accuracy degrades when GPT has to sift through large blocks of irrelevant text to find the answer.
The vector store solves this by separating two distinct problems, relevancy and reasoning.
At ingest time, the backend fetches and parses the target page, splits it into chunks, and sends each chunk to OpenAI's embedding model. The model returns a vector (an array of numbers representing the semantic meaning of that chunk). All vectors are stored in memory alongside their source text.
At query time, the user's question is embedded the same way, producing a query vector. The backend runs cosine similarity between the query vector and every stored chunk vector and selects the top matching chunks. Only those chunks are sent to GPT as context.
GPT receives a focused, pre-filtered context window. It does not see the entire page. This produces more accurate answers, lower token usage, and no context window overflow regardless of source content size.
Cosine similarity is used rather than keyword matching because it operates on meaning, not literal text. A question about "pricing" will match a chunk that says "cost per month" even if the word "pricing" never appears in that chunk.
The vector store lives in process memory by design. It is fast and requires no external infrastructure for an MVP. The trade-off is that content must be re-ingested after a container restart. Production hardening would persist embeddings to a dedicated vector database such as Pinecone or pgvector.
Both of these surfaced during real testing, not code review, and are documented here rather than quietly patched over:
- Source ID collision. URLs sharing a long common prefix (e.g. two Wikipedia pages under
/wiki/) were silently assigned identical internal source IDs, because the ID was a truncated base64 encoding of the URL that only captured the first 9 bytes. Two genuinely different ingested pages were merging into one source in the vector store without any error. Fixed by hashing the full URL (SHA-256) instead of truncating a prefix. - Empty-chunk ingestion crash. The chunking function could emit a whitespace-only chunk when a sentence ended right at the chunk-size boundary and was immediately followed by a long run of text with no period in it (common in citation/reference blocks). That empty string was rejected outright by OpenAI's embeddings API, crashing ingestion for the affected page. Fixed by guarding the chunk-flush condition against whitespace-only buffers and filtering any empty chunks that slip through regardless.
- CI (
.github/workflows/ci.yml): on every push/PR, syntax-checks the JavaScript files, type-checks the TypeScript files understrict: true, and runs the eval harness against a live boot of the server (skipped on forked PRs, which never receive the requiredOPENAI_API_KEYsecret). - CD (
.github/workflows/deploy.yml): on push tomain, first verifies the frontend actually builds cleanly on GitHub's own runner (fails fast, touches the VPS not at all), then only if that passes, syncs source to the VPS, rebuilds the backend container, rebuilds the frontend inside a throwaway Docker container on the VPS itself, deploys it to Apache, and verifies/api/healthresponds through the public domain. Deployment used to be a fully manual SSH process (still documented inDEPLOYMENT_AND_ARCHITECTURE.mdfor reference); it's automated now.
Two folders exist locally that are never uploaded to the VPS and aren't part of the deployed application: dev_scripts/ (batch scripts automating local dev, build, Docker, and git/TypeScript diagnostic workflows) and dev_reports/ (the output of those scripts, e.g. ts_check_report-SAFE_TO_DELETE.txt, staged-diffs-SAFE_TO_DELETE.txt). Both are tracked in git for portability across machines, but neither appears anywhere in the VPS deployment steps, consistent with DEPLOYMENT_AND_ARCHITECTURE.md section 1.2's upload table, which only lists frontend/ and backend/.
A secrets/ folder exists at the project root holding openai_key.txt, a plain-text reference copy of the OpenAI API key kept for quick lookup during local development. It is never read by the running application, which pulls the key from environment variables (.env / backend/.env) at runtime. secrets/ is excluded via the root .gitignore and is never committed.
Single root .gitignore by design. Despite the project being split into frontend/ and backend/ folders, a single .gitignore at the repo root covers both, rather than maintaining a separate .gitignore in each. backend/.gitignore and frontend/.gitignore previously existed and were identical to each other and fully redundant against the root file (same node_modules, .env, dist, and log patterns, just duplicated). They were removed in favor of one consolidated file, since per-folder gitignores added no coverage the root file didn't already provide and only created a second place to keep in sync.
| Layer | Technology |
|---|---|
| Frontend | React, Vite, Radix UI (Tabs, Dialog) |
| Backend | Node.js, Express |
| Embeddings | OpenAI text-embedding-3-small |
| Chat completion | OpenAI GPT-4o-mini |
| HTML extraction | Cheerio |
| HTTP client | Axios |
| API documentation | Swagger UI (OpenAPI 3.0) |
| CI/CD | GitHub Actions |
| Containerization | Docker |
| Web server | Apache with mod_proxy |
| Deployment | Bluehost VPS |
Every function, endpoint, and architectural decision is documented inline. The server.js opens with a full architectural overview covering endpoints, model choices, tradeoff analysis on the in-memory vector store, and production hardening guidance. New developers can orient themselves without asking a single question.
Type-checked TypeScript versions of the backend (server.ts, evals.ts, highlight-safe.ts, swagger-spec.ts) are maintained alongside the JavaScript originals, compiling cleanly under strict: true. The frontend follows the same documentation discipline as the backend. Each component file states not just what it renders but why it exists as a separate file (see Modular Frontend Architecture above). Secrets are isolated the same way. A single root-level .gitignore (see Secrets & Git Hygiene below) rather than duplicated per-folder files is a deliberate consistency decision, not an oversight.
See DEPLOYMENT_AND_ARCHITECTURE.md for the full deployment guide including VPS environment constraints, Docker build process, Apache configuration, and security notes.
- Not a ChatGPT wrapper
- Not a tutorial project
- Not vibe-coded
- Not deployed on a clean server with everything pre-installed
This was built locally, then deployed on a VPS with a broken Node environment, missing system libraries, and no native build capability. Every constraint was diagnosed and solved with a real engineering decision.
Lee Linkoff https://leelinkoff.com lee@leelinkoff.com