rag-pipeline-utils

Composable RAG for Node.js — with built-in evaluation, citations, guardrails, and observability.

Start small with three core primitives — pipeline, plugin, connector — and opt in to evaluation, citations, guardrails, agentic reasoning, and GraphRAG as you need them.

Installation

npm install @devilsdev/rag-pipeline-utils

Quick Start

import {
  createRagPipeline,
  OpenAIConnector,
  MemoryRetriever,
} from "@devilsdev/rag-pipeline-utils";

const pipeline = createRagPipeline({
  retriever: new MemoryRetriever(),
  llm: new OpenAIConnector({ apiKey: process.env.OPENAI_API_KEY }),
});

const result = await pipeline.run({
  query: "What is the vacation policy?",
  options: { citations: true, evaluate: true },
});

console.log(result.results); // retrieved documents
console.log(result.citations.groundednessScore); // 0.85
console.log(result.evaluation.scores); // { faithfulness, relevance, ... }

Requirements: Node.js ≥ 18 · ESM or CommonJS

Who This Is For

Teams building production Node.js RAG services who need:

• Grounded outputs they can trace back to sources
• Modular architecture that doesn't lock them into a single provider
• Operational discipline — tracing, metrics, cost controls, guardrails
• A stable foundation with clear plugin contracts that outlast any vendor

If you're looking for a framework you can grow into — start with retrieval, add evaluation when it matters, add guardrails when you ship to production — this is built for you.

Use Cases

Document Q&A with citations

const result = await pipeline.run({
  query: "Which policy covers remote work?",
  options: { citations: true },
});
// result.citations maps each answer sentence to its source documents

Internal knowledge assistant with evaluation

const result = await pipeline.run({
  query: userQuestion,
  options: { evaluate: true },
});
// result.evaluation.scores = { faithfulness, relevance, contextPrecision, ... }
// Alert when faithfulness drops below threshold in production

Enterprise service with guardrails

import { GuardrailsPipeline, createRagPipeline } from "@devilsdev/rag-pipeline-utils";

const safePipeline = new GuardrailsPipeline(createRagPipeline({ ... }), {
  preRetrieval:  { enableInjectionDetection: true },
  retrieval:     { minRelevanceScore: 0.6 },
  postGeneration:{ enablePIIDetection: true, enableGroundednessCheck: true },
});

Architecture

Ingestion:  Documents → Chunking → Embedder → Vector Store

Query:      User Query
              ↓
            Guardrails      → prompt injection, topic filtering
              ↓
            Query Planner   → decomposes complex queries
              ↓
            Hybrid Retriever → vector + BM25 with Reciprocal Rank Fusion
              ↓
            Reranker        → BM25 / embedding / cascade
              ↓
            LLM             → generates answer from context
              ↓
            Citation Tracker → maps sentences to sources
              ↓
            Evaluator       → scores faithfulness & groundedness
              ↓
            Response { answer, citations, evaluation }

Each stage is optional, pluggable, and observable.

Capabilities

Opt in only to what you need:

Capability	What you get
Chunking	5 strategies: sentence, fixed-size, recursive, semantic, structure-aware
Retrieval	Hybrid vector + BM25 with Reciprocal Rank Fusion
Reranking	LLM, BM25 scoring, embedding similarity, or cascade
Evaluation	Faithfulness, relevance, context precision/recall, groundedness
Citation	Per-sentence source attribution, hallucination detection
Agentic	Query planning, iterative retrieval, self-critique
Guardrails	Injection detection, PII filtering, ACL-aware access control
GraphRAG	Knowledge graph construction with entity extraction
Streaming	SSE and WebSocket adapters with backpressure control
Cost	Token tracking, budget enforcement, provider pricing
Debugging	Execution tracing, bottleneck detection
MCP	Expose pipelines as Model Context Protocol tools
Enterprise	Multi-tenancy, SSO (SAML/OAuth2/AD/OIDC), audit logs

See full documentation for complete API reference.

Provider Connectors

Built-in connectors for popular providers — or implement the contract for your own:

import {
  OpenAIConnector, // GPT-4, text-embedding-3
  AnthropicConnector, // Claude 3 Opus, Sonnet, Haiku
  CohereConnector, // Embed + Rerank
  OllamaConnector, // Llama 3, Mistral (local, offline)
  LocalEmbedder, // TF-IDF (offline, no API)
  MemoryRetriever, // In-memory cosine similarity
} from "@devilsdev/rag-pipeline-utils";

Documentation

• Documentation site — guides, API reference, deployment
• Architecture — plugin contracts, pipeline internals
• API reference — complete export index
• Examples — chatbot, Q&A, code search, support
• Comparison vs LangChain.js / LlamaIndex.TS — honest decision framework
• Runnable demo — Fastify + OpenAI + guardrails, deployable in one command
• Benchmarks & methodology — how we measure, how you reproduce
• Security policy — responsible disclosure, supported versions
• Security capability matrix — what's battle-tested vs. recommended vs. example
• API stability policy (SEMVER.md) — what counts as breaking, deprecation windows
• Changelog — release history

Roadmap

Current (v2.4): chunking, citation, evaluation, agentic RAG, hybrid retrieval, 3-layer guardrails, GraphRAG, streaming, cost management, MCP integration, 7 provider connectors.

Next (v3.0): intelligent caching, native Rust bindings, Kubernetes operator, edge deployment.

Vote on features in GitHub Discussions.

Community

• GitHub Discussions — ask questions, share use cases
• Issues — report bugs, request features
• Contributing guide

License

MIT — see LICENSE.