The private memory your AI plugs into over MCP — memory you can read and edit, not memory that reads you.
Live product: mdenh.dev · Role: Solo founder / Product — strategy, specs, evals, GTM · Status: Private beta, June 2026
A live AI product, built and shipped solo. Source is private (IP); this repo curates the product artifacts — the positioning, the decisions, the evaluation rigor, and the go-to-market — so the thinking is legible even though the code isn't.
I built this to demonstrate one thing: how I take an AI product from a sharp insight to a measurable, defensible, trustworthy outcome — and the judgment calls in between.
| Product muscle | The evidence |
|---|---|
| Positioning under competitive pressure | Killed the original "enhanced markdown reader" framing after a competitor teardown showed the mechanic was commoditizing at every price point, including free. Re-anchored on memory you own and can inspect. → positioning |
| Picking the metric that matters | The beta's primary metric is value/willingness (Sean-Ellis PMF, NPS, willingness-to-pay) and activation, not signups. I optimize for the question that de-risks the business, not the number that looks good. |
| The AI-specific tradeoff: quality vs. cost | The headline is ~81% fewer tokens — but it only counts because answer quality is measured separately and held at the same retrieval depth. Saving tokens by returning worse answers is a regression, not a win. |
| Shipping AI you can trust | A 7-suite eval program (ingestion fidelity → cross-tenant isolation → answer quality via LLM-as-judge) so quality regressions surface before users feel them. → scorecard |
| Intellectual honesty as a feature | Every eval ships an explicit "what this does not prove" section; the more conservative earlier token number is kept on the record, not quietly dropped. Credibility compounds. |
| Decisions written down | Non-trivial bets become ADRs — context, options weighed, consequences. 29 numbered decision records, written before the code. → ADR-0026 |
People want their AI to answer from their own notes, docs, and chat history. The default move — paste the whole library into the prompt — is expensive, hits context limits, and dilutes answer quality with irrelevant text.
The insight: the assistant shouldn't receive your library; it should query it. The Model Context Protocol (MCP) makes that real — Claude, Cursor, and ChatGPT connect directly to a hosted memory and pull only the relevant spans on demand.
That reframes the product's job into two things worth being excellent at:
- Make the memory good — hybrid retrieval, OCR ingestion, compression, reranking, so the right spans come back.
- Make the memory trustworthy — you can read, edit, and delete every item; document content never leaves first-party infrastructure.
The first framing was an enhanced markdown reader. A competitive teardown (Supermemory, Notion's official MCP connector, savemarkdown.co, free OSS vault servers) changed the thesis:
- "Connect your notes to your AI via MCP" is becoming table stakes — the mechanic is occupied at every price point, including free. So the mechanic is not a moat.
- The defensible axis is the bundle plus a philosophy: hosted zero-setup OAuth + hybrid/OCR ingestion + measured token economy + the differentiator — inspectable, curated memory you own, versus Supermemory's invisible, black-box auto-extraction. Every memory in mdenh is a full markdown document you can read, edit, and delete — imported chats and AI-saved notes land as readable documents, not opaque extracted facts.
The call I made: don't try to out-benchmark Supermemory on LongMemEval — that's a compute war I'd lose. Match precision and token economy with first-party components, and win on auditability, privacy, and curation ownership — the axis incumbents structurally won't copy.
What I deliberately deferred (and why): atomic-fact distillation and 768-dim embeddings are real quality levers — but they're Tier-2, gated on whether the eval needle actually moves on real cohort data, not on novelty. Shipping them now would burn the runway before the value hypothesis is even validated. Sequencing is the product decision.
Full teardown + differentiation matrix → artifacts/positioning.md.
The headline claim is quantified and reproducible, not asserted:
~81% fewer tokens (~5.3×) to answer a question from your library via retrieval vs. pasting the library into the prompt — measured with
gpt-tokenizer(cl100k_baseencoding) at production retrieval depth (k=5), over N=12 grounded-question scenarios.
The reason the win widens as you keep more notes: the MCP path holds flat at ~1,217 tokens per question no matter how large the library grows, while every paste baseline scales with library size.
But the number is meaningless on its own — of course you can send fewer tokens by sending less. The PM discipline is refusing to let the two halves of the tradeoff drift apart:
- Cost (tokens sent) and quality (faithfulness, citation correctness, abstention) are measured on the same retrieval depth, so a "win" on cost can't hide a loss on quality.
- The earlier, more conservative ~69% (~3.2×) figure (equal-Recall@5, from the retrieval eval) is kept on the record as historical context — I'd rather a recruiter trust the smaller number than catch me inflating the bigger one.
The stack earning the number — hybrid search (pgvector cosine + Postgres full-text, fused via Reciprocal Rank Fusion, K=60) → extractive sentence-window compression — runs by default; a first-party cross-encoder reranker (BAAI/bge-reranker-base on Modal) is the designed final stage, currently founder-ops-gated behind RERANK_SERVICE_URL and modeled as an oracle upper bound in the offline evals. The whole path is written up as a decision record with the options I weighed and rejected (third-party rerankers, LLM-based compression, single-arm retrieval): artifacts/ADR-0026-retrieval-architecture.md.
Two implementation honesties worth stating plainly: embeddings are first-party (all-MiniLM-L6-v2, 384-dim, via a Modal embed-service — no document content is sent to a third party to embed), and compression is extractive, not LLM-based — it scores sentences by query-token overlap and returns the tightest relevant window, preserving citation provenance, for ~60–73% per-passage reduction with zero LLM calls and no new sub-processor.
AI products fail quietly — a retrieval tweak that helps one query silently breaks ten others. So quality is a gate, not a vibe. The program is seven suites (E1–E7): E1–E6 are deterministic CI gates that block merges; E7 is a published, non-gating LLM-as-judge credibility report.
Deterministic gates (eval:all, runs in CI) — full scorecard:
| Suite | What it guards | Result |
|---|---|---|
| Ingestion fidelity | Turn count, role attribution, chronological order, coverage, code-block preservation (8 sub-metrics) | ✅ 1.0 across the board |
| Cross-tenant isolation | No query ever carries another owner's ID (5 reader fns, 16 queries, 2 seeded owners) | ✅ 0 leaks |
| Write-scope gating | Read-only / OAuth callers can't reach the write tool | ✅ 100% refused (3/3) |
| Retrieval | Recall@5 0.974 · MRR 0.930 · nDCG@10 0.927 · 68.9% compression efficiency at equal recall† | ✅ |
† Two different numbers, two different baselines — both correct. The 68.9% here is an internal efficiency metric: how much the compression + reranking stack shaves off the already-retrieved passages while holding Recall@5. The ~81% headline is the user-facing claim: retrieval vs. pasting the entire library into the prompt (
gpt-tokenizer, k=5). Pasting the whole library is a far larger baseline than naive retrieval, so the reduction against it is naturally larger. The ~81% number is intentionally the one users see; the more conservative ~69% is kept as the equal-recall reference.
Answer quality — separate, non-gating, LLM-as-judge with the answer model (owl-alpha via OpenRouter) and judge model (Claude Sonnet via the local CLI) deliberately distinct — no self-grading — over a 17-item QA set including 4 planted "not in corpus" probes to test that the assistant abstains instead of hallucinating (it abstained correctly on all 4) — full report:
| Metric | Score | Why it's the right metric |
|---|---|---|
| Faithfulness | 0.965 | Does the answer stay grounded in retrieved text? |
| Citation correctness | 0.988 | Can the user verify every claim? |
| Abstention accuracy | 0.941 | Does it say "I don't know" instead of inventing? |
| Completeness | 0.838 | Did it use everything relevant? (lowest — and tracked as the open weakness) |
The full taxonomy — promise → metric → bar → gate — is in artifacts/eval-taxonomy.md. The lowest score (completeness) is surfaced, not buried: knowing your weakest dimension is the deliverable.
- The token-savings benchmark runs against a synthetic proxy corpus (6,595 tokens / 8 docs / 32 passages,
cl100k), not live user libraries. - The offline harness uses a TF-IDF stand-in for the semantic ranker and models the reranker as an oracle upper bound, so the retrieval numbers are a favorable-but-bounded estimate; a drift test guards the ported
.mjsrunner against silently diverging from the production TypeScript (rrf.ts,compress.ts). - Abstention recall is 0.400 (against precision 1.0): the system almost never false-abstains, but it doesn't yet catch every unanswerable case. This is a real weakness, tracked as an open item — KI-1 stages the E5 answer-retention gate at 0.90 against a 0.95 target (numbers traced in artifacts/answer-quality-report.md).
Privacy is content-blind by construction: document content, query text, filenames, and emails stay first-party and encrypted — none of them ever reach any vendor. Usage-shape analytics (PostHog, US, consent-gated) emit only action types, counts, booleans, and opaque owner IDs; session replay is off. So the honest phrasing is: content stays first-party; usage-shape analytics go to PostHog with consent.
Not "ship it and watch signups." The beta is a fully instrumented funnel designed to answer "is this valuable enough that people would pay?" before spending a dollar on growth.
- Primary metric — value, not volume: Sean-Ellis PMF (target ≥40% "very disappointed" without it), NPS (≥+30), and explicit willingness-to-pay. Signups are a vanity number; these tell me whether there's a business.
- Activation funnel — find the leak before scaling: signup → first MCP search (≥45%), connect → search (≥60%). The "aha" is the assistant answering from your library; if users don't reach it, nothing downstream matters, so this is where I'd spend the next sprint.
- First-run engineered for the aha: a
/betagated front door, a/welcomeactivation checklist + micro-survey, one-click OAuth to Claude (zero config files, zero tokens to copy), D1/D7 lifecycle emails, and a/survey/pmfcapture — every step pruned to shorten time-to-value.
The corpus fills in as you work rather than requiring manual filing (ADR-0027):
- Chat-history import seeds the library from existing ChatGPT/Claude exports.
- A
save_to_libraryMCP write tool lets the assistant grow the library during sessions — a shipped differentiator, guarded by the write-scope gate above so read-only/OAuth callers can't reach it.
Landing — the memory-first pitch & the token wedge
The full landing page (positioning, the three pillars, how it works):
Beta access — qualifying testers by the AI they use and what they'd use memory for
Privacy, in plain language — the "what happens to your content" contract
- Close the activation leak first. Instrument connect → first-search in the live cohort, find the single biggest drop-off step, and remove it — before any growth spend. Acquisition into a leaky funnel is wasted money.
- Validate the flywheel. Chat-history import + the
save_to_librarywrite tool (shipped) should make the library self-populate as users work. Next is measuring whether it actually drives retention, not assuming it does. - Lift the tracked weaknesses. Completeness (0.838) and abstention recall (0.400) are the lowest scores; KI-1 stages the answer-retention gate toward its 0.95 target on real cohort data.
- Earn Tier-2, don't assume it. 768-dim embeddings and atomic-fact distillation only ship if the eval needle moves on real cohort queries — quality investment gated on measured lift, not on roadmap inertia.
| Artifact | What it is |
|---|---|
| positioning.md | Memory-first positioning, competitive teardown, differentiation matrix |
| ADR-0026-retrieval-architecture.md | Decision record for the hybrid-retrieval + compression + reranker stack |
| eval-scorecard.md | Deterministic eval gates (ingestion, isolation, write-scope, retrieval) |
| answer-quality-report.md | LLM-as-judge answer-quality report (faithfulness, citation, abstention) |
| eval-taxonomy.md | The full eval taxonomy: promise → metric → bar → gate |
MCP · Next.js 15 · PostgreSQL (Neon) · Drizzle ORM · pgvector · Postgres FTS · Hybrid search (RRF) · Cross-encoder reranker · all-MiniLM-L6-v2 · Modal · OCR ingestion · OAuth (Stytch) · OpenRouter · LLM-as-judge evals
Built and shipped solo — product strategy, specs, evaluation, and go-to-market. The application source is private to protect the IP; this repository is the curated product thinking behind it.
Christin Thomas · chrispm.dev · github.com/christinthomasdev
Live at mdenh.dev. The decisions, evals, and positioning excerpts here are real build artifacts, lightly curated for a public audience.