Design alignment pass — refactor existing docs & resolve open decisions

[0zzy4]

Design alignment pass — refactor existing docs & resolve open decisions

Goal

Get the whole team using the same source of truth by having everyone independently research the open design questions before next week's meeting. This is a refactor and fill-in-the-gaps pass, not a from-scratch redesign. We already have real decisions and real code; the point is to stop re-deciding those and figure out what's still open — and where the research points to a doc we don't have yet, propose it.

Why

We keep circling back to the same questions mid-sprint because they were never written down anywhere the whole team can point to. master-doc.md still accurately describes the big picture — what's missing is the layer underneath it: architecture specifics, the card schema, endpoint contracts, and a few decisions that got made verbally (Gemini) but never reconciled with what's written (Bedrock).

Repo & current state

Repo: https://github.com/CoderHub101/LingoLM
Open PRs (includes our current draft of endpoints and card types): https://github.com/CoderHub101/LingoLM/pulls

Read the repo before researching anything — docs/master-doc.md, docs/mvp-v1-tech-stack.md, and the open PRs are your starting point. This isn't a "design LingoLM" exercise, it's "here's what we have, here's what's still undecided, go figure out the undecided parts and tell us if anything in the settled parts needs to change."

What's already settled — treat as context, not up for re-debate

• master-doc.md: problem statement, solution, target audience, core use case. The only open thing here is scope (V1 vs. V2), not the vision itself.
• We're using Gemini, not Bedrock (supersedes mvp-v1-tech-stack.md's Bedrock references).
• API endpoints and card types have a real starting point in the open PRs above.
• Core stack: Next.js frontend, Lambda + API Gateway + DynamoDB backend.

What's genuinely open — starting questions, not a complete list

These are meant to kick off the research, not box it in. If working through one of these surfaces a gap — a doc, a decision, a dependency we haven't named — chase it and document how you got there.

• Caching approach and how it fits into the lookup flow
• Where/how Gemini gets called across use cases (base card gen, misspelling handling, nuance chat)
• Final card schema
• Endpoint details for the 3 Lambdas without logic yet (editCard, chatNuance, getAllCards)
• Auth implementation specifics for V1
• UI/UX direction
• User flow diagram (doesn't exist yet)

Suggested research order

Work top-down through architecture-level questions before detail-level ones — the details depend on the architecture, so nailing that first avoids redoing detailed work when an upstream decision changes:

1. Scope (V1 vs. V2)
2. Caching strategy
3. cardLookup order of operations
4. Gemini use-case map
5. Card schema
6. Endpoint contracts (editCard, chatNuance, getAllCards)
7. Auth
8. UI/UX direction (can happen in parallel with the backend chain)
9. User flow diagram (last, since it should reflect the settled flow, not a guess at it)

Guiding principles

• Cost and simplicity should be inputs to every decision, weighed alongside correctness and maintainability — not "always pick the cheapest option," but "know the tradeoff and be able to justify it."
• Stay focused on the actual problem. It's easy to go deep into an interesting technical rabbit hole and lose sight of what we're actually building: a fast, low-friction way to look up a word and save what you learned. If a thread stops connecting back to that, it's probably a V2 tangent — note it and move on.

How to research this

Use Claude, another AI, or your own reading — the tool doesn't matter. Start from the repo so you're reasoning from what we actually have. If using an AI, treat it like a consultant: get its recommendation, then question it rather than accepting it outright.

Starting prompt template (if using an AI):

Refer to our docs in the GitHub repo for information about LingoLM: [link]. Read docs/master-doc.md, docs/mvp-v1-tech-stack.md, and the open PRs for our current endpoints and card schema. Stack: Next.js frontend, AWS Lambda + API Gateway + DynamoDB backend, Gemini as our LLM. This is a student project, so cost and complexity matter, but I want a real analysis, not just "pick the cheapest option."

Help me think through [specific question]. Give me your actual recommendation, not just a list of options. Then I'm going to push back on it, so be ready to defend it or tell me where I'm right.

Things worth pushing on, whatever tool you use:

• Argue the opposite case — ask why the alternative might actually be better, and see if the reasoning holds up.
• Ask what it's assuming about our scale or usage that might not actually be true for us.
• Ask what the weakest part of its own recommendation is.
• If the justification is just "simpler" or "cheaper," push for simpler/cheaper than what, specifically, and what do we give up.
• If it proposes replacing something in the PR, ask what's concretely wrong with the current approach — not just that there's a cleaner pattern.
• Don't accept the first answer as final if something feels off — say so and ask it to reconsider.

Deliverable

Per question or topic explored: list the ideas considered and rejected (with why), the idea you're recommending, your reasoning for it, your confidence level, and any open questions for the group. If your research led you to propose a new doc or decision that wasn't on the original list, include how you got there. This isn't about documenting a back-and-forth with an AI — it's so the team can walk through what was considered and why we landed where we did, and actually remember it later.

Next meeting's outcome

The meeting is for talking through everyone's research and making actual decisions as a team — not for producing final documents on the spot. By the end of the meeting, we should have:

• A decision (or a clearly flagged unresolved item) for each question researched
• Agreement on which new docs are actually needed (beyond the ones already planned)

After the meeting, one person owns turning those decisions into the actual doc updates — master-doc.md (scope section), mvp-v1-tech-stack.md, api-endpoints.md, card schema, user flow diagram, Gemini prompts doc, and any newly-agreed docs — so we're not relying on everyone remembering to update their own piece. Each doc should get a "last updated" date so it's clear it reflects this pass and hasn't quietly gone stale again.

[aritazaman]

Arita's notes: https://docs.google.com/document/d/1Q0caACSHp7Wnhlf1KstYmXKUP9QkagCMOhp3aiVeAKw/edit?usp=sharing

[cboiteux2765]

Clement's notes: https://docs.google.com/document/d/1HimU0roVlL_U5LhJVdRa1c5ICau72Emd9nMqvpgEBFo/edit?usp=sharing

[0zzy4]

LingoLM Architecture: Card Schema, Caching & Search (Chinese MVP)

Scope

• Chinese (Mandarin) only for now, using CC-CEDICT as grounding source
• Spanish planned next, will use a dictionary API instead of CEDICT — grounding step is pluggable per language, not Chinese-specific
• LLM: Gemini (Bedrock is dropped from docs — needs updating there)

Tables (3)

CedictLookup — one-time CC-CEDICT import, keyed by lemma. Static grounding data (pinyin, gloss, POS). Includes a GSI on englishGloss for bidirectional (English→Chinese) search — same table, same import, two query paths. No separate EnglishIndex table needed.

WordCache — global, shared across all users, keyed by lang + lemma. Holds generated card content. Served on cache hit; written to on cache miss.

UserCards — per-user, keyed by userId. Full editable copy of a saved card + baseRef: { lang, lemma, schemaVersion, promptVersion } pointing back to its WordCache origin. Copy is independent once saved — edits never write back to WordCache, and a stale/changed/deleted reference is an expected harmless state, not a bug to guard against.

Card Generation Flow (cardLookup order of operations)

1. Determine input type: Chinese-script input → skip to step 3 with that as the lemma. English input → go to step 2.
2. Resolve English input to a lemma (only runs for English search):
   • Query CedictLookup via the englishGloss GSI
   • 1 candidate → take it, proceed to step 3
   • Multiple candidates → return raw candidates (lemma + pinyin + gloss, no generation yet) to frontend as a picker → wait for user selection → selected lemma proceeds to step 3
   • 0 candidates → Gemini translation fallback (open item, not yet fully decided) → resolved lemma proceeds to step 3
3. Check WordCache for lang + lemma
   • Hit → return cached card immediately, done
   • Miss → continue to step 4
4. Query CedictLookup by lemma for grounding data (pinyin, basic gloss, POS)
5. Call Gemini, passing the CEDICT grounding data, to generate only what CEDICT doesn't provide (examples, related words, etc.)
6. Write the assembled card to WordCache, keyed by lang + lemma
7. Return the card to the user
8. (Separate action, not part of cardLookup) — if the user saves the card, write a full copy + baseRef into UserCards

Note: steps 1–2 only apply to languages with a grounding table that supports search (currently just Chinese/CEDICT). Write step 2 as "resolve via language's grounding source" rather than hardcoding CEDICT, so adding Spanish's dictionary API later doesn't require restructuring this flow.

Schema Notes

Status: Main branch only has a flat VocabularyCard type (frontend-only, no backend schema in code). PR #26/#30's rich BaseCard is unmerged and built for a disconnected architecture (see Folder Organization note below) — not the basis for this schema. The schema below is derived from the actual live WordCache DynamoDB item, cleaned up and extended.

WordCache item (source of truth for a generated card):

{
  lang: string
  lemma: string
  baseCard: {
    partOfSpeech: string
    definitions: [{ text: string, register?: string }]   // register = formality level (formal/neutral/casual) — matters more for zh than en, e.g. 你 vs 您
    examples: [{ src: string, tgt: string }]
    relatedWords: string[]
    collocations: []
    romanization?: string   // numbered pinyin, e.g. "dian4nao3"; absent for Latin-script languages
  },
  schemaVersion: string
  promptVersion: string
  generatedAt: string
  modelId: string
}

UserCards item (personal saved copy — one per word per user, no cardId):

{
  userId: string
  lang: string
  lemma: string
  card: { ...same shape as baseCard, user-editable... }
  notes: string
  tags: string[]
  baseRef: { lang, lemma, schemaVersion, promptVersion }
  createdAt: string
  updatedAt: string
}

Changes from the current live WordCache item: dropped duplicate lemma inside baseCard (redundant with top-level key), dropped pitfalls (unused/undefined), shortDefinition (singular string) → definitions (array, supports multiple senses), added romanization (currently missing from live data despite design work done on it), added versioning metadata (schemaVersion/promptVersion/generatedAt/modelId) so bad generations can eventually be identified/regenerated. Breaking changes accepted — project is still in active development.

Full BaseCard (PR #30) fields intentionally cut: etymology, voiceout, learningAids, aiGrounding, qualityScore — speculative, not in master-doc's feature list, adds Gemini cost/complexity for fields nothing renders.

Folder Organization (flagged, not yet resolved)

• Working Lambda functions currently exist only in the AWS console — not in the repo, not version-controlled. Priority: extract into a new lambdas/ folder at repo root (one subfolder per function, e.g. lambdas/lookup/, lambdas/cards/, lambdas/chat/) as a direct copy-paste capture before any refactor.
• PR LingoLM-9 #30 built its files at root-level app/lib/... and app/app/api/..., structurally disconnected from the actual deployed app at frontend/app/... — did not import from or integrate with existing frontend types/components. Sign that structure needs an explicit team convention (Next.js API routes → frontend/app/api/, actual Lambda code → lambdas/, shared types → one canonical location) before more PRs guess at layout independently.

Performance / Caching Decisions

• DynamoDB (including the GSI query) is fast enough on its own — single-digit-ms reads, no separate caching layer needed in front of CedictLookup. It's static data with cheap queries; caching earns its keep on expensive operations (that's what WordCache already does in front of Gemini), not on lookups that are already fast.
• Rejected: dedicated cache service (Redis/ElastiCache) in front of CedictLookup. Not serverless — runs 24/7 regardless of traffic, needs VPC networking setup, disproportionate cost/complexity for a lookup that's already fast. Skip unless testing actually shows a measured bottleneck tracing back to the DynamoDB query itself.
• Likely real bottleneck if disambiguation feels slow: Lambda cold starts, not the database call. Same delay would exist regardless of which database/cache is queried, so swapping databases wouldn't fix it.
• Rejected (for now): Lambda in-memory (module-level) caching of CedictLookup data. Free and easy to add later, but only helps warm invocations, and DynamoDB reads are already fast enough that the gain is marginal — a later optimization if profiling shows it's worth it, not a v1 concern.
• Accepted: client-side (browser) caching of recent search/disambiguation results. Near-zero cost, no new infrastructure, directly helps the realistic "user searches the same term again shortly after" case. Pure frontend concern — doesn't touch backend architecture.

Considered & Rejected

• Skip Gemini at creation, only use it in chatNuance: rejected — CEDICT alone can't provide examples/related words, so the initial card would fail to meet the "auto-populated" pitch. Leaner initial card (fewer examples) is the accepted compromise instead.
• Generate all candidate cards during disambiguation: rejected — disambiguation only needs raw CEDICT data (lemma/pinyin/gloss) for the picker; card generation happens only after the user picks one lemma.
• Single-table DynamoDB design: rejected — team already moved away from generic PK/SK toward semantic names; separate tables are more readable for a small team even at slight efficiency cost.
• Separate EnglishIndex table: rejected in favor of a GSI on CedictLookup — same data, same import, no sync logic to maintain.
• UserCards as reference-only (no copy): rejected — user edits would drift/break if the source WordCache entry changed. Snapshot + reference (baseRef) gives edit safety and provenance.
• Dedicated cache service (Redis) for lookups: rejected — see Performance section above.

Known Gaps (accepted for MVP, not solved)

• No automatic cache invalidation for bad/outdated WordCache entries — manual fix path only (e.g. admin delete + regenerate)
• Concurrent cache misses on the same word can trigger duplicate Gemini calls — rare enough at current scale to ignore
• No automated check that Gemini's output agrees with CEDICT grounding data

Still Open

• Finalize trimmed BaseCard schema fields as a team (PR #26/LingoLM-9 #30 vs. this doc)
• Rewire PR LingoLM-9 #30's storage layer off in-memory Map onto this 3-table DynamoDB design
• Decide how minimal the creation-time card can be (how many examples = "pre-populated")
• Evaluate whether exact/prefix matching on englishGloss is sufficient, or fuzzy search needs revisiting later