Add TTL Caching to the Setup Guide Handler (Stop Calling the LLM on Every Visit)

[Vedant1703]

📋 Description

Every time a user visits /repo/[owner]/[repo]/setup, the frontend calls the Core Service GET /repo/setup-guide?repo=owner/repo&user_id=.... The handler in setup_guide.go always:

1. Fetches the repo's README from the GitHub API
2. Makes a full LLM call to the AI service to generate the guide

There is zero caching. This means:

• A user who clicks "Setup Guide" and then navigates away and comes back triggers two full LLM calls
• Multiple users with the same experience level viewing the same popular repo each trigger individual LLM calls
• LLM API calls cost money and have their own rate limits — this design wastes both

The fix is to build a SetupGuideCache that caches responses keyed by (owner, repo, experience_level) with a TTL. Since README content changes rarely, a TTL of several hours is appropriate.

This is harder than the generic cache cleanup issue (#18 ) because:

• The cache key must combine 3 fields
• The cached value is a complex JSON struct (SetupGuide), not a simple string
• You must integrate it cleanly into an existing HTTP handler without breaking the existing code path

---

📍 Files to Create / Modify

• Create: backend/core_service/internal/cache/setup_guide_cache.go
• Modify: backend/core_service/internal/handlers/setup_guide.go — check cache before calling AI, write to cache after

---

🔍 Current Handler (Always Calls AI)

// setup_guide.go
func (h *SetupGuideHandler) GetSetupGuide(w http.ResponseWriter, r *http.Request) {
    // ... parameter validation ...
    readme, _ := h.githubClient.FetchReadme(ctx, owner, repoName, token)  // GitHub call
    guide, _ := h.aiClient.GetSetupGuide(ctx, readme, user.ExperienceLevel)  // LLM call ← always happens
    json.NewEncoder(w).Encode(map[string]string{"guide": guide})
}

---

✅ What To Build

Step 1 — Create backend/core_service/internal/cache/setup_guide_cache.go:

package cache

import (
    "fmt"
    "sync"
    "time"
)

type SetupGuideCacheEntry struct {
    GuideJSON  string    // The raw JSON string from the AI
    CachedAt   time.Time
}

type SetupGuideCache struct {
    mu      sync.RWMutex
    entries map[string]SetupGuideCacheEntry
    ttl     time.Duration
}

func NewSetupGuideCache(ttl time.Duration) *SetupGuideCache {
    c := &SetupGuideCache{
        entries: make(map[string]SetupGuideCacheEntry),
        ttl:     ttl,
    }
    // Periodically clean up expired entries
    go c.startCleanup(ttl * 2)
    return c
}

// Key combines owner, repo, and experience level to ensure per-user-type caching
func (c *SetupGuideCache) cacheKey(owner, repo, experienceLevel string) string {
    return fmt.Sprintf("%s/%s::%s", owner, repo, experienceLevel)
}

func (c *SetupGuideCache) Get(owner, repo, experienceLevel string) (string, bool) {
    key := c.cacheKey(owner, repo, experienceLevel)
    
    c.mu.RLock()
    entry, ok := c.entries[key]
    c.mu.RUnlock()

    if !ok || time.Since(entry.CachedAt) > c.ttl {
        return "", false
    }
    return entry.GuideJSON, true
}

func (c *SetupGuideCache) Set(owner, repo, experienceLevel, guideJSON string) {
    key := c.cacheKey(owner, repo, experienceLevel)
    
    c.mu.Lock()
    c.entries[key] = SetupGuideCacheEntry{
        GuideJSON: guideJSON,
        CachedAt:  time.Now(),
    }
    c.mu.Unlock()
}

func (c *SetupGuideCache) startCleanup(interval time.Duration) {
    ticker := time.NewTicker(interval)
    defer ticker.Stop()
    for range ticker.C {
        c.mu.Lock()
        now := time.Now()
        for k, v := range c.entries {
            if now.Sub(v.CachedAt) > c.ttl {
                delete(c.entries, k)
            }
        }
        c.mu.Unlock()
    }
}

Step 2 — Add cache to SetupGuideHandler struct and constructor:

// In setup_guide.go:
type SetupGuideHandler struct {
    githubClient *clients.GitHubClient
    aiClient     *clients.AIClient
    userRepo     *users.Repository
    cache        *cache.SetupGuideCache    // ← add this
}

func NewSetupGuideHandler(gh *clients.GitHubClient, ai *clients.AIClient, ur *users.Repository, c *cache.SetupGuideCache) *SetupGuideHandler {
    return &SetupGuideHandler{githubClient: gh, aiClient: ai, userRepo: ur, cache: c}
}

Step 3 — Check cache before calling AI, populate cache after:

func (h *SetupGuideHandler) GetSetupGuide(w http.ResponseWriter, r *http.Request) {
    // ... existing parameter validation ...

    // ─── Cache Check ────────────────────────────────────
    if h.cache != nil {
        if cached, ok := h.cache.Get(owner, repoName, user.ExperienceLevel); ok {
            log.Printf("SetupGuide: cache HIT for %s/%s (%s)", owner, repoName, user.ExperienceLevel)
            w.Header().Set("Content-Type", "application/json")
            w.Header().Set("X-Cache", "HIT")  // helpful for debugging
            w.Write([]byte(`{"guide":` + cached + `}`))
            return
        }
        log.Printf("SetupGuide: cache MISS for %s/%s (%s)", owner, repoName, user.ExperienceLevel)
    }

    // ─── Existing logic (fetch readme + call AI) ─────────
    readme, err := h.githubClient.FetchReadme(r.Context(), owner, repoName, user.GitHubToken)
    if err != nil {
        http.Error(w, "failed to fetch readme: "+err.Error(), http.StatusInternalServerError)
        return
    }

    guide, err := h.aiClient.GetSetupGuide(r.Context(), readme, user.ExperienceLevel)
    if err != nil {
        http.Error(w, "failed to generate guide: "+err.Error(), http.StatusInternalServerError)
        return
    }

    // ─── Store in cache ───────────────────────────────────
    if h.cache != nil {
        h.cache.Set(owner, repoName, user.ExperienceLevel, guide)
    }

    w.Header().Set("Content-Type", "application/json")
    w.Header().Set("X-Cache", "MISS")
    json.NewEncoder(w).Encode(map[string]string{"guide": guide})
}

Step 4 — Wire it up in main.go or routes.go:

guideCache := cache.NewSetupGuideCache(4 * time.Hour)  // cached for 4 hours
setupHandler := handlers.NewSetupGuideHandler(githubClient, aiClient, userRepo, guideCache)

---

🏁 Acceptance Criteria

• A SetupGuideCache with Get, Set, and background cleanup is implemented
• Cache key includes owner, repo, and experience_level — two users with different levels get different cached responses
• On a cache HIT, the handler returns immediately without calling the AI service
• The response includes an X-Cache: HIT or X-Cache: MISS header (useful for observability)
• The TTL is configurable — passed to NewSetupGuideCache() as a parameter, not hardcoded
• Background cleanup goroutine removes expired entries to prevent unbounded memory growth
• All code compiles: cd backend/core_service && go build ./...
• Verify manually: hit the setup guide endpoint twice for the same repo, confirm the second response is significantly faster (< 10ms vs potentially seconds)

---

💡 Technical Hints

• The AI service returns a guide as a JSON string (look at AIClient.GetSetupGuide return type). Cache the raw JSON string — don't unmarshal and re-marshal it, that wastes CPU on every cache write/read
• experienceLevel is part of the cache key because a "beginner" and an "intermediate" user should get differently-worded guides for the same repo
• The handler currently updates NewSetupGuideHandler call — find where it's instantiated (likely main.go or routes/) and update that call to pass the cache

---

🚀 Getting Started

1. Fork the repository
2. Create a branch: git checkout -b feat/issue-32-setup-guide-cache
3. Create backend/core_service/internal/cache/setup_guide_cache.go
4. Update setup_guide.go handler
5. Update constructor call in main.go/routes/
6. Test: curl the same endpoint twice, 2nd response should be nearly instant
7. Open a PR with curl output or logs showing MISS then HIT!