Skip to content

[P2-Performance] Add intermediate result caching for LLM outputs #59

Description

@Ztrimus

Problem

Every pipeline run makes ~8 LLM calls with no caching. If:

  • User tweaks one section and re-runs → all 8 calls fire again
  • Same JD is used for multiple resumes → JD parsing repeats
  • Pipeline crashes mid-way → all progress lost, must restart from scratch

Solution

1. Hash-based file cache for LLM results

import hashlib, json, pathlib

CACHE_DIR = pathlib.Path(".zlm_cache")

def cached_llm_call(prompt: str, schema: type, model: str) -> dict:
    cache_key = hashlib.sha256(
        f"{prompt}:{schema.__name__}:{model}".encode()
    ).hexdigest()[:16]
    
    cache_file = CACHE_DIR / f"{cache_key}.json"
    if cache_file.exists():
        return json.loads(cache_file.read_text())
    
    result = llm_call(prompt, schema, model)
    cache_file.write_text(json.dumps(result))
    return result

2. Cache keys

  • JD extraction: hash(url + model)
  • Resume parsing: hash(resume_content + model)
  • Section generation: hash(section_name + jd_hash + user_data_hash + model)

3. Cache invalidation

  • Manual: --no-cache flag
  • Automatic: cache entries expire after 24h
  • Per-section: re-run only changed sections

Acceptance Criteria

  • JD extraction cached by URL
  • Section outputs cached by content hash
  • --no-cache CLI flag to bypass
  • Cache stats shown in output (hits/misses)

Files

  • zlm/utils/cache.py (new)
  • zlm/__init__.py (integrate caching)
  • main.py (add --no-cache flag)

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2-mediumMedium priority - quality & polishenhancementNew feature or requestperformancePerformance improvement

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions