Skip to content

[P2-Performance] Implement multi-model routing (cheap parse, expensive reason) #58

Description

@Ztrimus

Problem

Currently, the same expensive model (GPT-4o or Gemini Pro) is used for every LLM call. But not all tasks need the same reasoning power:

Task Reasoning Needed Current Model Ideal Model
Parse JD text → JSON Low GPT-4o ($5/M) GPT-4o-mini ($0.15/M)
Parse resume PDF → JSON Low GPT-4o ($5/M) GPT-4o-mini ($0.15/M)
Keyword extraction Medium GPT-4o ($5/M) GPT-4o-mini ($0.15/M)
Rewrite experience bullets High GPT-4o ($5/M) GPT-4o ($5/M)
Quality reflection High GPT-4o ($5/M) GPT-4o ($5/M)
Cover letter High GPT-4o ($5/M) GPT-4o ($5/M)

Potential savings: ~50-60% on total LLM costs.

Solution

Define a model routing config:

MODEL_ROUTING = {
    "jd_extraction": "gpt-4o-mini",      # Cheap: structured parsing
    "resume_parsing": "gpt-4o-mini",      # Cheap: structured parsing
    "keyword_extraction": "gpt-4o-mini",  # Cheap: classification
    "section_generation": "gpt-4o",       # Expensive: creative rewriting
    "quality_reflection": "gpt-4o",       # Expensive: evaluation
    "cover_letter": "gpt-4o",            # Expensive: creative writing
}

For Gemini equivalent:

  • Cheap: gemini-2.0-flash
  • Expensive: gemini-2.5-pro

Files

  • zlm/variables.py (add model routing config)
  • zlm/__init__.py (use routing in pipeline)

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2-mediumMedium priority - quality & polishenhancementNew feature or requestperformancePerformance improvement

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions