Skip to content

Use haiku instead of gpt-4o for more stable performance #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
wenatuhs merged 1 commit into from
Jan 30, 2026
Merged

Use haiku instead of gpt-4o for more stable performance #2

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 58 additions & 29 deletions config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,40 +23,69 @@ registry_path: ./src/otter/registry.py
# Common providers: cborg, openai, anthropic, google, ollama
# Budget tip: Use anthropic/claude-haiku via cborg for most roles

# models:
# orchestrator:
# provider: stanford
# model_id: claude-3-7-sonnet # Replaces cborg's anthropic/claude-sonnet
# response:
# provider: stanford
# model_id: gpt-4o # Replaces cborg's google/gemini-flash
# max_tokens: 5000
# classifier:
# provider: stanford
# model_id: gpt-4.omini # Replaces ollama's mistral:7b
# approval:
# provider: stanford
# model_id: gpt-4.omini # Replaces ollama's mistral:7b
# task_extraction:
# provider: stanford
# model_id: gpt-4o # Replaces cborg's google/gemini-flash
# max_tokens: 1024
# memory:
# provider: stanford
# model_id: gpt-4o # Replaces cborg's google/gemini-flash
# max_tokens: 256
# python_code_generator:
# provider: stanford
# model_id: claude-3-7-sonnet # Replaces cborg's anthropic/claude-haiku
# max_tokens: 4096
# time_parsing:
# provider: stanford
# model_id: gpt-4.omini # Replaces ollama's mistral:7b
# max_tokens: 512
# filter_extraction:
# provider: stanford
# model_id: gpt-4.omini # Fast model for extracting run query filters
# max_tokens: 1024 # More tokens than time_parsing for complex filter combinations

models:
orchestrator:
provider: stanford
model_id: claude-3-7-sonnet # Replaces cborg's anthropic/claude-sonnet
provider: cborg
model_id: anthropic/claude-haiku
response:
provider: stanford
model_id: gpt-4o # Replaces cborg's google/gemini-flash
max_tokens: 5000
provider: cborg
model_id: anthropic/claude-haiku
classifier:
provider: stanford
model_id: gpt-4.omini # Replaces ollama's mistral:7b
provider: cborg
model_id: anthropic/claude-haiku
approval:
provider: stanford
model_id: gpt-4.omini # Replaces ollama's mistral:7b
provider: cborg
model_id: anthropic/claude-haiku
task_extraction:
provider: stanford
model_id: gpt-4o # Replaces cborg's google/gemini-flash
max_tokens: 1024
provider: cborg
model_id: anthropic/claude-haiku
memory:
provider: stanford
model_id: gpt-4o # Replaces cborg's google/gemini-flash
max_tokens: 256
provider: cborg
model_id: anthropic/claude-haiku
python_code_generator:
provider: stanford
model_id: claude-3-7-sonnet # Replaces cborg's anthropic/claude-haiku
max_tokens: 4096
provider: cborg
model_id: anthropic/claude-haiku
time_parsing:
provider: stanford
model_id: gpt-4.omini # Replaces ollama's mistral:7b
max_tokens: 512
provider: cborg
model_id: anthropic/claude-haiku
filter_extraction:
provider: stanford
model_id: gpt-4.omini # Fast model for extracting run query filters
max_tokens: 1024 # More tokens than time_parsing for complex filter combinations
provider: cborg
model_id: anthropic/claude-haiku

# ============================================================
# SERVICE CONFIGURATION
Expand Down Expand Up @@ -225,7 +254,7 @@ development:
prompts:
show_all: false # Print prompts to console
print_all: true # Save prompts to files
latest_only: true # Keep only latest prompt version
latest_only: false # Keep only latest prompt version

# ============================================================
# LOGGING
Expand Down Expand Up @@ -282,10 +311,10 @@ api:
api_key: ${STANFORD_API_KEY}
base_url: https://aiapi-prod.stanford.edu/v1
timeout: 30
# cborg:
# api_key: ${CBORG_API_KEY}
# base_url: https://api.cborg.lbl.gov/v1
# timeout: 30
cborg:
api_key: ${CBORG_API_KEY}
base_url: https://api.cborg.lbl.gov/v1
timeout: 30
# openai:
# api_key: ${OPENAI_API_KEY}
# base_url: https://api.openai.com/v1
Expand Down