Skip to content

Bump litellm pin to <=1.88.1 and sync cost map to v1.88.1 #84

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
regan-huff merged 1 commit into from
Jun 15, 2026
Merged

Bump litellm pin to <=1.88.1 and sync cost map to v1.88.1 #84

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "agent-eval"
version = "0.1.50"
version = "0.1.51"
description = "Agent evaluation toolkit"
readme = "README.md"
requires-python = ">=3.10"
Expand All @@ -14,7 +14,7 @@ dependencies = [
"inspect-ai==0.3.203",
# pin litellm so that we know what model costs we're using
# see the Development.md doc before changing
"litellm>=1.67.4.post1,<=1.83.13",
"litellm>=1.67.4.post1,<=1.88.1",
"pydantic>=2.0.0",
# For leaderboard
"huggingface_hub",
Expand Down
4 changes: 3 additions & 1 deletion src/agenteval/cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,9 @@ def prep_litellm_cost_map():
# This snippet is mostly lifted from
# https://github.com/BerriAI/litellm/blob/b9621c760d3355e06dd17ec89b9eb6776755392e/litellm/litellm_core_utils/get_model_cost_map.py#L16
# See the Development.md before changing.
desired_model_costs_url = "https://raw.githubusercontent.com/BerriAI/litellm/319193604cfccb1092a17f58ec4ca049115da446/litellm/model_prices_and_context_window_backup.json"
# SHA below is the v1.88.1 release tag of BerriAI/litellm (upper bound of the
# litellm pin in pyproject.toml); keep the two in sync when bumping.
desired_model_costs_url = "https://raw.githubusercontent.com/BerriAI/litellm/9c117a58be8fb8066e833aa92dd98fdf15771e50/litellm/model_prices_and_context_window_backup.json"
response = httpx.get(desired_model_costs_url, timeout=5)
response.raise_for_status()
desired_model_costs = response.json()
Expand Down
Loading