Skip to content

feat: bump default Gemini model to 3.5-flash#324

Merged
luis5tb merged 3 commits into
RHEcosystemAppEng:mainfrom
luis5tb:worktree-gemini-model
Jul 10, 2026
Merged

feat: bump default Gemini model to 3.5-flash#324
luis5tb merged 3 commits into
RHEcosystemAppEng:mainfrom
luis5tb:worktree-gemini-model

Conversation

@luis5tb

@luis5tb luis5tb commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Bumps the default Gemini model from gemini-2.5-flash to gemini-3.5-flash (GA, released May 2026)
  • Updates config, all deployment manifests (Cloud Run, OpenShift, Podman), and documentation (13 files total)
  • Gemini 3.5 Flash offers significantly better agentic performance (83.6% MCP Atlas, beats 3.1 Pro on coding benchmarks) at ~4x faster output speed

Test plan

  • All 466 tests pass
  • Lint (ruff check) clean
  • Type check (mypy) clean
  • Verify model availability in target Vertex AI project
  • Smoke test with the new model in a dev environment

🤖 Generated with Claude Code

@yuvalk

yuvalk commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator
  1. No test locks in the new default — the PR changed Settings.gemini_model's default but added no test for it, unlike the existing
    test_default_provider_is_gemini pattern. Fixed in 76f213e by adding test_default_gemini_model.
  2. Cost impact undocumented — gemini-3.5-flash is priced substantially higher per token than gemini-2.5-flash (confirmed via Google's own docs: ~$1.50/M
    input, $9/M output). Since this agent bills usage through Google Marketplace metering, silently defaulting every deployment to a pricier model deserves a
    callout. Fixed in 8d80071 by adding cost notes to .env.example and docs/configuration.md.
  3. Stale mechanical find-replace — deploy/cloudrun/README.md's TPM-quota tuning tip was relabeled from gemini-2.5-flash to gemini-3.5-flash via straight text
    substitution without re-verifying the quota claim for the new model. Fixed in ae629af by rephrasing it to reference whatever model is actually configured,
    so it won't silently go stale on the next bump.

(claude)

@luis5tb luis5tb force-pushed the worktree-gemini-model branch from ae629af to d0e2ecf Compare July 10, 2026 13:09
luis5tb and others added 3 commits July 10, 2026 15:13
Gemini 3.5 Flash (GA, May 2026) offers significantly better agentic
and coding performance — 83.6% on MCP Atlas (vs 62% for 3 Flash),
beats 3.1 Pro on Terminal-Bench and Finance Agent benchmarks, and
runs ~4x faster than other frontier models.

Updated across config, deployment manifests (Cloud Run, OpenShift,
Podman), and documentation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The default bump to gemini-3.5-flash had no test asserting it, unlike
llm_provider's default which is covered by test_default_provider_is_gemini.
Without this, a future accidental revert of the default would go unnoticed.
The tip was mechanically relabeled from gemini-2.5-flash to
gemini-3.5-flash without re-verifying the TPM-quota claim for the new
model. Phrase it in terms of whatever model is actually configured so
it doesn't silently go stale on the next model bump.
@luis5tb luis5tb force-pushed the worktree-gemini-model branch from d0e2ecf to 3e037da Compare July 10, 2026 13:14

@yuvalk yuvalk left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with or even without my fixes

@luis5tb luis5tb merged commit a9c2b4a into RHEcosystemAppEng:main Jul 10, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants