feat(db): bound the connection pool with configurable sizing + pre-ping - #49
Merged
Conversation
app/database.py created the async engine with SQLAlchemy's defaults (pool_size=5, max_overflow=10 = up to 15 connections/instance) and no liveness check. On Cloud Run, which scales horizontally, total DB connections = per-instance pool × running instances, so an unbounded instance count can saturate Cloud SQL's max_connections — the morning-outage failure mode from the 2026-06-01 live event (aoe2-live-standings-api#171/#172). Idle connections dropped by Cloud SQL also surfaced as request errors without pre-ping. Make pool_size, max_overflow, pool_timeout, pool_recycle, and pool_pre_ping configurable via settings with conservative defaults (10 connections/instance, pre-ping on, 30-min recycle). Apply them to Postgres only — SQLite (dev/test) uses a StaticPool that rejects these args. Document the sizing formula vs. Cloud SQL max_connections in .env.example. Follow-up to #47.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #47 (the infra hardening it flagged from
aoe2-live-standings-api's 2026-06-01 launch incident). Mirrors aoe2#172.Problem
app/database.pybuilt the async engine with SQLAlchemy's defaults —pool_size=5, max_overflow=10= up to 15 connections per instance — and nopool_pre_ping/pool_recycle. On Cloud Run (horizontal scaling) total DB connections = per-instance pool × running instances, and with no instance cap that can saturate Cloud SQL'smax_connections(the morning connection-exhaustion outage in the referenced incident). Separately, Cloud SQL silently dropping idle connections surfaces as intermittent request errors withoutpool_pre_ping.Change
Make the pool configurable via settings, with conservative defaults:
DB_POOL_SIZEDB_MAX_OVERFLOWDB_POOL_TIMEOUTDB_POOL_RECYCLEDB_POOL_PRE_PINGApplied to Postgres only — SQLite (dev/test) uses a
StaticPoolthat rejects these args, so_pool_kwargs()returns{}for sqlite URLs.The sizing formula is documented in
.env.example:Defaults (10/instance) fit a 200-connection tier at
--max-instances=10(added in the companion deploy PR). Pairs with that instance cap to bound worst-case connections.Test plan
uv run pytest— 134 passed, 1 skippeduv run ruff check .+ruff format --check— cleantests/test_db_pool.py:_pool_kwargsempty for sqlite / populated for postgres; a throwaway asyncpg engine reflects the configuredpool.size(); a tripwire test that defaults stay ≤ 10/instance with pre-ping on.