docs(memory): budgets size from the process cgroup limit, not only a container limit - #2033
Merged
Merged
Conversation
…nly a container limit spiceai/spiceai#12263 replaces the cgroup-root probe with a walk of the process's own cgroup path (leaf -> mountpoint root), taking the smallest limit at any level. A limit set by `systemd-run -p MemoryMax=`, a capped parent slice, or a Kubernetes pod cgroup now sizes the DataFusion query pool default and the DuckDB coordinated budget, where previously only a cgroup-namespaced container limit was visible. vNext only — not in v2.1.2.
Contributor
✅ Pull with Spice PassedPassing checks:
|
Contributor
|
🚀 deployed to https://45bdb7c7.spiceai-org-website.pages.dev |
Collaborator
Author
|
@copilot review |
lukekim
approved these changes
Aug 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The docs described Spice's memory-budget sizing as container-aware / "the cgroup limit in a container". That was accurate for the old probe, which read the mountpoint root
/sys/fs/cgroup/memory.max— a path that holds the limit only inside a cgroup-namespaced container.spiceai/spiceai#12263 replaces it with
telemetry::hardware::cgroup_memory_limit(), which walks the process's own cgroup path leaf → mountpoint root and takes the minimum limit found at any level (cgroup v2memory.max, cgroup v1memory.limit_in_bytes). A cap set bysystemd-run -p MemoryMax=…, a capped parent slice, or a Kubernetes pod cgroup now binds sizing exactly like a container limit does; previously every one of those was invisible and budgets sized from full host RAM (an OOM kill within 25 minutes on a 121 GiB host with a 96 GiBMemoryMax, per the source PR).runtime::resource_monitor::get_total_memory()delegates to it, so this is the figure behind both theruntime.query.memory_limitdefault and the DuckDB coordinated memory budget.get_host_memory()is deliberately unchanged — DuckDB's own default still sizes from host RAM, which is why the coordinated budget must project that ceiling from the host figure; the pages keep saying so.Also notes the new startup INFO log when a cap binds (
Memory budgets sized from the cgroup memory limit: … (host total: …),crates/runtime/src/resource_monitor.rs:110).Source PRs
Doc pages changed (5) — all vNext only
reference/spicepod/runtime.md—runtime.query.memory_limitdefaultreference/memory.md— DataFusion query memory management (2 spots)reference/performance-tuning.md— Memory Limitcomponents/data-accelerators/duckdb/index.md— Coordinated memory budgetcomponents/data-accelerators/duckdb/deployment.md— MemoryCross-page sweep:
grep -rn 'container-aware\|container memory limit\|container limit' website/docs/→ 0 remaining hits. Scoped to vNext because the old container-only probe is what v2.1.2 and earlier actually ship, so the versioned snapshots are correct as written.Test plan
cd website && npm run buildpassestrunkonly, not in v2.1.2