Skip to content

docs(memory): budgets size from the process cgroup limit, not only a container limit - #2033

Merged
lukekim merged 1 commit into
trunkfrom
docs/cgroup-memory-limit-sizing
Aug 1, 2026
Merged

docs(memory): budgets size from the process cgroup limit, not only a container limit#2033
lukekim merged 1 commit into
trunkfrom
docs/cgroup-memory-limit-sizing

Conversation

@claudespice

Copy link
Copy Markdown
Collaborator

Summary

The docs described Spice's memory-budget sizing as container-aware / "the cgroup limit in a container". That was accurate for the old probe, which read the mountpoint root /sys/fs/cgroup/memory.max — a path that holds the limit only inside a cgroup-namespaced container.

spiceai/spiceai#12263 replaces it with telemetry::hardware::cgroup_memory_limit(), which walks the process's own cgroup path leaf → mountpoint root and takes the minimum limit found at any level (cgroup v2 memory.max, cgroup v1 memory.limit_in_bytes). A cap set by systemd-run -p MemoryMax=…, a capped parent slice, or a Kubernetes pod cgroup now binds sizing exactly like a container limit does; previously every one of those was invisible and budgets sized from full host RAM (an OOM kill within 25 minutes on a 121 GiB host with a 96 GiB MemoryMax, per the source PR).

runtime::resource_monitor::get_total_memory() delegates to it, so this is the figure behind both the runtime.query.memory_limit default and the DuckDB coordinated memory budget. get_host_memory() is deliberately unchanged — DuckDB's own default still sizes from host RAM, which is why the coordinated budget must project that ceiling from the host figure; the pages keep saying so.

Also notes the new startup INFO log when a cap binds (Memory budgets sized from the cgroup memory limit: … (host total: …), crates/runtime/src/resource_monitor.rs:110).

Source PRs

Doc pages changed (5) — all vNext only

  • reference/spicepod/runtime.mdruntime.query.memory_limit default
  • reference/memory.md — DataFusion query memory management (2 spots)
  • reference/performance-tuning.md — Memory Limit
  • components/data-accelerators/duckdb/index.md — Coordinated memory budget
  • components/data-accelerators/duckdb/deployment.md — Memory

Cross-page sweep: grep -rn 'container-aware\|container memory limit\|container limit' website/docs/ → 0 remaining hits. Scoped to vNext because the old container-only probe is what v2.1.2 and earlier actually ship, so the versioned snapshots are correct as written.

Test plan

  • cd website && npm run build passes
  • Versioned-docs propagation checked — current-behavior change on trunk only, not in v2.1.2
  • Files updated: 5

…nly a container limit

spiceai/spiceai#12263 replaces the cgroup-root probe with a walk of the
process's own cgroup path (leaf -> mountpoint root), taking the smallest
limit at any level. A limit set by `systemd-run -p MemoryMax=`, a capped
parent slice, or a Kubernetes pod cgroup now sizes the DataFusion query
pool default and the DuckDB coordinated budget, where previously only a
cgroup-namespaced container limit was visible.

vNext only — not in v2.1.2.
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

✅ Pull with Spice Passed

Passing checks:

  • ✅ Title meets minimum length requirement (10 characters)
  • ✅ Has at least one of the required labels: area/blog, area/docs, area/cookbook, dependencies
  • ✅ No banned labels detected
  • ✅ Has at least one assignee: claudespice

@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

🚀 deployed to https://45bdb7c7.spiceai-org-website.pages.dev

@claudespice

Copy link
Copy Markdown
Collaborator Author

@copilot review

@lukekim
lukekim merged commit 1087348 into trunk Aug 1, 2026
11 of 15 checks passed
@lukekim
lukekim deleted the docs/cgroup-memory-limit-sizing branch August 1, 2026 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants