docs: open-world, residency, and graph compute alignment

AGENTS.md

@@ -57,7 +57,9 @@ Build artifacts go to `build-vk-Release/` and are copied to `release/`.
 - **Billboard / flipbook / imposter (engine-native)**: Map entities **`misc_*`** parsed on **`LoadWorld`** (`r_spriteProps 1`, auto-disabled when **`CS_ENGINE_SPRITE_META`** set). Server: **`sv_engineSprites`**, **`sv_engineSpritesSpawn`**, console **`sv_sprite_spawn`**. Client: **`cl_engineSprites`**, **`sprite_spawn`**, cgame trap **`trap_EngineSpriteAddLocal`**. QVM game traps: **`G_ENGINE_SPRITE_SHADER_INDEX`**, **`G_ENGINE_SPRITE_SPAWN`**. Lua: **`Engine.Sprites.spawnLocal` / `spawnServer`**. Demo: **`demo_sprites.cfg`**, **`lua_run demo_run_sprites()`**.
 - **FreeUSD (default ON)**: Git submodule **`src/external/FreeUSD`** ([gopexllc/FreeUSD](https://github.com/gopexllc/FreeUSD)); **`USE_FREEUSD=ON`** in CMake and **`./scripts/compile_engine.sh`** (auto `git submodule update --init`; pass **`nofreeusd`** to disable). Targets link **`freeusd::runtime`** + **`freeusd::c`** via **`idtech3_freeusd`**. Renderer **`r_freeusd` 1**; client **`usd_*`** tools. Init: **`./scripts/init_optional_submodules.sh --freeusd`**. See **`docs/FREEUSD.md`**.
 - **World districts + proxy meshes**: **`r_district` 1** (default) — USD manifest via **`district_load world/playfield.usda`**, FreeUSD snapshot parse, proxy residency (`r_districtProxy`), optional **`cm_districtStream`** sector prefetch. Console: **`district_list`**, **`district_proxy`**, **`district_load_full`**. Demo: **`exec demo_districts.cfg`**. See **`docs/DISTRICTS.md`**.
-- **Infinite open worlds**: **`r_openWorld` 1** — view-driven sector residency: **`cm_stream`** + **`cm_streamMerge` 1** (overlay sector BSP collision), per-chunk **`nav/sector_X_Y.nav`**, **`sprites/sector_X_Y.ents`** billboard scatter. Console: **`openworld_start`**, **`openworld_sector`**, **`openworld_status`**. Demo: **`exec demo_openworld.cfg`**. See **`docs/OPEN_WORLD.md`**.
+- **Infinite open worlds**: **`r_openWorld` 1** — view-driven sector residency: **`cm_stream`** + **`cm_streamMerge` 1** (overlay sector BSP collision), per-chunk **`nav/sector_X_Y.nav`**, **`sprites/sector_X_Y.ents`** billboard scatter. Console: **`openworld_start`**, **`openworld_sector`**, **`openworld_status`**, **`nav_bake_sector`**. Demo: **`exec demo_openworld.cfg`**. See **`docs/OPEN_WORLD.md`**.
+- **World residency planner**: **`r_openWorldResidency` 1** — value-aware cardinality selection under per-layer budgets (`r_openWorldMaxSectors`, `r_openWorldMaxNavSectors`, `r_openWorldMaxSpriteSectors`) with hysteresis (`r_openWorldLoadRadius` / `r_openWorldUnloadRadius`). MP: **`sv_openWorldResidency` 1`** plans server collision; clients clamp to **`CS_ENGINE_OPENWORLD_SECTORS`**. See **`docs/WORLD_RESIDENCY.md`**. **`ctest -R test_openworld_residency`**.
+- **Sector graph / GPU BFS**: **`r_graphStreamReach` 1** filters residency to k-hop reachable cells (`r_graphStreamHops`); optional **`r_graphCompute` 1`** + **`vid_restart`** for Vulkan compute BFS (`graph_bfs_status`, `graph_bfs_bench`). See **`docs/GRAPH_COMPUTE.md`**. **`ctest -R test_graph_compute`** / **`unit_sector_graph`**.
 - **Procedural patterns**: **`r_proc` 1** — deterministic Voronoi/grid/hex/radial/stripe/noise sector typing. Console: **`proc_pattern`**, **`proc_map`**, **`proc_sample`**. Scatter fallback: **`sprites/region_<id>.ents`** when **`r_procScatterRegion` 1**. Demo: **`exec demo_proc.cfg`**. See **`docs/PROC_PATTERNS.md`**.
 - **Sector BSP fixtures**: **`python3 scripts/tools/gen_sector_bsp.py maps/sector_0_0.bsp`** — minimal collision overlay for **`cm_streamMerge`**. **`ctest -R test_cm_stream_merge`**.
 - **Nav sector bake**: **`nav_bake_sector 0 0`** / **`nav_bake_view`** — Recast tile from sector BSP → **`nav/sector_X_Y.nav`**. Server collision residency: **`sv_openWorld 1`**. **`ctest -R test_nav_bake`**.

CLAUDE.md

@@ -457,6 +457,9 @@ All significant changes require architectural review:
 - **[QUICKSTART.md](docs/QUICKSTART.md)**: End-user quick start (download, game data, run)
 - **[MINIMAL_GAME_SHELL.md](docs/MINIMAL_GAME_SHELL.md)**: Smallest valid `base/` + `.pk3` bootstrap (engine-only / tech demo)
 - **[RELEASE_CHECKLIST.md](docs/RELEASE_CHECKLIST.md)**: Pre-release validation and release steps
+- **[OPEN_WORLD.md](docs/OPEN_WORLD.md)**: Infinite open-world sector streaming (collision, nav, scatter)
+- **[WORLD_RESIDENCY.md](docs/WORLD_RESIDENCY.md)**: Consistent submodular sector residency planner
+- **[GRAPH_COMPUTE.md](docs/GRAPH_COMPUTE.md)**: Sector graph k-hop reachability (CPU + optional Vulkan BFS)
 
 ### Build Scripts
 - `./scripts/compile_engine.sh` - Primary build script

docs/ARCHITECTURE.md

@@ -34,6 +34,12 @@ src/
 ├── navigation/          Pathfinding
 │   ├── nav_recast.cpp/h          Recast/Detour navmesh + crowd
 │   └── nav_bsp_extract.c         BSP triangle extraction
+├── world/               Open-world streaming (sector residency)
+│   ├── world_open.c/h            Sector load/unload coordinator (collision, nav, sprites)
+│   ├── world_residency.c/h       Value-aware cardinality selection under per-layer budgets
+│   ├── world_district.c/h        USD district → sector streaming bridge
+│   ├── world_proc.c/h            Procedural sector typing (region/palette)
+│   └── sector_graph.c/h          Grid CSR + k-hop reachability (CPU BFS, optional GPU)
 ├── audio/               Audio
 │   ├── backends/                  OpenAL, SDL, null
 │   ├── codecs/                    WAV, MP3, Opus, FLAC, WebM
@@ -63,6 +69,8 @@ src/
 │   │   ├── vk_volumetric_pipelines.c Volumetric fog / fluid / luminance / CBT / veg-wind pipeline setup (split from vk.c)
 │   │   ├── vk_volumetric_internal.c/h MSAA depth resolve, fluid sim dispatch, volumetric perf queries (split from vk.c)
 │   │   ├── vk_volumetric_pass_compute.c Local volumetric shadows, froxel compute, composite, SMAA (split from vk.c)
+│   │   ├── vk_graph_bfs.c/h      Optional Vulkan compute BFS for sector graph reachability
+│   │   ├── tr_bsp_stream.c/h     Streamed sector BSP visual overlay (`r_bspStream`)
 │   │   ├── vk_shutdown.c          vk_shutdown, wait-idle, release_resources (split from vk.c)
 │   │   ├── vk_postfx_passes.c     Bloom, SSAO/HBAO, OIT, SSR passes (split from vk.c)
 │   │   ├── vk_clear_attachments.c In-pass color/depth clear + dynamic color write mask (split from vk.c)
@@ -116,10 +124,37 @@ CL_Frame(msec)
   │   ├── Choreo_Update
   │   ├── Horde_Update
   │   ├── BgMap_Frame
-  │   └── WinTitle_Update
+  │   ├── WinTitle_Update
+  │   └── CL_OpenWorld_Frame (when `r_openWorld` 1)
+  │       ├── WorldOpen_UpdateView → WorldResidency_UpdateView (when `r_openWorldResidency` 1)
+  │       │   └── SectorGraph_UpdateReachability (when `r_graphStreamReach` / `r_graphCompute`)
+  │       ├── BSP stream overlay sync (`r_bspStream`)
+  │       └── Billboard scatter draw for loaded sprite sectors
   └── Con_RunConsole()
 ```
 
+Dedicated server: `SV_OpenWorld_Frame` (when `sv_openWorld` 1) plans collision residency from player origins and publishes `CS_ENGINE_OPENWORLD_SECTORS` when `sv_openWorldSync` 1.
+
+## Open-world streaming
+
+View-driven **sector residency** for infinite worlds: modular BSP collision merge, per-chunk Detour nav tiles, and billboard scatter. Three cooperating layers:
+
+| Module | Role |
+|--------|------|
+| `world_open.c` | Sector table, layer load/unload callbacks, legacy radius disk or delegate to residency |
+| `world_residency.c` | Consistent submodular cardinality selection (`r_openWorldResidency` 1) |
+| `sector_graph.c` | k-hop reachability pre-filter (`r_graphStreamReach` 1); optional GPU via `vk_graph_bfs.c` |
+| `cl_openworld.cpp` | Client bridge: nav tile load, scatter parse, MP configstring sync |
+| `sv_openworld.c` | Server collision residency + sector list replication |
+| `cm_stream.c` / `cm_stream_merge.c` | Sector pk3 prefetch and BSP brush overlay merge |
+
+**Planner selection** (`WorldOpen_UpdateView` in `world_open.c`):
+
+1. `r_openWorldResidency 0` — load all cells inside `r_openWorldRadius` (legacy disk).
+2. `r_openWorldResidency 1` — `WorldResidency_UpdateView` scores candidates in a hysteresis annulus (`r_openWorldLoadRadius` / `r_openWorldUnloadRadius`), applies per-layer budgets, and bounds swaps per frame.
+
+Full usage, cvars, authoring, and MP rules: [OPEN_WORLD.md](OPEN_WORLD.md), [WORLD_RESIDENCY.md](WORLD_RESIDENCY.md), [GRAPH_COMPUTE.md](GRAPH_COMPUTE.md).
+
 ## Renderer Pipeline (Vulkan)
 
 The shipping Vulkan renderer is **forward-only** with a layered HDR/post-processing pipeline.

docs/CONTRIBUTING.md

@@ -42,6 +42,7 @@ Tests include:
 - `smoke_test` - Binary checks, server startup, shader validation
 - `check_artifacts` - Artifact format verification
 - `unit_macros` - PAD, PADLEN, MAX_QPATH, MAX_STRING_CHARS
+- Open-world wiring: `test_openworld`, `test_openworld_sync`, `test_openworld_residency`, `test_graph_compute`, `unit_world_residency`, `unit_sector_graph` (see [DEVELOPMENT_SETUP.md](DEVELOPMENT_SETUP.md#open-world--residency--graph-compute))
 
 ### Before Submitting
 

docs/DEVELOPMENT_SETUP.md

@@ -294,6 +294,34 @@ cd build-vk-Release && ctest -C Release --output-on-failure
 ctest --preset test-vulkan-release
 ```
 
+### Open-world / residency / graph compute
+
+After a Release build, these script tests validate wiring without a GPU client:
+
+```bash
+cd build-vk-Release
+ctest -R 'test_openworld|test_openworld_sync|test_openworld_residency|test_graph_compute|test_cm_stream_merge|test_nav_bake' -V
+```
+
+| Test | What it checks |
+|------|----------------|
+| `test_openworld` | Sector fixtures, scatter `.ents`, `world_open.c` symbols |
+| `test_openworld_sync` | MP configstring path (`sv_openworld.c`, `tr_bsp_stream.c`) |
+| `test_openworld_residency` | Residency planner cvars and source wiring |
+| `test_graph_compute` | `sector_graph.c` + `vk_graph_bfs.c` integration |
+| `test_cm_stream_merge` | Sector BSP merge API + `gen_sector_bsp.py` layout |
+| `test_nav_bake` | Recast sector tile bake from sector BSP |
+| `unit_world_residency` | Cardinality selection + symmetric difference (no engine link) |
+| `unit_sector_graph` | CPU BFS reachability (no engine link) |
+
+**Runtime collision smoke** (requires built `idtech3_server`):
+
+```bash
+ctest -R test_openworld_runtime -V
+```
+
+See [OPEN_WORLD.md](OPEN_WORLD.md), [WORLD_RESIDENCY.md](WORLD_RESIDENCY.md), and [GRAPH_COMPUTE.md](GRAPH_COMPUTE.md).
+
 ## IDE Setup
 
 ### VS Code / Cursor

docs/GRAPH_COMPUTE.md

@@ -54,6 +54,21 @@ Player origin(s) → SectorGraph_UpdateReachability (CPU BFS)
                  → WorldOpen load/unload
 ```
 
+## Vulkan compute path
+
+Shader: `src/renderers/vulkan/shaders/glsl/graph/graph_bfs_expand.comp` (compiled via `scripts/compile_shaders.sh` as `graph_bfs_expand_cs`).
+
+| Step | Action |
+|------|--------|
+| Enable | `set r_graphCompute 1` then **`vid_restart`** (pipeline created at renderer init) |
+| Verify | `graph_bfs_status` — reports `compute=on` when `R_GraphBfs_Active` |
+| Bench | `graph_bfs_bench` — 5×5 toy grid CPU vs GPU timing |
+| Debug mismatch | `set r_graphStreamVerify 1` — logs CPU vs GPU bitset differences (disables per-frame cache) |
+
+If `graph_bfs_bench` prints `pipeline not ready`, toggle `r_graphCompute 1` and run **`vid_restart`**. GPU path is **client-only**; dedicated server uses CPU BFS.
+
+`r_graphBlockUnloaded 1` treats edges into sectors without loaded collision as blocked — useful when walkable corridors must follow merged BSP, not the full grid.
+
 ## Phase 1 follow-ons (not implemented)
 
 - Nav influence heatmaps from sector reachability

docs/OPEN_WORLD.md

@@ -49,6 +49,8 @@ Demo mod: `exec demo_openworld.cfg`.
 | `openworld_status` | Cvars + loaded sector table |
 | `openworld_list` | List active sectors and layers |
 | `openworld_sector <x> <y>` | Force-load nav + scatter (+ collision if enabled) |
+| `nav_bake_sector <x> <y>` | Recast-bake `nav/sector_X_Y.nav` from sector BSP collision |
+| `nav_bake_view` | Bake nav tile for the sector under the current view |
 
 ## Cvars
 

docs/ROADMAP.md

@@ -65,6 +65,14 @@ Priorities that keep **CI green** and **README/build truth** aligned:
 - [x] Multiple model formats (glTF primary on Vulkan for GPU path; CPU tess fallback - see `docs/GLTF.md`)
 - [x] 6 image formats (EXR, PNG, TGA, JPG, PCX, BMP)
 
+### Open worlds -- In progress
+- [x] Sector streaming scaffold (`r_openWorld`, `cm_stream`, `cm_streamMerge`, nav tiles, scatter)
+- [x] Consistent submodular residency planner (`r_openWorldResidency`, per-layer budgets)
+- [x] Sector graph k-hop reachability filter + optional Vulkan compute BFS (`r_graphStreamReach`, `r_graphCompute`)
+- [x] MP sector sync (`sv_openWorldSync`, `CS_ENGINE_OPENWORLD_SECTORS`)
+- [ ] Streamed sector renderer VBO residency (overlay uses immediate tess path today)
+- See [OPEN_WORLD.md](OPEN_WORLD.md), [WORLD_RESIDENCY.md](WORLD_RESIDENCY.md), [GRAPH_COMPUTE.md](GRAPH_COMPUTE.md)
+
 ### Integration -- Complete
 - [x] All 16 systems wired into game loop
 - [x] 64 Lua-callable engine functions