Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,10 +58,14 @@ Build artifacts go to `build-vk-Release/` and are copied to `release/`.
- **FreeUSD (default ON)**: Git submodule **`src/external/FreeUSD`** ([gopexllc/FreeUSD](https://github.com/gopexllc/FreeUSD)); **`USE_FREEUSD=ON`** in CMake and **`./scripts/compile_engine.sh`** (auto `git submodule update --init`; pass **`nofreeusd`** to disable). Targets link **`freeusd::runtime`** + **`freeusd::c`** via **`idtech3_freeusd`**. Renderer **`r_freeusd` 1**; client **`usd_*`** tools. Init: **`./scripts/init_optional_submodules.sh --freeusd`**. See **`docs/FREEUSD.md`**.
- **World districts + proxy meshes**: **`r_district` 1** (default) — USD manifest via **`district_load world/playfield.usda`**, FreeUSD snapshot parse, proxy residency (`r_districtProxy`), optional **`cm_districtStream`** sector prefetch. Console: **`district_list`**, **`district_proxy`**, **`district_load_full`**. Demo: **`exec demo_districts.cfg`**. See **`docs/DISTRICTS.md`**.
- **Infinite open worlds**: **`r_openWorld` 1** — view-driven sector residency: **`cm_stream`** + **`cm_streamMerge` 1** (overlay sector BSP collision), per-chunk **`nav/sector_X_Y.nav`**, **`sprites/sector_X_Y.ents`** billboard scatter. Console: **`openworld_start`**, **`openworld_sector`**, **`openworld_status`**. Demo: **`exec demo_openworld.cfg`**. See **`docs/OPEN_WORLD.md`**.
- **Consistent sector residency (open world)**: **`r_openWorldResidency` 1** — value-aware cardinality selection under per-layer budgets (`r_openWorldMaxSectors`, `r_openWorldMaxNavSectors`, `r_openWorldMaxSpriteSectors`) with bounded swaps (`r_openWorldResidencyMaxSwaps`). MP: **`sv_openWorldResidency` 1** plans server collision from player unions; clients clamp to **`CS_ENGINE_OPENWORLD_SECTORS`**. Nav/sprites subject to **`cl_openWorldResidencyNavLocal`**. See **`docs/WORLD_RESIDENCY.md`**. **`ctest -R unit_world_residency`** / **`test_openworld_residency`**.
- **Sector graph reachability (open world)**: **`r_graphStreamReach` 1** — k-hop BFS filter on residency candidates (`r_graphStreamHops`, default 8); optional GPU path **`r_graphCompute` 1** + **`vid_restart`**. Console: **`graph_reach_test`**, **`graph_bfs_status`**, **`graph_bfs_bench`**. See **`docs/GRAPH_COMPUTE.md`**. **`ctest -R unit_sector_graph`** / **`test_graph_compute`**.
- **Procedural patterns**: **`r_proc` 1** — deterministic Voronoi/grid/hex/radial/stripe/noise sector typing. Console: **`proc_pattern`**, **`proc_map`**, **`proc_sample`**. Scatter fallback: **`sprites/region_<id>.ents`** when **`r_procScatterRegion` 1**. Demo: **`exec demo_proc.cfg`**. See **`docs/PROC_PATTERNS.md`**.
- **Sector BSP fixtures**: **`python3 scripts/tools/gen_sector_bsp.py maps/sector_0_0.bsp`** — minimal collision overlay for **`cm_streamMerge`**. **`ctest -R test_cm_stream_merge`**.
- **Nav sector bake**: **`nav_bake_sector 0 0`** / **`nav_bake_view`** — Recast tile from sector BSP → **`nav/sector_X_Y.nav`**. Server collision residency: **`sv_openWorld 1`**. **`ctest -R test_nav_bake`**.
- **MP sector sync**: **`sv_openWorldSync` 1** → **`CS_ENGINE_OPENWORLD_SECTORS`**; client **`cl_openWorldSync` 1**. Visual overlay: **`r_bspStream` 1**. **`ctest -R test_openworld_sync`**.
- **Wwise-style runtime mixer (audio)**: **`s_mixer_enable` 1** (default) — output buses (`s_bus_*`), RTPCs, state groups (`snd_setstate`), auto-duck, sound events (`snd_playevent`), replay capture. See **`docs/AUDIO_WWISE_PARITY.md`**.
- **Animgraph (native game module)**: **`g_animgraph` 1** — JSON state machine for glTF/IQM clip selection (`animgraph/*.json`); Lua **`Engine.AnimGraph.*`**. See **`docs/ANIMGRAPH.md`**.

### Linting / Static Analysis

Expand Down
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ Modern id Tech 3: **Vulkan renderer with PBR**, validation and smoke-tested buil
**Audio**:
* OpenAL backend with HRTF for 3D positional audio
* Real-time reverb and occlusion effects via OpenAL EFX (heuristic environmental acoustics)
* Wwise-inspired runtime mixer (buses, RTPCs, state groups, sound events) — [docs/AUDIO_WWISE_PARITY.md](docs/AUDIO_WWISE_PARITY.md)
* Audio codec support: mp3, ogg, wav, flac, webm, opus

**Video**:
Expand Down Expand Up @@ -61,7 +62,14 @@ Modern id Tech 3: **Vulkan renderer with PBR**, validation and smoke-tested buil
* Optional TLS-secured remote console (off by default)
* Console/logging for connection diagnostics and real-time network statistics

**World streaming** (engine-original open worlds):
* View-driven sector residency — collision merge, per-chunk nav tiles, billboard scatter — [docs/OPEN_WORLD.md](docs/OPEN_WORLD.md)
* Optional consistent cardinality planner (`r_openWorldResidency`) — [docs/WORLD_RESIDENCY.md](docs/WORLD_RESIDENCY.md)
* Sector graph k-hop reachability filter + optional Vulkan compute BFS — [docs/GRAPH_COMPUTE.md](docs/GRAPH_COMPUTE.md)
* USD world districts + proxy meshes — [docs/DISTRICTS.md](docs/DISTRICTS.md)

**Systems** (game and engine extensions):
* Animgraph JSON state machine for glTF/IQM clip selection — [docs/ANIMGRAPH.md](docs/ANIMGRAPH.md)
* AIML 2.1-oriented chatbot scripting (see `g_aiml.c` / Lua `Engine.AIML`)
* GOAP (AI planning)
* Flocking and boids (crowd AI)
Expand Down
32 changes: 32 additions & 0 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,14 @@ src/
├── qcommon/ Shared engine (VM, filesystem, network)
│ ├── vm.c / vm_local.h VM create, native load, QVM path
│ ├── vm_native_module.c/h Native `.so`/`.dll` filename candidates
│ ├── cm_stream.c Sector pk3 prefetch + collision merge overlay
│ └── files.c FS_LoadLibrary search (modules/vm/gamedir)
├── world/ Open-world streaming + districts
│ ├── world_open.c/h Sector residency (collision, nav, scatter)
│ ├── world_residency.c/h Value-aware cardinality planner (optional)
│ ├── sector_graph.c/h k-hop reachability pre-filter (CPU + optional GPU BFS)
│ ├── world_district.c/h USD district manifest + proxy residency
│ └── world_proc.c/h Procedural sector typing (Voronoi/grid/hex/…)
├── server/ Dedicated server
├── botlib/ Bot AI (Q3 AAS pathfinding)
└── external/ Vendored libraries
Expand Down Expand Up @@ -146,6 +153,31 @@ For goals and longer notes, see [RENDERER_2026_ARCHITECTURE_PASS.md](RENDERER_20

**Bootstrap game data** (minimal `base/` layout that still satisfies the filesystem): [MINIMAL_GAME_SHELL.md](MINIMAL_GAME_SHELL.md).

## Open-world streaming

View-driven **sector residency** lives in `src/world/` and is ticked from `CL_OpenWorld_Frame` / `SV_OpenWorld_Frame`:

```
View / player origin(s)
└── WorldOpen_UpdateView
├── r_openWorldResidency 0 → radius disk (r_openWorldRadius)
└── r_openWorldResidency 1 → WorldResidency (budget + bounded delta)
├── optional SectorGraph k-hop filter (r_graphStreamReach)
├── cm_stream merge → collision overlay
├── nav/sector_X_Y.nav → Detour addTile
└── sprites/sector_X_Y.ents → billboard scatter
```

| Module | Role |
|--------|------|
| `world_open.c` | Layer load/unload, `openworld_*` console, MP configstring sync |
| `world_residency.c` | Consistent submodular sector selection (collision/nav/sprite budgets) |
| `sector_graph.c` | Grid CSR + multi-source BFS; optional `vk_graph_bfs.c` compute path |
| `world_district.c` | USD districts; integrates with WorldOpen when `r_openWorld 1` |
| `world_proc.c` | Deterministic region/palette typing for scatter and matroid mode |

Docs: [OPEN_WORLD.md](OPEN_WORLD.md), [WORLD_RESIDENCY.md](WORLD_RESIDENCY.md), [GRAPH_COMPUTE.md](GRAPH_COMPUTE.md), [DISTRICTS.md](DISTRICTS.md), [PROC_PATTERNS.md](PROC_PATTERNS.md).

## Native game modules (VM)

When `fs_restrict` is **0** (default), `VM_Create` always tries a **native** shared library before falling back to a `.qvm` (`src/qcommon/vm.c`). Native load is disabled when `fs_restrict` is set (demo-style restriction).
Expand Down
7 changes: 7 additions & 0 deletions docs/DEVELOPMENT_SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -294,6 +294,13 @@ cd build-vk-Release && ctest -C Release --output-on-failure
ctest --preset test-vulkan-release
```

Open-world / residency / sector-graph unit and script tests (no GPU or game data required for unit targets):

```bash
cd build-vk-Release
ctest -R 'unit_world_residency|unit_sector_graph|test_openworld|test_graph_compute' -V
```

## IDE Setup

### VS Code / Cursor
Expand Down
2 changes: 2 additions & 0 deletions docs/DISTRICTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,8 @@ Shipped in `idtech3_demo.pk3` when built with `demo`: `exec demo_districts.cfg`

When **`r_openWorld 1`**, `district_load_full` calls **`WorldOpen_LoadSector`** for each cell in the district grid (collision + nav + scatter per cvars). When **`r_openWorld 0`**, full district load uses legacy **`CM_Stream_LoadSector`** (collision prefetch only). See [OPEN_WORLD.md](OPEN_WORLD.md#limitations).

With **`r_openWorldResidency 1`**, `WorldDistrict_StreamSectors` sets a district candidate filter and plans residency at the district centroid instead of brute-force loading every cell in the grid. Optional **`r_graphStreamReach 1`** further constrains candidates to k-hop reachable sectors. See [WORLD_RESIDENCY.md](WORLD_RESIDENCY.md) and [GRAPH_COMPUTE.md](GRAPH_COMPUTE.md).

## Related docs

- [OPEN_WORLD.md](OPEN_WORLD.md) — view-driven sector streaming (BSP prefetch, per-chunk nav, billboard scatter)
Expand Down
12 changes: 12 additions & 0 deletions docs/GRAPH_COMPUTE.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,18 @@ Player origin(s) → SectorGraph_UpdateReachability (CPU BFS)
→ WorldOpen load/unload
```

## Troubleshooting

| Symptom | Cause / fix |
|---------|-------------|
| `[GraphBfs] pipeline not ready` | Set **`r_graphCompute 1`**, then **`vid_restart`** so `vk_graph_bfs.c` registers the GPU reach fn |
| Residency ignores graph filter | **`r_graphStreamReach`** must be **1**; filter applies only when `WorldResidency` is active (`r_openWorldResidency 1`) |
| Cells unexpectedly unreachable | Window is centered on view cell with radius from **`r_openWorldUnloadRadius`**; cells outside the 64×64 window fail `SectorGraph_IsReachable` |
| Edges through unloaded sectors | Enable **`r_graphBlockUnloaded 1`** to cut adjacency into sectors not resident in `WorldOpen` |
| CPU/GPU mismatch warnings | **`r_graphStreamVerify 1`** compares bitsets; disable verify in shipping, fix shader or CSR upload if mismatches persist |

**`r_graphCompute 1`** alone does not change which sectors load — it only runs the GPU BFS path (and enables verify-friendly cache bypass). Pair with **`r_graphStreamReach 1`** for streaming behavior.

## Phase 1 follow-ons (not implemented)

- Nav influence heatmaps from sector reachability
Expand Down
22 changes: 22 additions & 0 deletions docs/WORLD_RESIDENCY.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,28 @@ Inspired by the consistent submodular optimization framework (Dütting et al., 2

Startup log when enabled: `[world_residency] enabled epsilon=… k_col=… max_swaps=… matroid=…`

## Quick start

After hub map load and `openworld_start` (see [OPEN_WORLD.md](OPEN_WORLD.md)):

```text
set r_openWorldResidency 1
set r_openWorldLoadRadius 12288
set r_openWorldUnloadRadius 14336
set r_openWorldMaxSectors 64
set r_openWorldMaxNavSectors 32
set r_openWorldMaxSpriteSectors 64
set r_openWorldResidencyMaxSwaps 4

// Optional: k-hop reachability pre-filter
set r_graphStreamReach 1
set r_graphStreamHops 8

openworld_status
```

Demo mod ships commented defaults in `demo_openworld.cfg` (`exec demo_openworld.cfg`). For MP, also set **`sv_openWorldResidency 1`** on the dedicated server.

## Multiplayer rules

- **Server collision is authoritative.** `SV_OpenWorld_Frame` with `sv_openWorldResidency 1` plans collision from the union of active player origins, then publishes via `CS_ENGINE_OPENWORLD_SECTORS`.
Expand Down