diff --git a/.gitignore b/.gitignore index 51b59868ce..3ded96a647 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,14 @@ **/books/* build/* +build-*/ +# Local distribution/release outputs +/dist/ +/release/ +tests/smoke/artifacts/ +docker/* +!docker/Dockerfile +!docker/docker-compose.yml +!docker/build-linux.sh **/data/* **/fonts/* **/images/* @@ -41,6 +50,7 @@ steam_appid.txt *.log *.user *log.txt +!docs/mod_release/mod-changelog.txt *.cache *.zip *.res @@ -75,6 +85,9 @@ steam_appid.txt xcode/Barony/Barony.xcodeproj/project.xcworkspace/* xcode/Barony/Barony.xcodeproj/xcuserdata/* *.ps1 +!setup_barony_vs2022_x64.ps1 +restore-env-*.ps1 +install.log !LICENSE.nativefiledialog.txt /VS.2015/Barony/x64/Debug *.sarif @@ -83,3 +96,7 @@ xcode/Barony/Barony.xcodeproj/xcuserdata/* *.ilk /VS.2015/x64 *.sqlite-journal +/deps/* +# Local smoke outputs and SDK drops +tests/smoke/artifacts/ +third_party/steamworks_sdk/ diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000000..15744311eb --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,294 @@ +# Repository Guidelines + +## Project Structure & Module Organization +The project is CMake-based with the root `CMakeLists.txt` orchestrating all targets. Core gameplay and editor code lives in `src/`, with major subsystems split into `src/interface/`, `src/magic/`, `src/ui/`, `src/engine/`, and `src/imgui/`. Runtime text and localization assets are in `lang/`. CI helper scripts are in `ci/`. Platform-specific project files are under `VS/`, `VS.2015/`, and `xcode/`. Start with `README.md`, `INSTALL.md`, and `CONTRIBUTING.md` for workflow context. + +## Build, Test, and Development Commands +Use out-of-source builds: + +```bash +mkdir -p build && cd build +cmake .. +cmake --build . -- -j +``` + +Common build variants: + +- `cmake -DCMAKE_BUILD_TYPE=Release -DFMOD_ENABLED=ON ..` builds release with FMOD. +- `cmake -DFMOD_ENABLED=OFF ..` builds without FMOD. + +CI-like Linux scripts (require CI secrets and packaged dependencies): + +- `cd ci && ./build-linux_fmod_steam.sh` +- `cd ci && ./build-linux_fmod_steam_eos-barony.sh` + +After building, run targets from the build directory (for example `./barony` or `./editor`). + +## Local macOS Run (Steam Assets) +For local gameplay/smoke validation on macOS, use the built binary but run it from the Steam app bundle so assets/config paths match normal runtime behavior. + +1. Build: + +```bash +cmake -S . -B build-mac -G Ninja -DFMOD_ENABLED=OFF +cmake --build build-mac -j8 +``` + +2. Copy the built executable into the Steam install (make a backup first): + +```bash +src="$PWD/build-mac/barony.app/Contents/MacOS/barony" +dstdir="$HOME/Library/Application Support/Steam/steamapps/common/Barony/Barony.app/Contents/MacOS" +cp "$dstdir/Barony" "$dstdir/Barony.backup-$(date +%Y%m%d-%H%M%S)" +cp "$src" "$dstdir/Barony" +chmod +x "$dstdir/Barony" +``` + +3. Run from the Steam path: + +```bash +"$HOME/Library/Application Support/Steam/steamapps/common/Barony/Barony.app/Contents/MacOS/Barony" -windowed -size=1280x720 +``` + +## Coding Style & Naming Conventions +Follow surrounding style in touched files; this codebase mixes legacy C/C++ patterns. Avoid broad formatting-only edits. Tabs are common in older files, while newer edits may use spaces; preserve local consistency. File naming is mostly subsystem-based lowercase (examples: `act*.cpp`, `monster_*.cpp`). Use `PascalCase` for types/classes (for example `Entity`) and uppercase names for macros/constants. + +## Testing Guidelines +There is no dedicated unit-test suite in this repository. Required validation is: + +- Build success for affected targets (`barony`, `editor` when relevant). +- Manual smoke test of the changed flow (menu/load/gameplay/editor path you touched). +- Keep GitHub Actions Linux build checks green for PRs. +- For multiplayer-expansion work, update `/Users/sayhiben/dev/Barony-8p/AGENTS.md` and the relevant `/Users/sayhiben/dev/Barony-8p/merge-prs/PR*.md` ticket inline as progress happens (checklist state + artifact paths + notable caveats). + +## Commit & Pull Request Guidelines +Create a topic branch per change. For bugfix work, target `master` (per `README.md`). Keep commits focused and message subjects short, imperative, and specific (recent history includes messages like `update hash` and `fix one who knocks achievement when parrying`). In PRs, include: what changed, why, test steps/results, and linked issues. Add screenshots for visible UI/editor changes. + +## Security & Configuration Tips +Do not commit secrets or environment tokens. CI scripts rely on `DEPENDENCIES_ZIP_KEY` and `DEPENDENCIES_ZIP_IV`; provide them via environment variables only. + +## Codex Sandbox / Permissions +When running in Codex with sandboxing, ask for sandbox breakout/escalation permission before running commands that may require access outside the sandbox or external services. + +- Common examples: `git ...`, `gh ...`, Steam app binary runs, and other commands that touch restricted paths/resources. +- If a command is blocked by sandboxing, rerun with escalation rather than changing the intended workflow. +- If launches fail with `Abort trap: 6` during smoke runs, treat it as a likely sandbox restriction signal and rerun with escalation. + +## Multiplayer Expansion (PR 940) Working Notes +- Expansion target is `MAXPLAYERS=15` (not 16). Preserve nibble-packed ownership assumptions unless a deliberate encoding refactor is planned. +- Keep smoke instrumentation isolated to `/Users/sayhiben/dev/Barony-8p/src/smoke/SmokeHooks*.cpp` and `/Users/sayhiben/dev/Barony-8p/src/smoke/SmokeTestHooks.hpp` with minimal call sites in gameplay/UI/network files. +- Keep headless mapgen integration plumbing (`-smoke-mapgen-integration*` parsing/validation/runner) in smoke hook implementation files; `src/game.cpp` should stay wiring-only for those options. +- Avoid adding ad-hoc smoke utility logic directly in core gameplay files; prefer hook APIs declared in `SmokeTestHooks.hpp` and keep base-game paths clean. +- Preferred local validation path is local build binary + Steam assets datadir (`--app .../build-mac/.../barony --datadir .../Barony.app/Contents/Resources`) instead of replacing the Steam executable. +- After long or high-instance smoke runs, clean generated cache bloat (especially `models.cache` under smoke artifact homes) while preserving logs/artifacts needed for debugging. +- If host performance degrades during smoke campaigns, check for lingering `smoke_runner.py mapgen-sweep`, `smoke_runner.py lan-helo-chunk`, and `barony` processes; terminate stale runs before launching new lanes. +- Known intermittent issue: churn/rejoin can show transient `lobby full` / join retries (`error code 16`). Track with artifacts and summaries, and avoid conflating it with unrelated feature-lane pass/fail unless assertions require it. +- Add and maintain compile-time gating for smoke hooks/call sites so smoke instrumentation compiles or executes only when a dedicated smoke-test flag is enabled. +- Smoke validation requires a smoke-enabled build (`-DBARONY_SMOKE_TESTS=ON`); if expected `[SMOKE]` logs are missing, verify generated config/build mode and rebuild the smoke target before rerunning tests. +- Keep generated `Config.hpp` build-local on Windows; writing it into `src/` cross-contaminates smoke/non-smoke build trees and can make OFF builds link smoke hooks by accident. +- Windows smoke launches must run with `cwd` set to the per-instance `.barony` home; otherwise relative config/log paths collapse back into the repo root and lane signal becomes unusable. +- Fresh per-instance smoke homes can stall in intro/title flow; ensure smoke homes are pre-seeded with profile data (`skipintro=true`, `mods=[]`, and compiled books cache) so autopilot reaches lobby/gameplay deterministically. +- Local splitscreen is a legacy path and should stay hard-capped at 4 players; retain dedicated smoke coverage for `/splitscreen > 4` clamp behavior and over-cap leakage checks. +- When parsing smoke status lines with similarly named keys (for example `connected` vs `over_cap_connected`), parse exact `key=value` tokens to avoid false negatives and lane hangs. +- During style/contribution cleanup, treat `#ifdef BARONY_SMOKE_TESTS` guards around smoke-hook callsites as an acceptable and idiomatic exception. +- Windows LAN gameplay lanes are timing-sensitive if `--auto-start-delay=0`; use the default `2` second delay (or higher) for reliable `GAMESTART`/`MAPGEN` assertions. +- Preferred balancing loop for mapgen tuning: hook-owned in-process integration preflight (`levels=1,7,16,33`, fixed seed) -> single-runtime matrix confirmation -> runs=5 volatility gate -> full-lobby confirmation. + +### Validation Summary (2026-02-12) +- Overall expansion status is near-finish: core LAN networking validation is green (HELO correctness, adversarial fail modes, soak/churn, high-slot regression lanes). +- Completed/green lanes include: save/reload owner-encoding sweep (`1..15`), lobby regression lanes (kick-target, slot-lock/copy, page navigation), remote-combat slot bounds, local splitscreen baseline, and `/splitscreen > 4` cap clamp. +- Steam backend handshake was validated for host-room/key flow; local same-account multi-instance joins remain a known Steam limitation. +- EOS-specific validation is intentionally not a release gate for this mod release; Epic players can use the matching Steam or NoDRM package. +- Known intermittent issue remains: churn/rejoin can hit transient `lobby full` / `error code 16` retries before recovery; track with artifacts and do not conflate with unrelated lane failures. +- Smoke compile/runtime gating is in place (`BARONY_SMOKE_TESTS`), and the preferred local lane path is local build binary + Steam `--datadir` assets. + +### Validation Addendum (2026-02-26) +- Implemented canonical duck ownership/color encoding for Hermit ducks (4 color variants, owner-safe at `MAXPLAYERS=15`) and removed duck-owner/color decode dependence on `items[TOOL_DUCK].variations`. +- Added runtime compatibility warning when `tool_duck` variations are below canonical span (`MAXPLAYERS * 4`); observed warning with Steam datadir (`variations=16`, canonical span `60`). +- Enemy HP bar forwarding now uses remote-slot/disconnect/local-player guards only; it no longer hard-blocks send on `net_clients[].host/port` zero values. +- LAN remote-combat validation passed with `client-ENHP` and `client-DAMI` contexts present. + - Artifact: `tests/smoke/artifacts/remote-combat-fix-20260226-001704` +- Save/reload owner-encoding compatibility lane passed after duck encoding changes. + - Artifact: `tests/smoke/artifacts/save-reload-compat-duck-fix-20260226-002659` +- Regression lanes passed: + - Splitscreen cap clamp (`/splitscreen 8 -> 4`): `tests/smoke/artifacts/splitscreen-cap-duck-fix-20260226-002744` + - Inventory fast-pass (lifecycle/edge/churn): `tests/smoke/artifacts/inventory-fast-pass-duck-fix-20260226-002822` +- Historical backend handshake follow-up lanes were attempted but could not enter room-key handshake in this local build context (no room key captured, launch prerequisites blocked). + - Steam artifact: `tests/smoke/artifacts/steam-remote-combat-fix-20260226-001807` + - EOS artifact: `tests/smoke/artifacts/eos-remote-combat-fix-20260226-002141` (tracked for history only; not a release gate) + +### Windows Validation Snapshot (2026-03-14) +- VS2022 x64 Windows release build (`build-vs2022-x64`) now coexists cleanly with a smoke build (`build-vs2022-x64-smoke-nosteam`) after moving generated `Config.hpp` to the build directory. +- Windows no-Steam smoke passes recorded at: + - `tests/smoke/artifacts/win-helo15-lobby-20260314-20260314-132233` (15p lobby / HELO / account-label coverage) + - `tests/smoke/artifacts/win-helo4-mapgen-delay2-20260314-20260314-133614` (4p gameplay + mapgen) + - `tests/smoke/artifacts/win-helo9-mapgen-delay2-20260314-20260314-133709` (9p gameplay + mapgen) + - `tests/smoke/artifacts/win-helo9-mapgen-v501compat-20260314-142002` (9p gameplay + mapgen after v5.0.1/v5.0.2 hash-compat patch) +- Windows false-fail examples with `--auto-start-delay=0`: + - `tests/smoke/artifacts/win-helo2-mapgen-20260314-20260314-133105` + - `tests/smoke/artifacts/win-helo4-mapgen-20260314-20260314-132443` +- Local Windows Steam install (`appmanifest_371970.acf`: `buildid=21759608`, `LastUpdated=2026-02-04 19:12:45 -08:00`) contains a fully self-consistent v5.0.1 map set: all 1922 `maps/*.lmp` files hash-match the upstream `v5.0.1` table, and exactly 19 files differ from the upstream `v5.0.2` table. +- Keep `v5.0.2` hashes canonical in `src/files.cpp`, but accept the 19 changed `v5.0.1` hashes as official compatibility values cross-platform until upstream asset packs catch up. Full audit artifact: `tests/smoke/artifacts/win-steam-map-hash-audit-20260314-142142` (`ACCEPTED_FILES=1922`, `COMPAT_HIT_FILES=19`). +- Because the broadly distributed asset packs still include those v5.0.1-era variants, these runs are treated as valid runtime-stability and compatibility signal rather than a release blocker pending exact upstream v5.0.2 asset certification. +- Windows overlay release artifacts were packaged from fresh full-feature build trees with `scripts/mod_release/package_windows_release.ps1`: + - `release-artifacts/barony-8p-windows-steam-20260314-195941.zip` + - `release-artifacts/barony-8p-windows-nodrm-20260314-195941.zip` + +### Validation Addendum (2026-03-21) +- Implemented authoritative level-load mapgen metadata for network clients: + - host now freezes a connected-player slot mask for the level load, computes a final tile checksum after generation, and appends both to `LVLC` / `LVLR` + - clients now consume that authoritative mask during map scaling/spawn filtering and compare their post-load tile checksum against the host value +- Added a shared tile checksum helper over width/height/skybox/flags/tile layers and a client warning path when host/local tile checksums disagree. +- Implemented automatic checksum-mismatch recovery for level-load map geometry: + - client now requests an authoritative host snapshot on checksum failure + - host streams map geometry snapshot chunks (`name/author/filename`, `width/height/skybox`, flags, tiles, tile attributes) over reliable packets + - client applies the snapshot, rebuilds pathing/chunks, and confirms the recovered checksum +- Extended level-load parity checks beyond raw tiles: + - the shared tile checksum now also covers `tileAttributes`, so slippery/slow/grease/treasure-room drift is detected together with wall/layout drift + - `LVLC` / `LVLR` level-load metadata now carries a second initial entity checksum covering the post-load entity scene before clients discard `NOUPDATE` placeholders + - clients log an `entity sync mismatch` when initial placements/content diverge even if final geometry still matches +- Smoke-only connected-player overrides now remain available for host/single-runtime mapgen lanes, but network clients no longer override the host's authoritative level-load mask. +- Targeted 2-instance LAN repro passed with a client-only smoke override of `5` connected players: + - host authoritative inputs: `players=2 mask=0x0003 checksum=2380154547` + - client received the same authoritative inputs before loading and generated the same `The Mines` floor (`players=2`, identical room/economy summary) + - artifact: `tests/smoke/artifacts/map-desync-authoritative-launch-20260321-210316` + - summary: `tests/smoke/artifacts/map-desync-authoritative-launch-20260321-210316/summary.env` +- Targeted 2-instance LAN recovery lane passed with a client-only forced checksum mismatch: + - client intentionally flipped one tile after local load, logged `local_checksum=1494320743` vs host `2380154547`, requested host recovery, and applied transfer `1` + - host streamed an authoritative `19621` byte snapshot in `11` reliable chunks; client rebuilt geometry and finished with checksum `2380154547` + - artifact: `tests/smoke/artifacts/map-desync-snapshot-recovery-20260321-213455` + - summary: `tests/smoke/artifacts/map-desync-snapshot-recovery-20260321-213455/summary.env` +- Post-hardening regression checks passed after adding tile-attribute coverage and the initial entity checksum: + - smoke-enabled target rebuild succeeded: `cmake --build build-mac-smoke -j8 --target barony` + - 2-instance forced-recovery lane remained green on the updated packet/checksum format + - artifact: `tests/smoke/artifacts/map-desync-entity-checksum-20260321-215354` + - summary: `tests/smoke/artifacts/map-desync-entity-checksum-20260321-215354/summary.env` +- Broader smoke pass on the same branch stayed green for the stable/high-signal lanes: + - 2-instance baseline dungeon transition: `tests/smoke/artifacts/level-sync-baseline-2p-20260321-221439` + - 4-instance dungeon/mapgen transition with a 3-second lobby settle delay: `tests/smoke/artifacts/level-sync-4p-mapgen-delay3-20260321-221847` + - 3-run 4-instance HELO soak: `tests/smoke/artifacts/helo-soak-level-sync-20260321-221956` + - HELO adversarial matrix (reverse/even-odd/duplicate-first expected-pass, drop-last/duplicate-conflict-first expected-fail): `tests/smoke/artifacts/helo-adversarial-level-sync-20260321-222152` + - standard 6-instance join/leave churn with ready-sync assertions: `tests/smoke/artifacts/join-leave-churn-standard-20260321-222825` + - save/reload owner-encoding compatibility: `tests/smoke/artifacts/save-reload-compat-level-sync-20260321-223036` +- Follow-up smoke gating closed the zero-delay lobby-start race: + - host smoke auto-start now waits for `connected` and `joined` parity, where `joined` means each remote client has actually entered the lobby (`JACK`-ack path) before auto-start fires + - 4-instance zero-delay auto-start lane now passes cleanly with all clients loading `start.lmp`: `tests/smoke/artifacts/level-sync-4p-mapgen-delay0-smokegate-20260321-230540` + - summary: `tests/smoke/artifacts/level-sync-4p-mapgen-delay0-smokegate-20260321-230540/summary.env` +- Same-level reload follow-up is now green on a networked lane: + - 2-instance procedural reload/regeneration lane with `mapgen-reload-same-level=1` passed and matched both requested reload seeds (`100100`, `100101`) + - artifact: `tests/smoke/artifacts/reload-procedural-level-sync-2p-20260321-230638` + - summary: `tests/smoke/artifacts/reload-procedural-level-sync-2p-20260321-230638/summary.env` +- Experimental lanes that were not counted toward confidence: + - churn-with-gameplay auto-start drifted into midgame-rejoin retries instead of standard lobby churn: `tests/smoke/artifacts/join-leave-churn-level-sync-20260321-222429` + - remote-combat lane aborted before gameplay because host launch never completed: `tests/smoke/artifacts/remote-combat-level-sync-20260321-223515` +- Current caveat: the fallback is geometry-scoped. It guarantees tile/flag/tile-attribute parity after load, but it is not a full host-authoritative level bootstrap for static entity/content drift. +- Cleanup follow-up (2026-03-22): + - extracted level-load authority and recovery ownership into `src/level_load_sync.cpp/.hpp`, so `game.cpp`, `net.cpp`, and `maps.cpp` no longer carry the packet-layout and snapshot-recovery state directly + - split map snapshot application so `files.cpp` now owns only geometry-data copy plus HDR/lightmap/minimap cache reset, while `level_load_sync.cpp` owns vismap/shoparea buffer replacement during recovery + - build verification passed: `cmake --build build-mac -j8 --target barony editor` + - no new smoke artifact for this cleanup-only pass; residual caveat remains that HDR/lightmap reset still lives in `files.cpp` because the ambience console variables are anchored there today + +### Balancing Lessons and Guardrails +- Hard rule: preserve `1..4p` gameplay parity; all new mapgen balancing logic must be overflow-only (`connectedPlayers > 4`). +- Use sweep confidence policy consistently: `runs=3` for directional iteration, `runs=5` for volatility gate/promotion decisions. +- Promotion requires both simulated and full-lobby confirmation: integration/single-runtime can pass while full-lobby diverges (observed in pass15g level-1 comparison), so do not skip full-lobby gates. +- Keep mapgen tuning explainable and measurable via telemetry (`rooms`, `monsters`, `gold`, `items`, `decorations`, food metrics, value metrics, seed/regeneration diagnostics). +- Current target bands for `p15 vs p4` reviews: + - rooms `1.62x-1.75x` + - monsters `1.38x-1.46x` + - monsters/room `0.82x-0.92x` + - gold/player `0.70x-0.80x` + - items/player `0.70x-0.80x` + - food/player `0.65x-0.78x` + - decorations `1.85x-2.25x`, blocking share `<= 45%` +- Maintain integration ownership boundaries: integration parser/validator/runner belong in smoke hook implementation files (`src/smoke/SmokeHooksMapgen.cpp`); `src/game.cpp` remains wiring-only. +- Keep operational hygiene between long runs: prune generated `models.cache`, and terminate stale `smoke_runner.py` lane processes plus `barony` before relaunch. + +### Technical Commands and Config Reference +- Windows overlay packaging after staging fresh release builds: +```powershell +powershell -ExecutionPolicy Bypass -File scripts\mod_release\package_windows_release.ps1 ` + -Label 20260314-195941 ` + -SteamBuildDir build-vs2022-x64-release-steam ` + -NoDrmBuildDir build-vs2022-x64-release-nodrm +``` +- Smoke-enabled build (required for `[SMOKE]` hooks/logs): +```bash +cmake -S . -B build-mac-smoke -G Ninja -DFMOD_ENABLED=OFF -DBARONY_SMOKE_TESTS=ON +cmake --build build-mac-smoke -j8 --target barony +``` +- Preferred mapgen tuning loop commands: + - Fast in-process integration preflight (`runs=2`): +```bash +USER_HOME="$HOME" +OUT="tests/smoke/artifacts/mapgen-integration-preflight-$(date +%Y%m%d-%H%M%S)" +mkdir -p "$OUT/home" +HOME="$OUT/home" build-mac-smoke/barony.app/Contents/MacOS/barony \ + -windowed -size=1280x720 -nosound \ + -datadir="$USER_HOME/Library/Application Support/Steam/steamapps/common/Barony/Barony.app/Contents/Resources" \ + -smoke-mapgen-integration \ + -smoke-mapgen-integration-csv="$OUT/mapgen_level_matrix.csv" \ + -smoke-mapgen-integration-levels=1,7,16,33 \ + -smoke-mapgen-integration-min-players=1 \ + -smoke-mapgen-integration-max-players=15 \ + -smoke-mapgen-integration-runs=2 +``` + - Volatility gate (`runs=5`): same command with `-smoke-mapgen-integration-runs=5`. + - Low-player parity guard (`2..4p`): same command with `-smoke-mapgen-integration-min-players=2`, `-smoke-mapgen-integration-max-players=4`, `-smoke-mapgen-integration-runs=5`. + - Scripted single-runtime matrix lane (`simulate-mapgen-players=1`, `runs=2/5`): +```bash +OUT="tests/smoke/artifacts/mapgen-level-matrix-passNN-$(date +%Y%m%d-%H%M%S)" +python3 tests/smoke/smoke_runner.py mapgen-level-matrix \ + --app "build-mac-smoke/barony.app/Contents/MacOS/barony" \ + --datadir "$HOME/Library/Application Support/Steam/steamapps/common/Barony/Barony.app/Contents/Resources" \ + --levels "1,7,16,33" \ + --min-players 1 --max-players 15 --runs-per-player 2 \ + --simulate-mapgen-players 1 --inprocess-sim-batch 1 --inprocess-player-sweep 1 \ + --mapgen-reload-same-level 1 \ + --outdir "$OUT" +``` + - Full-lobby confirmation (`simulate-mapgen-players=0`): +```bash +python3 tests/smoke/smoke_runner.py mapgen-sweep \ + --min-players 1 --max-players 15 --runs-per-player 5 \ + --simulate-mapgen-players 0 --auto-enter-dungeon 1 \ + --outdir "tests/smoke/artifacts/mapgen-full-posttune-$(date +%Y%m%d-%H%M%S)" +``` +- Additional validation lane commands: + - Same-level mapgen regeneration sanity lane (procedural floor): +```bash +python3 tests/smoke/smoke_runner.py lan-helo-chunk \ + --instances 1 --auto-start 1 --auto-start-delay 0 \ + --auto-enter-dungeon 1 --auto-enter-dungeon-delay 3 \ + --mapgen-samples 3 --require-mapgen 1 \ + --mapgen-reload-same-level 1 --mapgen-reload-seed-base 100100 \ + --start-floor 1 \ + --outdir "tests/smoke/artifacts/reload-procedural-verify-$(date +%Y%m%d-%H%M%S)" +``` + - Churn/rejoin retry investigation (`error code 16`): +```bash +python3 tests/smoke/smoke_runner.py join-leave-churn \ + --instances 8 --churn-cycles 3 --churn-count 2 \ + --force-chunk 1 --chunk-payload-max 200 \ + --auto-ready 1 --trace-ready-sync 1 --require-ready-sync 1 \ + --trace-join-rejects 1 \ + --outdir "tests/smoke/artifacts/churn-retry-investigation-$(date +%Y%m%d-%H%M%S)" +``` + - Optional historical Steam handshake lane: +```bash +python3 tests/smoke/smoke_runner.py lan-helo-chunk \ + --network-backend steam --instances 2 \ + --force-chunk 1 --chunk-payload-max 200 --timeout 360 \ + --outdir "tests/smoke/artifacts/steam-handshake-multiacct-$(date +%Y%m%d-%H%M%S)" +``` +- Technical config/guardrails: + - Use procedural floors for balancing sweeps (`1,7,16,33`); fixed/story floors may report `MAPGEN_WAIT_REASON=reload-complete-no-mapgen-samples`. + - Keep smoke-run homes isolated (`HOME="$OUT/home"`) to avoid cross-run config/data leakage. + - Integration seed root is now auto-generated per invocation; do not rely on the removed `-smoke-mapgen-integration-base-seed` override. + - Maintain compile-time/runtime gating with `BARONY_SMOKE_TESTS`; keep non-smoke gameplay paths clean. + - Keep integration parser/validator/runner in `src/smoke/SmokeHooksMapgen.cpp` (API in `src/smoke/SmokeTestHooks.hpp`); keep `src/game.cpp` wiring-only for `-smoke-mapgen-integration*`. +- Post-run hygiene commands: +```bash +find tests/smoke/artifacts -type f -name models.cache -delete +ps -Ao pid,ppid,etime,command | rg "smoke_runner.py (mapgen-level-matrix|mapgen-sweep|lan-helo-chunk)|barony.app/Contents/MacOS/barony" +``` diff --git a/CMakeLists.txt b/CMakeLists.txt index 4e9ed8fb08..9beb9f3f07 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1,4 +1,4 @@ -cmake_minimum_required(VERSION 3.0) +cmake_minimum_required(VERSION 3.10) project(barony) if (NOT CMAKE_BUILD_TYPE) @@ -17,6 +17,34 @@ if (NOT DEFINED STEAMWORKS_ENABLED) endif() endif() +if (DEFINED ENV{BARONY_SUPER_MULTIPLAYER} AND NOT DEFINED BARONY_SUPER_MULTIPLAYER) + set (BARONY_SUPER_MULTIPLAYER $ENV{BARONY_SUPER_MULTIPLAYER}) +endif() + +if (NOT DEFINED BARONY_SUPER_MULTIPLAYER) + option(BARONY_SUPER_MULTIPLAYER "Enable 15-Player Multiplayer" ON) +endif() + +if (BARONY_SUPER_MULTIPLAYER) + set (BARONY_SUPER_MULTIPLAYER 1) +else() + set (BARONY_SUPER_MULTIPLAYER 0) +endif() + +if (DEFINED ENV{BARONY_SMOKE_TESTS} AND NOT DEFINED BARONY_SMOKE_TESTS) + set (BARONY_SMOKE_TESTS $ENV{BARONY_SMOKE_TESTS}) +endif() + +if (NOT DEFINED BARONY_SMOKE_TESTS) + option(BARONY_SMOKE_TESTS "Compile smoke-test hooks and smoke instrumentation call sites" OFF) +endif() + +if (BARONY_SMOKE_TESTS) + set (BARONY_SMOKE_TESTS 1) +else() + set (BARONY_SMOKE_TESTS 0) +endif() + if (DEFINED ENV{OPTIMIZATION_LEVEL}) set (OPTIMIZATION_LEVEL $ENV{OPTIMIZATION_LEVEL}) else() @@ -193,7 +221,13 @@ endif() set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/cmake/Modules/") -set(CMAKE_OSX_ARCHITECTURES x86_64) +if (APPLE AND NOT DEFINED CMAKE_OSX_ARCHITECTURES) + if (CMAKE_HOST_SYSTEM_PROCESSOR MATCHES "arm64|aarch64") + set(CMAKE_OSX_ARCHITECTURES arm64 CACHE STRING "Build architectures for macOS") + else() + set(CMAKE_OSX_ARCHITECTURES x86_64 CACHE STRING "Build architectures for macOS") + endif() +endif() if (EDITOR_ENABLED) set(EDITOR_EXE_NAME "editor" CACHE STRING "Editor executable name") @@ -206,6 +240,18 @@ else() message("Building without steamworks") endif() +if (BARONY_SUPER_MULTIPLAYER) + message("Building with Barony Super Multiplayer") +else() + message("Building without Barony Super Multiplayer") +endif() + +if (BARONY_SMOKE_TESTS) + message("Building with smoke-test instrumentation") +else() + message("Building without smoke-test instrumentation") +endif() + if (EOS_ENABLED) message("Building with EOS") else() @@ -403,11 +449,12 @@ if (FMOD_ENABLED) set(FMOD 1) endif() endif() -if(NOT APPLE) - #Linux needs these two. - #Macs have their own special line. - find_package(PNG REQUIRED) -elseif (APPLE) +find_package(PNG REQUIRED) +if (WIN32) + find_package(GLEW REQUIRED) + include_directories(${GLEW_INCLUDE_DIR}) +endif() +if (APPLE) set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${FMOD_CXX_FLAGS}") MESSAGE("CMAKE_CXX_FLAGS: ${CMAKE_CXX_FLAGS}") set (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${FMOD_LINK_FLAGS}") @@ -462,12 +509,11 @@ endif() if( NOT WIN32 ) include_directories(${THREADS_INCLUDE_DIR}) endif() -if(NOT APPLE) - include_directories(${PNG_INCLUDE_DIR}) -endif() +include_directories(${PNG_INCLUDE_DIR}) if (FMOD_FOUND) include_directories(${FMOD_INCLUDE_DIR}) endif() +include_directories(BEFORE ${PROJECT_BINARY_DIR}) # Add the source directory #file(GLOB_RECURSE SOURCE_FILES src/*.c src/*.h) # Can't do this because it'd then compile barony and editor together. Which just won't work. @@ -481,7 +527,7 @@ endif() #add_subdirectory(books) -configure_file ( "${PROJECT_SOURCE_DIR}/src/Config.hpp.in" "${PROJECT_SOURCE_DIR}/src/Config.hpp") +configure_file("${PROJECT_SOURCE_DIR}/src/Config.hpp.in" "${PROJECT_BINARY_DIR}/Config.hpp") message("***************************") message("Debug flags: ") @@ -512,7 +558,8 @@ if (APPLE) #add_executable(barony MACOSX_BUNDLE OSX/SDLmain.m ${GAME_SOURCES} ${COPY_FRAMEWORKS} libfmodex.dylib /opt/local/lib/libpng16.16.dylib) #add_executable(barony MACOSX_BUNDLE OSX/SDLmain.m ${GAME_SOURCES} ${MACOSX_BUNDLE_ICON_FILE} libfmodex.dylib /opt/local/lib/libpng16.16.dylib) if (GAME_ENABLED) - add_executable(barony MACOSX_BUNDLE ${GAME_SOURCES} ${MACOSX_BUNDLE_ICON_FILE} ${PROJECT_SOURCE_DIR}/libpng16.16.dylib) + add_executable(barony MACOSX_BUNDLE ${GAME_SOURCES} ${MACOSX_BUNDLE_ICON_FILE}) + set_target_properties(barony PROPERTIES OUTPUT_NAME "Barony") #add_executable(barony OSX/SDLmain.m ${GAME_SOURCES}) #add_executable(barony ${GAME_SOURCES}) #SET_SOURCE_FILES_PROPERTIES(${COPY_FRAMEWORKS} PROPERTIES MACOSX_PACKAGE_LOCATION Frameworks) @@ -525,7 +572,6 @@ if (APPLE) #SET_SOURCE_FILES_PROPERTIES(${SOUND_FILES} PROPERTIES MACOSX_PACKAGE_LOCATION Resources/sound/) set_source_files_properties(${GAME_SOURCES} PROPERTIES COMPILE_FLAGS "-x objective-c++") endif(GAME_ENABLED) - SET_SOURCE_FILES_PROPERTIES(${PROJECT_SOURCE_DIR}/libpng16.16.dylib PROPERTIES MACOSX_PACKAGE_LOCATION MacOS) SET_SOURCE_FILES_PROPERTIES(${COPY_RESOURCES} PROPERTIES MACOSX_PACKAGE_LOCATION Resources) SET_SOURCE_FILES_PROPERTIES(${MACOSX_BUNDLE_ICON_FILE} PROPERTIES MACOSX_PACKAGE_LOCATION "Resources") #set_source_files_properties(${GAME_SOURCES} PROPERTIES COMPILE_FLAGS "-stdlib=libc++") @@ -553,7 +599,12 @@ if (GAME_ENABLED) # endif() set_property(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} PROPERTY VS_STARTUP_PROJECT barony) # Otherwise, hitting "debug" will default to "ALL_BUILD" :| if (DEFINED BARONY_DATADIR) - set_target_properties(barony PROPERTIES VS_DEBUGGER_WORKING_DIRECTORY "\$(BARONY_DATADIR)/") + set_target_properties(barony PROPERTIES VS_DEBUGGER_WORKING_DIRECTORY "${BARONY_DATADIR}/") + else() + set_target_properties(barony PROPERTIES VS_DEBUGGER_WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}/") + endif() + if (SDL2MAIN_LIBRARY) + target_link_libraries(barony ${SDL2MAIN_LIBRARY}) endif() target_link_libraries(barony ${SDL2_LIBRARIES} ${SDL2_LIBRARY} ${SDL2IMAGE_LIBRARIES} ${SDL2_IMAGE_LIBRARIES} ${SDL2_NET_LIBRARIES} ${SDL2_TTF_LIBRARIES} ${SDL2TTF_LIBRARY}) if (STEAMWORKS_ENABLED) @@ -608,9 +659,6 @@ if (GAME_ENABLED) if (EOS_ENABLED) target_link_libraries(barony ${EOS_LIBRARIES}) endif() - if (APPLE) - target_link_libraries(barony ${PROJECT_SOURCE_DIR}/libpng16.16.dylib) #Wait...what? if(APPLE) in if(WIN32)? What was I thinking back then, haha. - endif() if (${CMAKE_SYSTEM_NAME} MATCHES "BSD|DragonFly" OR ${CMAKE_SYSTEM_NAME} MATCHES "Haiku") # For backtrace find_path(EXECINFO_INC NAMES execinfo.h) @@ -624,24 +672,23 @@ if (GAME_ENABLED) endif() target_link_libraries(barony ${OPENGL_LIBRARIES}) target_link_libraries(barony ${THREADS_LIBRARIES}) # TODO: Not sure what this is doing, since it doesn't appear to be working in Linux! - target_link_libraries(barony -lm) # TODO: In Visual Studio, I bet this is something else, right? - if (NOT CMAKE_SYSTEM_NAME MATCHES "Haiku") - target_link_libraries(barony -lc) + if (NOT WIN32) + target_link_libraries(barony -lm) # TODO: In Visual Studio, I bet this is something else, right? + if (NOT CMAKE_SYSTEM_NAME MATCHES "Haiku") + target_link_libraries(barony -lc) + endif() endif() if (FMOD_FOUND) target_link_libraries(barony ${FMOD_LIBRARY}) endif() target_link_libraries(barony ${PHYSFS_LIBRARY}) - if(NOT APPLE) - #Remember, Mac isn't using find_package for PNG. - target_link_libraries(barony ${PNG_LIBRARY}) - endif() + target_link_libraries(barony ${PNG_LIBRARIES}) if (APPLE) target_link_libraries(barony ${IOKit_LIBRARY}) endif() # We need to link to Winsock if we're on Windows if(WIN32) - target_link_libraries(barony wsock32 ws2_32) + target_link_libraries(barony wsock32 ws2_32 ${GLEW_LIBRARIES}) #target_link_libraries(barony -lpng -lfmodex) Mingw, not Visual Studio. endif() target_link_libraries(barony ${EXTRA_LIBS}) #Apple needs this for OpenGL to work. @@ -689,14 +736,18 @@ if (EDITOR_ENABLED) #add_executable(editor MACOSX_BUNDLE OSX/SDLmain.m ${EDITOR_SOURCES} ${COPY_FRAMEWORKS} libfmodex.dylib /opt/local/lib/libpng16.16.dylib) #add_executable(editor MACOSX_BUNDLE OSX/SDLmain.m ${EDITOR_SOURCES} libfmodex.dylib /opt/local/lib/libpng16.16.dylib) set_source_files_properties("editor.icns" PROPERTIES MACOSX_PACKAGE_LOCATION "Resources") - add_executable(${EDITOR_EXE_NAME} MACOSX_BUNDLE "editor.icns" ${EDITOR_SOURCES} ${PROJECT_SOURCE_DIR}/libpng16.16.dylib) + add_executable(${EDITOR_EXE_NAME} MACOSX_BUNDLE "editor.icns" ${EDITOR_SOURCES}) #SET_SOURCE_FILES_PROPERTIES(${MACOSX_BUNDLE_ICON_FILE} PROPERTIES MACOSX_PACKAGE_LOCATION "Resources") set_source_files_properties(${EDITOR_SOURCES} PROPERTIES COMPILE_FLAGS "-x objective-c++") else() if (WIN32) # add_executable(${EDITOR_EXE_NAME} EXCLUDE_FROM_ALL ${EDITOR_SOURCES} ${EDITOR_RESOURCE_FILE}) add_executable(${EDITOR_EXE_NAME} ${EDITOR_SOURCES} ${EDITOR_RESOURCE_FILE}) - set_target_properties(${EDITOR_EXE_NAME} PROPERTIES VS_DEBUGGER_WORKING_DIRECTORY "\$(BARONY_DATADIR)/") + if (DEFINED BARONY_DATADIR) + set_target_properties(${EDITOR_EXE_NAME} PROPERTIES VS_DEBUGGER_WORKING_DIRECTORY "${BARONY_DATADIR}/") + else() + set_target_properties(${EDITOR_EXE_NAME} PROPERTIES VS_DEBUGGER_WORKING_DIRECTORY "${CMAKE_SOURCE_DIR}/") + endif() else() add_executable(${EDITOR_EXE_NAME} ${EDITOR_SOURCES} ${EDITOR_RESOURCE_FILE}) endif() @@ -715,6 +766,9 @@ if (EDITOR_ENABLED) #target_link_libraries(${EDITOR_EXE_NAME} OpenSSL::SSL) #target_link_libraries(${EDITOR_EXE_NAME} OpenSSL::Crypto) endif() + if (WIN32 AND SDL2MAIN_LIBRARY) + target_link_libraries(${EDITOR_EXE_NAME} ${SDL2MAIN_LIBRARY}) + endif() target_link_libraries(${EDITOR_EXE_NAME} ${SDL2_LIBRARIES} ${SDL2_LIBRARY} ${SDL2IMAGE_LIBRARIES} ${SDL2_IMAGE_LIBRARIES} ${SDL2_NET_LIBRARIES} ${SDL2_TTF_LIBRARIES} ${SDL2TTF_LIBRARY}) if(STEAMWORKS_ENABLED) target_link_libraries(${EDITOR_EXE_NAME} ${STEAMWORKS_LIBRARY}) @@ -735,28 +789,26 @@ if (EDITOR_ENABLED) endif() target_link_libraries(${EDITOR_EXE_NAME} ${OPENGL_LIBRARIES}) target_link_libraries(${EDITOR_EXE_NAME} ${THREADS_LIBRARIES}) - target_link_libraries(${EDITOR_EXE_NAME} -lm) # TODO: Only for Mingw? - if (NOT CMAKE_SYSTEM_NAME MATCHES "Haiku") - target_link_libraries(${EDITOR_EXE_NAME} -lc) + if (NOT WIN32) + target_link_libraries(${EDITOR_EXE_NAME} -lm) # TODO: Only for Mingw? + if (NOT CMAKE_SYSTEM_NAME MATCHES "Haiku") + target_link_libraries(${EDITOR_EXE_NAME} -lc) + endif() endif() # We need to link to Winsock if we're on Windows if (FMOD_FOUND) target_link_libraries(${EDITOR_EXE_NAME} ${FMOD_LIBRARY}) endif() - if(NOT APPLE) - #Remember, Mac isn't using find_package for FMOD and PNG. - target_link_libraries(${EDITOR_EXE_NAME} ${PHYSFS_LIBRARY}) - target_link_libraries(${EDITOR_EXE_NAME} ${PNG_LIBRARY}) - endif() + target_link_libraries(${EDITOR_EXE_NAME} ${PHYSFS_LIBRARY}) + target_link_libraries(${EDITOR_EXE_NAME} ${PNG_LIBRARY}) if (APPLE) - target_link_libraries(${EDITOR_EXE_NAME} ${PHYSFS_LIBRARY}) target_link_libraries(${EDITOR_EXE_NAME} ${IOKit_LIBRARY}) if (EOS_ENABLED) target_link_libraries(${EDITOR_EXE_NAME} ${EOS_LIBRARY}) endif() endif() if(WIN32) - target_link_libraries(${EDITOR_EXE_NAME} wsock32 ws2_32) + target_link_libraries(${EDITOR_EXE_NAME} wsock32 ws2_32 ${GLEW_LIBRARIES}) endif() if (APPLE) if (FMOD_ENABLED) diff --git a/HELO_ONLY_CHUNKING_PLAN.md b/HELO_ONLY_CHUNKING_PLAN.md new file mode 100644 index 0000000000..9bee59bbd1 --- /dev/null +++ b/HELO_ONLY_CHUNKING_PLAN.md @@ -0,0 +1,287 @@ +# HELO-Only Chunking Plan (Stable 16-Player Savegame Join Handshake) + +## Summary +This plan introduces **HELO-only chunking** for oversized lobby join snapshots while preserving existing behavior for all other packets. + +The goal is to eliminate oversized single-packet HELO joins (especially savegame joins with `MAXPLAYERS=16`) and reduce join fragility from UDP fragmentation. + +Target repo: `/Users/sayhiben/dev/Barony-8p` + +Primary touched files: +1. `/Users/sayhiben/dev/Barony-8p/src/ui/MainMenu.cpp` +2. `/Users/sayhiben/dev/Barony-8p/src/net.cpp` +3. `/Users/sayhiben/dev/Barony-8p/src/net.hpp` + +## Scope +1. Chunk only server->client HELO success payloads when large. +2. Keep existing `HELO` (single-packet) path intact for small payloads and fallback. +3. Keep existing failure/error HELO behavior unchanged. +4. Do not modify non-HELO protocols (`JOIN`, `REDY`, gameplay packets, etc.) except adding one `JOIN` capability byte. + +## Non-Goals +1. No generic packet chunking framework in this iteration. +2. No changes to gameplay packet cadence or entity sync protocol. +3. No migration of existing non-HELO chunk flows (`CSCN`, `CMPD`) to shared utilities. + +## Public Interface / Protocol Changes +1. `JOIN` request adds one capability byte. +2. New packet ID for HELO chunks: `'HLCN'`. +3. `lobbyPlayerJoinRequest` signature extends to expose whether HELO chunking should be used for this accepted join. + +### 1) JOIN capability extension +- Current `JOIN` length: `69`. +- New `JOIN` length: `70`. +- Byte `69` = capability bitfield. +- Bit `0` (`0x01`) means: `client supports HLCN HELO chunk reassembly`. + +### 2) New packet format (`'HLCN'`) +Header layout: +1. `[0..3]` `Uint32`: `'HLCN'` +2. `[4..5]` `Uint16`: `transferId` +3. `[6]` `Uint8`: `chunkIndex` (0-based) +4. `[7]` `Uint8`: `chunkCount` +5. `[8..9]` `Uint16`: `totalHeloLen` (full reassembled HELO payload length) +6. `[10..11]` `Uint16`: `chunkLen` +7. `[12..]` bytes: `chunk payload` + +Constraints: +1. `chunkCount >= 1` +2. `chunkIndex < chunkCount` +3. `totalHeloLen <= NET_PACKET_SIZE` +4. `12 + chunkLen == net_packet->len` + +### 3) Function signature update +Update declaration and definition: +- From: + - `NetworkingLobbyJoinRequestResult lobbyPlayerJoinRequest(int& outResult, bool lockedSlots[MAXPLAYERS]);` +- To: + - `NetworkingLobbyJoinRequestResult lobbyPlayerJoinRequest(int& outResult, bool lockedSlots[MAXPLAYERS], bool& outUseChunkedHelo);` + +## Design Decisions (Risk-Managed) +1. **Capability-gated chunking** + - Server only sends chunked HELO when client advertises support in `JOIN` byte 69 bit 0. + - If client does not advertise support, server sends legacy HELO. +2. **Chunking threshold** + - Use chunking when HELO payload length exceeds conservative non-fragment target. + - Default: `kHeloSinglePacketMax = 1100` bytes. +3. **Transport reliability** + - For chunked HELO sends, use `sendPacketSafe(...)` for both direct and P2P success paths. + - This leverages existing `SAFE/GOTP` ack path and retries. +4. **Transfer identity** + - Include `transferId` to prevent mixing stale chunks across retries/rejoins. +5. **Strict validation at client** + - Reject malformed chunks immediately. + - Do not partially apply invalid snapshot. +6. **No behavior change for non-chunk HELO** + - Preserve existing flow for compatibility and low regression risk. + +## Detailed Implementation Plan + +### A) Add constants and helper data structures +In `/Users/sayhiben/dev/Barony-8p/src/ui/MainMenu.cpp` and `/Users/sayhiben/dev/Barony-8p/src/net.cpp`: + +1. Add shared constants (local static in each translation unit if needed): + - `kJoinCapabilityHeloChunkV1 = 0x01` + - `kHeloChunkHeaderSize = 12` + - `kHeloChunkPayloadMax = 900` + - `kHeloSinglePacketMax = 1100` + - `kHeloChunkReassemblyTimeoutTicks = 5 * TICKS_PER_SECOND` +2. Add a per-player transfer counter on server side: + - `static Uint16 g_heloTransferId[MAXPLAYERS] = {0};` + +### B) Update JOIN request writer (client) +In `/Users/sayhiben/dev/Barony-8p/src/ui/MainMenu.cpp` `sendJoinRequest()`: + +1. Set `net_packet->data[69] = kJoinCapabilityHeloChunkV1`. +2. Change `net_packet->len = 70`. + +Code example: + +```cpp +memcpy(net_packet->data, "JOIN", 4); +// existing fields... +SDLNet_Write32(loadinglobbykey, &net_packet->data[65]); +net_packet->data[69] = kJoinCapabilityHeloChunkV1; +net_packet->len = 70; +``` + +### C) Update JOIN request parser / decision point (server) +In `/Users/sayhiben/dev/Barony-8p/src/net.cpp` `lobbyPlayerJoinRequest(...)`: + +1. Read client capability: + - `bool clientSupportsHeloChunk = (net_packet->len >= 70) && (net_packet->data[69] & kJoinCapabilityHeloChunkV1);` +2. Build HELO payload exactly as today (legacy bytes unchanged). +3. Compute `outUseChunkedHelo`: + - `outUseChunkedHelo = clientSupportsHeloChunk && loadingsavegame && (net_packet->len > kHeloSinglePacketMax);` +4. For direct-connect success path: + - If `outUseChunkedHelo` true, send chunked HELO directly from here. + - Else keep existing single `sendPacketSafe`. + +### D) Add server-side HELO chunk sender helper +Implement helper in `/Users/sayhiben/dev/Barony-8p/src/net.cpp` (or `MainMenu.cpp` static if preferred): + +1. Input: + - `hostnum`, destination address, source HELO bytes, source HELO length, `transferId`. +2. Split HELO payload into chunks of `kHeloChunkPayloadMax`. +3. For each chunk: + - Write `'HLCN'` header. + - Copy chunk bytes. + - Set `net_packet->len = kHeloChunkHeaderSize + chunkLen`. + - Send with `sendPacketSafe(...)`. + +Code example: + +```cpp +static void sendChunkedHeloToHost(int hostnum, const Uint8* heloData, int heloLen, Uint16 transferId) +{ + const int chunkCount = (heloLen + kHeloChunkPayloadMax - 1) / kHeloChunkPayloadMax; + for (int chunkIndex = 0; chunkIndex < chunkCount; ++chunkIndex) + { + const int offset = chunkIndex * kHeloChunkPayloadMax; + const int chunkLen = std::min(kHeloChunkPayloadMax, heloLen - offset); + + memcpy(net_packet->data, "HLCN", 4); + SDLNet_Write16(transferId, &net_packet->data[4]); + net_packet->data[6] = static_cast(chunkIndex); + net_packet->data[7] = static_cast(chunkCount); + SDLNet_Write16(static_cast(heloLen), &net_packet->data[8]); + SDLNet_Write16(static_cast(chunkLen), &net_packet->data[10]); + memcpy(&net_packet->data[12], heloData + offset, chunkLen); + net_packet->len = 12 + chunkLen; + + sendPacketSafe(net_sock, -1, net_packet, hostnum); + } +} +``` + +### E) P2P success-path changes in MainMenu server join handling +In `/Users/sayhiben/dev/Barony-8p/src/ui/MainMenu.cpp` where `lobbyPlayerJoinRequest(...)` result is processed: + +1. Capture `outUseChunkedHelo`. +2. Keep existing P2P failure behavior unchanged. +3. On P2P success: + - Keep current remote-ID assignment (`steamIDRemote`/EOS peer index). + - If `outUseChunkedHelo` true: + - Copy current HELO payload to local temp buffer. + - Increment/use `transferId` for this player. + - Send chunked HELO via `sendPacketSafe` helper using `hostnum = playerNum - 1`. + - Else: + - Keep current legacy single-HELO send behavior. + +### F) Client-side HLCN reassembly (pre-clientnum phase) +In `/Users/sayhiben/dev/Barony-8p/src/ui/MainMenu.cpp` `handlePacketsAsClient()` branch `receivedclientnum == false`: + +1. Add local static reassembly state: + - `active`, `transferId`, `chunkCount`, `totalLen`, `lastChunkTick`. + - `std::vector> chunks`. + - `std::vector received`. +2. When packet is `'HELO'`: + - Clear chunk state. + - Continue existing legacy parse flow unchanged. +3. When packet is `'HLCN'`: + - Validate header and bounds. + - Start or switch reassembly session by `transferId`. + - Store chunk only if first time received for that index. + - Update `lastChunkTick`. + - If all chunks received: + - Reassemble to contiguous buffer. + - Validate reconstructed packet begins with `'HELO'`. + - Copy reconstructed bytes into `net_packet->data`, set `net_packet->len`. + - Set `gotPacket = true` to reuse existing HELO parser. +4. Timeout: + - If active and stale (`ticks - lastChunkTick > kHeloChunkReassemblyTimeoutTicks`), reset state. + +Code example: + +```cpp +if (packetId == 'HELO') +{ + resetHeloChunkState(); + gotPacket = true; +} +else if (packetId == 'HLCN') +{ + if (ingestHeloChunkAndMaybeAssemble(*net_packet)) + { + // ingest function overwrote net_packet with full HELO bytes + gotPacket = true; + } +} +``` + +### G) Validation / guardrails +Add hard checks: + +1. Reject chunk packets where `totalHeloLen > NET_PACKET_SIZE`. +2. Reject chunk packets where `chunkLen` exceeds per-chunk max or packet length mismatch. +3. Reject chunk packets where `chunkCount == 0` or `chunkCount > 32`. +4. Reject out-of-range `chunkIndex`. +5. On any malformed chunk, reset active state and continue waiting (do not crash, do not partially apply). + +### H) Logging (stability-first) +Add `printlog` lines (debug-friendly, concise): + +1. Server: + - `sending chunked HELO: player=%d transfer=%u chunks=%d total=%d` +2. Client: + - `recv HLCN: transfer=%u idx=%u/%u len=%u` + - `HELO reassembled: transfer=%u total=%u` + - `HELO chunk timeout/reset transfer=%u` + +## Test Plan + +### 1) Functional matrix +1. Direct connect, non-savegame join: + - Expect legacy single HELO path unchanged. +2. Direct connect, savegame join, 16 players: + - Expect chunked HELO path when client advertises capability. +3. Steam join, non-savegame: + - Legacy path unchanged. +4. Steam join, savegame 16 players: + - Chunked HELO path; successful join. +5. EOS join, savegame 16 players: + - Chunked HELO path; successful join. +6. Legacy-capability client simulation (no capability bit): + - Server sends legacy HELO fallback. +7. Error HELO path (`MAXPLAYERS + n` codes): + - Ensure unchanged behavior and prompts. + +### 2) Robustness scenarios +1. Inject duplicate HLCN chunks: + - Reassembler ignores duplicates. +2. Out-of-order chunk arrival: + - Reassembler completes successfully. +3. Missing chunk: + - No malformed parse; timeout reset occurs; overall connect flow times out gracefully. +4. Mixed/stale transfer IDs: + - Old transfer chunks ignored/reset correctly. +5. Malformed header fields: + - Packet safely rejected; no crash. + +### 3) Acceptance criteria +1. No HELO handshake packet exceeding chunk threshold is sent when chunking is enabled. +2. Client can reassemble and parse chunked HELO deterministically. +3. Legacy HELO behavior remains unchanged for small payloads and no-capability clients. +4. No regressions in `JOIN`, `REDY`, `DISC`, `SVFL`, or lobby prompt flow. +5. Build success for affected targets. + +## Rollout / Verification Instructions +1. Implement in small commits: + - Commit 1: protocol constants + JOIN capability byte. + - Commit 2: server chunk sender + decision logic. + - Commit 3: client reassembly + timeout + logs. + - Commit 4: cleanup and guard assertions. +2. Run build: + - `cmake --build /Users/sayhiben/dev/Barony-8p/build-mac -j8` +3. Manual smoke tests: + - Host/join for direct + Steam + EOS in both savegame and non-savegame. +4. Keep logs enabled during first multiplayer verification pass. +5. If instability appears, temporarily force legacy HELO by setting chunk decision false server-side while keeping receiver code in place. + +## Assumptions and Defaults +1. This is a **HELO-only** chunking change. +2. Capability negotiation is via `JOIN` byte 69 bit 0. +3. Chunking only triggers for large HELO (default threshold `1100` bytes). +4. Chunk transport uses existing SAFE/GOTP reliability path. +5. Existing legacy parser remains source of truth for final HELO interpretation. +6. No version string bump is required for this patch; capability bit handles handshake behavior selection. diff --git a/INSTALL.md b/INSTALL.md index f63df4103e..1770c342a5 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -1,123 +1,581 @@ -# Dependencies -You will need the following libraries to build Barony: +# Barony Build and Setup Guide - * SDL2 (https://www.libsdl.org/download-2.0.php) - * SDL2_image (https://www.libsdl.org/projects/SDL_image/) - * SDL2_net (https://www.libsdl.org/projects/SDL_net/) - * SDL2_ttf (https://www.libsdl.org/projects/SDL_ttf/) - * libpng (http://www.libpng.org/pub/png/libpng.html) - * libz (https://zlib.net/) used by libpng - * PhysFS - * RapidJSON - * dirent.h (Comes with Linux/POSIX systems; need to acquire on Windows) - * OpenGL - * CMake (on Windows, use versions at least as new as 3.8.0) +This guide is for developers building Barony from source on Windows, macOS, and Linux. -OPTIONAL dependencies: - * FMOD Core API 2.02.14. +## Choose Your Setup Path -FMOD Studio API can be downloaded at https://www.fmod.com/download (you do need an account to download it). -You can disable FMOD by running cmake with -DFMOD_ENABLED=OFF (it's also disabled if not found). +- Windows (recommended): use `setup_barony_vs2022_x64.ps1` interactive installer. +- Windows (fallback): use manual CMake setup only if installer is intentionally skipped or troubleshooting is needed. +- macOS: Homebrew-based full-feature build. +- Linux: Docker full-feature build (recommended) or native package-based build. -OpenAL can be used with -DOPENAL_ENABLED - * Note - OpenAL support has been deprecated and is currently unmaintained. - -You will also need the following tools: +## Core Build Requirements - * A working C++ compiler (Visual Studio, MinGW _(not officially supported, CMakeLists.txt may require modification)_, GCC, Clang, or xtools) - * CMake - * Linux users will also need Make, or whatever alternate you may generate build files for. +Required for all platforms: -# Windows Instructions +- CMake (recent version; tested with `3.10+`, including current `4.x`) +- Git -## Acquire Dependencies +Platform package dependencies (SDL2/OpenGL/libpng/PhysFS/RapidJSON/etc.) are handled in the platform-specific sections below. -PhysFS: - * Download from https://icculus.org/physfs/downloads/physfs-3.0.1.tar.bz2 - * Open with .7zip or similar, open up docs/INSTALL.txt inside the archive for instructions to read along with this short guide. - * Download/install cmake-gui for Windows (or use a command line version if you have it -- you may select "Add CMake to the system PATH for all users/current user" during installation for ease of use) - * Open up the physfs-3.0.1 directory in cmake-gui and select configure, then generate - * You will now get Visual Studio files to open up and build - * Open up files in Visual Studio, in Solution Explorer right click the 'physfs' solution and 'build' - * You will now see a physfs.lib file in physfs-3.0.1/Release/ to use when building Barony. Put this in one of the VCC++ Library Directories folder for the Barony project. - * Similarly you can find physfs-3.0.1/src/physfs.h to put in one of the VC++ Include Directories from the Barony project. +CMake downloads: https://cmake.org/download/ -dirent.h is a POSIX header, you will need to obtain a Windows port. For example, from https://github.com/tronkko/dirent +## Optional Integrations -//TODO: Where get OpenGL? Did you need to install the "Windows SDK" when setting up Visual Studio? +- FMOD Core API (preferred audio path on Windows) +- Steamworks SDK +- Epic Online Services (EOS) SDK +- CURL + OpenSSL (workshop preview HTTP downloads) +- Opus (encoded voice chat) +- TheoraPlayer + Ogg/Vorbis/Theora (scripted video playback) +- PlayFab SDK (not automated by the Windows setup script) -Download everything else. You may need to build things. More explicit instructions here would be nice. -### TODO: vcpkg something or other +# Windows 11 + Visual Studio (2022-Compatible) -## Building Barony +Use the PowerShell installer by default. It handles dependency installation, configuration, build, and `INSTALL` target execution. +In automated Windows setup, open-source build dependencies are installed by the script via vcpkg. -Visual Studio Instructions: -* Rather than individually setting up environment variables for every dependency, you can simply create an environment variable named `BARONY_WIN32_LIBRARIES` and point it to a combined dependencies folder. Said folder should have a subdirectory named `include` with all libraries' header files in there, and a `lib` subdirectory for the library archives themselves. -* After that, create a directory named `build` (or somesuch) in the root Barony directory, and either run `cmake ..` inside it from a command prompt, or use cmake-gui. - * There are some additional options you can specify, such as -DFMOD_ENABLED, -DSTEAMWORKS_ENABLED, -DEOS_ENABLED, -DOPENAL_ENABLED. - * If you do not specify one of FMOD or OpenAL enabled, the game will build without sound support. -* Open barony.sln and the standard Visual Studio experience is now all yours. - * Build the whole solution to generate the .exe files. (Make sure that the appropriate Platform Toolset is installed) - * If the environment variable `BARONY_DATADIR` is defined, it will be used as the current working directory. Set this to wherever you have all of the game's assets and just mash the "play" button in Visual Studio :) - * If that variable doesn't exist, no worries, CMake will not modify the current working directory property. You'll either have to copy the executables to whever you have the assets yourself, or you can copy the assets over to the debugger's directory. -* MinGW probably not supported right now in the CMakeList.txt...PRs welcome :) -TODO: MinGW support, one master SLN with Steam, EOS, FMOD, OpenAL, etc variants all in one place. +## Automated Setup (Recommended) -If you're using MinGW or GCC, you'll need to run CMake first, then make: cmake . && make *(NOTE: Not supported, CMakeLists.txt is probably broken for MinGW support)* +### 1. Install tools -If you are missing GL header files like glext.h they are available from https://www.khronos.org/registry/OpenGL/api/GL/. +- Visual Studio with `Desktop development with C++` (downloads: https://visualstudio.microsoft.com/downloads/) +- Minimum compatibility target: VS 2022 toolchain (`Visual Studio 17 2022`, MSVC v143) +- If you use VS 2026, also install VS 2022-compatible C++ build tools/toolset +- CMake (recent version): https://cmake.org/download/ +- Git: https://git-scm.com/download/win +- PowerShell 5+ or PowerShell 7+: https://learn.microsoft.com/powershell/scripting/install/installing-powershell-on-windows -### Running Barony in Visual Studio. +Verify CMake is available: -By default, the "barony" project is set as the startup project. (The startup project is the project Visual Studio will launch when you hit the green "play" (debug) button) +```powershell +cmake --version +``` + +### 2. Place licensed SDKs for the integrations you enable + +These cannot be auto-downloaded by script due to licensing/account requirements. + +Download sources: + +- FMOD: https://www.fmod.com/download +- Steamworks SDK: https://partner.steamgames.com/downloads/list +- EOS SDK: https://onlineservices.epicgames.com/en-US/sdk + +Wizard defaults and required manual SDK layouts: + +| Integration | Wizard default | Manual SDK needed | Required files | +| --- | --- | --- | --- | +| FMOD | ON | Yes | `deps\fmod\api\core\inc\fmod.hpp`, `deps\fmod\api\core\lib\x86_64\fmod_vc.lib` | +| Steamworks | ON | Yes | `deps\steamworks\sdk\public\steam\steam_api.h`, `deps\steamworks\sdk\redistributable_bin\win64\steam_api64.lib` | +| EOS | OFF | Only if enabled | `deps\eos\SDK\Include\eos_sdk.h`, `deps\eos\SDK\Lib\EOSSDK-Win64-Shipping.lib` | +| TheoraPlayer | ON | No (script auto-builds) | Script generates `deps\theoraplayer\include`, `deps\theoraplayer\lib`, `deps\theoraplayer\bin` | + +Notes: + +- If FMOD ships `api\core\lib\x64`, the installer mirrors it to `x86_64`. +- Runtime DLLs (`fmod.dll`, `steam_api64.dll`, `theoraplayer.dll`, etc.) must be next to the executable for local runs. + +### 3. Run the installer wizard + +Before running commands: + +- Open PowerShell from the Windows Start menu (do not run these commands inside Visual Studio's terminal). +- Administrator privileges are not required for the standard workflow. +- Use **Run as administrator** only if you intentionally need elevated writes (for example, installing to protected system directories like `C:\Program Files`). + +Then change to the repository root and run: + +```powershell +cd "path\to\barony\repository" +Set-ExecutionPolicy -Scope Process Bypass +.\setup_barony_vs2022_x64.ps1 +``` + +What the wizard prompts for: + +- Feature toggles (`FMOD`, `Steamworks`, `EOS`, `CURL`, `Opus`, `TheoraPlayer`) +- `BARONY_DATADIR` auto-discovery/manual entry +- Re-check loop for missing manually placed SDK files +- Environment variable persistence +- EOS token values (only if EOS is enabled) + +Non-interactive mode (CI/automation) is available via `-NonInteractive`. +Environment variable persistence is enabled by default; to disable it, use `-NoPersistUserEnvironment`. + +### 4. Expected outputs and quick validation + +Expected artifacts: + +- Solution: `build-vs2022-x64\barony.sln` +- Game executable: typically under `build-vs2022-x64\Release\barony.exe` +- Editor executable: typically under `build-vs2022-x64\Release\editor.exe` + +If `BARONY_DATADIR` is set, installer `INSTALL` step copies runtime files there. + +### 5. EOS optional vs enabled + +If you do not have EOS credentials, keep EOS disabled (default). + +If EOS is enabled, CMake requires: + +- `BUILD_ENV_PR` +- `BUILD_ENV_SA` +- `BUILD_ENV_DE` +- `BUILD_ENV_CC` +- `BUILD_ENV_CS` +- `BUILD_ENV_GSE` + +## Manual Setup (Only if skipping installer or troubleshooting installer failures) + +Use out-of-source builds and pick one of the command sets below. +Run these in PowerShell opened from Start menu, after changing to the repository root: + +```powershell +cd "path\to\barony\repository" +``` + +### 0. Prepare vcpkg dependencies + +```powershell +if (-not (Test-Path "$PWD\deps\vcpkg\vcpkg.exe")) { + git clone --depth 1 https://github.com/microsoft/vcpkg "$PWD\deps\vcpkg" + cmd /c "$PWD\deps\vcpkg\bootstrap-vcpkg.bat -disableMetrics" +} + +& "$PWD\deps\vcpkg\vcpkg.exe" install ` + sdl2 sdl2-image sdl2-net sdl2-ttf ` + physfs rapidjson libpng zlib dirent glew ` + openssl curl[openssl] opus ` + libogg libvorbis libtheora nativefiledialog-extended ` + --triplet x64-windows +``` + +### A. Baseline manual build (minimal integrations) + +```powershell +$env:BARONY_WIN32_LIBRARIES = "$PWD\deps\vcpkg\installed\x64-windows" +$env:EOS_ENABLED = "0" + +cmake -S . -B build -G "Visual Studio 17 2022" -A x64 ` + -DCMAKE_POLICY_VERSION_MINIMUM=3.10 ` + -DCMAKE_TOOLCHAIN_FILE="$PWD\deps\vcpkg\scripts\buildsystems\vcpkg.cmake" ` + -DVCPKG_TARGET_TRIPLET=x64-windows ` + -DCMAKE_INSTALL_PREFIX="$PWD\build\install-root" ` + -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_STANDARD_REQUIRED=ON ` + -DFMOD_ENABLED=OFF -DSTEAMWORKS=OFF -DEOS=OFF -DOPENAL_ENABLED=OFF ` + -DTHEORAPLAYER=OFF -DCURL=ON -DOPUS=ON + +cmake --build build --config Release --parallel +cmake --build build --config Release --target INSTALL +``` + +### B. Full-feature manual build (FMOD + Steamworks + TheoraPlayer) + +```powershell +$env:BARONY_WIN32_LIBRARIES = "$PWD\deps\vcpkg\installed\x64-windows" +$env:FMOD_DIR = "$PWD\deps\fmod" +$env:STEAMWORKS_ROOT = "$PWD\deps\steamworks" +$env:THEORAPLAYER_DIR = "$PWD\deps\theoraplayer" +$env:EOS_ENABLED = "0" + +cmake -S . -B build -G "Visual Studio 17 2022" -A x64 ` + -DCMAKE_POLICY_VERSION_MINIMUM=3.10 ` + -DCMAKE_TOOLCHAIN_FILE="$PWD\deps\vcpkg\scripts\buildsystems\vcpkg.cmake" ` + -DVCPKG_TARGET_TRIPLET=x64-windows ` + -DCMAKE_INSTALL_PREFIX="$PWD\build\install-root" ` + -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_STANDARD_REQUIRED=ON ` + -DFMOD_ENABLED=ON -DSTEAMWORKS=ON -DEOS=OFF -DOPENAL_ENABLED=OFF ` + -DTHEORAPLAYER=ON -DCURL=ON -DOPUS=ON + +cmake --build build --config Release --parallel +cmake --build build --config Release --target INSTALL +``` + +Notes for full-feature manual build: + +- `THEORAPLAYER=ON` requires prebuilt TheoraPlayer files under `deps\theoraplayer`. +- Required files: `deps\theoraplayer\include\theoraplayer\theoraplayer.h` and `deps\theoraplayer\lib\theoraplayer.lib`. +- If those files are not present, set `-DTHEORAPLAYER=OFF`. + +If `INSTALL` tries to write to `C:\Program Files\barony` and fails with permission denied, reconfigure with a writable prefix, for example: + +```powershell +cmake -S . -B build -DCMAKE_INSTALL_PREFIX="$PWD\build\install-root" +``` + +### C. Package Windows mod releases (Steam and NoDRM overlays) + +After building the Windows release variants, package overlay zips with: + +```powershell +powershell -ExecutionPolicy Bypass -File scripts\mod_release\package_windows_release.ps1 ` + -Label v5.0.2-rc1 ` + -SteamBuildDir build-vs2022-x64 ` + -NoDrmBuildDir build-vs2022-x64-nodrm +``` + +The script accepts either a build directory or a direct `Release` directory for +each package input. For each package it stages: + +- `barony.exe` +- `editor.exe` if present +- every `.dll` next to the executable +- `steam_appid.txt` when present in the Steam build output +- the mod release readme from `docs\mod_release\README.txt` +- the packaged high-level changelog from `docs\mod_release\mod-changelog.txt` +- the detailed release notes from `docs\mod_release\changelog_v5.0.2.md` +- `SHA256SUMS.txt` + +Artifacts are written to `release-artifacts\barony-8p-windows-steam-