Skip to content

Add automated test suite for DLL core systems - #1211

Open
acato wants to merge 5 commits into
We-the-People-civ4col-mod:developfrom
acato:feature/test-suite
Open

Add automated test suite for DLL core systems#1211
acato wants to merge 5 commits into
We-the-People-civ4col-mod:developfrom
acato:feature/test-suite

Conversation

@acato

@acato acato commented Mar 15, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds a lightweight C++03 test framework (WTPTestFramework.h) that runs automatically on game launch in Assert builds, with zero overhead in Release/FinalRelease
  • 48,900+ automated checks across 17 test suites covering data structures, XML integrity, savegame serialization, and RNG determinism
  • Includes Python-side tests runnable from the debug console and Perl build-time validation scripts
  • Full design document with principles, usage guide, and examples for extending the suite

Test suites

Suite Tests What it validates
EnumMap (int, bool, enum) 53 Default values, set/get/reset, arithmetic, iteration, specializations
JustInTimeArray 21 Lazy allocation, reset, accumulation, min/max, non-zero defaults
Coordinates 5 Construction, invalid/null factories, equality
TradeRoute 20 Sentinel constants, default construction
CvIdVector 23 Add/remove/lookup, sequential IDs, no ID reuse after removal
XMLIntegrity 48,770 Profession yields, building classes, unit professions, founding fathers, equipment, terrain, yield costs
Savegame (7 sub-suites) 56 Primitive round-trips, stream state, strings, IDInfo, JustInTimeArray, enums, edge cases
Determinism 576 CvRandom seed reproducibility, range bounds, peek, float output, state round-trips
Python (on-demand) 7 DLL-Python bridge validation, XML data consistency from Python side

Files added

  • DLLSources/WTPTestFramework.h/.cpp — framework macros and reporting
  • DLLSources/TestDataStructures.cpp — EnumMap, JIT array, coordinates, trade route, ID vector tests
  • DLLSources/TestXMLIntegrity.cpp — XML cross-reference validation
  • DLLSources/TestSavegame.cpp — MemoryStream + round-trip serialization tests
  • DLLSources/TestDeterminism.cpp — CvRandom determinism tests
  • bin/test_determinism.pl — build-time scan for non-deterministic stdlib calls
  • bin/test_text_keys.pl — build-time TXT_KEY coverage check
  • Assets/Python/_DebugTools/WTPTests.py — Python test suite
  • docs/test-suite-design.md — design document

Test plan

  • All 17 suites pass (48,900+ checks) — verified in-game on Assert build
  • No assert popups on game launch
  • Results logged to Logs/WTPTests.log
  • Zero overhead in Release/FinalRelease builds (compiled out via #ifdef FASSERT_ENABLE)
  • Reviewer: verify build compiles cleanly

🤖 Generated with Claude Code

acato and others added 5 commits March 14, 2026 16:54
@acato @claude
Add WTP test framework and data structure tests (Phase 1)
0f16033 
Custom MinUnit-derived C++03 test framework with 35 tests across 6 suites
covering EnumMap, JustInTimeArray, Coordinates, CvTradeRoute sentinels,
and CvIdVector. Tests run at startup in Assert/Debug builds after XML load,
logging results to Logs\WTPTests.log.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@acato @claude
Add XML integrity tests and build-time validation scripts (Phase 2)
54b001d 
C++ tests (10) validate XML cross-references at startup: profession
yields, building classes, unit professions, father CivEffects/categories,
yield costs, equipment amounts, terrain yields, and trade sentinels.

Perl scripts: test_determinism.pl detects non-deterministic calls
(rand/srand/time/GetTickCount) outside allowed files; test_text_keys.pl
verifies TXT_KEY references in data XML have matching text definitions.
Both run as part of the test_DllExport build target.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@acato @claude
Add savegame round-trip tests (Phase 3) and fix test failures
da0f939 
Phase 3: MemoryStream (in-memory FDataStreamBase) with 23 round-trip tests
covering primitives, strings, IDInfo, JustInTimeArray, and enum types.

Fix XMLIntegrity test: allow NO_YIELD (-1) in profession yield lists, as
the game code explicitly checks and skips it. Fix int-to-enum casts for
VC++ 2003 strict type checking.

Fix Savegame test: JustInTimeArray::Read() always allocates, even for
empty arrays — removed incorrect !isAllocated() assertion.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@acato @claude
Add determinism tests (Phase 4) and Python validation tests (Phase 5)
9d9d90e 
Phase 4: CvRandom determinism tests — seed reproducibility, range bounds,
peek non-advancement, float output, and state round-trips (576 checks).

Phase 5: Python-side test suite (WTPTests.py) validating the DLL-Python
bridge and XML data consistency — profession yields, building classes,
unit professions, father categories, terrain yields, civilization info.
Runnable from debug console: import WTPTests; WTPTests.runAllTests()

Also fixes WTPTests.log double-nesting (gDLL->logMsg already writes to
the Logs/ directory).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@acato @claude
Remove trade-route-specific tests and add documentation
770992d 
Strip references to AFRICA_CITY_ID, PORT_ROYAL_CITY_ID, and
isOffMapTradeLocation() from test files — these depend on the trade
route feature branch and don't belong in the test-only PR.

Add test suite design document explaining framework principles, usage,
and examples for extending the suite.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@acato