Skip to content

Add balance analysis script with statistical reporting (Issue #63)#64

Merged
SorraTheOrc merged 3 commits into
mainfrom
copilot/applicable-takin
Dec 5, 2025
Merged

Add balance analysis script with statistical reporting (Issue #63)#64
SorraTheOrc merged 3 commits into
mainfrom
copilot/applicable-takin

Conversation

Copilot AI commented Dec 5, 2025

Copy link
Copy Markdown

Implements Task 11.3.1 - Analysis and Balance Reporting. Creates tooling to consume aggregated sweep data and generate actionable balance reports with statistical rigor.

Changes

New CLI Tool: scripts/analyze_balance.py

  • 4 subcommands: report, regression, trends, stats
  • Statistical analysis: confidence intervals (t-distribution), two-sample t-tests, linear regression trend detection
  • Balance detection: dominant strategies (>10% win rate delta), underperforming mechanics (<5% usage), unused story seeds, parameter sensitivity by difficulty
  • Visualizations: win rate bar charts, trend line charts, action distribution pie charts (matplotlib with base64 embedding)
  • Output formats: Markdown, HTML (with embedded charts), JSON

Tests: tests/scripts/test_analyze_balance.py

  • 39 tests across 12 test classes (requirement was 12+)
  • Covers statistical calculations, report generation, CLI commands, edge cases (empty DB, single run)

Usage

# Generate Markdown balance report
uv run python scripts/analyze_balance.py report --database build/sweep_results.db

# Generate HTML report with charts
uv run python scripts/analyze_balance.py report -d build/sweep_results.db -f html -o report.html

# Detect regressions between runs
uv run python scripts/analyze_balance.py regression -d build/sweep_results.db -b 1 -c 2

# Show metric trends
uv run python scripts/analyze_balance.py trends -d build/sweep_results.db --days 30

# Statistical analysis (CIs and t-tests)
uv run python scripts/analyze_balance.py stats -d build/sweep_results.db --json

Processes SQLite database from aggregate_sweep_results.py (Task 11.2.1). Falls back gracefully when scipy unavailable.

Original prompt

"Issue #63 - Analysis and Balance Reporting" to gamedev agent

Created from Copilot CLI via the copilot delegate command.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

…Issue #63)

Co-authored-by: SorraTheOrc <250240+SorraTheOrc@users.noreply.github.com>
Copilot AI changed the title [WIP] Add analysis and balance reporting feature Add balance analysis script with statistical reporting (Issue #63) Dec 5, 2025
Copilot AI requested a review from SorraTheOrc December 5, 2025 04:08
@SorraTheOrc SorraTheOrc merged commit 0379779 into main Dec 5, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants