Skip to content

lukifer23/Matrix0

Repository files navigation

Matrix0: Advanced AlphaZero-Style Chess Engine with SSL Integration

Code Quality Model Validation Training Pipeline Test Bugbot Review

Matrix0 is a production-ready AlphaZero-style chess engine featuring complete SSL (Self-Supervised Learning) integration designed for Apple Silicon. It provides a sophisticated multi-task learning pipeline combining policy/value optimization with advanced SSL capabilities for chess pattern recognition across five specialized tasks: piece, threat, pin, fork, and control.

Project Overview

Matrix0 implements cutting-edge multi-task learning combining reinforcement learning from AlphaZero with advanced SSL (Self-Supervised Learning) for chess pattern recognition, optimized for Apple Silicon (MPS). The project delivers:

  • SSL Architecture Integration: ARCHITECTURE READY - five specialized SSL heads for piece, threat, pin, fork, and control detection
  • Multi-Task Learning: Simultaneous optimization of policy, value, and SSL objectives
  • Advanced Architecture: 44.2M parameter ResNet-22 with chess-specific attention and SSL foundation
  • Apple Silicon Optimization: MPS GPU acceleration with 14GB memory management
  • Enhanced WebUI: Comprehensive monitoring platform with real-time SSL and training analytics
  • Advanced Benchmark System: Multi-engine tournaments, SSL performance tracking, and comprehensive evaluation

Project Status

ACTIVE DEVELOPMENT – SSL architecture integration, strict local-loop diagnostics, and EX0Bench benchmarking are in place. Current work is focused on reliable self-play improvement: verifying generated labels, diagnosing capped games, and promoting checkpoints only after stable heldout metrics plus candidate generator checks. For the active training/debugging plan, see the current working plan. For historical status and broader roadmap context, see the comprehensive status report, the enhanced WebUI guide, the EX0Bench documentation, and the development roadmap.

✨ Key Features

SSL Integration (COMPLETE)

  • 5 Specialized SSL Heads: Piece recognition, threat detection, pin detection, fork detection, control detection
  • Multi-Task Learning: Simultaneous optimization of policy, value, and SSL objectives
  • Dedicated SSL Parameters: SSL capacity with weighted loss functions
  • Real-Time SSL Monitoring: WebUI dashboard with SSL head performance tracking

Advanced Architecture

  • 44.2M Parameters: ResNet-22 with 288 channels, 22 blocks, 18 attention heads
  • Chess-Specific Attention: Optimized attention mechanisms for chess patterns
  • SSL Foundation: Complete SSL integration with multi-head architecture
  • Memory Optimized: 14GB MPS limit with efficient SSL processing

Enhanced WebUI Platform

  • Interactive Chess Board: Fully functional 8x8 board with proper alternating square colors
  • Multi-View Interface: Game, Training, SSL, Tournament, and Analysis views
  • Real-Time Monitoring: Live training status, SSL performance, and model analytics
  • Interactive Visualization: Charts, progress bars, and performance metrics
  • SSL Dashboard: Complete SSL head analysis and parameter tracking
  • Responsive Design: Optimized for desktop, tablet, and mobile devices
  • Modern UI: Clean, professional interface with efficient space utilization

Production Training Pipeline

  • Self-Play Generation: 3 workers by default (configurable) generating SSL-enhanced training data
  • Multi-Task Training: Combined policy/value/SSL optimization with proper gradient accumulation
  • Model Evaluation: Tournament system with SSL-aware strength estimation
  • Checkpoint Management: Advanced checkpoint creation preserving SSL architecture

Advanced Benchmark System

  • Multi-Engine Tournaments: Round-robin, Swiss, and single-elimination formats
  • EX0Bench External Engine Battles: Pure Stockfish vs LC0 comparisons with no neural network required
  • SSL Performance Tracking: Real-time monitoring of SSL head effectiveness
  • Apple Silicon Optimization: MPS memory monitoring and Metal backend support
  • Automated Engine Discovery: Intelligent detection and configuration of installed engines
  • Comprehensive Analysis: Statistical significance testing and performance regression analysis

Apple Silicon Optimization

  • MPS GPU Acceleration: Native Apple Silicon support with unified memory
  • 14GB Memory Management: Automatic cleanup and cache management
  • Mixed Precision Training: FP16 optimization with MPS compatibility
  • Performance Monitoring: Real-time MPS utilization and memory tracking

Enterprise Features

  • Robust Data Management: SQLite metadata, corruption detection, automatic backup
  • External Engine Integration: Stockfish and LC0 support for competitive evaluation
  • Comprehensive Logging: Structured logging with SSL performance metrics
  • Training Stability: Advanced error handling, gradient management, and recovery mechanisms

Project Structure

Matrix0/
├── azchess/                    # Core package (44.2M parameter model with SSL)
│   ├── model/                  # Neural network architecture
│   │   └── resnet.py          # ResNet-22 with attention and complete SSL integration
│   ├── ssl_algorithms.py      # Advanced SSL algorithms (threat, pin, fork, control)
│   ├── selfplay/               # Self-play generation with SSL data augmentation
│   ├── mcts/                   # Monte Carlo Tree Search engine
│   ├── training/               # Multi-task training pipeline (policy/value/SSL)
│   ├── eval/                   # Model evaluation with SSL-aware metrics
│   ├── tools/                  # Analysis and benchmarking tools
│   ├── data_manager.py        # SQLite metadata and backup system
│   ├── orchestrator.py        # Main training coordinator
│   └── config.py              # Configuration management
├── config.yaml                 # Main configuration (SSL enabled, 5 SSL tasks, 3 workers)
├── data/                       # Training data and replays
│   ├── backups/               # Automatic backup system
│   ├── selfplay/              # SSL-enhanced self-play game data
│   └── data_metadata.db       # SQLite database for data integrity
├── checkpoints/                # Model checkpoints with SSL architecture
│   ├── best.pt                # Current best checkpoint
│   ├── model_step_1000.pt     # Step 1000 checkpoint
│   └── v2_base.pt             # V2 base checkpoint
├── webui/                      # Enhanced FastAPI monitoring platform
│   ├── server.py              # Backend with SSL/training endpoints
│   └── static/                # Multi-view frontend interface
├── logs/                       # Comprehensive logging with SSL metrics
├── docs/                       # Complete documentation suite
└── requirements.txt            # Python dependencies

Quick Start

Prerequisites

  • macOS with Apple Silicon (M1/M2/M3/M4)
  • Python 3.11+ with virtual environment support
  • 18GB+ RAM (14GB for model, 4GB for system)
  • 100GB+ free disk space (50GB for checkpoints, 50GB for data)
  • 16GB+ unified memory recommended

1. Environment Setup

git clone https://github.com/lukifer23/Matrix0.git
cd Matrix0
/opt/homebrew/bin/python3.12 -m venv .venv
.venv/bin/python -m pip install --upgrade pip
.venv/bin/python -m pip install -r requirements.txt
make doctor

Use .venv/bin/python for local runs so training, benchmarks, and tests use the same supported Python and dependency set. make env-info prints Python, PyTorch, MPS, selected device, and current model details for baseline records.

2. Initial Model Checkpoint

.venv/bin/python create_v2_checkpoint.py  # Creates optimized 44.2M parameter model

3. Start Training (Current Session)

source .venv/bin/activate
python -m azchess.orchestrator --workers 3 --sims 300 --lr 0.001 --batch-size 96 --epochs 1 --eval-games 10 --device mps

Recommended fast, stable Apple‑Silicon run (200 sims)

# Slightly more aggressive self‑play and shorter cycles
MATRIX0_MPS_TARGET_BATCH=6 \
python -m azchess.orchestrator \
  --tui table \
  --workers 3 \
  --games 300 \
  --sims 200 \
  --eval-games 40 \
  --promotion-threshold 0.55 \
  --epochs 1 \
  --steps-per-epoch 15000 \
  --opening-plies 6 \
  --max-game-length 160 \
  --resign-threshold -0.6

3b. Generate Stockfish Data (Optional)

# Example: generate 2k tactical positions with SSL targets
python tools/generate_stockfish_data.py \
  --domain tactical \
  --subcategory pins \
  --positions 2000 \
  --stockfish-depth 12

# Datasets are saved under data/stockfish_games/** and automatically ingested
# by training via training.extra_replay_dirs in config.yaml

Run the training loop directly without the orchestrator:

python -m azchess.training.train --config config.yaml

4. Monitor & Interact

# Interactive play against current SSL-integrated model
python -m azchess.cli_play

# Enhanced WebUI monitoring platform
python webui/server.py
# Then visit: http://127.0.0.1:8000

# Check current training status with SSL metrics
tail -f logs/matrix0.log

# View SSL performance and training analytics
# Access WebUI at http://127.0.0.1:8000

Model Analysis & Benchmarks

# Model information and parameter count
python -m azchess.tools.model_info

# Tiny local learning loop: self-play -> train -> eval JSON
python -m azchess.tools.bench_local_loop --config config.yaml

# Inference performance benchmarking
python -m azchess.tools.bench_inference

# Export to CoreML for Apple Silicon optimization
python coreml_export.py --checkpoint checkpoints/best.pt --output matrix0.mlmodel --benchmark

# MCTS performance benchmarking
python -m azchess.tools.bench_mcts

# Compare two checkpoints on local replay/self-play data
python -m azchess.tools.eval_checkpoints \
  --model-a checkpoints/candidate.pt \
  --model-b checkpoints/best.pt \
  --data-dir data

# Enhanced Benchmark System (full Matrix0 capabilities)
python benchmarks/enhanced_runner.py --config benchmarks/configs/enhanced_scenarios.yaml

# EX0Bench - Pure external engine battles (Stockfish vs LC0, no neural network)
python benchmarks/ex0bench.py --engine1 stockfish --engine2 lc0 --games 50 --time 60+0.6

# Quick benchmark against external engines
python benchmarks/benchmark.py --model checkpoints/v2_base.pt --engine stockfish --games 10

engine vs engine tournament (lc0 vs Stockfish)
```bash
python benchmarks/enhanced_runner.py --config benchmarks/configs/enhanced_scenarios.yaml --scenario Multi_Engine_Tournament

Training data analysis

python -m azchess.tools.process_lichess


### Diagnostics & Evaluation Improvements
- Evaluation and benchmarks now report search diagnostics:
  - `mcts_empty_visits` (count of empty-search fallbacks)
  - average root policy entropy over legal moves (nats)
- PGN exports are validated: header `Result` is corrected if it mismatches the reconstructed board result.
- Evaluation fallbacks are policy-based (no random move injection) and exploration noise is disabled during eval.
- Recommended MCTS simulations for benchmarks: `--mcts-sims 800–1600`.

### Fast Iteration (Smaller Model)
- A smaller configuration for faster training/iteration is provided:
  - `config_small.yaml` (160 channels × 14 blocks, attention-enabled, SSL on)
- Example run:
```bash
python -m azchess.training.train --config config_small.yaml

Use this to iterate quickly on data/algorithms, then switch back to the main config for strength.

📚 Documentation

Current Training Status

The project runs a 44.2M parameter ResNet-22 with five SSL heads and an optimized data pipeline. Training metrics and update history are tracked in docs/CURRENT_STATUS_SUMMARY.md.

Development

Testing & Validation

The project includes comprehensive validation and monitoring:

  • Code Quality: Linting, formatting, and type checking workflows
  • Model Validation: Architecture integrity and encoding verification
  • Training Pipeline: End-to-end pipeline validation and stability tests
  • Performance Monitoring: Real-time training metrics and memory usage

Local Development

# Install development dependencies
pip install flake8 black mypy

# Run comprehensive tests
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
black --check --diff .
mypy azchess/ --ignore-missing-imports

# Model validation tests
python -m azchess.model.test_encoding
python -m azchess.model.test_attention

# Performance benchmarking
python -m azchess.tools.bench_inference
python -m azchess.tools.bench_mcts

Contributing

Contributions are welcome! Current focus areas include:

High Priority

  • Reliable Local-Loop Improvement: Validate generated labels, outcome mix, heldout metrics, and candidate generator quality before promotion
  • Search/Outcome Diagnosis: Reduce capped-game dominance while preserving sharp policy labels
  • Training Signal Quality: Keep legal-policy ranking and value MSE stable before accepting a candidate
  • SSL Performance Validation: Measure SSL learning effectiveness after the local-loop promotion gate is reliable

Medium Priority

  • SSL Learning Analytics: Deep analysis of SSL contribution to policy/value learning
  • Model Interpretability: SSL decision explanation and analysis tools
  • Performance Benchmarking: Comprehensive SSL and training validation suites
  • Advanced SSL Features: SSL curriculum progression and dynamic weighting

Development Guidelines

  • Follow PEP 8 style guidelines with 88-character line limits
  • Add comprehensive tests for new functionality
  • Update documentation for all changes
  • Use type hints and docstrings for all functions
  • Ensure MPS compatibility for all new features

License

This project is licensed under the MIT License - see the LICENSE file for details.

Note: This is a research project. No third-party model weights are included.

Acknowledgments

  • Inspired by AlphaZero, Leela Chess Zero, and modern chess AI research
  • Built with PyTorch and optimized for Apple Silicon MPS architecture
  • Advanced SSL concepts from recent computer vision and NLP research
  • Community contributions and feedback
  • Bugbot: Code review and quality assurance via Bugbot (14-day trial via Cursor)

Current Achievements & Next Steps

Major milestones, active priorities, and planned work are documented in docs/CURRENT_STATUS_SUMMARY.md and docs/roadmap.md. Refer to those documents for up‑to‑date progress and future tasks.


Matrix0 v2.2 - SSL Architecture Integration + EX0Bench System

Advanced chess AI research platform with 53.4M parameter model and SSL architecture integration. Multi-task learning framework operational with comprehensive monitoring, EX0Bench external engine battles, and data pipeline fixes.

About

Matrix0: AlphaZero-style chess engine with SSL integration for tactical pattern recognition (threats, pins, forks, control), multi-task RL training on ResNet-24 architecture, Apple Silicon MPS optimization, and advanced WebUI/benchmarking tools.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors