Compute error metrics

[BrendanKKrueger]

Depends on #28

Step 3 of #25

Summary from Claude below

Branch 3: Error Analysis

Title

Branch 3: Add comprehensive error analysis (eigenvalue, matrix norm, state-dependent errors)

Description

Overview

This PR implements comprehensive error analysis comparing exact Hamiltonian matrices with approximate algorithm unitaries. This is Branch 3 of a 5-branch implementation plan to add exact matrix analysis and error quantification capabilities to QHAT.

Purpose

• Quantify algorithm accuracy through three complementary error metrics
• Compare exact vs approximate eigenvalues (ground state energy validation)
• Compute matrix norm errors (physical bounds on algorithm error)
• Evaluate state-dependent errors (error on specific physically relevant states)
• Scale to 25-30 qubits using matrix-free methods

Changes Made

Modified Files:

1. analysis/config_types.py

   • Added error_num_eigenvalues field (default: 0, disabled)
   • Added error_matrix_norms field (default: None, disabled)
     • Accepts single string ("frobenius" or "spectral")
     • Accepts list of strings (["frobenius", "spectral"])
   • Added error_state_inputs field (default: None, disabled)
     • Accepts single filename (string)
     • Accepts list of filenames
   • Updated TOML serialization to include new fields

2. analysis/analysis.py

   • Added error_analysis() function (~350 lines)
     • Three independent error computation modes
     • Eigenvalue errors: Load or compute eigendecompositions, compare k smallest
     • Matrix norm errors: Dense path (small systems) and matrix-free path (large systems)
     • State-dependent errors: Apply both operators to states, compute difference norms
     • Automatic selection of computation strategy based on system size
     • Progress tracking for expensive matrix-free computations
     • Save all results to error_analysis.npz
   • Added import os for file existence checks
   • Updated analyze_algorithm() to:
     • Detect error analysis request
     • Include error analysis in validation
     • Mark both matrices as needed when error analysis requested
     • Dispatch to error_analysis() when requested

3. analysis/README.md

   • Added "Error Analysis" section with comprehensive documentation
   • Explained all three error types with physical interpretation
   • Provided system size guidance (≤15, 16-20, 20-30 qubits)
   • Documented when to use each error type
   • Included multiple usage examples
   • Explained matrix-free computation costs
   • Clarified Frobenius vs spectral norm differences

4. analysis/examples/config_full_analysis.py

   • Updated to show error analysis (replacing "Future Analyses" placeholder)
   • Added comprehensive examples for all three error types
   • Demonstrated minimal setup for large systems
   • Showed comprehensive validation setup
   • Included system size considerations in comments

New Files:

1. analysis/tests/test_error_analysis.py (775 lines)
   • 18 comprehensive tests covering:
     • Eigenvalue errors (2 tests): zero when identical, nonzero when different
     • Frobenius norm errors (2 tests): zero when identical, nonzero when different
     • Spectral norm errors (2 tests): zero when identical, nonzero when different
     • Both matrix norms (1 test)
     • State-dependent errors (3 tests): single state, multiple states
     • Integration (2 tests): all error types together, file output
     • Error handling (2 tests): invalid norm type, missing state file
     • Matrix-free operators (2 tests): state errors, Frobenius norm
     • Relative errors (2 tests): eigenvalue and state relative errors
   • All tests passing ✓

Technical Details

Three Independent Error Types:

1. Eigenvalue Errors (error_num_eigenvalues > 0):

   • Compares k smallest eigenvalues from exact vs approximate
   • Loads existing eigendecompositions or computes them
   • Reports both absolute and relative errors
   • Most common use: k=1 for ground state energy validation
   • Scales well: Uses partial eigendecomposition (sparse methods)

2. Matrix Norm Errors (error_matrix_norms not None):

   • Frobenius norm: ||H_exact - H_approx||_F = sqrt(sum of squares)

     • Fast for small systems (dense computation)
     • Matrix-free for large systems: 2^N matrix-vector products
     • Good for quick comparisons

   • Spectral norm: ||H_exact - H_approx||_2 = largest singular value

     • Physically meaningful (worst-case effect on any state)
     • Power iteration for large systems
     • More expensive than Frobenius

   • Implementation paths:

     • Dense (dimension ≤ 32768): Direct numpy computation
     • Matrix-free (dimension > 32768): Iterative methods with progress tracking

3. State-Dependent Errors (error_state_inputs not None):

   • Compares: ||H_exact|ψ⟩ - H_approx|ψ⟩||
   • Best-scaling error metric for large systems
   • Memory: O(2^N) for state vectors, not O(2^(2N)) for matrices
   • Can reach 30 qubits
   • Fast: just applies operators to states
   • Best for: Error on specific physically relevant states (ground state, thermal states, etc.)

Matrix-Free Computation:

For large systems (>15 qubits), matrix norm computation uses matrix-free methods:

• Frobenius norm: Computes ||diff e_i|| for each basis vector e_i

  • Requires 2^N matrix-vector products
  • Example: 20 qubits = 1M operations
  • Progress tracking every 10%

• Spectral norm: Power iteration on (H_exact - H_approx)† (H_exact - H_approx)

  • Converges in ~10-100 iterations typically
  • Each iteration: 2 forward, 2 adjoint applications
  • Progress tracking every 10 iterations

Automatic Strategy Selection:

The error_analysis() function automatically:

• Detects if matrices are dense arrays or matrix-free operators
• Chooses appropriate computation path
• Displays progress warnings for expensive computations
• Works with both Branch 1 (exact matrices) and Branch 2 (eigendecompositions)

Output Format (error_analysis.npz):

• eigenvalue_absolute_errors: Array of absolute errors (if eigenvalue errors enabled)
• eigenvalue_relative_errors: Array of relative errors (if eigenvalue errors enabled)
• eigenvalue_num: Number of eigenvalues compared
• matrix_frobenius_error: Scalar (if Frobenius norm requested)
• matrix_spectral_error: Scalar (if spectral norm requested)
• state_absolute_errors: Array of absolute errors (if state errors enabled)
• state_relative_errors: Array of relative errors (if state errors enabled)
• Filenames for states stored in results dict, not npz

Type of Change

• New feature (non-breaking change which adds functionality)
• Bug fix
• Documentation update
• Breaking change

Testing

Test Coverage:

• 18 unit/integration tests for error analysis (all passing)
• 200 existing tests from Branches 0-2 continue to pass
• Total: 218 tests, 100% passing

Test Highlights:

• Zero error verification for identical matrices/operators
• Nonzero error verification for different matrices
• Both matrix norm types verified
• Multiple input states verified
• All three error types together verified
• Matrix-free operators verified
• Relative error calculations verified
• Error handling verified (invalid parameters, missing files)
• Output file creation verified

Running Tests:

# All tests
python3.11 -m pytest analysis/tests/ -v

# Just error analysis
python3.11 -m pytest analysis/tests/test_error_analysis.py -v

# Quick run
python3.11 -m pytest analysis/tests/ -q

Usage Example

# In configuration file (e.g., config_full_analysis.py)

# Example 1: Ground state energy comparison (most common)
analysis.error_num_eigenvalues = 1

# Example 2: Matrix norm errors (physical bounds)
analysis.error_matrix_norms = "frobenius"  # Fast
# or
analysis.error_matrix_norms = ["frobenius", "spectral"]  # Both

# Example 3: State-dependent errors (best scaling)
analysis.error_state_inputs = "ground_state.npy"
# or multiple states
analysis.error_state_inputs = ["ground.npy", "excited.npy"]

# Example 4: Comprehensive validation (all three error types)
analysis.error_num_eigenvalues = 1
analysis.error_matrix_norms = "frobenius"
analysis.error_state_inputs = ["ground.npy", "excited.npy"]

# Example 5: Minimal setup for large systems (20+ qubits)
analysis.error_num_eigenvalues = 1  # Ground state only
analysis.error_state_inputs = "ground.npy"  # Scales well
# Skip error_matrix_norms (too slow for large systems)

Output:

• File: error_analysis.npz
• Load with: results = np.load('error_analysis.npz')
• Access: results['eigenvalue_absolute_errors'], results['matrix_frobenius_error'], etc.

Context: Implementation Plan

This is Branch 3 of a 5-branch sequence:

• Branch 0: Comprehensive configuration baseline ✓ (complete)
• Branch 1: Exact matrix computation ✓ (complete)
• Branch 2: Eigendecomposition analysis ✓ (complete)
• Branch 3: Error metrics ← YOU ARE HERE
• Branch 4: Exact numerical simulation

Each branch builds on the previous, delivering independent, testable functionality.

Benefits

For Users:

• Quantify algorithm accuracy with three complementary metrics
• Validate ground state energy calculations
• Compute physical bounds on algorithm error
• Evaluate error on specific states of interest
• Scale to 25-30 qubits using matrix-free methods

For Developers:

• Clean separation of three error types
• Automatic selection of computation strategy
• Reuses shared infrastructure from Branches 1 and 2
• Extensible design for additional error metrics
• Comprehensive test coverage

Performance

Scaling by error type:

Error Type	≤15 qubits	16-20 qubits	21-30 qubits	Notes
Eigenvalue	✓ Fast	✓ Fast	✓ Fast	Uses partial eigendecomp (Branch 2)
Frobenius norm	✓ Fast	⚠ Slow	⚠ Very slow	2^N ops for matrix-free (1M ops @ 20 qubits)
Spectral norm	✓ Fast	⚠ Slow	⚠ Very slow	Power iteration, even slower than Frobenius
State errors	✓ Fast	✓ Fast	✓ Fast	Best-scaling metric, just applies operators

Production recommendations:

• All systems: Use eigenvalue errors (k=1) for ground state validation
• ≤15 qubits: All error types fast, use all three
• 16-20 qubits: Use eigenvalue + state errors; use matrix norms only if needed
• 21-30 qubits: Use eigenvalue + state errors; avoid matrix norms (too slow)

Memory:

• Eigenvalue errors: O(k·2^N) for k eigenvectors
• Matrix norm errors (dense): O(2^(2N)) for full matrices
• Matrix norm errors (matrix-free): O(2^N) for state vectors
• State errors: O(2^N) for state vectors

Checklist

• Code follows project style guidelines
• All tests pass (218/218)
• Documentation updated (README, config examples)
• No breaking changes to existing API
• Backward compatible (error analysis disabled by default)
• Branch pushed to remote
• Code review completed
• Ready to merge

Merge Dependencies

• Base branch: bkk_eigendecomposition (Branch 2)
• Target branch: main
• Depends on: Branches 1 and 2 should be merged first

Commits

1. 0348180: Branch 3: Add error analysis (eigenvalue, matrix norm, state-dependent)

Related Issues

Part of the effort to add exact matrix analysis and error quantification capabilities to QHAT for validating algorithm accuracy.

Reviewers

@lanl/qhat-team

Notes for Reviewers

Key files to review:

• analysis/analysis.py: New error_analysis() function (~350 lines)
  • Three independent error computation modes
  • Automatic dense/matrix-free selection
  • Progress tracking for expensive computations
• analysis/config_types.py: Three new configuration fields
• Test file: Comprehensive coverage showing expected behavior (18 tests)

Design decisions to consider:

• Three independent error types (each can be enabled/disabled separately)
• Automatic selection of dense vs matrix-free computation
• Matrix norm errors expensive for large systems (warned in docs and logs)
• State-dependent errors scale best (recommended for large systems)
• Output file saves raw arrays (not metadata like filenames)

Integration with Branches 1-2:

• Reuses exact matrix computation from Branch 1
• Reuses eigendecomposition from Branch 2
• Can load existing eigendecompositions or compute on-the-fly
• Works with both dense matrices and matrix-free operators

Testing:

• 218 tests (128 original + 53 Branch 1 + 19 Branch 2 + 18 Branch 3), all passing
• Tests run in ~40 seconds on standard hardware
• No integration issues with existing functionality

Next steps (Branch 4):

• This error analysis infrastructure validates approximate algorithms
• Branch 4 will add exact numerical simulation
• Enables comparison: exact evolution vs approximate evolution