Summary
Several tests in tests/backtesting/ pass with green checkmarks but contain provably wrong assertions. These tests do not catch regressions; they provide false confidence. At least two cases confirmed:
Case 1 — Wrong win rate expected value
tests/backtesting/test_performance.py, test_win_rate_calculation:
trades = [100, -50, 200, 150, -75, 300, -25, 180, -100, 250]
expected_win_rate = 0.7 # WRONG
Counting: 6 positive values (100, 200, 150, 300, 180, 250) out of 10 trades = 0.6, not 0.7. The test asserts a value that doesn't match the data it defines. If the implementation ever computes the correct answer (0.6), this test will fail. Currently it passes only if the implementation also has a counting error.
Case 2 — Annualized return assertion in wrong units
tests/backtesting/test_performance.py, test_sharpe_ratio_calculation:
assert annualized_return > 10 # asserts >1000%
An annualized_return of 10 means 1000%. This assertion passes only because annualized_return uses 365.25 as the denominator instead of trading_days_held (see Issue #12), producing inflated nonsense values. If the annualized_return bug is fixed, this test will fail. The intended assertion was almost certainly > 0.10 (10%).
Broader concern
The combination of:
- Green tests with wrong assertions
INTERNALERROR crashes masking other tests
conftest.py injecting torch, F, np, pd as builtins
...means the test suite's green/red signals cannot be trusted at face value. Before treating any test pass as a correctness guarantee, each test's expected values should be independently verified against the documented spec.
Recommended fix
- Fix
test_win_rate_calculation: change expected_win_rate = 0.7 to 0.6
- Fix
test_sharpe_ratio_calculation: change assert annualized_return > 10 to assert annualized_return > 0.10
- Audit remaining backtesting tests for similar oracle errors — particularly any assertion that uses a round number without a derivation comment
Summary
Several tests in
tests/backtesting/pass with green checkmarks but contain provably wrong assertions. These tests do not catch regressions; they provide false confidence. At least two cases confirmed:Case 1 — Wrong win rate expected value
tests/backtesting/test_performance.py,test_win_rate_calculation:Counting: 6 positive values (100, 200, 150, 300, 180, 250) out of 10 trades = 0.6, not 0.7. The test asserts a value that doesn't match the data it defines. If the implementation ever computes the correct answer (0.6), this test will fail. Currently it passes only if the implementation also has a counting error.
Case 2 — Annualized return assertion in wrong units
tests/backtesting/test_performance.py,test_sharpe_ratio_calculation:An
annualized_returnof10means1000%. This assertion passes only becauseannualized_returnuses365.25as the denominator instead oftrading_days_held(see Issue #12), producing inflated nonsense values. If theannualized_returnbug is fixed, this test will fail. The intended assertion was almost certainly> 0.10(10%).Broader concern
The combination of:
INTERNALERRORcrashes masking other testsconftest.pyinjectingtorch,F,np,pdas builtins...means the test suite's green/red signals cannot be trusted at face value. Before treating any test pass as a correctness guarantee, each test's expected values should be independently verified against the documented spec.
Recommended fix
test_win_rate_calculation: changeexpected_win_rate = 0.7to0.6test_sharpe_ratio_calculation: changeassert annualized_return > 10toassert annualized_return > 0.10