Skip to content

test: add 54 unit tests for core evaluation components - #34

Open
Pranjal0410 wants to merge 1 commit into
ruxailab:mainfrom
Pranjal0410:test/add-unit-test-suite
Open

test: add 54 unit tests for core evaluation components#34
Pranjal0410 wants to merge 1 commit into
ruxailab:mainfrom
Pranjal0410:test/add-unit-test-suite

Conversation

@Pranjal0410

Copy link
Copy Markdown

test: add comprehensive unit test suite for core evaluation components

Problem

The project currently has a single integration test (test_integration_real.py) that requires OmniParser model weights and an OpenAI API key to run. This means:

  • Tests cannot run in CI without expensive external dependencies
  • Core logic bugs (scoring, serialization, data models) go untested
  • Contributors cannot verify their changes locally without model weights (~2GB+)

Solution

Added tests/test_unit.py with 42 unit tests across 13 test classes covering all core components that don't require external services:

Test Class Tests What It Covers
TestNielsenHeuristics 5 All 10 heuristic definitions, criteria IDs, severity weights
TestSeverityLevel 3 Enum values and string conversion
TestUIElement 9 Creation, properties, computed width/height, serialization, from_dict with both OmniParser and legacy formats, missing field defaults
TestUIElementDetectionResult 2 Full and empty result serialization
TestInferHeadingLevel 4 Heading level inference from bbox height
TestCalculateHeightVariance 4 Variance calculation for consistency checks
TestHeuristicViolation 4 Violation serialization across all severity levels
TestHeuristicScore 5 Score percentage calculation, violation inclusion, LLM explanation handling
TestHeuristicEvaluationResult 3 Full result serialization, ISO timestamp, metadata defaults
TestHeuristicEngineScoring 8 calculate_score() — deductions per severity, accumulation, floor at 0, unknown heuristic handling
TestExceptions 6 Exception hierarchy and field handling
TestHeuristicEngineInit 1 Engine starts uninitialized

Key Design Decisions

  • Zero external dependencies: No OmniParser weights, no OpenAI API key, no Firebase needed
  • Tests the scoring math directly: Verifies that H1.2 major deducts 6 points, critical deducts 10, etc. against the actual weights in constants.py
  • Covers both OmniParser and legacy from_dict formats: Ensures backward compatibility
  • Follows existing project patterns: Uses pytest as already used in test_integration_real.py

Running

pip install pytest
pytest tests/test_unit.py -v

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant