Skip to content

Commit b15b795

Browse files
psiddhfacebook-github-bot
authored andcommitted
Deselect VGF tests when model-converter is not installed (#19282)
Summary: VGF tests require the Arm MLSDK model-converter binary which is not provisioned in CI when `_ENABLE_VGF = False` (the current default in targets.bzl). These tests correctly skip via `SkipIfNoModelConverter`, but TestInfra reports skipped/disabled tests as FAILURE in the OMH dashboard, creating ~1,025 phantom failures for the ai_infra_mobile_platform oncall. This change uses the existing `pytest_collection_modifyitems` hook in conftest.py to deselect (not collect) tests that carry the `SkipIfNoModelConverter` marker (detected via `reason` string on `skipif` markers). This is more precise than name-based filtering because: - It catches parametrized variants like `test_roundtrip[VGF]` that use `marks=SkipIfNoModelConverter` - It does NOT deselect tests like `test_compile_spec_vgf_no_quant` that have "vgf" in the name but do not need model-converter Deselected tests are invisible to TestInfra, eliminating false failure reports. When `_ENABLE_VGF` is later flipped to True and model-converter is provisioned, VGF tests will automatically be collected and run again. Differential Revision: D103644035
1 parent a45c08a commit b15b795

1 file changed

Lines changed: 24 additions & 1 deletion

File tree

backends/arm/test/conftest.py

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,30 @@ def pytest_configure(config):
2929

3030

3131
def pytest_collection_modifyitems(config, items):
32-
pass
32+
from executorch.backends.arm.test import common
33+
from executorch.backends.arm.test.runner_utils import model_converter_installed
34+
35+
skip_marker = getattr(
36+
common.SkipIfNoModelConverter, "mark", common.SkipIfNoModelConverter
37+
)
38+
39+
if not model_converter_installed():
40+
deselected = []
41+
remaining = []
42+
for item in items:
43+
for marker in item.iter_markers("skipif"):
44+
if (
45+
marker.name == skip_marker.name
46+
and marker.args == skip_marker.args
47+
and marker.kwargs == skip_marker.kwargs
48+
):
49+
deselected.append(item)
50+
break
51+
else:
52+
remaining.append(item)
53+
if deselected:
54+
config.hook.pytest_deselected(items=deselected)
55+
items[:] = remaining
3356

3457

3558
def pytest_addoption(parser):

0 commit comments

Comments
 (0)