feat(cli): add --tc-params-file option to auto-populate test parameters from mapping file#96
Open
rquidute wants to merge 6 commits into
Conversation
…eters
Implements feature request #903: allow users to provide a pre-generated
TC parameters mapping JSON file that automatically populates
test_parameters for each selected TC ID, eliminating the need to
manually copy PIXIT values from the QA Verification Steps sheet.
Changes:
- utils.py: add load_tc_params_mapping() to load and merge test params
from a JSON mapping file keyed by TC ID. Normalises separators
(-/_/.) and performs case-insensitive matching, matching the
same normalisation used in build_test_selection(). Returns a tuple
of (merged_params, missing_ids) for graceful fallback reporting.
- validation.py: add validate_tc_params_file() for pre-flight format
checks (file exists, valid JSON, top-level dict, all values are
dicts). Surfaces bad TC IDs in the error message.
- commands/run_tests.py: add --tc-params-file / -m option. The mapping
is applied at priority level 2 in the merge chain:
project config < mapping file < --config < -- inline args
Missing TC IDs emit a warning but do not abort execution.
- tests/test_tc_params_mapping.py: 28 unit tests covering happy paths,
separator/case normalisation, merge-order semantics, missing IDs,
and all error paths for both new functions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…dation.py Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a new feature to auto-populate test parameters from a TC parameters mapping JSON file via the --tc-params-file option in the run_tests command. It includes validation logic, key normalization, parameter merging, and comprehensive unit tests. The review feedback highlights a PEP 8 style violation where a statement is placed on the same line as an if condition, and points out code duplication and redundant JSON parsing between the validation and loading functions, suggesting a shared helper to resolve it.
Address Gemini code review comments on PR #96: 1. Eliminate duplicate file I/O and validation logic between validate_tc_params_file (validation.py) and load_tc_params_mapping (utils.py). Both functions were independently opening, parsing, and structurally validating the same mapping file. 2. Extract parse_and_validate_tc_params_file() into validation.py as the single source of truth for mapping-file I/O and structural checks. - validate_tc_params_file() now delegates to it and returns the resolved Path. - load_tc_params_mapping() imports and calls it instead of repeating the try/except file-reading block. 3. Add 7 tests for parse_and_validate_tc_params_file in test_tc_params_mapping.py, including a cross-function test that proves both callers produce identical error messages for the same bad file. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test_run_tests_logger_configuration was asserting configure_logger_for_run was called with only title=, but the function now also receives enable_log_streaming=True (passed when --no-streaming is not set).
- Describe normalisation behaviour (case-insensitive, -/_/. separators)
- Note that unmatched TC IDs are silently skipped
- Add a numbered NOTE block showing the full config precedence chain:
1. project config 2. --tc-params-file 3. --config 4. -- inline args
oxesoft
approved these changes
Jun 12, 2026
Use Click's \b escape to prevent the NOTE block from being re-wrapped into a single run-on paragraph. Each precedence level now renders on its own line in the terminal.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements feature request #903: auto-populate
test_parametersfrom a pre-generated TC parameters mapping file.Users can now pass a
--tc-params-file/-margument toth-cli run-testspointing to a JSON file that maps TC IDs to their required run arguments (int-arg,string-arg,timeout, etc.). The CLI automatically looks up each selected TC in the file and merges the matching parameters before starting the test run — eliminating the need to manually copy PIXIT values from the QA Verification Steps sheet for every run.Related Issue
Closes project-chip/certification-tool#903
Changes
th_cli/utils.py—load_tc_params_mapping()New function that loads a TC parameters mapping JSON file and returns a merged
test_parametersdict for the requested TC IDs:-,_,.) and applies case-insensitive matching — consistent withbuild_test_selection().(merged_params, missing_ids)so callers can emit graceful fallback warnings without aborting execution.th_cli/validation.py—validate_tc_params_file()New pre-flight validator called before the test run starts:
dict.dict(params object), listing all offending TC IDs in the error message if not.th_cli/commands/run_tests.py—--tc-params-file/-moptionNew click option added after
--pics-config-folder.Validated early (before the API client is created) so bad files fail fast.
Mapping params are applied at priority level 2 in the merge chain, between the project config and explicit overrides:
TC IDs not found in the mapping emit a warning and continue normally.
Matched params are printed to the console as
TC Params from Mapping File (Execution Only).tests/test_tc_params_mapping.py— 28 unit testsCovers: exact match, separator/case normalisation, multi-ID merge, key-conflict order, missing IDs, empty mapping, empty test list, space-separated
int-argvalues, mixed separators, superset mapping files, and all error paths for both new functions.Motivation
Running large clusters of tests (or onboarding new users) required manually extracting PIXIT values from the QA Verification Steps sheet and pasting them into the TH project config for every test cycle. This was:
The mapping file acts as a single source of truth, maintained by QA and distributed with each TH build. Users only need to override values that are device-specific.
Impact
--tc-params-fileis fully optional; omitting it leaves all existing behaviour unchanged.test_run_config["test_parameters"]exactly as before.--configfile values and inline-- --int-argargs always take precedence over the mapping file.Testing
Unit Tests (automated)
28 new tests in
tests/test_tc_params_mapping.pycovering:test_exact_match_returns_params-/_/.)test_dash_underscore_normalisation,test_dot_normalisationtest_case_insensitive_matchtest_multiple_ids_merged,test_non_conflicting_keys_all_presenttest_key_conflict_last_sorted_id_winstest_unmatched_ids_reported_in_missing,test_all_ids_unmatched,test_empty_mapping_all_missingtest_empty_test_ids_list,test_empty_mapping_is_validint-arg, mixed separators)test_multiple_space_separated_values_in_int_arg,test_mapping_keys_with_mixed_separatorstest_mapping_superset_only_requested_ids_mergedRun with:
Manual Testing
To verify end-to-end behaviour, create a mapping file and pass it to
run-tests:{ "TC-ACE-1.1": { "int-arg": "PIXIT.ACE.APPENDPOINT:1", "timeout": "300" }, "TC-MCORE-FS-1.3": { "string-arg": "PIXIT.MCORE.ENDPOINT:3" } }Expected console output when mapping is applied:
run-test --help arg