Skip to content

Nightly Jobs Failing with false positive status #787

@jtrejoespinoza-grid

Description

@jtrejoespinoza-grid

Test running with run_python_tests fails when the test take more than 3 minutes to execute.

Description:

Currently Nightly jobs are failing, they are reported as complete but inspecting the logs will confirm the test is failing.
Example: https://github.com/project-chip/connectedhomeip/actions/runs/27077859958/job/79919136814

The issue is a TERMINATION_TIMEOUT_S introduced in the following PR: project-chip/connectedhomeip#72241 to avoid test to hang and terminate them but at the same time is terminating test that take more time than TERMINATION_TIMEOUT_S, as is the case for SU Tests.

Another issue that is observed is when running locally the Test keeps running while the run_python_test terminates, this seem to be happen because incorrect signal status is sent from the subprocess.

Steps to reproduce:

Run the test TC_SU_2_5 or TC_SU_2_7 or any test that takes more than 3 minutes to run and the test will fail.

python3 scripts/tests/run_python_test.py --load-from-env /tmp/test_env.yaml --script src/python_testing/TC_SU_2_5.py

Posible Solutions:

  • Add the custom --timeout of the test from the CI args when present to avoid the subprocess.wait() to use the default TERMINATION_TIMEOUT_S at run_python_test.py

Pending:

Investigate why at timeout is not sending the correct terminate signal.

Metadata

Metadata

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

Status
In review

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions