Skip to content

Create L1_L2_CoverageReport - #217

Open
nhanasi wants to merge 17 commits into
developfrom
feature/workflowtest
Open

Create L1_L2_CoverageReport#217
nhanasi wants to merge 17 commits into
developfrom
feature/workflowtest

Conversation

@nhanasi

@nhanasi nhanasi commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@nhanasi
nhanasi requested a review from a team as a code owner July 15, 2026 19:27
Copilot AI review requested due to automatic review settings July 15, 2026 19:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new GitHub Actions workflow intended to generate L1 (unit) and L2 (functional) coverage metrics on PRs, publish artifacts, regenerate coverage markdown docs, and post/update a PR comment with a summary.

Changes:

  • Introduces a multi-job PR workflow to compute and upload L1 and L2 coverage-related artifacts.
  • Regenerates L1/L2 coverage markdown files in-repo and (for same-repo PRs) commits them back to the PR branch.
  • Upserts a sticky PR comment containing the latest coverage summary.

Comment thread .github/workflows/L1_L2_CoverageReport.yml
Comment thread .github/workflows/L1_L2_CoverageReport.yml
Comment on lines +141 to +144
- name: Build and run L2 tests
run: |
docker exec -i native-platform /bin/bash -c "cd /mnt/L2_CONTAINER_SHARED_VOLUME/remote_debugger && sh cov_build.sh && export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/x86_64-linux-gnu:/lib/aarch64-linux-gnu:/usr/local/lib: && sh run_l2.sh"

Comment on lines +246 to +250
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0

Comment on lines +269 to +272
GENERATED_DATE="$(date -u +%Y-%m-%d)"

cat > test/functional-tests/docs/L1_Test_Coverage.md << EOF
# Remote Debugger L1 Coverage Report
Copilot AI review requested due to automatic review settings July 15, 2026 19:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 8 comments.

Comment on lines +7 to +9
permissions:
contents: write
pull-requests: write
Comment on lines +240 to +244
update-docs:
name: Update L1/L2 coverage markdowns
runs-on: ubuntu-latest
needs: [l1-coverage, l2-coverage]

Comment on lines +269 to +271
GENERATED_DATE="$(date -u +%Y-%m-%d)"

cat > test/functional-tests/docs/L1_Test_Coverage.md << EOF
test_functions=$(grep -R "^def test_" remote_debugger/test/functional-tests/tests/test_*.py | wc -l | tr -d ' ')
test_functions_disabled=$(grep -R "^[[:space:]]*#[[:space:]]*def test_" remote_debugger/test/functional-tests/tests/test_*.py | wc -l | tr -d ' ' || true)

src_functions=$(grep -R "^[A-Za-z_][A-Za-z0-9_[:space:]\*]*([^;]*)[[:space:]]*{" remote_debugger/src/*.c | wc -l | tr -d ' ')
Comment on lines +165 to +170
c_api_scenarios=$(grep -n "^[[:space:]]*Scenario:" remote_debugger/test/functional-tests/features/rrd_c_api_upload.feature | wc -l | tr -d ' ')
c_api_tests=$(grep -n "^def test_" remote_debugger/test/functional-tests/tests/test_rrd_c_api_upload.py | wc -l | tr -d ' ')
c_api_gap=$((c_api_scenarios - c_api_tests))
if [ "$c_api_gap" -lt 0 ]; then
c_api_gap=0
fi
Comment on lines +412 to +414
- name: Upsert PR coverage comment
uses: actions/github-script@v7
env:
Comment on lines +15 to +16
container:
image: ghcr.io/rdkcentral/docker-rdk-ci:latest
Comment on lines +127 to +130
- name: Pull required docker images
run: |
docker pull ghcr.io/rdkcentral/docker-device-mgt-service-test/mockxconf:latest
docker pull ghcr.io/rdkcentral/docker-device-mgt-service-test/native-platform:latest
@github-actions

Copy link
Copy Markdown

Code Coverage Summary

                             Total:|81.6%   2320|93.2%   133|    -      0

Copilot AI review requested due to automatic review settings July 15, 2026 19:43
@github-actions

Copy link
Copy Markdown

Code Coverage Summary

                             Total:|81.6%   2320|93.2%   133|    -      0

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 11 comments.

Comment thread .github/workflows/L1_L2_CoverageReport.yml Outdated
Comment thread .github/workflows/L1_L2_CoverageReport.yml Outdated

- name: Build and run L2 tests
run: |
docker exec -i native-platform /bin/bash -c "cd /mnt/L2_CONTAINER_SHARED_VOLUME/remote_debugger && sh cov_build.sh && export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/x86_64-linux-gnu:/lib/aarch64-linux-gnu:/usr/local/lib: && sh run_l2.sh"
Comment on lines +165 to +170
c_api_scenarios=$(grep -n "^[[:space:]]*Scenario:" remote_debugger/test/functional-tests/features/rrd_c_api_upload.feature | wc -l | tr -d ' ')
c_api_tests=$(grep -n "^def test_" remote_debugger/test/functional-tests/tests/test_rrd_c_api_upload.py | wc -l | tr -d ' ')
c_api_gap=$((c_api_scenarios - c_api_tests))
if [ "$c_api_gap" -lt 0 ]; then
c_api_gap=0
fi
Comment on lines +269 to +271
GENERATED_DATE="$(date -u +%Y-%m-%d)"

cat > test/functional-tests/docs/L1_Test_Coverage.md << EOF
Comment on lines +412 to +416
- name: Upsert PR coverage comment
uses: actions/github-script@v7
env:
COMMENT_PATH: /tmp/pr_coverage_comment.md
with:
Comment on lines +272 to +276
# Remote Debugger L1 Coverage Report

**Generated:** $GENERATED_DATE
**Component:** \`remotedebugger\`
**Source:** PR pipeline coverage run
Comment on lines +298 to +302
# Remote Debugger L2 Test Coverage Report

**Generated:** $GENERATED_DATE
**Component:** \`remotedebugger\`
**Test Suite:** \`test/functional-tests\`
Comment on lines +7 to +9
permissions:
contents: write
pull-requests: write
Comment on lines +15 to +16
container:
image: ghcr.io/rdkcentral/docker-rdk-ci:latest
@github-actions

Copy link
Copy Markdown

Code Coverage Summary

                             Total:|81.6%   2320|93.2%   133|    -      0

Copilot AI review requested due to automatic review settings July 16, 2026 19:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 12 comments.

Comment on lines +7 to +9
permissions:
contents: write
pull-requests: write
Comment on lines +240 to +244
update-docs:
name: Update L1/L2 coverage markdowns
runs-on: ubuntu-latest
needs: [l1-coverage, l2-coverage]

Comment on lines +31 to +35
COV_FILE="src/unittest/coverage.info"
if [ ! -f "$COV_FILE" ]; then
echo "coverage.info was not generated at $COV_FILE"
exit 1
fi
Comment thread .github/workflows/L1_L2_CoverageReport.yml Outdated
test_functions=$(grep -R "^def test_" remote_debugger/test/functional-tests/tests/test_*.py | wc -l | tr -d ' ')
test_functions_disabled=$(grep -R "^[[:space:]]*#[[:space:]]*def test_" remote_debugger/test/functional-tests/tests/test_*.py | wc -l | tr -d ' ' || true)

src_functions=$(grep -R "^[A-Za-z_][A-Za-z0-9_[:space:]\*]*([^;]*)[[:space:]]*{" remote_debugger/src/*.c | wc -l | tr -d ' ')
Comment on lines +272 to +276
# Remote Debugger L1 Coverage Report

**Generated:** $GENERATED_DATE
**Component:** `remotedebugger`
**Source:** PR pipeline coverage run
Comment on lines +292 to +294
- Coverage is generated by running `sh run_ut.sh --enable-cov`.
- Source artifact: `src/unittest/coverage.info`.
- This file is updated automatically on pull requests.
Comment on lines +298 to +302
# Remote Debugger L2 Test Coverage Report

**Generated:** $GENERATED_DATE
**Component:** `remotedebugger`
**Test Suite:** `test/functional-tests`
name: l1-coverage-artifacts
path: |
src/unittest/coverage.info
/tmp/coverage_summary.txt
Comment on lines +328 to +330
1. `rrd_c_api_upload.feature` scenarios: $C_API_SCENARIOS
`test_rrd_c_api_upload.py` test functions: $C_API_TESTS
**Current gap:** $C_API_GAP
Copilot AI review requested due to automatic review settings July 16, 2026 20:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 11 comments.

Comment on lines +7 to +9
permissions:
contents: write
pull-requests: write
Comment thread .github/workflows/L1_L2_CoverageReport.yml Outdated
Comment thread .github/workflows/L1_L2_CoverageReport.yml Outdated
Comment on lines +134 to +135
docker run -d --name mockxconf -p 50050:50050 -p 50051:50051 -p 50052:50052 -p 50054:50054 -v ${{ github.workspace }}:/mnt/L2_CONTAINER_SHARED_VOLUME ghcr.io/rdkcentral/docker-device-mgt-service-test/mockxconf:latest
docker run -d --name native-platform --link mockxconf -v ${{ github.workspace }}:/mnt/L2_CONTAINER_SHARED_VOLUME ghcr.io/rdkcentral/docker-device-mgt-service-test/native-platform:latest
Comment thread .github/workflows/L1_L2_CoverageReport.yml
Comment on lines +165 to +170
c_api_scenarios=$(grep -n "^[[:space:]]*Scenario:" remote_debugger/test/functional-tests/features/rrd_c_api_upload.feature | wc -l | tr -d ' ')
c_api_tests=$(grep -n "^def test_" remote_debugger/test/functional-tests/tests/test_rrd_c_api_upload.py | wc -l | tr -d ' ')
c_api_gap=$((c_api_scenarios - c_api_tests))
if [ "$c_api_gap" -lt 0 ]; then
c_api_gap=0
fi
Comment on lines +269 to +271
GENERATED_DATE="$(date -u +%Y-%m-%d)"

cat > test/functional-tests/docs/L1_Test_Coverage.md <<EOF
Comment on lines +272 to +276
# Remote Debugger L1 Coverage Report

**Generated:** $GENERATED_DATE
**Component:** `remotedebugger`
**Source:** PR pipeline coverage run
Comment thread .github/workflows/L1_L2_CoverageReport.yml Outdated
Comment on lines +298 to +302
# Remote Debugger L2 Test Coverage Report

**Generated:** $GENERATED_DATE
**Component:** `remotedebugger`
**Test Suite:** `test/functional-tests`
@github-actions

Copy link
Copy Markdown

Code Coverage Summary

                             Total:|81.6%   2320|93.2%   133|    -      0

2 similar comments
@github-actions

Copy link
Copy Markdown

Code Coverage Summary

                             Total:|81.6%   2320|93.2%   133|    -      0

@github-actions

Copy link
Copy Markdown

Code Coverage Summary

                             Total:|81.6%   2320|93.2%   133|    -      0

Copilot Bot review requested due to automatic review settings July 20, 2026 19:46
Copilot Bot review requested due to automatic review settings July 22, 2026 16:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (4)

.github/workflows/L1_L2_CoverageReport.yml:9

  • The workflow upserts PR issue comments via github.rest.issues.*, which requires issues: write permission. With only pull-requests: write, this step can fail with 403 when listing/creating/updating comments.
permissions:
  contents: write
  pull-requests: write

.github/workflows/L1_L2_CoverageReport.yml:216

  • This job later commits and pushes docs back to the PR branch, but actions/checkout on pull_request defaults to a detached merge ref (refs/pull/*/merge). Committing on that ref and pushing to github.head_ref can create an unintended history (or fail). Check out the PR head branch for same-repo PRs, otherwise keep the default ref.
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

.github/workflows/L1_L2_CoverageReport.yml:113

  • $LD_LIBRARY_PATH is expanded by the runner shell before docker exec runs, so the container may not inherit its own LD_LIBRARY_PATH value (often resulting in an empty/incorrect path). Quote or escape $ so it is evaluated inside the container.
          docker exec -i native-platform /bin/bash -c "cd /mnt/L2_CONTAINER_SHARED_VOLUME && sh cov_build.sh && export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/x86_64-linux-gnu:/lib/aarch64-linux-gnu:/usr/local/lib && sh run_l2.sh"

.github/workflows/L1_L2_CoverageReport.yml:130

  • With set -euo pipefail, this grep | wc -l pipeline will fail when there are 0 matching def test_ lines (grep exits 1), even though the desired count is 0. Wrap grep in a group with || true so the pipeline succeeds and yields 0.
          test_functions=$(grep -R "^def test_" test/functional-tests/tests/ | wc -l | tr -d ' ')

set -euo pipefail

feature_files=$(find test/functional-tests/features -name '*.feature' | wc -l | tr -d ' ')
feature_scenarios=$(grep -R "^[[:space:]]*Scenario:" test/functional-tests/features/ | wc -l | tr -d ' ')

feature_files=$(find test/functional-tests/features -name '*.feature' | wc -l | tr -d ' ')
feature_scenarios=$(grep -R "^[[:space:]]*Scenario:" test/functional-tests/features/ | wc -l | tr -d ' ')
feature_scenarios_disabled=$(grep -R "^[[:space:]]*#[[:space:]]*Scenario:" test/functional-tests/features/ | wc -l | tr -d ' ' || true)

test_files=$(find test/functional-tests/tests -maxdepth 1 -name 'test_*.py' | wc -l | tr -d ' ')
test_functions=$(grep -R "^def test_" test/functional-tests/tests/ | wc -l | tr -d ' ')
test_functions_disabled=$(grep -R "^[[:space:]]*#[[:space:]]*def test_" test/functional-tests/tests/ | wc -l | tr -d ' ' || true)
@github-actions

Copy link
Copy Markdown

Code Coverage Summary

                             Total:|81.6%   2320|93.2%   133|    -      0

Copilot Bot review requested due to automatic review settings July 22, 2026 18:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (4)

.github/workflows/L1_L2_CoverageReport.yml:9

  • Workflow-level permissions grant contents: write, which exposes a write-capable GITHUB_TOKEN to jobs that check out and execute PR code (unit/L2 tests). This is a security risk; keep the default token read-only and grant write scopes only to the job that actually needs to push docs / comment.
permissions:
  contents: write
  pull-requests: write

.github/workflows/L1_L2_CoverageReport.yml:210

  • After scoping the workflow-level token to read-only, the update-docs job needs explicit write permissions to push generated markdown and to create/update the PR issue comment via the Issues API.
  update-docs:
    name: Update L1/L2 coverage markdowns
    runs-on: ubuntu-latest
    needs: [l1-coverage, l2-coverage]

.github/workflows/L1_L2_CoverageReport.yml:108

  • mockxconf is mapped to port 50054 here, but the functional tests and test helpers use https://mockxconf:50053/... (see test/functional-tests/tests/rfc_test_helper.py). With the current mapping, L2 tests are likely to fail to reach mockxconf.
          docker run -d --name mockxconf -p 50050:50050 -p 50051:50051 -p 50052:50052 -p 50054:50054 -v ${{ github.workspace }}:/mnt/L2_CONTAINER_SHARED_VOLUME ghcr.io/rdkcentral/docker-device-mgt-service-test/mockxconf:latest

.github/workflows/L1_L2_CoverageReport.yml:376

  • On fork PRs, GITHUB_TOKEN permissions are forced to read-only, so this step will 403 when trying to create/update the PR comment. To avoid failing CI for fork PRs, guard the upsert step the same way the push step is guarded (or switch to a safer pull_request_target design).
      - name: Upsert PR coverage comment
        uses: actions/github-script@v7

@github-actions

Copy link
Copy Markdown

Code Coverage Summary

                             Total:|81.6%   2320|93.2%   133|    -      0

Copilot Bot review requested due to automatic review settings July 23, 2026 15:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (5)

.github/workflows/L1_L2_CoverageReport.yml:9

  • The workflow upserts a PR comment via github.rest.issues.*, but the explicit permissions: block doesn't grant issues: write, which can cause 403s when listing/creating/updating comments.
permissions:
  contents: write
  pull-requests: write

.github/workflows/L1_L2_CoverageReport.yml:16

  • This job runs PR-provided scripts (run_ut.sh, etc.) but inherits contents: write from the workflow-level token permissions. Restrict this job to read-only permissions to reduce the blast radius of untrusted PR code.
  l1-coverage:
    name: L1 unit coverage metrics
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/rdkcentral/docker-rdk-ci:latest

.github/workflows/L1_L2_CoverageReport.yml:96

  • This job also executes PR-controlled code inside containers and doesn't need write access to the repo. Add job-level read-only token permissions here as well.
  l2-coverage:
    name: L2 functional coverage metrics
    runs-on: ubuntu-latest

.github/workflows/L1_L2_CoverageReport.yml:227

  • The L2 job uploads l2-coverage-artifacts, but this step downloads l2-metrics. Align the artifact name so the workflow can source l2_metrics.env.
      - name: Download L2 metrics
        uses: actions/download-artifact@v4
        with:
          name: l2-metrics
          path: ci-artifacts/l2

.github/workflows/L1_L2_CoverageReport.yml:336

  • On pull_request events, actions/checkout typically checks out a detached PR merge ref. As written, git push origin HEAD:${{ github.head_ref }} can end up pushing that merge commit onto the PR branch (rewriting it). Switch to the PR head branch before committing/pushing.
          git add test/docs/L1_Test_Coverage.md test/docs/L2_Test_Coverage.md
          git commit -m "docs(coverage): refresh L1/L2 coverage for PR #${{ github.event.pull_request.number }}"
          git push origin HEAD:${{ github.head_ref }}

Comment on lines +165 to +166
functional_coverage_pct=$(awk -v passed="$L2_PASSED" -v total="$feature_scenarios" 'BEGIN { if (total > 0) printf "%.2f", (passed/total)*100; else print "0.00" }')
gap_to_100_pct=$(awk -v cov="$functional_coverage_pct" 'BEGIN { printf "%.2f", 100-cov }')
Comment on lines +217 to +221
- name: Download L1 metrics
uses: actions/download-artifact@v4
with:
name: l1-metrics
path: ci-artifacts/l1
@github-actions

Copy link
Copy Markdown

Code Coverage Summary

                             Total:|81.6%   2320|93.2%   133|    -      0

Copilot Bot review requested due to automatic review settings July 23, 2026 19:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (5)

.github/workflows/L1_L2_CoverageReport.yml:120

  • docker exec ... -c "... export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:..." expands $LD_LIBRARY_PATH on the GitHub runner before entering the container, so the container may get an empty/incorrect value (and an empty leading entry can unintentionally add the current directory to the loader search path). Use single quotes (or escape $) so expansion happens inside the container, and build the value without a leading colon.
          docker exec -i native-platform /bin/bash -c "cd /mnt/L2_CONTAINER_SHARED_VOLUME && sh cov_build.sh && export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/x86_64-linux-gnu:/lib/aarch64-linux-gnu:/usr/local/lib && sh run_l2.sh"

.github/workflows/L1_L2_CoverageReport.yml:9

  • Workflow-level permissions grants contents: write and pull-requests: write to all jobs, including jobs that execute PR-provided code. To reduce blast radius, default the workflow to read-only and grant write permissions only to the job(s) that actually need them (the docs commit + PR comment job).
permissions:
  contents: write
  pull-requests: write

.github/workflows/L1_L2_CoverageReport.yml:221

  • After reducing workflow-level permissions, update-docs needs explicit write permissions for pushing docs commits and posting/updating PR comments. Add job-level permissions here to keep other jobs least-privileged.
  update-docs:
    name: Update L1/L2 coverage markdowns
    runs-on: ubuntu-latest
    needs: [l1-coverage, l2-coverage]

.github/workflows/L1_L2_CoverageReport.yml:391

  • On fork-based PRs, pull_request workflows run with a read-only GITHUB_TOKEN, so the actions/github-script comment upsert will fail and can make the whole workflow red even though you already detect fork PRs for doc commits. Gate this step (or make it non-fatal) for fork PRs.
      - name: Upsert PR coverage comment
        uses: actions/github-script@v7
        env:
          COMMENT_PATH: /tmp/pr_coverage_comment.md
        with:

.github/workflows/L1_L2_CoverageReport.yml:17

  • This workflow pulls/runs multiple container images using the moving :latest tag. For supply-chain safety and reproducibility, pin these images to an immutable version (or ideally a digest, e.g. @sha256:...).
    container:
      image: ghcr.io/rdkcentral/docker-rdk-ci:latest

Comment on lines +144 to +163
import glob
import json

totals = {"collected": 0, "passed": 0, "failed": 0, "skipped": 0}
for path in glob.glob("/tmp/L2_TEST_RESULTS/*.json"):
try:
with open(path, "r", encoding="utf-8") as f:
data = json.load(f)
s = data.get("summary", {})
totals["collected"] += int(s.get("collected", s.get("total", 0) or 0))
totals["passed"] += int(s.get("passed", 0) or 0)
totals["failed"] += int(s.get("failed", 0) or 0)
totals["skipped"] += int(s.get("skipped", 0) or 0)
except Exception:
continue

print(f"L2_COLLECTED={totals['collected']}")
print(f"L2_PASSED={totals['passed']}")
print(f"L2_FAILED={totals['failed']}")
print(f"L2_SKIPPED={totals['skipped']}")
@github-actions

Copy link
Copy Markdown

Code Coverage Summary

                             Total:|81.6%   2320|93.2%   133|    -      0

@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown

PR Coverage Summary

Metric Value
L1 line coverage 81.6% (1893 / 2320)
L1 function coverage 93.2% (124 / 133)
L2 collected/passed/failed/skipped 30 / 22 / 8 / 0
L2 feature scenarios 35
L2 test functions 35
Estimated functional coverage 62.86%
Gap to 100% 37.14%

Coverage docs status: updated in PR branch

Updated files:

  • test/docs/L1_Test_Coverage.md
  • test/docs/L2_Test_Coverage.md

Copilot Bot review requested due to automatic review settings July 24, 2026 15:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (5)

.github/workflows/L1_L2_CoverageReport.yml:120

  • $LD_LIBRARY_PATH is expanded by the runner shell before docker exec runs, so the container receives the runner's value (often empty) instead of its own environment. Escape the $ so expansion happens inside the container.
      - name: Build and run L2 tests
        run: |
          docker exec -i native-platform /bin/bash -c "cd /mnt/L2_CONTAINER_SHARED_VOLUME && sh cov_build.sh && export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/x86_64-linux-gnu:/lib/aarch64-linux-gnu:/usr/local/lib && sh run_l2.sh"

.github/workflows/L1_L2_CoverageReport.yml:134

  • With set -euo pipefail, the grep | wc -l pipelines will fail when there are 0 matches (grep exits 1), which turns a valid "0 scenarios" case into a hard workflow failure. Wrap grep/find with || true before piping into wc.
          feature_files=$(find test/functional-tests/features -name '*.feature' | wc -l | tr -d ' ')
          feature_scenarios=$(grep -R "^[[:space:]]*Scenario:" test/functional-tests/features/ | wc -l | tr -d ' ')
          feature_scenarios_disabled=$(grep -R "^[[:space:]]*#[[:space:]]*Scenario:" test/functional-tests/features/ | wc -l | tr -d ' ' || true)

.github/workflows/L1_L2_CoverageReport.yml:138

  • Same set -euo pipefail issue for the test counts: if there are no def test_ matches, grep exits 1 and the step fails even though the correct count is 0.
          test_files=$(find test/functional-tests/tests -maxdepth 1 -name 'test_*.py' | wc -l | tr -d ' ')
          test_functions=$(grep -R "^def test_" test/functional-tests/tests/ | wc -l | tr -d ' ')
          test_functions_disabled=$(grep -R "^[[:space:]]*#[[:space:]]*def test_" test/functional-tests/tests/ | wc -l | tr -d ' ' || true)

.github/workflows/L1_L2_CoverageReport.yml:140

  • src_functions uses grep without -E but the pattern relies on ERE grouping ((...)) semantics; in basic regex this changes the meaning and can severely skew the count. Also, this pipeline can fail with pipefail when there are 0 matches. Use -E (or adjust the pattern) and guard the pipeline similarly.
          src_functions=$(grep -R "^[A-Za-z_][A-Za-z0-9_[:space:]\*]*([^;]*)[[:space:]]*{" rfcMgr/ rfcapi/ tr181api/ utils/ --include='*.cpp' --include='*.c' | wc -l | tr -d ' ')

.github/workflows/L1_L2_CoverageReport.yml:391

  • On pull_request workflows, fork PRs do not get a write-capable GITHUB_TOKEN, so this step will fail when trying to create/update a PR comment. Gate the upsert step to same-repo PRs (matching the commit step) or make it non-fatal.
      - name: Upsert PR coverage comment
        uses: actions/github-script@v7
        env:
          COMMENT_PATH: /tmp/pr_coverage_comment.md
        with:

@github-actions

Copy link
Copy Markdown

Code Coverage Summary

                             Total:|81.6%   2320|93.2%   133|    -      0

Copilot Bot review requested due to automatic review settings July 24, 2026 16:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants