RDKOSS-954: Extraction of SHA256 checksum values for downloading/consolidating layered CVE artifacts

[jomothomas]

This PR supports extraction of SHA256 checksum values for downloading/consolidating layered CVE artifacts

Changes:
Adds a new generate-cve-sha256.bbclass with functions to fetch layer sha256 checksum required for downloading CVE artifacts to generate CVE reports and consolidated manifests by populating a dynamic-sha.inc file.
Adds conditional inherit of this class depending on "Enable CVE check" for CVE packages accordingly.

[Copilot]

Pull request overview

Adds a new BitBake class to extract per-layer CVE artifact SHA256 values at parse time and persist them into an include file for reuse across BitBake invocations.

Changes:

• Introduces generate-cve-sha256.bbclass to parse CVE_LAYER_FEED_PATH, HEAD each artifact URL, and read X-Checksum-Sha256.
• Writes extracted checksums into ${TOPDIR}/conf/dynamic_sha.inc and sets them in the current datastore.
• Adds a sentinel mechanism intended to avoid repeated network fetches.

Suppressed comments (1)

classes/generate-cve-sha256.bbclass:108

• The sentinel file is written with an empty payload, so even if you later change CVE_LAYER_FEED_PATH there’s no way to detect that and force regeneration. Writing the computed signature into the sentinel allows safe cache invalidation.

    # Write sentinel so subsequent recipe parses skip this block
    with open(sentinel, 'w') as f:
        f.write('')

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

Suppressed comments (5)

classes/generate-cve-sha256.bbclass:110

• The SHA256 value is only checked for being non-empty; it should be validated as a 64-hex digest to avoid persisting malformed header values into dynamic_sha.inc.

        if not sha:
            bb.warn("generate-cve-sha256: empty X-Checksum-Sha256 header for %s (%s) - skipping entry"
                    % (var_name, url))
            continue

classes/generate-cve-sha256.bbclass:123

• The sentinel file is written unconditionally. If all entries are skipped (e.g., transient network failure), the sentinel will still prevent retries in later parses, leaving dynamic_sha.inc missing/empty until the user manually deletes the sentinel.

    # Write sentinel so subsequent recipe parses skip this block
    with open(sentinel, 'w') as f:
        f.write('')

classes/generate-cve-sha256.bbclass:51

• PR description mentions writing dynamic-sha.inc, but this class writes ${TOPDIR}/conf/dynamic_sha.inc. Please align the intended filename between documentation/PR description and implementation (and any consumers that include it).

    inc_file = os.path.join(topdir, 'conf', 'dynamic_sha.inc')

    bb.note("generate-cve-sha256: generating CVE feed SHA256 checksums -> %s" % inc_file)

classes/generate-cve-sha256.bbclass:29

• This class performs network I/O at parse time via urllib, which can make parsing non-deterministic and may bypass BitBake fetch controls (e.g., BB_NO_NETWORK, mirrors, proxies). Consider moving the fetch into a task using BitBake’s fetcher, or explicitly honoring BB_NO_NETWORK.

python () {
    import re
    import urllib.request
    import urllib.parse

classes/generate-cve-sha256.bbclass:8

• This comment references generate_sha256.sh, but no such file exists in this repository. This is misleading for future maintainers.

#      (mirrors the curl --netrc -sI -L behaviour of generate_sha256.sh)

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Suppressed comments (6)

classes/generate-cve-sha256.bbclass:50

• PR description calls out a dynamic-sha.inc file, but the implementation writes dynamic_sha.inc (underscore). If the intent is to create dynamic-sha.inc, rename this to match so downstream includes/use are consistent.

    inc_file = os.path.join(topdir, 'conf', 'dynamic_sha.inc')

classes/generate-cve-sha256.bbclass:10

• PR description mentions populating a "dynamic-sha.inc" file, but this class documents/writes "dynamic_sha.inc" (underscore). Please align the filename with the PR description (or update the PR description if underscore is intended).

#   4. Appends  VAR = "SHA"  to ${TOPDIR}/conf/dynamic_sha.inc so the

classes/generate-cve-sha256.bbclass:49

• This class performs network access at parse time via urllib. As written, it will still attempt network fetches even when BitBake is configured for offline/no-network operation (e.g., BB_NO_NETWORK="1"). It should explicitly respect BB_NO_NETWORK to avoid unexpected parse-time failures in restricted build environments.

    cve_layer_feed_path = d.getVar('CVE_LAYER_FEED_PATH') or ''
    if not cve_layer_feed_path.strip():
        bb.debug(1, "generate-cve-sha256: CVE_LAYER_FEED_PATH is not set, nothing to do")
        return

classes/generate-cve-sha256.bbclass:124

• The sentinel file is written unconditionally even if no SHA256 values were successfully generated (e.g., all entries failed to fetch/parse). That would cause subsequent parses in the same cooker session to skip regeneration and leave dynamic_sha.inc missing/empty.

    # Write sentinel so subsequent recipe parses skip this block
    with open(sentinel, 'w') as f:
        f.write('')

classes/generate-cve-sha256.bbclass:8

• The header comment references generate_sha256.sh, but that script doesn’t exist in this repository (only this bbclass mentions it). This makes the documentation misleading for anyone trying to trace the behavior.

#      (mirrors the curl --netrc -sI -L behaviour of generate_sha256.sh)

classes/generate-cve-sha256.bbclass:56

• The class removes and rewrites ${TOPDIR}/conf/dynamic_sha.inc without any file locking. If two BitBake processes share the same TOPDIR (or if multiple parses overlap), this can race and corrupt the include. In this repo, classes/tag_to_sha_converter.bbclass uses fcntl.flock (e.g., lines 34-47) to serialize cache file access; a similar lock around dynamic_sha.inc + the sentinel would make this robust.

    # Remove any stale inc file before writing fresh values
    if os.path.exists(inc_file):
        os.remove(inc_file)