RDKOSS-580: Implement recipes_info.bbclass - #106
Conversation
Reason for this change: Implement a bbclass that outputs recipe metadata after parsing, without triggering any build tasks. Signed-off-by: mselva006c <mani_selvaraj@comcast.com>
There was a problem hiding this comment.
Pull request overview
This PR implements a new BitBake class (recipes_info.bbclass) that outputs recipe metadata to a file after recipe parsing, without triggering build tasks. The implementation collects key recipe information (PN, PV, PR, PE, PACKAGE_ARCH, recipe path, and SRC_URI) and writes it to a configurable output file in a delimited format.
Key changes:
- New
recipes_info.bbclasswith a custom task that extracts and logs recipe metadata before the fetch stage - Automatic enablement of the class through
user-classes.incconfiguration
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 9 comments.
| File | Description |
|---|---|
| classes/recipes_info.bbclass | New bbclass implementing do_print_recipes_info task to capture recipe metadata (PN, PV, PR, PE, PACKAGE_ARCH, SRC_URI) and write to a delimited log file with configurable output format |
| conf/include/user-classes.inc | Enables recipes_info class globally by appending it to USER_CLASSES |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if not os.path.exists(log_file): | ||
| # Create empty file | ||
| with open(log_file, "w"): | ||
| pass |
There was a problem hiding this comment.
The empty file creation pattern (lines 64-67) can be simplified. Using open(log_file, "w") and immediately closing it without writing anything is inefficient. Consider using open(log_file, "a") mode when reading/writing, or use Path(log_file).touch() from pathlib to create an empty file more idiomatically.
| # Ensure the task runs before fetch, so SRC_URI/SRCREV are available but | ||
| # we capture info early in the pipeline. | ||
| addtask do_print_recipes_info before do_fetch |
There was a problem hiding this comment.
The task is scheduled to run before do_fetch, but the comment (line 97) states it should capture SRC_URI/SRCREV info. However, SRCREV values may not be fully resolved until fetch time, especially for auto-increment SRCREV values or when using AUTOREV. Consider whether this task should run after parsing but verify if SRCREV is actually available at this stage. If SRCREV needs to be included (as the comment suggests but the code doesn't currently do), the task ordering may need adjustment.
| srcuri_str = " ".join(src_list) | ||
|
|
||
| # Prepare the line to write/update | ||
| # Format: PN:PV-PR-PE:PACKAGE_ARCH:recipe_name:SRC_URI:SRCREV |
There was a problem hiding this comment.
The comment states the format includes "SRCREV" but the actual code (lines 54-60) doesn't include SRCREV in the output. Either update the comment to reflect the actual format (PN:PV-PR-PE:PACKAGE_ARCH:recipe_name:SRC_URI) or add SRCREV to the output if it's intended to be included.
|
|
||
| # Build a stable, deduped SRC_URI string | ||
| srcuri = d.getVar("SRC_URI", True) or "" | ||
| src_list = sorted(set(srcuri.split())) |
There was a problem hiding this comment.
Sorting the SRC_URI entries (line 49) may not be ideal because the order of entries in SRC_URI can be significant in BitBake (e.g., patches are applied in order). While deduplication is useful, consider preserving the original order by using a pattern like list(dict.fromkeys(srcuri.split())) which removes duplicates while maintaining order, instead of sorted(set(...)).
| src_list = sorted(set(srcuri.split())) | |
| src_list = list(dict.fromkeys(srcuri.split())) |
| delim = d.getVar("PRINT_SRC_DELIM", True) or ":" | ||
| log_file = d.getVar("PRINT_SRC_LOG_FILE", True) or os.path.join(d.getVar("TMPDIR", True), "recipes_info.txt") | ||
|
|
||
| # Collect metadata | ||
| pkg_pn = d.getVar("PN", True) or "" | ||
| pkg_pv = d.getVar("PV", True) or "" | ||
| pkg_pr = d.getVar("PR", True) or "" | ||
| pkg_pe = d.getVar("PE", True) or "" | ||
| pkg_arch = d.getVar("PACKAGE_ARCH", True) or "" | ||
| bbfn = d.getVar("BB_FILENAME", True) or "" | ||
|
|
||
| # Derive recipe_name, with optional path prefix stripping | ||
| prefix = d.getVar("PRINT_SRC_RECIPE_PATH_PREFIX", True) or "" |
There was a problem hiding this comment.
The second parameter True in d.getVar() calls is deprecated in modern BitBake versions. This parameter used to control variable expansion but is now the default behavior. Remove the True parameter from all d.getVar() calls throughout this file (lines 19, 20, 23-28, 31, 48).
| # Read once, update or append | ||
| updated = False | ||
| lines = [] | ||
| with open(log_file, "r", encoding="utf-8") as f: | ||
| lines = f.readlines() | ||
|
|
||
| # Replace the line that starts with "PN<delim>" | ||
| start_token = f"{pkg_pn}{delim}" | ||
| for idx, line in enumerate(lines): | ||
| if line.startswith(start_token): | ||
| if line != line_value: | ||
| lines[idx] = line_value | ||
| updated = True | ||
| else: | ||
| # Already identical: nothing to do | ||
| updated = False | ||
| break | ||
| else: | ||
| # Not found; append | ||
| lines.append(line_value) | ||
| updated = True | ||
|
|
||
| # Write only if changed | ||
| if updated: | ||
| with open(log_file, "w", encoding="utf-8") as f: | ||
| f.writelines(lines) |
There was a problem hiding this comment.
This file I/O implementation lacks proper file locking for concurrent access. When BitBake processes multiple recipes in parallel, multiple tasks can simultaneously read, modify, and write to the same log file, leading to race conditions and data corruption. Implement file locking (using fcntl.flock) similar to the pattern used in tag_to_sha_converter.bbclass to ensure thread-safe file operations.
| # Format: PN:PV-PR-PE:PACKAGE_ARCH:recipe_name:SRC_URI:SRCREV | ||
| line_value = delim.join([ | ||
| pkg_pn, | ||
| f"{pkg_pv}-{pkg_pr}-{pkg_pe}", |
There was a problem hiding this comment.
The formatting of version string includes PE (Package Epoch) even when it's empty or not set. This will result in output like "1.0-r0-" with a trailing hyphen when PE is empty. Consider omitting PE from the output when it's empty, or use a different format that handles this case more gracefully (e.g., only include "-PE" when PE is non-empty).
| f"{pkg_pv}-{pkg_pr}-{pkg_pe}", | |
| f"{pkg_pv}-{pkg_pr}" + (f"-{pkg_pe}" if pkg_pe else ""), |
| recipe_name = os.path.basename(bbfn) | ||
|
|
||
| # Build a stable, deduped SRC_URI string | ||
| srcuri = d.getVar("SRC_URI", True) or "" |
There was a problem hiding this comment.
The second parameter True in d.getVar() is deprecated. Remove it to follow modern BitBake conventions.
| srcuri = d.getVar("SRC_URI", True) or "" | |
| srcuri = d.getVar("SRC_URI") or "" |
| try: | ||
| if prefix: | ||
| # Remove everything up to and including the first occurrence of "/{prefix}/" | ||
| marker = f"/{prefix}/" | ||
| if marker in bbfn: | ||
| recipe_name = bbfn.split(marker, 1)[1] | ||
| else: | ||
| # Fallback to basename if prefix not found | ||
| recipe_name = bbfn | ||
| else: | ||
| recipe_name = bbfn | ||
| except Exception: | ||
| recipe_name = os.path.basename(bbfn) |
There was a problem hiding this comment.
The exception handling in the recipe_name derivation is too broad. If an exception occurs (line 44), the code falls back to using basename(bbfn), but the try block only contains operations that shouldn't raise exceptions under normal circumstances (string operations). More critically, if the prefix splitting succeeds but results in an unexpected value, the code won't catch it. Consider removing the try-except block or making the exception handling more specific to avoid masking genuine errors.
| try: | |
| if prefix: | |
| # Remove everything up to and including the first occurrence of "/{prefix}/" | |
| marker = f"/{prefix}/" | |
| if marker in bbfn: | |
| recipe_name = bbfn.split(marker, 1)[1] | |
| else: | |
| # Fallback to basename if prefix not found | |
| recipe_name = bbfn | |
| else: | |
| recipe_name = bbfn | |
| except Exception: | |
| recipe_name = os.path.basename(bbfn) | |
| if prefix: | |
| # Remove everything up to and including the first occurrence of "/{prefix}/" | |
| marker = f"/{prefix}/" | |
| if marker in bbfn: | |
| recipe_name = bbfn.split(marker, 1)[1] | |
| else: | |
| # Fallback to basename if prefix not found | |
| recipe_name = bbfn | |
| else: | |
| recipe_name = bbfn |
Reason for this change: Implement a bbclass that outputs recipe metadata after parsing, without triggering any build tasks.
bitbake --runonly print_recipes_info