Try#176
Open
TamarW0 wants to merge 365 commits into
Open
Conversation
Signed-off-by: Vladimir Belousov <vbelouso@redhat.com>
Signed-off-by: Vladimir Belousov <vbelouso@redhat.com>
bug: correctly handle checkouts in SourceCodeGitLoader
Signed-off-by: Vladimir Belousov <vbelouso@redhat.com>
ci: enable unit tests with shared cache storage
* feat: Add C/C++ support to transitive code search tool - Add C language support and enhance function parsing capabilities - Implement singleton pattern for RPMDependencyManager - Add container source download for C/C++ dependencies - Add C extended segmenter class with comprehensive testing - Add new Function Name Locator tool with fuzzy matching - Enhance transitive code search with assistant tool - Add C/C++ function parsers and language support - Add comprehensive test suite for C code analysis - Improve function name extraction and parsing - Add source RPM downloader for C/C++ dependencies - Update configuration files for NIM and OpenAI endpoints - Fix various bugs and improve code quality - Add proper documentation and type hints - Clean up imports and remove debug prints This commit consolidates 39 individual commits that add comprehensive C/C++ language support to the vulnerability analysis toolchain. * update docs, kustomize service config, tests and rpm downloader support UT * review fix move C_DEP_LIBS_NAME locaiton * fixed review comments * Add cache for std lib names for lang python,go,java,javascript * Update src/vuln_analysis/utils/standard_library_cache.py Co-authored-by: Zvi Grinberg <75700623+zvigrinberg@users.noreply.github.com> * fix indentation * add persistance to cache * code review comments * add comments and update docker for bsdtar app to be install in image * update docfile * small improvments * handle invalid utf-8 files --------- Co-authored-by: Zvi Grinberg <75700623+zvigrinberg@users.noreply.github.com>
…as escape chars. Originally was fixed in old git commit 710ab46, which belongs to commits tree of the previous version Signed-off-by: Zvi Grinberg <zgrinber@redhat.com>
…-parsing-error fix: Fix checklist escaping parsing error
Signed-off-by: Zvi Grinberg <zgrinber@redhat.com>
Signed-off-by: Zvi Grinberg <zgrinber@redhat.com>
…check Signed-off-by: Zvi Grinberg <zgrinber@redhat.com>
fix: golang tests failed for some standard libs checks
* fix: csegmenter sometimes place two functions in segment * create UT for c segmenter
, just log them and replace exception message in response with configured value as the output for the checklist item' answer. Signed-off-by: Zvi Grinberg <zgrinber@redhat.com>
…agent-loop fix: do not throw exceptions in agent loop.
Signed-off-by: Zvi Grinberg <zgrinber@redhat.com>
Signed-off-by: Zvi Grinberg <zgrinber@redhat.com>
…gging feat: improve tools errors handling and logging
Signed-off-by: Vladimir Belousov <vbelouso@redhat.com>
feat: add caching of Go modules to persistent storage
* chore: update kustomize files to include argille * feat: adding argilla deployment folder to kustomize-base * chore: updating image in deployment file * chore: updating to a temporary image * fix: creating secret to argilla environment variables * chore: add Argilla pull secret instructions to README * fix: update argilla pull secret on README file * fix: update image on deployment file * refactor: move argilla secrets to dedicated base/argilla/feedback_secret.env * docs: exchange the order of level 2 and 3 * fix: revert to resources on kustomization file
Signed-off-by: Zvi Grinberg <zgrinber@redhat.com>
* bugfix in the testing env * update tool descriptions for clarity * refactor tool names to be class constants instead of disparate strings * add initial unit tests * rename tool names to be more consistent and distinct * update unit tests with tool names and tool constants * cleanup startup guide notebook * rework intel source score section * update agent execution stage prompts and make tool descriptions dynamic * add tests for dynamic tool descriptions * revamp the tool description list, as well as the checklist prompt for better clarity and quality * revamp checklist prompt implementation, as well as add in dynamic tool descriptions to both checklist and agent prompts * update tests for tool descriptions * add more detailed agent examples with more useful MRKL-formatted steps * update for summary prompt * update justification prompt with more logic and explanations on how to pick the class * update CVSS prompts and cleanup examples and guidance * bugfix on intel source * bug patch for vdb generation adding in constants during the vdb generation check * bugfix by Tamar * update register_function() and transitive_search() descriptions * bugfix in the testing env * update tool descriptions for clarity * refactor tool names to be class constants instead of disparate strings * add initial unit tests * rename tool names to be more consistent and distinct * update unit tests with tool names and tool constants * cleanup startup guide notebook * rework intel source score section * update agent execution stage prompts and make tool descriptions dynamic * add tests for dynamic tool descriptions * revamp the tool description list, as well as the checklist prompt for better clarity and quality * revamp checklist prompt implementation, as well as add in dynamic tool descriptions to both checklist and agent prompts * update tests for tool descriptions * add more detailed agent examples with more useful MRKL-formatted steps * update for summary prompt * update justification prompt with more logic and explanations on how to pick the class * update CVSS prompts and cleanup examples and guidance * bugfix on intel source * bug patch for vdb generation adding in constants during the vdb generation check * bugfix by Tamar * update register_function() and transitive_search() descriptions * add function locator descriptions * add names to configs * add local output for local testing * bugfix * Update tool_names.py
* chore: add pvc for postgresql and elasticsearch * fix: update pvc settings and names
Fixes: https://issues.redhat.com/browse/APPENG-4088 Signed-off-by: Zvi Grinberg <zgrinber@redhat.com>
fix: regressive bugs of agent
* Java transitive search Signed-off-by: Theodor Mihalache <tmihalac@tmihalac-thinkpadp1gen7.rmtusfl.csb>
Signed-off-by: Vladimir Belousov <vbelouso@redhat.com>
Signed-off-by: Vladimir Belousov <vbelouso@redhat.com>
In addition, in case credential is not provided and when cloning for git repo result with user/pass prompt, fail the operation * PPENG-5400: Fail scan request as soon as echosystem fail to be detected for the the git repo * Handle error none exist of private repository when expect to be public repository by the code * add unit tests for public source code clone --------- Co-authored-by: Gal Netanel <gnetanel@redhat.com>
- Add dedicated ServiceAccount, Role, and RoleBinding for MCP server (anyuid SCC) - Switch MCP server image to quay.io/ecosystem-appeng/ registry - Add runAsNonRoot security context and EXPLOITIQ_READ_ONLY/REDACT_RESPONSES env vars - Update kustomization image reference to match new registry - Upgrade git loader fetch failure logs from debug to warning for better troubleshooting - Deduplicate checked_not_vulnerable packages using seen_not_vulnerable set - Improve justification output: group packages by shared reason, show package names without version * Address PR #257 review feedback: - Restore git error details in warning logs for fetch failure diagnostics - Include package version in justification output to disambiguate duplicates Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
…te-rendering feat(nginx): replace docker.io nginx with Red Hat hardened image
This PR adds Commit Discovery Pipeline - a 4-phase enhancement that extracts fix commit information from upstream repositories when no downstream patch is available. This enables better vulnerability analysis by discovering the actual fix code from advisory references. Key Features OSIDB Intel Integration - New intel source for internal Red Hat users Reference Mining - Extract commit hints from advisory URLs Repository Resolution - Map package names to upstream git repositories Git Commit Search - Clone repos and find fix commits using extracted hints
* Rewrite JS parser and Segmenter:
- Replaced esprima-based JavaScript segmenter with tree-sitter for reliable
parsing of modern JS syntax (optional chaining, nullish coalescing, top-level
await)
- Fixed JS function name extraction: keyword filtering, position-aware matching,
redundant pattern removal, generator/TypeScript/anonymous-export support
- Added build-artifact filtering (should_skip) that excludes app-level dist/,
build/static/, .min.js while preserving node_modules/*/dist/ as legitimate
third-party source
- Added empty-name guards in CCA BFS to prevent documents with unextractable
function names from entering call-chain analysis
- Fixed _get_function_calls regex to detect calls through optional chaining
(obj?.method())
* Remove parser_threshold parameter from ExtendedLanguageParser creation
* Fix JS get_function_name catastrophic regex backtracking and harden JS parser
- Replace 3 catastrophic regex patterns (P4/P9/P11) in get_function_name with
safe arrow_header check + simple name extraction, eliminating 12-38s hangs
- Add arrow_header extension for destructured params like ({a,b}) => {body}
where first '{' is in params, with guard to skip non-arrow functions
- Use negative lookahead =(?!>) in P9 to distinguish assignment from arrow
- Fix should_skip to only match top-level exclude dirs (dist/, build/) not
nested ones (node_modules/*/dist/)
- Add thread-safe double-checked locking for tree-sitter language init
- Refactor class/method extraction to handle anonymous classes, generators,
getters/setters, and export patterns
- Fix is_function to use class\b word boundary instead of class\s+[\w$]+
- Strip optional chaining '?' from _get_function_calls results
- Guard empty function names in CCA and create_map_of_local_vars with
ValueError handling
- Add direct export detection (export function/const) in is_function_exported
- Add 387 JS parser tests and 139 JS segmenter tests covering backtracking
regression, arrow functions, anonymous classes, and should_skip filtering
* Harden JS parser and CCA: $-sign support, alias recursion, empty-name guards
- Fix _build_class_hierarchy regex to include $ in identifier char class ([\w$]+)
- Pass code_documents to recursive _get_function_calls alias resolution calls
- Add empty-name guard in CCA BFS to skip documents with unextractable function
names
- Add try/except + empty guard in function_name_extractor for robust name
extraction
- Use [-1] instead of [1] for get_package_names to handle variable-length returns
- Simplify should_skip with direct index checks instead of loop
- Add regex comments describing direct-call and callback-reference patterns
- Add 8 tests: $-sign class hierarchy (6) and chained alias resolution (2)
- Fix test assertions: generator star spacing, empty-string guard, jQuery dollar
detection
* - Log source path of documents skipped due to empty get_function_name
result in __find_initial_function
* Clean up test comments and add _JS_KEYWORDS intent documentation
- Add comment explaining _JS_KEYWORDS filters control-flow false
positives, not invalid identifiers
- Remove all "Fix X", "Fix A-I", and "Pattern N" references from test
comments and docstrings
- Replace internal references with behavioral descriptions of what
each test verifies
- Add warning log when skipping documents with empty function names
in CCA __find_initial_function
Signed-off-by: Theodor Mihalache <tmihalac@redhat.com>
…#252) * APPENG-5327: generate VEX report even when no vulnerabilities are found * Update flags labels with thh correct value * Add uuid to tracking-id to add uniquness to the id * Add purl to vex report * Bugfix in purl for oci-image * Remove namespace from oci purl * Update full image name in repository_url --------- Co-authored-by: Gal Netanel <gnetanel@redhat.com>
Signed-off-by: Zvi Grinberg <zgrinber@redhat.com>
Signed-off-by: Zvi Grinberg <zgrinber@redhat.com>
Signed-off-by: Zvi Grinberg <zgrinber@redhat.com>
…-revamp docs(kustomize): revamp deployment guide, add RBAC and overlay improvements
* BugFix 1.support package flavors kernel for example rt or debug need to parse and find correct config file * BugFix 2. in case grep tool return empty results need to clasify the results correctly. * BugFix 3. grep tool failed in multi query make robust
…oading-twice fix: prevent documents loading process from running twice
…store oc get route
docs(kustomize): fix OAuth flow — restore oc get route for redirectURIs
* bugFix rpm check: fine tune prompt for grep kernel config flags * BugFix remove .git/index files from lexical search or docs * BugFix separate observation seeding and add token-bounded critical-context merge in BaseGraphAgent
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.