CCA pre-filters (Java import, C arg-count, Go sub-package), Java CCA cycle detection & perf, RPM version guard, ~40 bug fixes, ~200 new tests

[tmihalac]

Java CCA

• Add _can_reference_class() import-based pre-filter (simple class name, wildcard import, same package, same artifact) to eliminate irrelevant uber-JAR candidates before expensive type resolution
• Only filter third-party candidates; application code (root docs) always passes to avoid false negatives from polymorphic interface calls
• Add DFS cycle detection guard in get_relevant_documents to prevent infinite loops from self-recursive or mutually recursive method calls
• Switch logger.debug in __check_identifier_resolved_to_callee_function_package from f-strings to %s lazy formatting

C CCA

• Add argument-count pre-filter to C parser's search_for_called_function to reject cross-package false positives where a same-named function has different arity (e.g. rsync read_byte(1) vs PostgreSQL read_byte(3))

Go CCA

• Add Go sub-package awareness to Function Locator (_extract_go_subpackage, _go_subpackage_flow_control)
• Add CCA sub-package filtering via resolve_subpackage_to_module in Go parser and tree fallback in chain_of_calls_retriever
• Add Go sub-package enrichment from patches in intel_utils (extract_go_subpackages_from_patch)
• Add Go sub-package prefix matching to Rule 8 so github.com/lib/foo/bar matches target github.com/lib/foo
• Fix Go FL short_name dict collision: store list of packages per short name instead of overwriting

CCA / all ecosystems

• Pre-index documents in chain_of_calls_retriever.__init__ (sort_docs, _root_docs, _source_path_index) for O(1) lookups in get_possible_docs instead of scanning all documents
• Optimize _is_doc_excluded to compare source path (cheap) before page content (expensive)
• Deduplicate parents list in non-Java CCA tree_dict to prevent duplicate entries from dependency tree builder
• Add distinct "function not found" message when CCA returns empty call_hierarchy_list so agent distinguishes missing function from unreachable function
• Escape regex metacharacters in Function Caller Finder query builder to handle identifiers containing dots, brackets, and plus sign
• Reorder direct_parents in __find_caller_function_dfs so root-level packages are searched before library packages, fixing nondeterministic JS transitive search timeouts

JS parser fixes

• Cap get_function_name regex searches to first 2000 chars to prevent catastrophic backtracking on huge functions
• Fix is_comment_line to handle *-prefixed JSDoc continuation lines vs generator *method() syntax
• Fix _extract_class_name regex to support $ in JS identifiers (\w+ → [\w$]+)
• Fix resolve_chain backreference [^\1] in string pattern (changed to .*?)
• Remove unused is_multiline parameter from _parse_declarations

RPM checker

• Emit TARGET_IN_VULNERABLE_RANGE in VulnerabilityIntel.format_for_prompt() so the L1 agent sees the field referenced by the version-based fallback rules
• Add VERSION GUARD clause to Case B sys prompt: when target is in vulnerable range, a grep match alone is not sufficient to conclude PATCHED — the fix must be at the exact CVE location
• Shorten cve_verify_vuln_package LLM response instruction to one sentence

Config scanner

• Add _CONFIG_DIR_ALLOWED_EXTENSIONS allowlist for directory-matched files, filtering out .js/.css/.map/.java/.py from config collection
• Add build/tool config patterns: pyproject.toml, setup.cfg, tox.ini, tsconfig.json, .eslintrc.json, Makefile, CMakeLists.txt, meson.build

Source code bug fixes (~40)

• Fix dep_tree.py missing comma causing --pythonvenv_python string concatenation
• Fix dep_tree.py C/C++ detect_ecosystem walk not using _WALK_EXCLUDE_DIRS
• Fix dep_tree.py remove unused _ensure_venv method
• Fix dep_tree.py deduplicate types- package candidates via dict.fromkeys
• Fix c_segmenter_custom.py remove_comments stripping patterns inside string literals
• Fix c_lang_function_parsers.py debug print statement left in production code
• Fix golang_functions_parsers.py len(declaration_parts) == (2 or 3) always-true comparison
• Fix golang_functions_parsers.py no-op re.search("") call
• Fix golang_functions_parsers.py is_package_imported raw string split, missing quote stripping, and unescaped identifier in regex
• Fix golang_functions_parsers.py is_same_package crash on empty input
• Fix python_functions_parser.py is_same_package returning True for two empty strings
• Fix python_segmenters_with_classes_methods.py annotating all methods with last class name and skipping async def methods
• Fix source_code_git_loader.py safe.directory guard unnecessarily gated on clone_url
• Fix brew_downloader.py returning path with zero downloads and extracting SRPM from cache path instead of target path
• Fix configuration_scanner.py re.match allowing partial filename matches (fullmatch), max_results=0 returning 1 result, cache race condition on concurrent eviction, and missing docker-compose*.yaml pattern
• Fix import_usage_analyzer.py empty short_name matching everything
• Fix async_http_utils.py off-by-one in retry count, consumer errors caught by retry loop, retry_on_client_errors overridden by Retry-After check, negative sleep from past X-RateLimit-Reset, and missing @functools.wraps
• Fix function_name_locator.py python_flow_control crash on non-function documents, Go versioned module short-name collision, and get_function_name ValueError exception handling
• Fix git_commit_searcher.py _rank_results mutating confidence in-place
• Fix git_repo_manager.py double-wrapping GitCommandError
• Fix intel_utils.py parse_cpe checking split_cpe[5] instead of split_cpe[10] for system
• Fix llm_engine_utils.py assert False in production code replaced with RuntimeError
• Fix repo_resolver.py case-sensitivity bug in normalize_package_name dropping original case for mixed-case JSON keys
• Fix serp_api_wrapper.py key index not reset after full rotation, dead max_retries field, and callers passing removed field
• Fix csaf_generator.py GHSA description dropped when no pre-existing note and notes appended with text: None
• Fix web_patch_fetcher.py missing asyncio import, _is_commit_url false positive on /c/ outside kernel.org, dropping Gitiles commit URLs, yarl double-encoding %5E%21, and rewrite _fetch_gitiles_patch from sync requests to async aiohttp
• Fix prompting.py build_tool_descriptions missing FL, CONFIG, IUA, GREP entries
• Fix java_functions_parsers.py _count_call_args treating < comparison as generic bracket

Build & infra

• Parallelize source JAR extraction using ThreadPoolExecutor with cgroup-aware worker count
• Add -Dmaven.artifact.threads=10 to all mvn dependency:copy-dependencies invocations in dep_tree.py
• Set MAVEN_OPTS with -Dmaven.repo.local on shared PVC in on-cm-runner.yaml, on-pull-request.yaml, and exploit_iq_service.yaml
• Add GOCACHE env var to on-cm-runner.yaml, on-pull-request.yaml, exploit_iq_service.yaml
• Increase batch runner CPU request/limit from 1/2 cores to 3/3 cores

Tests

• Fix tautological assertions (disjunctive or, truthiness-only, conditional if-then-assert) and tests that reimplemented source logic instead of calling real functions
• Consolidate 9 duplicate in-tree test files into canonical tests/ location
• Add ~200 new tests: Java CCA (_can_reference_class, function_called_from_caller_body, __find_caller_function), JS parser/segmenter (backtracking cap, comment line, $ identifiers), C parser (argument-count filter, find_top_level_blocks), Go sub-package (FL, CCA, intel enrichment), RPM checker (format_for_prompt, VERSION GUARD), config scanner (allowlist, build patterns), and tools (FL, IUA, SerpAPI, git, async HTTP, web patch fetcher)

[vbelouso]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Code Security	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[tmihalac]

/test-heavy

[tmihalac]

/test-heavy

[tmihalac]

/test-heavy

[tmihalac]

/test-heavy

[tmihalac]

/test vulnerability-analysis-on-pr

[tmihalac]

/test-heavy

[tmihalac]

/test vulnerability-analysis-on-pr

[tmihalac]

/test vulnerability-analysis-on-pr

[tmihalac]

/test vulnerability-analysis-on-pr

[tmihalac]

/test-heavy

[tmihalac]

/test vulnerability-analysis-on-pr

[tmihalac]

/test vulnerability-analysis-on-pr

[tmihalac]

/test vulnerability-analysis-on-pr

[tmihalac]

/test-heavy

[tmihalac]

/test-heavy

[tmihalac]

/test-heavy

[tmihalac]

/test vulnerability-analysis-on-pr

[tmihalac]

/test-heavy

[tmihalac]

/test-heavy

[zvigrinberg]

Hi @tmihalac ,

Please see the comments below.

[zvigrinberg]

Bug: module_path receives unresolved short name — sub-package disambiguation is dead code

The refactoring that introduced packages_to_search stopped reassigning package to the resolved full module path. This call:

return self._go_subpackage_flow_control(function, package_docs, package)

passes the original short name (e.g., 'protojson') as module_path. Inside _extract_go_subpackage, the check dir_path.startswith(norm_module + '/') never matches a short name against a full source path like google.golang.org/protobuf/encoding/protojson/.... Every function falls through to return module_path, collapsing all sub-packages into one bucket — so len(subpkg_to_funcs) <= 1 is always true and the multi-sub-package logic never triggers.

Fix: Pass the resolved module path(s) from packages_to_search:

return self._go_subpackage_flow_control(function, package_docs, packages_to_search[0])

Or iterate over all resolved paths if multiple are possible.

[zvigrinberg]

Bug: Comma counting ignores string/char literals — causes false rejections

_count_call_site_args iterates characters tracking only parenthesis depth:

for ch in args_str:
    if ch == '(':
        depth += 1
    elif ch == ')':
        depth -= 1
    elif ch == ',' and depth == 0:
        count += 1

This doesn't skip commas inside string or character literals. A call like func("error: a,b", x) is counted as 3 args instead of 2, causing a spurious mismatch against the declared parameter count and rejecting a valid call chain.

This is a common pattern in C code (format strings, error messages like fprintf(stderr, "expected %d, got %d", a, b)).

The Java counterpart _count_call_args has a related issue where < in comparison expressions (e.g., a < b) is treated as a generic angle-bracket opener.

[zvigrinberg]

Type mixing: Guidance string appended to function-name list

result = list(close_matches)
result.append(guidance)
return result

This appends a multi-line INFO string (e.g., "INFO: Matched functions exist in multiple sub-packages of 'module':\n subpkg1: func1\n...") to a list that otherwise contains only function names. The caller locate_functions returns this as "result": result, documented as [function_names].

While the LLM consumer may parse this gracefully, any programmatic downstream code that iterates over result treating every element as a valid function name will produce incorrect lookups or regex errors when it hits the guidance text (which contains spaces, colons, and newlines).

Suggestion: Return the guidance separately:

return {"functions": list(close_matches), "guidance": guidance}

Or log it instead of embedding it in the return value.

[zvigrinberg]

Race condition: Resetting shared _serp_api_key_index outside the lock

self.__class__._serp_api_key_index = 0
raise Exception("All API keys exhausted")

_serp_api_key_index is a ClassVar shared across all instances. All other mutations (validate_serp_api_keys, _rotate_next_key) properly guard writes with _key_rotation_lock, but this reset happens outside the lock.

If a concurrent caller is mid-rotation (e.g., on key index 3 of 5), this unlocked reset to 0 causes it to re-try already-exhausted keys (getting 402/429 loops) or skip keys it hasn't tried.

Fix: Move inside the lock:

with self.__class__._key_rotation_lock:
    self.__class__._serp_api_key_index = 0
raise Exception("All API keys exhausted")

[zvigrinberg]

Nit: Pre-index doesn't achieve algorithmic speedup — still O(unique_paths) linear scan

candidates = [doc for path, docs in self._source_path_index.items()
              if package in path for doc in docs]

This iterates all unique source paths doing a substring match — same algorithmic complexity as the old linear scan over all documents. The reduction from N (total docs) to U (unique paths) is a constant-factor improvement when files yield many functions, but it's not the O(1) lookup the pre-indexing pattern suggests.

Not a blocker — the constant-factor improvement plus the reordered search_token check before expensive is_function/_is_doc_excluded calls is a net positive. But if this becomes a bottleneck, an inverted index on path segments (mapping each package name to matching paths) would give true O(1) lookup.

[RedTanny]

A few questions before we merge the VERSION GUARD changes:

1. What is the basis for this change?
   Can you provide a specific CVE case where the current behavior caused an incorrect verdict? Specifically:
   Without a concrete example demonstrating the problem, it's hard to evaluate whether this change is necessary or correctly scoped.

Did you run the modified prompts against actual RPM checker scenarios to verify:

In my experience, LLMs frequently ignore or misinterpret prompt instructions — especially conditional logic like "if X then require Y". Adding a VERSION GUARD clause sounds reasonable in theory, but:

The model may still conclude PATCHED on a grep match alone
The model may become overly conservative and mark everything VULNERABLE
The interaction between this guard and other prompt rules is unpredictable

Suggestion: Before merging

1. find a real use case where this change is needed
2. verify that the prompt really works and the llm does not ignores it
3. run a regression test i have a dataset of 10 cases which i can send to check that changes does not create regression issues

[zvigrinberg]

Another cycle of review

[zvigrinberg]

High: is_comment_line misclassifies most JSDoc continuation lines as non-comment

A line like * Returns the cached value has rest='Returns...' and rest[0]='R' is alphanumeric, so the method returns False. This means the majority of JSDoc prose lines survive the comment filter. At line 207, unfiltered JSDoc prose is scanned for call patterns, producing false-positive caller-callee edges when prose mentions function names (e.g., * Delegates to parseJSON).

The intent was to distinguish *method() generator syntax from * JSDoc text, but the heuristic catches far too much.

Suggestion: Check whether the line is inside a /* ... */ block comment rather than inspecting the character after *. Or use a narrower heuristic — generator methods start with * immediately followed by an identifier without a space:

if stripped.startswith('*'):
    rest = stripped[1:]
    # Generator syntax: *methodName() — no space after *
    if rest and not rest[0].isspace():
        return False
    # JSDoc continuation: * some text — space after *
    return True

[zvigrinberg]

Refined suggestion: The space-based heuristic in my original suggestion doesn't work because both JSDoc and generator methods can have or omit the space after *. A better approach is to use a regex — a generator method always has * followed by an identifier and (, while JSDoc prose almost never matches that pattern:

@classmethod
def is_comment_line(cls, line: str) -> bool:
    stripped = line.strip()
    if stripped.startswith('//') or stripped.startswith('/*'):
        return True
    if stripped.startswith('*'):
        # Generator method: *methodName( or * methodName(
        if re.match(r'\*\s*[\w$]+\s*\(', stripped):
            return False
        return True
    return False

This correctly handles all combinations:

• * Returns the value → no identifier+( → True (comment ✓)
• *Returns the value → no ( → True (comment ✓)
• *method() → matches → False (generator ✓)
• * method() → matches → False (generator ✓)
• * @param x → no match → True (comment ✓)

The only edge case is JSDoc prose that looks like * Call foo(bar) to initialize — but that's rare, and the safe direction (treating it as code means the line isn't blanked, which just avoids one filtered line rather than creating a false-positive call match).

[zvigrinberg]

High: is_package_imported regex adds trailing .*, matching identifier anywhere in import path

Old regex: import ['"].*{identifier}['"] — identifier at end of path.
New regex: import ['"].*{esc_id}.*['"] — identifier anywhere in path.

So identifier='json' now matches import "github.com/json-iterator/go" because json appears mid-path. This creates false-positive import matches and allows unrelated packages to pass the call-chain filter.

The re.escape(identifier) fix is correct and needed — but the trailing .* changes the matching semantics.

Suggestion: Keep the anchor at the end, just add the escape:

esc_id = re.escape(identifier)
matching = re.search(rf"import [\'\"].*{esc_id}[\'\"]" , code_content)

Or if you need to match identifier as a path segment (not just at the end), use a more precise pattern:

matching = re.search(rf"import [\'\"].*[/]{esc_id}[\'\"]" , code_content)

[zvigrinberg]

@zvigrinberg This comment issue addresses an existing code, not new added one, thus need to embrace new needed escaping of identifier, and keep the greedy quantifier * at the end as is.

[zvigrinberg]

High: _count_call_args miscounts when </> are comparison operators balancing across a comma

For assertTrue(x < 10, y > 5):

1. < increments depth_a to 1
2. At the comma, depth_a != 0, so only commas_without_angles is incremented (not commas_with_angles)
3. > decrements depth_a back to 0
4. Since depth_a == 0, returns commas_with_angles + 1 = 1 instead of the correct 2

This causes the arg-count pre-filter to reject valid call sites wherever assertions, comparisons, or ternaries use </> across argument boundaries.

Suggestion: Use a two-pass approach — first try treating </> as angle brackets. If depth_a goes negative at any point (a > without a preceding <), restart treating all </> as operators and count only parenthesis/bracket depth:

# If depth_a ever goes negative, it means > was a comparison, not a bracket close.
# Fall back to ignoring angle brackets entirely.
if depth_a < 0:
    return _count_ignoring_angles(s, open_idx, close_idx)

[zvigrinberg]

Medium: C arg-count pre-filter checks only the first call site — rejects valid matches when a same-named local function appears first

_count_call_site_args uses pattern.search(function_body) (first match only). The docstring confirms: "Count arguments at the first call site."

If a caller has init() (0 args, local helper) textually before init(ctx, cfg) (2 args, the real callee), re.search finds the wrong one. The arity mismatch causes return False, rejecting a valid caller-callee edge.

Suggestion: Use re.finditer and check ALL call sites — accept if ANY matches the declared param count:

def _count_call_site_args(function_body: str, func_name: str) -> list[int]:
    """Count arguments at all call sites of func_name(...) in function_body."""
    pattern = re.compile(r'\b' + re.escape(func_name) + r'\s*\(')
    counts = []
    for m in pattern.finditer(function_body):
        count = _count_args_from_match(function_body, m)
        if count is not None:
            counts.append(count)
    return counts

Then in the caller: if declared not in call_arg_counts: return False

[zvigrinberg]

Medium: _go_subpackage_flow_control doesn't catch ValueError from get_function_name, unlike python_flow_control fixed in the same PR

This PR adds try/except ValueError: continue to python_flow_control (line ~188 of the diff) but the new _go_subpackage_flow_control calls get_function_name unguarded:

function_name = self.lang_parser.get_function_name(doc)

Go's get_function_name raises ValueError on malformed function headers (line 416 in golang_functions_parsers.py: raise ValueError(f"Invalid function header")). A single malformed document crashes the entire locate_functions call.

Suggestion: Add the same guard:

for doc in package_docs:
    if self.lang_parser:
        try:
            function_name = self.lang_parser.get_function_name(doc)
        except ValueError:
            continue
        if function_name:
            func_to_docs[function_name].append(doc)

[zvigrinberg]

Suggested change

	if not path.endswith(".go"):
	continue
	go_func_parser=GoLanguageFunctionsParser()
	extensions=go_func_parser.supported_files_extensions()
	if not any (path.endswith(ext) for ext in extensions):
	continue

[zvigrinberg]

Hey @tmihalac — great work on all these fixes and improvements. The scope here is impressive, but at 30K+ additions across 113 files, this PR is very difficult to review safely. Individual bugs can hide behind the sheer volume, and a single revert would roll back dozens of unrelated changes.

I'd recommend breaking this into self-contained PRs, roughly one per section in your description. I've created a Jira Epic to track this: APPENG-5610 with 10 child tasks:

Task	Scope
APPENG-5611	Java CCA: import pre-filter + cycle detection
APPENG-5612	C CCA: argument-count pre-filter
APPENG-5613	Go CCA: sub-package awareness + filtering
APPENG-5614	CCA all ecosystems: pre-indexing, dedup, regex escaping, DFS ordering
APPENG-5615	JS parser fixes
APPENG-5616	RPM checker: VERSION GUARD + format_for_prompt
APPENG-5617	Config scanner: extension allowlist + build patterns
APPENG-5618	Source code bug fixes (~40)
APPENG-5619	Build & infra: parallel JAR extraction, Maven threads, caching
APPENG-5620	Tests: fix tautological assertions, consolidate duplicates, add ~200 new tests

Each of these can be reviewed, tested, and merged independently. Smaller PRs are easier to review thoroughly, safer to revert if something breaks, and faster to get through CI.

I've also left inline review comments on several issues I found — We'll address all of that independently.

[zvigrinberg]

A Better implementation, taking into account process cpu count if it's available in python version ( python 3.13+) and also fallbacks to old cgroup v1 that might still be presented in some clusters.

Suggested change

	def _available_cpus() -> int:
	"""Return the number of CPUs available to this process, respecting cgroup limits."""
	try:
	with open("/sys/fs/cgroup/cpu.max") as f:
	quota, period = f.read().strip().split()
	if quota != "max":
	return max(1, int(quota) // int(period))
	except (FileNotFoundError, ValueError):
	pass
	return os.cpu_count() or 4
	def _available_cpus() -> int:
	"""Return the number of CPUs available to this process, respecting cgroup limits."""
	# Python 3.13+
	if hasattr(os, "process_cpu_count"):
	return os.process_cpu_count() or 1
	# cgroup v2
	try:
	with open("/sys/fs/cgroup/cpu.max") as f:
	quota, period = f.read().strip().split()
	if quota != "max":
	return max(1, int(quota) // int(period))
	except (FileNotFoundError, ValueError):
	pass
	# cgroup v1
	try:
	with open("/sys/fs/cgroup/cpu/cpu.cfs_quota_us") as f:
	quota = int(f.read().strip())
	if quota != -1:
	with open("/sys/fs/cgroup/cpu/cpu.cfs_period_us") as f:
	period = int(f.read().strip())
	return max(1, quota // period)
	except (FileNotFoundError, ValueError):
	pass
	return os.cpu_count() or 4

[vbelouso]

@zvigrinberg I think we should keep the implementation cgroup-first and only use os.process_cpu_count() as a fallback.

In my test on a H200 OpenShift node, a Python 3.13 pod with limits.cpu: 4 still reported 224 from both os.cpu_count() and os.process_cpu_count(), while /sys/fs/cgroup/cpu.max correctly showed 400000 100000.

So I think the safer order is:

1. cgroup v2 cpu.max
2. cgroup v1 quota/period fallback
3. CPU affinity
4. os.process_cpu_count()
5. os.cpu_count()

This keeps the Python 3.13 API as a useful fallback, but avoids ignoring the actual container CPU limit when cgroup quota is available.

WDYT?

[vbelouso]

cpu.py
something like this

[zvigrinberg]

@vbelouso Looks even better and safer to me, thanks!.

[zvigrinberg]

Suggested change

	def _available_cpus() -> int:
	"""Return the number of CPUs available to this process, respecting cgroup limits."""
	try:
	with open("/sys/fs/cgroup/cpu.max") as f:
	quota, period = f.read().strip().split()
	if quota != "max":
	return max(1, int(quota) // int(period))
	except (FileNotFoundError, ValueError):
	pass
	return os.cpu_count() or 4
	def _available_cpus() -> int:
	"""Return the number of CPUs available to this process.
	Respects container CPU limits from cgroup v2/v1 before falling back to
	process affinity and Python CPU APIs.
	"""
	# cgroup v2
	try:
	with open("/sys/fs/cgroup/cpu.max", encoding="utf-8") as f:
	quota_s, period_s = f.read().strip().split()
	if quota_s != "max":
	quota = int(quota_s)
	period = int(period_s)
	if quota > 0 and period > 0:
	return max(1, math.ceil(quota / period))
	except (OSError, ValueError):
	pass
	# cgroup v1
	for base in (
	"/sys/fs/cgroup/cpu",
	"/sys/fs/cgroup/cpu,cpuacct",
	):
	try:
	with open(f"{base}/cpu.cfs_quota_us", encoding="utf-8") as f:
	quota = int(f.read().strip())
	with open(f"{base}/cpu.cfs_period_us", encoding="utf-8") as f:
	period = int(f.read().strip())
	# cgroup v1 uses quota == -1 to mean "no CPU quota".
	if quota > 0 and period > 0:
	return max(1, math.ceil(quota / period))
	except (OSError, ValueError):
	pass
	# CPU affinity fallback.
	try:
	return max(1, len(os.sched_getaffinity(0)))
	except (AttributeError, OSError):
	pass
	# Python 3.13+ fallback.
	# Keep this after cgroup checks: in OpenShift/Kubernetes it may expose
	# the full node-visible CPU set rather than the pod CPU limit.
	if hasattr(os, "process_cpu_count"):
	count = os.process_cpu_count()
	if count:
	return max(1, count)
	return os.cpu_count() or 4