Skip to content

[BUG] Recursive multi-skill scans silently ignore --baseline and --show-suppressed #201

Description

@wernerkasselman-au

What I'm seeing

I was checking how the suppression work (#88) and the multi-skill work (#136) interact, and they do not: in scan --recursive, the --baseline and --show-suppressed options are accepted on the command line and then silently dropped, so a baselined finding is not actually suppressed. It still counts toward each sub-skill's risk score, and it can still flip the exit code to 1. The single-skill path honours both options, so only the recursive path regresses, which makes it easy to miss.

Walking the code (commit 7bc9c0f, src/skillspector/cli.py)

  • The recursive dispatch hands over neither option (cli.py:284):
    _scan_multi_skill(detection, format, output, no_llm, yara_rules_dir, verbose)
  • _scan_multi_skill() has no baseline / show_suppressed parameters at all (cli.py:359-366), and its per-skill state build leaves them out (cli.py:379):
    state = _scan_state(str(skill.path), format, no_llm, yara_rules_dir=yara_dir)
  • The single-skill path, for contrast, passes both through (cli.py:302-309).
  • And _scan_state() only loads the baseline and sets show_suppressed when baseline is actually provided (cli.py:142-145), so omitting the arguments means no suppression is applied, not "default suppression".

Reproduction

# A collection dir with at least two sub-skills (each with its own SKILL.md),
# plus a baseline generated from a prior scan of that collection.
skillspector scan ./skill-collection/ --recursive \
    --baseline .skillspector-baseline.yaml --show-suppressed
  • What I get: every finding in the baseline is still reported and still counted in each sub-skill's score, --show-suppressed does nothing, and a fully baselined collection can still exit 1 (max_score > 50).
  • What I expected: baselined findings suppressed per sub-skill the same way single-skill mode does it, kept out of the score, and listed only when --show-suppressed is set.

Why it matters

It is silent (no warning that the flags were ignored), so the natural assumption is that suppression is being applied. That breaks the incremental, "only the new findings matter" workflow #88 was built for, and it breaks it precisely on --recursive, which is the mode you would reach for to scan a whole skill registry in one pass.

What I think the fix is

Thread the two options through: add baseline: Path | None and show_suppressed: bool to _scan_multi_skill(), pass them from the call site at cli.py:284, and forward them into the _scan_state(...) call at cli.py:379, mirroring cli.py:302-309. Happy to put that up with a test that runs a baselined collection through --recursive and asserts the suppressed findings stay out of the per-skill scores.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions