Skip to content

feat: SCALIBR e2e & Fix all the snapshot problems for ever!*#2934

Draft
another-rex wants to merge 39 commits into
google:mainfrom
another-rex:new-scalibr-clients
Draft

feat: SCALIBR e2e & Fix all the snapshot problems for ever!*#2934
another-rex wants to merge 39 commits into
google:mainfrom
another-rex:new-scalibr-clients

Conversation

@another-rex

@another-rex another-rex commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Note

I'll try to split this up

With the scalibr clients PR finally merged in, we now have access to everything we need to fully migrate to use scalibr end2end.

This PR does the following:

  • Remove vulnerability and license matching from osv-scanner, and enable them as enrichers when running osv-scalibr
  • All scanning and matching logic is in scalibr now, making it much easier to extend/onboard new functionality.
  • Setup filtering as a plugin (temporary until we get a better solution, see design doc).
  • Setup clients to allow cassette of enricher requests
  • Create a grpc vcr library to allow cassetting of deps.dev requests
  • Update how offline vulnerabilities are being tested so it's not downloading the full zips every time, making it much much faster to run.
  • Full tests reduced from 60+ seconds or so on my machine to 7 seconds.
  • Basically no more snapshot changes without code changes. *

*Not quite true, as we still download live full vulns, which has a severity field that is likely to change.

@another-rex another-rex changed the title feat: Fix all the snapshot problems for ever!* feat: SCALIBR e2e & Fix all the snapshot problems for ever!* Jul 21, 2026
}

// Delete existing directory to start clean
if err := os.RemoveAll(localDBPath); err != nil {
if err := os.RemoveAll(localDBPath); err != nil {
return err
}
if err := os.MkdirAll(localDBPath, 0750); err != nil {
Comment on lines +872 to +901
for _, file := range zipReader.File {
if !strings.HasSuffix(file.Name, ".json") {
continue
}

vulnID := strings.TrimSuffix(file.Name, ".json")
if !shouldKeepVuln(vulnID) {
continue
}

f, err := file.Open()
if err != nil {
return err
}
content, err := io.ReadAll(f)
f.Close()
if err != nil {
return err
}

var prettyJSON bytes.Buffer
if err := json.Indent(&prettyJSON, content, "", " "); err != nil {
return err
}

targetFilePath := filepath.Join(localDBPath, vulnID+".json")
if err := os.WriteFile(targetFilePath, prettyJSON.Bytes(), 0644); err != nil { //nolint:gosec
return err
}
}
}

targetFilePath := filepath.Join(localDBPath, vulnID+".json")
if err := os.WriteFile(targetFilePath, prettyJSON.Bytes(), 0644); err != nil { //nolint:gosec
repoRoot := findRepoRoot()
localDBPath := filepath.Join(repoRoot, offlineDBRelativePath, ecosystem)

files, err := os.ReadDir(localDBPath)
}

filePath := filepath.Join(localDBPath, file.Name())
content, err := os.ReadFile(filePath)
@another-rex
another-rex marked this pull request as draft July 21, 2026 06:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants