Skip to content

[CI] Security workflow failing every weekly run since April 2026 — protobuf RUSTSEC-2024-0437 + Trivy CRITICAL/HIGH findings unresolved for 3+ months #15

Description

@cerebral-work

Summary

The Security workflow (scheduled weekly) has been failing on every single run since at least 2026-04-13 — that is 13+ consecutive weekly failures spanning 3+ months, all on the same code sha e09ec977. The workflow has two failing jobs: Cargo Audit and Trivy Container Scan.

Affected Runs (sample)

Run ID Date
27543738125 2026-06-15
27130804333 2026-06-08
26751042021 2026-06-01
26394133246 2026-05-25
24331799588 2026-04-13

Cargo Audit Failure

Advisory: RUSTSEC-2024-0437

error: 1 vulnerability found!
Crate:    protobuf
Version:  2.28.0
Title:    Crash due to uncontrolled recursion in protobuf crate
Date:     2024-12-12
ID:       RUSTSEC-2024-0437
URL:      https://rustsec.org/advisories/RUSTSEC-2024-0437
Solution: Upgrade to >=3.7.2

The protobuf crate at version 2.28.0 contains a vulnerability where uncontrolled recursion can cause a crash. The fix is to upgrade to protobuf >= 3.7.2. This advisory was published 2024-12-12 and has been in the dependency tree unresolved for over 7 months.

Trivy Container Scan Failure

The Trivy scan is configured with severity: CRITICAL,HIGH and is failing on CRITICAL/HIGH findings in the container image. Specific CVEs were not captured in this log excerpt but the scan exits non-zero.

Root Cause

The protobuf dependency (likely a transitive dependency) has not been updated from 2.28.0 to >= 3.7.2. This is a major version bump (2.x3.x) which may require API changes at the call sites. The Trivy findings represent OS or runtime vulnerabilities in the container base image that have accumulated over time without a base image refresh.

Impact

Security signals have been silently failing for 3+ months. The Security workflow is effectively dead as an alert mechanism — operators have likely normalized seeing it red.

Suggested Fix Direction

For cargo audit:

  • Upgrade protobuf to >= 3.7.2. If protobuf is a transitive dependency (via e.g. grpcio or similar), upgrading the parent crate or pinning to the fixed version in Cargo.toml may be needed.
  • Alternatively, if protobuf 2.x cannot be upgraded immediately, add a [advisories] ignore = ["RUSTSEC-2024-0437"] entry to .cargo/deny.toml with a documented plan and timeline — but do not ignore indefinitely.

For Trivy:

  • Rebuild the Docker image with updated base image layers (apt-get upgrade in Dockerfile or bump the base image tag).
  • Review and triage the CRITICAL/HIGH CVE list from a recent Trivy report.

Filed by CI/Ops Watchdog — 2026-07-16

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions