[ingestion/data-quality issue] Inconsistency on Namespace in Sources Table

[Jayashree138]

Describe the bug

Inconsistency on Namespace in Sources Table

Overview

GUAC's SourceInputSpec model identifies source repositories using three mandatory fields: Type, Namespace, and Name. Different certifiers and parsers construct these fields using different logic, which means the same Git repository can be represented as multiple distinct source nodes in the graph.

For example, https://github.com/facebook/react may produce:

Certifier/Parser	Type	Namespace	Name
deps.dev (via VcsToSrc)	git	github.com/facebook	react
ClearlyDefined (via SourceToSourceInput)	git	facebook	react
Scorecard (direct construction)	git	github.com/facebook	react

---

How Each Certifier/Parser Constructs Source Nodes

deps.dev (VcsToSrc path)

File: internal/client/depsdevclient/deps_dev_client.go:484-488

The deps.dev client receives a source repository URL from the deps.dev API (via the SOURCE_REPO link label) and passes it to helpers.VcsToSrc().

File: pkg/assembler/helpers/vcs.go:37-90

VcsToSrc parses the URL and sets:

• Type: derived from URL scheme (e.g., git for github.com, gitlab.com, etc.)
• Namespace: u.Host + u.Path[:lastSlashIndex] (e.g., github.com/facebook)
• Name: last path segment (e.g., react)

m.Namespace = u.Host                  // "github.com"
idx := strings.LastIndex(u.Path, "/")
if idx > 0 {
    m.Name = u.Path[idx+1:]           // "react"
    m.Namespace += u.Path[:idx]       // "github.com/facebook"
}

Result for https://github.com/facebook/react:

Type:      "git"
Namespace: "github.com/facebook"
Name:      "react"

---

ClearlyDefined (SourceToSourceInput path)

File: pkg/certifier/clearlydefined/clearlydefined.go:214-228

When ClearlyDefined returns a sourceLocation in a definition response, the certifier builds a coordinate directly from the response fields:

srcInput := helpers.SourceToSourceInput(
    sourceLocation.Type,       // from ClearlyDefined API
    sourceLocation.Namespace,  // from ClearlyDefined API
    sourceLocation.Name,
    &sourceLocation.Revision,
)

File: pkg/assembler/helpers/source.go:117-132

SourceToSourceInput is a simple passthrough — it does not normalize the namespace:

func SourceToSourceInput(srcType, namespace, name string, revision *string) *generated.SourceInputSpec {
    srcInput := &generated.SourceInputSpec{
        Type:      srcType,
        Namespace: namespace,
        Name:      name,
    }
    // ...
}

File: pkg/certifier/attestation/license/attestation_license.go:112-119

The SourceLocation struct maps directly from the ClearlyDefined API response:

type SourceLocation struct {
    Type      string `json:"type"`
    Provider  string `json:"provider"`
    Namespace string `json:"namespace"` // e.g., "facebook" (no host prefix)
    Name      string `json:"name"`
    Revision  string `json:"revision"`
}

ClearlyDefined coordinates use the format git/github/<namespace>/<name>/<revision>. The namespace field contains only the GitHub org/user, not the full host path.

Result for the same repository:

Type:      "git"
Namespace: "facebook"
Name:      "react"

---

Scorecard Parser (direct construction)

File: pkg/ingestor/parser/scorecard/parser_scorecard.go:101-147

The getPredicates function splits s.Repo.Name on the last / to derive namespace and name directly:

var ns, name string
idx := strings.LastIndex(s.Repo.Name, "/")
if idx < 0 {
    name = s.Repo.Name
} else {
    ns = s.Repo.Name[:idx]
    name = s.Repo.Name[idx+1:]
}

Repo.Name is typically a full URL like github.com/facebook/react, so this produces:

Type:      "git"
Namespace: "github.com/facebook"
Name:      "react"

This is consistent with VcsToSrc but arrived at through different code.

---

Inconsistency Scenarios

Scenario 1: Same repo, different namespace formats

A package pkg:npm/react@18.0.0 is processed by both deps.dev and ClearlyDefined:

• deps.dev retrieves https://github.com/facebook/react, calls VcsToSrc, producing: {Type: "git", Namespace: "github.com/facebook", Name: "react"}
• ClearlyDefined returns sourceLocation: {type: "git", provider: "github", namespace: "facebook", name: "react"}, calls SourceToSourceInput, producing: {Type: "git", Namespace: "facebook", Name: "react"}

These are two different source nodes in the graph. The HasSourceAt edge from ClearlyDefined points to a different node than the one from deps.dev.

Scenario 2: Scorecard data fragmentation

deps.dev also retrieves Scorecard data and attaches it to the source node it created (via VcsToSrc). If the standalone Scorecard parser processes the same repo, it constructs the source node directly from Repo.Name, which typically includes the host. These two should match, but the code paths are entirely independent — any format change in either would silently break consistency.

Note: Similar behaviour will be seen in the Scorecard api fetch and ingestion logic.

---

Suggestions

Use sourceLocation.URL with VcsToSrc for consistency

if sourceLocation.URL != "" {
    srcInput, err := helpers.VcsToSrc(sourceLocation.URL)
    // {git, github.com/facebook, react} — consistent with deps.dev
}

This would produce the same namespace format across certifiers.

Test: curl command

curl -s -X POST "https://api.clearlydefined.io/definitions" \
  -H "Content-Type: application/json" \
  -d '["npm/npmjs/-/react/18.2.0"]' | python3 -m json.tool | grep -A 10 '"sourceLocation"'

The sourceLocation for npm/react@18.2.0:

"sourceLocation": {
  "type": "git",
  "provider": "github",
  "namespace": "facebook",
  "name": "react",
  "revision": "9e3b772b8cabbd8cadc7522ebe3dde3279e79d9e",
  "url": "https://github.com/facebook/react/tree/9e3b772b8cabbd8cadc7522ebe3dde3279e79d9e"
}

Known issues with this approach

1. URL contains a /tree/<revision> suffix that must be truncated before passing to VcsToSrc.
2. URL may not always be present in the API response — a fallback is needed.

To Reproduce
Steps to reproduce the behavior:
Ingest a package covered by both deps.dev and ClearlyDefined (e.g., pkg:npm/react@18.2.0).
Query source nodes — two distinct nodes exist for the same repository due to differing namespace formats.

Expected behavior
The same repository always maps to a single canonical source node, regardless of which certifier or parser processes it.

Inconsitancy_on_namespace_SourceTable.pdf