Skip to content

[ci]: Add pure database config adapter#722

Open
hdwhdw wants to merge 6 commits into
sonic-net:masterfrom
hdwhdw:ci/pure-sonic-db-config
Open

[ci]: Add pure database config adapter#722
hdwhdw wants to merge 6 commits into
sonic-net:masterfrom
hdwhdw:ci/pure-sonic-db-config

Conversation

@hdwhdw

@hdwhdw hdwhdw commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Why I did it

This is the first adapter slice after
sonic-net/sonic-gnmi#719.

Canonical packages need a repeatable way to isolate dependencies that cannot
compile outside the SONiC build environment. Database configuration is the
first slice because its namespace API contains reusable derived logic while its
implementation depends on swsscommon.

How I did it

  • extracted namespace-based database configuration into internal/dbconfig
  • kept derived behavior untagged behind a repository-owned provider contract
  • added a !pure provider that wraps the existing swsscommon calls
  • added a pure provider that parses database_config.json
  • preserved the legacy sonic_db_config namespace signatures through delegates
  • left DBKey APIs unchanged for a later migration
  • added CGO-free build proof, dependency leakage checks, and build-tag placement
    checks to the pure workflow
  • removed the sibling repository checkout from hosted pure tests
  • shared a provider contract between pure CI and SONiC integration CI

How to verify it

  • make -f pure.mk ci
  • make -f pure.mk junit-xml
  • make -f pure.mk PACKAGES=internal/dbconfig check-deps check-tags build-test test
  • run make check_gotest_junit in the SONiC integration environment
  • review the successful package, image, and KVM checks in
    sonic-net/sonic-buildimage#28409

Which release branch to backport (provide reason below if selected)

  • 202305
  • 202311
  • 202405
  • 202411
  • 202505
  • 202511
  • 202605

Tracking issue/work item for backport/cherry-pick request (GitHub issue or Microsoft ADO): N/A
Failure type: N/A

Tested branch

  • master
  • 202305
  • 202311
  • 202405
  • 202411
  • 202505
  • 202511
  • 202605
  • N/A

Test result

  • master: pure CI and hosted-style JUnit passed after rebasing onto master.
  • sonic-buildimage: all checks passed in
    #28409.
    git range-diff reports both step-1 patches unchanged by the rebase.

Description for the changelog

Add the first pure/real adapter seam for standalone database configuration
tests.

Link to config_db schema for YANG module changes

Not applicable.

A picture of a cute animal (not mandatory but encouraged)

hdwhdw and others added 2 commits July 22, 2026 09:29
Extract namespace database configuration behind an internal provider seam. The normal build uses swsscommon, while the pure build parses database_config.json and rejects unsupported global configurations explicitly.

Run canonical packages with the pure build tag, prove CGO-free builds, reject SONiC-only dependency leaks, and remove the sibling checkout from hosted pure tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
Validate test dependencies and build-tag placement, run a CGO-free build before hosted race tests, and preserve default namespace lookup behavior.

Share the database configuration contract with the real swsscommon provider and publish its integration result separately.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
Copilot AI review requested due to automatic review settings July 22, 2026 14:33
@mssonicbld

Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new internal/dbconfig adapter layer to decouple SONiC/CGO-dependent database configuration (via swsscommon) from reusable namespace/derived logic, enabling “pure” (CGO-free) builds and tests that run outside the SONiC build environment.

Changes:

  • Added internal/dbconfig with a provider contract and two implementations: !pure (swsscommon-backed) and pure (file-backed database_config.json parsing).
  • Updated sonic_db_config to preserve legacy APIs by delegating namespace-based behavior to internal/dbconfig.
  • Strengthened pure CI validation (dependency leakage + build-tag placement checks) and adjusted Azure pipeline checkout/pathing.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
sonic_db_config/db_config.go Delegates namespace-based dbconfig APIs to internal/dbconfig while preserving legacy surface.
pure.mk Adds pure-tag flags, dependency leakage checks, and build-tag placement checks to pure CI targets.
Makefile Adds integration test/coverage runs for internal/dbconfig and separates dbconfig contract tests for JUnit.
internal/dbconfig/testdata/database_config.json Adds standalone JSON config fixture used by the pure provider tests.
internal/dbconfig/provider_swss.go Implements !pure provider backed by swsscommon.
internal/dbconfig/provider_swss_test.go Adds provider contract test coverage for the swsscommon provider.
internal/dbconfig/provider_pure.go Implements pure provider that reads/parses database_config.json.
internal/dbconfig/provider_pure_test.go Adds unit tests validating pure-provider behavior and error paths.
internal/dbconfig/provider_contract_test.go Adds shared provider contract test helper used by both providers.
internal/dbconfig/dbconfig.go Introduces provider interface + common derived logic with initialization/reset control.
azure-pipelines.yml Removes extra checkout for pure tests, updates paths, and publishes integration dbconfig JUnit output.

Comment thread internal/dbconfig/provider_swss.go Outdated
Comment thread internal/dbconfig/provider_pure.go
Comment thread Makefile Outdated
Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
Copilot AI review requested due to automatic review settings July 22, 2026 17:02
@mssonicbld

Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

Comments suppressed due to low confidence (1)

internal/dbconfig/provider_swss.go:19

  • swssProvider.initialize() treats any os.Stat() error as equivalent to "file missing" (via os.IsExist) and falls back to SonicDBConfigInitialize(). If the global config file is unreadable (e.g., permission/IO error), this silently initializes the wrong config instead of surfacing the error. Use os.IsNotExist(err) to distinguish missing vs other errors and return the unexpected error.
func (swssProvider) initialize() error {
	if _, err := os.Stat(GlobalConfigFile); err == nil || os.IsExist(err) {
		if !swsscommon.SonicDBConfigIsGlobalInit() {
			swsscommon.SonicDBConfigInitializeGlobalConfig()
		}

Comment thread pure.mk Outdated
Comment thread internal/dbconfig/provider_pure_test.go
Comment thread internal/dbconfig/provider_pure_test.go
Comment thread internal/dbconfig/provider_pure_test.go
Comment thread internal/dbconfig/provider_pure_test.go
Comment thread internal/dbconfig/provider_pure_test.go
Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
Copilot AI review requested due to automatic review settings July 22, 2026 17:28
@mssonicbld

Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (7)

internal/dbconfig/provider_swss.go:16

  • os.IsExist(err) is not meaningful for errors returned by os.Stat and can cause non-"not exist" errors (e.g., permission/IO) to be treated as “no global config”, silently falling back to SonicDBConfigInitialize(). Handle os.IsNotExist(err) explicitly and return unexpected errors.
	if _, err := os.Stat(GlobalConfigFile); err == nil || os.IsExist(err) {

internal/dbconfig/provider_pure.go:62

  • When INCLUDES is present in database_config.json, the error message says “global database configuration”, but this case is specifically about unsupported includes. This makes failures harder to understand.
	if len(p.config.Includes) > 0 {
		return fmt.Errorf("global database configuration is not supported by the pure provider")
	}

internal/dbconfig/provider_pure_test.go:41

  • %q is not a valid/meaningful format for a []string and will print %!q([]string=...). Use %v (or %#v) for slices in test failure messages.
		t.Errorf("GetDbAllNamespaces() = %q, want default namespace", namespaces)

internal/dbconfig/provider_pure_test.go:49

  • %q is not a valid/meaningful format for a []string and will print %!q([]string=...). Use %v (or %#v) for slices in test failure messages.
		t.Errorf("GetDbNonDefaultNamespaces() = %q, want none", nonDefault)

internal/dbconfig/provider_pure_test.go:66

  • %q is not a valid/meaningful format for a []string and will print %!q([]string=...). Use %v (or %#v) for slices in test failure messages.
		t.Errorf("GetDbList() = %q, want %q", databases, wantDatabases)

internal/dbconfig/provider_pure_test.go:157

  • %q is not a valid/meaningful format for a []string and will print %!q([]string=...). Use %v (or %#v) for slices in test failure messages.
		t.Errorf("GetDbAllNamespaces() = %q, want namespace %q", namespaces, contract.namespace)

internal/dbconfig/provider_pure_test.go:165

  • %q is not a valid/meaningful format for a []string and will print %!q([]string=...). Use %v (or %#v) for slices in test failure messages.
		t.Errorf("GetDbList() = %q, want database %q", databases, contract.database)

Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
Copilot AI review requested due to automatic review settings July 22, 2026 20:31
@mssonicbld

Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Comment thread internal/adaptors/dbconfig/provider_swss.go
Comment thread internal/adaptors/dbconfig/provider_pure.go
Signed-off-by: Dawei Huang <daweihuang@microsoft.com>
Copilot AI review requested due to automatic review settings July 22, 2026 23:08
@mssonicbld

Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants