[sonic-grpc]: Add gNOI Python gRPC client framework wheel by hdwhdw · Pull Request #28341 · sonic-net/sonic-buildimage

hdwhdw · 2026-07-09T22:01:25Z

Why I did it

The new PTF-free native gNOI test suite in sonic-mgmt (and other in-box
components) need an in-process Python gNOI gRPC client. This adds it as a
reusable, minimal-dependency standalone wheel so lightweight consumers — notably
the docker-sonic-mgmt test image — can install it without pulling in a heavier
package or its dependency tree.

The gNOI client code originates from the framework in #27760; this PR delivers
it as a standalone sonic-grpc wheel that #27760 and the sonic-mgmt gNOI test
suite can both consume.

How I did it

Added a new src/sonic-grpc/ package (sonic_grpc):

sonic_grpc.gnoi.GnoiClient — context-managed gRPC channel wrapper exposing
gNOI service stubs as properties (client.system.*, client.file.*) over
insecure TCP, unix:// UDS, or mTLS. Service-agnostic.
Vendored flat gNOI proto stubs (System, File, types, common), regenerated
against protobuf 4.x/5.x. No stub generation or .proto files at install time.
FakeGnoiServer test doubles for offline unit tests.
Trivial setup.py: depends only on grpcio + protobuf>=4.21.

Wired into docker-sonic-mgmt:

rules/sonic-grpc.mk (SONIC_GRPC_PY3, added to SONIC_PYTHON_WHEELS).
rules/docker-sonic-mgmt.mk: += $(SONIC_GRPC_PY3) to _PYTHON_WHEELS.
Dockerfile.j2: a uv-based install of the wheel (the image /opt/venv has no
pip, so the generic pip3 install_python_wheels macro is not used), plus
grpcio added to the package list as an explicit dependency.

How to verify it

Unit: cd src/sonic-grpc && python3 -m pytest (client + FakeGnoiServer).

Import smoke test in a container with the wheel installed:

from sonic_grpc.gnoi import GnoiClient, system_pb2, file_pb2
from sonic_grpc.gnoi.testing import FakeGnoiServer
with FakeGnoiServer() as s, GnoiClient(s.target) as c:
    c.system.Reboot(system_pb2.RebootRequest(method=system_pb2.RebootMethod.COLD))

End-to-end: the sonic-mgmt tests/gnoi/ suite ([gnoi] PTF-free native gNOI test suite (sonic_grpc.GnoiClient) sonic-mgmt#26039)
drives System.Time and File.Stat over this client against a DUT.
Image wiring is validated by the build pipeline (a clean docker-sonic-mgmt
build installs the wheel).

Which release branch to backport (provide reason below if selected)

None.

Description for the changelog

Add sonic-grpc wheel: native Python gNOI gRPC client framework for SONiC.

Dependencies

Consumed by [gnoi] PTF-free native gNOI test suite (sonic_grpc.GnoiClient) sonic-mgmt#26039 (native gNOI test suite).
Supersedes the in-place gNOI framework in [sonic-py-common] Add gRPC framework with gNOI client #27760, which can be rerouted to
consume this wheel.

Add a new standalone `sonic-grpc` Python distribution providing a native, in-process gNOI gRPC client for SONiC, and wire it into docker-sonic-mgmt. sonic_grpc.gnoi provides: - GnoiClient: a context-managed gRPC channel wrapper exposing gNOI service stubs as properties (client.system.*, client.file.*), over insecure TCP, unix:// UDS, or mTLS. Service-agnostic; new services plug in as properties. - Vendored flat gNOI proto stubs (System, File, types, common) regenerated against protobuf 4.x/5.x. No stub generation or gnoi.proto files needed at install time. - FakeGnoiServer test doubles for offline unit testing of gNOI-driven code. Packaged as its own minimal-dependency wheel (grpcio + protobuf only) so lightweight consumers - notably the docker-sonic-mgmt test image - can install it without pulling in a heavier package or its dependency tree. It installs anywhere, including: pip install "git+https://github.com/sonic-net/sonic-buildimage#subdirectory=src/sonic-grpc" Wired into docker-sonic-mgmt via rules/sonic-grpc.mk (SONIC_GRPC_PY3) and a uv-based install block in the Dockerfile (the image's /opt/venv has no pip, so the generic pip3 install_python_wheels macro is not used). grpcio is added to the image's package list as an explicit dependency. The gNOI client code originates from the framework in sonic-net#27760; this PR delivers it as a reusable standalone wheel that sonic-net#27760 and the sonic-mgmt gNOI test suite can both consume. Signed-off-by: Dawei Huang <daweihuang@microsoft.com>

mssonicbld · 2026-07-09T22:01:33Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-07-09T22:01:56Z

Azure Pipelines successfully started running 1 pipeline(s).

The wheel-install block uses the copy_files macro, which must be imported at the top of the Dockerfile.j2 (as every other wheel-consuming Dockerfile does). Without the import the j2 template fails to render. Add the import. Signed-off-by: Dawei Huang <daweihuang@microsoft.com>

mssonicbld · 2026-07-09T22:09:55Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-07-09T22:10:08Z

Azure Pipelines successfully started running 1 pipeline(s).

Every buildable component with GIT_CONTENT_SHA caching has a matching .dep file (included via Makefile.cache) that declares its DEP_FILES and cache mode. Add rules/sonic-grpc.dep for the new SONIC_GRPC_PY3 wheel, mirroring rules/sonic-py-common.dep, so the wheel participates in content-SHA build caching and rebuilds when its sources change. Signed-off-by: Dawei Huang <daweihuang@microsoft.com>

mssonicbld · 2026-07-09T22:15:25Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-07-09T22:15:38Z

Azure Pipelines successfully started running 1 pipeline(s).

Copilot

Pull request overview

This PR adds a new standalone Python wheel (sonic-grpc) under src/sonic-grpc/ that provides a minimal gNOI gRPC client framework (client wrapper + vendored gNOI proto stubs + test fakes) and wires that wheel into the docker-sonic-mgmt image build so consumers (notably sonic-mgmt’s native gNOI tests) can install it without pulling in a larger dependency tree.

Changes:

Introduces sonic_grpc.gnoi.GnoiClient plus a FakeGnoiServer test double, and vendors generated gNOI stubs (System/File/types/common) in the new sonic-grpc wheel.
Adds build rules for producing the new wheel and includes it in SONIC_PYTHON_WHEELS.
Updates docker-sonic-mgmt to include and install locally-built wheels via uv, and adds grpcio to the image’s Python dependencies.

Reviewed changes

Copilot reviewed 13 out of 17 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/sonic-grpc/sonic_grpc/gnoi/types_pb2.py	Vendored generated protobuf stubs for gNOI types.
src/sonic-grpc/sonic_grpc/gnoi/types_pb2_grpc.py	Placeholder generated gRPC module for types (no services).
src/sonic-grpc/sonic_grpc/gnoi/common_pb2.py	Vendored generated protobuf stubs for gNOI common messages.
src/sonic-grpc/sonic_grpc/gnoi/common_pb2_grpc.py	Placeholder generated gRPC module for common (no services).
src/sonic-grpc/sonic_grpc/gnoi/system_pb2.py	Vendored generated protobuf stubs for gNOI System service messages.
src/sonic-grpc/sonic_grpc/gnoi/system_pb2_grpc.py	Vendored generated gRPC stubs for gNOI System service.
src/sonic-grpc/sonic_grpc/gnoi/file_pb2.py	Vendored generated protobuf stubs for gNOI File service messages.
src/sonic-grpc/sonic_grpc/gnoi/file_pb2_grpc.py	Vendored generated gRPC stubs for gNOI File service.
src/sonic-grpc/sonic_grpc/gnoi/client.py	Implements `GnoiClient` channel wrapper with per-service stub properties.
src/sonic-grpc/sonic_grpc/gnoi/testing.py	Implements `FakeGnoiServer` and `FakeSystemServicer` test doubles.
src/sonic-grpc/sonic_grpc/gnoi/init.py	Package entry point exporting `GnoiClient`.
src/sonic-grpc/sonic_grpc/init.py	Top-level `sonic_grpc` package docstring and future roadmap.
src/sonic-grpc/setup.py	Wheel metadata and runtime dependency declarations.
rules/sonic-grpc.mk	Adds wheel definition and registers it in `SONIC_PYTHON_WHEELS`.
rules/sonic-grpc.dep	Adds dependency tracking for caching/rebuild correctness.
rules/docker-sonic-mgmt.mk	Adds the `sonic-grpc` wheel to the docker-sonic-mgmt wheel set.
dockers/docker-sonic-mgmt/Dockerfile.j2	Installs locally-built wheels via `uv` and adds `grpcio` to dependencies.

Files not reviewed (4)

src/sonic-grpc/sonic_grpc/gnoi/common_pb2.py: Generated file
src/sonic-grpc/sonic_grpc/gnoi/file_pb2.py: Generated file
src/sonic-grpc/sonic_grpc/gnoi/system_pb2.py: Generated file
src/sonic-grpc/sonic_grpc/gnoi/types_pb2.py: Generated file

+# sonic-grpc Python wheel (gNOI / gRPC client framework for SONiC)
+
+SONIC_GRPC_PY3 = sonic_grpc-1.0-py3-none-any.whl
+$(SONIC_GRPC_PY3)_SRC_PATH = $(SRC_PATH)/sonic-grpc
+$(SONIC_GRPC_PY3)_PYTHON_VERSION = 3
+SONIC_PYTHON_WHEELS += $(SONIC_GRPC_PY3)


+from google.protobuf import runtime_version as _runtime_version
+from google.protobuf import symbol_database as _symbol_database
+from google.protobuf.internal import builder as _builder
+_runtime_version.ValidateProtobufRuntimeVersion(
+    _runtime_version.Domain.PUBLIC,
+    5,
+    29,
+    0,
+    '',
+    'github.com/openconfig/gnoi/file/file.proto'
+)


+GRPC_GENERATED_VERSION = '1.70.0'
+GRPC_VERSION = grpc.__version__
+_version_not_supported = False
+
+try:
+    from grpc._utilities import first_version_is_lower
+    _version_not_supported = first_version_is_lower(GRPC_VERSION, GRPC_GENERATED_VERSION)
+except ImportError:
+    _version_not_supported = True
+
+if _version_not_supported:
+    raise RuntimeError(
+        f'The grpc package installed is at version {GRPC_VERSION},'
+        + f' but the generated code in github.com/openconfig/gnoi/file/file_pb2_grpc.py depends on'
+        + f' grpcio>={GRPC_GENERATED_VERSION}.'
+        + f' Please upgrade your grpc module to grpcio>={GRPC_GENERATED_VERSION}'
+        + f' or downgrade your generated code using grpcio-tools<={GRPC_VERSION}.'
+    )


+    install_requires=[
+        'grpcio',
+        'protobuf>=4.21',
+    ],


The SONiC bookworm/trixie wheel build rule runs `pytest` on each wheel's source unless <pkg>_TEST=n. sonic-grpc had no tests, so pytest collected 0 items and the wheel target failed. Add the gNOI client and FakeGnoiServer unit tests (31 tests, lifted from the framework in sonic-net#27760) under src/sonic-grpc/tests/, and declare a `testing` extra so the build's `pip install ".[testing]"` step is satisfied. Verified: `target/python-wheels/bookworm/sonic_grpc-1.0-py3-none-any.whl` now builds with `31 passed` in the test step. Signed-off-by: Dawei Huang <daweihuang@microsoft.com>

mssonicbld · 2026-07-10T16:45:31Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-07-10T16:45:45Z

Azure Pipelines successfully started running 1 pipeline(s).

Copilot

Pull request overview

Copilot reviewed 15 out of 20 changed files in this pull request and generated 2 comments.

Files not reviewed (4)

src/sonic-grpc/sonic_grpc/gnoi/common_pb2.py: Generated file
src/sonic-grpc/sonic_grpc/gnoi/file_pb2.py: Generated file
src/sonic-grpc/sonic_grpc/gnoi/system_pb2.py: Generated file
src/sonic-grpc/sonic_grpc/gnoi/types_pb2.py: Generated file

+    install_requires=[
+        'grpcio',
+        'protobuf>=4.21',
+    ],


+        self.assertEqual(len(self.server.system.reboot_status_calls), 1)
+        self.assertEqual(len(self.server.system.cancel_reboot_calls), 1)
+
+    def test_credentials_none_uses_insecure_channel(self):


hdwhdw · 2026-07-14T18:20:30Z

@ronan-nexthop @ndas7 cannot tag you for review so FYI. Following a similar approach as #28339

@sneelam20 for review

ronan-nexthop · 2026-07-14T22:43:25Z

Interesting, is the a real use case for this on the box? If so the approach you take is good, althought I would have thought most of these gNOI operations would be handled off box? I appreciate gNOI RPC's don't change that frequently, in any case the regeneration story here isn't the best, appears file_pb2.py was generated by protobuf 5.29 tooling and grpcio-tools 1.70. system_pb2.py, types_pb2.py, common_pb2.py were generated by protobuf 4.25 tooling, and system_pb2_grpc.py by an older grpcio-tools. setup.py declares protobuf>=4.21 and unpinned grpcio. It would probably be good to have some form of regeneration script

hdwhdw · 2026-07-15T15:46:26Z

@ronan-nexthop Yes. There are Nvidia platform with multiple cards and a midplane network. Also Microsoft also have some orchestration that uses gnoi as a way to standardize on device operation.

This change is mostly for porting to sonic-mgmt instead of on the box container. (Although as a byproduct we also want it to be available on the box).

Agree with the generation. Probably can be built into the build system.

hdwhdw mentioned this pull request Jul 9, 2026

[gnoi]: Add PTF-free native gNOI test suite (System.Time, File.Stat) sonic-net/sonic-mgmt#26040

Draft

8 tasks

hdwhdw mentioned this pull request Jul 9, 2026

[gnoi] PTF-free native gNOI test suite (sonic_grpc.GnoiClient) sonic-net/sonic-mgmt#26039

Open

3 tasks

hdwhdw marked this pull request as ready for review July 10, 2026 14:51

hdwhdw requested a review from qiluo-msft as a code owner July 10, 2026 14:51

Copilot AI review requested due to automatic review settings July 10, 2026 14:51

hdwhdw requested review from lguohan and xumia as code owners July 10, 2026 14:51

Copilot started reviewing on behalf of hdwhdw July 10, 2026 14:51 View session

Copilot AI reviewed Jul 10, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 10, 2026 16:45

Copilot started reviewing on behalf of hdwhdw July 10, 2026 16:45 View session

Copilot AI reviewed Jul 10, 2026

View reviewed changes

hdwhdw requested a review from sneelam20 July 14, 2026 18:20

This was referenced Jul 17, 2026

[gnoi_shutdown_daemon] Log gNOI Reboot failure details sonic-net/sonic-host-services#404

Merged

[gNMI] Migrate legacy tests to native managed gRPC suites sonic-net/sonic-mgmt#26384

Open

Uh oh!

Conversation

hdwhdw commented Jul 9, 2026

Why I did it

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Description for the changelog

Dependencies

Uh oh!

mssonicbld commented Jul 9, 2026

Uh oh!

azure-pipelines Bot commented Jul 9, 2026

Uh oh!

mssonicbld commented Jul 9, 2026

Uh oh!

azure-pipelines Bot commented Jul 9, 2026

Uh oh!

mssonicbld commented Jul 9, 2026

Uh oh!

azure-pipelines Bot commented Jul 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

mssonicbld commented Jul 10, 2026

Uh oh!

azure-pipelines Bot commented Jul 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

hdwhdw commented Jul 14, 2026

Uh oh!

ronan-nexthop commented Jul 14, 2026

Uh oh!

hdwhdw commented Jul 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants