Skip to content

sonic-yang-mgmt: uses clause with leaf-list, choice not honored#21907

Merged
qiluo-msft merged 2 commits into
sonic-net:masterfrom
bradh352:bradh352/uses-leaflist
Jun 24, 2025
Merged

sonic-yang-mgmt: uses clause with leaf-list, choice not honored#21907
qiluo-msft merged 2 commits into
sonic-net:masterfrom
bradh352:bradh352/uses-leaflist

Conversation

@bradh352

@bradh352 bradh352 commented Mar 4, 2025

Copy link
Copy Markdown
Collaborator

Why I did it

When a uses clause imports a grouping, it was only processing leaf entries and ignoring leaf-list and choice clauses. That means that for instance in bgp route maps, route_map_in and route_map_out validations would fail.

Honoring the uses refine clause is now also honored which is depended upon in sonic-utilities.

This now precompiles the uses clause and integrates it as if the uses clause was not part of the schema as multiple end users were having to do this additional processing.

This fixes that behavior and adds test cases to ensure it doesn't regress in the future.

This is the proper fix, replacing #21078 that just worked around it.

Removal of uses logic in sonic-utilities here: sonic-net/sonic-utilities#3814

Fixes #22382

Work item tracking
  • Microsoft ADO (number only):

How I did it

Added leaf-list lookup.

How to verify it

See test cases pass

Which release branch to backport (provide reason below if selected)

  • 202405
  • 202411

Tested branch (Please provide the tested image version)

master as of 20250304

Description for the changelog

sonic-yang-mgmt: uses clause with leaf-list, choice not honored

Link to config_db schema for YANG module changes

N/A

A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: Brad House (@bradh352)

@bradh352
bradh352 requested a review from qiluo-msft as a code owner March 4, 2025 02:44
@bradh352
bradh352 marked this pull request as draft March 4, 2025 02:44
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run Azure.sonic-buildimage

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run Azure.sonic-buildimage

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@bradh352
bradh352 force-pushed the bradh352/uses-leaflist branch from 5e93791 to 0a069b5 Compare March 5, 2025 01:04
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run Azure.sonic-buildimage

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run Azure.sonic-buildimage

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run Azure.sonic-buildimage

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run Azure.sonic-buildimage

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run Azure.sonic-buildimage

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@bradh352
bradh352 force-pushed the bradh352/uses-leaflist branch from f7775f7 to 1c8718c Compare March 5, 2025 02:11
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run Azure.sonic-buildimage

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@bradh352 bradh352 changed the title yang: add route_map_in/route_map_out bgp valid tests to trigger leaflist bug under uses sonic-yang-mgmt: uses clause with leaf-list not honored Mar 5, 2025
@bradh352

bradh352 commented Mar 5, 2025

Copy link
Copy Markdown
Collaborator Author

@ganglyu This is the proper fix for #21078 that resolves the underlying issue instead of attempting to work around it. Please review and hopefully approve.

@bradh352
bradh352 marked this pull request as ready for review March 5, 2025 02:16
@bradh352

bradh352 commented Mar 5, 2025

Copy link
Copy Markdown
Collaborator Author

@wen587, @venkatmahalingam you approved the PR that this is replacing. Can you review and approve this instead?

@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run Azure.sonic-buildimage

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@bradh352
bradh352 force-pushed the bradh352/uses-leaflist branch from 363be19 to 15ef7ab Compare March 6, 2025 01:05
@bradh352

bradh352 commented Apr 3, 2025

Copy link
Copy Markdown
Collaborator Author

@adyeung just wanted to see if we could move this forward. Thanks.

@zhangyanzhao

Copy link
Copy Markdown

@qiluo-msft @dgsudharsan can you please help to take a look at this issue? Thanks.

@zjswhhh

zjswhhh commented Jun 18, 2025

Copy link
Copy Markdown
Contributor

Hi @vaibhavhd - can you find a reviewer for this one?

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes handling of YANG uses clauses by including leaf-list and choice entries and honoring refine, and adds corresponding test cases and sample config updates for BGP route-map validation.

  • Extend grouping preprocessing to capture leaf-list and choice nodes.
  • Introduce _compileUsesClause to inline uses clauses into the schema.
  • Add tests and sample config entries for valid BGP neighbor/peergroup route-map references.

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
src/sonic-yang-models/tests/yang_model_tests/tests_config/bgp.json Add valid BGP_NEIGHBOR_AF_ROUTE_MAP_VALID and BGP_PEERGROUP_AF_ROUTE_MAP_VALID configs
src/sonic-yang-models/tests/yang_model_tests/tests/bgp.json Define new test cases for valid route-map references
src/sonic-yang-models/tests/files/sample_config_db.json Populate route_map_in / route_map_out entries in sample DB
src/sonic-yang-mgmt/sonic_yang_ext.py Add _compileUsesClause* methods, update grouping preprocessor
Comments suppressed due to low confidence (2)

src/sonic-yang-mgmt/sonic_yang_ext.py:167

  • This new helper lacks a docstring. Consider adding a brief description of its purpose, parameters (refine, model), and effects to aid future maintainers.
    def _compileUsesClauseRefineApply(self, refine, model):

src/sonic-yang-mgmt/sonic_yang_ext.py:217

  • The new compileUsesClause* methods introduce significant logic but lack direct unit tests. Consider adding focused tests for _compileUsesClauseModel (e.g., grouping with leaf-list, choice, and refine) to ensure correct behavior and prevent regressions.
    def _compileUsesClauseModel(self, module, model):

Comment thread src/sonic-yang-mgmt/sonic_yang_ext.py Outdated
Comment thread src/sonic-yang-mgmt/sonic_yang_ext.py Outdated
Comment thread src/sonic-yang-mgmt/sonic_yang_ext.py Outdated
Comment thread src/sonic-yang-mgmt/sonic_yang_ext.py Outdated
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run Azure.sonic-buildimage

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@bradh352

Copy link
Copy Markdown
Collaborator Author

another spurious test failure ... rebased to get a clean build

@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run Azure.sonic-buildimage

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

bradh352 added 2 commits June 23, 2025 08:56
When a uses clause imports a grouping, it was only processing leaf
entries and ignoring leaf-list and choice clauses.  That means that
for instance in bgp route maps, route_map_in and route_map_out
validations would fail.

Honoring the uses `refine` clause is now also honored which is
depended upon in sonic-utilities.

This now precompiles the uses clause and integrates it as if the
uses clause was not part of the schema as multiple end users were
having to do this additional processing.

This fixes that behavior and adds test cases to ensure it doesn't
regress in the future.

Signed-off-by: Brad House (@bradh352)
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run Azure.sonic-buildimage

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@bradh352

Copy link
Copy Markdown
Collaborator Author

and the last build failed due to mellanox build failure related to some nfs mount failure on the build system ... then also a test timeout. Rebasing again to force another rebuild. None of the failure are related to the content in this PR.

@bradh352

Copy link
Copy Markdown
Collaborator Author

All checks passed. @qiluo-msft can you merge?

@yejianquan

Copy link
Copy Markdown
Contributor

Hi @qiluo-msft , could you help to suggest whether this change should goto 202505 branch?

@qiluo-msft

Copy link
Copy Markdown
Collaborator

I consider this as a new feature for use case like BGP_NEIGHBOR_AF. No need to backport.

@hsw0

hsw0 commented Aug 3, 2025

Copy link
Copy Markdown

I consider this as a new feature for use case like BGP_NEIGHBOR_AF. No need to backport.

@qiluo-msft It fixed #21078, which affects 202505 and older versions. Please reconsider backporting.

@Verma-Anukul

Copy link
Copy Markdown
Contributor

@qiluo-msft @bradh352 @r12f

Can you please get it merged in 202505 also

@mssonicbld

Copy link
Copy Markdown
Collaborator

Cherry-pick PR to msft-202412: Azure/sonic-buildimage-msft#1731

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Bug: sonic-yang-mgmt uses clause evaluation ignores leaf-list and refine