[doc] Update stages doc with top-level stages#597
Draft
martin-velay wants to merge 1 commit into
Draft
Conversation
Contributor
Author
|
@marnovandermaas, WDYT? |
Signed-off-by: martin-velay <mvelay@lowrisc.org>
martin-velay
force-pushed
the
top_level_stages
branch
from
2d50ddb to
f094b94
Compare
Collaborator
marnovandermaas
left a comment
marnovandermaas
left a comment
There was a problem hiding this comment.
Thank you so much for this PR. I really enjoyed the motivation sections you added. I put some of my thoughts in here, which I hope are useful.
| It should also update [the table](#current-status) documenting the current status of each block. | ||
|
|
||
| ## Design stages | ||
| ## IP block design stages |
Collaborator
There was a problem hiding this comment.
How about "Hardware IP block"?
| These stages apply to the `top_chip` integration testbench. | ||
| The two key documents governing chip-level verification are: | ||
|
|
||
| - **Verification plan** (primary): [`hw/top_chip/dv/data/top_mocha_vplan.hjson`](../../hw/top_chip/dv/data/top_mocha_vplan.hjson) - defines the coverage metrics and their mapping to tests. |
Collaborator
There was a problem hiding this comment.
This doesn't exist yet right?
| The two key documents governing chip-level verification are: | ||
|
|
||
| - **Verification plan** (primary): [`hw/top_chip/dv/data/top_mocha_vplan.hjson`](../../hw/top_chip/dv/data/top_mocha_vplan.hjson) - defines the coverage metrics and their mapping to tests. | ||
| - **Testplan**: [`hw/top_chip/data/chip_testplan.hjson`](../../hw/top_chip/data/chip_testplan.hjson) - captures individual testpoints and their associated tests. |
Collaborator
There was a problem hiding this comment.
This also doesn't exist yet right?
|
|
||
| A test that could pass or fail based solely on IP-internal behaviour, with no observable effect at the chip level, does not belong in the chip-level testplan. | ||
|
|
||
| | **Stage** | **Name** | **Definition** | |
Collaborator
There was a problem hiding this comment.
This should have it's own header I think.
| |-----------|----------|----------------| | ||
| | V0 | Initial Work | <ul> <li> Chip-level testbench being set up </li> <li> Chip-level verification plan being written </li> <li> Chip-level testplan being written </li> </ul> | | ||
| | V1 | Smoke Passing | <ul> <li> All IPs smoke-tested at chip level </li> <li> Testbench infrastructure validated </li> <li> CI smoke regression running </li> </ul> | | ||
| | V2 | Integration Complete | <ul> <li> All planned chip-level tests passing </li> <li> All chip interfaces connected to an active agent and exercised end-to-end </li> <li> End-to-end interrupt routing confirmed for all interrupt-capable IPs </li> <li> Cross-IP integration paths and reset sequences exercised </li> <li> Chip-level coverage targets met </li> </ul> | |
Collaborator
There was a problem hiding this comment.
Can we say that the tests are passing with at least 90% of random seeds?
| | **Item name** | **Description** | | ||
| |---------------|-----------------| | ||
| | TOP_DV_DOC_DRAFTED | DV document drafted covering testbench architecture, agent topology, firmware-driven stimulus model, and chip-level coverage intent. | | ||
| | TOP_VPLAN_COMPLETED | Verification plan (`top_mocha_vplan.hjson`) complete with the metric-to-test mapping for each coverage item and milestone specified. Reviewed by designers, a peer DV engineer, firmware author, and chip architect. | |
Collaborator
There was a problem hiding this comment.
I think completed is too ambitious here. I would like drafted here. That doesn't mean it cannot be complete but I think it might prevent us from doing the sign-off with a vplan that is good enough.
| |---------------|-----------------| | ||
| | TOP_DV_DOC_DRAFTED | DV document drafted covering testbench architecture, agent topology, firmware-driven stimulus model, and chip-level coverage intent. | | ||
| | TOP_VPLAN_COMPLETED | Verification plan (`top_mocha_vplan.hjson`) complete with the metric-to-test mapping for each coverage item and milestone specified. Reviewed by designers, a peer DV engineer, firmware author, and chip architect. | | ||
| | TOP_TESTPLAN_COMPLETED | Chip-level testplan (`chip_testplan.hjson`) complete with at least one testpoint per integrated IP. Reviewed by designers, a peer DV engineer, firmware author, and chip architect. | |
Collaborator
There was a problem hiding this comment.
Same as for vplan.
| | TOP_DV_DOC_DRAFTED | DV document drafted covering testbench architecture, agent topology, firmware-driven stimulus model, and chip-level coverage intent. | | ||
| | TOP_VPLAN_COMPLETED | Verification plan (`top_mocha_vplan.hjson`) complete with the metric-to-test mapping for each coverage item and milestone specified. Reviewed by designers, a peer DV engineer, firmware author, and chip architect. | | ||
| | TOP_TESTPLAN_COMPLETED | Chip-level testplan (`chip_testplan.hjson`) complete with at least one testpoint per integrated IP. Reviewed by designers, a peer DV engineer, firmware author, and chip architect. | | ||
| | TOP_TB_COMPLETED | Top-level testbench instantiates the DUT with all chip interfaces connected to a UVM agent, an interface or a module that can actively drive or passively observe them. Tie-offs are only permitted for interfaces that are architecturally unused; each must be documented with justification. | |
Collaborator
There was a problem hiding this comment.
I think maybe here we can have wording on "mostly complete" or allow "waivers by the DV lead".
| | TOP_TESTPLAN_COMPLETED | Chip-level testplan (`chip_testplan.hjson`) complete with at least one testpoint per integrated IP. Reviewed by designers, a peer DV engineer, firmware author, and chip architect. | | ||
| | TOP_TB_COMPLETED | Top-level testbench instantiates the DUT with all chip interfaces connected to a UVM agent, an interface or a module that can actively drive or passively observe them. Tie-offs are only permitted for interfaces that are architecturally unused; each must be documented with justification. | | ||
| | TOP_BOOT_INFRA_PASSING | The SW-to-DV pass/fail signalling mechanism is confirmed working before any other firmware-driven test result is trusted. | | ||
| | TOP_ALL_TESTS_PASSING_V1 | All V1 testpoints in the testplan passing. | |
Collaborator
There was a problem hiding this comment.
Passing for at least one random seed.
| | TOP_BOOT_INFRA_PASSING | The SW-to-DV pass/fail signalling mechanism is confirmed working before any other firmware-driven test result is trusted. | | ||
| | TOP_ALL_TESTS_PASSING_V1 | All V1 testpoints in the testplan passing. | | ||
| | TOP_VPLAN_COVERAGE_V1 | All V1 items defined in the verification plan achieved. | | ||
| | TOP_SMOKE_REGRESSION_IN_CI | V1 smoke suite runs automatically on PRs touching top-level RTL or testbench and failures block merge. | |
Collaborator
There was a problem hiding this comment.
Can't we do the same with out current nightly setup?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.