Zcmp cosim mismatches in llvm-snippy generated tests

[AnastasiyaChernikova]

Observed Behavior

When running Snippy-generated Zcmp tests with cosimulation enabled, some tests fail with cosim mismatches.

The issue was found while experimenting with the llvm-snippy integration from this PR

Spike is used as the cosimulation reference model, built from the ibex_cosim branch as described in the Ibex documentation.

Two different types of cosim mismatches were observed.
One failure mode is a register write index mismatch:

Cosim mismatch Register write index mismatch, DUT: x21 expected: x9

Another failure mode is a missing expected store access:

Cosim mismatch A store at address 800a6e50 was expected but there are no pending accesses

At the moment it is not clear whether this points to an issue in the Ibex implementation of the Zcmp extension, in the Spike model used for cosimulation, or in the interaction between the two.

Expected Behavior

Snippy-generated Zcmp tests should pass RTL cosimulation without register writeback or memory access mismatches.

In particular, the architectural state observed from the DUT should match the Spike reference model for Zcmp instructions such as cm.push, cm.pop, cm.popret, cm.popretz, cm.mva01s, and cm.mvsa01.

Steps to reproduce the issue

The issue was reproduced using the llvm-snippy flow from: #2444

Run the following command from dv/uvm/core_ibex:

make run_snippy \
  OUT=out \
  COV=1 \
  SIMULATOR=vcs \
  IBEX_CONFIG=opentitan \
  RV32ZC=ibex_pkg::RV32ZcaZcbZcmp \
  SNIPPY_ITERATIONS=2 \
  SNIPPY_YAML_DIR=snippy/yaml_tests/zcmp

After the run, inspect the logs under:

dv/uvm/core_ibex/out/run/tests

All generated Zcmp tests fail during cosimulation with errors similar to:

Cosim mismatch Register write index mismatch, DUT: x21 expected: x9

and:

Cosim mismatch A store at address 800a6e50 was expected but there are no pending accesses

My Environment

EDA tool and version:

Synopsys VCS S-2021.09
vcs script version: S-2021.09
machine OS: Linux 5.15.0-164-generic

Operating system:

Rocky Linux release 8.10

Version of the Ibex source code:

Ibex source based on this pr

Spike is used for cosimulation as described in the Ibex documentation, built from the ibex_cosim branch.

[AnastasiyaChernikova]

Attaching the snippy_layout_mvsa_call_test launch logs with a seed value of 1780581282555988937 and the snippy_layout_push_pop_test launch logs with a seed value of 1780581282555989068

[SamuelRiedel]

Hi @AnastasiyaChernikova, thank you for looking into Snippy with Ibex! I wasn't familiar with it before, but it looks like a really interesting tool.

I want to be upfront regarding integrating Snippy into Ibex. I still need to discuss this with our DV team to see if it's something we can merge. Integrating a new tool into our environment means committing to supporting and maintaining it long-term. I can't promise that we will have the bandwidth for that, but it is great to see it being tested against our core and we really appreciate you sharing your findings!

Concerning the cosim mismatches with the Zcmp extension: this extension is very experimental at the moment and not fully supported in our DV and software environment, as you might have already noticed in #2398. Did you use the master branch to run your cosim?

We have extended our DV environment with the necessary changes in the cosim and riscv-dv to support the Zc* extensions in this PR: #2324. However, we are currently waiting for the compiler to update before we can merge it, and we're hoping to upstream our riscv-dv changes first.

That PR has a commit (5f3e8a6) that currently reverts and disables some of the changes necessary to run Zc* with the latest compiler to not break the current CI. If you rebase your work on top of that PR without this last commit, you should be able to run Zc* tests with a cosim environment that works.

[AnastasiyaChernikova]

@SamuelRiedel, thank you very much for the clarification and for pointing me to the relevant changes.

I updated my local branch by applying the changes from the referenced PR. After that, the Snippy-based tests that previously led to cosimulation mismatches started passing successfully.

The PR applied cleanly on top of the current integration, and after the update I was able to run more than 30 different test runs from the Snippy PR. All of them completed successfully.

Therefore, it looks like the issue I observed was not related to the Ibex implementation itself, but rather to the fact that these experimental Zc*/Zcmp scenarios required the corresponding updates in the cosimulation/reference flow. Thank you again for the help and for pointing me in the right direction for checking this.