Informative

Non-volatile, non-changing DUT operational parameters are of interest to almost all test cases. A test should be written so that all variations of hardware are supported dynamically. In other words, tests written to assume a DUT has a known number of namespaces or always has 20 doorbell registers is not generic and thus not useful for all implementations of the NVMe Specification. This category of data can be gathered once at application launch and be referenced by all tests for convenience. This would alleviate each test from issuing the same commands to learn of these boundary conditions. This type of data is informative and is available in this design by a singleton called gInformative. It is created automatically by the framework and is guaranteed to exist before any test assumes control and is guaranteed to contain data.

The framework is designed to be passive in that it does not create resources which act upon the DUT unless the action is seen as unavoidable. For example, it is impossible to place the DUT into a well known state between groups unless the framework forces a reset and a disabling of interrupts. Furthermore, the framework will not create resources to interact with the DUT, unless the command line specifically forces this as in the case of using the --queues, --format and --golden command line options. In all other cases the framework is completely passive, meaning only test case(s) interact with the DUT. This means singleton gInformative is not allowed to interact and gather data to learn of the boundary conditions which the test cases are interested. However, gInformative only gathers data using administration commands which are core to every DUT. If a DUT doesn't support the simple Identify command or the mandatory Get Features identifiers then it isn't ready for rigorous testing. In other words, core functionality is essential for minimal testing. Therefore, the only exception to the rule the framework must be 100% passive is during the initialization, construction, and re-initialization of the singleton gInformative, from this point forward referred to as the initialization of gInformative.

The initialization of gInformative is designed to gather common useful data that won't change as a result of some test acting upon the DUT. This data is embedded within the DUT, set at manufacturing time and cannot be modified. All tests will have access to gInformative's data via its public methods. However, the data is given out as a const references/pointer so that a rogue test doesn't changed the singleton's internal data inadvertently.

It is conceivable that some future enhancement to the NVMe spec, or the creation of a new command set could cause a DUT's identify data to change during run time. These future unknowns are being planned for today by providing the ability to re-initialize gInformative at any time. It is imperative, and a rather large mistake to act upon the DUT, thus changing its so called "constant data" and not updating gInformative. The end result will yield a rather hard bug to find, for the test cases won't be covering, or learning the new boundaries of the DUT. This will ultimately lead to incomplete test coverage and false compliance validations.