Extending Testing Lua Plugins

Testing Lua Plugins

Prev: Lua Plugin Cookbook | Up: Extending ReqPack | Next: Building Registry Entries

rqp test-plugin runs hermetic plugin conformance cases without touching real package managers.

Use it when you want to verify wrapper behavior such as:

• commands executed through context.exec.run(...) or reqpack.exec.run(...)
• emitted events and payloads
• reported query results
• artifact registration
• success/failure behavior

It is intentionally narrower than full end-to-end testing. It does not validate real package-manager integration.

CLI

rqp test-plugin --plugin <path-or-id> --preset core [--report <file.json>]
rqp test-plugin --plugin <path-or-id> --case <file.lua> [--case <file.lua> ...]
rqp test-plugin --plugin <path-or-id> --cases <directory> [--report <file.json>]

Options:

• --plugin <value>: plugin script path, plugin bundle directory, or plugin id under config.registry.pluginDirectory
• --preset <name>: add preset cases from <plugin>/.reqpack-test/<name>/
• --case <file.lua>: add one case file
• --cases <directory>: add all *.lua files in directory
• --report <file.json>: write JSON summary report

Known preset today:

• core

Case collection behavior:

• all paths are normalized to absolute paths
• duplicates are removed
• final case list is sorted

Plugin Resolution

--plugin is resolved in this order:

1. direct script path
2. plugin directory containing run.lua
3. installed plugin id under config.registry.pluginDirectory/<id>/run.lua

If directory is passed and run.lua is missing, command fails.

What Test Runner Executes

Supported test actions:

• install
• remove
• update
• list
• search
• outdated
• info

Local install case is represented as:

• request.action = "install"
• plus non-empty request.localPath

That dispatches to plugin.installLocal(context, localPath).

Not supported by current test runner:

• pack
• proxy resolution as top-level action
• arbitrary executor/orchestrator flows outside single plugin call

Case File Shape

Case file must return Lua table.

Top-level sections:

• name
• request
• fakeExec
• expect

Minimal passing install case:

return {
  name = "install success",
  request = {
    action = "install",
    system = "demo",
    packages = {
      { name = "curl", version = "8.0" }
    }
  },
  fakeExec = {
    {
      match = "fake-pm install curl",
      exitCode = 0,
      stdout = "ok\n",
      stderr = "",
      success = true,
    }
  },
  expect = {
    success = true,
    commands = { "fake-pm install curl" },
    stdout = { "ok\n" },
    events = { "installed", "success" },
    eventPayloads = {
      installed = "{name=curl}",
      success = "ok",
    },
  }
}

request Section

Recognized request fields:

request = {
  action = "install",
  system = "demo",
  flags = { "flag-a" },
  prompt = "curl",
  localPath = "./artifact.rpm",
  packages = {
    {
      action = "install",
      system = "demo",
      name = "curl",
      version = "8.0",
      sourcePath = "/tmp/curl.rpm",
      localTarget = true,
      flags = { "dep-flag" },
    }
  }
}

Rules:

• request.action is required
• request.system should normally be set
• request.prompt is used for search and info
• request.localPath switches install case to installLocal
• request.packages entries map to normal Package fields

Action notes:

• list and outdated ignore prompt
• info treats call as successful when returned record has non-empty displayable content
• search passes request.prompt directly into plugin search(context, prompt)

fakeExec Section

fakeExec is ordered array of substring-match rules.

Each entry:

{
  match = "fake-pm install curl",
  exitCode = 0,
  stdout = "ok\n",
  stderr = "",
  success = true,
}

Behavior:

• first rule whose match string appears inside executed command wins
• if success is omitted, runner derives it from exitCode == 0
• if no rule matches command, fake exec returns failure with exitCode = 127 and stderr no fakeExec rule matched command: ...

This affects both:

• context.exec.run(...)
• reqpack.exec.run(...)

expect Section

Supported expectation keys:

expect = {
  success = true,
  commands = { "cmd" },
  stdout = { "line\n" },
  stderr = { "boom\n" },
  artifacts = { "/tmp/out.txt" },
  events = { "installed", "success" },
  eventPayloads = {
    installed = "{name=curl}",
  },
  resultCount = 1,
  resultName = "curl",
  resultVersion = "8.0",
}

Expectation semantics:

• commands, stdout, stderr, artifacts, and events use exact ordered comparison when provided
• eventPayloads checks first event with same name
• resultCount, resultName, and resultVersion inspect first query result, or single info() result
• omitted expectation fields are not checked

Report JSON

--report writes JSON with:

• plugin
• pluginPath
• passed
• failed
• cases[]

Each case entry contains:

• name
• success
• message
• commands[]
• stdout[]
• stderr[]
• events[]
• artifacts[]
• eventRecords[] with name and payload

Core Presets

Preset cases live inside plugin bundle:

<plugin>/
  .reqpack-test/
    core/
      list.lua
      search.lua

--preset core fails if directory is missing or contains no *.lua files.

Good preset candidates:

• one happy-path list
• one happy-path search
• one mutating case
• one failure case

What Runner Verifies Well

• command construction
• parsing logic
• emitted event names
• event payload formatting
• query result mapping
• artifact registration
• wrapper thinness and deterministic behavior

What Runner Does Not Replace

• real package-manager smoke tests
• privileged execution tests
• network/auth tests against real repositories
• full orchestrator flows with planning, graph edges, and transaction recovery

Keep both:

1. hermetic test-plugin cases for fast regression coverage
2. a few real host smoke tests for final confidence

Best Practices

• keep match strings specific enough that only one fake command fits
• prefer deterministic stdout fixtures over complex free-form output
• add at least one failure case
• use preset core for baseline contract coverage
• test emitted payload text explicitly when parser logic matters

Related Pages

• Writing Lua Plugins
• Lua Plugin Cookbook
• Building Registry Entries

Prev: Lua Plugin Cookbook | Up: Extending ReqPack | Next: Building Registry Entries