Skip to content

feat(e2e): add JSON file support for extra test cases#727

Draft
mdryaan wants to merge 1 commit into
urunc-dev:mainfrom
mdryaan:feat/e2e-json-test-cases
Draft

feat(e2e): add JSON file support for extra test cases#727
mdryaan wants to merge 1 commit into
urunc-dev:mainfrom
mdryaan:feat/e2e-json-test-cases

Conversation

@mdryaan
Copy link
Copy Markdown
Contributor

@mdryaan mdryaan commented May 26, 2026

Description

Adds a JSON loading layer to the e2e test suite so operators can inject additional foreground test cases into nerdctlTestCases() and ctrTestCases() by editing JSON files under tests/e2e/testdata/ without modifying Go source.

JSON-defined cases always use runForegroundTest (output matching via ExpectOut) since TestFunc is a function
pointer that cannot be serialized to JSON. Docker and crictl are excluded because their test runners call tc.TestFunc(tool) directly and would panic on a nil TestFunc — those can be extended in a follow-up once a function registry or similar mechanism is added.

New files:

  • tests/e2e/json_test_cases.gojsonTestCase struct,
    loadTestCasesFromJSON, optionalJSONTestCases
  • tests/e2e/json_test_cases_test.go — unit tests
  • tests/e2e/testdata/nerdctl_extra_test_cases.json — empty by default
  • tests/e2e/testdata/ctr_extra_test_cases.json — empty by default

Modified:

  • tests/e2e/test_cases.gonerdctlTestCases() and
    ctrTestCases() now append optionalJSONTestCases(...)

Related issues

How was this tested?

Added TestLoadTestCasesFromJSON (6 subtests) and TestOptionalJSONTestCases (3 subtests) in tests/e2e/json_test_cases_test.go. All 9 subtests pass: go test -count=1 -timeout 20m ./tests/e2e/... -v -run "TestLoadTestCasesFromJSON|TestOptionalJSONTestCases" Linter passes (golangci-lint run, 0 issues). Build passes (make).

LLM usage

N/A

Checklist

  • I have read the contribution guide.
  • The linter passes locally (make lint).
  • The e2e tests of at least one tool pass locally (make test_ctr, make test_nerdctl, make test_docker, make test_crictl).
  • If LLMs were used: I have read the llm policy.

Fixes: urunc-dev#123
Signed-off-by: Md Raiyan <alikhurshid842001@gmail.com>
@netlify
Copy link
Copy Markdown

netlify Bot commented May 26, 2026

Deploy Preview for urunc canceled.

Name Link
🔨 Latest commit 3a009e6
🔍 Latest deploy log https://app.netlify.com/projects/urunc/deploys/6a14ebc65af0820009c9ac65

@mdryaan mdryaan marked this pull request as draft May 26, 2026 00:42
@mdryaan mdryaan marked this pull request as ready for review May 26, 2026 12:43
@mdryaan mdryaan marked this pull request as draft May 27, 2026 05:15
@mdryaan
Copy link
Copy Markdown
Contributor Author

mdryaan commented May 27, 2026

Implementation JSON test cases work for nerdctl and ctr via the foreground path. Docker and crictl are excluded for now since their runners call TestFunc directly and a nil value would panic, Happy to extend those in a follow up if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make end-to-end testing read from a JSON file

1 participant