diff --git a/README.md b/README.md index 181cb88..9e91145 100644 --- a/README.md +++ b/README.md @@ -13,15 +13,18 @@ Currently supported: - Ruby with RSpec or Minitest. - Python with pytest. +- JavaScript with Jest. ## Prerequisites Before using DDTest, you must have **Datadog Test Optimization** already set up and enabled with a Datadog Test Optimization library for your language and framework. DDTest relies on this integration to discover your tests and plan test execution accordingly. -Minimum supported library versions: +Minimum supported library and runtime requirements: - Ruby requires the `datadog-ci` gem **1.31.0** or higher. - Python requires the `ddtrace` package **4.10.3** or higher and `pytest`. +- JavaScript requires the `dd-trace` package **6.0.0** or higher, Node.js, and + Jest. For instructions on setting up Test Optimization, see the [Datadog Test Optimization documentation](https://docs.datadoghq.com/tests/setup/). @@ -66,6 +69,16 @@ ddtest plan \ --max-parallelism 32 ``` +For JavaScript/Jest: + +```bash +ddtest plan \ + --platform javascript \ + --framework jest \ + --min-parallelism 8 \ + --max-parallelism 32 +``` + This prepares the plan and writes it to `.testoptimization/` folder for later reuse. Copy `.testoptimization/` to any CI job that runs `ddtest run` or reads DDTest's plan file lists. For the full file layout and formats, see @@ -87,6 +100,12 @@ For Python/pytest: ddtest run --platform python --framework pytest ``` +For JavaScript/Jest: + +```bash +ddtest run --platform javascript --framework jest +``` + For CI-node mode, worker environment variables, custom commands, and parallelism details, see [Running DDTest](docs/running.md). @@ -94,9 +113,9 @@ parallelism details, see [Running DDTest](docs/running.md). | CLI flag | What it does | | --- | --- | -| `--platform` | Language/platform. Currently supported: `ruby`, `python`. | -| `--framework` | Test framework. Currently supported: `rspec`, `minitest`, `pytest`. | -| `--command` | Override the default test command used by Ruby frameworks. For pytest, use `PYTEST_ADDOPTS` for pytest flags. | +| `--platform` | Language/platform. Currently supported: `ruby`, `python`, `javascript`. | +| `--framework` | Test framework. Currently supported: `rspec`, `minitest`, `pytest`, `jest`. | +| `--command` | Override the default base command for supported framework modes. Currently used by RSpec and Minitest run/discovery, and Jest run/discovery. For pytest, use `PYTEST_ADDOPTS` for pytest flags. | | `--min-parallelism` | Minimum CI node or worker count DDTest considers when planning. | | `--max-parallelism` | Maximum CI node or worker count DDTest considers when planning. | | `--target-time` | Target wall time DDTest tries to satisfy when selecting parallelism. | diff --git a/docs/best_practices.md b/docs/best_practices.md index 1ed9ea0..83a7d2a 100644 --- a/docs/best_practices.md +++ b/docs/best_practices.md @@ -3,8 +3,10 @@ ## Optimize Planning Step When using ddtest, you need to add a planning step that performs test discovery -(for example, RSpec dry-run or pytest collection) before execution. This stage -adds overhead: you can optimize it with the practices below. +before execution. For Ruby and Python, this can involve full framework +discovery such as RSpec dry-run or pytest collection. For Jest, DDTest uses +Jest's `--listTests` command to discover test files. This planning stage adds +overhead: you can optimize it with the practices below. ### Preinstall System Dependencies Via Docker @@ -40,6 +42,15 @@ For GitHub Actions + pip: cache: pip ``` +For GitHub Actions + npm: + +```yaml +- uses: actions/setup-node@v4 + with: + node-version: "22" + cache: npm +``` + ### Disable Seeds/Fixtures During Discovery Discovery (planning) does not execute tests; you don't have to setup DB, @@ -123,7 +134,8 @@ corrupt, produced for a different platform/framework/test location/exclude pattern, or based on a commit that is not available locally. It also invalidates the cache when files under the current project's test root changed. For example, the default RSpec root is `spec/**`, the default Minitest root is `test/**`, -and pytest uses `testpaths` from pytest config when available. With +pytest uses `testpaths` from pytest config when available, and Jest uses +DDTest's built-in Jest test file pattern unless `--tests-location` is set. With `--tests-location custom/spec/**/*_spec.rb`, the root is `custom/**`. In monorepos, run DDTest from the project subdirectory whose tests you are @@ -141,6 +153,17 @@ For discovery, DDTest reads `testpaths` and `python_files` from `pytest.ini`, `pyproject.toml`, `tox.ini`, or `setup.cfg`. If no pytest config defines those settings, DDTest uses `**/{test_*,*_test}.py`. +## Jest Support + +Use `--command` when your project runs Jest through a package manager or wrapper: + +```bash +ddtest run --platform javascript --framework jest --command "pnpm jest --runInBand" +``` + +Do not include test files or a `--` separator in the command; DDTest appends the +file list and Jest flags itself. + ## Minitest Support In Non-Rails Projects We use `bundle exec rake test` command when we don't detect `rails` command to diff --git a/docs/examples/circleci.md b/docs/examples/circleci.md index 9b76885..93ebf8c 100644 --- a/docs/examples/circleci.md +++ b/docs/examples/circleci.md @@ -170,3 +170,43 @@ CircleCI node index to DDTest: NODE_INDEX=${CIRCLE_NODE_INDEX:-0} ./bin/ddtest run --platform python --framework pytest --ci-node "${NODE_INDEX}" ``` + +## JavaScript / Jest Variant + +Use the same setup workflow pattern for JavaScript projects, but use a Node.js +image, install project dependencies, and configure the runner for Jest: + +```yaml +jobs: + plan: + docker: + - image: cimg/node:22.14 + environment: + DD_ENV: ci + DD_TEST_OPTIMIZATION_RUNNER_PLATFORM: javascript + DD_TEST_OPTIMIZATION_RUNNER_FRAMEWORK: jest + DD_TEST_OPTIMIZATION_RUNNER_MIN_PARALLELISM: 1 + DD_TEST_OPTIMIZATION_RUNNER_MAX_PARALLELISM: 4 + steps: + - checkout + - run: + name: Install JavaScript dependencies + command: npm ci + - test-optimization-circleci-orb/autoinstrument: + languages: js + site: datadoghq.eu + - run: + name: Plan tests with ddtest + command: ./bin/ddtest plan +``` + +In the test job, keep the restored `.testoptimization/` plan, install +dependencies, and pass the CircleCI node index to DDTest: + +```yaml +- run: + name: Run tests with ddtest + command: | + NODE_INDEX=${CIRCLE_NODE_INDEX:-0} + ./bin/ddtest run --platform javascript --framework jest --ci-node "${NODE_INDEX}" +``` diff --git a/docs/examples/github-actions.md b/docs/examples/github-actions.md index 33e8e77..bb50d6a 100644 --- a/docs/examples/github-actions.md +++ b/docs/examples/github-actions.md @@ -133,3 +133,45 @@ Configure Datadog Test Optimization for Python: The `ddtest plan` and `ddtest run --ci-node ${{ matrix.ci_node_index }}` commands can stay the same when the platform and framework are provided through the environment. + +## JavaScript / Jest Variant + +Use the same plan/test job structure for JavaScript projects, but configure the +runner and setup steps for Jest: + +```yaml +env: + DD_TEST_OPTIMIZATION_RUNNER_PLATFORM: javascript + DD_TEST_OPTIMIZATION_RUNNER_FRAMEWORK: jest + DD_TEST_OPTIMIZATION_RUNNER_MIN_PARALLELISM: 1 + DD_TEST_OPTIMIZATION_RUNNER_MAX_PARALLELISM: 8 +``` + +Replace each Ruby setup step with Node.js dependency installation: + +```yaml +- name: Setup Node.js + uses: actions/setup-node@v4 + with: + node-version: "22" + cache: npm +- name: Install JavaScript dependencies + run: npm ci +``` + +Configure Datadog Test Optimization for JavaScript: + +```yaml +- name: Configure Datadog Test Optimization + uses: datadog/test-visibility-github-action@v2 + with: + languages: js + api_key: ${{ secrets.DD_API_KEY }} + site: datadoghq.com +``` + +DDTest sets `NODE_OPTIONS=-r dd-trace/ci/init` for Jest worker processes, so the +project dependencies installed before `ddtest plan` must include `dd-trace`. +The `ddtest plan` and `ddtest run --ci-node ${{ matrix.ci_node_index }}` +commands can stay the same when the platform and framework are provided through +the environment. diff --git a/docs/layout.md b/docs/layout.md index 887d9e3..c44ca27 100644 --- a/docs/layout.md +++ b/docs/layout.md @@ -237,7 +237,7 @@ Fields: | --- | --- | | `name` | Test name reported by the framework. | | `suite` | Test suite name reported by the framework. | -| `module` | Framework module name, such as `rspec`, `minitest`, or `pytest`. | +| `module` | Framework module name, such as `rspec`, `minitest`, `pytest`, or `jest`. | | `parameters` | Serialized test parameters. | | `suiteSourceFile` | Source file containing the suite. | diff --git a/docs/local-ci-skippable-tests.md b/docs/local-ci-skippable-tests.md index e970c3a..8fde8e9 100644 --- a/docs/local-ci-skippable-tests.md +++ b/docs/local-ci-skippable-tests.md @@ -30,7 +30,10 @@ Use `--runtime-tags` to override your local runtime tags with the tags from CI. For Python projects, `runtime.name` is the Python implementation name (for example `CPython` for standard Python), and `runtime.version` is the Python interpreter version used in CI. - Python interpreter version used in CI. + + For JavaScript projects, `runtime.name` is `node`, and `runtime.version` is + the Node.js version used in CI, including the leading `v` (for example, + `v22.16.0`). 2. Create the runtime tags JSON. diff --git a/docs/running.md b/docs/running.md index a506a76..9d0e16b 100644 --- a/docs/running.md +++ b/docs/running.md @@ -26,6 +26,12 @@ For Python/pytest: ddtest run --platform python --framework pytest ``` +For JavaScript/Jest: + +```bash +ddtest run --platform javascript --framework jest +``` + On one CI node, the default `--min-parallelism` and `--max-parallelism` equal the available physical CPU core count, so DDTest can start one worker per physical core without defaulting to one worker per hyperthread. @@ -45,6 +51,12 @@ For Python/pytest: ddtest run --platform python --framework pytest --ci-node ``` +For JavaScript/Jest: + +```bash +ddtest run --platform javascript --framework jest --ci-node +``` + In CI-node mode, DDTest uses one local worker by default so database and other per-worker resources stay easy to isolate. To fan out within each CI node, set `--ci-node-workers` to a positive integer, or use `--ci-node-workers ncpu` to @@ -78,12 +90,19 @@ starting each worker. Use `--command` to override the framework's default base test command where supported. DDTest currently applies this override to RSpec run and full -discovery, Minitest run and full discovery, and Jest run: +discovery, Minitest run and full discovery, and Jest run and file discovery: ```bash ddtest run --platform ruby --framework rspec --command "bundle exec rspec --profile" ``` +For JavaScript/Jest, DDTest automatically appends `--listTests` during planning +and `--runTestsByPath ` during execution: + +```bash +ddtest run --platform javascript --framework jest --command "pnpm jest --runInBand" +``` + When using `--command`, do not include the `--` separator or test files in your command. DDTest automatically appends selected tests and framework-specific flags to the command you provide. @@ -118,6 +137,23 @@ For Python/pytest, DDTest discovers test files using this priority: Pytest does not have an equivalent to RSpec's pattern flag, so DDTest resolves the pattern to explicit file paths before invoking `python -m pytest`. +## Jest Discovery And Instrumentation + +For JavaScript/Jest, DDTest discovers test files with Jest's own `--listTests` +command. It uses this priority: + +1. `--command` when set, with `--listTests` appended. +2. The local executable `node_modules/.bin/jest` when present. +3. `npx jest`. + +Jest uses its own configuration and default test matching for `--listTests`. +When `--tests-location` or `--tests-exclude-pattern` is set, DDTest filters the +file list returned by Jest after discovery; it does not pass `--tests-location` +as Jest's `--testMatch`. + +DDTest prepends `-r dd-trace/ci/init` to `NODE_OPTIONS` for worker processes +unless `NODE_OPTIONS` already loads `dd-trace/ci/init`. + ## Parallelism Selection DDTest chooses parallelism by estimating the runnable duration of each test file, diff --git a/docs/settings.md b/docs/settings.md index 7a26fe1..7a6956e 100644 --- a/docs/settings.md +++ b/docs/settings.md @@ -5,9 +5,9 @@ CLI flags take precedence over environment variables. | CLI flag | Environment variable | Env alias | Default | What it does | | --- | --- | --- | ---: | --- | -| `--platform` | `DD_TEST_OPTIMIZATION_RUNNER_PLATFORM` | | `ruby` | Language/platform. Currently supported: `ruby`, `python`. | -| `--framework` | `DD_TEST_OPTIMIZATION_RUNNER_FRAMEWORK` | | `rspec` | Test framework. Currently supported: `rspec`, `minitest`, `pytest`. | -| `--command` | `DD_TEST_OPTIMIZATION_RUNNER_COMMAND` | | `""` | Override the default base test command for supported framework modes. Currently used by RSpec and Minitest run/discovery, and Jest run; pytest ignores it. DDTest appends selected tests and framework-specific flags. For pytest, use `PYTEST_ADDOPTS` for pytest flags. | +| `--platform` | `DD_TEST_OPTIMIZATION_RUNNER_PLATFORM` | | `ruby` | Language/platform. Currently supported: `ruby`, `python`, `javascript`. | +| `--framework` | `DD_TEST_OPTIMIZATION_RUNNER_FRAMEWORK` | | `rspec` | Test framework. Currently supported: `rspec`, `minitest`, `pytest`, `jest`. | +| `--command` | `DD_TEST_OPTIMIZATION_RUNNER_COMMAND` | | `""` | Override the default base test command for supported framework modes. Currently used by RSpec and Minitest run/discovery, and Jest run/discovery; pytest ignores it. DDTest appends selected tests and framework-specific flags. For pytest, use `PYTEST_ADDOPTS` for pytest flags. | | `--min-parallelism` | `DD_TEST_OPTIMIZATION_RUNNER_MIN_PARALLELISM` | | physical CPU count | Minimum count DDTest considers when planning. Interpret it as CI nodes in CI-node mode, or workers in a single-node run. | | `--max-parallelism` | `DD_TEST_OPTIMIZATION_RUNNER_MAX_PARALLELISM` | | physical CPU count | Maximum count DDTest considers when planning. Interpret it as CI nodes in CI-node mode, or workers in a single-node run. | | `--ci-job-overhead` | `DD_TEST_OPTIMIZATION_RUNNER_CI_JOB_OVERHEAD` | | `25s` | Modeled overhead for adding one more CI node. Accepts durations such as `25s`, `1m`, `1500ms`, or `0s` to disable this bias. Increase it to use fewer CI nodes; decrease it to prefer faster wall time. | @@ -15,7 +15,7 @@ CLI flags take precedence over environment variables. | `--ci-node` | `DD_TEST_OPTIMIZATION_RUNNER_CI_NODE` | | `-1` (off) | Restrict this run to files assigned to CI node **N** (0-indexed). | | `--ci-node-workers` | `DD_TEST_OPTIMIZATION_RUNNER_CI_NODE_WORKERS` | | `1` | Number of workers to start on this CI node. Use a positive integer, or `ncpu` to use the node's available physical CPU cores. | | `--worker-env` | `DD_TEST_OPTIMIZATION_RUNNER_WORKER_ENV` | | `""` | Template env vars per worker: `--worker-env "DATABASE_NAME_TEST=app_test{{nodeIndex}}_{{workerIndex}}"`. `{{nodeIndex}}` is the CI node index (`0` for single-node runs); `{{workerIndex}}` is the worker process index within that CI node. | -| `--tests-location` | `DD_TEST_OPTIMIZATION_RUNNER_TESTS_LOCATION` | `KNAPSACK_PRO_TEST_FILE_PATTERN` | `""` | Custom glob pattern to discover test files, such as `--tests-location "custom/spec/**/*_spec.rb"` or `--tests-location "tests/**/*_test.py"`. Defaults to `spec/**/*_spec.rb` for RSpec, `test/**/*_test.rb` for Minitest, and pytest config or `**/{test_*,*_test}.py` for pytest. | +| `--tests-location` | `DD_TEST_OPTIMIZATION_RUNNER_TESTS_LOCATION` | `KNAPSACK_PRO_TEST_FILE_PATTERN` | `""` | Custom glob pattern to discover test files, such as `--tests-location "custom/spec/**/*_spec.rb"`, `--tests-location "tests/**/*_test.py"`, or `--tests-location "packages/**/__tests__/**/*.test.ts"`. Defaults to `spec/**/*_spec.rb` for RSpec, `test/**/*_test.rb` for Minitest, pytest config or `**/{test_*,*_test}.py` for pytest, and Jest's configured/default test matching for Jest. | | `--tests-exclude-pattern` | `DD_TEST_OPTIMIZATION_RUNNER_TESTS_EXCLUDE_PATTERN` | `KNAPSACK_PRO_TEST_FILE_EXCLUDE_PATTERN` | `""` | Glob pattern to exclude test files from discovery, such as `--tests-exclude-pattern "spec/system/**/*_spec.rb"`. | | `--test-discovery-cache` | `DD_TEST_OPTIMIZATION_RUNNER_TEST_DISCOVERY_CACHE` | | `""` | Path to a restored test discovery cache file. DDTest imports it before planning and refreshes the internal discovery cache after successful full discovery. | | `--force-full-test-discovery` | `DD_TEST_OPTIMIZATION_RUNNER_FORCE_FULL_TEST_DISCOVERY` | | `false` | Force full test discovery when the framework supports it, including in suite-level skipping mode. | diff --git a/docs/third-party-runners.md b/docs/third-party-runners.md index 0639a71..1ad3821 100644 --- a/docs/third-party-runners.md +++ b/docs/third-party-runners.md @@ -30,6 +30,21 @@ if [ -s .testoptimization/runner/test-files.txt ]; then fi ``` +## Jest + +When another runner consumes DDTest's file list for Jest, make sure the +`dd-trace` Test Optimization init is loaded the same way `ddtest run` loads it: + +```bash +case "${NODE_OPTIONS:-}" in + *dd-trace/ci/init*) ;; + *) export NODE_OPTIONS="-r dd-trace/ci/init${NODE_OPTIONS:+ $NODE_OPTIONS}" ;; +esac +if [ -s .testoptimization/runner/test-files.txt ]; then + xargs ./node_modules/.bin/jest --runTestsByPath < .testoptimization/runner/test-files.txt +fi +``` + ## Custom Runners Read `.testoptimization/runner/test-files.txt` when your runner should handle diff --git a/docs/upgrade-1.0.md b/docs/upgrade-1.0.md index d751d1d..19a7f45 100644 --- a/docs/upgrade-1.0.md +++ b/docs/upgrade-1.0.md @@ -6,10 +6,12 @@ Before upgrading to 1.0, update CI jobs and custom integrations to consume the ## Required Changes -Minimum supported library versions: +Minimum supported library and runtime requirements: - Ruby requires the `datadog-ci` gem 1.31.0 or higher. - Python requires the `ddtrace` package 4.10.3 or higher and `pytest`. +- JavaScript requires the `dd-trace` package 6.0.0 or higher, Node.js, and + Jest. DDTest 1.0 writes plan files under `.testoptimization/runner/*`. If your scripts, CI jobs, or custom test runners read DDTest plan files directly, update @@ -26,7 +28,7 @@ these paths: 1. Verify the Datadog Test Optimization library version for your platform: `datadog-ci` 1.31.0 or higher for Ruby, or `ddtrace` 4.10.3 or higher for - Python. + Python, or `dd-trace` 6.0.0 or higher for JavaScript. 2. Remove references to legacy root plan paths from CI templates and custom scripts. 3. Run `ddtest plan` in CI. 4. Run one CI shard with `DD_TEST_OPTIMIZATION_RUNNER_CI_NODE=0 ddtest run`.