[Backport pipeline] Add PR-driven backport branch creation

[mrodm]

Proposed commit message

Adds an automated, PR-driven workflow to create backport branches from `.backports.yml`.
When a PR that adds new entries to `.backports.yml` is merged into `main`, the CI pipeline
detects the new entries and triggers the `integrations-backport` pipeline to create the
corresponding branches automatically. A comment is then posted back to the originating PR
reporting success or failure.

WHAT:
Adds an automated, PR-driven workflow to create backport branches from .backports.yml.
When a PR that adds new entries to .backports.yml is merged into main, the CI pipeline
detects the new entries and triggers the integrations-backport pipeline to create the
corresponding branches automatically. A comment is then posted back to the originating PR
reporting success or failure.

Key components:

• .backports.yml inventory — source of truth listing backport branches with their
  package, base version, base commit, and lifecycle metadata (archived, maintained_until).
• trigger_backport.sh / trigger_backport_lib.sh — diff the old and new inventory on each
  push to main, validate the new inventory via mage ValidateBackportsInventory, and
  emit Buildkite trigger steps for any newly-added active entries.
• pipeline.yml — adds check-backports-inventory (schema validation on PRs),
  trigger-backport-dryrun (dry-run on PRs), and trigger-backport-create (real creation
  on merge to main) steps.
• pipeline.backport.yml — updated to accept BACKPORT_BRANCH_NAME via meta_data
  (passed by the trigger step), restricts UI access to the ecosystem team, adds a
  notify-pr step that posts a success/failure comment back to the originating PR.
• notify_backport_pr.sh — posts an idempotent, per-run comment on the PR using a
  hidden HTML marker to avoid duplicate comments on retries.
• AddBackportEntry mage target — developer tool to add a new entry to .backports.yml,
  resolving the base commit via dev/scripts/get_release_commit.sh and inserting the entry
  in sorted order (package ascending, version descending).
• AddEntry in dev/backports/inventory.go — validates the package name against the
  known packages set before writing, preventing branches that would fail branchRE.
• Shell tests in test_trigger_backport.sh — unit-tests generate_trigger_pipeline with
  a per-target mage mock (MOCK_MAGE_VALIDATE_EXIT / MOCK_MAGE_CHECK_EXIT).

WHY:
Previously, backport branches had to be created manually by running backport_branch.sh
by hand. This automation eliminates that toil: once a developer adds an entry to
.backports.yml in a PR, the branch is created automatically on merge, and the PR author
is notified of the outcome without needing to monitor CI separately.

Author's Checklist

• New shell scripts are covered by tests in test_trigger_backport.sh
• mage ValidateBackportsInventory passes on .backports.yml
• pipeline.backport.yml dry-run mode tested end-to-end in a Buildkite build
• Documentation updated (docs/extend/developer-workflow-support-old-package.md and docs/ci_pipelines.md)

How to test this PR locally

1. Run Go tests: go test ./dev/backports/...
2. Run shell tests: bash .buildkite/scripts/test_trigger_backport.sh
3. Run inventory validation: mage ValidateBackportsInventory
4. Test AddBackportEntry mage target:

   mage AddBackportEntry <package_name> <base_version>
   

5. Trigger a dry-run build on a branch that modifies .backports.yml and verify that trigger-backport-dryrun emits the correct trigger steps.

Related issues

• Relates backport: make branch creation PR-driven via .backports.yml #19211

---

This PR was generated with the assistance of Claude (claude-sonnet-4-6).

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: f6a7168

Failed CI Steps

• :bash: 🐍 Buildkite scripts unit tests

History

• 💔 Build #44732 failed 7b1948b

cc @mrodm

[github-actions]

TL;DR

The :bash: :snake: Buildkite scripts unit tests job is failing because the new test_trigger_backport.sh path calls generate_trigger_pipeline, which immediately runs yq -e '.backports' ..., but .buildkite/scripts/run_buildkite_scripts_tests.sh never bootstraps the repo-pinned yq before running that test.

Remediation

• In .buildkite/scripts/run_buildkite_scripts_tests.sh, source .buildkite/scripts/common.sh and run add_bin_path + with_yq before test_trigger_backport.sh is invoked.
• Re-run .buildkite/scripts/run_buildkite_scripts_tests.sh in the same Buildkite image/step to confirm the trigger backport tests reach the mage mocks and generate the expected pipeline YAML.

Investigation details

Root Cause

This is a test dependency/setup issue, not a backport inventory fixture issue.

The PR adds test_trigger_backport.sh, whose header says it requires yq, and the test sources trigger_backport_lib.sh. In that library, generate_trigger_pipeline validates both inventories with yq -e '.backports' before doing any mocked mage work. However, the Buildkite scripts test runner only creates the Python venv and installs .buildkite/scripts/requirements-ci-python-scripts.txt (requests, PyYAML), then runs the shell tests; it does not call the existing .buildkite/scripts/common.sh helper with_yq.

Relevant code paths:

• .buildkite/scripts/run_buildkite_scripts_tests.sh: runs test_trigger_backport.sh after the changelog tests, but has no source .buildkite/scripts/common.sh, add_bin_path, or with_yq setup.
• .buildkite/scripts/trigger_backport_lib.sh: generate_trigger_pipeline starts by executing yq -e '.backports' "${old_inventory}" and yq -e '.backports' "${new_inventory}".
• .buildkite/scripts/common.sh:199-210: with_yq downloads and puts the expected Mike Farah yq on PATH.

Evidence

• Build: https://buildkite.com/elastic/integrations/builds/44732
• Job/step: :bash: :snake: Buildkite scripts unit tests
• Key log excerpt:

=== Running trigger_backport_lib.sh tests ===
--- new entry — dry-run mode
ERROR: old inventory is not valid YAML or missing 'backports' key: /tmp/tmp.KP8o5VJNMp/old.yml
FAIL: dry-run: step header present — 'steps:' not found in /tmp/tmp.KP8o5VJNMp/pipeline.yml
...
--- multiple new entries
ERROR: old inventory is not valid YAML or missing 'backports' key: /tmp/tmp.KP8o5VJNMp/old.yml
FAIL: multiple: aws step present — 'backport-aws-2.0' not found in /tmp/tmp.KP8o5VJNMp/pipeline.yml
FAIL: multiple: gcp step present — 'backport-gcp-1.5' not found in /tmp/tmp.KP8o5VJNMp/pipeline.yml

Results: 7 passed, 12 failed

Those repeated failures happen before any trigger YAML is written, which is why every downstream assertion looking for steps:, DRY_RUN, PACKAGE_NAME, etc. fails.

Verification

• I verified the generated inventory fixture shape is accepted by Mike Farah yq with yq -e '.backports', so the fixture itself is valid.
• I did not run the full PR test suite locally because this checkout is on main, not the PR head.

Follow-up

If the Buildkite image happens to contain some other yq implementation, calling with_yq still avoids ambiguity by forcing the repo-pinned YQ_VERSION onto PATH.

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 89f7d03

History

• 💚 Build #44740 succeeded c8f110a
• 💚 Build #44739 succeeded 6deb3e8
• 💚 Build #44738 succeeded a813501
• 💚 Build #44736 succeeded 5b262b4
• 💔 Build #44733 failed f6a7168
• 💔 Build #44732 failed 7b1948b

cc @mrodm