Skip to content

Commit 12ad2ce

Browse files
committed
ci: add weekly automated codegen workflow
Adds .github/workflows/codegen.yml which runs every Monday at 07:00 UTC (and on workflow_dispatch) to regenerate the auto-generated ES and Cloud API bindings from elastic/elastic-client-generator-js. When the diff is non-empty it opens a PR via peter-evans/create-pull-request. Introduces two package.json scripts, codegen:es and codegen:cloud, which wrap a small orchestrator (scripts/codegen.mjs) that clones the generator, runs its zod / cli-es / cli-cloud targets, and copies the outputs into src/es/apis, src/es/apis/schemas and src/cloud/apis. The orchestrator is a plain Node script with no new runtime or dev dependencies. Refs: #79
1 parent d2b4a43 commit 12ad2ce

4 files changed

Lines changed: 279 additions & 0 deletions

File tree

.github/workflows/codegen.yml

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
name: Weekly Codegen
2+
3+
# Regenerates the auto-generated ES and Cloud API bindings against the latest
4+
# upstream `elastic/elastic-client-generator-js` output and, when the diff is
5+
# non-empty, opens a PR. See https://github.com/elastic/cli/issues/79.
6+
on:
7+
schedule:
8+
# Monday 07:00 UTC — matches the cadence of the wider client-codegen suite
9+
# run by elastic/devtools-team (which fires one hour earlier).
10+
- cron: "0 7 * * 1"
11+
workflow_dispatch:
12+
13+
permissions:
14+
contents: write
15+
pull-requests: write
16+
17+
concurrency:
18+
group: ${{ github.workflow }}
19+
cancel-in-progress: false
20+
21+
jobs:
22+
codegen:
23+
name: Regenerate API bindings
24+
runs-on: ubuntu-latest
25+
steps:
26+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
27+
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
28+
with:
29+
node-version: 24
30+
- name: Install repo dependencies
31+
run: npm ci
32+
33+
- name: Prepare generator checkout (shared between codegen steps)
34+
id: generator
35+
shell: bash
36+
run: |
37+
dir="$RUNNER_TEMP/elastic-client-generator-js"
38+
git clone --depth 1 --branch main https://github.com/elastic/elastic-client-generator-js.git "$dir"
39+
(cd "$dir" && npm install --no-audit --no-fund)
40+
echo "dir=$dir" >> "$GITHUB_OUTPUT"
41+
42+
- name: Regenerate ES bindings
43+
env:
44+
CODEGEN_GENERATOR_DIR: ${{ steps.generator.outputs.dir }}
45+
run: npm run codegen:es
46+
- name: Regenerate Cloud bindings
47+
env:
48+
CODEGEN_GENERATOR_DIR: ${{ steps.generator.outputs.dir }}
49+
run: npm run codegen:cloud
50+
# TODO(#79): add a `codegen:kibana` step here once the Kibana target
51+
# lands in elastic/elastic-client-generator-js. The Kibana bindings
52+
# tracking issue (elastic/cli#72) will gate this.
53+
54+
- name: Build
55+
run: npm run build
56+
- name: Unit tests
57+
run: npm run test:unit
58+
59+
- name: Detect generated diff
60+
id: diff
61+
shell: bash
62+
run: |
63+
if git diff --quiet -- src/es src/cloud; then
64+
echo "changed=false" >> "$GITHUB_OUTPUT"
65+
echo "No generated changes — nothing to PR."
66+
else
67+
echo "changed=true" >> "$GITHUB_OUTPUT"
68+
git --no-pager diff --stat -- src/es src/cloud
69+
fi
70+
71+
# NOTE: Using GITHUB_TOKEN means the resulting PR will NOT trigger further
72+
# workflow runs (including `ci.yml`). To get CI on weekly codegen PRs,
73+
# replace `GITHUB_TOKEN` below with a fine-grained PAT stored as
74+
# `CODEGEN_PR_TOKEN`. See:
75+
# https://docs.github.com/en/actions/security-for-github-actions/security-guides/automatic-token-authentication#using-the-github_token-in-a-workflow
76+
- name: Create Pull Request
77+
if: steps.diff.outputs.changed == 'true'
78+
id: cpr
79+
uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
80+
with:
81+
token: ${{ secrets.CODEGEN_PR_TOKEN || secrets.GITHUB_TOKEN }}
82+
branch: chore/weekly-codegen
83+
delete-branch: true
84+
base: main
85+
commit-message: "chore(codegen): weekly update of generated API bindings"
86+
title: "chore(codegen): weekly update of generated API bindings"
87+
body: |
88+
Automated weekly regeneration of the ES and Cloud API bindings
89+
against the latest `elastic/elastic-client-generator-js` output.
90+
91+
- `src/es/apis/**` and `src/es/apis.ts` — produced by `npm run cli-es`
92+
- `src/es/apis/schemas/**` — produced by `npm run zod`
93+
- `src/cloud/apis/**` and `src/cloud/apis.ts` — produced by `npm run cli-cloud`
94+
95+
Run locally with `npm run codegen:es` / `npm run codegen:cloud`.
96+
97+
Refs: #79
98+
committer: "elastic-machine <elasticmachine@users.noreply.github.com>"
99+
author: "elastic-machine <elasticmachine@users.noreply.github.com>"
100+
add-paths: |
101+
src/es
102+
src/cloud
103+
104+
- name: PR summary
105+
if: steps.cpr.outputs.pull-request-url
106+
shell: bash
107+
run: |
108+
echo "Opened: ${{ steps.cpr.outputs.pull-request-url }}" >> "$GITHUB_STEP_SUMMARY"

CONTRIBUTING.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,17 @@ Thanks for your interest in contributing! We love receiving contributions from o
5454
| `npm run test:lint -- --fix` | Fix linter errors automatically |
5555
| `npm run test:megalinter` | Run MegaLinter locally (requires Docker) |
5656

57+
## Regenerating API bindings
58+
59+
The files under `src/es/apis/`, `src/es/apis/schemas/` and `src/cloud/apis/` are auto-generated from [`elastic/elastic-client-generator-js`](https://github.com/elastic/elastic-client-generator-js) by CI (see `.github/workflows/codegen.yml`, which opens a PR every Monday). To reproduce locally:
60+
61+
```bash
62+
npm run codegen:es # regenerates ES namespaces + Zod schemas
63+
npm run codegen:cloud # regenerates Cloud namespaces
64+
```
65+
66+
Both scripts clone the generator into a temp directory and run its `npm run zod` / `npm run cli-es` / `npm run cli-cloud` targets. Set `CODEGEN_GENERATOR_DIR` to point at an existing checkout if you want to iterate on generator changes without cloning each time.
67+
5768
## Linting
5869

5970
This project uses [MegaLinter](https://megalinter.io/) for comprehensive linting across TypeScript, YAML, GitHub Actions, Dockerfiles, and more. It also scans for secrets, copy-paste, and vulnerabilities.

package.json

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,8 @@
1212
"test:unit": "node --import tsx/esm --test --experimental-test-coverage --test-coverage-lines=90 --test-coverage-branches=90 --test-coverage-functions=90 --test-coverage-include='src/**/*.ts' --test-coverage-exclude='src/es/apis/schemas/**' --test-coverage-exclude='src/cloud/apis/**' --test-coverage-exclude='src/es/apis.ts' --test-coverage-exclude='src/cloud/apis.ts' --test-coverage-exclude='src/cloud/serverless-apis.ts' --test-coverage-exclude='node_modules/**'",
1313
"test:lint": "eslint src",
1414
"codegen:functional": "npx tsx codegen/functional/index.ts --tests-dir ../elasticsearch-clients-tests/tests",
15+
"codegen:es": "node scripts/codegen.mjs es",
16+
"codegen:cloud": "node scripts/codegen.mjs cloud",
1517
"test:functional:es": "bash test/functional/es/run.sh",
1618
"test:functional:cloud": "bash test/functional/cloud/smoke.sh",
1719
"test:license": "license-checker --production",

scripts/codegen.mjs

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
#!/usr/bin/env node
2+
/*
3+
* Copyright Elasticsearch B.V. and contributors
4+
* SPDX-License-Identifier: Apache-2.0
5+
*/
6+
7+
/**
8+
* Orchestrates regeneration of the ES and Cloud API bindings against the
9+
* upstream elastic/elastic-client-generator-js repo.
10+
*
11+
* Usage (from package.json scripts):
12+
* node scripts/codegen.mjs es
13+
* node scripts/codegen.mjs cloud
14+
*
15+
* Environment:
16+
* CODEGEN_GENERATOR_DIR Reuse an existing generator checkout (absolute
17+
* path). When unset, the script clones the repo
18+
* into a fresh temporary directory and installs
19+
* its dependencies.
20+
* CODEGEN_GENERATOR_REF Ref/branch/tag to clone (default: main).
21+
* CODEGEN_ES_VERSION Elasticsearch schema version (default: main).
22+
*
23+
* Design rationale:
24+
* The upstream generator's npm scripts (`npm run zod`, `npm run cli-es`,
25+
* `npm run cli-cloud`) share a single `output/` directory that each run
26+
* wipes via `npm run clean`. This script runs them sequentially and copies
27+
* the relevant files out of `output/` between runs, mapping them onto the
28+
* repo layout at `src/es/apis/`, `src/es/apis/schemas/` and
29+
* `src/cloud/apis/`.
30+
*/
31+
32+
import { spawnSync } from 'node:child_process'
33+
import { cpSync, existsSync, mkdirSync, mkdtempSync, readdirSync, rmSync, statSync } from 'node:fs'
34+
import { tmpdir } from 'node:os'
35+
import { dirname, join, resolve } from 'node:path'
36+
import { fileURLToPath } from 'node:url'
37+
38+
const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..')
39+
const GENERATOR_REPO = 'https://github.com/elastic/elastic-client-generator-js.git'
40+
41+
function run (command, args, opts = {}) {
42+
const result = spawnSync(command, args, { stdio: 'inherit', shell: false, ...opts })
43+
if (result.status !== 0) {
44+
const cwd = opts.cwd ?? process.cwd()
45+
throw new Error(`Command failed (${command} ${args.join(' ')}) in ${cwd}: exit ${result.status ?? 'signal'}`)
46+
}
47+
}
48+
49+
/**
50+
* Ensure a checkout of the generator exists and has its dependencies installed.
51+
* Returns an absolute path to it. When CODEGEN_GENERATOR_DIR is set the caller
52+
* is responsible for the checkout; this function only verifies it exists and
53+
* skips install (the caller should have installed). Otherwise a fresh temp
54+
* clone is produced and its dependencies installed via npm.
55+
*/
56+
function ensureGenerator () {
57+
const existing = process.env.CODEGEN_GENERATOR_DIR
58+
if (existing != null && existing.length > 0) {
59+
if (!existsSync(existing)) {
60+
throw new Error(`CODEGEN_GENERATOR_DIR="${existing}" does not exist`)
61+
}
62+
console.log(`[codegen] Using existing generator checkout: ${existing}`)
63+
return existing
64+
}
65+
66+
const ref = process.env.CODEGEN_GENERATOR_REF ?? 'main'
67+
const dir = mkdtempSync(join(tmpdir(), 'elastic-client-generator-'))
68+
console.log(`[codegen] Cloning ${GENERATOR_REPO} @ ${ref} -> ${dir}`)
69+
run('git', ['clone', '--depth', '1', '--branch', ref, GENERATOR_REPO, dir])
70+
console.log('[codegen] Installing generator dependencies (npm install)')
71+
run('npm', ['install', '--no-audit', '--no-fund'], { cwd: dir, shell: process.platform === 'win32' })
72+
return dir
73+
}
74+
75+
/** Remove every file in `dir` matching `predicate`. Non-recursive. */
76+
function clearDir (dir, predicate = () => true) {
77+
if (!existsSync(dir)) {
78+
mkdirSync(dir, { recursive: true })
79+
return
80+
}
81+
for (const entry of readdirSync(dir)) {
82+
if (!predicate(entry)) continue
83+
const full = join(dir, entry)
84+
if (statSync(full).isDirectory()) continue // leave sub-directories alone
85+
rmSync(full, { force: true })
86+
}
87+
}
88+
89+
/** Copy every *.ts file from `src` (non-recursive) into `dest`. */
90+
function copyTsFiles (src, dest) {
91+
if (!existsSync(src)) throw new Error(`Expected generator output at ${src}`)
92+
mkdirSync(dest, { recursive: true })
93+
for (const entry of readdirSync(src)) {
94+
if (!entry.endsWith('.ts')) continue
95+
cpSync(join(src, entry), join(dest, entry))
96+
}
97+
}
98+
99+
function generateEs (generatorDir) {
100+
const version = process.env.CODEGEN_ES_VERSION ?? 'main'
101+
const output = join(generatorDir, 'output')
102+
103+
console.log(`[codegen] Step 1/2: Zod schemas (version: ${version})`)
104+
run('npm', ['run', 'zod', '--', '--version', version], { cwd: generatorDir, shell: process.platform === 'win32' })
105+
const schemasDest = join(REPO_ROOT, 'src', 'es', 'apis', 'schemas')
106+
clearDir(schemasDest, (entry) => entry.endsWith('.ts'))
107+
copyTsFiles(output, schemasDest)
108+
console.log(`[codegen] wrote schemas -> ${schemasDest}`)
109+
110+
console.log(`[codegen] Step 2/2: ES API namespace files (version: ${version})`)
111+
run('npm', ['run', 'cli-es', '--', '--version', version], { cwd: generatorDir, shell: process.platform === 'win32' })
112+
const apisDest = join(REPO_ROOT, 'src', 'es', 'apis')
113+
clearDir(apisDest, (entry) => entry.endsWith('.ts'))
114+
copyTsFiles(join(output, 'es', 'apis'), apisDest)
115+
// Generator emits the barrel as `output/es/index.ts`; the repo expects it
116+
// as `src/es/apis.ts` next to the `apis/` directory.
117+
const barrelSrc = join(output, 'es', 'index.ts')
118+
const barrelDest = join(REPO_ROOT, 'src', 'es', 'apis.ts')
119+
if (!existsSync(barrelSrc)) throw new Error(`Missing barrel: ${barrelSrc}`)
120+
cpSync(barrelSrc, barrelDest)
121+
console.log(`[codegen] wrote APIs -> ${apisDest}`)
122+
console.log(`[codegen] wrote barrel -> ${barrelDest}`)
123+
}
124+
125+
function generateCloud (generatorDir) {
126+
const output = join(generatorDir, 'output')
127+
128+
console.log('[codegen] Cloud API namespace files')
129+
// `npm run cli-cloud` does not call clean itself, so wipe any leftover
130+
// output from a previous step before running the generator.
131+
rmSync(output, { recursive: true, force: true })
132+
run('npm', ['run', 'cli-cloud'], { cwd: generatorDir, shell: process.platform === 'win32' })
133+
const apisDest = join(REPO_ROOT, 'src', 'cloud', 'apis')
134+
clearDir(apisDest, (entry) => entry.endsWith('.ts'))
135+
copyTsFiles(join(output, 'cloud', 'apis'), apisDest)
136+
const barrelSrc = join(output, 'cloud', 'apis.ts')
137+
const barrelDest = join(REPO_ROOT, 'src', 'cloud', 'apis.ts')
138+
if (!existsSync(barrelSrc)) throw new Error(`Missing barrel: ${barrelSrc}`)
139+
cpSync(barrelSrc, barrelDest)
140+
console.log(`[codegen] wrote APIs -> ${apisDest}`)
141+
console.log(`[codegen] wrote barrel -> ${barrelDest}`)
142+
}
143+
144+
const target = process.argv[2]
145+
if (target !== 'es' && target !== 'cloud') {
146+
console.error('Usage: node scripts/codegen.mjs <es|cloud>')
147+
process.exit(1)
148+
}
149+
150+
try {
151+
const generatorDir = ensureGenerator()
152+
if (target === 'es') generateEs(generatorDir)
153+
else generateCloud(generatorDir)
154+
console.log(`[codegen] Done (${target}).`)
155+
} catch (err) {
156+
console.error(`[codegen] Failed: ${err instanceof Error ? err.message : String(err)}`)
157+
process.exit(1)
158+
}

0 commit comments

Comments
 (0)