From 9066a4ff5c138f926a007f7b17c4b3ffefadf164 Mon Sep 17 00:00:00 2001 From: PBNZ Date: Sat, 4 Jul 2026 00:20:16 +1200 Subject: [PATCH 01/10] feat: add power-platform-connectors type to /new-repo MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A new repo type that turns a committed Postman collection into OpenAPI 2.0 custom-connector definitions for Microsoft Power Platform (2.0 + <1MB are hard requirements — verified from MS Learn). - Pinned-Docker generator (Node 18 + postman-to-openapi + api-spec-converter + swagger-cli): converts Postman -> OpenAPI 3.0 -> Swagger 2.0, normalises the lossy output to valid 2.0 (resolves {{vars}} for host/basePath, derives securityDefinitions from the Postman auth, backfills response descriptions, adds missing path params), splits per top-level folder ONLY when a single def would hit 1MB, and self-validates every output (fails on invalid or oversize). - CI regenerates + validates on source change; a scheduled sync workflow detects upstream changes (hash-diff), regenerates, validates inline (the bot PR won't trigger CI), and opens a PR. Committed source -> builds with just Docker, no Postman account; account-free sync when the source has a public URL. - Core + Public tier (no Published — connectors are imported manually). Proven end-to-end against a real public Postman collection. Registered in the SKILL enum, the standard's Types table, the testing matrix (+ backfilled the missing docker-compose row), the README, and docs/adr/0003. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01DXtxaAQHcsDThJ6BaqwXES --- CHANGELOG.md | 6 + README.md | 5 +- .../0003-power-platform-connectors-type.md | 55 ++++++ plugins/repokit/skills/new-repo/SKILL.md | 6 +- .../core/.postman/manifest.json | 5 + .../power-platform-connectors/core/Dockerfile | 12 ++ .../core/README.md.tmpl | 62 ++++++ .../core/connectors.config.json | 5 + .../core/connectors/.gitkeep | 0 .../core/scripts/generate.mjs | 184 ++++++++++++++++++ .../core/source/.gitkeep | 0 .../public/.github/workflows/ci.yml.tmpl | 23 +++ .../public/.github/workflows/sync.yml.tmpl | 83 ++++++++ .../repo-standard/standard/testing-matrix.md | 2 + .../repo-standard/standard/the-standard.md | 1 + 15 files changed, 444 insertions(+), 5 deletions(-) create mode 100644 docs/adr/0003-power-platform-connectors-type.md create mode 100644 plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/.postman/manifest.json create mode 100644 plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/Dockerfile create mode 100644 plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/README.md.tmpl create mode 100644 plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/connectors.config.json create mode 100644 plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/connectors/.gitkeep create mode 100644 plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs create mode 100644 plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/source/.gitkeep create mode 100644 plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/ci.yml.tmpl create mode 100644 plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl diff --git a/CHANGELOG.md b/CHANGELOG.md index 919bf28..ebfcc5d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - A `docker-compose` repo type for `/new-repo`: a minimal `compose.yaml` (named volumes for data, committed config, `.env` secrets), `.env.example`, `.dockerignore`, a README with a workflow diagram, and a `docker compose config -q` validation CI (Public tier). +- A `power-platform-connectors` repo type for `/new-repo`: turns a committed Postman collection + into **OpenAPI 2.0** custom-connector definitions for Microsoft Power Platform. A pinned-Docker + generator (`postman-to-openapi` + `api-spec-converter`, normalised to valid Swagger 2.0) splits + per top-level folder **only when a single definition would exceed the 1 MB limit**, self-validates + every output, and a scheduled sync workflow opens a PR when the upstream collection changes. The + collection is committed, so the repo builds with just Docker — no Postman account. - The `repo-standard` skill now tells agents to check the remote after a push/PR — CI/Actions status and GitHub Copilot / reviewer feedback — before calling work done. ### Changed diff --git a/README.md b/README.md index ee0d172..9ed06b6 100644 --- a/README.md +++ b/README.md @@ -90,8 +90,9 @@ make the first commit. Nothing is pushed or published — you do that when you'r ## Profiles -- **Type — what the repo *is*:** `powershell-module` and `docker-compose` (built out), plus `skill-plugin`, - `collection`, `mcp-server`, `app-ts`, `app-python`, `script-collection` (stubs, filled as needed). +- **Type — what the repo *is*:** `powershell-module`, `docker-compose`, and + `power-platform-connectors` (built out), plus `skill-plugin`, `collection`, `mcp-server`, + `app-ts`, `app-python`, `script-collection` (stubs, filled as needed). - **Tier — ceremony, set by visibility:** **Core** (every repo), **+Public**, **+Published**. See [`the standard`](plugins/repokit/skills/repo-standard/standard/the-standard.md) for the full diff --git a/docs/adr/0003-power-platform-connectors-type.md b/docs/adr/0003-power-platform-connectors-type.md new file mode 100644 index 0000000..bc78caf --- /dev/null +++ b/docs/adr/0003-power-platform-connectors-type.md @@ -0,0 +1,55 @@ +# ADR-0003: `power-platform-connectors` repo type + +- **Status:** accepted +- **Date:** 2026-07-04 + +## Context + +We needed a repo **type** that turns a Postman collection into OpenAPI definitions ready to import +as **Microsoft Power Platform custom connectors**. Verified hard constraints from Microsoft Learn +(*Create a custom connector from an OpenAPI definition*): the definition must be **OpenAPI 2.0 / +Swagger** ("OpenAPI 3.0 format is not supported") and **< 1 MB**, `.json` or `.yaml`, with a single +top security definition (client-credentials OAuth is rejected). + +## Decision + +- **Converter (free, pinned in Docker):** `postman-to-openapi@3.0.1` outputs OpenAPI **3.0** only, + so we downconvert with `api-spec-converter@2.12.0` (`--from=openapi_3 --to=swagger_2`). Both are + unmaintained (2023 / 2021; the latter depends on the deprecated `request`) and do **not** run on + current Node — the image is pinned to **Node 18**. Do not bump without re-testing. APIMatic (paid, + direct Postman→2.0) is the documented escape hatch, not built. +- **The downconvert is lossy, so `generate.mjs` normalises the output to valid Swagger 2.0** — this + was proven empirically (a real public collection failed validation four different ways before the + fixups): (1) **pre-resolve collection `{{variables}}`** so `host`/`basePath`/`schemes` are real + instead of `%7B%7Bbaseurl%7D%7D`; (2) **derive `securityDefinitions` from the Postman `auth`** + block (p2o mis-maps apikey to an invalid `type:http` and drops the header name); (3) **backfill a + `description` on every response** (required in 2.0; p2o only sets it from the Postman `status`); + (4) **add missing path parameters and leading `/`** on path keys. The generator then + **self-validates** each output with `swagger-cli` and asserts **< 1 MB**, exiting non-zero on any + failure. +- **Split policy:** convert the whole collection to one definition; ship one file if it's < 1 MB; + otherwise split **per top-level folder** (each carrying the collection `variable`/`auth`); if a + single folder is still ≥ 1 MB after stripping example fields, **flag it** rather than ship an + un-importable file. +- **Self-containment / source:** the collection is **committed** (`source/collection.json`), so the + repo builds with just Docker — no Postman account. **Sync** fetches a **configurable `sourceUrl`**: + a public URL (GitHub raw / vendor site) is **account-free**; a Postman-platform-only collection + needs the maintainer's optional `POSTMAN_API_KEY` secret (never needed by cloners). Postman has no + reliable anonymous fetch for a collection you don't own (public JSON links are deprecated). +- **Change detection = scheduled workflow + hash:** `sync.yml` (daily cron + dispatch) canonicalises + the fetched collection, SHA-256s it, compares to `.postman/manifest.json`, and on a change updates + the snapshot, regenerates, **validates inline**, and opens a PR. Validation is inline because a PR + opened by the default `GITHUB_TOKEN` does not trigger `ci.yml`. This needs + `permissions: pull-requests: write` and the repo setting *Allow GitHub Actions to create and + approve pull requests*. +- **Tier: Core + Public, no Published** (like `docker-compose`) — "publishing" is manually importing + a definition into Power Platform; there is no registry step to automate. + +## Consequences + +- The conversion is lossy — the pipeline validates and opens a **PR to review**; it never + auto-imports. Richer collections (saved example responses, one clear auth scheme) produce better + connectors. +- The toolchain is stale but pinned in Docker; if a future Node breaks it, the pin holds. +- Auto-sync is account-free for public-URL sources; Postman-platform-only collections need the + maintainer's optional key. diff --git a/plugins/repokit/skills/new-repo/SKILL.md b/plugins/repokit/skills/new-repo/SKILL.md index dc7dc25..ab6bd40 100644 --- a/plugins/repokit/skills/new-repo/SKILL.md +++ b/plugins/repokit/skills/new-repo/SKILL.md @@ -26,14 +26,14 @@ Gather these from the user's arguments / message, else ask — keep it to load-b - **name** — the repo / directory name (kebab-case). - **description** — one line. -- **type** — one of: `powershell-module`, `docker-compose`, `skill-plugin`, `collection`, `mcp-server`, `app-ts`, - `app-python`, `script-collection`. +- **type** — one of: `powershell-module`, `docker-compose`, `power-platform-connectors`, + `skill-plugin`, `collection`, `mcp-server`, `app-ts`, `app-python`, `script-collection`. - **visibility** — `private` (= Core tier), `public` (= +Public), or `published` (= +Published). - **author** — default `Peter Braun` (`PBNZ`). - **license** — default `Apache-2.0`. - For `powershell-module` only: **ModuleName** (PascalCase, e.g. `MyModule`). -If the chosen type is a **stub** (anything other than `powershell-module` or `docker-compose`), tell the user so: the +If the chosen type is a **stub** (anything other than `powershell-module`, `docker-compose`, or `power-platform-connectors`), tell the user so: the Core/Public/Published files get stamped, but there's no type-specific structure yet. Confirm they want to continue. diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/.postman/manifest.json b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/.postman/manifest.json new file mode 100644 index 0000000..2467e98 --- /dev/null +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/.postman/manifest.json @@ -0,0 +1,5 @@ +{ + "sourceUrl": "", + "sha256": "", + "updatedAt": "" +} diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/Dockerfile b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/Dockerfile new file mode 100644 index 0000000..e50a6a3 --- /dev/null +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/Dockerfile @@ -0,0 +1,12 @@ +# Pinned toolchain for generating Power Platform custom-connector definitions. +# +# Node is pinned to 18 on purpose: the converters are unmaintained +# (postman-to-openapi 2023, api-spec-converter 2021 — the latter depends on the +# deprecated `request` module) and do NOT run on current Node. Do not bump the +# Node line or the tool versions without re-testing the whole pipeline. +FROM node:18-bullseye-slim + +RUN npm install -g postman-to-openapi@3.0.1 api-spec-converter@2.12.0 @apidevtools/swagger-cli@4.0.4 + +WORKDIR /work +# The repo is mounted at /work at run time; run: node scripts/generate.mjs diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/README.md.tmpl b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/README.md.tmpl new file mode 100644 index 0000000..b6d5cf1 --- /dev/null +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/README.md.tmpl @@ -0,0 +1,62 @@ +# {{name}} + +{{description}} + +OpenAPI **2.0** connector definitions for **Microsoft Power Platform** custom connectors, +generated from a Postman collection. Each definition is valid Swagger 2.0 and kept **under 1 MB** +(Power Platform's limits — 3.0 is not supported). The collection is **committed to this repo**, so +anyone can regenerate the connectors with just Docker — no Postman account needed. + +## How it works + +```mermaid +flowchart LR + Src["source/collection.json
(committed Postman collection)"] --> Gen["generate.mjs, in Docker"] + Gen --> Q{"whole def under 1 MB?"} + Q -->|yes| One["one connectors/*.swagger.json"] + Q -->|no| Split["split per top-level folder"] + One --> Val["validate: Swagger 2.0 + under 1 MB"] + Split --> Val + Val --> Imp["import into Power Platform"] +``` + +## Use it + +1. Export your Postman collection (v2.1) and save it as **`source/collection.json`**. +2. Set `sourceUrl` in **`connectors.config.json`** — a public URL to the collection JSON (a GitHub + raw file or the vendor's site), or the Postman API endpoint if the collection lives only in + Postman. +3. Generate the definitions (one Docker command — no local Node needed): + + ```sh + docker build -t {{name}}-gen . + docker run --rm -v "${PWD}:/work" {{name}}-gen node scripts/generate.mjs + ``` + + It writes `connectors/*.swagger.json` — **one file if the whole collection fits under 1 MB, + otherwise one per top-level folder**. Every file is validated as Swagger 2.0 and checked < 1 MB; + the run fails if any definition is invalid or oversize. +4. In Power Platform (Power Automate / Power Apps / Logic Apps): **New custom connector → Import an + OpenAPI file** → pick a `connectors/*.swagger.json`. + +## Keeping it up to date + +- **CI** (`.github/workflows/ci.yml`) rebuilds + regenerates + validates on every change to + `source/`, `scripts/`, the `Dockerfile`, or the config. +- **Auto-sync** (`.github/workflows/sync.yml`) runs daily: it fetches `sourceUrl`, and if the + upstream collection changed, it updates `source/collection.json`, regenerates, validates, and + opens a **PR** for you to review. Account-free when `sourceUrl` is a public URL. If the collection + lives only in Postman, add a `POSTMAN_API_KEY` repo secret (only the maintainer needs it — never + cloners), and enable *Settings → Actions → Allow GitHub Actions to create and approve pull requests*. + +## Heads-up — review before importing + +The Postman → Swagger 2.0 conversion is **lossy** (it drops `oneOf/anyOf`, `nullable`, and derives +response schemas from saved example responses). The pipeline fixes the common breakages +(host/basePath, security definitions, response descriptions) and validates every output, but +**review the connector in the PR before importing**. Richer collections — with saved example +responses and a single clear auth scheme — produce better connectors. Power Platform also picks the +single top security definition and rejects OAuth client-credentials. + +See [`AGENTS.md`](AGENTS.md) for the START-HERE map. Follows the +[RepoKit](https://github.com/PBNZ/repo-kit) standard. diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/connectors.config.json b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/connectors.config.json new file mode 100644 index 0000000..e09328e --- /dev/null +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/connectors.config.json @@ -0,0 +1,5 @@ +{ + "sourceUrl": "", + "sizeLimitBytes": 1048576, + "output": "connectors" +} diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/connectors/.gitkeep b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/connectors/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs new file mode 100644 index 0000000..6c01fbc --- /dev/null +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs @@ -0,0 +1,184 @@ +// Generate Power Platform custom-connector definitions (OpenAPI 2.0 / Swagger) +// from the committed Postman collection in source/. +// +// Postman collection -> resolve {{vars}} -> postman-to-openapi (3.0) +// -> api-spec-converter (Swagger 2.0) -> fix securityDefinitions from the +// Postman auth -> measure -> split per top-level folder ONLY if a single +// definition would be >= the size limit. +// +// Runs inside the pinned Docker image (p2o + api-spec-converter on PATH). + +import { readFileSync, writeFileSync, mkdirSync, readdirSync, rmSync } from 'node:fs'; +import { execFileSync } from 'node:child_process'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; + +const cfg = JSON.parse(readFileSync('connectors.config.json', 'utf8')); +const LIMIT = cfg.sizeLimitBytes ?? 1048576; +const OUT = cfg.output ?? 'connectors'; +const work = tmpdir(); +const warnings = []; + +// --- locate + load the committed collection --- +const srcName = readdirSync('source').find((f) => f.endsWith('.json')); +if (!srcName) { + console.log('source/ has no *.json collection yet — export your Postman collection there. Nothing to do.'); + process.exit(0); +} +const rawObj = JSON.parse(readFileSync(join('source', srcName), 'utf8')); +const collection = rawObj.collection ?? rawObj; // unwrap Postman API {"collection":…} +const rootVars = collection.variable || []; +const rootAuth = collection.auth || null; + +// --- helpers --- +const sizeOf = (o) => Buffer.byteLength(JSON.stringify(o, null, 2)); +const slug = (s) => (s || 'connector').toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/(^-|-$)/g, '') || 'connector'; +const kv = (arr, k) => arr?.find((e) => e.key === k)?.value; + +// Pre-resolve collection variables (concrete {{vars}} with a value) so host/basePath come out +// real instead of "%7B%7Bbaseurl%7D%7D". Variables with no value are left untouched. +function resolveVars(obj, vars) { + let s = JSON.stringify(obj); + for (const v of vars) if (v && v.key && v.value) s = s.split(`{{${v.key}}}`).join(v.value); + return JSON.parse(s); +} + +// Map a Postman auth block to a valid Swagger 2.0 security definition. p2o mis-maps these +// (e.g. apikey -> {type:http, scheme:apikey}, which is invalid 2.0 AND drops the header name), +// so we build them from the source collection instead. +function pmAuthToSwagger(auth) { + if (!auth || !auth.type) return null; + switch (auth.type) { + case 'apikey': { + const a = auth.apikey || []; + const loc = (kv(a, 'in') || 'header').toLowerCase() === 'query' ? 'query' : 'header'; + return { name: 'apiKeyAuth', def: { type: 'apiKey', name: kv(a, 'key') || 'Authorization', in: loc } }; + } + case 'bearer': + return { name: 'bearerAuth', def: { type: 'apiKey', name: 'Authorization', in: 'header' } }; + case 'basic': + return { name: 'basicAuth', def: { type: 'basic' } }; + case 'oauth2': { + const a = auth.oauth2 || []; + return { name: 'oauth2Auth', def: { type: 'oauth2', flow: 'accessCode', authorizationUrl: kv(a, 'authUrl') || '', tokenUrl: kv(a, 'accessTokenUrl') || '', scopes: {} } }; + } + default: + return null; + } +} + +// Swagger 2.0 requires every response to have a `description`. p2o only sets it from the Postman +// response's `status` (reason phrase); collections that omit `status` yield a description-less +// (invalid) response. Backfill a sensible one so the output always validates. +const REASON = { 200: 'OK', 201: 'Created', 202: 'Accepted', 203: 'Non-Authoritative Information', 204: 'No Content', 206: 'Partial Content', 301: 'Moved Permanently', 302: 'Found', 304: 'Not Modified', 400: 'Bad Request', 401: 'Unauthorized', 403: 'Forbidden', 404: 'Not Found', 405: 'Method Not Allowed', 409: 'Conflict', 422: 'Unprocessable Entity', 429: 'Too Many Requests', 500: 'Internal Server Error', 502: 'Bad Gateway', 503: 'Service Unavailable' }; +function fixResponses(sw) { + for (const path of Object.values(sw.paths || {})) + for (const op of Object.values(path)) + if (op && typeof op === 'object' && op.responses) + for (const [code, r] of Object.entries(op.responses)) + if (r && typeof r === 'object' && !r.description) r.description = REASON[code] || 'Response'; +} + +// Swagger 2.0 requires (a) every path key to start with "/", and (b) every "{token}" in a path to +// have a matching path parameter. p2o sometimes emits neither. Backfill both. +function fixPaths(sw) { + if (!sw.paths) return; + const fixed = {}; + for (const [key, item] of Object.entries(sw.paths)) { + const pathKey = key.startsWith('/') ? key : '/' + key; + const tokens = [...pathKey.matchAll(/\{([^}]+)\}/g)].map((m) => m[1]); + if (tokens.length && item && typeof item === 'object') { + for (const op of Object.values(item)) { + if (op && typeof op === 'object' && op.responses) { + op.parameters = op.parameters || []; + for (const t of tokens) + if (!op.parameters.some((p) => p.in === 'path' && p.name === t)) + op.parameters.push({ name: t, in: 'path', required: true, type: 'string' }); + } + } + } + fixed[pathKey] = item; + } + sw.paths = fixed; +} + +// Replace whatever the converter produced with a correct security definition derived from the +// Postman auth (Power Platform picks the single top securityDefinition, so we emit exactly one). +function fixSecurity(sw, auth) { + const s = pmAuthToSwagger(auth); + if (s) { + sw.securityDefinitions = { [s.name]: s.def }; + sw.security = [{ [s.name]: [] }]; + } else { + if (auth?.type) warnings.push(`auth type "${auth.type}" not mapped — add security manually in the connector`); + delete sw.securityDefinitions; // never ship an invalid securityDefinitions block + delete sw.security; + } +} + +// One Postman (sub)collection -> validated Swagger 2.0 object. +function convert(coll, tag, effectiveAuth) { + const pin = join(work, `${tag}.postman.json`); + const oas = join(work, `${tag}.oas3.yml`); + writeFileSync(pin, JSON.stringify(resolveVars(coll, coll.variable?.length ? coll.variable : rootVars))); + execFileSync('p2o', [pin, '-f', oas], { stdio: 'pipe' }); + const raw = execFileSync('api-spec-converter', ['--from=openapi_3', '--to=swagger_2', '--syntax=json', oas], { stdio: 'pipe' }); + const sw = JSON.parse(raw.toString()); + fixPaths(sw); + fixResponses(sw); + fixSecurity(sw, effectiveAuth); + return sw; +} + +// Strip verbose example fields (shrinks size; also improves Power Platform compatibility). +function strip(o) { + if (Array.isArray(o)) o.forEach(strip); + else if (o && typeof o === 'object') { delete o.example; delete o.examples; for (const k of Object.keys(o)) strip(o[k]); } + return o; +} + +// --- fresh output dir --- +mkdirSync(OUT, { recursive: true }); +for (const f of readdirSync(OUT)) if (f.endsWith('.swagger.json')) rmSync(join(OUT, f)); + +const written = []; +function emit(name, sw) { + let bytes = sizeOf(sw); + if (bytes >= LIMIT) { strip(sw); bytes = sizeOf(sw); } // last-ditch shrink for an oversize def + const file = join(OUT, `${slug(name)}.swagger.json`); + writeFileSync(file, JSON.stringify(sw, null, 2)); + let valid = true; + try { execFileSync('swagger-cli', ['validate', file], { stdio: 'pipe' }); } catch { valid = false; } + written.push({ file, bytes, over: bytes >= LIMIT, valid }); +} + +// --- convert whole; split only if it wouldn't fit --- +const whole = convert(collection, 'whole', rootAuth); +if (sizeOf(whole) < LIMIT) { + emit(collection.info?.name || srcName.replace(/\.json$/, ''), whole); +} else { + const roots = collection.item || []; + for (const folder of roots.filter((it) => it.item)) { + const sub = { + info: { ...collection.info, name: `${collection.info?.name || ''} - ${folder.name}`.trim() }, + variable: rootVars, + item: folder.item, + }; + emit(folder.name, convert(sub, slug(folder.name), folder.auth || rootAuth)); + } + const loose = roots.filter((it) => it.request); + if (loose.length) { + const sub = { info: { ...collection.info, name: `${collection.info?.name || ''} - misc` }, variable: rootVars, item: loose }; + emit('misc', convert(sub, 'misc', rootAuth)); + } +} + +// --- report --- +for (const w of written) console.log(`${(w.bytes / 1024).toFixed(1)} KB ${w.file}${w.over ? ' ** OVER 1MB **' : ''}${w.valid ? '' : ' ** INVALID Swagger 2.0 **'}`); +for (const w of [...new Set(warnings)]) console.warn(`warning: ${w}`); +const bad = written.filter((w) => w.over || !w.valid); +if (bad.length) { + console.error(`\n${bad.length} definition(s) are over 1 MB or not valid Swagger 2.0 — split that folder further, trim the collection, or fix the source. See flags above.`); + process.exit(2); +} +console.log(`\n${written.length} connector definition(s) written to ${OUT}/ — all valid Swagger 2.0, all < 1 MB.`); diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/source/.gitkeep b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/source/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/ci.yml.tmpl b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/ci.yml.tmpl new file mode 100644 index 0000000..a058144 --- /dev/null +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/ci.yml.tmpl @@ -0,0 +1,23 @@ +name: CI + +on: + push: + branches: [main] + paths: ['source/**', 'scripts/**', 'Dockerfile', 'connectors.config.json'] + pull_request: + branches: [main] + paths: ['source/**', 'scripts/**', 'Dockerfile', 'connectors.config.json'] + workflow_dispatch: + +jobs: + generate: + name: generate + validate connector definitions + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Build the pinned toolchain image + run: docker build -t ppc-gen . + + - name: Generate + self-validate (fails on invalid Swagger 2.0 or >= 1 MB) + run: docker run --rm -v "${{ github.workspace }}:/work" -w /work ppc-gen node scripts/generate.mjs diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl new file mode 100644 index 0000000..e756d45 --- /dev/null +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl @@ -0,0 +1,83 @@ +name: sync-postman + +# Detect changes to the upstream Postman collection and open a PR with regenerated +# connector definitions. Account-free when `sourceUrl` is a public URL; add a +# POSTMAN_API_KEY secret only if the collection lives inside Postman's platform. +# +# Requires: Settings -> Actions -> "Allow GitHub Actions to create and approve pull requests". + +on: + schedule: + - cron: '17 6 * * *' # daily, 06:17 UTC + workflow_dispatch: + +permissions: + contents: write + pull-requests: write + +jobs: + sync: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Fetch upstream collection and detect a change + id: fetch + env: + POSTMAN_API_KEY: ${{ secrets.POSTMAN_API_KEY }} + run: | + set -euo pipefail + URL=$(jq -r '.sourceUrl // ""' connectors.config.json) + case "$URL" in + *"> "$GITHUB_OUTPUT"; exit 0;; + esac + # Send the Postman API key only when hitting the Postman API. + AUTH=() + case "$URL" in *api.getpostman.com*) + [ -n "${POSTMAN_API_KEY:-}" ] && AUTH=(-H "X-Api-Key: ${POSTMAN_API_KEY}");; + esac + curl -fsSL "${AUTH[@]}" "$URL" -o /tmp/fetched.json + # Unwrap the Postman API's {"collection": …} envelope if present. + if jq -e '.collection' /tmp/fetched.json >/dev/null 2>&1; then + jq '.collection' /tmp/fetched.json > /tmp/coll.json + else + cp /tmp/fetched.json /tmp/coll.json + fi + NEW=$(jq -S 'del(.info._postman_id)' /tmp/coll.json | sha256sum | cut -d' ' -f1) + OLD=$(jq -r '.sha256 // ""' .postman/manifest.json) + if [ "$NEW" = "$OLD" ]; then + echo "No upstream change." + echo "changed=false" >> "$GITHUB_OUTPUT"; exit 0 + fi + SRC=$(ls source/*.json 2>/dev/null | head -1); SRC=${SRC:-source/collection.json} + cp /tmp/coll.json "$SRC" + jq -n --arg u "$URL" --arg s "$NEW" --arg t "$(date -u +%FT%TZ)" \ + '{sourceUrl:$u, sha256:$s, updatedAt:$t}' > .postman/manifest.json + echo "changed=true" >> "$GITHUB_OUTPUT" + + - name: Regenerate + validate INLINE + # Inline because a PR opened by the default GITHUB_TOKEN does not trigger ci.yml, + # so we can't rely on CI validating this bot's PR. + if: steps.fetch.outputs.changed == 'true' + run: | + docker build -t ppc-gen . + docker run --rm -v "${{ github.workspace }}:/work" -w /work ppc-gen node scripts/generate.mjs + + - name: Open a pull request + if: steps.fetch.outputs.changed == 'true' + env: + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + BR="sync/postman-$(date -u +%Y%m%d%H%M%S)" + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git checkout -b "$BR" + git add -A + git commit -m "chore: sync connector definitions from upstream Postman collection" + git push -u origin "$BR" + gh pr create --base main --head "$BR" \ + --title "Sync connector definitions from upstream Postman collection" \ + --body "The upstream Postman collection changed. Regenerated OpenAPI 2.0 definitions (validated as Swagger 2.0, each < 1 MB). **Review before importing to Power Platform** — the Postman-to-2.0 conversion is lossy." diff --git a/plugins/repokit/skills/repo-standard/standard/testing-matrix.md b/plugins/repokit/skills/repo-standard/standard/testing-matrix.md index 93a0997..50c655e 100644 --- a/plugins/repokit/skills/repo-standard/standard/testing-matrix.md +++ b/plugins/repokit/skills/repo-standard/standard/testing-matrix.md @@ -6,6 +6,8 @@ minimum, whatever proves the code runs.) | Type | Test / check | |------|--------------| | `powershell-module` | `Test-ModuleManifest` parses the `.psd1`; **Pester** for behaviour; **PSScriptAnalyzer** (`Invoke-ScriptAnalyzer -Recurse`, fail on `Error` severity). CI runs on `ubuntu-latest` with `shell: pwsh`. | +| `docker-compose` | `docker compose config -q` validates the compose file. CI runs it on `ubuntu-latest` (Public tier). | +| `power-platform-connectors` | `docker build` then `node scripts/generate.mjs` in the pinned image: every generated `connectors/*.swagger.json` must be **valid Swagger 2.0** (self-validated via `swagger-cli`) and **< 1 MB** — the generator exits non-zero otherwise. CI runs it on `ubuntu-latest` (Public tier). | | `skill-plugin` | TBD when the overlay is built — validate `plugin.json` / `marketplace.json` / `SKILL.md` frontmatter (reuse `scripts/validate_*.py`) and scan SKILL bodies for safety. | | `collection` | TBD when the overlay is built. | | `mcp-server` | TBD when the overlay is built. | diff --git a/plugins/repokit/skills/repo-standard/standard/the-standard.md b/plugins/repokit/skills/repo-standard/standard/the-standard.md index 8e62f2a..1736b8d 100644 --- a/plugins/repokit/skills/repo-standard/standard/the-standard.md +++ b/plugins/repokit/skills/repo-standard/standard/the-standard.md @@ -55,6 +55,7 @@ No governance overhead. A forever-private repo stays here and stays effortless. |------|------| | `powershell-module` | `.psd1` manifest, `.psm1` root module, `Public/` + `Private/`, `Tests/` (Pester); PSScriptAnalyzer + Pester CI; `Publish-PSResource` publish (Published) | | `docker-compose` | `compose.yaml` (no `version:`; named volumes for data, commented config/build patterns), `.env.example`, `.dockerignore`; `docker compose config -q` validation CI (Public). Config committed, data in named volumes, secrets in `.env` | +| `power-platform-connectors` | a committed Postman collection → **OpenAPI 2.0** custom-connector definitions for Microsoft Power Platform. A pinned-Docker generator (`postman-to-openapi` + `api-spec-converter`) converts + normalises to valid Swagger 2.0, splitting per top-level folder **only when a single def would hit the 1 MB limit**; self-validates; a scheduled sync workflow PRs upstream changes. Builds with just Docker — no Postman account (Public) | | `skill-plugin` | a Claude Code plugin: `.claude-plugin/`, `skills//SKILL.md`, validation *(stub — fill when first needed)* | | `collection` | a multi-component repo with a top-level map + per-component subdirs *(stub)* | | `mcp-server` | an MCP server *(stub)* | From 40459c708b54c59af5b8e4d2f4bbc8797f49a9eb Mon Sep 17 00:00:00 2001 From: PBNZ Date: Sat, 4 Jul 2026 00:43:54 +1200 Subject: [PATCH 02/10] =?UTF-8?q?fix(power-platform-connectors):=20address?= =?UTF-8?q?=20review=20=E2=80=94=20determinism=20+=20temp=20hygiene?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From Copilot review of PR #1: - generate.mjs: use a unique mkdtemp work dir (removed on exit) instead of a predictable path in the shared tmpdir. - generate.mjs: resolve collection variables whose value is set including "" and "0" (check != null, not truthiness). - generate.mjs + sync.yml: standardise on source/collection.json — the generator prefers it, and sync writes it — so the two never disagree when multiple JSONs exist. Re-proven: USPS collection -> valid Swagger 2.0; a 50 KB split limit -> 25 valid < 1 MB definitions. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01DXtxaAQHcsDThJ6BaqwXES --- .../core/scripts/generate.mjs | 16 +++++++++------- .../public/.github/workflows/sync.yml.tmpl | 4 ++-- 2 files changed, 11 insertions(+), 9 deletions(-) diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs index 6c01fbc..7964f61 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs @@ -8,7 +8,7 @@ // // Runs inside the pinned Docker image (p2o + api-spec-converter on PATH). -import { readFileSync, writeFileSync, mkdirSync, readdirSync, rmSync } from 'node:fs'; +import { readFileSync, writeFileSync, mkdirSync, mkdtempSync, readdirSync, rmSync } from 'node:fs'; import { execFileSync } from 'node:child_process'; import { join } from 'node:path'; import { tmpdir } from 'node:os'; @@ -16,11 +16,13 @@ import { tmpdir } from 'node:os'; const cfg = JSON.parse(readFileSync('connectors.config.json', 'utf8')); const LIMIT = cfg.sizeLimitBytes ?? 1048576; const OUT = cfg.output ?? 'connectors'; -const work = tmpdir(); +const work = mkdtempSync(join(tmpdir(), 'ppc-')); // unique per run; removed on exit +process.on('exit', () => { try { rmSync(work, { recursive: true, force: true }); } catch { /* best-effort */ } }); const warnings = []; -// --- locate + load the committed collection --- -const srcName = readdirSync('source').find((f) => f.endsWith('.json')); +// --- locate + load the committed collection (prefer the documented source/collection.json) --- +const jsons = readdirSync('source').filter((f) => f.endsWith('.json')).sort(); +const srcName = jsons.includes('collection.json') ? 'collection.json' : jsons[0]; if (!srcName) { console.log('source/ has no *.json collection yet — export your Postman collection there. Nothing to do.'); process.exit(0); @@ -35,11 +37,11 @@ const sizeOf = (o) => Buffer.byteLength(JSON.stringify(o, null, 2)); const slug = (s) => (s || 'connector').toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/(^-|-$)/g, '') || 'connector'; const kv = (arr, k) => arr?.find((e) => e.key === k)?.value; -// Pre-resolve collection variables (concrete {{vars}} with a value) so host/basePath come out -// real instead of "%7B%7Bbaseurl%7D%7D". Variables with no value are left untouched. +// Pre-resolve collection variables that are set (including "" and "0") so host/basePath come out +// real instead of "%7B%7Bbaseurl%7D%7D". Only variables with no value at all are left untouched. function resolveVars(obj, vars) { let s = JSON.stringify(obj); - for (const v of vars) if (v && v.key && v.value) s = s.split(`{{${v.key}}}`).join(v.value); + for (const v of vars) if (v && v.key && v.value != null) s = s.split(`{{${v.key}}}`).join(v.value); return JSON.parse(s); } diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl index e756d45..81da6f8 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl @@ -51,8 +51,8 @@ jobs: echo "No upstream change." echo "changed=false" >> "$GITHUB_OUTPUT"; exit 0 fi - SRC=$(ls source/*.json 2>/dev/null | head -1); SRC=${SRC:-source/collection.json} - cp /tmp/coll.json "$SRC" + # Write to the canonical committed path (the generator prefers source/collection.json). + cp /tmp/coll.json source/collection.json jq -n --arg u "$URL" --arg s "$NEW" --arg t "$(date -u +%FT%TZ)" \ '{sourceUrl:$u, sha256:$s, updatedAt:$t}' > .postman/manifest.json echo "changed=true" >> "$GITHUB_OUTPUT" From 7600ef8ac57523d1435d6705a48059f748047cf1 Mon Sep 17 00:00:00 2001 From: PBNZ Date: Sat, 4 Jul 2026 00:50:21 +1200 Subject: [PATCH 03/10] =?UTF-8?q?fix(power-platform-connectors):=20address?= =?UTF-8?q?=20Gemini=20review=20=E2=80=94=20robustness=20+=20safety?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - generate.mjs: guard a missing source/ dir (existsSync) instead of crashing; rewrite resolveVars to recurse the parsed object and substitute inside string values (safe against quotes/backslashes/newlines in a variable value — a raw string-replace on serialized JSON could yield invalid JSON); surface p2o/api-spec-converter stderr and capture swagger-cli validation errors into the warnings for debuggability. - sync.yml: mkdir -p source/.postman before writing; reuse one sync/postman branch with force-push (+ `|| true` on gh pr create) so daily runs update the existing PR instead of piling up new branches/PRs. Re-proven: USPS -> valid Swagger 2.0; a special-char variable value resolves safely; missing source/ exits cleanly; workflows parse as YAML. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01DXtxaAQHcsDThJ6BaqwXES --- .../core/scripts/generate.mjs | 30 ++++++++++++------- .../public/.github/workflows/sync.yml.tmpl | 10 ++++--- 2 files changed, 26 insertions(+), 14 deletions(-) diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs index 7964f61..e1cf2b3 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs @@ -8,7 +8,7 @@ // // Runs inside the pinned Docker image (p2o + api-spec-converter on PATH). -import { readFileSync, writeFileSync, mkdirSync, mkdtempSync, readdirSync, rmSync } from 'node:fs'; +import { readFileSync, writeFileSync, mkdirSync, mkdtempSync, readdirSync, rmSync, existsSync } from 'node:fs'; import { execFileSync } from 'node:child_process'; import { join } from 'node:path'; import { tmpdir } from 'node:os'; @@ -21,10 +21,10 @@ process.on('exit', () => { try { rmSync(work, { recursive: true, force: true }); const warnings = []; // --- locate + load the committed collection (prefer the documented source/collection.json) --- -const jsons = readdirSync('source').filter((f) => f.endsWith('.json')).sort(); +const jsons = existsSync('source') ? readdirSync('source').filter((f) => f.endsWith('.json')).sort() : []; const srcName = jsons.includes('collection.json') ? 'collection.json' : jsons[0]; if (!srcName) { - console.log('source/ has no *.json collection yet — export your Postman collection there. Nothing to do.'); + console.log('source/ has no *.json collection yet — export your Postman collection to source/collection.json. Nothing to do.'); process.exit(0); } const rawObj = JSON.parse(readFileSync(join('source', srcName), 'utf8')); @@ -38,11 +38,18 @@ const slug = (s) => (s || 'connector').toLowerCase().replace(/[^a-z0-9]+/g, '-') const kv = (arr, k) => arr?.find((e) => e.key === k)?.value; // Pre-resolve collection variables that are set (including "" and "0") so host/basePath come out -// real instead of "%7B%7Bbaseurl%7D%7D". Only variables with no value at all are left untouched. +// real instead of "%7B%7Bbaseurl%7D%7D". Walk the parsed object and substitute inside string values +// only — safe against values containing quotes/backslashes/newlines (a raw string-replace on the +// serialized JSON could produce invalid JSON). function resolveVars(obj, vars) { - let s = JSON.stringify(obj); - for (const v of vars) if (v && v.key && v.value != null) s = s.split(`{{${v.key}}}`).join(v.value); - return JSON.parse(s); + if (typeof obj === 'string') { + let s = obj; + for (const v of vars) if (v && v.key && v.value != null) s = s.split(`{{${v.key}}}`).join(String(v.value)); + return s; + } + if (Array.isArray(obj)) return obj.map((x) => resolveVars(x, vars)); + if (obj && typeof obj === 'object') { const r = {}; for (const [k, val] of Object.entries(obj)) r[k] = resolveVars(val, vars); return r; } + return obj; } // Map a Postman auth block to a valid Swagger 2.0 security definition. p2o mis-maps these @@ -123,8 +130,10 @@ function convert(coll, tag, effectiveAuth) { const pin = join(work, `${tag}.postman.json`); const oas = join(work, `${tag}.oas3.yml`); writeFileSync(pin, JSON.stringify(resolveVars(coll, coll.variable?.length ? coll.variable : rootVars))); - execFileSync('p2o', [pin, '-f', oas], { stdio: 'pipe' }); - const raw = execFileSync('api-spec-converter', ['--from=openapi_3', '--to=swagger_2', '--syntax=json', oas], { stdio: 'pipe' }); + // Show the converters' own warnings/errors (stderr) so a bad collection is debuggable; still + // capture api-spec-converter's stdout (the Swagger JSON). + execFileSync('p2o', [pin, '-f', oas], { stdio: ['ignore', 'ignore', 'inherit'] }); + const raw = execFileSync('api-spec-converter', ['--from=openapi_3', '--to=swagger_2', '--syntax=json', oas], { stdio: ['ignore', 'pipe', 'inherit'] }); const sw = JSON.parse(raw.toString()); fixPaths(sw); fixResponses(sw); @@ -150,7 +159,8 @@ function emit(name, sw) { const file = join(OUT, `${slug(name)}.swagger.json`); writeFileSync(file, JSON.stringify(sw, null, 2)); let valid = true; - try { execFileSync('swagger-cli', ['validate', file], { stdio: 'pipe' }); } catch { valid = false; } + try { execFileSync('swagger-cli', ['validate', file], { stdio: 'pipe' }); } + catch (err) { valid = false; warnings.push(`validation failed for ${file}: ${(err.stderr?.toString() || err.message || '').trim()}`); } written.push({ file, bytes, over: bytes >= LIMIT, valid }); } diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl index 81da6f8..08552b6 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl @@ -52,6 +52,7 @@ jobs: echo "changed=false" >> "$GITHUB_OUTPUT"; exit 0 fi # Write to the canonical committed path (the generator prefers source/collection.json). + mkdir -p source .postman cp /tmp/coll.json source/collection.json jq -n --arg u "$URL" --arg s "$NEW" --arg t "$(date -u +%FT%TZ)" \ '{sourceUrl:$u, sha256:$s, updatedAt:$t}' > .postman/manifest.json @@ -71,13 +72,14 @@ jobs: GH_TOKEN: ${{ github.token }} run: | set -euo pipefail - BR="sync/postman-$(date -u +%Y%m%d%H%M%S)" + # Reuse one branch/PR so daily runs update the existing PR instead of piling up new ones. + BR="sync/postman" git config user.name "github-actions[bot]" git config user.email "41898282+github-actions[bot]@users.noreply.github.com" - git checkout -b "$BR" + git checkout -B "$BR" git add -A git commit -m "chore: sync connector definitions from upstream Postman collection" - git push -u origin "$BR" + git push -f -u origin "$BR" gh pr create --base main --head "$BR" \ --title "Sync connector definitions from upstream Postman collection" \ - --body "The upstream Postman collection changed. Regenerated OpenAPI 2.0 definitions (validated as Swagger 2.0, each < 1 MB). **Review before importing to Power Platform** — the Postman-to-2.0 conversion is lossy." + --body "The upstream Postman collection changed. Regenerated OpenAPI 2.0 definitions (validated as Swagger 2.0, each < 1 MB). **Review before importing to Power Platform** — the Postman-to-2.0 conversion is lossy." || true From f6e40fc4efa5d18ae3d9bcf77bee003225e38031 Mon Sep 17 00:00:00 2001 From: PBNZ Date: Sat, 4 Jul 2026 00:59:50 +1200 Subject: [PATCH 04/10] fix(power-platform-connectors): report the configured size limit, not "1 MB" Copilot re-review: the generator's flags and summary hard-coded "1 MB" even though sizeLimitBytes is configurable. Derive a human-readable limitStr from LIMIT and use it in all three messages (e.g. "< 49 KB" when sizeLimitBytes is lowered). Proven at 1 MB and at 50 KB. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01DXtxaAQHcsDThJ6BaqwXES --- .../power-platform-connectors/core/scripts/generate.mjs | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs index e1cf2b3..0d57222 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs @@ -15,6 +15,7 @@ import { tmpdir } from 'node:os'; const cfg = JSON.parse(readFileSync('connectors.config.json', 'utf8')); const LIMIT = cfg.sizeLimitBytes ?? 1048576; +const limitStr = LIMIT >= 1048576 ? `${+(LIMIT / 1048576).toFixed(2)} MB` : `${Math.round(LIMIT / 1024)} KB`; const OUT = cfg.output ?? 'connectors'; const work = mkdtempSync(join(tmpdir(), 'ppc-')); // unique per run; removed on exit process.on('exit', () => { try { rmSync(work, { recursive: true, force: true }); } catch { /* best-effort */ } }); @@ -186,11 +187,11 @@ if (sizeOf(whole) < LIMIT) { } // --- report --- -for (const w of written) console.log(`${(w.bytes / 1024).toFixed(1)} KB ${w.file}${w.over ? ' ** OVER 1MB **' : ''}${w.valid ? '' : ' ** INVALID Swagger 2.0 **'}`); +for (const w of written) console.log(`${(w.bytes / 1024).toFixed(1)} KB ${w.file}${w.over ? ` ** OVER ${limitStr} **` : ''}${w.valid ? '' : ' ** INVALID Swagger 2.0 **'}`); for (const w of [...new Set(warnings)]) console.warn(`warning: ${w}`); const bad = written.filter((w) => w.over || !w.valid); if (bad.length) { - console.error(`\n${bad.length} definition(s) are over 1 MB or not valid Swagger 2.0 — split that folder further, trim the collection, or fix the source. See flags above.`); + console.error(`\n${bad.length} definition(s) are over ${limitStr} or not valid Swagger 2.0 — split that folder further, trim the collection, or fix the source. See flags above.`); process.exit(2); } -console.log(`\n${written.length} connector definition(s) written to ${OUT}/ — all valid Swagger 2.0, all < 1 MB.`); +console.log(`\n${written.length} connector definition(s) written to ${OUT}/ — all valid Swagger 2.0, all < ${limitStr}.`); From eb2db3302b285604eec8855e5fd300a141fe0799 Mon Sep 17 00:00:00 2001 From: PBNZ Date: Sat, 4 Jul 2026 01:13:01 +1200 Subject: [PATCH 05/10] =?UTF-8?q?fix(power-platform-connectors):=20round-3?= =?UTF-8?q?=20review=20=E2=80=94=20robustness=20+=20non-root=20docker?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses the second Copilot + Gemini re-review: - generate.mjs: clean the temp dir on SIGINT/SIGTERM too (not just normal exit); restrict fixResponses/fixPaths to real HTTP methods (skip path-level parameters/$ref/x-* extensions); disambiguate output filenames that slug to the same name (uniqueSlug); guard against null items / missing folder names. - ci.yml + sync.yml: run the container with --user "$(id -u):$(id -g)" so generated files aren't root-owned (avoids permission issues before git add); de-hard-code "1 MB" in the CI step name. - README: note the --user tip for Linux/macOS (kept out of the command so the Windows/PowerShell copy-paste still works). Re-proven: USPS -> valid; 50 KB limit -> 25 valid split files; runs correctly as a non-root user. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01DXtxaAQHcsDThJ6BaqwXES --- .../core/README.md.tmpl | 3 ++ .../core/scripts/generate.mjs | 38 +++++++++++++------ .../public/.github/workflows/ci.yml.tmpl | 4 +- .../public/.github/workflows/sync.yml.tmpl | 2 +- 4 files changed, 33 insertions(+), 14 deletions(-) diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/README.md.tmpl b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/README.md.tmpl index b6d5cf1..8d32a7d 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/README.md.tmpl +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/README.md.tmpl @@ -33,6 +33,9 @@ flowchart LR docker run --rm -v "${PWD}:/work" {{name}}-gen node scripts/generate.mjs ``` + *(On Linux/macOS, add `--user "$(id -u):$(id -g)"` to the `docker run` so the generated files + aren't owned by root.)* + It writes `connectors/*.swagger.json` — **one file if the whole collection fits under 1 MB, otherwise one per top-level folder**. Every file is validated as Swagger 2.0 and checked < 1 MB; the run fails if any definition is invalid or oversize. diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs index 0d57222..e62af3c 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs @@ -17,8 +17,10 @@ const cfg = JSON.parse(readFileSync('connectors.config.json', 'utf8')); const LIMIT = cfg.sizeLimitBytes ?? 1048576; const limitStr = LIMIT >= 1048576 ? `${+(LIMIT / 1048576).toFixed(2)} MB` : `${Math.round(LIMIT / 1024)} KB`; const OUT = cfg.output ?? 'connectors'; -const work = mkdtempSync(join(tmpdir(), 'ppc-')); // unique per run; removed on exit -process.on('exit', () => { try { rmSync(work, { recursive: true, force: true }); } catch { /* best-effort */ } }); +const work = mkdtempSync(join(tmpdir(), 'ppc-')); // unique per run; removed on exit / interrupt +const cleanup = () => { try { rmSync(work, { recursive: true, force: true }); } catch { /* best-effort */ } }; +process.on('exit', cleanup); +for (const sig of ['SIGINT', 'SIGTERM']) process.on(sig, () => { cleanup(); process.exit(1); }); const warnings = []; // --- locate + load the committed collection (prefer the documented source/collection.json) --- @@ -81,12 +83,17 @@ function pmAuthToSwagger(auth) { // response's `status` (reason phrase); collections that omit `status` yield a description-less // (invalid) response. Backfill a sensible one so the output always validates. const REASON = { 200: 'OK', 201: 'Created', 202: 'Accepted', 203: 'Non-Authoritative Information', 204: 'No Content', 206: 'Partial Content', 301: 'Moved Permanently', 302: 'Found', 304: 'Not Modified', 400: 'Bad Request', 401: 'Unauthorized', 403: 'Forbidden', 404: 'Not Found', 405: 'Method Not Allowed', 409: 'Conflict', 422: 'Unprocessable Entity', 429: 'Too Many Requests', 500: 'Internal Server Error', 502: 'Bad Gateway', 503: 'Service Unavailable' }; +const METHODS = ['get', 'put', 'post', 'delete', 'options', 'head', 'patch']; function fixResponses(sw) { - for (const path of Object.values(sw.paths || {})) - for (const op of Object.values(path)) + for (const path of Object.values(sw.paths || {})) { + if (!path || typeof path !== 'object') continue; + for (const m of METHODS) { + const op = path[m]; if (op && typeof op === 'object' && op.responses) for (const [code, r] of Object.entries(op.responses)) if (r && typeof r === 'object' && !r.description) r.description = REASON[code] || 'Response'; + } + } } // Swagger 2.0 requires (a) every path key to start with "/", and (b) every "{token}" in a path to @@ -98,8 +105,9 @@ function fixPaths(sw) { const pathKey = key.startsWith('/') ? key : '/' + key; const tokens = [...pathKey.matchAll(/\{([^}]+)\}/g)].map((m) => m[1]); if (tokens.length && item && typeof item === 'object') { - for (const op of Object.values(item)) { - if (op && typeof op === 'object' && op.responses) { + for (const m of METHODS) { + const op = item[m]; + if (op && typeof op === 'object') { op.parameters = op.parameters || []; for (const t of tokens) if (!op.parameters.some((p) => p.in === 'path' && p.name === t)) @@ -154,10 +162,17 @@ mkdirSync(OUT, { recursive: true }); for (const f of readdirSync(OUT)) if (f.endsWith('.swagger.json')) rmSync(join(OUT, f)); const written = []; +const usedSlugs = new Map(); +function uniqueSlug(name) { + const base = slug(name); + const n = (usedSlugs.get(base) || 0) + 1; + usedSlugs.set(base, n); + return n === 1 ? base : `${base}-${n}`; // disambiguate folders that slug to the same name +} function emit(name, sw) { let bytes = sizeOf(sw); if (bytes >= LIMIT) { strip(sw); bytes = sizeOf(sw); } // last-ditch shrink for an oversize def - const file = join(OUT, `${slug(name)}.swagger.json`); + const file = join(OUT, `${uniqueSlug(name)}.swagger.json`); writeFileSync(file, JSON.stringify(sw, null, 2)); let valid = true; try { execFileSync('swagger-cli', ['validate', file], { stdio: 'pipe' }); } @@ -171,15 +186,16 @@ if (sizeOf(whole) < LIMIT) { emit(collection.info?.name || srcName.replace(/\.json$/, ''), whole); } else { const roots = collection.item || []; - for (const folder of roots.filter((it) => it.item)) { + for (const folder of roots.filter((it) => it && it.item)) { + const fname = folder.name || 'folder'; const sub = { - info: { ...collection.info, name: `${collection.info?.name || ''} - ${folder.name}`.trim() }, + info: { ...collection.info, name: `${collection.info?.name || ''} - ${fname}`.trim() }, variable: rootVars, item: folder.item, }; - emit(folder.name, convert(sub, slug(folder.name), folder.auth || rootAuth)); + emit(fname, convert(sub, slug(fname), folder.auth || rootAuth)); } - const loose = roots.filter((it) => it.request); + const loose = roots.filter((it) => it && it.request); if (loose.length) { const sub = { info: { ...collection.info, name: `${collection.info?.name || ''} - misc` }, variable: rootVars, item: loose }; emit('misc', convert(sub, 'misc', rootAuth)); diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/ci.yml.tmpl b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/ci.yml.tmpl index a058144..d1ed7b0 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/ci.yml.tmpl +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/ci.yml.tmpl @@ -19,5 +19,5 @@ jobs: - name: Build the pinned toolchain image run: docker build -t ppc-gen . - - name: Generate + self-validate (fails on invalid Swagger 2.0 or >= 1 MB) - run: docker run --rm -v "${{ github.workspace }}:/work" -w /work ppc-gen node scripts/generate.mjs + - name: Generate + self-validate (fails on invalid Swagger 2.0 or over the size limit) + run: docker run --rm --user "$(id -u):$(id -g)" -v "${{ github.workspace }}:/work" -w /work ppc-gen node scripts/generate.mjs diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl index 08552b6..9b47bfb 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl @@ -64,7 +64,7 @@ jobs: if: steps.fetch.outputs.changed == 'true' run: | docker build -t ppc-gen . - docker run --rm -v "${{ github.workspace }}:/work" -w /work ppc-gen node scripts/generate.mjs + docker run --rm --user "$(id -u):$(id -g)" -v "${{ github.workspace }}:/work" -w /work ppc-gen node scripts/generate.mjs - name: Open a pull request if: steps.fetch.outputs.changed == 'true' From fa6298cfd52f3fe3cd8f2bad0ee11df36a48766e Mon Sep 17 00:00:00 2001 From: PBNZ Date: Sat, 4 Jul 2026 01:23:50 +1200 Subject: [PATCH 06/10] =?UTF-8?q?fix(power-platform-connectors):=20round-4?= =?UTF-8?q?=20review=20=E2=80=94=20valid=20oauth2=20+=20no=20dangling=20se?= =?UTF-8?q?curity?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two real Swagger 2.0 validity bugs caught by the re-review (USPS uses apikey, so earlier tests missed them): - pmAuthToSwagger: an oauth2 block with missing authorizationUrl/tokenUrl produced an invalid def (accessCode flow requires both URLs). Now returns null in that case -> no security def + a warning, so the output stays valid. - fixSecurity: overriding the root securityDefinitions left operation-level `security` blocks pointing at the deleted scheme (a dangling ref). Now strip operation-level security so only the single root-level definition applies. Also (hardening): sync.yml only sends the Postman API key to https://api.getpostman.com/ (not look-alike hosts), adds curl --max-time/--retry, and tolerates a missing manifest; split sub-collections carry auth for p2o context. Proven: oauth2 with URLs -> valid oauth2 def; without -> valid (no security); USPS apikey regression still valid. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01DXtxaAQHcsDThJ6BaqwXES --- .../core/scripts/generate.mjs | 15 +++++++++++++-- .../public/.github/workflows/sync.yml.tmpl | 9 +++++---- 2 files changed, 18 insertions(+), 6 deletions(-) diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs index e62af3c..26f5e78 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs @@ -72,7 +72,10 @@ function pmAuthToSwagger(auth) { return { name: 'basicAuth', def: { type: 'basic' } }; case 'oauth2': { const a = auth.oauth2 || []; - return { name: 'oauth2Auth', def: { type: 'oauth2', flow: 'accessCode', authorizationUrl: kv(a, 'authUrl') || '', tokenUrl: kv(a, 'accessTokenUrl') || '', scopes: {} } }; + const authorizationUrl = kv(a, 'authUrl'); + const tokenUrl = kv(a, 'accessTokenUrl'); + if (!authorizationUrl || !tokenUrl) return null; // 2.0 accessCode flow requires both URLs + return { name: 'oauth2Auth', def: { type: 'oauth2', flow: 'accessCode', authorizationUrl, tokenUrl, scopes: {} } }; } default: return null; @@ -132,6 +135,13 @@ function fixSecurity(sw, auth) { delete sw.securityDefinitions; // never ship an invalid securityDefinitions block delete sw.security; } + // Drop any operation-level security the converter emitted — it points at the old (now replaced + // or deleted) definition, i.e. a dangling reference. The single root-level security applies to + // every operation. + for (const path of Object.values(sw.paths || {})) { + if (!path || typeof path !== 'object') continue; + for (const m of METHODS) if (path[m] && typeof path[m] === 'object') delete path[m].security; + } } // One Postman (sub)collection -> validated Swagger 2.0 object. @@ -191,13 +201,14 @@ if (sizeOf(whole) < LIMIT) { const sub = { info: { ...collection.info, name: `${collection.info?.name || ''} - ${fname}`.trim() }, variable: rootVars, + auth: folder.auth || rootAuth, item: folder.item, }; emit(fname, convert(sub, slug(fname), folder.auth || rootAuth)); } const loose = roots.filter((it) => it && it.request); if (loose.length) { - const sub = { info: { ...collection.info, name: `${collection.info?.name || ''} - misc` }, variable: rootVars, item: loose }; + const sub = { info: { ...collection.info, name: `${collection.info?.name || ''} - misc` }, variable: rootVars, auth: rootAuth, item: loose }; emit('misc', convert(sub, 'misc', rootAuth)); } } diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl index 9b47bfb..7922578 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl @@ -33,12 +33,13 @@ jobs: echo "sourceUrl is not set in connectors.config.json — skipping." echo "changed=false" >> "$GITHUB_OUTPUT"; exit 0;; esac - # Send the Postman API key only when hitting the Postman API. + # Send the Postman API key only to the real Postman API host (a substring match would also + # hit look-alikes like api.getpostman.com.evil.tld). AUTH=() - case "$URL" in *api.getpostman.com*) + case "$URL" in "https://api.getpostman.com/"*) [ -n "${POSTMAN_API_KEY:-}" ] && AUTH=(-H "X-Api-Key: ${POSTMAN_API_KEY}");; esac - curl -fsSL "${AUTH[@]}" "$URL" -o /tmp/fetched.json + curl -fsSL --max-time 60 --retry 3 "${AUTH[@]}" "$URL" -o /tmp/fetched.json # Unwrap the Postman API's {"collection": …} envelope if present. if jq -e '.collection' /tmp/fetched.json >/dev/null 2>&1; then jq '.collection' /tmp/fetched.json > /tmp/coll.json @@ -46,7 +47,7 @@ jobs: cp /tmp/fetched.json /tmp/coll.json fi NEW=$(jq -S 'del(.info._postman_id)' /tmp/coll.json | sha256sum | cut -d' ' -f1) - OLD=$(jq -r '.sha256 // ""' .postman/manifest.json) + OLD=$(jq -r '.sha256 // ""' .postman/manifest.json 2>/dev/null || echo "") if [ "$NEW" = "$OLD" ]; then echo "No upstream change." echo "changed=false" >> "$GITHUB_OUTPUT"; exit 0 From 94293cba5235065981d1f37a383f421338737ffe Mon Sep 17 00:00:00 2001 From: PBNZ Date: Sat, 4 Jul 2026 01:32:09 +1200 Subject: [PATCH 07/10] =?UTF-8?q?fix(power-platform-connectors):=20round-5?= =?UTF-8?q?=20review=20=E2=80=94=20$ref=20responses=20+=20slug=20collision?= =?UTF-8?q?s?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two more edge-case validity fixes from the re-review: - fixResponses: don't add a `description` to a `$ref` response object (in Swagger 2.0 a `$ref` must be the only key; a sibling `description` is invalid). - uniqueSlug: track the actual set of used filenames and increment until free, so two "My Folder" folders plus a real "My Folder 2" produce three distinct files instead of colliding. Proven: USPS still valid; duplicate folder names -> 3 distinct outputs. Deliberately not changed (evaluated, low value): --force-with-lease on the solo bot-managed sync branch; an O(N*V) micro-opt in resolveVars; an explicit uncaughtException cleanup handler (Node's 'exit' already fires on that, so the temp dir is cleaned). Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01DXtxaAQHcsDThJ6BaqwXES --- .../core/scripts/generate.mjs | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs index 26f5e78..18ecaa8 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs @@ -94,7 +94,7 @@ function fixResponses(sw) { const op = path[m]; if (op && typeof op === 'object' && op.responses) for (const [code, r] of Object.entries(op.responses)) - if (r && typeof r === 'object' && !r.description) r.description = REASON[code] || 'Response'; + if (r && typeof r === 'object' && !r.$ref && !r.description) r.description = REASON[code] || 'Response'; } } } @@ -172,12 +172,13 @@ mkdirSync(OUT, { recursive: true }); for (const f of readdirSync(OUT)) if (f.endsWith('.swagger.json')) rmSync(join(OUT, f)); const written = []; -const usedSlugs = new Map(); +const usedNames = new Set(); function uniqueSlug(name) { const base = slug(name); - const n = (usedSlugs.get(base) || 0) + 1; - usedSlugs.set(base, n); - return n === 1 ? base : `${base}-${n}`; // disambiguate folders that slug to the same name + let candidate = base, n = 1; + while (usedNames.has(candidate)) candidate = `${base}-${++n}`; // avoids cross-collisions with real "…-2" names + usedNames.add(candidate); + return candidate; } function emit(name, sw) { let bytes = sizeOf(sw); From 8b3b8de289b3b1194edcc485f1a778f77d4c7aaf Mon Sep 17 00:00:00 2001 From: PBNZ Date: Sat, 4 Jul 2026 02:03:38 +1200 Subject: [PATCH 08/10] fix(power-platform-connectors): harden generator + workflows from self-review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fresh critical-review pass (own reviewer agents) on generate.mjs + the workflows, fixing correctness and robustness gaps that swagger-cli's validateFormats:false gate would let pass silently: generate.mjs - C1: scan each output for residual {{var}} / %7b%7b (unresolved Postman *environment* variables) and fail the build — previously shipped a broken connector reported as "all valid Swagger 2.0". Verified: exit 2 + clear flag. - H1: resolve committed {{vars}} inside the auth block before mapping, so OAuth URLs / apiKey header names don't leak literally. Verified. - H2: map every OAuth2 grant to the correct Swagger 2.0 flow (client_credentials ->application, password->password, implicit->implicit, auth_code->accessCode), requiring only the URL(s) that flow needs (verified vs the OpenAPI 2.0 spec). Previously hardcoded accessCode -> shipped an authless connector for the common client-credentials case. Verified: client_credentials -> flow:application. - H3: warn when auth is set only per-request (not mapped) instead of silently emitting no securityDefinitions. - M1: try dropping examples on the whole collection before deciding to split, so a barely-over collection stays one file instead of fragmenting. - M2: per-(sub)collection try/catch so one folder that fails conversion no longer aborts the rest of the split; failures are reported and still exit non-zero. - LOW: guard kv() against v2.0 object-form auth (no more TypeError crash, verified), treat noauth as silent no-security, and make resolveVars a bounded fixed-point so nested/out-of-order vars resolve. workflows - sync H1: --max-redirs 0 on the authed Postman fetch + --proto/--proto-redir =https so the API key can't follow a redirect to another host. - sync H2: open the PR only if none exists; don't mask other gh errors. - ci/sync: least-privilege permissions on ci.yml, concurrency guard on sync, scoped git add, HOME=/tmp for the container, drop the daily-churn timestamp. - Dockerfile: npm install --ignore-scripts (defence-in-depth; tools verified to still run). Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01DXtxaAQHcsDThJ6BaqwXES --- .../core/.postman/manifest.json | 3 +- .../power-platform-connectors/core/Dockerfile | 4 +- .../core/scripts/generate.mjs | 115 +++++++++++++++--- .../public/.github/workflows/ci.yml.tmpl | 5 +- .../public/.github/workflows/sync.yml.tmpl | 33 +++-- 5 files changed, 127 insertions(+), 33 deletions(-) diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/.postman/manifest.json b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/.postman/manifest.json index 2467e98..27e981a 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/.postman/manifest.json +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/.postman/manifest.json @@ -1,5 +1,4 @@ { "sourceUrl": "", - "sha256": "", - "updatedAt": "" + "sha256": "" } diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/Dockerfile b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/Dockerfile index e50a6a3..feda011 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/Dockerfile +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/Dockerfile @@ -6,7 +6,9 @@ # Node line or the tool versions without re-testing the whole pipeline. FROM node:18-bullseye-slim -RUN npm install -g postman-to-openapi@3.0.1 api-spec-converter@2.12.0 @apidevtools/swagger-cli@4.0.4 +# --ignore-scripts blocks install-time lifecycle scripts (defence-in-depth for this stale, +# transitive-heavy dependency tree). These three tools are pure JS and run fine without them. +RUN npm install -g --ignore-scripts postman-to-openapi@3.0.1 api-spec-converter@2.12.0 @apidevtools/swagger-cli@4.0.4 WORKDIR /work # The repo is mounted at /work at run time; run: node scripts/generate.mjs diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs index 18ecaa8..d1e1d32 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs @@ -22,6 +22,7 @@ const cleanup = () => { try { rmSync(work, { recursive: true, force: true }); } process.on('exit', cleanup); for (const sig of ['SIGINT', 'SIGTERM']) process.on(sig, () => { cleanup(); process.exit(1); }); const warnings = []; +let hadError = false; // set when a (sub)collection fails to convert, so we still exit non-zero // --- locate + load the committed collection (prefer the documented source/collection.json) --- const jsons = existsSync('source') ? readdirSync('source').filter((f) => f.endsWith('.json')).sort() : []; @@ -40,6 +41,17 @@ const sizeOf = (o) => Buffer.byteLength(JSON.stringify(o, null, 2)); const slug = (s) => (s || 'connector').toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/(^-|-$)/g, '') || 'connector'; const kv = (arr, k) => arr?.find((e) => e.key === k)?.value; +// Detect auth set only on individual requests. We map collection/folder-level auth, not per-request +// auth, so without this a request-auth-only collection would ship with no securityDefinitions and no +// warning at all. +function hasRequestAuth(items) { + for (const it of items || []) { + if (it?.request?.auth?.type && it.request.auth.type !== 'noauth') return true; + if (it?.item && hasRequestAuth(it.item)) return true; + } + return false; +} + // Pre-resolve collection variables that are set (including "" and "0") so host/basePath come out // real instead of "%7B%7Bbaseurl%7D%7D". Walk the parsed object and substitute inside string values // only — safe against values containing quotes/backslashes/newlines (a raw string-replace on the @@ -47,7 +59,13 @@ const kv = (arr, k) => arr?.find((e) => e.key === k)?.value; function resolveVars(obj, vars) { if (typeof obj === 'string') { let s = obj; - for (const v of vars) if (v && v.key && v.value != null) s = s.split(`{{${v.key}}}`).join(String(v.value)); + // Repeat until stable (bounded) so a nested var resolves regardless of its order in vars[]: + // baseUrl="{{proto}}://host" needs a second pass to also resolve {{proto}}. + for (let pass = 0; pass < 5 && s.includes('{{'); pass++) { + const before = s; + for (const v of vars) if (v && v.key && v.value != null) s = s.split(`{{${v.key}}}`).join(String(v.value)); + if (s === before) break; + } return s; } if (Array.isArray(obj)) return obj.map((x) => resolveVars(x, vars)); @@ -59,10 +77,11 @@ function resolveVars(obj, vars) { // (e.g. apikey -> {type:http, scheme:apikey}, which is invalid 2.0 AND drops the header name), // so we build them from the source collection instead. function pmAuthToSwagger(auth) { - if (!auth || !auth.type) return null; + if (!auth || !auth.type || auth.type === 'noauth') return null; // noauth = intentional no security + const attrs = (k) => (Array.isArray(auth[k]) ? auth[k] : []); // v2.0 object-form auth would crash kv() switch (auth.type) { case 'apikey': { - const a = auth.apikey || []; + const a = attrs('apikey'); const loc = (kv(a, 'in') || 'header').toLowerCase() === 'query' ? 'query' : 'header'; return { name: 'apiKeyAuth', def: { type: 'apiKey', name: kv(a, 'key') || 'Authorization', in: loc } }; } @@ -71,11 +90,34 @@ function pmAuthToSwagger(auth) { case 'basic': return { name: 'basicAuth', def: { type: 'basic' } }; case 'oauth2': { - const a = auth.oauth2 || []; + const a = attrs('oauth2'); const authorizationUrl = kv(a, 'authUrl'); const tokenUrl = kv(a, 'accessTokenUrl'); - if (!authorizationUrl || !tokenUrl) return null; // 2.0 accessCode flow requires both URLs - return { name: 'oauth2Auth', def: { type: 'oauth2', flow: 'accessCode', authorizationUrl, tokenUrl, scopes: {} } }; + const grant = (kv(a, 'grant_type') || '').toLowerCase(); + // Pick the Swagger 2.0 oauth2 flow. Prefer Postman's grant_type (substring match — robust to + // the exact string), else infer from which URLs are present. Emit only the URL(s) that flow + // requires, verified against the OpenAPI 2.0 spec: implicit->authorizationUrl, + // password/application->tokenUrl, accessCode->both. (Postman's earlier hardcoded-accessCode + // path shipped a NO-auth connector for client-credentials/password/implicit grants.) + let flow; + if (grant.includes('client')) flow = 'application'; + else if (grant.includes('password')) flow = 'password'; + else if (grant.includes('implicit')) flow = 'implicit'; + else if (grant.includes('authorization') || grant.includes('code')) flow = 'accessCode'; + else if (authorizationUrl && tokenUrl) flow = 'accessCode'; + else if (tokenUrl) flow = 'application'; + else if (authorizationUrl) flow = 'implicit'; + else return null; // no grant_type and no URLs — nothing usable + const def = { type: 'oauth2', flow, scopes: {} }; + if (flow === 'accessCode' || flow === 'implicit') { + if (!authorizationUrl) return null; + def.authorizationUrl = authorizationUrl; + } + if (flow === 'accessCode' || flow === 'application' || flow === 'password') { + if (!tokenUrl) return null; + def.tokenUrl = tokenUrl; + } + return { name: 'oauth2Auth', def }; } default: return null; @@ -126,12 +168,16 @@ function fixPaths(sw) { // Replace whatever the converter produced with a correct security definition derived from the // Postman auth (Power Platform picks the single top securityDefinition, so we emit exactly one). function fixSecurity(sw, auth) { - const s = pmAuthToSwagger(auth); + // Resolve committed {{vars}} in the auth block too — otherwise a variable used as an OAuth URL or + // an apiKey header name would leak into securityDefinitions literally (and swagger-cli validates + // with validateFormats:false, so a bad URL would pass silently). + const s = pmAuthToSwagger(resolveVars(auth, rootVars)); if (s) { sw.securityDefinitions = { [s.name]: s.def }; sw.security = [{ [s.name]: [] }]; } else { - if (auth?.type) warnings.push(`auth type "${auth.type}" not mapped — add security manually in the connector`); + if (auth?.type && auth.type !== 'noauth') + warnings.push(`auth type "${auth.type}" not fully mapped (missing URL/grant details?) — add security manually in the connector`); delete sw.securityDefinitions; // never ship an invalid securityDefinitions block delete sw.security; } @@ -184,16 +230,47 @@ function emit(name, sw) { let bytes = sizeOf(sw); if (bytes >= LIMIT) { strip(sw); bytes = sizeOf(sw); } // last-ditch shrink for an oversize def const file = join(OUT, `${uniqueSlug(name)}.swagger.json`); - writeFileSync(file, JSON.stringify(sw, null, 2)); + const serialized = JSON.stringify(sw, null, 2); + writeFileSync(file, serialized); let valid = true; + // Residual Postman template tokens — {{var}} or its %7b%7b URL-encoding — mean a variable never + // resolved, almost always a Postman *environment* variable that isn't in the committed + // collection.json. swagger-cli validates with validateFormats:false, so a host/URL like + // "%7b%7bbaseurl%7d%7d" would otherwise pass and the connector would ship broken. + const unresolved = /\{\{|%7[bB]%7[bB]/.test(serialized); + if (unresolved) + warnings.push(`${file}: unresolved {{variables}} — likely a Postman environment variable not in the committed collection; the connector will not work until these are provided`); try { execFileSync('swagger-cli', ['validate', file], { stdio: 'pipe' }); } catch (err) { valid = false; warnings.push(`validation failed for ${file}: ${(err.stderr?.toString() || err.message || '').trim()}`); } - written.push({ file, bytes, over: bytes >= LIMIT, valid }); + written.push({ file, bytes, over: bytes >= LIMIT, valid, unresolved }); } -// --- convert whole; split only if it wouldn't fit --- -const whole = convert(collection, 'whole', rootAuth); -if (sizeOf(whole) < LIMIT) { +// Convert one (sub)collection and emit it; on converter failure, record it and keep going so one +// bad folder doesn't abort the rest of the split (p2o / api-spec-converter throw on any non-zero). +function tryEmit(name, coll, tag, auth) { + try { + emit(name, convert(coll, tag, auth)); + } catch (err) { + hadError = true; + warnings.push(`"${name}" failed to convert: ${(err.message || err).toString().trim()}`); + } +} + +// H3: auth on individual requests is not mapped — warn rather than silently ship an authless connector. +if (!rootAuth && hasRequestAuth(collection.item)) { + warnings.push('auth is set per-request in this collection; only collection/folder-level auth is mapped, so the connector(s) have no securityDefinitions — add auth manually after import'); +} + +// --- convert whole; split only if it *still* wouldn't fit after dropping examples --- +let whole = null, wholeBytes = Infinity; +try { + whole = convert(collection, 'whole', rootAuth); + wholeBytes = sizeOf(whole); + if (wholeBytes >= LIMIT) { strip(whole); wholeBytes = sizeOf(whole); } // try examples-off before splitting +} catch (err) { + warnings.push(`whole-collection conversion failed (${(err.message || err).toString().trim()}); falling back to per-folder`); +} +if (whole && wholeBytes < LIMIT) { emit(collection.info?.name || srcName.replace(/\.json$/, ''), whole); } else { const roots = collection.item || []; @@ -205,21 +282,21 @@ if (sizeOf(whole) < LIMIT) { auth: folder.auth || rootAuth, item: folder.item, }; - emit(fname, convert(sub, slug(fname), folder.auth || rootAuth)); + tryEmit(fname, sub, slug(fname), folder.auth || rootAuth); } const loose = roots.filter((it) => it && it.request); if (loose.length) { const sub = { info: { ...collection.info, name: `${collection.info?.name || ''} - misc` }, variable: rootVars, auth: rootAuth, item: loose }; - emit('misc', convert(sub, 'misc', rootAuth)); + tryEmit('misc', sub, 'misc', rootAuth); } } // --- report --- -for (const w of written) console.log(`${(w.bytes / 1024).toFixed(1)} KB ${w.file}${w.over ? ` ** OVER ${limitStr} **` : ''}${w.valid ? '' : ' ** INVALID Swagger 2.0 **'}`); +for (const w of written) console.log(`${(w.bytes / 1024).toFixed(1)} KB ${w.file}${w.over ? ` ** OVER ${limitStr} **` : ''}${!w.valid ? ' ** INVALID Swagger 2.0 **' : ''}${w.unresolved ? ' ** UNRESOLVED {{variables}} **' : ''}`); for (const w of [...new Set(warnings)]) console.warn(`warning: ${w}`); -const bad = written.filter((w) => w.over || !w.valid); -if (bad.length) { - console.error(`\n${bad.length} definition(s) are over ${limitStr} or not valid Swagger 2.0 — split that folder further, trim the collection, or fix the source. See flags above.`); +const bad = written.filter((w) => w.over || !w.valid || w.unresolved); +if (bad.length || hadError) { + console.error(`\n${bad.length + (hadError ? 1 : 0)} problem(s): definitions over ${limitStr}, not valid Swagger 2.0, with unresolved {{variables}}, or a folder that failed to convert — split that folder further, trim the collection, or fix the source. See flags above.`); process.exit(2); } console.log(`\n${written.length} connector definition(s) written to ${OUT}/ — all valid Swagger 2.0, all < ${limitStr}.`); diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/ci.yml.tmpl b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/ci.yml.tmpl index d1ed7b0..97288ab 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/ci.yml.tmpl +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/ci.yml.tmpl @@ -9,6 +9,9 @@ on: paths: ['source/**', 'scripts/**', 'Dockerfile', 'connectors.config.json'] workflow_dispatch: +permissions: + contents: read + jobs: generate: name: generate + validate connector definitions @@ -20,4 +23,4 @@ jobs: run: docker build -t ppc-gen . - name: Generate + self-validate (fails on invalid Swagger 2.0 or over the size limit) - run: docker run --rm --user "$(id -u):$(id -g)" -v "${{ github.workspace }}:/work" -w /work ppc-gen node scripts/generate.mjs + run: docker run --rm --user "$(id -u):$(id -g)" -e HOME=/tmp -v "${{ github.workspace }}:/work" -w /work ppc-gen node scripts/generate.mjs diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl index 7922578..0c5f145 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl @@ -11,6 +11,10 @@ on: - cron: '17 6 * * *' # daily, 06:17 UTC workflow_dispatch: +concurrency: + group: sync-postman + cancel-in-progress: false + permissions: contents: write pull-requests: write @@ -34,12 +38,14 @@ jobs: echo "changed=false" >> "$GITHUB_OUTPUT"; exit 0;; esac # Send the Postman API key only to the real Postman API host (a substring match would also - # hit look-alikes like api.getpostman.com.evil.tld). + # hit look-alikes like api.getpostman.com.evil.tld). --max-redirs 0 fails closed if that + # host ever 3xx-redirects: curl re-sends custom headers across hosts on redirect, so this + # prevents the key leaking to another host. AUTH=() case "$URL" in "https://api.getpostman.com/"*) - [ -n "${POSTMAN_API_KEY:-}" ] && AUTH=(-H "X-Api-Key: ${POSTMAN_API_KEY}");; + [ -n "${POSTMAN_API_KEY:-}" ] && AUTH=(--max-redirs 0 -H "X-Api-Key: ${POSTMAN_API_KEY}");; esac - curl -fsSL --max-time 60 --retry 3 "${AUTH[@]}" "$URL" -o /tmp/fetched.json + curl -fsSL --proto '=https' --proto-redir '=https' --max-time 60 --retry 3 "${AUTH[@]}" "$URL" -o /tmp/fetched.json # Unwrap the Postman API's {"collection": …} envelope if present. if jq -e '.collection' /tmp/fetched.json >/dev/null 2>&1; then jq '.collection' /tmp/fetched.json > /tmp/coll.json @@ -55,8 +61,9 @@ jobs: # Write to the canonical committed path (the generator prefers source/collection.json). mkdir -p source .postman cp /tmp/coll.json source/collection.json - jq -n --arg u "$URL" --arg s "$NEW" --arg t "$(date -u +%FT%TZ)" \ - '{sourceUrl:$u, sha256:$s, updatedAt:$t}' > .postman/manifest.json + # No timestamp in the committed manifest — an unchanged collection would otherwise produce + # a fresh commit every day. Only sha256 gates change detection. + jq -n --arg u "$URL" --arg s "$NEW" '{sourceUrl:$u, sha256:$s}' > .postman/manifest.json echo "changed=true" >> "$GITHUB_OUTPUT" - name: Regenerate + validate INLINE @@ -65,7 +72,7 @@ jobs: if: steps.fetch.outputs.changed == 'true' run: | docker build -t ppc-gen . - docker run --rm --user "$(id -u):$(id -g)" -v "${{ github.workspace }}:/work" -w /work ppc-gen node scripts/generate.mjs + docker run --rm --user "$(id -u):$(id -g)" -e HOME=/tmp -v "${{ github.workspace }}:/work" -w /work ppc-gen node scripts/generate.mjs - name: Open a pull request if: steps.fetch.outputs.changed == 'true' @@ -78,9 +85,15 @@ jobs: git config user.name "github-actions[bot]" git config user.email "41898282+github-actions[bot]@users.noreply.github.com" git checkout -B "$BR" - git add -A + # Stage only the sync outputs, not any other tree state the runner may have. + git add source/collection.json .postman/manifest.json connectors/ git commit -m "chore: sync connector definitions from upstream Postman collection" git push -f -u origin "$BR" - gh pr create --base main --head "$BR" \ - --title "Sync connector definitions from upstream Postman collection" \ - --body "The upstream Postman collection changed. Regenerated OpenAPI 2.0 definitions (validated as Swagger 2.0, each < 1 MB). **Review before importing to Power Platform** — the Postman-to-2.0 conversion is lossy." || true + # Open a PR only if one isn't already open for this branch. Don't swallow other errors + # (e.g. "GitHub Actions is not permitted to create pull requests") — a silently-dead sync + # must fail loudly. + if ! gh pr view "$BR" --json number >/dev/null 2>&1; then + gh pr create --base main --head "$BR" \ + --title "Sync connector definitions from upstream Postman collection" \ + --body "The upstream Postman collection changed. Regenerated OpenAPI 2.0 definitions (validated as Swagger 2.0, each < 1 MB). **Review before importing to Power Platform** — the Postman-to-2.0 conversion is lossy." + fi From 8f2d7d25377db3b5b9f3447b139ecec78bd08f58 Mon Sep 17 00:00:00 2001 From: PBNZ Date: Sat, 4 Jul 2026 02:11:52 +1200 Subject: [PATCH 09/10] fix(power-platform-connectors): scope the unresolved-var scan to connector-critical fields MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follow-up to the C1 fix. The whole-document regex scan would false-fail on a collection that legitimately documents a {{var}} in a description/example. Scope it to the fields Power Platform actually calls — host, basePath, schemes, path keys, and the security URLs / apiKey header name — which is both precise and the exact set swagger-cli can't check (it runs validateFormats:false). Verified empirically: an unresolved env var in an oauth2 tokenUrl with an otherwise-valid host is reported *valid* by swagger-cli alone (exit 0) but flagged by the scoped scan (exit 2, naming oauth2Auth.tokenUrl); a benign {{var}} in a description now passes (exit 0) instead of false-failing. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01DXtxaAQHcsDThJ6BaqwXES --- .../core/scripts/generate.mjs | 32 ++++++++++++++----- 1 file changed, 24 insertions(+), 8 deletions(-) diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs index d1e1d32..0cd101c 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs @@ -226,20 +226,36 @@ function uniqueSlug(name) { usedNames.add(candidate); return candidate; } +// Report unresolved Postman tokens ({{var}} or its %7b%7b URL-encoding) ONLY in the fields Power +// Platform actually calls: host, basePath, schemes, path keys, and the security URLs / header name. +// A leftover in a description or example doesn't break the connector, so scanning the whole document +// would false-fail on collections that legitimately document a {{var}}. swagger-cli can't catch +// these itself — it validates with validateFormats:false, so an unresolved host or tokenUrl passes. +const TOKEN = /\{\{|%7[bB]%7[bB]/; +function unresolvedFields(sw) { + const hits = []; + const chk = (label, val) => { if (typeof val === 'string' && TOKEN.test(val)) hits.push(label); }; + chk('host', sw.host); + chk('basePath', sw.basePath); + (sw.schemes || []).forEach((s, i) => chk(`schemes[${i}]`, s)); + for (const k of Object.keys(sw.paths || {})) chk(`path "${k}"`, k); + for (const [n, d] of Object.entries(sw.securityDefinitions || {})) { + chk(`${n}.authorizationUrl`, d?.authorizationUrl); + chk(`${n}.tokenUrl`, d?.tokenUrl); + chk(`${n}.name`, d?.name); + } + return hits; +} function emit(name, sw) { let bytes = sizeOf(sw); if (bytes >= LIMIT) { strip(sw); bytes = sizeOf(sw); } // last-ditch shrink for an oversize def const file = join(OUT, `${uniqueSlug(name)}.swagger.json`); - const serialized = JSON.stringify(sw, null, 2); - writeFileSync(file, serialized); + writeFileSync(file, JSON.stringify(sw, null, 2)); let valid = true; - // Residual Postman template tokens — {{var}} or its %7b%7b URL-encoding — mean a variable never - // resolved, almost always a Postman *environment* variable that isn't in the committed - // collection.json. swagger-cli validates with validateFormats:false, so a host/URL like - // "%7b%7bbaseurl%7d%7d" would otherwise pass and the connector would ship broken. - const unresolved = /\{\{|%7[bB]%7[bB]/.test(serialized); + const hits = unresolvedFields(sw); + const unresolved = hits.length > 0; if (unresolved) - warnings.push(`${file}: unresolved {{variables}} — likely a Postman environment variable not in the committed collection; the connector will not work until these are provided`); + warnings.push(`${file}: unresolved {{variables}} in ${hits.join(', ')} — likely a Postman environment variable not in the committed collection; the connector will not work until these are provided`); try { execFileSync('swagger-cli', ['validate', file], { stdio: 'pipe' }); } catch (err) { valid = false; warnings.push(`validation failed for ${file}: ${(err.stderr?.toString() || err.message || '').trim()}`); } written.push({ file, bytes, over: bytes >= LIMIT, valid, unresolved }); From 3c53aa8ab57ff0270f8271e30a399d5e628fb6e7 Mon Sep 17 00:00:00 2001 From: PBNZ Date: Sat, 4 Jul 2026 02:31:33 +1200 Subject: [PATCH 10/10] fix(power-platform-connectors): address final-round review (1 HIGH + 3 MEDIUM) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From the final Gemini review round: - HIGH (sync.yml): use `gh pr list --head --state open` instead of `gh pr view` to decide whether a sync PR already exists. `gh pr view` returns the branch's most recent PR even when merged/closed, so after the first sync PR merged the workflow would wrongly assume a PR still existed and never open another — silently dropping all future upstream changes. (Regression from the previous round's PR-guard; now open-only.) - MEDIUM (sync.yml): also match api.postman.com and api.eu.postman.com (EU data residency) when attaching the API key — both verified as official Postman API hosts; the trailing-"/" anchor still blocks look-alikes. - MEDIUM (generate.mjs): Array.isArray guards on op.parameters (fixPaths) and sw.schemes (the new unresolved-var scan) so malformed converter output can't throw a TypeError; plus a typeof-object guard on op.responses. Verified: Test A (client_credentials) still exits 0; all 5 repo validators green. Copilot produced no new feedback this round. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01DXtxaAQHcsDThJ6BaqwXES --- .../core/scripts/generate.mjs | 6 ++--- .../public/.github/workflows/sync.yml.tmpl | 26 ++++++++++++------- 2 files changed, 19 insertions(+), 13 deletions(-) diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs index 0cd101c..92485a3 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs @@ -134,7 +134,7 @@ function fixResponses(sw) { if (!path || typeof path !== 'object') continue; for (const m of METHODS) { const op = path[m]; - if (op && typeof op === 'object' && op.responses) + if (op && typeof op === 'object' && op.responses && typeof op.responses === 'object') for (const [code, r] of Object.entries(op.responses)) if (r && typeof r === 'object' && !r.$ref && !r.description) r.description = REASON[code] || 'Response'; } @@ -153,7 +153,7 @@ function fixPaths(sw) { for (const m of METHODS) { const op = item[m]; if (op && typeof op === 'object') { - op.parameters = op.parameters || []; + if (!Array.isArray(op.parameters)) op.parameters = []; // tolerate a malformed non-array for (const t of tokens) if (!op.parameters.some((p) => p.in === 'path' && p.name === t)) op.parameters.push({ name: t, in: 'path', required: true, type: 'string' }); @@ -237,7 +237,7 @@ function unresolvedFields(sw) { const chk = (label, val) => { if (typeof val === 'string' && TOKEN.test(val)) hits.push(label); }; chk('host', sw.host); chk('basePath', sw.basePath); - (sw.schemes || []).forEach((s, i) => chk(`schemes[${i}]`, s)); + (Array.isArray(sw.schemes) ? sw.schemes : []).forEach((s, i) => chk(`schemes[${i}]`, s)); for (const k of Object.keys(sw.paths || {})) chk(`path "${k}"`, k); for (const [n, d] of Object.entries(sw.securityDefinitions || {})) { chk(`${n}.authorizationUrl`, d?.authorizationUrl); diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl index 0c5f145..26c21c1 100644 --- a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl @@ -37,13 +37,16 @@ jobs: echo "sourceUrl is not set in connectors.config.json — skipping." echo "changed=false" >> "$GITHUB_OUTPUT"; exit 0;; esac - # Send the Postman API key only to the real Postman API host (a substring match would also - # hit look-alikes like api.getpostman.com.evil.tld). --max-redirs 0 fails closed if that - # host ever 3xx-redirects: curl re-sends custom headers across hosts on redirect, so this - # prevents the key leaking to another host. + # Send the Postman API key only to a genuine Postman API host (all three verified official: + # api.getpostman.com legacy, api.postman.com standard, api.eu.postman.com EU data + # residency). The trailing "/" anchors each match so a look-alike like + # api.getpostman.com.evil.tld cannot match. --max-redirs 0 fails closed if a host ever + # 3xx-redirects: curl re-sends custom headers across hosts on redirect, so this prevents + # the key leaking to another host. AUTH=() - case "$URL" in "https://api.getpostman.com/"*) - [ -n "${POSTMAN_API_KEY:-}" ] && AUTH=(--max-redirs 0 -H "X-Api-Key: ${POSTMAN_API_KEY}");; + case "$URL" in + "https://api.getpostman.com/"* | "https://api.postman.com/"* | "https://api.eu.postman.com/"*) + [ -n "${POSTMAN_API_KEY:-}" ] && AUTH=(--max-redirs 0 -H "X-Api-Key: ${POSTMAN_API_KEY}");; esac curl -fsSL --proto '=https' --proto-redir '=https' --max-time 60 --retry 3 "${AUTH[@]}" "$URL" -o /tmp/fetched.json # Unwrap the Postman API's {"collection": …} envelope if present. @@ -89,10 +92,13 @@ jobs: git add source/collection.json .postman/manifest.json connectors/ git commit -m "chore: sync connector definitions from upstream Postman collection" git push -f -u origin "$BR" - # Open a PR only if one isn't already open for this branch. Don't swallow other errors - # (e.g. "GitHub Actions is not permitted to create pull requests") — a silently-dead sync - # must fail loudly. - if ! gh pr view "$BR" --json number >/dev/null 2>&1; then + # Open a PR only if one isn't already OPEN for this branch. Use `gh pr list` (open PRs + # only), NOT `gh pr view` — the latter returns the branch's most recent PR even when it is + # merged/closed, so after the first sync PR merges we'd wrongly think one still exists and + # never open another. Command substitution under `set -e` still fails loudly if gh errors + # (e.g. "GitHub Actions is not permitted to create pull requests") — a dead sync must be visible. + OPEN=$(gh pr list --head "$BR" --state open --json number --jq 'length') + if [ "${OPEN:-0}" -eq 0 ]; then gh pr create --base main --head "$BR" \ --title "Sync connector definitions from upstream Postman collection" \ --body "The upstream Postman collection changed. Regenerated OpenAPI 2.0 definitions (validated as Swagger 2.0, each < 1 MB). **Review before importing to Power Platform** — the Postman-to-2.0 conversion is lossy."