diff --git a/CHANGELOG.md b/CHANGELOG.md index 919bf28..ebfcc5d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - A `docker-compose` repo type for `/new-repo`: a minimal `compose.yaml` (named volumes for data, committed config, `.env` secrets), `.env.example`, `.dockerignore`, a README with a workflow diagram, and a `docker compose config -q` validation CI (Public tier). +- A `power-platform-connectors` repo type for `/new-repo`: turns a committed Postman collection + into **OpenAPI 2.0** custom-connector definitions for Microsoft Power Platform. A pinned-Docker + generator (`postman-to-openapi` + `api-spec-converter`, normalised to valid Swagger 2.0) splits + per top-level folder **only when a single definition would exceed the 1 MB limit**, self-validates + every output, and a scheduled sync workflow opens a PR when the upstream collection changes. The + collection is committed, so the repo builds with just Docker — no Postman account. - The `repo-standard` skill now tells agents to check the remote after a push/PR — CI/Actions status and GitHub Copilot / reviewer feedback — before calling work done. ### Changed diff --git a/README.md b/README.md index ee0d172..9ed06b6 100644 --- a/README.md +++ b/README.md @@ -90,8 +90,9 @@ make the first commit. Nothing is pushed or published — you do that when you'r ## Profiles -- **Type — what the repo *is*:** `powershell-module` and `docker-compose` (built out), plus `skill-plugin`, - `collection`, `mcp-server`, `app-ts`, `app-python`, `script-collection` (stubs, filled as needed). +- **Type — what the repo *is*:** `powershell-module`, `docker-compose`, and + `power-platform-connectors` (built out), plus `skill-plugin`, `collection`, `mcp-server`, + `app-ts`, `app-python`, `script-collection` (stubs, filled as needed). - **Tier — ceremony, set by visibility:** **Core** (every repo), **+Public**, **+Published**. See [`the standard`](plugins/repokit/skills/repo-standard/standard/the-standard.md) for the full diff --git a/docs/adr/0003-power-platform-connectors-type.md b/docs/adr/0003-power-platform-connectors-type.md new file mode 100644 index 0000000..bc78caf --- /dev/null +++ b/docs/adr/0003-power-platform-connectors-type.md @@ -0,0 +1,55 @@ +# ADR-0003: `power-platform-connectors` repo type + +- **Status:** accepted +- **Date:** 2026-07-04 + +## Context + +We needed a repo **type** that turns a Postman collection into OpenAPI definitions ready to import +as **Microsoft Power Platform custom connectors**. Verified hard constraints from Microsoft Learn +(*Create a custom connector from an OpenAPI definition*): the definition must be **OpenAPI 2.0 / +Swagger** ("OpenAPI 3.0 format is not supported") and **< 1 MB**, `.json` or `.yaml`, with a single +top security definition (client-credentials OAuth is rejected). + +## Decision + +- **Converter (free, pinned in Docker):** `postman-to-openapi@3.0.1` outputs OpenAPI **3.0** only, + so we downconvert with `api-spec-converter@2.12.0` (`--from=openapi_3 --to=swagger_2`). Both are + unmaintained (2023 / 2021; the latter depends on the deprecated `request`) and do **not** run on + current Node — the image is pinned to **Node 18**. Do not bump without re-testing. APIMatic (paid, + direct Postman→2.0) is the documented escape hatch, not built. +- **The downconvert is lossy, so `generate.mjs` normalises the output to valid Swagger 2.0** — this + was proven empirically (a real public collection failed validation four different ways before the + fixups): (1) **pre-resolve collection `{{variables}}`** so `host`/`basePath`/`schemes` are real + instead of `%7B%7Bbaseurl%7D%7D`; (2) **derive `securityDefinitions` from the Postman `auth`** + block (p2o mis-maps apikey to an invalid `type:http` and drops the header name); (3) **backfill a + `description` on every response** (required in 2.0; p2o only sets it from the Postman `status`); + (4) **add missing path parameters and leading `/`** on path keys. The generator then + **self-validates** each output with `swagger-cli` and asserts **< 1 MB**, exiting non-zero on any + failure. +- **Split policy:** convert the whole collection to one definition; ship one file if it's < 1 MB; + otherwise split **per top-level folder** (each carrying the collection `variable`/`auth`); if a + single folder is still ≥ 1 MB after stripping example fields, **flag it** rather than ship an + un-importable file. +- **Self-containment / source:** the collection is **committed** (`source/collection.json`), so the + repo builds with just Docker — no Postman account. **Sync** fetches a **configurable `sourceUrl`**: + a public URL (GitHub raw / vendor site) is **account-free**; a Postman-platform-only collection + needs the maintainer's optional `POSTMAN_API_KEY` secret (never needed by cloners). Postman has no + reliable anonymous fetch for a collection you don't own (public JSON links are deprecated). +- **Change detection = scheduled workflow + hash:** `sync.yml` (daily cron + dispatch) canonicalises + the fetched collection, SHA-256s it, compares to `.postman/manifest.json`, and on a change updates + the snapshot, regenerates, **validates inline**, and opens a PR. Validation is inline because a PR + opened by the default `GITHUB_TOKEN` does not trigger `ci.yml`. This needs + `permissions: pull-requests: write` and the repo setting *Allow GitHub Actions to create and + approve pull requests*. +- **Tier: Core + Public, no Published** (like `docker-compose`) — "publishing" is manually importing + a definition into Power Platform; there is no registry step to automate. + +## Consequences + +- The conversion is lossy — the pipeline validates and opens a **PR to review**; it never + auto-imports. Richer collections (saved example responses, one clear auth scheme) produce better + connectors. +- The toolchain is stale but pinned in Docker; if a future Node breaks it, the pin holds. +- Auto-sync is account-free for public-URL sources; Postman-platform-only collections need the + maintainer's optional key. diff --git a/plugins/repokit/skills/new-repo/SKILL.md b/plugins/repokit/skills/new-repo/SKILL.md index dc7dc25..ab6bd40 100644 --- a/plugins/repokit/skills/new-repo/SKILL.md +++ b/plugins/repokit/skills/new-repo/SKILL.md @@ -26,14 +26,14 @@ Gather these from the user's arguments / message, else ask — keep it to load-b - **name** — the repo / directory name (kebab-case). - **description** — one line. -- **type** — one of: `powershell-module`, `docker-compose`, `skill-plugin`, `collection`, `mcp-server`, `app-ts`, - `app-python`, `script-collection`. +- **type** — one of: `powershell-module`, `docker-compose`, `power-platform-connectors`, + `skill-plugin`, `collection`, `mcp-server`, `app-ts`, `app-python`, `script-collection`. - **visibility** — `private` (= Core tier), `public` (= +Public), or `published` (= +Published). - **author** — default `Peter Braun` (`PBNZ`). - **license** — default `Apache-2.0`. - For `powershell-module` only: **ModuleName** (PascalCase, e.g. `MyModule`). -If the chosen type is a **stub** (anything other than `powershell-module` or `docker-compose`), tell the user so: the +If the chosen type is a **stub** (anything other than `powershell-module`, `docker-compose`, or `power-platform-connectors`), tell the user so: the Core/Public/Published files get stamped, but there's no type-specific structure yet. Confirm they want to continue. diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/.postman/manifest.json b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/.postman/manifest.json new file mode 100644 index 0000000..27e981a --- /dev/null +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/.postman/manifest.json @@ -0,0 +1,4 @@ +{ + "sourceUrl": "", + "sha256": "" +} diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/Dockerfile b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/Dockerfile new file mode 100644 index 0000000..feda011 --- /dev/null +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/Dockerfile @@ -0,0 +1,14 @@ +# Pinned toolchain for generating Power Platform custom-connector definitions. +# +# Node is pinned to 18 on purpose: the converters are unmaintained +# (postman-to-openapi 2023, api-spec-converter 2021 — the latter depends on the +# deprecated `request` module) and do NOT run on current Node. Do not bump the +# Node line or the tool versions without re-testing the whole pipeline. +FROM node:18-bullseye-slim + +# --ignore-scripts blocks install-time lifecycle scripts (defence-in-depth for this stale, +# transitive-heavy dependency tree). These three tools are pure JS and run fine without them. +RUN npm install -g --ignore-scripts postman-to-openapi@3.0.1 api-spec-converter@2.12.0 @apidevtools/swagger-cli@4.0.4 + +WORKDIR /work +# The repo is mounted at /work at run time; run: node scripts/generate.mjs diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/README.md.tmpl b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/README.md.tmpl new file mode 100644 index 0000000..8d32a7d --- /dev/null +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/README.md.tmpl @@ -0,0 +1,65 @@ +# {{name}} + +{{description}} + +OpenAPI **2.0** connector definitions for **Microsoft Power Platform** custom connectors, +generated from a Postman collection. Each definition is valid Swagger 2.0 and kept **under 1 MB** +(Power Platform's limits — 3.0 is not supported). The collection is **committed to this repo**, so +anyone can regenerate the connectors with just Docker — no Postman account needed. + +## How it works + +```mermaid +flowchart LR + Src["source/collection.json
(committed Postman collection)"] --> Gen["generate.mjs, in Docker"] + Gen --> Q{"whole def under 1 MB?"} + Q -->|yes| One["one connectors/*.swagger.json"] + Q -->|no| Split["split per top-level folder"] + One --> Val["validate: Swagger 2.0 + under 1 MB"] + Split --> Val + Val --> Imp["import into Power Platform"] +``` + +## Use it + +1. Export your Postman collection (v2.1) and save it as **`source/collection.json`**. +2. Set `sourceUrl` in **`connectors.config.json`** — a public URL to the collection JSON (a GitHub + raw file or the vendor's site), or the Postman API endpoint if the collection lives only in + Postman. +3. Generate the definitions (one Docker command — no local Node needed): + + ```sh + docker build -t {{name}}-gen . + docker run --rm -v "${PWD}:/work" {{name}}-gen node scripts/generate.mjs + ``` + + *(On Linux/macOS, add `--user "$(id -u):$(id -g)"` to the `docker run` so the generated files + aren't owned by root.)* + + It writes `connectors/*.swagger.json` — **one file if the whole collection fits under 1 MB, + otherwise one per top-level folder**. Every file is validated as Swagger 2.0 and checked < 1 MB; + the run fails if any definition is invalid or oversize. +4. In Power Platform (Power Automate / Power Apps / Logic Apps): **New custom connector → Import an + OpenAPI file** → pick a `connectors/*.swagger.json`. + +## Keeping it up to date + +- **CI** (`.github/workflows/ci.yml`) rebuilds + regenerates + validates on every change to + `source/`, `scripts/`, the `Dockerfile`, or the config. +- **Auto-sync** (`.github/workflows/sync.yml`) runs daily: it fetches `sourceUrl`, and if the + upstream collection changed, it updates `source/collection.json`, regenerates, validates, and + opens a **PR** for you to review. Account-free when `sourceUrl` is a public URL. If the collection + lives only in Postman, add a `POSTMAN_API_KEY` repo secret (only the maintainer needs it — never + cloners), and enable *Settings → Actions → Allow GitHub Actions to create and approve pull requests*. + +## Heads-up — review before importing + +The Postman → Swagger 2.0 conversion is **lossy** (it drops `oneOf/anyOf`, `nullable`, and derives +response schemas from saved example responses). The pipeline fixes the common breakages +(host/basePath, security definitions, response descriptions) and validates every output, but +**review the connector in the PR before importing**. Richer collections — with saved example +responses and a single clear auth scheme — produce better connectors. Power Platform also picks the +single top security definition and rejects OAuth client-credentials. + +See [`AGENTS.md`](AGENTS.md) for the START-HERE map. Follows the +[RepoKit](https://github.com/PBNZ/repo-kit) standard. diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/connectors.config.json b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/connectors.config.json new file mode 100644 index 0000000..e09328e --- /dev/null +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/connectors.config.json @@ -0,0 +1,5 @@ +{ + "sourceUrl": "", + "sizeLimitBytes": 1048576, + "output": "connectors" +} diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/connectors/.gitkeep b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/connectors/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs new file mode 100644 index 0000000..92485a3 --- /dev/null +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/scripts/generate.mjs @@ -0,0 +1,318 @@ +// Generate Power Platform custom-connector definitions (OpenAPI 2.0 / Swagger) +// from the committed Postman collection in source/. +// +// Postman collection -> resolve {{vars}} -> postman-to-openapi (3.0) +// -> api-spec-converter (Swagger 2.0) -> fix securityDefinitions from the +// Postman auth -> measure -> split per top-level folder ONLY if a single +// definition would be >= the size limit. +// +// Runs inside the pinned Docker image (p2o + api-spec-converter on PATH). + +import { readFileSync, writeFileSync, mkdirSync, mkdtempSync, readdirSync, rmSync, existsSync } from 'node:fs'; +import { execFileSync } from 'node:child_process'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; + +const cfg = JSON.parse(readFileSync('connectors.config.json', 'utf8')); +const LIMIT = cfg.sizeLimitBytes ?? 1048576; +const limitStr = LIMIT >= 1048576 ? `${+(LIMIT / 1048576).toFixed(2)} MB` : `${Math.round(LIMIT / 1024)} KB`; +const OUT = cfg.output ?? 'connectors'; +const work = mkdtempSync(join(tmpdir(), 'ppc-')); // unique per run; removed on exit / interrupt +const cleanup = () => { try { rmSync(work, { recursive: true, force: true }); } catch { /* best-effort */ } }; +process.on('exit', cleanup); +for (const sig of ['SIGINT', 'SIGTERM']) process.on(sig, () => { cleanup(); process.exit(1); }); +const warnings = []; +let hadError = false; // set when a (sub)collection fails to convert, so we still exit non-zero + +// --- locate + load the committed collection (prefer the documented source/collection.json) --- +const jsons = existsSync('source') ? readdirSync('source').filter((f) => f.endsWith('.json')).sort() : []; +const srcName = jsons.includes('collection.json') ? 'collection.json' : jsons[0]; +if (!srcName) { + console.log('source/ has no *.json collection yet — export your Postman collection to source/collection.json. Nothing to do.'); + process.exit(0); +} +const rawObj = JSON.parse(readFileSync(join('source', srcName), 'utf8')); +const collection = rawObj.collection ?? rawObj; // unwrap Postman API {"collection":…} +const rootVars = collection.variable || []; +const rootAuth = collection.auth || null; + +// --- helpers --- +const sizeOf = (o) => Buffer.byteLength(JSON.stringify(o, null, 2)); +const slug = (s) => (s || 'connector').toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/(^-|-$)/g, '') || 'connector'; +const kv = (arr, k) => arr?.find((e) => e.key === k)?.value; + +// Detect auth set only on individual requests. We map collection/folder-level auth, not per-request +// auth, so without this a request-auth-only collection would ship with no securityDefinitions and no +// warning at all. +function hasRequestAuth(items) { + for (const it of items || []) { + if (it?.request?.auth?.type && it.request.auth.type !== 'noauth') return true; + if (it?.item && hasRequestAuth(it.item)) return true; + } + return false; +} + +// Pre-resolve collection variables that are set (including "" and "0") so host/basePath come out +// real instead of "%7B%7Bbaseurl%7D%7D". Walk the parsed object and substitute inside string values +// only — safe against values containing quotes/backslashes/newlines (a raw string-replace on the +// serialized JSON could produce invalid JSON). +function resolveVars(obj, vars) { + if (typeof obj === 'string') { + let s = obj; + // Repeat until stable (bounded) so a nested var resolves regardless of its order in vars[]: + // baseUrl="{{proto}}://host" needs a second pass to also resolve {{proto}}. + for (let pass = 0; pass < 5 && s.includes('{{'); pass++) { + const before = s; + for (const v of vars) if (v && v.key && v.value != null) s = s.split(`{{${v.key}}}`).join(String(v.value)); + if (s === before) break; + } + return s; + } + if (Array.isArray(obj)) return obj.map((x) => resolveVars(x, vars)); + if (obj && typeof obj === 'object') { const r = {}; for (const [k, val] of Object.entries(obj)) r[k] = resolveVars(val, vars); return r; } + return obj; +} + +// Map a Postman auth block to a valid Swagger 2.0 security definition. p2o mis-maps these +// (e.g. apikey -> {type:http, scheme:apikey}, which is invalid 2.0 AND drops the header name), +// so we build them from the source collection instead. +function pmAuthToSwagger(auth) { + if (!auth || !auth.type || auth.type === 'noauth') return null; // noauth = intentional no security + const attrs = (k) => (Array.isArray(auth[k]) ? auth[k] : []); // v2.0 object-form auth would crash kv() + switch (auth.type) { + case 'apikey': { + const a = attrs('apikey'); + const loc = (kv(a, 'in') || 'header').toLowerCase() === 'query' ? 'query' : 'header'; + return { name: 'apiKeyAuth', def: { type: 'apiKey', name: kv(a, 'key') || 'Authorization', in: loc } }; + } + case 'bearer': + return { name: 'bearerAuth', def: { type: 'apiKey', name: 'Authorization', in: 'header' } }; + case 'basic': + return { name: 'basicAuth', def: { type: 'basic' } }; + case 'oauth2': { + const a = attrs('oauth2'); + const authorizationUrl = kv(a, 'authUrl'); + const tokenUrl = kv(a, 'accessTokenUrl'); + const grant = (kv(a, 'grant_type') || '').toLowerCase(); + // Pick the Swagger 2.0 oauth2 flow. Prefer Postman's grant_type (substring match — robust to + // the exact string), else infer from which URLs are present. Emit only the URL(s) that flow + // requires, verified against the OpenAPI 2.0 spec: implicit->authorizationUrl, + // password/application->tokenUrl, accessCode->both. (Postman's earlier hardcoded-accessCode + // path shipped a NO-auth connector for client-credentials/password/implicit grants.) + let flow; + if (grant.includes('client')) flow = 'application'; + else if (grant.includes('password')) flow = 'password'; + else if (grant.includes('implicit')) flow = 'implicit'; + else if (grant.includes('authorization') || grant.includes('code')) flow = 'accessCode'; + else if (authorizationUrl && tokenUrl) flow = 'accessCode'; + else if (tokenUrl) flow = 'application'; + else if (authorizationUrl) flow = 'implicit'; + else return null; // no grant_type and no URLs — nothing usable + const def = { type: 'oauth2', flow, scopes: {} }; + if (flow === 'accessCode' || flow === 'implicit') { + if (!authorizationUrl) return null; + def.authorizationUrl = authorizationUrl; + } + if (flow === 'accessCode' || flow === 'application' || flow === 'password') { + if (!tokenUrl) return null; + def.tokenUrl = tokenUrl; + } + return { name: 'oauth2Auth', def }; + } + default: + return null; + } +} + +// Swagger 2.0 requires every response to have a `description`. p2o only sets it from the Postman +// response's `status` (reason phrase); collections that omit `status` yield a description-less +// (invalid) response. Backfill a sensible one so the output always validates. +const REASON = { 200: 'OK', 201: 'Created', 202: 'Accepted', 203: 'Non-Authoritative Information', 204: 'No Content', 206: 'Partial Content', 301: 'Moved Permanently', 302: 'Found', 304: 'Not Modified', 400: 'Bad Request', 401: 'Unauthorized', 403: 'Forbidden', 404: 'Not Found', 405: 'Method Not Allowed', 409: 'Conflict', 422: 'Unprocessable Entity', 429: 'Too Many Requests', 500: 'Internal Server Error', 502: 'Bad Gateway', 503: 'Service Unavailable' }; +const METHODS = ['get', 'put', 'post', 'delete', 'options', 'head', 'patch']; +function fixResponses(sw) { + for (const path of Object.values(sw.paths || {})) { + if (!path || typeof path !== 'object') continue; + for (const m of METHODS) { + const op = path[m]; + if (op && typeof op === 'object' && op.responses && typeof op.responses === 'object') + for (const [code, r] of Object.entries(op.responses)) + if (r && typeof r === 'object' && !r.$ref && !r.description) r.description = REASON[code] || 'Response'; + } + } +} + +// Swagger 2.0 requires (a) every path key to start with "/", and (b) every "{token}" in a path to +// have a matching path parameter. p2o sometimes emits neither. Backfill both. +function fixPaths(sw) { + if (!sw.paths) return; + const fixed = {}; + for (const [key, item] of Object.entries(sw.paths)) { + const pathKey = key.startsWith('/') ? key : '/' + key; + const tokens = [...pathKey.matchAll(/\{([^}]+)\}/g)].map((m) => m[1]); + if (tokens.length && item && typeof item === 'object') { + for (const m of METHODS) { + const op = item[m]; + if (op && typeof op === 'object') { + if (!Array.isArray(op.parameters)) op.parameters = []; // tolerate a malformed non-array + for (const t of tokens) + if (!op.parameters.some((p) => p.in === 'path' && p.name === t)) + op.parameters.push({ name: t, in: 'path', required: true, type: 'string' }); + } + } + } + fixed[pathKey] = item; + } + sw.paths = fixed; +} + +// Replace whatever the converter produced with a correct security definition derived from the +// Postman auth (Power Platform picks the single top securityDefinition, so we emit exactly one). +function fixSecurity(sw, auth) { + // Resolve committed {{vars}} in the auth block too — otherwise a variable used as an OAuth URL or + // an apiKey header name would leak into securityDefinitions literally (and swagger-cli validates + // with validateFormats:false, so a bad URL would pass silently). + const s = pmAuthToSwagger(resolveVars(auth, rootVars)); + if (s) { + sw.securityDefinitions = { [s.name]: s.def }; + sw.security = [{ [s.name]: [] }]; + } else { + if (auth?.type && auth.type !== 'noauth') + warnings.push(`auth type "${auth.type}" not fully mapped (missing URL/grant details?) — add security manually in the connector`); + delete sw.securityDefinitions; // never ship an invalid securityDefinitions block + delete sw.security; + } + // Drop any operation-level security the converter emitted — it points at the old (now replaced + // or deleted) definition, i.e. a dangling reference. The single root-level security applies to + // every operation. + for (const path of Object.values(sw.paths || {})) { + if (!path || typeof path !== 'object') continue; + for (const m of METHODS) if (path[m] && typeof path[m] === 'object') delete path[m].security; + } +} + +// One Postman (sub)collection -> validated Swagger 2.0 object. +function convert(coll, tag, effectiveAuth) { + const pin = join(work, `${tag}.postman.json`); + const oas = join(work, `${tag}.oas3.yml`); + writeFileSync(pin, JSON.stringify(resolveVars(coll, coll.variable?.length ? coll.variable : rootVars))); + // Show the converters' own warnings/errors (stderr) so a bad collection is debuggable; still + // capture api-spec-converter's stdout (the Swagger JSON). + execFileSync('p2o', [pin, '-f', oas], { stdio: ['ignore', 'ignore', 'inherit'] }); + const raw = execFileSync('api-spec-converter', ['--from=openapi_3', '--to=swagger_2', '--syntax=json', oas], { stdio: ['ignore', 'pipe', 'inherit'] }); + const sw = JSON.parse(raw.toString()); + fixPaths(sw); + fixResponses(sw); + fixSecurity(sw, effectiveAuth); + return sw; +} + +// Strip verbose example fields (shrinks size; also improves Power Platform compatibility). +function strip(o) { + if (Array.isArray(o)) o.forEach(strip); + else if (o && typeof o === 'object') { delete o.example; delete o.examples; for (const k of Object.keys(o)) strip(o[k]); } + return o; +} + +// --- fresh output dir --- +mkdirSync(OUT, { recursive: true }); +for (const f of readdirSync(OUT)) if (f.endsWith('.swagger.json')) rmSync(join(OUT, f)); + +const written = []; +const usedNames = new Set(); +function uniqueSlug(name) { + const base = slug(name); + let candidate = base, n = 1; + while (usedNames.has(candidate)) candidate = `${base}-${++n}`; // avoids cross-collisions with real "…-2" names + usedNames.add(candidate); + return candidate; +} +// Report unresolved Postman tokens ({{var}} or its %7b%7b URL-encoding) ONLY in the fields Power +// Platform actually calls: host, basePath, schemes, path keys, and the security URLs / header name. +// A leftover in a description or example doesn't break the connector, so scanning the whole document +// would false-fail on collections that legitimately document a {{var}}. swagger-cli can't catch +// these itself — it validates with validateFormats:false, so an unresolved host or tokenUrl passes. +const TOKEN = /\{\{|%7[bB]%7[bB]/; +function unresolvedFields(sw) { + const hits = []; + const chk = (label, val) => { if (typeof val === 'string' && TOKEN.test(val)) hits.push(label); }; + chk('host', sw.host); + chk('basePath', sw.basePath); + (Array.isArray(sw.schemes) ? sw.schemes : []).forEach((s, i) => chk(`schemes[${i}]`, s)); + for (const k of Object.keys(sw.paths || {})) chk(`path "${k}"`, k); + for (const [n, d] of Object.entries(sw.securityDefinitions || {})) { + chk(`${n}.authorizationUrl`, d?.authorizationUrl); + chk(`${n}.tokenUrl`, d?.tokenUrl); + chk(`${n}.name`, d?.name); + } + return hits; +} +function emit(name, sw) { + let bytes = sizeOf(sw); + if (bytes >= LIMIT) { strip(sw); bytes = sizeOf(sw); } // last-ditch shrink for an oversize def + const file = join(OUT, `${uniqueSlug(name)}.swagger.json`); + writeFileSync(file, JSON.stringify(sw, null, 2)); + let valid = true; + const hits = unresolvedFields(sw); + const unresolved = hits.length > 0; + if (unresolved) + warnings.push(`${file}: unresolved {{variables}} in ${hits.join(', ')} — likely a Postman environment variable not in the committed collection; the connector will not work until these are provided`); + try { execFileSync('swagger-cli', ['validate', file], { stdio: 'pipe' }); } + catch (err) { valid = false; warnings.push(`validation failed for ${file}: ${(err.stderr?.toString() || err.message || '').trim()}`); } + written.push({ file, bytes, over: bytes >= LIMIT, valid, unresolved }); +} + +// Convert one (sub)collection and emit it; on converter failure, record it and keep going so one +// bad folder doesn't abort the rest of the split (p2o / api-spec-converter throw on any non-zero). +function tryEmit(name, coll, tag, auth) { + try { + emit(name, convert(coll, tag, auth)); + } catch (err) { + hadError = true; + warnings.push(`"${name}" failed to convert: ${(err.message || err).toString().trim()}`); + } +} + +// H3: auth on individual requests is not mapped — warn rather than silently ship an authless connector. +if (!rootAuth && hasRequestAuth(collection.item)) { + warnings.push('auth is set per-request in this collection; only collection/folder-level auth is mapped, so the connector(s) have no securityDefinitions — add auth manually after import'); +} + +// --- convert whole; split only if it *still* wouldn't fit after dropping examples --- +let whole = null, wholeBytes = Infinity; +try { + whole = convert(collection, 'whole', rootAuth); + wholeBytes = sizeOf(whole); + if (wholeBytes >= LIMIT) { strip(whole); wholeBytes = sizeOf(whole); } // try examples-off before splitting +} catch (err) { + warnings.push(`whole-collection conversion failed (${(err.message || err).toString().trim()}); falling back to per-folder`); +} +if (whole && wholeBytes < LIMIT) { + emit(collection.info?.name || srcName.replace(/\.json$/, ''), whole); +} else { + const roots = collection.item || []; + for (const folder of roots.filter((it) => it && it.item)) { + const fname = folder.name || 'folder'; + const sub = { + info: { ...collection.info, name: `${collection.info?.name || ''} - ${fname}`.trim() }, + variable: rootVars, + auth: folder.auth || rootAuth, + item: folder.item, + }; + tryEmit(fname, sub, slug(fname), folder.auth || rootAuth); + } + const loose = roots.filter((it) => it && it.request); + if (loose.length) { + const sub = { info: { ...collection.info, name: `${collection.info?.name || ''} - misc` }, variable: rootVars, auth: rootAuth, item: loose }; + tryEmit('misc', sub, 'misc', rootAuth); + } +} + +// --- report --- +for (const w of written) console.log(`${(w.bytes / 1024).toFixed(1)} KB ${w.file}${w.over ? ` ** OVER ${limitStr} **` : ''}${!w.valid ? ' ** INVALID Swagger 2.0 **' : ''}${w.unresolved ? ' ** UNRESOLVED {{variables}} **' : ''}`); +for (const w of [...new Set(warnings)]) console.warn(`warning: ${w}`); +const bad = written.filter((w) => w.over || !w.valid || w.unresolved); +if (bad.length || hadError) { + console.error(`\n${bad.length + (hadError ? 1 : 0)} problem(s): definitions over ${limitStr}, not valid Swagger 2.0, with unresolved {{variables}}, or a folder that failed to convert — split that folder further, trim the collection, or fix the source. See flags above.`); + process.exit(2); +} +console.log(`\n${written.length} connector definition(s) written to ${OUT}/ — all valid Swagger 2.0, all < ${limitStr}.`); diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/source/.gitkeep b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/core/source/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/ci.yml.tmpl b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/ci.yml.tmpl new file mode 100644 index 0000000..97288ab --- /dev/null +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/ci.yml.tmpl @@ -0,0 +1,26 @@ +name: CI + +on: + push: + branches: [main] + paths: ['source/**', 'scripts/**', 'Dockerfile', 'connectors.config.json'] + pull_request: + branches: [main] + paths: ['source/**', 'scripts/**', 'Dockerfile', 'connectors.config.json'] + workflow_dispatch: + +permissions: + contents: read + +jobs: + generate: + name: generate + validate connector definitions + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Build the pinned toolchain image + run: docker build -t ppc-gen . + + - name: Generate + self-validate (fails on invalid Swagger 2.0 or over the size limit) + run: docker run --rm --user "$(id -u):$(id -g)" -e HOME=/tmp -v "${{ github.workspace }}:/work" -w /work ppc-gen node scripts/generate.mjs diff --git a/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl new file mode 100644 index 0000000..26c21c1 --- /dev/null +++ b/plugins/repokit/skills/new-repo/templates/types/power-platform-connectors/public/.github/workflows/sync.yml.tmpl @@ -0,0 +1,105 @@ +name: sync-postman + +# Detect changes to the upstream Postman collection and open a PR with regenerated +# connector definitions. Account-free when `sourceUrl` is a public URL; add a +# POSTMAN_API_KEY secret only if the collection lives inside Postman's platform. +# +# Requires: Settings -> Actions -> "Allow GitHub Actions to create and approve pull requests". + +on: + schedule: + - cron: '17 6 * * *' # daily, 06:17 UTC + workflow_dispatch: + +concurrency: + group: sync-postman + cancel-in-progress: false + +permissions: + contents: write + pull-requests: write + +jobs: + sync: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Fetch upstream collection and detect a change + id: fetch + env: + POSTMAN_API_KEY: ${{ secrets.POSTMAN_API_KEY }} + run: | + set -euo pipefail + URL=$(jq -r '.sourceUrl // ""' connectors.config.json) + case "$URL" in + *"> "$GITHUB_OUTPUT"; exit 0;; + esac + # Send the Postman API key only to a genuine Postman API host (all three verified official: + # api.getpostman.com legacy, api.postman.com standard, api.eu.postman.com EU data + # residency). The trailing "/" anchors each match so a look-alike like + # api.getpostman.com.evil.tld cannot match. --max-redirs 0 fails closed if a host ever + # 3xx-redirects: curl re-sends custom headers across hosts on redirect, so this prevents + # the key leaking to another host. + AUTH=() + case "$URL" in + "https://api.getpostman.com/"* | "https://api.postman.com/"* | "https://api.eu.postman.com/"*) + [ -n "${POSTMAN_API_KEY:-}" ] && AUTH=(--max-redirs 0 -H "X-Api-Key: ${POSTMAN_API_KEY}");; + esac + curl -fsSL --proto '=https' --proto-redir '=https' --max-time 60 --retry 3 "${AUTH[@]}" "$URL" -o /tmp/fetched.json + # Unwrap the Postman API's {"collection": …} envelope if present. + if jq -e '.collection' /tmp/fetched.json >/dev/null 2>&1; then + jq '.collection' /tmp/fetched.json > /tmp/coll.json + else + cp /tmp/fetched.json /tmp/coll.json + fi + NEW=$(jq -S 'del(.info._postman_id)' /tmp/coll.json | sha256sum | cut -d' ' -f1) + OLD=$(jq -r '.sha256 // ""' .postman/manifest.json 2>/dev/null || echo "") + if [ "$NEW" = "$OLD" ]; then + echo "No upstream change." + echo "changed=false" >> "$GITHUB_OUTPUT"; exit 0 + fi + # Write to the canonical committed path (the generator prefers source/collection.json). + mkdir -p source .postman + cp /tmp/coll.json source/collection.json + # No timestamp in the committed manifest — an unchanged collection would otherwise produce + # a fresh commit every day. Only sha256 gates change detection. + jq -n --arg u "$URL" --arg s "$NEW" '{sourceUrl:$u, sha256:$s}' > .postman/manifest.json + echo "changed=true" >> "$GITHUB_OUTPUT" + + - name: Regenerate + validate INLINE + # Inline because a PR opened by the default GITHUB_TOKEN does not trigger ci.yml, + # so we can't rely on CI validating this bot's PR. + if: steps.fetch.outputs.changed == 'true' + run: | + docker build -t ppc-gen . + docker run --rm --user "$(id -u):$(id -g)" -e HOME=/tmp -v "${{ github.workspace }}:/work" -w /work ppc-gen node scripts/generate.mjs + + - name: Open a pull request + if: steps.fetch.outputs.changed == 'true' + env: + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + # Reuse one branch/PR so daily runs update the existing PR instead of piling up new ones. + BR="sync/postman" + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + git checkout -B "$BR" + # Stage only the sync outputs, not any other tree state the runner may have. + git add source/collection.json .postman/manifest.json connectors/ + git commit -m "chore: sync connector definitions from upstream Postman collection" + git push -f -u origin "$BR" + # Open a PR only if one isn't already OPEN for this branch. Use `gh pr list` (open PRs + # only), NOT `gh pr view` — the latter returns the branch's most recent PR even when it is + # merged/closed, so after the first sync PR merges we'd wrongly think one still exists and + # never open another. Command substitution under `set -e` still fails loudly if gh errors + # (e.g. "GitHub Actions is not permitted to create pull requests") — a dead sync must be visible. + OPEN=$(gh pr list --head "$BR" --state open --json number --jq 'length') + if [ "${OPEN:-0}" -eq 0 ]; then + gh pr create --base main --head "$BR" \ + --title "Sync connector definitions from upstream Postman collection" \ + --body "The upstream Postman collection changed. Regenerated OpenAPI 2.0 definitions (validated as Swagger 2.0, each < 1 MB). **Review before importing to Power Platform** — the Postman-to-2.0 conversion is lossy." + fi diff --git a/plugins/repokit/skills/repo-standard/standard/testing-matrix.md b/plugins/repokit/skills/repo-standard/standard/testing-matrix.md index 93a0997..50c655e 100644 --- a/plugins/repokit/skills/repo-standard/standard/testing-matrix.md +++ b/plugins/repokit/skills/repo-standard/standard/testing-matrix.md @@ -6,6 +6,8 @@ minimum, whatever proves the code runs.) | Type | Test / check | |------|--------------| | `powershell-module` | `Test-ModuleManifest` parses the `.psd1`; **Pester** for behaviour; **PSScriptAnalyzer** (`Invoke-ScriptAnalyzer -Recurse`, fail on `Error` severity). CI runs on `ubuntu-latest` with `shell: pwsh`. | +| `docker-compose` | `docker compose config -q` validates the compose file. CI runs it on `ubuntu-latest` (Public tier). | +| `power-platform-connectors` | `docker build` then `node scripts/generate.mjs` in the pinned image: every generated `connectors/*.swagger.json` must be **valid Swagger 2.0** (self-validated via `swagger-cli`) and **< 1 MB** — the generator exits non-zero otherwise. CI runs it on `ubuntu-latest` (Public tier). | | `skill-plugin` | TBD when the overlay is built — validate `plugin.json` / `marketplace.json` / `SKILL.md` frontmatter (reuse `scripts/validate_*.py`) and scan SKILL bodies for safety. | | `collection` | TBD when the overlay is built. | | `mcp-server` | TBD when the overlay is built. | diff --git a/plugins/repokit/skills/repo-standard/standard/the-standard.md b/plugins/repokit/skills/repo-standard/standard/the-standard.md index 8e62f2a..1736b8d 100644 --- a/plugins/repokit/skills/repo-standard/standard/the-standard.md +++ b/plugins/repokit/skills/repo-standard/standard/the-standard.md @@ -55,6 +55,7 @@ No governance overhead. A forever-private repo stays here and stays effortless. |------|------| | `powershell-module` | `.psd1` manifest, `.psm1` root module, `Public/` + `Private/`, `Tests/` (Pester); PSScriptAnalyzer + Pester CI; `Publish-PSResource` publish (Published) | | `docker-compose` | `compose.yaml` (no `version:`; named volumes for data, commented config/build patterns), `.env.example`, `.dockerignore`; `docker compose config -q` validation CI (Public). Config committed, data in named volumes, secrets in `.env` | +| `power-platform-connectors` | a committed Postman collection → **OpenAPI 2.0** custom-connector definitions for Microsoft Power Platform. A pinned-Docker generator (`postman-to-openapi` + `api-spec-converter`) converts + normalises to valid Swagger 2.0, splitting per top-level folder **only when a single def would hit the 1 MB limit**; self-validates; a scheduled sync workflow PRs upstream changes. Builds with just Docker — no Postman account (Public) | | `skill-plugin` | a Claude Code plugin: `.claude-plugin/`, `skills//SKILL.md`, validation *(stub — fill when first needed)* | | `collection` | a multi-component repo with a top-level map + per-component subdirs *(stub)* | | `mcp-server` | an MCP server *(stub)* |