Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .claude/skills/verify/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
name: verify
description: Verify await-parallel-limit at its real surface — the npm package boundary (packed tarball installed into a fresh consumer), never via ./src imports.
---

# Verifying await-parallel-limit

Surface: the package boundary. Pack → install into a throwaway consumer → drive as a dependent would.

## Recipe

1. Build + pack: `npx tsc && npm pack --pack-destination <tmpdir>`
2. Fresh consumer: `mkdir <tmpdir>/consumer && cd there && npm init -y && npm i <tmpdir>/await-parallel-limit-<ver>.tgz`
3. Drive CJS: `require('await-parallel-limit').default` and named `{ parallel, settle, map, mapSettled, DEFAULT_CONCURRENCY }`.

## Flows worth driving

- Legacy dependent shape (`@paperbits/core`): `await parallel(thunks, 30)`, result discarded.
- Ordering (slow first job must still come back first) and peak-concurrency (active/peak counters in jobs).
- Abort: mid-flight, pre-aborted, **synchronous abort from inside a job during startup** (regression: must reject, not fulfill with holes), abort after completion (no-op), one signal reused across 30 runs (no MaxListeners warning).
- Fail-fast: after first rejection, started-count must not grow (remaining jobs abandoned).
- Garbage: non-array, junk limits (0, -1, 2.5, NaN, string, null → default 5), non-function array element (TypeError rejection), sparse arrays through map (holes → mapper gets undefined).
- TS surface: compile a consumer .ts against the *installed* d.ts. Floor is **TS 3.4** (`readonly I[]` in map/mapSettled signatures) — pin with `npm i --no-save typescript@3.4.5`.

## Gotchas

- Node ESM default import works via the `exports` map + `esm/index.mjs` wrapper. If `import parallel from` ever yields an object instead of a function, the exports map or wrapper broke — run `test/boundary/smoke.mjs` against the packed tarball.
- `tsc ... | head` masks the exit code — check `$?` on the tsc command itself, not the pipe.
- Scale check: 100k-item `map` should be O(10ms); if it regresses to O(100ms), per-item thunk allocation crept back in.
46 changes: 45 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [16.x, 18.x, 20.x, 22.x]
# 16.14 is the declared engines floor — tested exactly.
node-version: ['16.14', 16.x, 18.x, 20.x, 22.x, 24.x]
steps:
- uses: actions/checkout@v4
- name: Use Node.js ${{ matrix.node-version }}
Expand All @@ -19,3 +20,46 @@ jobs:
node-version: ${{ matrix.node-version }}
- run: npm install
- run: npm test

fuzz:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22.x
- run: npm install
- name: Differential fuzz (500 seeded scenarios)
run: npm run fuzz -- 500

package-boundary:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22.x
- run: npm install
- name: Pack tarball
run: |
npm pack --pack-destination /tmp
echo "TARBALL=$(ls /tmp/await-parallel-limit-*.tgz)" >> "$GITHUB_ENV"
- name: Install into a fresh consumer and run the smoke tests (CJS + ESM)
run: |
mkdir -p /tmp/consumer && cd /tmp/consumer
npm init -y > /dev/null
npm install "$TARBALL"
cp "$GITHUB_WORKSPACE/test/boundary/smoke.cjs" "$GITHUB_WORKSPACE/test/boundary/smoke.mjs" .
node --unhandled-rejections=strict smoke.cjs
node --unhandled-rejections=strict smoke.mjs
- name: Typings floor gate — TypeScript 3.4 compiles the installed d.ts
run: |
cd /tmp/consumer
cp "$GITHUB_WORKSPACE/test/boundary/consumer.ts" .
npm install --no-save typescript@3.4.5
./node_modules/.bin/tsc --noEmit --strict --target es2017 --lib es2017 --moduleResolution node consumer.ts
- name: Typings gate — latest TypeScript compiles the installed d.ts
run: |
cd /tmp/consumer
npm install --no-save typescript@latest
./node_modules/.bin/tsc --noEmit --strict --target es2017 --lib es2017 --moduleResolution node consumer.ts
63 changes: 63 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
name: Publish

# Publishes to npm with provenance attestations. Trigger by publishing a
# GitHub Release (tag `vX.Y.Z`), or manually via workflow_dispatch.
#
# One-time setup: add an npm Automation token as the `NPM_TOKEN` repository
# secret (Settings → Secrets and variables → Actions). Provenance requires a
# public repo and the `id-token: write` permission below — npm links the
# published tarball to this exact workflow run and commit.

on:
release:
types: [published]
workflow_dispatch:

permissions:
contents: read
id-token: write # required for npm provenance (OIDC)

jobs:
publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: 22.x
registry-url: 'https://registry.npmjs.org'

- run: npm install

- name: Guard — tag matches package.json version
if: github.event_name == 'release'
run: |
PKG_VERSION="$(node -p "require('./package.json').version")"
TAG="${{ github.event.release.tag_name }}"
if [ "v$PKG_VERSION" != "$TAG" ] && [ "$PKG_VERSION" != "$TAG" ]; then
echo "::error::package.json is $PKG_VERSION but the release tag is $TAG"
exit 1
fi

- name: Full gate — unit suite + type gate
run: npm test

- name: Full gate — differential fuzz
run: npm run fuzz -- 500

- name: Full gate — package-boundary smoke (CJS + ESM) from the tarball
run: |
npm pack --pack-destination /tmp
TARBALL="$(ls /tmp/await-parallel-limit-*.tgz)"
mkdir -p /tmp/consumer && cd /tmp/consumer
npm init -y > /dev/null
npm install "$TARBALL"
cp "$GITHUB_WORKSPACE/test/boundary/smoke.cjs" "$GITHUB_WORKSPACE/test/boundary/smoke.mjs" .
node --unhandled-rejections=strict smoke.cjs
node --unhandled-rejections=strict smoke.mjs

- name: Publish with provenance
run: npm publish --provenance --access public
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
56 changes: 56 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Agent guide — await-parallel-limit

Zero-dependency npm package: a concurrency-limited promise pool. ~230 lines of
TypeScript in `src/index.ts`; everything else is tests and packaging.

## Consumer API

See `llms.txt` (shipped in the npm tarball) for the complete API, semantics,
recipes, and gotchas in one file.

## Working on this repo

- Build: `npm run build` (tsc → `dist/`). `dist/` is git-ignored; `prepare`
builds on install/publish.
- Test: `npm test` — builds, compiles `test/types.test.ts` (exact-type
assertions; a wrong inference is a build failure), then runs the unit suite
under `--unhandled-rejections=strict`.
- Fuzz: `npm run fuzz -- [iterations] [seed]` — seeded differential fuzzer
(`test/fuzz.cjs`); failures print the seed for exact reproduction.
- Verify at the real surface: follow `.claude/skills/verify/SKILL.md` — pack
the tarball, install into a fresh consumer, drive `require`/`import`
`'await-parallel-limit'`. Never verify via `./src` imports.

## Architecture

One core, four one-line wrappers:

- `run(items, invoke, limit, settle, signal)` in `src/index.ts` — spawns up to
`limit` workers pulling from a shared cursor; workers call
`invoke(items[i], i)` directly (no per-item thunk allocation). A single
`stopped` flag unifies fail-fast and abort cancellation.
- `parallel`/`settle` pass the jobs array + a call-the-thunk invoker;
`map`/`mapSettled` pass the user's items + the mapper itself.
- ESM entry is a static wrapper (`esm/index.mjs`) re-exporting the CJS build —
do NOT introduce a second compiled implementation (dual-package hazard).

## Invariants you must not break (enforced by tests + fuzzer + CI)

1. Results in input order; peak in-flight ≤ effective limit; sliding pool
(finished task replaced immediately — never batches).
2. Legacy call shape `parallel(jobs, 30)` and the default export are frozen —
published dependents rely on them.
3. Limit normalization: non-positive/non-integer → 5; `Infinity` → unbounded.
4. Fail-fast stops scheduling; settle mode never rejects on job errors.
5. Abort (even synchronous, from inside a job) rejects with `signal.reason`;
listener always removed; no unhandled rejections ever.
6. Zero runtime dependencies; compiles without DOM/es2020 libs; typings floor
TS 3.4 (`readonly` array syntax — nothing newer in the public d.ts).
7. Node floor 16.14 (`engines`), tested exactly in CI.

## Conventions

- No new runtime dependencies. Keep declaration-emit clean (JSDoc flows into
`dist/index.d.ts` — it is consumer documentation).
- Every bug fix lands with a regression test; behavioral claims in README must
have a matching CI gate.
12 changes: 12 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# await-parallel-limit

Follow `AGENTS.md` — it has the build/test/fuzz commands, architecture map,
and the invariants this package must never break. Consumer-facing API
reference lives in `llms.txt`.

Quick anchors:
- `npm test` = build + exact-type gate + unit suite (strict unhandled rejections)
- `npm run fuzz -- 300` before any change to `src/index.ts`
- Package-boundary verification recipe: `.claude/skills/verify/SKILL.md`
- Never break: input-order results, sliding-pool concurrency, the v2 call shape
`parallel(jobs, limit)`, zero dependencies, TS 3.4 typings floor.
124 changes: 96 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,63 +1,131 @@
# await-parallel-limit

Run an array of async functions with a bounded number running concurrently.
Run async work with a bounded number of tasks in flight at once.
Zero dependencies, first-class TypeScript types.

```bash
npm install await-parallel-limit --save
```

## Behaviour

- Runs at most `concurrency` jobs at a time (default `5`).
- Results are returned in **input order**, not completion order.
- If any job rejects, the returned promise rejects with the first error
(same semantics as `Promise.all`). Jobs already in flight still run to
completion, but their results are discarded.
- A `concurrency` value that is not a positive integer (e.g. `0`, `-1`, `2.5`)
falls back to the default of `5`.
## Guarantees

- **Concurrency ceiling:** at most `concurrency` tasks are in flight at any
moment (default `5`; `Infinity` means unbounded). A `concurrency` that is not
a positive integer or `Infinity` (e.g. `0`, `-1`, `2.5`, `NaN`) falls back to
the default of `5`.
- **Sustained concurrency:** when a task finishes, the next one starts
immediately — a sliding worker pool, not fixed batches.
- **Ordering:** results are returned in **input order**, not completion order.
- **Fail-fast** (`parallel` / `map`): on the first rejection the returned
promise rejects with that error (like `Promise.all`); no further tasks are
started, and in-flight tasks run to completion but their results are
discarded. Use `settle` / `mapSettled` to collect every outcome instead
(like `Promise.allSettled`).
- **Abort** (`{ signal }`): an already-aborted signal rejects before any task
starts; aborting mid-run — even synchronously from inside a task — rejects
with the signal's `reason` and stops scheduling. The abort listener is always
removed, so one long-lived signal can be reused across many runs.
- **No stray rejections:** after the returned promise settles, a still-running
task that later rejects is absorbed — it never surfaces as an unhandled
rejection.
- **Input snapshot:** the input array's length is captured at call time, so
mutating the array mid-run cannot change the result set. Elements are read
lazily at dispatch; don't mutate the input during a run.

These guarantees are enforced by the unit suite plus a seeded differential
fuzzer (`npm run fuzz`) and a package-boundary smoke test in CI.

## API

```typescript
parallel<T>(
jobs: Array<() => Promise<T>>,
concurrency?: number, // default: 5
): Promise<T[]>
type Options = { signal?: AbortSignal }

// Run an array of thunks, fail-fast, results in input order.
parallel<T>(jobs: Array<() => Promise<T>>, concurrency?: number, options?: Options): Promise<T[]>

// Like parallel, but never rejects on a failing job — returns per-job outcomes.
settle<T>(jobs: Array<() => Promise<T>>, concurrency?: number, options?: Options): Promise<SettledResult<T>[]>

// Map over data with a mapper, fail-fast, results in input order.
map<I, R>(items: I[], mapper: (item: I, index: number) => R | Promise<R>, concurrency?: number, options?: Options): Promise<R[]>

// Like map, but never rejects — returns per-item outcomes.
mapSettled<I, R>(items: I[], mapper: (item: I, index: number) => R | Promise<R>, concurrency?: number, options?: Options): Promise<SettledResult<R>[]>

type SettledResult<T> =
| { status: 'fulfilled'; value: T }
| { status: 'rejected'; reason: any }
```

## TypeScript
All four functions are named exports; `parallel` is also the default export.

## Examples

```typescript
import parallel from 'await-parallel-limit'
// named import also available: import { parallel } from 'await-parallel-limit'
import parallel, { settle, map, mapSettled } from 'await-parallel-limit'

// 1. Array of thunks (ordered-tuple typing when declared `as const`).
const jobs = [
async () => true,
async () => 2,
] as const
const results = await parallel(jobs, 2) // const results: [boolean, number]

// 2. Map over data — no need to pre-build closures.
const bodies = await map(urls, (url) => fetch(url).then((r) => r.text()), 10)

// 3. Don't fail fast — inspect every outcome.
const outcomes = await mapSettled(urls, (url) => fetch(url), 10)
const failed = outcomes.filter((o) => o.status === 'rejected')

// When `jobs` is a readonly tuple (`as const`), the result is an ordered tuple
// typed from each job's return type:
// const results: [boolean, number]
const results = await parallel(jobs, 2)
// 4. Cancel early with an AbortSignal.
const controller = new AbortController()
setTimeout(() => controller.abort(), 5000)
await parallel(jobs, 5, { signal: controller.signal })
```

## JavaScript
## Importing

Both module systems are first-class (an `exports` map routes each to the right
entry — the ESM default import is the function, not a namespace object):

```javascript
// ESM
import parallel, { settle, map, mapSettled } from 'await-parallel-limit'

// CommonJS
const parallel = require('await-parallel-limit').default
const { settle, map, mapSettled } = require('await-parallel-limit')

const jobs = [
async () => { /* ... */ },
async () => { /* ... */ },
const results = await parallel([
async () => { /* ... */ },
async () => { /* ... */ },
]

const results = await parallel(jobs, 2)
], 2)
```

## For AI agents

The npm tarball ships an [`llms.txt`](./llms.txt) — the complete API,
semantics, recipes, and gotchas in one compact file
(`node_modules/await-parallel-limit/llms.txt`). Contributors: see
[`AGENTS.md`](./AGENTS.md) for build/test/fuzz commands and the invariants
this package guarantees.

## Compatibility

`3.x` keeps the `2.x` API: the default export and the `parallel(jobs, limit)`
call shape are unchanged, and `settle`, `map`, `mapSettled`, and the `options`
argument are additive.

One deliberate behavioural change: after a fail-fast rejection, `3.x` stops
starting the remaining jobs. In `2.x`, surviving workers kept executing every
remaining job in the background even though the batch promise had already
rejected and the results were discarded. If you relied on those background side
effects, use `settle`, which always runs every job.

Requires Node >= 16.14. TypeScript consumers need TS >= 3.4 (the published
typings use `readonly` array syntax).

## License

MIT
11 changes: 11 additions & 0 deletions esm/index.d.mts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
export {
default,
parallel,
settle,
map,
mapSettled,
DEFAULT_CONCURRENCY,
SettledResult,
AbortSignalLike,
ParallelOptions,
} from '../dist/index.js'
Loading
Loading