diff --git a/AGENTS.md b/AGENTS.md index 7e60ad3..e9d5e17 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -68,7 +68,10 @@ src/ # gitignored — cloned upstream trees live here - **`.build-status.json`** is written atomically (`tmp` → `mv`) so a reader never sees a half-written file. Phases: `init → clone → venv → prebuild → build_start → building → build_done → wheel → done`, or `failed`. Anything - interpolated into it goes through `json_escape`. + interpolated into it goes through `json_escape`. It lives inside the source + tree, so on a first build nothing is written until the clone lands — `init` + and `clone` only show up on a re-run. Emitting it is best-effort by design: + a status write must never take down a build that has been running for hours. - **Every env var takes the form `${VAR:-default}`**, so any single value can be overridden without editing a file. Keep it that way. - **Paths derive from the repo root**, computed from the script's own location. diff --git a/CHANGELOG.md b/CHANGELOG.md index bbbf757..8537979 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,23 @@ and the project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0. ## [Unreleased] +### Fixed + +- **The first build on a fresh clone always failed.** `emit_status` writes + `.build-status.json` into the target's source tree, but the `init` phase fires + before the clone has created that directory — so `make build-pytorch` died + instantly with `src/pytorch/.build-status.json.tmp: No such file or directory` + on any checkout that had never been built. Status emission now skips until the + tree is a git repo, and never fails the build if the write does not land. This + also unbreaks the case where the destination directory exists but is empty: + the old code seeded a status file into it, and `git clone` then refused the + now-non-empty directory. +- **The pytorch preset was missing two build requirements.** Upstream builds + through `scikit-build-core` now, and `--no-build-isolation` installs nothing on + our behalf, so `numpy` and `scikit-build-core` have to be seeded into the venv + alongside the rest of `[build-system] requires`. Without them the build fails + at backend import, before a single file compiles. + ## [0.1.0] - 2026-08-03 First public release. conceit began as the private build orchestration behind a diff --git a/scripts/build-upstream.sh b/scripts/build-upstream.sh index 43189c5..39f9f9d 100755 --- a/scripts/build-upstream.sh +++ b/scripts/build-upstream.sh @@ -59,11 +59,23 @@ json_escape() { emit_status() { local phase=$1 step=${2:-null} jobs=${3:-0} detail=${4:-} - [[ -z "$_status_file" ]] && return - printf '{"phase":"%s","ts":"%s","step":"%s","jobs":%s,"detail":"%s"}\n' \ + [[ -z "$_status_file" ]] && return 0 + + # The status file lives inside the source tree, which does not exist yet on a + # fresh checkout — and must stay empty until the clone lands, because git + # refuses to clone into a directory that already has files in it. So skip the + # write until the tree is a repo: the init/clone phases have nowhere to go. + # Never fail here either; a status write must not kill a three-hour build. + [[ -d "${_status_file%/*}/.git" ]] || return 0 + + if printf '{"phase":"%s","ts":"%s","step":"%s","jobs":%s,"detail":"%s"}\n' \ "$(json_escape "$phase")" "$(ts_iso)" "$(json_escape "$step")" \ "$jobs" "$(json_escape "$detail")" \ - > "${_status_file}.tmp" && mv "${_status_file}.tmp" "$_status_file" + > "${_status_file}.tmp" 2>/dev/null + then + mv -f "${_status_file}.tmp" "$_status_file" 2>/dev/null || true + fi + return 0 } # Count all descendants of a PID (not just direct children). @@ -219,7 +231,11 @@ case "$preset" in pytorch) build_cmd=${BUILD_CMD:-"MAX_JOBS=${MAX_JOBS} NVCC_THREADS=${NVCC_THREADS} python -m pip install -e . -v --no-build-isolation"} build_dist_cmd=${BUILD_DIST_CMD:-"python -m pip wheel --no-build-isolation --no-deps -w dist ."} - extra_pip_deps=${EXTRA_PIP_DEPS:-"cmake ninja packaging pyyaml typing_extensions six"} + # --no-build-isolation means pip installs nothing on our behalf: every entry + # in pytorch's [build-system] requires has to already be in the venv. numpy + # and scikit-build-core (the build backend since the setup.py retirement) + # are load-bearing — without them the build dies before the first compile. + extra_pip_deps=${EXTRA_PIP_DEPS:-"cmake ninja packaging pyyaml typing_extensions six numpy scikit-build-core"} ;; vllm) # vllm uses uv (per AGENTS.md). We override the torch==2.11.0 pin with our