Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,15 @@ and the project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.
also unbreaks the case where the destination directory exists but is empty:
the old code seeded a status file into it, and `git clone` then refused the
now-non-empty directory.
- **`check-env` passed on a machine that could not compile.** `cuda-env.sh`
hardcoded `CC=/usr/bin/gcc-15`, and `check-env` printed that path without ever
testing it — so a host without gcc 15 installed sailed through the gate and
died in cmake twelve minutes later, after a 1.4 GB clone and a full submodule
sync (`is not a full path to an existing compiler tool`). The host compiler is
now probed (`gcc-15`, `gcc-14`, `gcc-13`, then plain `gcc`), and `check-env`
fails if `CC`/`CXX` are not executable, or if the compiler is newer than the
cap in the toolkit's own `crt/host_config.h` — which is what a rolling distro
hands you by default, since CUDA 13.3 stops at gcc 15 and Manjaro is on 16.
- **The pytorch preset was missing two build requirements.** Upstream builds
through `scikit-build-core` now, and `--no-build-isolation` installs nothing on
our behalf, so `numpy` and `scikit-build-core` have to be seeded into the venv
Expand Down
10 changes: 10 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,16 @@ check-env: ## Validate prerequisites: CUDA, Python, uv, GPU, disk
@[ -n "$(CUDA_HOME)" ] || { echo "ERROR: CUDA_HOME not set. Run: source scripts/cuda-env.sh"; exit 1; }
@[ -d "$(CUDA_HOME)" ] || { echo "ERROR: CUDA_HOME=$(CUDA_HOME) does not exist"; exit 1; }
@$(call require_bins,nvidia-smi python)
@[ -x "$(CC)" ] || { echo "ERROR: CC=$(CC) is not an executable compiler. Install a CUDA-supported gcc (pacman -S gcc15 | apt install gcc-15 g++-15), or export CC/CXX"; exit 1; }
@[ -x "$(CXX)" ] || { echo "ERROR: CXX=$(CXX) is not an executable compiler. Install a CUDA-supported g++ (pacman -S gcc15 | apt install gcc-15 g++-15), or export CC/CXX"; exit 1; }
@hdr="$(CUDA_HOME)/include/crt/host_config.h"; \
max=$$(grep -oE '__GNUC__ > [0-9]+' "$$hdr" 2>/dev/null | head -1 | grep -oE '[0-9]+$$' || true); \
have=$$("$(CXX)" -dumpversion 2>/dev/null | cut -d. -f1 || true); \
if [ -n "$$max" ] && [ -n "$$have" ] && [ "$$have" -gt "$$max" ]; then \
echo "ERROR: $(CXX) is gcc $$have, and this CUDA supports up to gcc $$max — nvcc will refuse to compile."; \
echo " Install gcc $$max and re-source scripts/cuda-env.sh, or export CC/CXX to one."; \
exit 1; \
fi
@[ -x "$(UV)" ] || { echo "ERROR: uv not found at $(UV). Install: curl -LsSf https://astral.sh/uv/install.sh | sh"; exit 1; }
@[ -n "$(TORCH_CUDA_ARCH_LIST)" ] || echo "WARN: TORCH_CUDA_ARCH_LIST unset — will compile every arch (~9x slower)"
@echo "python $$(python --version 2>&1)"
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,15 +75,15 @@ Anything scoring 4–5 gets resolved before you compile. Every row cost a real b
```bash
git clone https://github.com/TGPSKI/conceit.git
cd conceit
source scripts/cuda-env.sh # CUDA_HOME, TORCH_CUDA_ARCH_LIST=12.0+PTX, gcc-15
source scripts/cuda-env.sh # CUDA_HOME, TORCH_CUDA_ARCH_LIST=12.0+PTX, host compiler
make check-env # fails fast on anything missing
```

`make help` lists every target. Nothing is installed system-wide; the source trees and venvs live under `src/` in the checkout.

**Tested on:** Blackwell `sm_120` (RTX PRO 4500), CUDA 13.3, Python 3.14.6 via asdf, gcc/g++ 15, x86_64 Manjaro. Hopper and Ampere work by setting `TORCH_CUDA_ARCH_LIST`. Budget ~200 GB of disk and 32 GB of RAM for parallel nvcc.

**You need:** the CUDA 13.3 toolkit (`scripts/install-cuda-toolkit.sh` handles the gcc-15 shim), `uv`, and — for the full torchvision/torchaudio builds — `libjpeg-turbo libpng ffmpeg sox`.
**You need:** the CUDA 13.3 toolkit (`scripts/install-cuda-toolkit.sh` handles the gcc-15 shim), a gcc no newer than your CUDA supports (13.3 caps at 15 — `pacman -S gcc15`, `apt install gcc-15 g++-15`), `uv`, and — for the full torchvision/torchaudio builds — `libjpeg-turbo libpng ffmpeg sox`. `cuda-env.sh` picks the newest supported `gcc-*` it finds; `make check-env` fails if the result is missing or too new for the toolkit.

## Configure

Expand Down
18 changes: 16 additions & 2 deletions scripts/cuda-env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,22 @@ unset _root
# Leaving this unset compiles every supported arch — roughly 9x the nvcc time.
export TORCH_CUDA_ARCH_LIST="${TORCH_CUDA_ARCH_LIST:-12.0+PTX}"

export CC="${CC:-/usr/bin/gcc-15}"
export CXX="${CXX:-/usr/bin/g++-15}"
# nvcc refuses a host compiler newer than the cap in its own host_config.h —
# CUDA 13.3 stops at gcc 15 — and a rolling distro's default gcc runs ahead of
# that (Manjaro is on 16). So prefer an explicitly versioned binary, newest
# first, and fall back to the unversioned one only when no versioned gcc is
# installed. Hardcoding a path here is what makes `make check-env` pass on a
# machine that cannot compile: check-env now verifies whatever this resolves to
# against the toolkit's own cap.
_host_cc() {
local c
for c in "$1-15" "$1-14" "$1-13" "$1"; do
command -v "$c" 2>/dev/null && return 0
done
}

export CC="${CC:-$(_host_cc gcc)}"
export CXX="${CXX:-$(_host_cc g++)}"
export CUDAHOSTCXX="${CUDAHOSTCXX:-$CXX}"
export CMAKE_C_COMPILER="${CMAKE_C_COMPILER:-$CC}"
export CMAKE_CXX_COMPILER="${CMAKE_CXX_COMPILER:-$CXX}"
Expand Down