From d97dd95674ad9bf2deae69652ffc3875b0673ae5 Mon Sep 17 00:00:00 2001 From: elliot-barn Date: Fri, 26 Jun 2026 09:34:32 +0000 Subject: [PATCH 1/3] [ci] windows: pin pyopenssl in conda base to fix cryptography skew (#64034) Backport of #64034 to releases/2.56.0. The Windows base image build (ci/ray_ci/windows/build_base.sh) crashes when running `conda update -c conda-forge ca-certificates certifi`: AttributeError: module 'lib' has no attribute 'X509_V_FLAG_CB_ISSUER_CHECK' Upgrading the conda base env to python 3.10 (`conda install python=...`) pulls cryptography>=38, which removed `_lib.X509_V_FLAG_CB_ISSUER_CHECK`. pyopenssl is not part of that transaction, so the stale py3.8-era pyopenssl is left behind and still references the removed attribute at import. The next conda invocation imports requests -> urllib3.contrib.pyopenssl -> OpenSSL.crypto and detonates before conda can run, failing the base image build. Co-resolve pyopenssl 23.2.0 in the same conda install transaction so it stays compatible with the cryptography 38.x that gets installed, and --freeze-installed the cert update so it can't perturb the just-resolved env. Co-Authored-By: Claude Opus 4.8 (1M context) Signed-off-by: elliot-barn --- ci/ray_ci/windows/build_base.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ci/ray_ci/windows/build_base.sh b/ci/ray_ci/windows/build_base.sh index 2398179d5c83..080cfc179dce 100644 --- a/ci/ray_ci/windows/build_base.sh +++ b/ci/ray_ci/windows/build_base.sh @@ -16,9 +16,9 @@ conda init # Conda 26.3.1 crashes on Windows when it tries to clean up a locked exe during # a build-variant swap. Preventing self-update avoids that code path. conda config --set auto_update_conda false -conda install -q -y python="${PYTHON_FULL_VERSION}" requests=2.32.3 +conda install -q -y python="${PYTHON_FULL_VERSION}" requests=2.32.3 pyopenssl=23.2.0 # Force CA trust stack to the newest versions available at build time. -conda update -c conda-forge -q -y ca-certificates certifi +conda update --freeze-installed -c conda-forge -q -y ca-certificates certifi # Install torch first, as some dependencies (e.g. torch-spline-conv) need torch to be # installed for their own install. From 79b63b7897251167bc9bacfc683c97bee8ef22ef Mon Sep 17 00:00:00 2001 From: elliot-barn Date: Fri, 26 Jun 2026 09:39:58 +0000 Subject: [PATCH 2/3] [ci] windows: TEMP release-build BUILDKITE_COMMIT override (DO NOT MERGE) One-off release maneuver for shipping 2.56.0 Windows wheels. This branch's HEAD carries the windowsbuild conda fix (so the image builds), but the wheels must be stamped with the real release commit 637fd062205393b9e1929996bfe1d49bd3f8469d, not this branch's commit. build-wheel-windows.sh stamps ray.__commit__ from BUILDKITE_COMMIT, which is forwarded into the build container via _DOCKER_ENV, so exporting it inside the windows wheel step overrides the stamp without affecting the checkout. Wheels will upload to s3://ray-wheels/// and then be copied into the releases/2.56.0// prefix for the PyPI upload. This commit is intentionally NOT for merge - drop it before opening the clean backport PR (which should contain only the build_base.sh fix). Co-Authored-By: Claude Opus 4.8 (1M context) Signed-off-by: elliot-barn --- .buildkite/windows.rayci.yml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.buildkite/windows.rayci.yml b/.buildkite/windows.rayci.yml index d6621a220db3..bcc92c5b2c7b 100644 --- a/.buildkite/windows.rayci.yml +++ b/.buildkite/windows.rayci.yml @@ -39,6 +39,11 @@ steps: job_env: WINDOWS instance_type: windows commands: + # RELEASE-BUILD OVERRIDE (DO NOT MERGE): stamp the wheels with the real 2.56.0 + # release commit, even though this branch's HEAD carries the windowsbuild fix. + # build-wheel-windows.sh reads BUILDKITE_COMMIT for ray.__commit__; the env is + # forwarded into the container via _DOCKER_ENV. Drop this line for the clean PR. + - export BUILDKITE_COMMIT=637fd062205393b9e1929996bfe1d49bd3f8469d - bash ci/ray_ci/windows/install_tools.sh - bazel run //ci/ray_ci:build_in_docker_windows -- wheel --python-version {{matrix}} --operating-system windows --upload matrix: From 9b7ae07da920e00d6dba1ef2aba9c07f883ccde3 Mon Sep 17 00:00:00 2001 From: elliot-barn Date: Sun, 28 Jun 2026 23:37:18 +0000 Subject: [PATCH 3/3] [ci] windows: route broker wheel upload to release prefix (DO NOT MERGE) Adds export BUILDKITE_BRANCH=releases/2.56.0 inside the windows wheel step so the postmerge upload broker (.buildkite/copy_files.py, key = {BUILDKITE_BRANCH}/{BUILDKITE_COMMIT}/{fn}) writes the wheels straight to s3://ray-wheels/releases/2.56.0/637fd062.../ -- the same prefix the release PyPI upload reads from. There is no direct write access to the cross-account ray-wheels bucket, so routing via BUILDKITE_BRANCH is the only way to land them in the release prefix. Stacks on top of the BUILDKITE_COMMIT override; both TEMP commits are dropped before the clean build_base.sh backport PR. Signed-off-by: elliot-barn --- .buildkite/windows.rayci.yml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/.buildkite/windows.rayci.yml b/.buildkite/windows.rayci.yml index bcc92c5b2c7b..c270a9486a8d 100644 --- a/.buildkite/windows.rayci.yml +++ b/.buildkite/windows.rayci.yml @@ -42,8 +42,12 @@ steps: # RELEASE-BUILD OVERRIDE (DO NOT MERGE): stamp the wheels with the real 2.56.0 # release commit, even though this branch's HEAD carries the windowsbuild fix. # build-wheel-windows.sh reads BUILDKITE_COMMIT for ray.__commit__; the env is - # forwarded into the container via _DOCKER_ENV. Drop this line for the clean PR. + # forwarded into the container via _DOCKER_ENV. Drop these lines for the clean PR. + # BUILDKITE_BRANCH routes the broker upload (copy_files.py key is + # {BUILDKITE_BRANCH}/{BUILDKITE_COMMIT}/{fn}) to the release prefix the PyPI + # upload reads from, so no post-hoc S3 copy is needed. - export BUILDKITE_COMMIT=637fd062205393b9e1929996bfe1d49bd3f8469d + - export BUILDKITE_BRANCH=releases/2.56.0 - bash ci/ray_ci/windows/install_tools.sh - bazel run //ci/ray_ci:build_in_docker_windows -- wheel --python-version {{matrix}} --operating-system windows --upload matrix: