diff --git a/docs/superpowers/specs/2026-07-24-relay-deploy-cn-mirrors-design.md b/docs/superpowers/specs/2026-07-24-relay-deploy-cn-mirrors-design.md new file mode 100644 index 0000000000..4e1c3ae385 --- /dev/null +++ b/docs/superpowers/specs/2026-07-24-relay-deploy-cn-mirrors-design.md @@ -0,0 +1,101 @@ +# Relay Deploy: China Mirror Acceleration + +**Date:** 2026-07-24 +**Status:** Approved for implementation (user: design + implement end-to-end) + +## Problem + +One-click / `deploy.sh` relay deployment pulls Docker Hub images, Debian apt +packages, crates.io crates, GitHub source, and `get.docker.com`. On mainland +China hosts these endpoints are slow or unreliable, so deploy often stalls. + +## Goals + +1. Auto-detect mainland China at deploy start; allow force override. +2. Cover the full path: Desktop SSH (Docker install, GitHub sync) + `deploy.sh` + + Dockerfile (apt + cargo) + Docker Hub pulls. +3. Persist host-level apt and Docker mirror config; keep Cargo mirroring scoped + to the relay Docker build so deployment never rewrites the SSH user's Cargo + configuration. +4. Ship built-in default CN mirrors; allow env overrides. +5. Keep non-CN hosts unchanged. + +## Non-goals + +- Changing relay runtime / account / port behavior +- Building a private mirror service +- Guaranteeing third-party public mirror uptime (defaults + overrides only) + +## Detection + +Priority: + +1. `BITFUN_MIRROR=cn|global` or flags `--cn-mirror` / `--global-mirror` +2. Auto (`BITFUN_MIRROR=auto` default): + - Public IP country lookup (short timeout) + - Timezone `Asia/Shanghai` / `Asia/Chongqing` / `Asia/Urumqi` + - Connectivity heuristic: GitHub slow/fail + Aliyun mirror reachable → CN +3. On ambiguity → `global` (safe default) + +## Default CN mirrors (overridable) + +| Surface | Default | Override env | +|---|---|---| +| Docker Hub registry-mirrors | `https://docker.1ms.run`, `https://dockerproxy.net`, `https://docker.m.daocloud.io` | `BITFUN_DOCKER_REGISTRY_MIRRORS` (space/comma separated) | +| Debian/Ubuntu apt | `mirrors.aliyun.com` | `BITFUN_APT_MIRROR` | +| RHEL/CentOS yum/dnf docker-ce | Aliyun docker-ce | same family | +| Relay Docker build Cargo / crates.io | `sparse+https://rsproxy.cn/index/` | `BITFUN_CARGO_SPARSE_URL` | +| Rustup (host, if used) | `https://rsproxy.cn` | `BITFUN_RUSTUP_DIST_SERVER` | +| GitHub git / tarball | `https://ghfast.top/` prefix | `BITFUN_GITHUB_PROXY` | +| Docker Engine install | Aliyun docker-ce packages; fallback proxied `get.docker.com` | `BITFUN_DOCKER_INSTALL_URL` | + +## Architecture + +Canonical script: `src/apps/relay-server/mirror.sh` + +- Sourced by `deploy.sh` +- Embedded into Desktop orchestration via `include_str!` from + `relay_deploy.rs` (single source of truth; no Cargo crate dependency on apps) +- Idempotent apply with backups under `/etc/bitfun/mirror-backup-*` and + `$HOME/.bitfun/mirror-backup-*` + +Flow: + +``` +detect mode → if cn: apply host mirrors → export build/env vars + → Docker install / GitHub sync use CN URLs + → deploy.sh passes BITFUN_USE_CN_MIRROR=1 build-args + → Dockerfile rewrites apt + writes build-local cargo config +``` + +## Persistence / merge rules + +- **apt:** backup `sources.list` (+ `sources.list.d` bitfun file); write BitFun-owned + `sources.list.d/bitfun-cn-mirror.list` when possible; otherwise rewrite + `deb.debian.org` / `archive.ubuntu.com` hosts in place. +- **Docker daemon.json:** JSON-merge `registry-mirrors` (python3 when available); + never drop unrelated keys; `systemctl restart docker` only if daemon was + already manageable. +- **Cargo:** leave `$HOME/.cargo/config.toml` untouched. The builder writes + `/usr/local/cargo/config.toml` inside Docker for rsproxy sparse. +- Marker file: `$HOME/.bitfun/mirror-mode` = `cn|global` for logs/idempotency. +- **Rollback:** `BITFUN_MIRROR=global` removes the BitFun apt list, restores + source files renamed with `.bitfun-disabled`, removes only Docker mirrors + recorded as BitFun additions, and cleans the legacy managed Cargo block from + early deployments. + +## Failure behavior + +- Mirror apply failures log warnings and continue with best effort; do not abort + deploy solely because a public mirror endpoint is down. +- GitHub proxy failure keeps existing tarball fallback chain (try CN URL then + upstream if override allows). +- `BITFUN_MIRROR=global` skips CN writes and rolls back BitFun-owned host + mirror changes even on a China IP. + +## Verification + +- `bash -n` on `mirror.sh` / `deploy.sh` +- `cargo test -p bitfun-services-integrations --features remote-ssh` (include + embedded mirror script + existing deploy tests) +- `cargo check -p bitfun-desktop` when orchestration wiring changes diff --git a/src/apps/relay-server/Dockerfile b/src/apps/relay-server/Dockerfile index a5a92b05f0..b3bafb000a 100644 --- a/src/apps/relay-server/Dockerfile +++ b/src/apps/relay-server/Dockerfile @@ -7,6 +7,10 @@ # downloaded crates and compiled dependency objects across builds. # - Binaries are copied out of the target cache mount into /out within the same # RUN (cache mounts are not part of the image filesystem). +# +# China mirrors (optional): +# docker compose build --build-arg BITFUN_USE_CN_MIRROR=1 +# Also: BITFUN_APT_MIRROR, BITFUN_CARGO_SPARSE_URL # Pin a minor toolchain so floating `rust:1-bookworm` updates do not bust the # entire builder cache on every upstream image refresh. @@ -18,10 +22,43 @@ WORKDIR /build/src/apps/relay-server # docker compose build --build-arg CARGO_BUILD_JOBS=1 # Note: empty value must NOT be set as ENV — cargo chokes on empty string. ARG CARGO_BUILD_JOBS= +ARG BITFUN_USE_CN_MIRROR=0 +ARG BITFUN_APT_MIRROR=mirrors.aliyun.com +ARG BITFUN_CARGO_SPARSE_URL=sparse+https://rsproxy.cn/index/ + ENV DEBIAN_FRONTEND=noninteractive \ CARGO_TERM_COLOR=always \ CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse +# Configure apt + cargo mirrors inside the builder when deploying from China. +RUN set -eux; \ + if [ "${BITFUN_USE_CN_MIRROR}" = "1" ]; then \ + sed -i \ + -e "s|deb.debian.org/debian|${BITFUN_APT_MIRROR}/debian|g" \ + -e "s|security.debian.org/debian-security|${BITFUN_APT_MIRROR}/debian-security|g" \ + /etc/apt/sources.list 2>/dev/null || true; \ + if [ -f /etc/apt/sources.list.d/debian.sources ]; then \ + sed -i \ + -e "s|deb.debian.org/debian|${BITFUN_APT_MIRROR}/debian|g" \ + -e "s|security.debian.org/debian-security|${BITFUN_APT_MIRROR}/debian-security|g" \ + /etc/apt/sources.list.d/debian.sources; \ + fi; \ + mkdir -p /usr/local/cargo; \ + printf '%s\n' \ + '[source.crates-io]' \ + 'replace-with = "bitfun-rsproxy-sparse"' \ + '' \ + '[source.bitfun-rsproxy-sparse]' \ + "registry = \"${BITFUN_CARGO_SPARSE_URL}\"" \ + '' \ + '[registries.bitfun-rsproxy-sparse]' \ + "index = \"${BITFUN_CARGO_SPARSE_URL}\"" \ + '' \ + '[net]' \ + 'git-fetch-with-cli = true' \ + > /usr/local/cargo/config.toml; \ + fi + RUN apt-get update \ && apt-get install -y --no-install-recommends \ pkg-config \ @@ -88,9 +125,25 @@ RUN --mount=type=cache,id=bitfun-relay-cargo-registry,target=/usr/local/cargo/re FROM debian:bookworm-slim +ARG BITFUN_USE_CN_MIRROR=0 +ARG BITFUN_APT_MIRROR=mirrors.aliyun.com + ENV DEBIAN_FRONTEND=noninteractive -RUN apt-get update \ +RUN set -eux; \ + if [ "${BITFUN_USE_CN_MIRROR}" = "1" ]; then \ + sed -i \ + -e "s|deb.debian.org/debian|${BITFUN_APT_MIRROR}/debian|g" \ + -e "s|security.debian.org/debian-security|${BITFUN_APT_MIRROR}/debian-security|g" \ + /etc/apt/sources.list 2>/dev/null || true; \ + if [ -f /etc/apt/sources.list.d/debian.sources ]; then \ + sed -i \ + -e "s|deb.debian.org/debian|${BITFUN_APT_MIRROR}/debian|g" \ + -e "s|security.debian.org/debian-security|${BITFUN_APT_MIRROR}/debian-security|g" \ + /etc/apt/sources.list.d/debian.sources; \ + fi; \ + fi; \ + apt-get update \ && apt-get install -y --no-install-recommends ca-certificates curl \ && rm -rf /var/lib/apt/lists/* diff --git a/src/apps/relay-server/README.md b/src/apps/relay-server/README.md index 91653a7fda..09444be541 100644 --- a/src/apps/relay-server/README.md +++ b/src/apps/relay-server/README.md @@ -36,6 +36,32 @@ memory VPS (common on arm64), use: RELAY_CARGO_BUILD_JOBS=1 bash deploy.sh ``` +### Mainland China hosts + +`deploy.sh` (and Desktop one-click deploy) auto-detects mainland China and +configures host mirrors for apt, Docker Hub, and GitHub source retrieval plus +a build-local Cargo/crates.io mirror. Docker Engine installation also uses a +mainland mirror. Override when needed: + +```bash +BITFUN_MIRROR=cn bash deploy.sh # force China mirrors +BITFUN_MIRROR=global bash deploy.sh # restore BitFun-managed upstream sources +bash deploy.sh --cn-mirror +bash deploy.sh --global-mirror +``` + +Defaults (overridable via env): Aliyun apt, Docker registry mirrors +(`docker.1ms.run` / `dockerproxy.net` / `docker.m.daocloud.io`), +rsproxy Cargo sparse index, `ghfast.top` GitHub prefix, Aliyun docker-ce +for Engine install (fallback: jsDelivr docker-install). See `mirror.sh` +for the full list (`BITFUN_APT_MIRROR`, `BITFUN_DOCKER_REGISTRY_MIRRORS`, +`BITFUN_CARGO_SPARSE_URL`, `BITFUN_GITHUB_PROXY`, …). + +China mode does not modify the SSH user's global `~/.cargo/config.toml`; Cargo +mirroring is scoped to the relay image build. Switching to `global` restores +apt files disabled by BitFun and removes only Docker registry mirrors recorded +as BitFun additions. + `deploy.sh` enables Docker BuildKit so the Dockerfile can reuse Cargo registry/git/`target` cache mounts across redeploys. Keep BuildKit enabled (`DOCKER_BUILDKIT=1`, the deploy default) and avoid `docker builder prune` diff --git a/src/apps/relay-server/deploy.sh b/src/apps/relay-server/deploy.sh index ffe9584491..4b88e9a393 100755 --- a/src/apps/relay-server/deploy.sh +++ b/src/apps/relay-server/deploy.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash # BitFun Relay Server — one-click deploy script. -# Usage: bash deploy.sh [--skip-build] [--skip-health-check] +# Usage: bash deploy.sh [--skip-build] [--skip-health-check] [--cn-mirror|--global-mirror] # # Run this script on the target server itself after SSH login. # It deploys to the current machine only; it does not SSH to a remote host. @@ -11,15 +11,21 @@ # # Low-memory VPS tip (especially arm64): # RELAY_CARGO_BUILD_JOBS=1 bash deploy.sh +# +# China hosts: auto-detects mainland China and configures apt/Docker/cargo/GitHub +# mirrors (override with BITFUN_MIRROR=cn|global or --cn-mirror/--global-mirror). set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" # shellcheck source=common.sh source "${SCRIPT_DIR}/common.sh" +# shellcheck source=mirror.sh +source "${SCRIPT_DIR}/mirror.sh" SKIP_BUILD=false SKIP_HEALTH_CHECK=false +MIRROR_ARGS=() usage() { cat <<'EOF' @@ -38,12 +44,19 @@ Supported architectures: Options: --skip-build Skip docker compose build, only recreate/start services --skip-health-check Skip post-deploy health check + --cn-mirror Force China mirrors (apt/Docker/cargo/GitHub) + --global-mirror Force global upstream mirrors -h, --help Show this help message Environment: RELAY_HOST_BIND_IP Host bind address for published port (default 0.0.0.0) RELAY_CARGO_BUILD_JOBS Limit rustc parallelism inside Docker (e.g. 1 on small VPS) DOCKER_DEFAULT_PLATFORM Leave unset for native host builds (recommended) + BITFUN_MIRROR auto|cn|global (default auto) + BITFUN_APT_MIRROR Debian/Ubuntu apt host (default mirrors.aliyun.com) + BITFUN_DOCKER_REGISTRY_MIRRORS Space/comma-separated Docker Hub mirrors + BITFUN_CARGO_SPARSE_URL Cargo sparse registry URL (default rsproxy) + BITFUN_GITHUB_PROXY GitHub HTTPS proxy prefix (default https://ghfast.top/) EOF } @@ -51,6 +64,9 @@ for arg in "$@"; do case "$arg" in --skip-build) SKIP_BUILD=true ;; --skip-health-check) SKIP_HEALTH_CHECK=true ;; + --cn-mirror|--global-mirror|--no-cn-mirror|--skip-mirror-apply) + MIRROR_ARGS+=("$arg") + ;; -h|--help) usage exit 0 @@ -70,6 +86,9 @@ echo "Target: current machine ($(uname -s) / ${HOST_ARCH}, uname=$(uname -m))" echo "Note: run this script on the target server after SSH login." assert_supported_arch +# Detect region and persist host mirrors before Docker pulls / image build. +# Validate the host first so unsupported machines are not modified. +bitfun_mirror_init "${MIRROR_ARGS[@]+"${MIRROR_ARGS[@]}"}" require_docker_daemon resolve_compose warn_if_forced_foreign_platform @@ -77,6 +96,21 @@ warn_if_forced_foreign_platform echo "Compose: ${COMPOSE[*]}" cd "$SCRIPT_DIR" +# Persist compose build-args for CN builds (and subsequent restarts). +touch .env +chmod 600 .env 2>/dev/null || true +# Refresh BitFun-managed mirror keys without wiping unrelated .env entries. +if [ -f .env ]; then + tmp_env="$(mktemp)" + grep -Ev '^(BITFUN_USE_CN_MIRROR|BITFUN_APT_MIRROR|BITFUN_CARGO_SPARSE_URL)=' .env >"$tmp_env" || true + mv "$tmp_env" .env +fi +{ + echo "BITFUN_USE_CN_MIRROR=${BITFUN_USE_CN_MIRROR:-0}" + echo "BITFUN_APT_MIRROR=${BITFUN_APT_MIRROR:-mirrors.aliyun.com}" + echo "BITFUN_CARGO_SPARSE_URL=${BITFUN_CARGO_SPARSE_URL:-sparse+https://rsproxy.cn/index/}" +} >>.env + # Build first so a compile failure does not take down a running relay. if [ "$SKIP_BUILD" = true ]; then echo "[1/2] Skipping Docker build (--skip-build)" @@ -87,6 +121,12 @@ else BUILD_ARGS+=(--build-arg "CARGO_BUILD_JOBS=${RELAY_CARGO_BUILD_JOBS}") echo " Using CARGO_BUILD_JOBS=${RELAY_CARGO_BUILD_JOBS}" fi + BUILD_ARGS+=(--build-arg "BITFUN_USE_CN_MIRROR=${BITFUN_USE_CN_MIRROR:-0}") + BUILD_ARGS+=(--build-arg "BITFUN_APT_MIRROR=${BITFUN_APT_MIRROR:-mirrors.aliyun.com}") + BUILD_ARGS+=(--build-arg "BITFUN_CARGO_SPARSE_URL=${BITFUN_CARGO_SPARSE_URL:-sparse+https://rsproxy.cn/index/}") + if [ "${BITFUN_USE_CN_MIRROR:-0}" = "1" ]; then + echo " Using China mirrors inside Docker build (apt + cargo)" + fi # BuildKit is required for Dockerfile cargo registry/git/target cache mounts. # Plain progress so nohup/file-redirected deploys still stream build lines. export DOCKER_BUILDKIT=1 @@ -99,11 +139,14 @@ else case "${BITFUN_DOCKER_MODE:-direct}" in sudo) sudo env DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 BUILDKIT_PROGRESS="${BUILDKIT_PROGRESS}" \ + BITFUN_USE_CN_MIRROR="${BITFUN_USE_CN_MIRROR:-0}" \ + BITFUN_APT_MIRROR="${BITFUN_APT_MIRROR:-mirrors.aliyun.com}" \ + BITFUN_CARGO_SPARSE_URL="${BITFUN_CARGO_SPARSE_URL:-sparse+https://rsproxy.cn/index/}" \ docker compose --progress=plain build "${BUILD_ARGS[@]}" ;; sg) # shellcheck disable=SC2086 - sg docker -c "env DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 BUILDKIT_PROGRESS='${BUILDKIT_PROGRESS}' docker compose --progress=plain build ${BUILD_ARGS[*]}" + sg docker -c "env DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 BUILDKIT_PROGRESS='${BUILDKIT_PROGRESS}' BITFUN_USE_CN_MIRROR='${BITFUN_USE_CN_MIRROR:-0}' BITFUN_APT_MIRROR='${BITFUN_APT_MIRROR:-mirrors.aliyun.com}' BITFUN_CARGO_SPARSE_URL='${BITFUN_CARGO_SPARSE_URL:-sparse+https://rsproxy.cn/index/}' docker compose --progress=plain build ${BUILD_ARGS[*]}" ;; *) if [ "${#COMPOSE[@]}" -ge 2 ] && [ "${COMPOSE[0]}" = "docker" ] && [ "${COMPOSE[1]}" = "compose" ]; then diff --git a/src/apps/relay-server/docker-compose.yml b/src/apps/relay-server/docker-compose.yml index 6569b63570..1fe2e1b05d 100644 --- a/src/apps/relay-server/docker-compose.yml +++ b/src/apps/relay-server/docker-compose.yml @@ -7,6 +7,10 @@ services: # Pass through optional parallelism limit for low-memory hosts: # RELAY_CARGO_BUILD_JOBS=1 docker compose build CARGO_BUILD_JOBS: ${RELAY_CARGO_BUILD_JOBS:-} + # China mirror switch (set by deploy.sh / BITFUN_MIRROR=cn): + BITFUN_USE_CN_MIRROR: ${BITFUN_USE_CN_MIRROR:-0} + BITFUN_APT_MIRROR: ${BITFUN_APT_MIRROR:-mirrors.aliyun.com} + BITFUN_CARGO_SPARSE_URL: ${BITFUN_CARGO_SPARSE_URL:-sparse+https://rsproxy.cn/index/} # Intentionally no `platform:` pin — build/run for the host CPU # (linux/amd64 on x86_64, linux/arm64 on aarch64). container_name: bitfun-relay diff --git a/src/apps/relay-server/mirror.sh b/src/apps/relay-server/mirror.sh new file mode 100644 index 0000000000..7bb6a7d9dc --- /dev/null +++ b/src/apps/relay-server/mirror.sh @@ -0,0 +1,993 @@ +#!/usr/bin/env bash +# BitFun Relay deploy — region detection and China mirror configuration. +# +# Source this file, then call: +# bitfun_mirror_init [--cn-mirror|--global-mirror] +# +# Or execute directly: +# bash mirror.sh [--cn-mirror|--global-mirror] +# +# Environment: +# BITFUN_MIRROR=auto|cn|global +# BITFUN_APT_MIRROR=mirrors.aliyun.com +# BITFUN_DOCKER_REGISTRY_MIRRORS="https://docker.1ms.run https://dockerproxy.net https://docker.m.daocloud.io" +# BITFUN_CARGO_SPARSE_URL=sparse+https://rsproxy.cn/index/ +# BITFUN_RUSTUP_DIST_SERVER=https://rsproxy.cn +# BITFUN_GITHUB_PROXY=https://ghfast.top/ +# BITFUN_DOCKER_INSTALL_URL= # optional full URL override for get.docker.com script +# +# Sets / exports (when mode=cn): +# BITFUN_MIRROR_MODE=cn|global +# BITFUN_USE_CN_MIRROR=0|1 +# BITFUN_GITHUB_GIT_URL / BITFUN_GITHUB_TARBALL_URL +# BITFUN_DOCKER_GET_URL +# BITFUN_APT_MIRROR / BITFUN_CARGO_SPARSE_URL / BITFUN_DOCKER_REGISTRY_MIRRORS +# RUSTUP_DIST_SERVER / RUSTUP_UPDATE_ROOT (cn only) + +# shellcheck disable=SC2034 + +bitfun_mirror_default_docker_mirrors() { + # Order from Beijing CN re-probe (2026-07-25): + # - 1ms: fastest digests for hello-world/debian/rust (~0.5s) + # - dockerproxy.net: stable digests (~1.5-2.5s) + # - daocloud: usable fallback (occasionally slower digest) + # xuanyuan free tier dropped: TOOMANYREQUESTS on debian/rust + echo "https://docker.1ms.run https://dockerproxy.net https://docker.m.daocloud.io" +} + +bitfun_mirror_normalize_list() { + # Portable: BSD/GNU sed differ on \n in character classes; use tr. + echo "$1" | tr ',\t\n' ' ' | tr -s ' ' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//' +} + +bitfun_mirror_priv() { + if [ "$(id -u)" = "0" ]; then + "$@" + elif command -v sudo >/dev/null 2>&1 && sudo -n true >/dev/null 2>&1; then + sudo -n "$@" + elif command -v sudo >/dev/null 2>&1; then + sudo "$@" + else + "$@" + fi +} + +bitfun_mirror_parse_args() { + local arg + for arg in "$@"; do + case "$arg" in + --cn-mirror) + export BITFUN_MIRROR=cn + ;; + --global-mirror|--no-cn-mirror) + export BITFUN_MIRROR=global + ;; + --skip-mirror-apply) + export BITFUN_MIRROR_SKIP_APPLY=1 + ;; + esac + done +} + +bitfun_mirror_http_ok() { + local url="$1" + local timeout="${2:-3}" + if command -v curl >/dev/null 2>&1; then + curl -fsS -m "$timeout" -o /dev/null "$url" >/dev/null 2>&1 + return + fi + if command -v wget >/dev/null 2>&1; then + wget -q -T "$timeout" -O /dev/null "$url" >/dev/null 2>&1 + return + fi + if command -v python3 >/dev/null 2>&1; then + python3 - "$url" "$timeout" >/dev/null 2>&1 <<'PY' +import sys +import urllib.request + +with urllib.request.urlopen(sys.argv[1], timeout=float(sys.argv[2])) as response: + response.read(1) +PY + return + fi + return 1 +} + +bitfun_mirror_http_body() { + local url="$1" + local timeout="${2:-3}" + if command -v curl >/dev/null 2>&1; then + curl -fsS -m "$timeout" "$url" 2>/dev/null + return + fi + if command -v wget >/dev/null 2>&1; then + wget -q -T "$timeout" -O - "$url" 2>/dev/null + return + fi + if command -v python3 >/dev/null 2>&1; then + python3 - "$url" "$timeout" 2>/dev/null <<'PY' +import sys +import urllib.request + +with urllib.request.urlopen(sys.argv[1], timeout=float(sys.argv[2])) as response: + sys.stdout.buffer.write(response.read(65536)) +PY + return + fi + return 1 +} + +# Last-resort country lookup for minimal hosts without curl/wget/python. +# Uses bash /dev/tcp against the same plain-HTTP ip-api endpoint already used +# above, bounded by coreutils/busybox `timeout`. +bitfun_mirror_country_via_bash_tcp() { + if ! command -v timeout >/dev/null 2>&1; then + return 1 + fi + local response code + response="$( + timeout 4 bash -c ' + exec 3<>/dev/tcp/ip-api.com/80 || exit 1 + printf "GET /line/?fields=countryCode HTTP/1.1\r\nHost: ip-api.com\r\nConnection: close\r\n\r\n" >&3 + cat <&3 + ' 2>/dev/null || true + )" + code="$( + printf '%s' "$response" \ + | tr -d '\r' \ + | awk ' + body && /^[[:alpha:]][[:alpha:]]$/ { print toupper($0); exit } + /^$/ { body=1 } + ' + )" + if [ "${#code}" -eq 2 ]; then + echo "$code" + return 0 + fi + return 1 +} + +bitfun_mirror_detect_country() { + local code="" + code="$(bitfun_mirror_http_body "https://ipinfo.io/country" 3 | tr -d '[:space:]' | tr '[:lower:]' '[:upper:]')" + if [ "${#code}" -eq 2 ]; then + echo "$code" + return 0 + fi + code="$(bitfun_mirror_http_body "http://ip-api.com/line/?fields=countryCode" 3 | tr -d '[:space:]' | tr '[:lower:]' '[:upper:]')" + if [ "${#code}" -eq 2 ]; then + echo "$code" + return 0 + fi + code="$(bitfun_mirror_http_body "https://ifconfig.co/country-iso" 3 | tr -d '[:space:]' | tr '[:lower:]' '[:upper:]')" + if [ "${#code}" -eq 2 ]; then + echo "$code" + return 0 + fi + code="$(bitfun_mirror_country_via_bash_tcp 2>/dev/null || true)" + if [ "${#code}" -eq 2 ]; then + echo "$code" + return 0 + fi + return 1 +} + +bitfun_mirror_timezone_suggests_cn() { + local tz="" + if [ -n "${TZ:-}" ]; then + tz="$TZ" + elif [ -f /etc/timezone ]; then + tz="$(tr -d '[:space:]' /dev/null 2>&1; then + tz="$(timedatectl show -p Timezone --value 2>/dev/null || true)" + fi + case "$tz" in + Asia/Shanghai|Asia/Chongqing|Asia/Urumqi|Asia/Harbin|PRC) + return 0 + ;; + esac + return 1 +} + +bitfun_mirror_connectivity_suggests_cn() { + # GitHub hard to reach, but a mainland mirror works → likely CN. + if bitfun_mirror_http_ok "https://mirrors.aliyun.com/" 4; then + if ! bitfun_mirror_http_ok "https://github.com/" 4; then + return 0 + fi + fi + return 1 +} + +# Resolve BITFUN_MIRROR_MODE to cn|global. Returns 0 always. +bitfun_mirror_resolve_mode() { + local forced="${BITFUN_MIRROR:-auto}" + forced="$(echo "$forced" | tr '[:upper:]' '[:lower:]')" + case "$forced" in + cn|china|zh|zh-cn|zh_cn|1|true|yes) + export BITFUN_MIRROR_MODE=cn + export BITFUN_USE_CN_MIRROR=1 + return 0 + ;; + global|intl|international|off|0|false|no|overseas) + export BITFUN_MIRROR_MODE=global + export BITFUN_USE_CN_MIRROR=0 + return 0 + ;; + esac + + local country="" + country="$(bitfun_mirror_detect_country || true)" + if [ "$country" = "CN" ]; then + echo ">>> Region detect: public IP country=CN → China mirrors" + export BITFUN_MIRROR_MODE=cn + export BITFUN_USE_CN_MIRROR=1 + return 0 + fi + + if bitfun_mirror_timezone_suggests_cn; then + echo ">>> Region detect: timezone suggests mainland China → China mirrors" + export BITFUN_MIRROR_MODE=cn + export BITFUN_USE_CN_MIRROR=1 + return 0 + fi + + if bitfun_mirror_connectivity_suggests_cn; then + echo ">>> Region detect: GitHub unreachable + Aliyun reachable → China mirrors" + export BITFUN_MIRROR_MODE=cn + export BITFUN_USE_CN_MIRROR=1 + return 0 + fi + + if [ -n "$country" ]; then + echo ">>> Region detect: public IP country=${country} → global mirrors" + else + echo ">>> Region detect: inconclusive → global mirrors" + fi + export BITFUN_MIRROR_MODE=global + export BITFUN_USE_CN_MIRROR=0 + return 0 +} + +bitfun_mirror_export_urls() { + local git_upstream="${BITFUN_REPO_GIT_URL:-https://github.com/GCWing/BitFun.git}" + local tarball_upstream="${BITFUN_REPO_TARBALL_URL:-https://github.com/GCWing/BitFun/archive/refs/heads/main.tar.gz}" + local proxy="${BITFUN_GITHUB_PROXY:-https://ghfast.top/}" + local docker_get_upstream="${BITFUN_DOCKER_INSTALL_URL:-https://get.docker.com}" + + export BITFUN_APT_MIRROR="${BITFUN_APT_MIRROR:-mirrors.aliyun.com}" + export BITFUN_CARGO_SPARSE_URL="${BITFUN_CARGO_SPARSE_URL:-sparse+https://rsproxy.cn/index/}" + export BITFUN_RUSTUP_DIST_SERVER="${BITFUN_RUSTUP_DIST_SERVER:-https://rsproxy.cn}" + export BITFUN_DOCKER_REGISTRY_MIRRORS + BITFUN_DOCKER_REGISTRY_MIRRORS="$(bitfun_mirror_normalize_list "${BITFUN_DOCKER_REGISTRY_MIRRORS:-$(bitfun_mirror_default_docker_mirrors)}")" + + if [ "${BITFUN_MIRROR_MODE:-global}" != "cn" ]; then + export BITFUN_GITHUB_GIT_URL="$git_upstream" + export BITFUN_GITHUB_TARBALL_URL="$tarball_upstream" + export BITFUN_DOCKER_GET_URL="$docker_get_upstream" + export BITFUN_USE_CN_MIRROR=0 + return 0 + fi + + case "$proxy" in + */) ;; + *) proxy="${proxy}/" ;; + esac + export BITFUN_GITHUB_PROXY="$proxy" + + # Prefix-style proxy: https://ghfast.top/https://github.com/... + if [[ "$git_upstream" == https://github.com/* ]] || [[ "$git_upstream" == http://github.com/* ]]; then + export BITFUN_GITHUB_GIT_URL="${proxy}${git_upstream}" + else + export BITFUN_GITHUB_GIT_URL="$git_upstream" + fi + if [[ "$tarball_upstream" == https://github.com/* ]] || [[ "$tarball_upstream" == http://github.com/* ]]; then + export BITFUN_GITHUB_TARBALL_URL="${proxy}${tarball_upstream}" + else + export BITFUN_GITHUB_TARBALL_URL="$tarball_upstream" + fi + + if [ -n "${BITFUN_DOCKER_INSTALL_URL:-}" ]; then + export BITFUN_DOCKER_GET_URL="$BITFUN_DOCKER_INSTALL_URL" + else + # get.docker.com and most GitHub-prefix proxies return 403 from CN. + # Prefer the upstream install script mirrored on jsDelivr (same docker/docker-install). + export BITFUN_DOCKER_GET_URL="${BITFUN_DOCKER_GET_URL:-https://cdn.jsdelivr.net/gh/docker/docker-install@master/install.sh}" + fi + + export RUSTUP_DIST_SERVER="$BITFUN_RUSTUP_DIST_SERVER" + export RUSTUP_UPDATE_ROOT="${BITFUN_RUSTUP_UPDATE_ROOT:-${BITFUN_RUSTUP_DIST_SERVER}/rustup}" + export BITFUN_USE_CN_MIRROR=1 +} + +bitfun_mirror_backup_file() { + local src="$1" + local stamp="${BITFUN_MIRROR_BACKUP_STAMP:-$(date +%Y%m%d%H%M%S)}" + local dest_dir="${2:-/etc/bitfun}" + if [ ! -e "$src" ]; then + return 0 + fi + bitfun_mirror_priv mkdir -p "$dest_dir" 2>/dev/null || mkdir -p "$HOME/.bitfun/mirror-backup" 2>/dev/null || true + local base dest + base="$(basename "$src")" + if bitfun_mirror_priv test -d "$dest_dir" 2>/dev/null; then + dest="${dest_dir}/mirror-backup-${stamp}-${base}" + bitfun_mirror_priv cp -a "$src" "$dest" 2>/dev/null || true + else + dest="$HOME/.bitfun/mirror-backup/mirror-backup-${stamp}-${base}" + mkdir -p "$(dirname "$dest")" 2>/dev/null || true + cp -a "$src" "$dest" 2>/dev/null || true + fi +} + +bitfun_mirror_chown_to_home_owner() { + if [ ! -d "$HOME" ] || ! command -v stat >/dev/null 2>&1; then + return 0 + fi + local owner path_owner path + owner="$(stat -c '%u:%g' "$HOME" 2>/dev/null || true)" + if [ -z "$owner" ]; then + return 0 + fi + for path in "$@"; do + path_owner="$(stat -c '%u:%g' "$path" 2>/dev/null || true)" + if [ -z "$path_owner" ] || [ "$path_owner" = "$owner" ]; then + continue + fi + if [ "$(id -u)" = "0" ]; then + chown "$owner" "$path" 2>/dev/null || true + else + bitfun_mirror_priv chown "$owner" "$path" 2>/dev/null || true + fi + done +} + +bitfun_mirror_file_cksum() { + local path="$1" + if ! command -v cksum >/dev/null 2>&1; then + return 1 + fi + cksum "$path" 2>/dev/null | awk '{print $1 " " $2}' \ + || bitfun_mirror_priv cksum "$path" 2>/dev/null | awk '{print $1 " " $2}' +} + +bitfun_mirror_apply_apt_debian_family() { + local mirror="${BITFUN_APT_MIRROR:-mirrors.aliyun.com}" + local id="" version_codename="" id_like="" + # shellcheck disable=SC1091 + . /etc/os-release 2>/dev/null || true + id="${ID:-}" + version_codename="${VERSION_CODENAME:-}" + id_like="${ID_LIKE:-}" + + if [ -z "$version_codename" ]; then + echo ">>> apt mirror: skip (missing VERSION_CODENAME)" + return 0 + fi + + local suite_security="" + case "$id" in + ubuntu) + suite_security="${version_codename}-security" + ;; + debian) + suite_security="${version_codename}-security" + ;; + *) + case "$id_like" in + *ubuntu*) + id=ubuntu + suite_security="${version_codename}-security" + ;; + *debian*) + id=debian + suite_security="${version_codename}-security" + ;; + *) + echo ">>> apt mirror: unsupported distro '${id}'; rewriting common hosts only" + ;; + esac + ;; + esac + + bitfun_mirror_priv mkdir -p /etc/apt/sources.list.d /etc/bitfun 2>/dev/null || true + if [ -f /etc/apt/sources.list ]; then + bitfun_mirror_backup_file /etc/apt/sources.list + fi + + # Prefer a BitFun-owned list so cloud-init vendor files stay intact. + local list_file="/etc/apt/sources.list.d/bitfun-cn-mirror.list" + local tmp + tmp="$(mktemp)" + case "$id" in + ubuntu) + cat >"$tmp" <"$tmp" <"$rewritten" + bitfun_mirror_priv cp "$rewritten" /etc/apt/sources.list + rm -f "$rewritten" + fi + echo ">>> apt mirror: rewrote common upstream hosts → ${mirror}" + return 0 + ;; + esac + + bitfun_mirror_priv cp "$tmp" "$list_file" + rm -f "$tmp" + + # Disable conflicting default lists that still point overseas (keep backups). + local f + for f in /etc/apt/sources.list /etc/apt/sources.list.d/debian.sources \ + /etc/apt/sources.list.d/ubuntu.sources /etc/apt/sources.list.d/official-package-repositories.list; do + if [ -f "$f" ] && grep -Eq 'deb\.debian\.org|security\.debian\.org|archive\.ubuntu\.com|security\.ubuntu\.com' "$f" 2>/dev/null; then + bitfun_mirror_backup_file "$f" + bitfun_mirror_priv mv "$f" "${f}.bitfun-disabled" 2>/dev/null || true + fi + done + + echo ">>> apt mirror: enabled ${list_file} → ${mirror}" +} + +bitfun_mirror_apply_apt() { + if ! command -v apt-get >/dev/null 2>&1; then + return 0 + fi + if [ ! -f /etc/os-release ]; then + return 0 + fi + bitfun_mirror_apply_apt_debian_family || echo ">>> apt mirror: apply failed (continuing)" >&2 +} + +bitfun_mirror_write_docker_daemon_json() { + local mirrors_csv="$1" + local tmp py added_tmp state_dir state_file created_state prior_added daemon_json mirror checksum + tmp="$(mktemp)" + py="$(mktemp)" + added_tmp="$(mktemp)" + daemon_json="${BITFUN_DOCKER_DAEMON_JSON:-/etc/docker/daemon.json}" + state_dir="$HOME/.bitfun/mirror-state" + state_file="${state_dir}/docker-added-mirrors" + created_state="${state_dir}/docker-daemon-created.cksum" + mkdir -p "$state_dir" 2>/dev/null || true + prior_added="" + if [ -f "$state_file" ]; then + prior_added="$(cat "$state_file" 2>/dev/null || bitfun_mirror_priv cat "$state_file" 2>/dev/null || true)" + fi + cat >"$py" <<'PY' +import json, os, sys +path = sys.argv[5] +mirrors = [m for m in sys.argv[1].split() if m] +prior_added = [m for m in sys.argv[3].split() if m] +data = {} +if os.path.exists(path): + try: + with open(path, "r", encoding="utf-8") as f: + raw = f.read().strip() + if raw: + data = json.loads(raw) + if not isinstance(data, dict): + raise ValueError("daemon.json root must be an object") + except Exception as exc: + print(f"cannot safely parse {path}: {exc}", file=sys.stderr) + sys.exit(2) +existing = data.get("registry-mirrors") or [] +if not isinstance(existing, list): + print("daemon.json registry-mirrors must be an array", file=sys.stderr) + sys.exit(2) +legacy_managed = bool(data.pop("bitfun-cn-mirror", False)) +merged = [] +for item in list(existing) + mirrors: + if item and item not in merged: + merged.append(item) +data["registry-mirrors"] = merged +added = [] +for item in prior_added: + if item and item not in added: + added.append(item) +for item in mirrors: + if item not in existing or legacy_managed: + if item not in added: + added.append(item) +out = sys.argv[2] +with open(out, "w", encoding="utf-8") as f: + json.dump(data, f, indent=2) + f.write("\n") +with open(sys.argv[4], "w", encoding="utf-8") as f: + for item in added: + f.write(item + "\n") +PY + if command -v python3 >/dev/null 2>&1; then + if [ -f "$daemon_json" ]; then + bitfun_mirror_backup_file "$daemon_json" + fi + if ! bitfun_mirror_priv mkdir -p "$(dirname "$daemon_json")"; then + echo ">>> docker mirror: cannot create $(dirname "$daemon_json")" >&2 + rm -f "$tmp" "$py" "$added_tmp" + return 1 + fi + if bitfun_mirror_priv python3 "$py" "$mirrors_csv" "$tmp" "$prior_added" "$added_tmp" "$daemon_json"; then + if command -v dockerd >/dev/null 2>&1 \ + && ! bitfun_mirror_priv dockerd --validate --config-file "$tmp" >/dev/null; then + echo ">>> docker mirror: generated daemon.json failed dockerd validation; not installing it" >&2 + rm -f "$tmp" "$py" "$added_tmp" + return 1 + fi + if ! bitfun_mirror_priv cp "$tmp" "$daemon_json"; then + echo ">>> docker mirror: cannot install ${daemon_json}" >&2 + rm -f "$tmp" "$py" "$added_tmp" + return 1 + fi + if cp "$added_tmp" "$state_file" 2>/dev/null \ + || bitfun_mirror_priv cp "$added_tmp" "$state_file" 2>/dev/null; then + : + else + echo ">>> docker mirror: could not persist rollback state" >&2 + rm -f "$state_dir/version" 2>/dev/null \ + || bitfun_mirror_priv rm -f "$state_dir/version" 2>/dev/null \ + || true + fi + chmod 644 "$state_file" 2>/dev/null || true + bitfun_mirror_chown_to_home_owner "$state_dir" "$state_file" + if [ -f "$created_state" ]; then + checksum="$(bitfun_mirror_file_cksum "$daemon_json" || true)" + if [ -n "$checksum" ]; then + echo "$checksum" >"$created_state" + chmod 644 "$created_state" 2>/dev/null || true + bitfun_mirror_chown_to_home_owner "$created_state" + fi + fi + echo ">>> docker mirror: merged registry-mirrors into ${daemon_json}" + else + echo ">>> docker mirror: safe JSON merge failed; leaving daemon.json untouched" >&2 + rm -f "$tmp" "$py" "$added_tmp" + return 1 + fi + else + if [ -f "$daemon_json" ]; then + echo ">>> docker mirror: python3 missing; leaving existing daemon.json untouched" >&2 + rm -f "$tmp" "$py" "$added_tmp" + return 1 + else + if ! bitfun_mirror_priv mkdir -p "$(dirname "$daemon_json")"; then + echo ">>> docker mirror: cannot create $(dirname "$daemon_json")" >&2 + rm -f "$tmp" "$py" "$added_tmp" + return 1 + fi + { + echo '{' + echo ' "registry-mirrors": [' + local first=1 m + for m in $mirrors_csv; do + if [ "$first" -eq 1 ]; then first=0; else echo ','; fi + printf ' "%s"' "$m" + done + echo '' + echo ' ]' + echo '}' + } >"$tmp" + if ! bitfun_mirror_priv cp "$tmp" "$daemon_json"; then + echo ">>> docker mirror: cannot install ${daemon_json}" >&2 + rm -f "$tmp" "$py" "$added_tmp" + return 1 + fi + : >"$added_tmp" + for mirror in $mirrors_csv; do + printf '%s\n' "$mirror" >>"$added_tmp" + done + if cp "$added_tmp" "$state_file" 2>/dev/null \ + || bitfun_mirror_priv cp "$added_tmp" "$state_file" 2>/dev/null; then + : + else + echo ">>> docker mirror: could not persist rollback state" >&2 + rm -f "$state_dir/version" 2>/dev/null \ + || bitfun_mirror_priv rm -f "$state_dir/version" 2>/dev/null \ + || true + fi + chmod 644 "$state_file" 2>/dev/null || true + bitfun_mirror_chown_to_home_owner "$state_dir" "$state_file" + checksum="$(bitfun_mirror_file_cksum "$daemon_json" || true)" + if [ -n "$checksum" ]; then + if echo "$checksum" >"$created_state"; then + chmod 644 "$created_state" 2>/dev/null || true + bitfun_mirror_chown_to_home_owner "$created_state" + else + echo ">>> docker mirror: could not persist created-file checksum for rollback" >&2 + rm -f "$state_dir/version" 2>/dev/null \ + || bitfun_mirror_priv rm -f "$state_dir/version" 2>/dev/null \ + || true + fi + fi + echo ">>> docker mirror: wrote ${daemon_json}" + fi + fi + rm -f "$tmp" "$py" "$added_tmp" +} + +bitfun_mirror_restart_docker_if_needed() { + if ! command -v docker >/dev/null 2>&1; then + return 0 + fi + if docker info >/dev/null 2>&1 || bitfun_mirror_priv docker info >/dev/null 2>&1; then + echo ">>> docker mirror: restarting docker to apply registry-mirrors..." + bitfun_mirror_priv systemctl restart docker 2>/dev/null \ + || bitfun_mirror_priv service docker restart 2>/dev/null \ + || true + sleep 1 + fi +} + +bitfun_mirror_apply_docker_daemon() { + local mirrors + mirrors="$(bitfun_mirror_normalize_list "${BITFUN_DOCKER_REGISTRY_MIRRORS:-$(bitfun_mirror_default_docker_mirrors)}")" + if [ -z "$mirrors" ]; then + return 0 + fi + if bitfun_mirror_write_docker_daemon_json "$mirrors"; then + bitfun_mirror_restart_docker_if_needed || true + else + echo ">>> docker mirror: apply failed (continuing)" >&2 + fi +} + +bitfun_mirror_restore_apt() { + local list_file="/etc/apt/sources.list.d/bitfun-cn-mirror.list" + local changed=0 restore_failed=0 original disabled + for original in /etc/apt/sources.list /etc/apt/sources.list.d/debian.sources \ + /etc/apt/sources.list.d/ubuntu.sources /etc/apt/sources.list.d/official-package-repositories.list; do + disabled="${original}.bitfun-disabled" + if [ ! -f "$disabled" ]; then + continue + fi + if [ -e "$original" ]; then + echo ">>> apt mirror: not restoring ${original}; a replacement already exists" >&2 + restore_failed=1 + continue + fi + if bitfun_mirror_priv mv "$disabled" "$original" 2>/dev/null; then + changed=1 + else + echo ">>> apt mirror: failed to restore ${original}" >&2 + restore_failed=1 + fi + done + if [ -f "$list_file" ] && [ "$restore_failed" -eq 0 ]; then + if bitfun_mirror_priv rm -f "$list_file" 2>/dev/null; then + changed=1 + else + echo ">>> apt mirror: failed to remove ${list_file}" >&2 + fi + elif [ -f "$list_file" ]; then + echo ">>> apt mirror: keeping ${list_file} because an upstream source could not be restored" >&2 + fi + if [ "$changed" -eq 1 ]; then + echo ">>> apt mirror: removed BitFun mirror list and restored disabled upstream sources" + fi +} + +bitfun_mirror_remove_docker_daemon() { + local daemon_json="${BITFUN_DOCKER_DAEMON_JSON:-/etc/docker/daemon.json}" + local state_file="$HOME/.bitfun/mirror-state/docker-added-mirrors" + local created_state="$HOME/.bitfun/mirror-state/docker-daemon-created.cksum" + local version_file="$HOME/.bitfun/mirror-state/version" + local mode_file="$HOME/.bitfun/mirror-mode" + local managed=0 mirrors_to_remove="" tmp py status expected_checksum actual_checksum + + if [ -f "$state_file" ]; then + managed=1 + mirrors_to_remove="$(cat "$state_file" 2>/dev/null || bitfun_mirror_priv cat "$state_file" 2>/dev/null || true)" + elif [ ! -f "$version_file" ] && [ "$(cat "$mode_file" 2>/dev/null || true)" = "cn" ]; then + # Compatibility cleanup for hosts touched by early versions that did not + # record exactly which registry mirrors they added. + managed=1 + mirrors_to_remove="$(bitfun_mirror_default_docker_mirrors)" + elif [ -r "$daemon_json" ] && grep -q '"bitfun-cn-mirror"' "$daemon_json" 2>/dev/null; then + managed=1 + mirrors_to_remove="$(bitfun_mirror_default_docker_mirrors)" + fi + if [ "$managed" -ne 1 ]; then + if [ -f "$version_file" ] && [ ! -f "$state_file" ]; then + rm -f "$version_file" "$created_state" 2>/dev/null \ + || bitfun_mirror_priv rm -f "$version_file" "$created_state" 2>/dev/null \ + || true + fi + return 0 + fi + if [ ! -f "$daemon_json" ]; then + rm -f "$state_file" "$version_file" "$created_state" 2>/dev/null \ + || bitfun_mirror_priv rm -f "$state_file" "$version_file" "$created_state" 2>/dev/null \ + || true + return 0 + fi + if ! command -v python3 >/dev/null 2>&1; then + if [ -f "$created_state" ]; then + expected_checksum="$(cat "$created_state" 2>/dev/null || true)" + actual_checksum="$(bitfun_mirror_file_cksum "$daemon_json" || true)" + if [ -n "$expected_checksum" ] && [ "$actual_checksum" = "$expected_checksum" ]; then + bitfun_mirror_backup_file "$daemon_json" + if bitfun_mirror_priv rm -f "$daemon_json"; then + rm -f "$state_file" "$version_file" "$created_state" 2>/dev/null \ + || bitfun_mirror_priv rm -f "$state_file" "$version_file" "$created_state" 2>/dev/null \ + || true + echo ">>> docker mirror: removed BitFun-created ${daemon_json}" + if command -v docker >/dev/null 2>&1; then + bitfun_mirror_priv systemctl restart docker 2>/dev/null \ + || bitfun_mirror_priv service docker restart 2>/dev/null \ + || true + fi + return 0 + fi + fi + fi + echo ">>> docker mirror: python3 missing; cannot safely remove managed daemon.json entries" >&2 + return 1 + fi + + tmp="$(mktemp)" + py="$(mktemp)" + cat >"$py" <<'PY' +import json, os, sys + +path = sys.argv[3] +remove = {m for m in sys.argv[1].split() if m} +try: + with open(path, "r", encoding="utf-8") as f: + data = json.load(f) +except Exception as exc: + print(f"cannot safely parse {path}: {exc}", file=sys.stderr) + sys.exit(2) +if not isinstance(data, dict): + print("daemon.json root must be an object", file=sys.stderr) + sys.exit(2) +before = json.dumps(data, sort_keys=True) +data.pop("bitfun-cn-mirror", None) +existing = data.get("registry-mirrors") +if isinstance(existing, list): + kept = [item for item in existing if item not in remove] + if kept: + data["registry-mirrors"] = kept + else: + data.pop("registry-mirrors", None) +after = json.dumps(data, sort_keys=True) +if before == after: + sys.exit(3) +with open(sys.argv[2], "w", encoding="utf-8") as f: + json.dump(data, f, indent=2) + f.write("\n") +PY + + status=0 + bitfun_mirror_priv python3 "$py" "$mirrors_to_remove" "$tmp" "$daemon_json" || status=$? + if [ "$status" -eq 3 ]; then + rm -f "$tmp" "$py" + rm -f "$state_file" 2>/dev/null || bitfun_mirror_priv rm -f "$state_file" 2>/dev/null || true + rm -f "$version_file" 2>/dev/null || bitfun_mirror_priv rm -f "$version_file" 2>/dev/null || true + rm -f "$created_state" 2>/dev/null || bitfun_mirror_priv rm -f "$created_state" 2>/dev/null || true + return 0 + fi + if [ "$status" -ne 0 ]; then + echo ">>> docker mirror: failed to remove managed daemon.json entries" >&2 + rm -f "$tmp" "$py" + return 1 + fi + if command -v dockerd >/dev/null 2>&1 \ + && ! bitfun_mirror_priv dockerd --validate --config-file "$tmp" >/dev/null; then + echo ">>> docker mirror: restored daemon.json failed dockerd validation; leaving current file untouched" >&2 + rm -f "$tmp" "$py" + return 1 + fi + bitfun_mirror_backup_file "$daemon_json" + if ! bitfun_mirror_priv cp "$tmp" "$daemon_json"; then + echo ">>> docker mirror: failed to restore ${daemon_json}" >&2 + rm -f "$tmp" "$py" + return 1 + fi + rm -f "$tmp" "$py" + rm -f "$state_file" 2>/dev/null || bitfun_mirror_priv rm -f "$state_file" 2>/dev/null || true + rm -f "$version_file" 2>/dev/null || bitfun_mirror_priv rm -f "$version_file" 2>/dev/null || true + rm -f "$created_state" 2>/dev/null || bitfun_mirror_priv rm -f "$created_state" 2>/dev/null || true + echo ">>> docker mirror: removed BitFun-managed registry mirrors from ${daemon_json}" + if command -v docker >/dev/null 2>&1; then + bitfun_mirror_priv systemctl restart docker 2>/dev/null \ + || bitfun_mirror_priv service docker restart 2>/dev/null \ + || true + fi +} + +bitfun_mirror_remove_cargo_managed_block() { + local cargo_home="${CARGO_HOME:-$HOME/.cargo}" + local cfg="${cargo_home}/config.toml" + local tmp backup + if [ ! -f "$cfg" ]; then + return 0 + fi + if ! grep -q '^# >>> BITFUN-CN-MIRROR$' "$cfg" 2>/dev/null \ + && ! bitfun_mirror_priv grep -q '^# >>> BITFUN-CN-MIRROR$' "$cfg" 2>/dev/null; then + return 0 + fi + mkdir -p "$HOME/.bitfun/mirror-backup" 2>/dev/null || true + backup="$HOME/.bitfun/mirror-backup/cargo-config.toml.$(date +%Y%m%d%H%M%S)" + if cp -a "$cfg" "$backup" 2>/dev/null || bitfun_mirror_priv cp -a "$cfg" "$backup" 2>/dev/null; then + : + else + echo ">>> cargo mirror: cannot back up ${cfg}; leaving it untouched" >&2 + return 1 + fi + bitfun_mirror_chown_to_home_owner "$HOME/.bitfun/mirror-backup" "$backup" + tmp="$(mktemp)" + # shellcheck disable=SC2016 + if ! awk ' + BEGIN {skip=0} + /^# >>> BITFUN-CN-MIRROR$/ {skip=1; next} + /^# <<< BITFUN-CN-MIRROR$/ {skip=0; next} + skip==0 {print} + ' "$cfg" >"$tmp" 2>/dev/null; then + bitfun_mirror_priv awk ' + BEGIN {skip=0} + /^# >>> BITFUN-CN-MIRROR$/ {skip=1; next} + /^# <<< BITFUN-CN-MIRROR$/ {skip=0; next} + skip==0 {print} + ' "$cfg" >"$tmp" + fi + if cp "$tmp" "$cfg" 2>/dev/null || bitfun_mirror_priv cp "$tmp" "$cfg" 2>/dev/null; then + : + else + echo ">>> cargo mirror: failed to remove legacy managed block from ${cfg}" >&2 + rm -f "$tmp" + return 1 + fi + bitfun_mirror_chown_to_home_owner "$cargo_home" "$cfg" + rm -f "$tmp" + echo ">>> cargo mirror: removed legacy BitFun block from ${cfg}; relay Cargo mirroring is build-local" +} + +bitfun_mirror_restore_host() { + echo ">>> Restoring global host sources managed by BitFun..." + bitfun_mirror_restore_apt || true + bitfun_mirror_remove_docker_daemon || true + bitfun_mirror_remove_cargo_managed_block || true +} + +bitfun_mirror_apply_host() { + if [ "${BITFUN_MIRROR_SKIP_APPLY:-0}" = "1" ]; then + echo ">>> mirror apply skipped (BITFUN_MIRROR_SKIP_APPLY=1)" + return 0 + fi + if [ "${BITFUN_MIRROR_MODE:-global}" != "cn" ]; then + return 0 + fi + mkdir -p "$HOME/.bitfun/mirror-state" 2>/dev/null || true + echo "2" >"$HOME/.bitfun/mirror-state/version" 2>/dev/null || true + bitfun_mirror_chown_to_home_owner \ + "$HOME/.bitfun" "$HOME/.bitfun/mirror-state" "$HOME/.bitfun/mirror-state/version" + echo ">>> Applying China host mirrors (apt / docker; Cargo stays build-local)..." + bitfun_mirror_apply_apt || true + bitfun_mirror_apply_docker_daemon || true + # Relay compilation happens inside Docker. Do not mutate the SSH user's + # global Cargo config; older versions did and could create duplicate TOML + # tables or root-owned ~/.cargo directories. + bitfun_mirror_remove_cargo_managed_block || true + mkdir -p "$HOME/.bitfun" 2>/dev/null || true + echo "cn" >"$HOME/.bitfun/mirror-mode" 2>/dev/null || true + bitfun_mirror_chown_to_home_owner "$HOME/.bitfun" "$HOME/.bitfun/mirror-mode" +} + +# Install Docker Engine from Aliyun docker-ce (CN). Returns 0 on success. +bitfun_mirror_install_docker_aliyun() { + if ! command -v apt-get >/dev/null 2>&1 && ! command -v dnf >/dev/null 2>&1 && ! command -v yum >/dev/null 2>&1; then + return 1 + fi + # shellcheck disable=SC1091 + . /etc/os-release 2>/dev/null || true + local id="${ID:-}" version_codename="${VERSION_CODENAME:-}" arch + arch="$(dpkg --print-architecture 2>/dev/null || uname -m)" + case "$arch" in + x86_64) arch=amd64 ;; + aarch64) arch=arm64 ;; + esac + + echo ">>> Installing Docker from Aliyun docker-ce mirror..." + if command -v apt-get >/dev/null 2>&1; then + local docker_ce_distro="" + case "$id" in + ubuntu|linuxmint|pop) docker_ce_distro=ubuntu ;; + debian|raspbian) docker_ce_distro=debian ;; + *) + case "${ID_LIKE:-}" in + *ubuntu*) docker_ce_distro=ubuntu ;; + *debian*) docker_ce_distro=debian ;; + *) return 1 ;; + esac + ;; + esac + [ -n "$version_codename" ] || return 1 + bitfun_mirror_priv apt-get update -y + bitfun_mirror_priv apt-get install -y ca-certificates curl + bitfun_mirror_priv install -m 0755 -d /etc/apt/keyrings + curl -fsSL --retry 3 "https://mirrors.aliyun.com/docker-ce/linux/${docker_ce_distro}/gpg" \ + | bitfun_mirror_priv tee /etc/apt/keyrings/docker.asc >/dev/null + bitfun_mirror_priv chmod a+r /etc/apt/keyrings/docker.asc + echo "deb [arch=${arch} signed-by=/etc/apt/keyrings/docker.asc] https://mirrors.aliyun.com/docker-ce/linux/${docker_ce_distro} ${version_codename} stable" \ + | bitfun_mirror_priv tee /etc/apt/sources.list.d/docker.list >/dev/null + bitfun_mirror_priv apt-get update -y + bitfun_mirror_priv apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin + return 0 + fi + + if command -v dnf >/dev/null 2>&1 || command -v yum >/dev/null 2>&1; then + local pkg=yum + command -v dnf >/dev/null 2>&1 && pkg=dnf + bitfun_mirror_priv tee /etc/yum.repos.d/docker-ce.repo >/dev/null <>> Fetching Docker install script: ${url}" + curl -fsSL --retry 3 "$url" -o "$dest" +} + +bitfun_mirror_init() { + bitfun_mirror_parse_args "$@" + bitfun_mirror_resolve_mode + bitfun_mirror_export_urls + echo ">>> Mirror mode: ${BITFUN_MIRROR_MODE} (BITFUN_USE_CN_MIRROR=${BITFUN_USE_CN_MIRROR})" + if [ "${BITFUN_MIRROR_MODE}" = "cn" ]; then + echo ">>> GitHub git URL: ${BITFUN_GITHUB_GIT_URL}" + echo ">>> GitHub tarball URL: ${BITFUN_GITHUB_TARBALL_URL}" + echo ">>> Docker get URL: ${BITFUN_DOCKER_GET_URL}" + echo ">>> apt mirror: ${BITFUN_APT_MIRROR}" + echo ">>> cargo sparse: ${BITFUN_CARGO_SPARSE_URL}" + echo ">>> docker registries: ${BITFUN_DOCKER_REGISTRY_MIRRORS}" + bitfun_mirror_apply_host + else + bitfun_mirror_restore_host + mkdir -p "$HOME/.bitfun" 2>/dev/null || true + echo "global" >"$HOME/.bitfun/mirror-mode" 2>/dev/null || true + bitfun_mirror_chown_to_home_owner "$HOME/.bitfun" "$HOME/.bitfun/mirror-mode" + fi +} + +# When executed directly as mirror.sh (not sourced, not text-embedded), run init. +# Basename guard prevents auto-run when this file is concatenated into Desktop +# driver scripts where BASH_SOURCE[0] == $0. +if [[ "${BASH_SOURCE[0]:-}" == "${0}" ]] \ + && [[ "$(basename "${BASH_SOURCE[0]}")" == "mirror.sh" ]]; then + set -euo pipefail + bitfun_mirror_init "$@" +fi diff --git a/src/crates/services/services-integrations/AGENTS.md b/src/crates/services/services-integrations/AGENTS.md index 23e9e7811e..be1529a65d 100644 --- a/src/crates/services/services-integrations/AGENTS.md +++ b/src/crates/services/services-integrations/AGENTS.md @@ -37,9 +37,11 @@ slices that are outside pure product logic but still platform-neutral. explicit remote SSH features. - One-click relay self-deploy (`remote_ssh/relay_deploy.rs`) stages embedded scripts under `~/.bitfun/relay-deploy/` and clones source to - `~/.bitfun/relay-src/` (never `$HOME/bitfun`). Invariants: - `src/web-ui/src/features/relay-deploy/README.md`. Desktop Tauri wrapper: - `src/apps/desktop/src/api/relay_deploy_api.rs`. + `~/.bitfun/relay-src/` (never `$HOME/bitfun`). Embeds + `src/apps/relay-server/mirror.sh` and runs `bitfun_mirror_init` before apt / + Docker install / GitHub sync so mainland China hosts use configured mirrors. + Invariants: `src/web-ui/src/features/relay-deploy/README.md`. Desktop Tauri + wrapper: `src/apps/desktop/src/api/relay_deploy_api.rs`. - Workspace search owns the local flashgrep daemon/session lifecycle and indexed-search result conversion behind `workspace-search`; product config and workspace bootstrap stay in the core facade as injected hooks. diff --git a/src/crates/services/services-integrations/src/remote_ssh/relay_deploy.rs b/src/crates/services/services-integrations/src/remote_ssh/relay_deploy.rs index 12b6ab7513..51feb827f3 100644 --- a/src/crates/services/services-integrations/src/remote_ssh/relay_deploy.rs +++ b/src/crates/services/services-integrations/src/remote_ssh/relay_deploy.rs @@ -19,6 +19,8 @@ //! Product / regression invariants (wizard + entry points): //! `src/web-ui/src/features/relay-deploy/README.md`. Do not change clone destination, //! password handoff, or “already deployed” semantics without updating that doc. +//! China mirror helpers live in `src/apps/relay-server/mirror.sh` and are embedded +//! here so detection/apply runs before GitHub/Docker downloads. use anyhow::{anyhow, Result}; use serde::{Deserialize, Serialize}; @@ -47,6 +49,12 @@ const REPO_GIT_URL: &str = "https://github.com/GCWing/BitFun.git"; const REPO_GIT_BRANCH: &str = "main"; /// Tarball fallback when git is unavailable or clone/fetch fails. const REPO_TARBALL_URL: &str = "https://github.com/GCWing/BitFun/archive/refs/heads/main.tar.gz"; +/// Canonical China-mirror helper (shared with `src/apps/relay-server/deploy.sh`). +/// Embedded so Desktop orchestration can apply mirrors before the git clone. +const RELAY_MIRROR_SH: &str = include_str!(concat!( + env!("CARGO_MANIFEST_DIR"), + "/../../../apps/relay-server/mirror.sh" +)); /// Remote directory (relative to the SSH user's home) holding deploy state. const DEPLOY_STATE_DIR: &str = ".bitfun/relay-deploy"; /// BitFun-managed source checkout (relative to home). Must stay under `.bitfun/` @@ -773,8 +781,17 @@ async fn exec_ok(manager: &SSHConnectionManager, connection_id: &str, command: & } /// Shared interactive prepare helpers embedded in driver scripts. -fn prepare_helpers_bash() -> &'static str { - r#" +fn prepare_helpers_bash() -> String { + // Mirror helpers first so prepare/install/deploy can call bitfun_mirror_init + // before apt/git/docker downloads. + format!( + r#" +# --- begin BitFun relay mirror.sh (embedded) --- +{mirror} +# --- end BitFun relay mirror.sh --- +"#, + mirror = RELAY_MIRROR_SH + ) + r#" # Privilege helpers: # - Never use `sudo -v` when NOPASSWD is set — on many cloud images `sudo -v` # still demands a password even though `sudo -n true` works. @@ -919,6 +936,8 @@ bitfun_docker() { bitfun_run_deploy_sh() { local dir="$1" local port="${RELAY_PORT:-9700}" + # Prefer already-resolved mirror mode so deploy.sh does not re-probe. + local mirror_mode="${BITFUN_MIRROR:-${BITFUN_MIRROR_MODE:-auto}}" # DOCKER_BUILDKIT is required for Dockerfile cargo registry/git/target mounts. case "${BITFUN_DOCKER_MODE:-direct}" in sudo) @@ -926,20 +945,38 @@ bitfun_run_deploy_sh() { sudo -n -E env RELAY_PORT="$port" RELAY_CARGO_BUILD_JOBS="${RELAY_CARGO_BUILD_JOBS:-}" \ DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 BUILDKIT_PROGRESS=plain \ DOCKER_CONFIG="${DOCKER_CONFIG:-}" \ + BITFUN_MIRROR="$mirror_mode" \ + BITFUN_USE_CN_MIRROR="${BITFUN_USE_CN_MIRROR:-0}" \ + BITFUN_APT_MIRROR="${BITFUN_APT_MIRROR:-}" \ + BITFUN_CARGO_SPARSE_URL="${BITFUN_CARGO_SPARSE_URL:-}" \ + BITFUN_DOCKER_REGISTRY_MIRRORS="${BITFUN_DOCKER_REGISTRY_MIRRORS:-}" \ + BITFUN_GITHUB_PROXY="${BITFUN_GITHUB_PROXY:-}" \ bash "$dir/deploy.sh" else sudo -E env RELAY_PORT="$port" RELAY_CARGO_BUILD_JOBS="${RELAY_CARGO_BUILD_JOBS:-}" \ DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 BUILDKIT_PROGRESS=plain \ DOCKER_CONFIG="${DOCKER_CONFIG:-}" \ + BITFUN_MIRROR="$mirror_mode" \ + BITFUN_USE_CN_MIRROR="${BITFUN_USE_CN_MIRROR:-0}" \ + BITFUN_APT_MIRROR="${BITFUN_APT_MIRROR:-}" \ + BITFUN_CARGO_SPARSE_URL="${BITFUN_CARGO_SPARSE_URL:-}" \ + BITFUN_DOCKER_REGISTRY_MIRRORS="${BITFUN_DOCKER_REGISTRY_MIRRORS:-}" \ + BITFUN_GITHUB_PROXY="${BITFUN_GITHUB_PROXY:-}" \ bash "$dir/deploy.sh" fi ;; sg) - sg docker -c "env RELAY_PORT='$port' RELAY_CARGO_BUILD_JOBS='${RELAY_CARGO_BUILD_JOBS:-}' DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 BUILDKIT_PROGRESS=plain DOCKER_CONFIG='${DOCKER_CONFIG:-}' bash '$dir/deploy.sh'" + sg docker -c "env RELAY_PORT='$port' RELAY_CARGO_BUILD_JOBS='${RELAY_CARGO_BUILD_JOBS:-}' DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 BUILDKIT_PROGRESS=plain DOCKER_CONFIG='${DOCKER_CONFIG:-}' BITFUN_MIRROR='$mirror_mode' BITFUN_USE_CN_MIRROR='${BITFUN_USE_CN_MIRROR:-0}' BITFUN_APT_MIRROR='${BITFUN_APT_MIRROR:-}' BITFUN_CARGO_SPARSE_URL='${BITFUN_CARGO_SPARSE_URL:-}' BITFUN_DOCKER_REGISTRY_MIRRORS='${BITFUN_DOCKER_REGISTRY_MIRRORS:-}' BITFUN_GITHUB_PROXY='${BITFUN_GITHUB_PROXY:-}' bash '$dir/deploy.sh'" ;; *) env RELAY_PORT="$port" RELAY_CARGO_BUILD_JOBS="${RELAY_CARGO_BUILD_JOBS:-}" \ DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 BUILDKIT_PROGRESS=plain \ + BITFUN_MIRROR="$mirror_mode" \ + BITFUN_USE_CN_MIRROR="${BITFUN_USE_CN_MIRROR:-0}" \ + BITFUN_APT_MIRROR="${BITFUN_APT_MIRROR:-}" \ + BITFUN_CARGO_SPARSE_URL="${BITFUN_CARGO_SPARSE_URL:-}" \ + BITFUN_DOCKER_REGISTRY_MIRRORS="${BITFUN_DOCKER_REGISTRY_MIRRORS:-}" \ + BITFUN_GITHUB_PROXY="${BITFUN_GITHUB_PROXY:-}" \ bash "$dir/deploy.sh" ;; esac @@ -988,6 +1025,10 @@ touch "$PREPARE_FLAG" echo ">>> prepare starting (uid=$(id -u) home=$HOME)" | tee -a "$LOG" cleanup_prepare() {{ rm -f "$PREPARE_FLAG"; }} trap cleanup_prepare EXIT +# Region/mirrors before apt tool install and Docker/GitHub downloads. +export BITFUN_REPO_GIT_URL="{REPO_GIT_URL}" +export BITFUN_REPO_TARBALL_URL="{REPO_TARBALL_URL}" +bitfun_mirror_init bitfun_ensure_tools export DOCKER_CONFIG="${{DOCKER_CONFIG:-$HOME/.bitfun/docker-config}}" mkdir -p "$DOCKER_CONFIG" @@ -1024,9 +1065,13 @@ if [ "{kind}" = "install" ]; then export BITFUN_KEEP_HOME="${{BITFUN_KEEP_HOME:-$HOME}}" set +e if command -v stdbuf >/dev/null 2>&1; then - stdbuf -oL -eL env BITFUN_KEEP_HOME="$BITFUN_KEEP_HOME" bash "$BODY" 2>&1 | tee -a "$LOG" + stdbuf -oL -eL env BITFUN_KEEP_HOME="$BITFUN_KEEP_HOME" \ + BITFUN_MIRROR="${{BITFUN_MIRROR:-${{BITFUN_MIRROR_MODE:-auto}}}}" \ + bash "$BODY" 2>&1 | tee -a "$LOG" else - env BITFUN_KEEP_HOME="$BITFUN_KEEP_HOME" bash "$BODY" 2>&1 | tee -a "$LOG" + env BITFUN_KEEP_HOME="$BITFUN_KEEP_HOME" \ + BITFUN_MIRROR="${{BITFUN_MIRROR:-${{BITFUN_MIRROR_MODE:-auto}}}}" \ + bash "$BODY" 2>&1 | tee -a "$LOG" fi code=${{PIPESTATUS[0]}} set -e @@ -1045,6 +1090,14 @@ echo ">>> Starting background task (log: $LOG)" | tee -a "$LOG" nohup env BITFUN_DOCKER_MODE="$BITFUN_DOCKER_MODE" DOCKER_CONFIG="$DOCKER_CONFIG" \ RELAY_CARGO_BUILD_JOBS="${{RELAY_CARGO_BUILD_JOBS:-}}" \ DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 BUILDKIT_PROGRESS=plain \ + BITFUN_MIRROR="${{BITFUN_MIRROR:-${{BITFUN_MIRROR_MODE:-auto}}}}" \ + BITFUN_USE_CN_MIRROR="${{BITFUN_USE_CN_MIRROR:-0}}" \ + BITFUN_APT_MIRROR="${{BITFUN_APT_MIRROR:-}}" \ + BITFUN_CARGO_SPARSE_URL="${{BITFUN_CARGO_SPARSE_URL:-}}" \ + BITFUN_DOCKER_REGISTRY_MIRRORS="${{BITFUN_DOCKER_REGISTRY_MIRRORS:-}}" \ + BITFUN_GITHUB_PROXY="${{BITFUN_GITHUB_PROXY:-}}" \ + BITFUN_REPO_GIT_URL="${{BITFUN_REPO_GIT_URL:-}}" \ + BITFUN_REPO_TARBALL_URL="${{BITFUN_REPO_TARBALL_URL:-}}" \ "${{RUNNER[@]}}" "$BODY" >"$LOG" 2>&1 < /dev/null & echo $! >"$PIDF" rm -f "$PREPARE_FLAG" @@ -1056,6 +1109,8 @@ exec tail -n +1 -f "$LOG" stem = stem, kind = kind, helpers = helpers, + REPO_GIT_URL = REPO_GIT_URL, + REPO_TARBALL_URL = REPO_TARBALL_URL, ) } @@ -1080,18 +1135,39 @@ fi if [ -z "$DEPLOY_USER" ] || [ "$DEPLOY_USER" = "root" ]; then DEPLOY_USER="$(id -un)" fi -echo ">>> Installing Docker (get.docker.com) as uid=$(id -u) for user=$DEPLOY_USER ..." -curl -fsSL --retry 3 https://get.docker.com -o /tmp/bitfun-get-docker.sh +export BITFUN_REPO_GIT_URL="{REPO_GIT_URL}" +export BITFUN_REPO_TARBALL_URL="{REPO_TARBALL_URL}" +bitfun_mirror_init +echo ">>> Installing Docker as uid=$(id -u) for user=$DEPLOY_USER (mirror_mode=${{BITFUN_MIRROR_MODE:-global}}) ..." +INSTALLED=0 +if [ "${{BITFUN_MIRROR_MODE:-}}" = "cn" ]; then + if bitfun_mirror_install_docker_aliyun; then + INSTALLED=1 + else + echo ">>> Aliyun docker-ce install failed; falling back to get.docker.com mirror..." + fi +fi +if [ "$INSTALLED" != "1" ]; then + bitfun_mirror_fetch_docker_install_script /tmp/bitfun-get-docker.sh \ + || curl -fsSL --retry 3 https://get.docker.com -o /tmp/bitfun-get-docker.sh + if [ "$(id -u)" = "0" ]; then + sh /tmp/bitfun-get-docker.sh + else + bitfun_priv sh /tmp/bitfun-get-docker.sh + fi + rm -f /tmp/bitfun-get-docker.sh +fi if [ "$(id -u)" = "0" ]; then - sh /tmp/bitfun-get-docker.sh systemctl enable --now docker usermod -aG docker "$DEPLOY_USER" || true else - bitfun_priv sh /tmp/bitfun-get-docker.sh bitfun_priv systemctl enable --now docker bitfun_priv usermod -aG docker "$DEPLOY_USER" fi -rm -f /tmp/bitfun-get-docker.sh +# Re-apply Docker registry mirrors after engine install (daemon.json may be new). +if [ "${{BITFUN_MIRROR_MODE:-}}" = "cn" ]; then + bitfun_mirror_apply_docker_daemon || true +fi bitfun_fix_docker_home # Verify without relying on a new login session if docker info >/dev/null 2>&1 \ @@ -1106,6 +1182,8 @@ else fi "#, helpers = helpers, + REPO_GIT_URL = REPO_GIT_URL, + REPO_TARBALL_URL = REPO_TARBALL_URL, TASK_DONE_MARKER = TASK_DONE_MARKER, ) } @@ -1122,9 +1200,11 @@ bitfun_sync_source() {{ # `git clone ` without a path would create ./BitFun — we always pass "$src". # Tarball extracts BitFun-main/; we use --strip-components=1 into "$src". local src="$1" - local git_url="{REPO_GIT_URL}" + local git_upstream="{REPO_GIT_URL}" + local tarball_upstream="{REPO_TARBALL_URL}" + local git_url="${{BITFUN_GITHUB_GIT_URL:-$git_upstream}}" + local tarball_url="${{BITFUN_GITHUB_TARBALL_URL:-$tarball_upstream}}" local branch="{REPO_GIT_BRANCH}" - local tarball_url="{REPO_TARBALL_URL}" local managed_prefix="$HOME/.bitfun/" local relay_deploy_sh="src/apps/relay-server/deploy.sh" @@ -1173,13 +1253,14 @@ bitfun_sync_source() {{ }} bitfun_fetch_tarball() {{ - echo ">>> Downloading BitFun source (tarball fallback)..." + local url="$1" + echo ">>> Downloading BitFun source (tarball): $url" command -v curl >/dev/null 2>&1 || bitfun_ensure_tools command -v tar >/dev/null 2>&1 || bitfun_ensure_tools bitfun_replace_managed_src mkdir -p "$src" # Archive root is BitFun-main/; strip so files land directly in "$src". - curl -fsSL --retry 3 "$tarball_url" | tar xz -C "$src" --strip-components=1 + curl -fsSL --retry 3 "$url" | tar xz -C "$src" --strip-components=1 bitfun_assert_source_layout }} @@ -1189,7 +1270,7 @@ bitfun_sync_source() {{ if command -v git >/dev/null 2>&1; then if [ -d "$src/.git" ]; then - echo ">>> Updating BitFun source (git fetch)..." + echo ">>> Updating BitFun source (git fetch via $git_url)..." git -C "$src" remote set-url origin "$git_url" 2>/dev/null || true if git -C "$src" fetch --depth 1 origin "$branch" \ && git -C "$src" checkout -f -B "$branch" "origin/$branch" \ @@ -1198,13 +1279,24 @@ bitfun_sync_source() {{ echo ">>> Source updated to $(git -C "$src" rev-parse --short HEAD 2>/dev/null || echo unknown)" return 0 fi + if [ "$git_url" != "$git_upstream" ]; then + echo ">>> git update via mirror failed; retrying upstream..." + git -C "$src" remote set-url origin "$git_upstream" 2>/dev/null || true + if git -C "$src" fetch --depth 1 origin "$branch" \ + && git -C "$src" checkout -f -B "$branch" "origin/$branch" \ + && git -C "$src" clean -fd \ + && bitfun_assert_source_layout; then + echo ">>> Source updated to $(git -C "$src" rev-parse --short HEAD 2>/dev/null || echo unknown)" + return 0 + fi + fi echo ">>> git update failed; recloning managed source..." bitfun_replace_managed_src elif [ -e "$src" ]; then echo ">>> Managed source exists but is not a git checkout; replacing..." bitfun_replace_managed_src fi - echo ">>> Cloning into $src (explicit path; not default BitFun/)..." + echo ">>> Cloning into $src via $git_url ..." mkdir -p "$(dirname "$src")" # Explicit destination avoids creating $PWD/BitFun from the repo name. if git clone --depth 1 --branch "$branch" "$git_url" "$src" \ @@ -1212,11 +1304,27 @@ bitfun_sync_source() {{ echo ">>> Source cloned at $(git -C "$src" rev-parse --short HEAD 2>/dev/null || echo unknown)" return 0 fi + if [ "$git_url" != "$git_upstream" ]; then + echo ">>> git clone via mirror failed; retrying upstream $git_upstream ..." + bitfun_replace_managed_src + mkdir -p "$(dirname "$src")" + if git clone --depth 1 --branch "$branch" "$git_upstream" "$src" \ + && bitfun_assert_source_layout; then + echo ">>> Source cloned at $(git -C "$src" rev-parse --short HEAD 2>/dev/null || echo unknown)" + return 0 + fi + fi echo ">>> git clone failed; falling back to tarball" else echo ">>> git unavailable; using tarball fallback" fi - bitfun_fetch_tarball + if bitfun_fetch_tarball "$tarball_url"; then + return 0 + fi + if [ "$tarball_url" != "$tarball_upstream" ]; then + echo ">>> tarball mirror failed; retrying upstream..." + bitfun_fetch_tarball "$tarball_upstream" + fi }} "#, REPO_GIT_URL = REPO_GIT_URL, @@ -1250,6 +1358,9 @@ fi RELAY_PORT="${{RELAY_PORT:-{port}}}" export RELAY_PORT echo ">>> Using RELAY_PORT=$RELAY_PORT" +export BITFUN_REPO_GIT_URL="{REPO_GIT_URL}" +export BITFUN_REPO_TARBALL_URL="{REPO_TARBALL_URL}" +bitfun_mirror_init SRC="$HOME/{SOURCE_DIR}" bitfun_sync_source "$SRC" cd "$SRC/src/apps/relay-server" @@ -1279,16 +1390,140 @@ echo {TASK_DONE_MARKER} SOURCE_DIR = SOURCE_DIR, port = port, TASK_DONE_MARKER = TASK_DONE_MARKER, + REPO_GIT_URL = REPO_GIT_URL, + REPO_TARBALL_URL = REPO_TARBALL_URL, ) } #[cfg(test)] mod tests { use super::{ - classify_docker_access, decide_task_status, parse_preflight, split_poll_stdout, - DockerAccessMode, RelayTaskStatus, + classify_docker_access, decide_task_status, parse_preflight, prepare_helpers_bash, + split_poll_stdout, sync_source_bash, DockerAccessMode, RelayTaskStatus, RELAY_MIRROR_SH, }; + #[test] + fn embedded_mirror_script_exposes_init_and_cn_defaults() { + assert!( + RELAY_MIRROR_SH.contains("bitfun_mirror_init"), + "mirror.sh must define bitfun_mirror_init" + ); + assert!( + RELAY_MIRROR_SH.contains("rsproxy.cn"), + "mirror.sh must default cargo to rsproxy" + ); + assert!( + RELAY_MIRROR_SH.contains("ghfast.top"), + "mirror.sh must default GitHub proxy" + ); + assert!( + RELAY_MIRROR_SH.contains("bitfun_mirror_country_via_bash_tcp"), + "mirror.sh must keep a country fallback for minimal hosts without curl" + ); + assert!( + RELAY_MIRROR_SH.contains("bitfun_mirror_restore_host"), + "mirror.sh must support switching a managed host back to global mode" + ); + assert!( + !RELAY_MIRROR_SH.contains("data[\"bitfun-cn-mirror\"]"), + "daemon.json must contain only dockerd-supported directives" + ); + assert!( + !RELAY_MIRROR_SH.contains("bitfun_mirror_apply_cargo_config"), + "relay deploy must not rewrite the SSH user's global Cargo config" + ); + let helpers = prepare_helpers_bash(); + assert!( + helpers.contains("bitfun_mirror_init"), + "prepare helpers must embed mirror.sh" + ); + assert!( + helpers.contains("bitfun_run_deploy_sh"), + "prepare helpers must keep deploy runner" + ); + } + + #[test] + fn sync_source_uses_mirror_url_env_with_upstream_fallback() { + let sync = sync_source_bash(); + assert!(sync.contains("BITFUN_GITHUB_GIT_URL")); + assert!(sync.contains("BITFUN_GITHUB_TARBALL_URL")); + assert!(sync.contains("retrying upstream")); + } + + #[cfg(unix)] + #[test] + fn mirror_docker_config_round_trip_preserves_unmanaged_settings() { + use std::{fs, process::Command}; + + let temp = tempfile::tempdir().expect("create mirror test dir"); + let mirror_path = temp.path().join("mirror.sh"); + let daemon_path = temp.path().join("etc/docker/daemon.json"); + fs::create_dir_all(daemon_path.parent().expect("daemon parent")) + .expect("create daemon dir"); + fs::write(&mirror_path, RELAY_MIRROR_SH).expect("write embedded mirror script"); + fs::write( + &daemon_path, + r#"{ + "debug": true, + "registry-mirrors": ["https://user.example"] +} +"#, + ) + .expect("write initial daemon config"); + + let output = Command::new("bash") + .arg("-c") + .arg( + r#" +set -euo pipefail +export HOME="$2/home" +export BITFUN_DOCKER_DAEMON_JSON="$2/etc/docker/daemon.json" +mkdir -p "$HOME" +source "$1" +bitfun_mirror_priv() { "$@"; } +bitfun_mirror_backup_file() { :; } +bitfun_mirror_restart_docker_if_needed() { :; } +bitfun_mirror_write_docker_daemon_json \ + "https://docker.1ms.run https://dockerproxy.net" +python3 - "$BITFUN_DOCKER_DAEMON_JSON" <<'PY' +import json, sys +with open(sys.argv[1], encoding="utf-8") as f: + data = json.load(f) +assert data["debug"] is True +assert data["registry-mirrors"] == [ + "https://user.example", + "https://docker.1ms.run", + "https://dockerproxy.net", +] +assert "bitfun-cn-mirror" not in data +PY +bitfun_mirror_remove_docker_daemon +python3 - "$BITFUN_DOCKER_DAEMON_JSON" <<'PY' +import json, sys +with open(sys.argv[1], encoding="utf-8") as f: + data = json.load(f) +assert data == { + "debug": True, + "registry-mirrors": ["https://user.example"], +} +PY +"#, + ) + .arg("mirror-round-trip") + .arg(&mirror_path) + .arg(temp.path()) + .output() + .expect("run mirror round-trip test"); + + assert!( + output.status.success(), + "mirror round trip failed:\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + } + #[test] fn decide_status_pending_before_pty_is_running() { assert_eq!( diff --git a/src/web-ui/src/features/relay-deploy/README.md b/src/web-ui/src/features/relay-deploy/README.md index db10d83780..561f69a304 100644 --- a/src/web-ui/src/features/relay-deploy/README.md +++ b/src/web-ui/src/features/relay-deploy/README.md @@ -46,6 +46,14 @@ Desktop Tauri surface: `src/apps/desktop/src/api/relay_deploy_api.rs` 8. **Scripts are embedded Rust templates** staged via SFTP. Do not rely on a static repo `.sh` alone on the server until the desktop binary re-stages. +9. **China mirrors before overseas downloads.** Desktop orchestration embeds + `src/apps/relay-server/mirror.sh` and runs `bitfun_mirror_init` before apt + tool install, Docker Engine install, and GitHub sync. `deploy.sh` sources + the same file so manual and one-click paths stay aligned. Force with + `BITFUN_MIRROR=cn|global`. Docker daemon metadata must stay outside + `daemon.json`; host Cargo config must remain untouched; global mode rolls + back only BitFun-managed apt and Docker entries. + ## Related docs - Relay runtime / admin: [`src/apps/relay-server/README.md`](../../../apps/relay-server/README.md)