diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 2378d3e..ca86fba 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -82,6 +82,22 @@ jobs: set -euo pipefail python -m pip install --upgrade pip python -m pip install -r requirements.txt pytest ruff + + - name: Smoke import pinned shared packages + run: | + set -euo pipefail + python - <<'PY' + from quant_platform_kit.common.port_adapters import CallableNotificationPort, CallablePortfolioPort + from us_equity_strategies import resolve_canonical_profile + + assert CallableNotificationPort + assert CallablePortfolioPort + assert resolve_canonical_profile("mega_cap_leader_rotation_top50_balanced") == "mega_cap_leader_rotation_top50_balanced" + PY + + - name: Install editable shared repositories + run: | + set -euo pipefail python -m pip install --no-deps -e external/QuantPlatformKit -e external/UsEquityStrategies -e external/UsEquitySnapshotPipelines - name: Run ruff diff --git a/README.md b/README.md index f17469d..d989a8c 100644 --- a/README.md +++ b/README.md @@ -244,6 +244,7 @@ Important: - Here "shared config" still only means the **IBKR pair** (`InteractiveBrokersPlatform` + `IBKRGatewayManager`). `TELEGRAM_TOKEN` and `TELEGRAM_TOKEN_SECRET_NAME` remain repository-specific. - If `IB_ACCOUNT_GROUP_CONFIG_SECRET_NAME` is set, the Cloud Run runtime needs Secret Manager access to that secret. - GitHub now authenticates to Google Cloud with OIDC + Workload Identity Federation, so `GCP_SA_KEY` is no longer required for this workflow. +- If you deploy with `gcloud run deploy --source` or a Cloud Run source trigger, the staging bucket `gs://run-sources--` also needs `roles/storage.objectViewer` for the build service account, the deploy service account, and the default compute service account. Missing this binding fails the deploy before Cloud Build starts with `storage.objects.get` denied. ### Deployment unit and naming @@ -258,6 +259,7 @@ Important: 1. **GCE**: Set up IB Gateway (paper or live) on a GCE instance. Ensure API access is enabled, remote clients are allowed when needed, and use `4001` for `live` or `4002` for `paper`. 2. **VPC / Subnet**: Put Cloud Run and GCE in the same VPC. For cleaner firewall rules, reserve a dedicated subnet for Cloud Run Direct VPC egress. 3. **Cloud Run**: Deploy or update this Flask app with Direct VPC egress. Set `STRATEGY_PROFILE`, `ACCOUNT_GROUP`, and `IB_ACCOUNT_GROUP_CONFIG_SECRET_NAME`. Keep `IB_GATEWAY_ZONE` / `IB_GATEWAY_IP_MODE` only as transition fallbacks if the selected account-group payload does not already contain them. The runtime service account needs `roles/secretmanager.secretAccessor` and, for instance-name resolution, `roles/compute.viewer`. + - For Cloud Run source deploy, also grant `roles/storage.objectViewer` on `gs://run-sources-${PROJECT_ID}-${REGION}` to the build service account, the deploy service account, and `${PROJECT_NUMBER}-compute@developer.gserviceaccount.com`. 4. **Firewall**: Allow TCP `4001` (`live`) or `4002` (`paper`) from the Cloud Run egress subnet CIDR to the GCE instance. 5. **Cloud Scheduler**: Create a job that POSTs to the Cloud Run URL. Choose the cron from the strategy-layer cadence in `UsEquityStrategies`; daily profiles can still use a near-close weekday schedule such as `45 15 * * 1-5` in `America/New_York`. 6. **Optional public-IP mode**: Only if you cannot use VPC, set `IB_GATEWAY_IP_MODE=external`, expose the GCE public IP deliberately, and restrict source ranges tightly. This is not the default path. @@ -449,6 +451,7 @@ IB_GATEWAY_IP_MODE=internal - 这里说的“共享配置”仍然只针对 **IBKR 这一组系统**。`TELEGRAM_TOKEN` 和 `TELEGRAM_TOKEN_SECRET_NAME` 都还是这个仓库自己的配置,不建议提升成所有 quant 共用的全局配置。 - 如果设置了 `IB_ACCOUNT_GROUP_CONFIG_SECRET_NAME`,Cloud Run 运行时还需要有对应 Secret 的访问权限。 - GitHub 现在通过 OIDC + Workload Identity Federation 登录 Google Cloud,这个 workflow 不再需要 `GCP_SA_KEY`。 +- 如果你用 `gcloud run deploy --source` 或 Cloud Run source trigger 部署,还要给 staging bucket `gs://run-sources--` 补 `roles/storage.objectViewer`,对象包括 build service account、deploy service account、默认 compute service account。少了这层权限,会在 Cloud Build 启动前直接报 `storage.objects.get denied`。 ### 部署单元和命名建议 @@ -463,6 +466,7 @@ IB_GATEWAY_IP_MODE=internal 1. **GCE**: 部署 IB Gateway(模拟或实盘),确认 API 已开启、需要远程连接时已允许非 localhost 客户端,并确认 `live` 使用 `4001`、`paper` 使用 `4002`。 2. **VPC / 子网**: 让 Cloud Run 和 GCE 处于同一个 VPC。为了让防火墙规则更干净,建议给 Cloud Run Direct VPC egress 单独准备一个子网。 3. **Cloud Run**: 部署此 Flask 应用时启用 Direct VPC egress。设置 `STRATEGY_PROFILE`、`ACCOUNT_GROUP`、`IB_ACCOUNT_GROUP_CONFIG_SECRET_NAME`;只有在账号组配置里还没放 `ib_gateway_zone` / `ib_gateway_ip_mode` 时,才临时保留 `IB_GATEWAY_ZONE` / `IB_GATEWAY_IP_MODE` 作为过渡 fallback。runtime service account 需要 `roles/secretmanager.secretAccessor`,若走实例名解析,还需要 `roles/compute.viewer`。 + - 如果使用 Cloud Run source deploy,还要给 `gs://run-sources-${PROJECT_ID}-${REGION}` 这个 bucket 授权 `roles/storage.objectViewer`,对象是 build service account、deploy service account,以及 `${PROJECT_NUMBER}-compute@developer.gserviceaccount.com`。 4. **防火墙**: 只允许 Cloud Run 出口子网访问 GCE 的 `TCP 4001`(`live`)或 `TCP 4002`(`paper`)。 5. **Cloud Scheduler**: 创建定时任务,POST 到 Cloud Run URL。cron 频率以 `UsEquityStrategies` 里的策略层 cadence 为准;日频 profile 仍可使用美股临近收盘的工作日计划,例如 `45 15 * * 1-5`(America/New_York 时区)。 6. **可选公网模式**: 只有在不能走 VPC 时,才设置 `IB_GATEWAY_IP_MODE=external`,并且要明确开放 GCE 公网 IP,同时严格限制来源 IP 和防火墙规则。 diff --git a/docs/ibkr_runtime_rollout.md b/docs/ibkr_runtime_rollout.md index c78fc15..24f2b71 100644 --- a/docs/ibkr_runtime_rollout.md +++ b/docs/ibkr_runtime_rollout.md @@ -151,6 +151,38 @@ gcloud projects add-iam-policy-binding "${PROJECT_ID}" \ 如果你后面改成直接配固定私网 host,而且代码不再走实例名解析,这个权限可以再收回。**但按现在这套推荐部署,先给上更稳。** +## 3.1 Cloud Run source deploy 的 staging bucket 权限 + +如果部署路径是 `gcloud run deploy --source`,或 Cloud Run 绑定 GitHub/source trigger,那么还要检查 staging bucket: + +- bucket 一般是 `gs://run-sources-${PROJECT_ID}-${REGION}` +- 至少需要给下面这些身份授予 `roles/storage.objectViewer` + - build service account + - deploy service account + - 默认 compute service account:`${PROJECT_NUMBER}-compute@developer.gserviceaccount.com` + +少了这层权限,部署会在“源码上传完成后、Cloud Build 启动前”直接失败,报错类似: + +```text +storage.objects.get denied on gs://run-sources-... +``` + +示例: + +```bash +gcloud storage buckets add-iam-policy-binding "gs://run-sources-${PROJECT_ID}-${REGION}" \ + --member="serviceAccount:${BUILD_SA}" \ + --role="roles/storage.objectViewer" + +gcloud storage buckets add-iam-policy-binding "gs://run-sources-${PROJECT_ID}-${REGION}" \ + --member="serviceAccount:${DEPLOY_SA}" \ + --role="roles/storage.objectViewer" + +gcloud storage buckets add-iam-policy-binding "gs://run-sources-${PROJECT_ID}-${REGION}" \ + --member="serviceAccount:${PROJECT_NUMBER}-compute@developer.gserviceaccount.com" \ + --role="roles/storage.objectViewer" +``` + ### 当前这一步不用先加的权限 - 不需要为了这个仓库再加 Storage 锁相关权限;当前代码已经不靠 `EXECUTION_LOCK_BUCKET` 那套 GCS 锁运行。 diff --git a/requirements.txt b/requirements.txt index 0b4163f..e0c8850 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,7 +1,7 @@ flask gunicorn -quant-platform-kit @ git+https://github.com/QuantStrategyLab/QuantPlatformKit.git@v0.7.19 -us-equity-strategies @ git+https://github.com/QuantStrategyLab/UsEquityStrategies.git@v0.7.31 +quant-platform-kit @ git+https://github.com/QuantStrategyLab/QuantPlatformKit.git@ac550fee3ca33bbf28c8a886848dede4e4e09a1b +us-equity-strategies @ git+https://github.com/QuantStrategyLab/UsEquityStrategies.git@0f4373f69c9d9f800eda476ff213b7013e51aba4 pandas numpy requests