Skip to content

fix: add AI strategy review workflow #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Pigbibi merged 1 commit into from
Apr 1, 2026
Merged

fix: add AI strategy review workflow #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
181 changes: 121 additions & 60 deletions .github/workflows/ai_review.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,78 +3,124 @@ name: AI Monthly Review
"on":
issues:
types: [opened]
workflow_dispatch:
inputs:
issue_number:
description: "Issue number to review"
required: true

jobs:
ai-review:
if: contains(github.event.issue.labels.*.name, 'monthly-review')
if: contains(github.event.issue.labels.*.name, 'monthly-review') || inputs.issue_number != ''
runs-on: ubuntu-latest
permissions:
contents: write
contents: read
id-token: write
issues: write
pull-requests: write

steps:
- name: Checkout
uses: actions/checkout@v5

- name: Load review issue context
id: issue_context
run: |
python3 - <<'PY'
import json
import os
import urllib.request

repo = os.environ["GITHUB_REPOSITORY"]
issue_number = os.environ["ISSUE_NUMBER"]
token = os.environ["GITHUB_TOKEN"]
api_url = f"https://api.github.com/repos/{repo}/issues/{issue_number}"
request = urllib.request.Request(
api_url,
headers={
"Accept": "application/vnd.github+json",
"Authorization": f"Bearer {token}",
"X-GitHub-Api-Version": "2022-11-28",
"User-Agent": "crypto-leader-rotation-ai-review",
},
)
with urllib.request.urlopen(request) as response:
issue = json.load(response)

with open(os.environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as output:
print("issue_title<<EOF", file=output)
print(issue["title"], file=output)
print("EOF", file=output)
print("issue_body<<EOF", file=output)
print(issue["body"], file=output)
print("EOF", file=output)
PY
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
ISSUE_NUMBER: ${{ inputs.issue_number || github.event.issue.number }}

- name: Run Claude Code Review
id: claude_review
uses: anthropics/claude-code-action@v1
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
model: claude-sonnet-4-6-20250514
github_token: ${{ secrets.GITHUB_TOKEN }}
use_bedrock: false
use_vertex: false
allowed_tools: |
Read
Glob
Grep
Bash(git log:*)
Bash(git diff:*)
Bash(python:*)
custom_instructions: |
You are reviewing the monthly upstream release report for CryptoLeaderRotation,
a research and release system that publishes a monthly 5-symbol Binance Spot
leader universe to its downstream execution engine (BinanceQuant).

The Issue body contains the full monthly report (ai_review_input.md).
Analyze it and provide your review in BOTH English and Chinese.
prompt: |
Do not use Bash or ask for additional approval.
Return only the final bilingual review in markdown.
The workflow will publish your final review as the issue comment.

You are reviewing the monthly upstream selector report for CryptoLeaderRotation.
This repository owns monthly universe selection, ranking, challenger evaluation,
and publication of a 5-symbol Binance Spot leader pool for downstream execution.

This is a strategy-optimization review, not only a release QA check.
Your job is to help the operator judge:
- whether the published official pool looks internally consistent and research-valid,
- whether the current evidence is sufficient to keep or change the selector,
- what the next highest-value shadow/challenger experiments should be.

Important interpretation rules:
- Do not recommend changing production selector logic from one monthly snapshot alone.
- If shadow or challenger tracks are missing, say evidence is incomplete rather than making strong claims.
- Treat release consistency and validation as a prerequisite, but spend most attention on selector quality and research direction.
- BinancePlatform is a downstream execution engine that consumes this 5-symbol pool and then selects its own final live targets from it.

## Issue Title
${{ steps.issue_context.outputs.issue_title }}

## Issue Body
${{ steps.issue_context.outputs.issue_body }}

## Review Scope

1. **Release Consistency Check**
- Do live_pool.json, release_manifest.json, and release_status_summary.json agree
on as_of_date, version, mode, pool_size, and symbols?
- Is the validation status ok with no errors?
- Are there any warnings that need operator attention?

2. **Anomaly Detection**
- Are any artifact ages unexpectedly high (age_days > 7)?
- Is dry_run=True expected or a sign that publish was not actually enabled?
- Are ranking scores reasonable (no extreme values or suspicious clustering)?
- Is pool_size consistent with the expected value (5)?

3. **Downstream Impact (BinanceQuant)**
- BinanceQuant is a downstream execution engine that selects the top 2 symbols
from this pool for trend rotation trading on Binance Spot.
- Note any symbols that are new to the pool vs last month (if determinable from context).
- Flag if any pool symbols might have low liquidity or high volatility concerns
for a real-money execution system.
- Remind the operator that pool changes will affect live positions at the next
BinanceQuant hourly cycle after the Firestore document updates.

4. **Operator Action Items**
- Summarize the operator checklist items from the report.
- Add any additional follow-up items based on your analysis.
- If publish was a dry run, note that the operator should confirm whether
a real publish is intended.

5. **Code Improvements (Optional)**
- If you identify concrete, low-risk improvements to scripts or configuration
in this repository, you may open a Pull Request.
- Do NOT modify strategy logic, ranking parameters, or universe filters.
- Acceptable improvements: documentation, validation checks, error messages,
report formatting, workflow configuration.
- Do NOT auto-merge. The operator will review and decide.
1. **Release Consistency & Contract Health**
- Do release metadata, symbols, pool_size, version, and validation state agree?
- Are there any warnings, stale artifacts, or publish flags that weaken trust in this release?

2. **Selector Quality Review**
- Does the official 5-symbol pool look reasonable for a Binance Spot mainstream leader universe?
- Does the ranking preview show healthy score separation, or suspicious clustering/tie behavior?
- Are any symbols potentially borderline on liquidity, stability, or crowding risk for downstream live use?

3. **Research Coverage & Evidence Gaps**
- Is shadow/challenger coverage present?
- If not present, explicitly say that strategy-optimization evidence is incomplete.
- If present, explain whether official baseline still looks justified versus challengers.

4. **Downstream Impact (BinancePlatform)**
- What are the likely downstream implications for BinancePlatform, which consumes this pool monthly?
- Highlight any pool changes or symbol characteristics that could affect downstream rotation behavior or turnover.

5. **Strategy Optimization Directions**
- Suggest the 1-3 highest-value next research directions in this repository.
- Prefer ideas such as challenger track design, shadow build coverage, ranking weight review, regime-weight review, universe-filter tuning, and validation additions.
- Do not suggest arbitrary complexity; prioritize explainable, low-risk, testable improvements.

6. **Operator Action Items**
- Summarize immediate operator checks for this month.
- Separate "safe to keep production as-is" from "needs more research evidence".

## Output Format

Expand All @@ -85,25 +131,40 @@ jobs:

### Release Consistency
...
### Anomalies
### Selector Quality
...
### Downstream Impact (BinanceQuant)
### Research Coverage Gaps
...
### Operator Action Items
### Downstream Impact (BinancePlatform)
...
### Code Improvement Suggestions
### Strategy Optimization Directions
...
### Operator Action Items
...

---
## 中文

### 发布一致性检查
...
### 异常检测
### 选币器质量评估
...
### 下游影响（BinanceQuant）
### 研究覆盖缺口
...
### 操作员待办事项
### 下游影响（BinancePlatform）
...
### 策略优化方向
...
### 代码改进建议
### 操作员待办事项
...

- name: Post AI review issue comment
if: steps.claude_review.outcome == 'success'
run: |
python3 scripts/post_monthly_ai_review_comment.py \
--repo "${GITHUB_REPOSITORY}" \
--issue-number "${{ inputs.issue_number || github.event.issue.number }}" \
--execution-file "${{ steps.claude_review.outputs.execution_file }}" \
--run-url "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
43 changes: 43 additions & 0 deletions .github/workflows/monthly_publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ jobs:
- Linux
- X64
permissions:
actions: write
contents: write
id-token: write
issues: write
Expand Down Expand Up @@ -91,9 +92,11 @@ jobs:

- name: Create Monthly Review Issue
if: success() && env.PUBLISH_ENABLED != 'false'
id: review_issue
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_OUTPUT: ${{ github.output }}
run: |
set -euo pipefail
python - <<'PY'
Expand Down Expand Up @@ -154,6 +157,46 @@ jobs:
},
)
print(f"Created issue #{issue['number']}: {issue['html_url']}")
with open(os.environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as output:
print(f"issue_number={issue['number']}", file=output)
print(f"issue_url={issue['html_url']}", file=output)
PY

- name: Trigger AI Monthly Review
if: success() && env.PUBLISH_ENABLED != 'false'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_REF_NAME: ${{ github.ref_name }}
ISSUE_NUMBER: ${{ steps.review_issue.outputs.issue_number }}
run: |
set -euo pipefail
python - <<'PY'
import json
import os
import urllib.request

token = os.environ["GITHUB_TOKEN"]
repository = os.environ["GITHUB_REPOSITORY"]
ref_name = os.environ["GITHUB_REF_NAME"]
issue_number = os.environ["ISSUE_NUMBER"]
payload = {"ref": ref_name, "inputs": {"issue_number": issue_number}}
request = urllib.request.Request(
f"https://api.github.com/repos/{repository}/actions/workflows/ai_review.yml/dispatches",
data=json.dumps(payload).encode("utf-8"),
method="POST",
headers={
"Authorization": f"Bearer {token}",
"Accept": "application/vnd.github+json",
"Content-Type": "application/json",
"X-GitHub-Api-Version": "2022-11-28",
"User-Agent": "crypto-leader-rotation-monthly-publish",
},
)
with urllib.request.urlopen(request) as response:
if response.status not in (201, 204):
raise RuntimeError(f"Unexpected dispatch status: {response.status}")
print(f"Triggered ai_review.yml for issue #{issue_number} on ref {ref_name}")
PY

- name: Write Release Heartbeat
Expand Down
Loading
Loading