Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/tdx2db-query/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ SELECT
1. **code 一律 6 位纯数字**。`sh600036` / `sz000001` 是 `stock_info` 和文件名的格式,行情表里没有。
2. **停牌日显式处理**:`WHERE date = :d` 查不到就是停牌,需要回退到前一交易日时显式写,禁止用 `<= :d ORDER BY date DESC LIMIT 1` 静默错配。
3. **价格不复权**。除权除息造成的跳空是数据特征不是 bug。
4. **股票名称不在库里**:`stock_info.name` 是占位符。需要名称/板块时用通达信导出 CSV(code 先 `zfill(6)`),或直接用 code
4. **股票名称**:`stock_info.name` 为真实名称(含退市名)。注意 `stock_info.code` 带前缀(`sz000001`),与行情表按 code 关联要 `RIGHT(stock_info.code, 6)`。板块归属查 `block_stock_relation`
5. **统计结果踩到极端值(0%/100%/历史新低)先复核口径再下结论**。

## 排查同步问题
Expand Down
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ tdx2db stock-list --db-only # 同步股票列表
|----|----------|------|
| `daily_data` | (code, date) | 日线 OHLCV + 11 条均线 |
| `minute{5,15,30,60}_data` | (code, datetime) | 分钟线;15/30/60 由 5 分钟重采样 |
| `stock_info` | code | 股票列表(name 是占位符,见"陷阱") |
| `stock_info` | code | 股票列表(name 为真实名称,来自本地 infoharbor_ex.code) |
| `block_stock_relation` | (block_type, block_name, code) | 板块-个股关系,全量快照;block_type ∈ 行业/概念/指数/地区/风格/特殊 |

- 行情表通用列:`code, market, datetime, date, open, high, low, close, volume, amount, ma5, ma10, ma13, ma21, ma34, ma55, ma60, ma89, ma144, ma233, ma250`
Expand Down Expand Up @@ -71,7 +71,7 @@ WHERE code = '000001';

1. **code 格式跨表不一致**:`stock_info.code` 带前缀(`sz000001`),行情表是 6 位纯数字(`000001`)。用 `LIKE 'sh688%'` 查 `daily_data` 会静默零匹配。跨表需 `RIGHT(stock_info.code, 6)`——但内部实践根本不用 `stock_info`(见 3)。
2. **停牌日必须严格 `date = X` 等值匹配**:想"取不到就用前一天"时要显式写出回退逻辑,`<=` 取最末行会静默错配到停牌前一天。
3. **`stock_info.name` `深Asz000001` / `上Ash600000` 式占位符**,不是真实股票名。真实名称/板块归属从通达信导出的 CSV 获取(code 记得 `zfill(6)` 对齐)
3. **`stock_info.name` 现为真实股票名称**(来自本地 infoharbor_ex.code,含退市名;名称文件缺失时回退 `深Asz000001` 式占位符)。注意 `stock_info.code` 带前缀,与行情表 JOIN 仍需 `RIGHT(code, 6)`。板块归属查 `block_stock_relation`
4. **数据不复权**(设计决策,不会改):与行情软件默认前复权对比时,分红除权的股票形状会不同。复权在消费端自行处理。
5. **同步不完整会静默算出子集**:`MAX(date)` 已是今天不代表全市场都进来了。见下面的验证流程。

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ tdx2db minutes --csv-only
**⚠️ code 格式差异(跨表查询必读)**:`stock_info.code` 带市场前缀(`sz000001` / `sh600000`),而 `daily_data` / `minute*_data` 的 code 是 **6 位纯数字**(`000001`)。跨表 JOIN 需要 `RIGHT(stock_info.code, 6)` 或等价处理——这是最容易踩的坑。

**已知限制**:
- `stock_info.name` 目前是 `深Asz000001` 式占位符,不是真实股票名称
- `stock_info.name` 为真实股票名称(来自通达信本地 infoharbor_ex.code,缺失时回退占位符)
- 收录范围:深市 `000 / 001 / 002 / 300 / 301`,沪市 `60xxxx / 688xxx`;**北交所、ETF、指数暂未纳入**

## FAQ
Expand Down
31 changes: 26 additions & 5 deletions tdx2db/reader.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,32 @@ def __init__(self, tdx_path: Optional[str] = None) -> None:
self.min_reader = TdxMinBarReader()
self.lc_min_reader = TdxLCMinBarReader()

def _load_real_names(self) -> dict:
"""读取真实股票名称:T0002/hq_cache/infoharbor_ex.code(issue #42)

格式:``6位code|股票名|关联人``,GBK,全 A 股含退市名,随盘后下载更新。
文件缺失时返回空 dict,get_stock_list 回退占位符命名。

Returns:
dict: 6位纯数字 code -> 股票名
"""
f = self.tdx_path / 'T0002' / 'hq_cache' / 'infoharbor_ex.code'
if not f.exists():
logger.warning(f"缺少 {f},stock_info.name 回退为占位符(深A/上A + code)")
return {}
names = {}
for line in f.read_text(encoding='gbk', errors='replace').splitlines():
p = line.strip().split('|')
if len(p) >= 2 and p[0] and p[1]:
names[p[0]] = p[1]
return names

def get_stock_list(self) -> pd.DataFrame:
"""获取股票列表

Returns:
DataFrame: 包含A股股票代码和名称的DataFrame(不包含B股、基金、等)
DataFrame: 包含A股股票代码和名称的DataFrame(不包含B股、基金、等)。
name 优先取 infoharbor_ex.code 真实名称,缺失时回退占位符
"""
# 尝试查找通达信股票数据文件
sz_path = self.tdx_path / 'vipdoc' / 'sz' / 'lday'
Expand All @@ -65,17 +86,18 @@ def get_stock_list(self) -> pd.DataFrame:
if not (sz_path.exists() or sh_path.exists()):
raise FileNotFoundError(f"无法找到股票列表文件或股票数据目录")

real_names = self._load_real_names()

# 从目录中获取股票代码
stocks = []

# 处理深圳股票
if sz_path.exists():
for file in sz_path.glob('*.day'):
code = file.stem
name = f"深A{code}"

pure_code = code[-6:]
code_str = str(pure_code).zfill(6) # 补齐为6位字符串
name = real_names.get(code_str, f"深A{code}")
# 深证A股:主板 000/001/002 + 创业板 300/301
if re.match(r'^(000|001|002|300|301)\d{3}$', code_str):
stocks.append({'code': code, 'name': name})
Expand All @@ -84,10 +106,9 @@ def get_stock_list(self) -> pd.DataFrame:
if sh_path.exists():
for file in sh_path.glob('*.day'):
code = file.stem
name = f"上A{code}"

pure_code = code[-6:]
code_str = str(pure_code).zfill(6) # 补齐为6位字符串
name = real_names.get(code_str, f"上A{code}")
# 上证A股:主板 60xxxx + 科创板 688xxx(旧正则 688\d{4} 共 7 位,永远匹配不上 6 位代码)
if re.match(r'^(60\d{4}|688\d{3})$', code_str):
stocks.append({'code': code, 'name': name})
Expand Down
24 changes: 24 additions & 0 deletions tests/test_reader.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,30 @@ def test_missing_file_raises(self, tdx_reader):
tdx_reader.read_daily_data(0, 'sz999999')


class TestStockListNames:
def test_real_names_from_infoharbor_ex(self, tmp_path):
"""stock_info.name 用 infoharbor_ex.code 真名(issue #42),code 保持带前缀"""
for m, f in [('sz', 'sz000001.day'), ('sh', 'sh688001.day')]:
d = tmp_path / 'vipdoc' / m / 'lday'
d.mkdir(parents=True)
shutil.copy(FIXTURES / f, d / f)
hq = tmp_path / 'T0002' / 'hq_cache'
hq.mkdir(parents=True)
(hq / 'infoharbor_ex.code').write_bytes(
'000001|平安银行|平安保险,谢永林\n688001|华兴源创|陈文源\n'.encode('gbk')
)
df = TdxDataReader(tdx_path=str(tmp_path)).get_stock_list()
names = dict(zip(df.code, df.name))
assert names == {'sz000001': '平安银行', 'sh688001': '华兴源创'}

def test_fallback_placeholder_when_file_missing(self, tdx_reader):
"""名称文件缺失回退占位符,不报错"""
df = tdx_reader.get_stock_list()
names = dict(zip(df.code, df.name))
assert names['sz000001'] == '深Asz000001'
assert names['sh688001'] == '上Ash688001'


@pytest.fixture
def real_tdx_reader(tmp_path):
"""真实文件切片(dd bs=32 count=3 自实际 vipdoc,2026-07-07)——
Expand Down
Loading