m02 数据工程补四问结论卡与派生回边脚本，打通对 m03/m04 的标准交接#11
Merged
Conversation
补 m02→m03/m04 此前靠聊天传的单向挂载：四问结论收敛成可落盘工件 data_feasibility.md，m03 开工读它判该不该立项、m04 复核数据声明以它为锚点。 - data_feasibility.py：四问各 ok/warn/insufficient + 整体 verdict， insufficient 退出码 1 当"不进 m03"闸门，可吃 sample_size_check --json - derive_eval_set.py：把 m05 派生数据回边做成可执行——据规格生成 加噪/缺失/跨域/扫参评测集 + 对齐 db04 的 dataset_card 字段， 只动特征不碰标签、固定种子、不回流训练折 - drift_check.py：两数据集分布漂移检验，数值列 KS+PSI、类别列卡方+PSI， 纯 numpy 无 scipy 硬依赖，以 PSI 效应量为主 - emit_artifacts.py：核三件套落 CONVENTIONS §6.1 标准名 + 打印 passport 登记命令 - data_doctor.py 补 inf/混合类型/类不均衡/偏态/稀有类 5 个检测器 - CONVENTIONS §6.1 m02 下游补 m03/m04 并加 data_feasibility.md - m03/m04/m05 SKILL 加对应衔接指针 - run_skill_selftests.py 重配 stdout/stderr 为 UTF-8，修 Windows 残留门打印崩溃
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
对标 pandas_dq / ydata-profiling / Altimate / 官方 skills 后发现：m02 与同类的差距不在检测器多寡，而在衔接接口——SKILL 反复说"四问结论喂 m03/m04"，但 CONVENTIONS §6.1 工件表里 m02 下游只写 m05/a03/m06，m03/m04 拿不到落盘工件、靠聊天传，踩了 §6.1 自己"避免单向挂载"的红线。本 PR 按四块杠杆把这个地基环节做到位。
改动
杠杆 1 — 补单向挂载（最高价值）
data_feasibility.py：四问各 ok/warn/insufficient → 整体 verdict，insufficient 退出码 1 当"不进 m03"闸门，可吃sample_size_check --json自动填规模问项m03/m04(四问结论)，标准工件加data_feasibility.md杠杆 2 — 派生回边脚本化
derive_eval_set.py+examples/derive_spec.example.json：据 m05 派生规格生成加噪/缺失/跨域/扫参评测集 + 对齐 db04 的 dataset_card 字段。铁律：只动特征不碰标签、固定种子、不回流训练折杠杆 3 — 工件落位
emit_artifacts.py：核 quality_report/data_card/data_feasibility 三件套落 §6.1 标准名，打印委托 orchestrator passport.py 的登记命令杠杆 4 — 单点检测追平成熟库
data_doctor.py补 inf / 混合类型 / 类不均衡 / 强偏态 / 稀有类 5 个检测器drift_check.py：数值列 KS+PSI、类别列卡方+PSI，纯 numpy 无 scipy 硬依赖，以 PSI 效应量为主收尾
run_skill_selftests.py重配 stdout/stderr 为 UTF-8，修 Windows 残留门打印 ✓/✗ 的 GBK 崩溃验证
--selftest通过、零文件残留（已精确验证）影响范围
CONVENTIONS.md（真相源，§6.1 一行）、WHATS_INCLUDED.md（登记 4 新脚本）、m02/m03/m04/m05 SKILL.md、1 个 CI 脚本。无破坏性改动，纯增量 + 一处契约修正。