Skip to content

docs: add medical CAMEL evaluation setting report#1

Open
ZhaoyangHan04 wants to merge 1 commit into
mainfrom
codex/evaluate-setting-report-for-qa-synthesis
Open

docs: add medical CAMEL evaluation setting report#1
ZhaoyangHan04 wants to merge 1 commit into
mainfrom
codex/evaluate-setting-report-for-qa-synthesis

Conversation

@ZhaoyangHan04

Copy link
Copy Markdown
Owner

Motivation

  • Provide a concise, reproducible Setting Report that documents the medical QA synthesis and evaluation workflow driven by CAMEL-scraped sources and DataFlow pipelines.
  • Capture data provenance and pipeline mapping to ensure the published experiment can be re-run and validated by others.
  • Reserve a clear placeholder for final evaluation results ({result}) and recommend a metrics table for downstream reporting.

Description

  • Add a new markdown report at reports/medical_camel_setting_report.md that summarizes the overall task, data sources, pipeline mapping, and final synthesized QA size (17774).
  • Include a representative QA sample in JSON format and list the two input PDFs: Microbiology-LR.pdf and AnatomyAndPhysiology-LR.pdf.
  • Document training environment and framework details including LLaMA-Factory, hardware A800 x 4, and base model Qwen2.5-7B-Instruct.
  • Record the evaluation datasets (medqa, bioasq, pubmedqa, covert, pubhealth) and the full training hyperparameters block with the swanlab_api_key redacted.

Testing

  • Validated the created file exists and reviewed its contents using nl -ba reports/medical_camel_setting_report.md | sed -n '1,260p', which displayed the full report.
  • Manually inspected the markdown formatting and the included YAML/code blocks for readability and completeness.

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant