This repository is a compact computational showcase for an education-policy question-answering experiment. It contains implementation code and aggregate claim metadata. It deliberately excludes source documents, processed text, question-level annotations, retrieved passages, model answers, embeddings, indexes, and record-level results.
The benchmark contains 80 questions over 84 cleaned education-policy documents: 20 factual, 20 rule, 15 exception, 15 multi-document, and 10 unanswerable questions. Seventy answerable questions include document-, page-, and span-level evidence, although those questions reference only three unique evidence documents.
The retrieval metric checks whether the target evidence document appears among the source documents of the top-five retrieved chunks; it is not answer accuracy. The hit rate was 20/70 (28.6%) for answerable questions and 0/15 for multi-document questions. No confidence interval was reported.
Four 80-row output sets were produced for closed-book, full-context, basic-RAG, and citation-RAG prompting. Answer and citation quality have not been fully scored, so their existence does not establish a QA improvement.
These findings are documented-only in this public candidate because the question rows, source texts, and result rows are not redistributed. The included synthetic tests exercise bounded code paths but do not reproduce the reported benchmark results.
Previously public collection, metadata-cleaning, document-loading, Kaggle-runner, and local-check utilities are retained as documented-only implementation history. Their required inputs and outputs remain excluded, and the embedded manual question-fix script is not distributed because it contains record-level questions and reference answers.
src/: chunking, embedding/index construction, retrieval evaluation, and four prompting runners.data/SOURCES.md: sanitized upstream-source and redistribution boundary; no document or record-level payload.results/: aggregate claims and provenance boundaries.experiments/manifest.csv: status of each computational path.docs/reproducing.md: what can and cannot be run from this tree.docs/data-and-models.md: excluded data and external model requirements.docs/limitations.md: evidence concentration, metric, and evaluation limits.docs/repository-scope.md: public/private boundary.
With the listed Python dependencies installed in an isolated environment:
python -m unittest discover -s tests -vThis command uses synthetic records only. It does not contact Ollama, download a model, or reconstruct the benchmark.
| Path | Status | Meaning |
|---|---|---|
| Python syntax/import checks | verified |
All 15 Python files parsed; the nine bounded showcase modules imported in the isolated environment. |
| Retained historical utilities | documented-only |
Five restored utilities were statically parsed but not executed against excluded inputs. |
| Synthetic chunking and retrieval-metric tests | smoke-tested |
Bounded tests use fabricated inputs only. |
| 80-question benchmark results | documented-only |
Aggregate claims are provenance-tracked; record-level inputs/outputs are excluded. |
| End-to-end corpus reconstruction | unavailable |
Third-party source texts and one upstream manual-fix dependency are not included. |
| Ollama/model execution | unavailable |
No model was downloaded or invoked during validation. |
Project-authored code and documentation are distributed under the existing Apache License 2.0 in LICENSE. That license does not cover the third-party course documents. No source document is redistributed here, and dependency/model names are references rather than bundled assets.
This is a computational showcase, not a complete replication package or a publication-ready dataset release.