Context
The primary benchmark uses deepset/prompt-injections (n=662). The import script already supports additional datasets via --preset, but no results have been published for them yet.
Evaluating on OWASP v2 and Antijection would:
- Reduce single-dataset bias in reported metrics
- Validate that 95.1% PINT accuracy generalizes beyond deepset
- Strengthen cross-dataset claims for research credibility
Goal
Run and publish benchmark results for at least two additional datasets already supported by the pipeline.
Tasks
References
- Import script:
scripts/import_external_dataset.py
- Existing deepset results:
notebooks/data/external/results/deepset-sentinel.json
Context
The primary benchmark uses
deepset/prompt-injections(n=662). The import script already supports additional datasets via--preset, but no results have been published for them yet.Evaluating on OWASP v2 and Antijection would:
Goal
Run and publish benchmark results for at least two additional datasets already supported by the pipeline.
Tasks
scripts/import_external_dataset.py --preset owasp-v2scripts/import_external_dataset.py --preset antijectionscripts/run_pint_benchmark.pyfor each dataset in bothbenchmarkandproductionmodesnotebooks/data/external/results/References
scripts/import_external_dataset.pynotebooks/data/external/results/deepset-sentinel.json