vlabs-evidence

Redacted evidence artifacts from the Verifiable Labs clean promotion gate: sample assurance cards, aggregate metrics, and sanitized export manifests.

Verifiable Labs builds clean feedback and promotion gates for increasingly general AI agents.

Artifacts under evidence/ and the HF/W&B manifests are illustrative — synthetic numbers published to show the evidence format, not customer results. results/ separately contains clearly labelled measurements reproduced on public benchmark datasets. There are no customer results in this repository.

What is never published, here or anywhere: hidden evaluation content, gold answers, anti-hack detection details, raw traces, customer data, secrets, private anti-hack traps, or private engine internals.

Contents

• evidence/sample_assurance_card_redacted.json
• evidence/aggregate_metrics_sample.md
• results/atlas_honeypot_ipt.json — real result: IPT vs LLM-judges on the Obfuscation Atlas MBPP-Honeypot
• results/conformal_fp_control_mbpp.json — real public-benchmark result (not synthetic)
• reproducibility-notes.md
• manifests/ — sanitized HF/W&B export manifests (below)

Sanitized evidence manifests

manifests/ holds the sanitized export manifests used to publish public evidence to Hugging Face and Weights & Biases:

• manifests/hf_upload_manifest.json
• manifests/wandb_upload_manifest.json
• manifests/evidence_safety_report.md

The artifacts they reference are synthetic / redacted. They contain no hidden evals, gold answers, raw traces, customer data, secrets, private anti-hack traps, or private engine internals. Export tooling routes candidates through the platform's assert_public_export_safe policy gate, which refuses such content and any restricted/secret classification.

HF and W&B uploads are gated by explicit approval flags (VLABS_APPROVE_HF_UPLOAD=1, VLABS_APPROVE_WANDB_UPLOAD=1); the exporters are dry-run by default and never touch the network without them.

Published evidence

Public, synthetic / redacted demo evidence:

• Hugging Face dataset — https://huggingface.co/datasets/verifiablelabs/vlabs-clean-gate-evidence
• Weights & Biases (entity verifiable-labs): clean-generalization-gate · contamination-firewall · anti-hack-engine · scenario-compiler · runpod-costs

The HF/W&B demo evidence is synthetic / redacted and is not a training dataset. The separate JSON files under results/ are real public-benchmark measurements and contain no customer data, hidden evaluations, gold answers, raw traces, private anti-hack traps, or private engine internals.

SHA256SUMS commits the exact bytes of every JSON/Markdown artifact under evidence/, results/, and manifests/; CI rejects missing or stale hashes.

Install the SDK: pip install "vlabs-sdk==0.0.2"

Install the SDK

pip install "vlabs-sdk==0.0.2"

Formal scope

Selected mathematical properties behind the contamination-resistant promotion gate are machine-verified in Lean 4. A hand-maintained Python mirror has property tests derived from selected definitions; no mechanized code-to-proof parity is claimed.