Codex/expose voice assets for behavior audit

.gitignore

@@ -27,8 +27,14 @@ tmp/
 scripts/**/reports/**/report_*/
 scripts/reports/**/report_*/
 
-# Research paper workspace lives outside the package release.
-/papers/
+# Research paper workspace lives outside the package release, except the
+# packaged voice helper APIs consumed by sibling products.
+/papers/*
+!/papers/__init__.py
+!/papers/voice/
+!/papers/voice/**
+/papers/**/__pycache__/
+/papers/**/*.py[cod]
 
 # Local ingestion artifacts
 /data/
@@ -49,3 +55,4 @@ dashboard/dist/
 dashboard/.vite/
 dashboard/tsconfig.tsbuildinfo
 dashboard/src/**/*.js
+projects/proxy_readers/reports/assistant_axis_message_score/report_22bceb208eaf_fc421b6a/

papers/__init__.py

@@ -0,0 +1 @@
+"""Paper replication workflows."""

papers/voice/BYOD.md

@@ -0,0 +1,127 @@
+# BYOD Method Schemas
+
+BYOD in this tree means "bring data that fits the method," not "bring the
+paper's original semantic domain." The paper directories keep their historical
+names, but their schemas are method-level contracts.
+
+## Where To Edit
+
+- `assistant_axis/method.py`:
+  default-vs-role contrast data and precomputed axis/trait scoring.
+- `emotions/synth_data.py`:
+  concept-labeled vector spaces with optional neutral/background projection.
+- `refusal_direction/synth_data.py`:
+  two-pole contrast directions.
+- `honest_llama/synth_data.py`:
+  binary truth/correctness readouts.
+- `schemas/*.schema.json`:
+  row contracts for BYOD files and generated-data outputs.
+
+These method/data files are intentionally small. Edit labels, topics, seed
+items, counts, split policy, and prompt templates there before wiring a larger
+workflow. The JSON schemas are the stable boundary: if your imported or
+generated rows satisfy the relevant schema, the adapter can turn them into
+Xenon `Dataset` examples with the right labels and token sections.
+
+## Method Contracts
+
+### Assistant Axis Method
+
+Schema: `schemas/assistant_axis_method.schema.json`
+
+Use for:
+
+- scoring traces against released assistant-axis or trait coordinates
+- deriving a new default-vs-role axis from your own grouped conditions
+
+Required scoring row shape:
+
+```json
+{
+  "example_id": "trace_001",
+  "text": "Human: ...\n\nAssistant: ...",
+  "assistant_response": {"char_start": 42, "char_end": 180},
+  "model_id": "meta-llama/Llama-3.3-70B-Instruct"
+}
+```
+
+Additional fields for deriving a new axis:
+
+```json
+{
+  "axis_kind": "role",
+  "role": "analyst_mode",
+  "adherence_score": 3,
+  "split": "train"
+}
+```
+
+### Concept Vector Space Method
+
+Schema: `schemas/concept_vector_space_method.schema.json`
+
+Use for any labeled concept set. The paper default is emotions, but concepts
+can be financial states, support intents, failure classes, styles, or other
+semantic groups.
+
+```json
+{
+  "example_id": "row_001",
+  "text": "The market opened with widening credit spreads and defensive positioning.",
+  "row_role": "concept",
+  "concept": "risk_off",
+  "split": "train",
+  "topic": "morning market note"
+}
+```
+
+Neutral/background rows use `row_role: "neutral"` and do not need `concept`.
+
+### Contrast Direction Method
+
+Schema: `schemas/contrast_direction_method.schema.json`
+
+Use for any two-pole direction. The paper default is refusal, but the method
+only needs a defined positive pole and negative pole.
+
+```json
+{
+  "example_id": "contrast_001",
+  "text": "Rendered prompt or trace text.",
+  "label": "positive_pole",
+  "positive_label": "positive_pole",
+  "negative_label": "negative_pole",
+  "split": "train"
+}
+```
+
+### Binary Truth/Correctness Method
+
+Schema: `schemas/binary_truth_readout_method.schema.json`
+
+Use for true-vs-false, correct-vs-incorrect, or equivalent binary answer
+contrasts.
+
+```json
+{
+  "example_id": "claim_001_true",
+  "prompt": "Question or claim prompt.",
+  "answer": "Candidate answer text.",
+  "label": "true",
+  "claim_id": "claim_001",
+  "split": "train"
+}
+```
+
+## Adapter Rule
+
+Adapters should do the boring work:
+
+1. Validate rows against the method schema.
+2. Render text if the schema stores prompt/answer separately.
+3. Attach `token_sections` and `section_records` from char spans.
+4. Preserve method labels under stable names.
+5. Pass the resulting `Dataset` into the existing workflow specs.
+
+Do not bake paper-domain assumptions into adapters. Put domain defaults in
+the paper's method-data file, and keep workflow specs pointed at method labels.

papers/voice/README.md

@@ -0,0 +1,80 @@
+# Voice Paper Replications
+
+Xenon-native smoke replications for direction/intervention papers about model
+"voice": persona, emotion, refusal, and truthfulness. These are public example
+workflows, not normal `projects/<project>/phase_XX` research phases.
+
+These workflows live in `xenon-projects`, a small `uv` project that depends on
+the sibling `../xenon` checkout in editable mode. Run workflow commands from
+the `xenon-projects` repository root; `uv run ...` will expose `pipelines_v2`
+without a manual `PYTHONPATH` or `--project ../xenon` override.
+
+Reusable vector and geometry artifacts use Modal volume `xenon-data` under
+`/data/artifacts/model-assets/vectors/<paper-or-method>/<run>`. Product demos
+should consume those shared assets instead of owning their own vector copies.
+Model weights stay on their existing volumes, including transitional
+`yora-models` for Llama 70B Assistant Axis and emotion asset work and
+`xenon-models` for historical Qwen emotion replication work.
+
+Each paper directory should expose four runnable surfaces: paper rerun, BYOD,
+BYOT, and BYOP. Assistant Axis is the current reference implementation for
+that shape.
+
+## Workflows
+
+```bash
+uv run python -m pipelines_v2.cli workflow plan --file papers/voice/assistant_axis/specs/paper_rerun.py
+uv run python -m pipelines_v2.cli workflow plan --file papers/voice/assistant_axis/specs/asset_score.py
+uv run python -m pipelines_v2.cli workflow plan --file papers/voice/assistant_axis/specs/byod_axis.py
+uv run python -m pipelines_v2.cli workflow plan --file papers/voice/assistant_axis/specs/byot_score.py
+uv run python -m pipelines_v2.cli workflow plan --file papers/voice/assistant_axis/specs/byop_generate.py
+
+uv run python -m pipelines_v2.cli workflow plan --file papers/voice/emotions/specs/workflow.py
+XENON_NEON_DATABASE_URL=postgresql://placeholder EMOTION_ASSET_MODE=pilot uv run python -m pipelines_v2.cli workflow plan --file papers/voice/emotions/replication/specs/llama70b_vector_workflow.py
+uv run python -m pipelines_v2.cli workflow plan --file papers/voice/refusal_direction/specs/workflow.py
+uv run python -m pipelines_v2.cli workflow plan --file papers/voice/honest_llama/specs/workflow.py
+```
+
+## Layout
+
+- `assistant_axis/`: default-assistant vs role-play persona axis; reference
+  package layout with paper/BYOD/BYOT/BYOP specs.
+- `emotions/`: emotion concept vectors, scoring, geometry, and steering export;
+  Llama 70B asset package mirrors Assistant Axis while computing vectors from
+  generated/captured data.
+- `refusal_direction/`: harmful-minus-harmless direction, selection, add-direction, and project-out smoke.
+- `honest_llama/`: residual TruthfulQA-style truthful-minus-untruthful ITI smoke.
+- `common/`: shared smoke constants and local runner fixtures.
+- `schemas/`: method-level BYOD row contracts.
+- `BYOD.md`: where to edit data recipes and how schemas map onto methods.
+
+The paper directory names are historical. BYOD schemas are method schemas, not
+domain schemas. For example, the emotion-concepts workflow can build concept
+vectors over finance labels, support intents, or failure modes if the rows fit
+the concept-vector-space schema.
+
+## Lightweight Data Hooks
+
+The smoke workflows keep fixtures in code, but the reusable APIs now point at
+the paper-style sources where possible:
+
+- Assistant Axis prompts and released vectors: `assistant_axis_prompt_dataset`,
+  `AssistantAxisPrecomputedCoordinateSpec`, and
+  `AssistantAxisTraitCoordinateSpec(model_id="llama-3.3-70b", trait="calm")`.
+- Emotions: `emotion_probe_story_dataset` for a public generated-probe mirror,
+  or `emotion_contrast_dataset(records=...)` for agent/product logs.
+- Refusal Direction: `refusal_direction_split_dataset(split="train")` and
+  `refusal_direction_processed_dataset("advbench")`.
+- Honest LLaMA / ITI: `truthfulqa_generation_dataset()` and
+  `truthfulqa_answer_contrast_dataset(prompt_template=...)`.
+
+These helpers are deliberately prompt-template friendly. Users can keep the
+same labels and vector workflow while changing the actual chat/completion
+format to match their product.
+
+## Claim Boundary
+
+Planning a workflow checks API plumbing and artifact shapes. It does not prove
+paper-scale claims. A paper-scale result requires real datasets, behavioral
+sanity checks, confound checks, selected loci that transfer to real data, and
+intervention controls.

papers/voice/__init__.py

@@ -0,0 +1 @@
+"""Voice-related mech-interp paper replications."""

papers/voice/assistant_axis/.env.example

@@ -0,0 +1,14 @@
+# Assistant Axis paper-rerun judge config.
+#
+# Keep raw API keys in your shell or Modal secret. This file should usually
+# name which env var/secret to use, not contain the secret value itself.
+
+ASSISTANT_AXIS_JUDGE_MODEL=gpt-4.1-mini
+ASSISTANT_AXIS_JUDGE_API_KEY_ENV=OPENAI_API_KEY
+ASSISTANT_AXIS_JUDGE_SECRET_NAME=OPENAI_API_KEY
+ASSISTANT_AXIS_JUDGE_BASE_URL=
+ASSISTANT_AXIS_JUDGE_MAX_TOKENS=64
+ASSISTANT_AXIS_JUDGE_DRY_RUN=0
+
+# Full paper threshold is 50. Use 1 for cheap smoke slices.
+ASSISTANT_AXIS_MIN_ROLE_EXAMPLES_PER_ROLE=1