Skip to content

contrib: MolecularFullerene - dsh-schema-ablation - #7

Merged
0liveiraaa merged 1 commit into
0liveiraaa:mainfrom
MolecularFullerene:contrib/molecularfullerene-dsh-schema-ablation
Aug 16, 2026
Merged

contrib: MolecularFullerene - dsh-schema-ablation#7
0liveiraaa merged 1 commit into
0liveiraaa:mainfrom
MolecularFullerene:contrib/molecularfullerene-dsh-schema-ablation

Conversation

@MolecularFullerene

Copy link
Copy Markdown
Contributor

Summary

This adds an independent controlled ablation of the DeepSeek Harness
first-request tool surface, plus a small second-request transport protocol pilot.
It is adjacent to, but does not replace or modify,
contributions/xiaobright-deepseek-v4-harness.

Main observations:

  • With the official Minimal system, contexts=[], exactly two visible tools,
    and reasoningEffort=max, the persistent-bash schema bundle produced
    17/20 minimal-like lexical trajectories versus 6/20 for the one-shot
    bash bundle.
  • The editor/read marginal comparison was 13/20 versus 10/20; the evidence for
    that file-tool main effect is weak.
  • All 40 first actions were legal and task-relevant. We need / Let me is
    reported only as a trajectory diagnostic, not as an ability score or proof
    of two internal personas.
  • A retain/drop reasoning_content × same/new session protocol pilot used
    one independent first-request source per treatment. All four cells (n=1 each)
    were accepted and returned the preregistered simple JSON answer. This does
    not establish equivalence on complex agent tasks.
  • The evidence audit keeps the historical Project2 98/99 runs attached to
    their recorded legacy Windows pwsh/read → 25 tools configuration rather
    than the later exact-Minimal pair. The docs-only provenance request is
    tracked in xiaobright/dsh-anchored-standard#60.

Included material

  • one contribution README with environment and limitations;
  • three deidentified reports;
  • three aggregate JSON summaries;
  • append-only index rows in the two repository READMEs.

No raw trajectories, complete reasoning, request bodies, API/user/session IDs
or hashes, keys, absolute paths, Project2 prompt text, evaluator, or hidden
tests are included.

The reusable CLI, fixtures, and security tests live separately at
MolecularFullerene/deepseek-v4-pro-harness-ablation.

Validation

  • python3 scripts/check_pr.py --base origin/main --head HEAD: 24/24 checks
    passed.
  • git diff --check origin/main...HEAD: passed.
  • The contribution is one commit on top of upstream
    aceaa0612ef7c9e7558f9dd5a6b694662cdb6e66.

Important limitations

  • The shell factor changes a full schema bundle, not persistence semantics
    alone.
  • The 40 trajectories use two prompt templates and are not 40 independent
    tasks.
  • Prompt, identity, batch, and time are partially confounded; the reported
    Fisher p-values are nominal.
  • The model is a mutable service alias with no exposed immutable server build.

This is opened as a draft so the maintainer can first review scope,
provenance, and wording.

@0liveiraaa
0liveiraaa marked this pull request as ready for review August 16, 2026 12:40
@0liveiraaa
0liveiraaa merged commit b17ee51 into 0liveiraaa:main Aug 16, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants