Skip to content

Recommend the Cantor-preserving operating point - #17

Draft
Al-does wants to merge 2 commits into
cursor/token-guess-operating-point-0668from
cursor/token-guess-fractal-preserving-point-0668
Draft

Recommend the Cantor-preserving operating point#17
Al-does wants to merge 2 commits into
cursor/token-guess-operating-point-0668from
cursor/token-guess-fractal-preserving-point-0668

Conversation

@Al-does

@Al-does Al-does commented Jul 27, 2026

Copy link
Copy Markdown
Owner

Stacked on #16, which traded the fractal away for global-R² headroom. That was the wrong trade, and this PR supersedes its recommendation.

Why less visible structure looked like more sensitivity

The two things are the same number. The mixed-state set is the attractor of an iterated function system with one contractive map per token. Disjoint first-level images need a contraction ratio below 1/sqrt(3), which puts at least two thirds of belief variance between the branches — and between-branch variance is exactly what a rank-2 probe on the last token reads.

alpha at stay=0.95 0.85 0.80 0.75 0.70 0.55
box dimension 0.85 1.01 1.22 1.38 1.82
last-token probe floor 0.730 0.633 0.543 0.461 0.264

So a low global-R² floor requires closing the gaps. stay carries no such cost: it moves the accuracy axis while leaving the geometry alone.

Mixed-state geometry at four operating points

Recommendation: stay = 0.95, alpha = 0.75

Four points, matched PPO and IQN arms, three seeds, 2.5M steps, identical recipe and probe.

All four operating points on three axes

point stay/alpha band arm belief R² within-branch R² accuracy range share
shipped 0.90 / 0.85 0.036 untrained 0.883 −0.717
ppo 0.8461 ± 0.0065 −1.252 ± 0.091 −0% ± 0%
iqn 0.9756 ± 0.0032 +0.642 ± 0.048 47% ± 2%
cantor_sharp 0.95 / 0.85 0.056 untrained 0.866 −0.089
ppo 0.9138 ± 0.0356 +0.300 ± 0.290 41% ± 31%
iqn 0.9793 ± 0.0015 +0.832 ± 0.011 90% ± 1%
cantor 0.95 / 0.75 0.085 untrained 0.823 +0.300
ppo 0.8990 ± 0.0188 +0.601 ± 0.076 63% ± 13%
iqn 0.9737 ± 0.0012 +0.896 ± 0.005 92% ± 1%
proposed 0.96 / 0.55 0.168 untrained 0.833 +0.715
ppo 0.8281 ± 0.0548 +0.708 ± 0.093 52% ± 22%
iqn 0.9646 ± 0.0056 +0.940 ± 0.010 91% ± 1%

cantor is the only point where the three conditions order cleanly with non-overlapping error bars on the metric that has a floor of zero by construction: untrained 0.300 < PPO 0.601 ± 0.076 < IQN 0.896 ± 0.005. It also has the lowest IQN seed spread of the four, keeps visible level-one gaps at box dimension 1.22, and takes 2.4× the global-R² band and 4.1× the accuracy resolution of the shipped point.

cantor_sharp is the zero-risk option: alpha is untouched so the picture is identical in character, and PPO still moves from 0% to 41% of the accuracy range.

The deeper fix is the metric

Nearly all of a Cantor-structured belief's variance is which cluster you are in, which the last token already gives away. conditional_residual_r2 at depth two discards exactly that term, so any depth-two token model scores zero there by construction at every operating point. That is why the sparse points still discriminate well on it while their global-R² bands look hopeless — and why proposed, with the widest global band, is the most compressed on it: a smooth belief is something a random causal filter already tracks (untrained scores 0.715 there).

Reward-only PPO fails to clear the global-R² floor at all four points, so that finding is not an artifact of a badly chosen process.

Cost

One RTX 4090 for ~1.2 h, destroyed on completion; no instances remain.

To show artifacts inline, enable in settings.

Open in Web Open in Cursor 

cursoragent and others added 2 commits July 27, 2026 06:45
The mixed-state set is the attractor of one contractive map per token, so
disjoint first-level images need a contraction ratio below 1/sqrt(3), which
puts at least two thirds of belief variance between the branches. That
between-branch variance is exactly what a probe on the last token reads, so
visible gaps and a low global-R2 floor are the same quantity with opposite
signs. Measured across alpha 0.85 to 0.55 the box dimension runs 0.93 to 1.68
while the last-token floor runs 0.80 to 0.26.

stay carries no such cost: at fixed alpha, 0.90 to 0.95 leaves the box
dimension at 0.93 to 0.85 while accuracy resolution goes from 6 to 21 sigma.
cantor_sharp takes that for free at the shipped alpha; cantor drops alpha to
0.75, the widest global-R2 band still showing visible level-one gaps.

fractal_preserving_validation reuses the same head-to-head so all four points
land in one synthesis.

Co-authored-by: Alex Vardakostas <Al-does@users.noreply.github.com>
Validating the two fractal-preserving points changes the recommendation.
cantor (stay=0.95, alpha=0.75) keeps visible level-one gaps at box dimension
1.22 and is the only point where the three conditions order cleanly with
non-overlapping error bars on the metric whose floor is zero by construction:
untrained 0.300 < PPO 0.601 +/- 0.076 < IQN 0.896 +/- 0.005. It also has the
lowest IQN seed spread of the four, 2.4x the global-R2 band, and 4.1x the
accuracy resolution.

cantor_sharp leaves alpha untouched, so the picture is unchanged, and still
moves PPO from 0% to 41% of the accuracy range.

proposed keeps the widest global-R2 band but its mixed-state set is a dense
triangle, its PPO seed spread is the worst of the four, and it is the most
compressed on the within-branch metric -- because a smooth belief is something
a random causal filter already tracks.

Co-authored-by: Alex Vardakostas <Al-does@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants