Significant drop in sort performance at commit 7133a70

[philrz]

On my Intel Macbook, the following sort operation took would run nearly 20x faster at the commit just prior.

$ python3 -c '
import random
for i in range(50000):
  fl=", ".join(f"f{j}:\"v{j}\"" for j in range(120))
  print(f"{{key:{random.randint(0,50000)}, {fl}}}")' |
super -dynamic -f csup - > data.csup &&
super -version &&
hyperfine --warmup 1 --runs 3 --time-unit second 'super -dynamic -f null -c "sort key" data.csup'

Version: v0.3.0-222-g7133a7029

Benchmark 1: super -dynamic -f null -c "sort key" data.csup
  Time (mean ± σ):      9.415 s ±  0.079 s    [User: 17.170 s, System: 0.664 s]
  Range (min … max):    9.326 s …  9.478 s    3 runs

Details

Repro is with super commit 7133a70, which is associated with the merge of the changes in #7073.

Here was the performance at the commit just prior to that:

$ super -version &&
hyperfine --warmup 1 --runs 3 --time-unit second 'super -dynamic -f null -c "sort key" data.csup'

Version: v0.3.0-221-g2c7cf3b83

Benchmark 1: super -dynamic -f null -c "sort key" data.csup
  Time (mean ± σ):      0.504 s ±  0.002 s    [User: 1.860 s, System: 0.309 s]
  Range (min … max):    0.502 s …  0.506 s    3 runs

The problem originally surfaced in a frequently-studied "crunchy query" that contains a lot of other complex query logic, so I enlisted the help of Claude to do some profiling and create the simplified repro above that it claimed exercises the parts of the code that was causing most of the observed slow-down. As often happens in such exercises, Claude also offered its opinions on possible fixes, so I've pasted that into a Gist if anyone cares to take a look.