feat(Query): query complexity framework with sorting examples

[kim-em]

This PR implements Sebastian Graf's unified approach to query complexity (discussed in the CSLib Algorithm frameworks thread), combining the strengths of #372 (explicit Prog/FreeM query types) and #376 (monad-parametric approach).

Programs are Prog Q α (free monad over query type Q), and the oracle is supplied after the program produces its query plan — giving anti-cheating guarantees for both upper and lower bounds. No WP/Hoare triple machinery is needed: correctness is just equations about Prog.eval oracle, and cost is just equations about Prog.queriesOn oracle.

This provides an alternative to the TimeM-based cost analysis already in the repo: here query counting is structural (derived from the Prog tree) rather than annotation-based.

New files

File	Contents
Query/Prog.lean	Core Prog type, eval, queriesOn, simp lemmas
Query/Bounds.lean	UpperBound and LowerBound definitions
Query/QueryTree.lean	Decision trees with fixed response type, for lower bound proofs
Query/Sort/LEQuery.lean	Comparison query type for sorting
Query/Sort/IsSort.lean	IsSort correctness specification
Query/Sort/Insertion/{Defs,Lemmas}.lean	Insertion sort: correctness + O(n²) upper bound
Query/Sort/Merge/{Defs,Lemmas}.lean	Merge sort: correctness + n·⌈log₂ n⌉ upper bound
Query/Sort/QueryTree.lean	Prog-to-QueryTree bridge + pigeonhole depth lemma
Query/Sort/LowerBound.lean	Any correct comparison sort needs ≥ ⌈log₂(n!)⌉ queries

Results

• Insertion sort: correctness (permutation + sortedness), n² upper bound, IsSort instance
• Merge sort: correctness, n·⌈log₂ n⌉ upper bound, IsSort instance
• Lower bound: any IsSort on an infinite type makes ≥ ⌈log₂(n!)⌉ queries. The proof constructs n! distinct total orders via permutations of embedded elements, shows they force distinct sorted outputs, then applies an adversarial pigeonhole argument on QueryTree depth.

🤖 Prepared with Claude Code

[Shreyas4991]

1. I wish to point out that the authorship and copyright headers must also be changed. This is my code. The FRO can't claim copyright.
2. Same for author comments. The order of authorship matters.
3. This PR still doesn't address the technical deficiencies w.r.t. mine, because it entirely overwrites my content.
4. Query Trees are still redundant. They are trivially derivable from Progs.
5. The Arith query should use my query and build on top of it.
6. Per Fabrizio's message to me there was no mention of "superseding" any of my work. He said the pr would build on top of mine.
7. The overall technical merits of adding extra monadic polymorphism becomes even less apparent. Technically it doesn't even make sense to add another monad parameter since FreeM is already parametrising a functor (more general than a monad). That is it is already monad parametric.
8. All the features claimed to be derived from model hiding can already be achieved in the model of 372. In fact this is as simple as quantifying over arbitrary models in theorem statements. The extra monad parameter is redundant. It can actually be substituted by any type constructor (which is essentially what the F in FreeM F a really is).
9. The examples I added should be preserved.

[sgraf812]

I think 3 of my 4 items in #401 (review) haven't been addressed yet. I think the doX stuff has been addressed.

[sgraf812]

Still relevant

[sgraf812]

Delete this file. Use ProgM throughout.

[Shreyas4991]

Programs are Prog Q α (free monad over query type Q), and the oracle is supplied after the program produces its query plan — giving anti-cheating guarantees for both upper and lower bounds. No WP/Hoare triple machinery is needed: correctness is just equations about Prog.eval oracle, and cost is just equations about Prog.queriesOn oracle.

One small nitpick, in the counting of time complexity, the complexity of an oracle call can depend on size/parameters of inputs supplied to it. For example the cost of calling a vertex connectivity query to a subgraph depends on the edge and vertex size of a subgraph. The current definition simply assumes each oracle call costs 1.

[kim-em]

• Define queriesOn as an abbreviation of cost, kill lemmas for the former

@sgraf812 could you take a look again and see if you'd still like me to remove things here?

[eric-wieser]

This is missing pure and Id.run

[eric-wieser]

Requiring this for even bad oracles is an interesting but reasonable choice.

[sgraf812]

Although is it really reasonable? I think it could unnecessarily rule out some high performance implementations. I don't have a concrete example, though...

[sorrachai]

The name isSort is misleading because you also include perm inside. More descriptive/explicit name is better e.g., isSortPerm.

[eric-wieser]

I'd suggest making all these wrappers for FreeM.lift abbrevs

[eric-wieser]

(as I do in #525)

[sgraf812]

Much better 👍

[sgraf812]

Although is it really reasonable? I think it could unnecessarily rule out some high performance implementations. I don't have a concrete example, though...

[sgraf812]

Minor: I tend to read queriesOn and think "this gives back a trace of the queries done by p" even if it doesn't make sense typically. Maybe cost1? Or countQueries (which nicely matches existing use of count in Std)?

[sgraf812]

LEQuery.ask here hints at a missing simp lemma? Shouldn't eval_ask have fired? Maybe it is fixed by making LEQuery.ask an abbrev, as Eric suggests...

[sgraf812]

Similarly here. Is there a reusable simp lemma missing?

[sgraf812]

and here

[Shreyas4991]

Queries can have more generic, input-size dependent costs.

[eric-wieser]

I don't think there's a good reason to restrict to Nat here.

[eric-wieser]

I think we should just prove that (orderedInsert x l).eval oracle = l.orderedInsert x (fun x y => oracle (.le x y)), and then we can delete most of this file.

[sorrachai]

Beautiful.

[sorrachai]

FreeM is more robust than TimeM, but I understand that FreeM is not immune to cost cheating, right? If so, we should explicitly write a warning/caveat to the users so they aware some weaknesses. In particular, we should write how much trust do we require in this model so that the complexity is counted correctly.

[Shreyas4991]

This is fairly trivial. The scope for cheating is essentially "sneak in operations via pure". This is different from TimeM, where additionally the location and individual cost annotations can vary from place to place which can also cause problems. One must also ideally avoiding adding extraneous typeclass instances and choose Boolean propositions.

This is however one form of "cheating" that I am not too worried about anymore.

1. On the one hand, we do want pure operations when we don't care too much about their implementation details. This came up in my MPI talk.

2. I have two ideas on how to eliminate pure operations based issues with model-level lower bound proofs entirely, in addition to model/input hiding. Can elaborate on Zulip if asked.

[sorrachai]

Can you elaborate on why using List.length is bad in the termination argument?

[Shreyas4991]

The odd even split uses structural recursion instead of well founded recursion.

[sorrachai]

Suggested change

	| [], ys => pure ys
	| [], ys => pure ys

return ys has better stylistic reading. This applies to other similar lines.

[sorrachai]

Suggested change

	@[expose] def orderedInsert (x : α) : List α → FreeM (LEQuery α) (List α)
	| [] => pure [x]
	| y :: ys => do
	let le ← LEQuery.ask x y
	if le then
	pure (x :: y :: ys)
	else do
	let rest ← orderedInsert x ys
	pure (y :: rest)
	@[expose] def orderedInsert (x : α) : List α → FreeM (LEQuery α) (List α)
	| [] => pure [x]
	| y :: ys => do
	let le ← LEQuery.ask x y
	if le then
	pure (x :: y :: ys)
	else do
	let rest ← orderedInsert x ys
	pure (y :: rest)

Stylistic change suggestion:

@[expose] def orderedInsert (x : α) : List α → FreeM (LEQuery α) (List α)
| [] => return [x]
| y :: ys => do
let le ← LEQuery.ask x y
if le then
return (x :: y :: ys)
else do
let rest ← orderedInsert x ys
return (y :: rest)