Skip to content

quantile(v, p; sorted=true) is O(n) due to unconditional NaN scan #203

Description

@Beforerr

quantile(v, p; sorted=true) scales linearly in length(v) despite the sort being skipped.

using Chairmarks, Statistics
for n in (10_000, 100_000, 1_000_000)
    v = sort(rand(n))
    @info "n=$n" (@b quantile($v, 0.25; sorted=true))
end
# n=10000     2.9 μs
# n=100000   28.5 μs
# n=1000000  300  μs   ≈ time of any(isnan, v) alone

Cause — _quantilesort! (src/Statistics.jl:1038):

if (sorted && (ismissing(v[end]) || (v[end] isa Number && isnan(v[end])))) ||
    any(x -> ismissing(x) || (x isa Number && isnan(x)), v)
    throw(ArgumentError(...))
end

Shape (sorted && check_end) || any(check_all): when sorted=true and v[end] is finite, falls through to the full any scan. The v[end] fast-path never fires on the happy path.

Two options:

  • Trust the contract (sorted ? check_end : any(check_all)) — O(1) when sorted, matches docstring ("can be assumed to be sorted"). Requires updating the test at test/runtests.jl:707-713 which plants NaN mid-array under sorted=true.
  • Drop the dead v[end] branch — preserves current behavior, no perf change, just clarity.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions