feature request: min µs

[petaflot]

from the examples in https://github.com/belm0/perf-timer:

timer "process thumbnail": avg 73.1 µs ± 18.0 µs, max 320.5 µs in 292 runs

• average is massively cool
• deviation is massively cool
• max is massively cool
• number of runs is also very cool

why is min missing?!? that makes no sense to me.

[petaflot]

one very useful example is as follows:

I write a function which caches its result for the next call ; the first call takes a long time, while the subsequent ones will (supposedly) be faster. I do not necessarily want to restart my program (and do the necessary user interactions for the function calls) ; since the timing of the first call can be very variable (I suppose it has a greater deviation than subsequent calls) the average will be greatly affected by the first call when all I want is an order of magnitude between the first call and the subsequent calls -> the difference between min and max is basically what I'm looking for (unless something quite resource-hungry like a bell curve is generated instead of a list of 5 values)

[belm0]

Thank you for your suggestion and use case explanation.

Besides caching that you mention, using a JIT is another case where the first run (and perhaps later runs if the argument types change) may be significantly slower.

Rather than providing a "min" statistic however, looking at histogram output would be a lot more revealing, and perf-timer already supports that. See https://github.com/belm0/perf-timer#histogram-statistics

[petaflot]

Histogram is nice, but not what I'm looking for.

Choosing max over min (or the opposite) is massively arbitrary ; both values are IMHO (my maths teachers said something similar) just as important.

[belm0]

Though it's a roundabout way, histogram can provide "min"-- it's the same as 0 percentile. Here you will get a report of min, median, and max:

PerfTimer('foo', observer=HistogramObserver, quantiles=(0, .5, 1))

Use of minimum duration is making the assumption that the code being measured only has two modes-- one slow and one fast-- and furthermore that the duration of the fast one doesn't vary. Generally that isn't true, and computation work of functions or code will vary based on the input. This is why minimum isn't of that much use. Maximum is somewhat more useful because we often want to know the worst case when optimizing.

One idea I've considered in the past is to have a "skip first observation" option, which would allow the default stats to be more useful for scenarios with JIT or caching on the first run.