DynamicPPL 0.41.8 vs 0.42.0: `posteriordb-bench` Comparison

[shravanngoswamii]

Environment (versioninfo())

Julia Version 1.12.6
Commit 15346901f00 (2026-04-09 19:20 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 16 × 12th Gen Intel(R) Core(TM) i5-12500H
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, alderlake)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 16 virtual cores)

Results

DPPL 0.41.8

==========================================================================================
                               eval                             gradient
                      ----------------------  --------------------------------------------
Model            dim       Turing       Stan      FwdDiff     Enzyme   Mooncake       Stan
------------------------------------------------------------------------------------------
arma-arma11        4     775.8 ns     1.2 μs       2.7 μs     2.2 μs    12.9 μs     8.0 μs
earnings-lh        3       1.1 μs     1.5 μs       4.6 μs     9.1 μs    42.2 μs    14.3 μs
earnings-lhm       4       1.6 μs     1.6 μs      14.2 μs     9.5 μs    51.3 μs    23.4 μs
es-esc            10     124.8 ns   246.9 ns     521.8 ns   336.6 ns     1.1 μs   496.2 ns
es-esn            10     110.2 ns   369.1 ns     580.3 ns   313.4 ns     1.2 μs   745.3 ns
garch-garch11      4       2.7 μs     6.9 μs       6.3 μs     5.2 μs    16.1 μs    14.6 μs
gpr-gpr           13       1.7 μs     2.7 μs      17.8 μs     8.0 μs    40.5 μs     5.5 μs
kidiq-km           3       1.2 μs   904.3 ns       4.0 μs     6.4 μs    23.0 μs     5.8 μs
rm-rhin           90       6.8 μs    11.5 μs     265.2 μs    14.7 μs   108.6 μs    61.4 μs
rd-rm             65       1.1 μs     1.4 μs      26.9 μs     3.2 μs    11.2 μs     6.1 μs
sblrc-blr          6     280.4 ns   347.0 ns       2.3 μs     1.3 μs     5.4 μs     1.1 μs
sblri-blr          6     350.5 ns   437.5 ns       3.3 μs     1.8 μs     5.1 μs     1.4 μs
==========================================================================================

DPPL 0.42.0

==========================================================================================
                               eval                             gradient
                      ----------------------  --------------------------------------------
Model            dim       Turing       Stan      FwdDiff     Enzyme   Mooncake       Stan
------------------------------------------------------------------------------------------
arma-arma11        4       1.5 μs     2.1 μs       6.5 μs     4.0 μs    25.4 μs    13.1 μs
earnings-lh        3       2.4 μs     3.0 μs       7.7 μs    19.3 μs    74.2 μs    26.5 μs
earnings-lhm       4       2.7 μs     4.4 μs      17.0 μs    25.3 μs   116.3 μs    44.6 μs
es-esc            10     271.5 ns   447.2 ns       1.4 μs   737.0 ns     3.0 μs   845.8 ns
es-esn            10     273.4 ns   571.0 ns       1.7 μs   718.8 ns     2.7 μs     1.1 μs
garch-garch11      4       6.5 μs    14.2 μs      18.5 μs    11.6 μs    36.6 μs    27.1 μs
gpr-gpr           13       4.2 μs     4.1 μs      48.3 μs    25.0 μs    87.5 μs     8.2 μs
kidiq-km           3       2.2 μs     1.5 μs      10.1 μs    15.5 μs    50.0 μs     9.8 μs
rm-rhin           90      18.4 μs    16.9 μs     504.2 μs    38.1 μs   227.2 μs    92.9 μs
rd-rm             65       2.5 μs     2.4 μs      76.2 μs     7.0 μs    34.9 μs    11.2 μs
sblrc-blr          6     682.4 ns   626.4 ns       6.5 μs     3.7 μs    10.0 μs     2.1 μs
sblri-blr          6     630.5 ns   619.8 ns       6.1 μs     4.0 μs    10.6 μs     2.1 μs
==========================================================================================

Gradient / primal ratio per backend (each run normalises by its own primal)

This removes the per-run noise floor and isolates how expensive AD is relative to one primal call.

Model	FwdDiff 0.41	FwdDiff 0.42	Enzyme 0.41	Enzyme 0.42	Mooncake 0.41	Mooncake 0.42
arma-arma11	3.48	4.33	2.84	2.67	16.63	16.93
earnings-lh	4.18	3.21	8.27	8.04	38.36	30.92
earnings-lhm	8.88	6.30	5.94	9.37	32.06	43.07
es-esc	4.18	5.16	2.70	2.71	8.81	11.05
es-esn	5.27	6.22	2.84	2.63	10.89	9.88
garch-garch11	2.33	2.85	1.93	1.78	5.96	5.63
gpr-gpr	10.47	11.50	4.71	5.95	23.82	20.83
kidiq-km	3.33	4.59	5.33	7.05	19.17	22.73
rm-rhin	39.00	27.40	2.16	2.07	15.97	12.35
rd-rm	24.45	30.48	2.91	2.80	10.18	13.96
sblrc-blr	8.20	9.53	4.64	5.42	19.26	14.66
sblri-blr	9.41	9.68	5.13	6.35	14.55	16.81

Resolved versions

Package	DPPL 0.41 Baseline	DPPL 0.42 Candidate
Turing	v0.45.0 (released)	v0.45.0 (locally patched compat)
DynamicPPL	v0.41.8	v0.42.0
AbstractPPL	v0.14.x	v0.15.x
Bijectors	v0.15.24	v0.16.0
AdvancedVI	v0.6.2	v0.7.0 (PR #255)
FlexiChains	v0.6.12	v0.6.12 (PR #234 + local patch)

Reproducing locally

cd YOUR-WORKING-DIR
git clone https://github.com/stan-dev/posteriordb

cd posteriordb-bench
julia --project=. -e '
using Pkg
Pkg.develop([
    Pkg.PackageSpec(path="/home/local-path/Turing.jl"),       # patched compat
    Pkg.PackageSpec(path="/home/local-path/AdvancedVI.jl"),   # PR #255
    Pkg.PackageSpec(path="/home/local-path/FlexiChains.jl"),  # PR #234 + DPPL compat patch
])
Pkg.add(Pkg.PackageSpec(name="DynamicPPL", version="0.42.0"))   # or 0.41.8
'
julia --project=. bench.jl

ref: https://github.com/JuliaBayes/posteriordb-bench

[shravanngoswamii]

I don't think there are any unexpected major performance regressions, few are improved a lot and few worsened and these are acceptable.