Benchmarks
Interim numbers
The figures and numbers on this page are the development state: one round per case, measured on the machines listed at the bottom under their normal load, at the commits given below. The measurement campaign on quiet machines will replace them; the protocol below is already the one the campaign follows.
The numbers in the text are read from the summaries that the figure scripts write (bench/fig_speedup.jl, bench/fig_standing.jl), so text and figures come from the same data.
Which library to depend on
This figure asks the question a package author faces: which single library to depend on. Per case, the fold change is the time of the fastest other library divided by this library's time; right of the parity line is a win, left of it a loss, and wins are not clipped at 1. The box is the quartile rectangle with the median line and Tukey whiskers, the points are the raw cases. Colour marks the library. The label under each column carries the fold factor (the geometric mean of the fold changes over the cases the library itself computes), the win rate, and the coverage: libraries that lack transform kinds (ducc0 without DCT/DST, vDSP with c2c and DCT-II only, MPSGraph and cuFFT without real-to-real at all) compute their factor over a smaller subset, so the coverage is part of the label.
FFTL is the only library that runs in all six panels and computes every kind. Its fold factor ranges from 0.88× to 1.44× over the panels. It is below 1 on the NVIDIA A6000 (0.88×), where VkFFT has the highest factor (1.03×); see Platforms and limits.
How FFTL compares
The speed-up is the time of the fastest other library for that case divided by FFTL's time; above 1 (shaded), FFTL is faster. One marker per transform; colour is the precision, shape the kind; open triangles at the panel edge mark values beyond the axis. In the two multi-thread panels every library is timed at its better setting per case, the minimum over one and all threads, so FFTW is not charged for spreading tiny transforms over many threads. Transforms that no competitor computes (DCT and DST in cuFFT and MPSGraph, most real-to-real kinds) are not plotted. The table gives the share of cases in which FFTL is faster than the fastest other library, and the median speed-up; on the A6000, small transforms are bound by kernel launches.
| platform | competitors | FFTL faster in | median speed-up | cases |
|---|---|---|---|---|
| Apple M1 Max, 1 thread | FFTW, ducc0, vDSP | 90 % | 1.34× | 386 |
| AMD EPYC 7702, 1 thread | FFTW, ducc0 | 79 % | 1.27× | 386 |
| Apple M1 Max, 8 threads | FFTW, ducc0 | 92 % | 1.39× | 386 |
| AMD EPYC 7702, 16 threads | FFTW, ducc0 | 75 % | 1.26× | 386 |
| Apple M1 Max GPU | MPSGraph, VkFFT | 73 % | 1.24× | 119 |
| NVIDIA RTX A6000 | cuFFT, VkFFT | 22 % | 0.88× | 119 |
The data is the overview set in bench/results. Each data file has a .meta file with its provenance; the table lists the FFTL commit and the start of each run.
| data | FFTL commit | measured |
|---|---|---|
overview-a6000-gpu-kinds.csv | 49648b3 | 2026-11-03 |
overview-m1-gpu-kinds-0ab2ed0.csv | fec5b70 | 2026-11-06 |
overview-m1-t1-kinds-0ab2ed0.csv | 9e9bcc2 | 2026-11-06 |
overview-m1-t1-kinds-8e31d3a.csv | 8e31d3a | 2026-11-07 |
overview-m1-t8-kinds-0ab2ed0.csv | fec5b70 | 2026-11-06 |
overview-m1-t8-kinds-8e31d3a.csv | 8e31d3a | 2026-11-07 |
overview-reco2-t1-kinds-8e31d3a.csv | 8e31d3a | 2026-11-07 |
overview-reco2-t1-kinds-f825bee.csv | f825bee | 2026-11-06 |
overview-reco2-t16-kinds-8e31d3a.csv | 8e31d3a | 2026-11-07 |
overview-reco2-t16-kinds-f825bee.csv | f825bee | 2026-11-06 |
What the campaign covers
Complex, real and real-to-real transforms on the CPU, 1D to 3D, Float32 and Float64, against the competitors that support the same kind. Each kind has its own column; kinds without a competitor are shown separately.
Threads. The same sizes with one thread and with the number of threads of the machine, with the noise band of each machine.
GPU. Complex, real and real-to-real transforms against the vendor library (cuFFT on CUDA, MPSGraph and Metal.jl's FFT on Metal) and against VkFFT.
libfftl. The C library against the Julia package.
Planning. The cost of the first plan in a fresh process, separate from the execution time.
Accuracy. Error against a high-precision reference over the same sizes.
How it is measured
The protocol is described on the Performance page: A/B runs on the same machine, medians of interleaved runs, A/A runs for the noise band, and sizes reported separately from the sizes used for tuning.
Machines
| machine | role |
|---|---|
| Apple M1 Max | development machine, CPU and Metal GPU |
| AMD EPYC 7702 (Zen 2, 2 sockets, AVX2) | x86-64 CPU of the figures above |
| AMD EPYC 7763 (Zen 3, 2 sockets, AVX2) | x86-64 development machine; hosts the A6000 |
| NVIDIA RTX A6000 | CUDA GPU |
Each result names the library, its version, the build flags and the thread count. Competitors are built from their current releases with native CPU flags; FFTW is planned with MEASURE.
Status
| part | status |
|---|---|
| harness, protocol and figures | prepared |
| campaign (quiet machines, several rounds, held-out sizes) | planned |