RK4 oscillators: results#
The RK4 oscillator workload runs on both a GPU and a CPU; each device gets
its own card, generated by its own script (perf_card.py / cpu_card.py),
so the arm lists differ (the GPU card also includes a CPU OpenMP row as its
correctness reference, at 8 threads only; the CPU card is the fuller
CPU-side comparison, with 1- and 8-thread hawk + eagle rows plus Numba, JAX,
multiprocessing and plain Python).
Pick a device with the tabs; the choice carries over to the other results pages. Inside a device, the tabs switch the workload (spread or uniform stop steps), and the table of every arm and every N is collapsed under its charts. How to read the numbers: Reading the numbers.
GPU#
Run on: driver 580.178.04; CUDA runtime 12.9; host CPU Intel Xeon Platinum 8481C CPU @ 2.70GHz; compiler x86_64-conda-linux-gnu-g++ (conda-forge gcc 14.4.0-7) 14.4.0; Python 3.12.14; Warp 1.17.0, JAX 0.11.2, PyTorch 2.12.1+cu129, CuPy 14.2.0, NumPy 2.5.3.
N = 1,000,000
Log scale, from 106 µs to 58.2 ms.
N = 1,000,000
Log scale, from 383 µs to 589 ms.
N = 1,000,000
Log scale, from 1.07 ms to 584 ms.
Table of every arm and every N
Device: NVIDIA H100 80GB HBM3 (compute capability 9.0, 132 SMs). Peak FP64: 3.35e13 FLOP/s. Peak DRAM bandwidth: 3.35e12 B/s. Mean of the middle 3 of 5 runs of each arm (the highest and the lowest dropped), range of those 3 in parentheses.
Each arm’s block starts once the GPU is within 3 °C of its idle 30 °C and reads idle, or after 20 s; the SM clock is read right after each timed run and its range is in the last column.
The FP64 peak above is at the device’s reported maximum clock (1980 MHz). During the timed repetitions the SM clock read 1980 MHz; at that clock the FP64 peak is 3.35e13 FLOP/s, and every FP64 percentage below is 1× larger against it.
“% of issue peak” (the eagle arms only) counts DP instructions (DFMA/DMUL/DADD/DSETP/MUFU), not useful FLOPs: the step’s SASS has ? of them, against the device’s DP issue-slot throughput at the observed clock – a kernel that is already at the issue pipe’s limit can still read well under 100% of FP64 peak, because useful FLOPs/step is smaller than issued DP instructions/step.
eagle_graph checks its stop guard every step; eagle_graph_compact checks it once every 16 steps (the cadence its compaction runs on) and reads an active-set map built from it. The two rows below differ in both the cadence and the map; the compaction cadence sweep further down holds the cadence fixed at a few values and isolates the map alone.
PyTorch, CUDA graph of 16 steps captures 16 masked steps once as a torch.cuda.CUDAGraph over static buffers and replays it, with one host check of “any sample running” per replay: it removes the per-kernel launch overhead and keeps the masking (every sample is computed every step, finished ones held by torch.where, so the steps a replay runs past the last stop change nothing). JAX’s while loop is one compiled executable with no host check per step; batched by vmap, it runs until the slowest sample stops. Warp runs one thread per sample, each stopping at its own step.
Every arm runs each sample to its own stop and no further than max_steps (S): eagle and Warp enforce the cap per sample, JAX in its while-loop condition, PyTorch and CuPy by the outer step count (a batch-level cap); an arm that runs whole blocks of steps per launch or replay stops at the first block boundary at or after it.
What each arm needed to get there: Warp, a hand-written step counter and cap inside the kernel; JAX, a cap in the while_loop condition; eagle, the kernel source unchanged, with its loop shape, unrolling and device entry chosen by hawk + eagle (per device, per batch).
Cap fixture (checked before any timing counts): N = 4,096, 8 samples with stop step 2S = 256, S = 128; every arm (16) reported k = min(stop step, S) per sample and the same finished count (4,088): passed.
arm |
loop structure |
|---|---|
eagle graph (device loop) |
per launch over the batch, inside one CUDA graph: the plain decorator’s kernel, which finishes its own samples and runs the steps per launch eagle’s policy picks (8 to 64), guard checked on the device every launch |
eagle graph + compaction |
as the graph arm, the step launched over the active-set map only (recomputed every 16 steps) |
eagle graph + compaction + reorder |
as the compaction arm, plus a physical reorder of the per-sample planes when the live samples spread thin |
eagle graph, K fused steps per launch |
per launch over the batch, inside one CUDA graph: the step kernel runs up to K = 16 steps per sample with the state in registers, each sample leaving at its own stop step; guard checked on the device every launch |
eagle graph, K fused steps per launch + compaction |
as the K-steps arm, the launch over the active-set map only (recomputed every 64 steps = 4 launches) |
eagle auto (eagle picks the launch mode) |
the active-set kernel under eagle’s automatic policy, which picks the launch mode from the batch: one launch running every sample to its end for a small batch, the persistent launch above that, and the compaction loop (active-set map, steps per launch picked on the device) only when the caller asks for compaction or reordering; the row records the mode taken |
hawk + eagle persistent |
ONE launch, no graph, no map, no policy kernel: a grid sized from the SM count and the kernel’s occupancy, each lane fetching base + atomicAdd(counter, 1) whenever its sample is done, running the whole budget’s worth of steps per sample it picks up |
eagle.simulate (eagle picks the launch mode) |
as the automatic arm: eagle.simulate takes the kernel, its state and its parameters by name and eagle picks the launch mode the same way |
eager launch loop |
per step over the batch: one kernel launch from Python, a host read of the finished count every step |
CuPy masked |
per step over the batch: whole-batch array expressions, finished samples held by cupy.where, a host check every step; the cap is batch-level (the outer step loop runs at most max_steps) |
PyTorch masked |
per step over the batch: whole-batch tensor expressions, finished samples held by torch.where, a host check every step; the cap is batch-level (the outer step loop runs at most max_steps) |
PyTorch, CUDA graph of 16 steps |
as PyTorch masked, 16 steps captured as one CUDA graph over static buffers, replayed with one host check per replay |
PyTorch torch.compile (reduce-overhead) |
as PyTorch masked, the step compiled by Inductor and replayed as a CUDA graph, a host check every step |
JAX jit + vmap(while_loop) |
per step over the batch, inside one compiled executable: the vmapped while loop runs until the slowest sample stops or the step cap is reached, finished samples held by a select |
Warp per-thread kernel |
per sample: one launch, each thread loops over its own sample’s steps and stops at its own stop step or the step cap, whichever comes first |
CPU OpenMP |
per step over the batch on the host: one host-team launch per step, the kernel marks and counts its finished samples |
Spread stop steps (log-uniform in [1, 100]), up to 100 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
313 µs (298 µs–325 µs) |
– |
404 µs |
7.05e7 |
3.10e9 (0.0093 %) |
2.33e9 (0.07 %) |
2 MiB (58.3×) |
2.04 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eagle graph + compaction |
556 µs (554 µs–557 µs) |
– |
675 µs |
3.97e7 |
1.75e9 (0.0052 %) |
3.55e9 (0.11 %) |
2 MiB (58.3×) |
2.07 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eagle graph + compaction + reorder |
645 µs (641 µs–647 µs) |
– |
791 µs |
3.42e7 |
1.51e9 (0.0045 %) |
3.71e9 (0.11 %) |
2 MiB (58.3×) |
2.14 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eagle graph, K fused steps per launch |
153 µs (149 µs–159 µs) |
– |
241 µs |
1.44e8 |
6.34e9 (0.019 %) |
1.87e9 (0.056 %) |
2 MiB (58.3×) |
2.03 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eagle graph, K fused steps per launch + compaction |
204 µs (197 µs–208 µs) |
– |
330 µs |
1.08e8 |
4.76e9 (0.014 %) |
1.35e9 (0.04 %) |
2 MiB (58.3×) |
2.04 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eagle auto (eagle picks the launch mode) |
42.4 µs (42.1 µs–42.8 µs) |
– |
148 µs |
5.21e8 |
2.29e10 (0.068 %) |
1.54e10 (0.46 %) |
0 MiB (0×) |
2.05 MiB |
– |
– |
– |
1980–1980 |
1,000 |
hawk + eagle persistent |
81.1 µs (79.9 µs–82.7 µs) |
– |
161 µs |
2.72e8 |
1.20e10 (0.036 %) |
8.04e9 (0.24 %) |
0 MiB (0×) |
2.04 MiB |
– |
0.2098712531887755 |
– |
1980–1980 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
44 µs (42.9 µs–45.9 µs) |
– |
144 µs |
5.01e8 |
2.21e10 (0.066 %) |
1.48e10 (0.44 %) |
0 MiB (0×) |
2.06 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eager launch loop |
1.96 ms (1.95 ms–1.98 ms) |
– |
2.04 ms |
1.12e7 |
4.94e8 (0.0015 %) |
1.99e9 (0.06 %) |
0 MiB (0×) |
0.0312 MiB |
– |
– |
– |
1980–1980 |
1,000 |
CuPy masked |
34.2 ms (33.8 ms–34.7 ms) |
– |
34.4 ms |
6.44e5 |
2.83e7 (8.5e-05 %) |
2.63e9 (0.079 %) |
0 MiB (0×) |
4.98 MiB |
– |
– |
– |
1980–1980 |
1,000 |
PyTorch masked |
24.2 ms (24.1 ms–24.4 ms) |
– |
24.3 ms |
9.11e5 |
4.01e7 (0.00012 %) |
3.72e9 (0.11 %) |
90 MiB (2.62e+03×) |
0.0703 MiB |
– |
– |
– |
1980–1980 |
1,000 |
PyTorch, CUDA graph of 16 steps |
7.84 ms (7.83 ms–7.84 ms) |
– |
7.96 ms |
2.82e6 |
1.24e8 (0.00037 %) |
1.29e10 (0.39 %) |
162 MiB (4.72e+03×) |
0.16 MiB |
– |
– |
– |
1980–1980 |
1,000 |
PyTorch torch.compile (reduce-overhead) |
6.71 ms (6.61 ms–6.78 ms) |
– |
6.86 ms |
3.29e6 |
1.45e8 (0.00043 %) |
1.31e9 (0.039 %) |
162 MiB (4.72e+03×) |
32.1 MiB |
– |
– |
– |
1980–1980 |
1,000 |
JAX jit + vmap(while_loop) |
2.49 ms (2.48 ms–2.49 ms) |
– |
3.59 ms |
8.87e6 |
3.90e8 (0.0012 %) |
3.55e9 (0.11 %) |
158 MiB (4.6e+03×) |
0.0273 MiB |
– |
– |
– |
1980–1980 |
1,000 |
Warp per-thread kernel |
51.9 µs (50.9 µs–52.5 µs) |
– |
233 µs |
4.25e8 |
1.87e10 (0.056 %) |
1.23e9 (0.037 %) |
30 MiB (874×) |
0.0156 MiB |
– |
– |
– |
1980–1980 |
1,000 |
CPU OpenMP |
182 µs (176 µs–186 µs) |
– |
192 µs |
1.21e8 |
5.32e9 (–) |
2.15e10 (–) |
– |
0.363 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
324 µs (321 µs–326 µs) |
– |
484 µs |
7.06e8 |
3.10e10 (0.093 %) |
2.32e10 (0.69 %) |
2 MiB (5.83×) |
2.45 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eagle graph + compaction |
586 µs (577 µs–598 µs) |
– |
787 µs |
3.90e8 |
1.72e10 (0.051 %) |
3.46e10 (1 %) |
2 MiB (5.83×) |
2.45 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eagle graph + compaction + reorder |
835 µs (829 µs–839 µs) |
– |
1.04 ms |
2.74e8 |
1.20e10 (0.036 %) |
2.94e10 (0.88 %) |
2 MiB (5.83×) |
3 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eagle graph, K fused steps per launch |
160 µs (156 µs–165 µs) |
– |
318 µs |
1.43e9 |
6.30e10 (0.19 %) |
1.80e10 (0.54 %) |
2 MiB (5.83×) |
2.44 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eagle graph, K fused steps per launch + compaction |
214 µs (206 µs–222 µs) |
– |
400 µs |
1.07e9 |
4.71e10 (0.14 %) |
1.31e10 (0.39 %) |
2 MiB (5.83×) |
2.45 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eagle auto (eagle picks the launch mode) |
55.1 µs (51.6 µs–61 µs) |
– |
232 µs |
4.15e9 |
1.83e11 (0.55 %) |
1.22e11 (3.7 %) |
0 MiB (0×) |
2.62 MiB |
– |
– |
– |
1980–1980 |
10,000 |
hawk + eagle persistent |
83.3 µs (82.8 µs–84 µs) |
– |
223 µs |
2.74e9 |
1.21e11 (0.36 %) |
8.09e10 (2.4 %) |
0 MiB (0×) |
2.56 MiB |
– |
0.21156370013627354 |
– |
1980–1980 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
61.4 µs (60.6 µs–62.3 µs) |
– |
242 µs |
3.72e9 |
1.64e11 (0.49 %) |
1.10e11 (3.3 %) |
0 MiB (0×) |
2.62 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eager launch loop |
1.97 ms (1.97 ms–1.98 ms) |
– |
2.11 ms |
1.16e8 |
5.10e9 (0.015 %) |
2.00e10 (0.6 %) |
0 MiB (0×) |
0.406 MiB |
– |
– |
– |
1980–1980 |
10,000 |
CuPy masked |
35.4 ms (35.3 ms–35.6 ms) |
– |
35.6 ms |
6.45e6 |
2.84e8 (0.00085 %) |
2.54e10 (0.76 %) |
0 MiB (0×) |
5.27 MiB |
– |
– |
– |
1980–1980 |
10,000 |
PyTorch masked |
24.7 ms (24.5 ms–24.9 ms) |
– |
24.9 ms |
9.26e6 |
4.07e8 (0.0012 %) |
3.65e10 (1.1 %) |
90 MiB (262×) |
0.684 MiB |
– |
– |
– |
1980–1980 |
10,000 |
PyTorch, CUDA graph of 16 steps |
7.91 ms (7.91 ms–7.91 ms) |
– |
8.1 ms |
2.89e7 |
1.27e9 (0.0038 %) |
1.28e11 (3.8 %) |
162 MiB (472×) |
0.762 MiB |
– |
– |
– |
1980–1980 |
10,000 |
PyTorch torch.compile (reduce-overhead) |
6.83 ms (6.69 ms–6.91 ms) |
– |
7.07 ms |
3.35e7 |
1.47e9 (0.0044 %) |
1.29e10 (0.38 %) |
162 MiB (472×) |
32.7 MiB |
– |
– |
– |
1980–1980 |
10,000 |
JAX jit + vmap(while_loop) |
2.65 ms (2.63 ms–2.66 ms) |
– |
3.61 ms |
8.64e7 |
3.80e9 (0.011 %) |
3.34e10 (1 %) |
158 MiB (460×) |
0.727 MiB |
– |
– |
– |
1980–1980 |
10,000 |
Warp per-thread kernel |
59.2 µs (58.9 µs–59.6 µs) |
– |
330 µs |
3.86e9 |
1.70e11 (0.51 %) |
1.08e10 (0.32 %) |
30 MiB (87.4×) |
0.688 MiB |
– |
– |
– |
1980–1980 |
10,000 |
CPU OpenMP |
414 µs (408 µs–420 µs) |
– |
454 µs |
5.53e8 |
2.43e10 (–) |
9.53e10 (–) |
– |
1.26 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
401 µs (394 µs–410 µs) |
– |
1.04 ms |
5.70e9 |
2.51e11 (0.75 %) |
1.88e11 (5.6 %) |
6 MiB (1.75×) |
7.87 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eagle graph + compaction |
690 µs (688 µs–693 µs) |
– |
1.36 ms |
3.31e9 |
1.46e11 (0.44 %) |
2.94e11 (8.8 %) |
6 MiB (1.75×) |
7.89 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eagle graph + compaction + reorder |
1.09 ms (1.08 ms–1.1 ms) |
– |
1.78 ms |
2.10e9 |
9.24e10 (0.28 %) |
2.25e11 (6.7 %) |
10 MiB (2.91×) |
8.54 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eagle graph, K fused steps per launch |
235 µs (230 µs–239 µs) |
– |
871 µs |
9.72e9 |
4.28e11 (1.3 %) |
1.22e11 (3.6 %) |
6 MiB (1.75×) |
7.86 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eagle graph, K fused steps per launch + compaction |
265 µs (258 µs–275 µs) |
– |
911 µs |
8.65e9 |
3.81e11 (1.1 %) |
1.06e11 (3.1 %) |
6 MiB (1.75×) |
7.89 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eagle auto (eagle picks the launch mode) |
82 µs (80.9 µs–84.1 µs) |
– |
732 µs |
2.79e10 |
1.23e12 (3.7 %) |
8.23e11 (25 %) |
4 MiB (1.17×) |
7.84 MiB |
– |
– |
– |
1980–1980 |
100,000 |
hawk + eagle persistent |
125 µs (124 µs–127 µs) |
– |
731 µs |
1.83e10 |
8.05e11 (2.4 %) |
5.40e11 (16 %) |
4 MiB (1.17×) |
7.82 MiB |
– |
0.35324525439049587 |
– |
1980–1980 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
87.2 µs (86.3 µs–88.9 µs) |
– |
765 µs |
2.63e10 |
1.16e12 (3.5 %) |
7.74e11 (23 %) |
4 MiB (1.17×) |
7.89 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eager launch loop |
1.97 ms (1.96 ms–1.98 ms) |
– |
2.57 ms |
1.16e9 |
5.11e10 (0.15 %) |
2.00e11 (6 %) |
4 MiB (1.17×) |
5.76 MiB |
– |
– |
– |
1980–1980 |
100,000 |
CuPy masked |
36.2 ms (35.8 ms–36.4 ms) |
– |
37 ms |
6.33e7 |
2.78e9 (0.0083 %) |
2.49e11 (7.4 %) |
22 MiB (6.41×) |
8.59 MiB |
– |
– |
– |
1980–1980 |
100,000 |
PyTorch masked |
26.8 ms (26.5 ms–27.1 ms) |
– |
27.8 ms |
8.55e7 |
3.76e9 (0.011 %) |
3.37e11 (10 %) |
114 MiB (33.2×) |
6.86 MiB |
– |
– |
– |
1980–1980 |
100,000 |
PyTorch, CUDA graph of 16 steps |
8.44 ms (8.44 ms–8.44 ms) |
– |
9.23 ms |
2.71e8 |
1.19e10 (0.036 %) |
1.20e12 (36 %) |
184 MiB (53.6×) |
6.87 MiB |
– |
– |
– |
1980–1980 |
100,000 |
PyTorch torch.compile (reduce-overhead) |
6.99 ms (6.82 ms–7.11 ms) |
– |
7.8 ms |
3.28e8 |
1.44e10 (0.043 %) |
1.26e11 (3.8 %) |
168 MiB (48.9×) |
36.5 MiB |
– |
– |
– |
1980–1980 |
100,000 |
JAX jit + vmap(while_loop) |
2.73 ms (2.72 ms–2.74 ms) |
– |
4.35 ms |
8.37e8 |
3.68e10 (0.11 %) |
3.23e11 (9.6 %) |
158 MiB (46×) |
10.1 MiB |
– |
– |
– |
1980–1980 |
100,000 |
Warp per-thread kernel |
83.7 µs (82.7 µs–84.4 µs) |
– |
1.14 ms |
2.73e10 |
1.20e12 (3.6 %) |
7.64e10 (2.3 %) |
30 MiB (8.74×) |
6.87 MiB |
– |
– |
– |
1980–1980 |
100,000 |
CPU OpenMP |
2.36 ms (2.36 ms–2.37 ms) |
– |
2.83 ms |
9.68e8 |
4.26e10 (–) |
1.67e11 (–) |
– |
10.1 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
573 µs (561 µs–580 µs) |
– |
5.4 ms |
4.01e10 |
1.77e12 (5.3 %) |
1.32e12 (39 %) |
42 MiB (1.22×) |
62.8 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eagle graph + compaction |
2.02 ms (2.02 ms–2.03 ms) |
– |
6.96 ms |
1.14e10 |
5.00e11 (1.5 %) |
1.01e12 (30 %) |
58 MiB (1.69×) |
62.9 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eagle graph + compaction + reorder |
2.15 ms (2.14 ms–2.15 ms) |
– |
7.03 ms |
1.07e10 |
4.71e11 (1.4 %) |
1.14e12 (34 %) |
86 MiB (2.5×) |
63.1 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eagle graph, K fused steps per launch |
419 µs (417 µs–422 µs) |
– |
5.42 ms |
5.49e10 |
2.41e12 (7.2 %) |
6.87e11 (20 %) |
42 MiB (1.22×) |
62.8 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
457 µs (455 µs–459 µs) |
– |
5.33 ms |
5.03e10 |
2.21e12 (6.6 %) |
6.11e11 (18 %) |
58 MiB (1.69×) |
62.8 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
163 µs (161 µs–164 µs) |
– |
5.04 ms |
1.41e11 |
6.20e12 (19 %) |
4.16e12 (1.2e+02 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
hawk + eagle persistent |
246 µs (245 µs–247 µs) |
– |
5.05 ms |
9.35e10 |
4.12e12 (12 %) |
2.76e12 (82 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
0.44834174898930623 |
– |
1980–1980 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
169 µs (164 µs–174 µs) |
– |
5.02 ms |
1.36e11 |
5.99e12 (18 %) |
4.01e12 (1.2e+02 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eager launch loop |
3.52 ms (3.51 ms–3.54 ms) |
– |
8.42 ms |
6.53e9 |
2.87e11 (0.86 %) |
1.12e12 (33 %) |
40 MiB (1.17×) |
58.2 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
CuPy masked |
36.4 ms (36.3 ms–36.5 ms) |
– |
42.3 ms |
6.32e8 |
2.78e10 (0.083 %) |
2.48e12 (74 %) |
192 MiB (5.59×) |
68.8 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
PyTorch masked |
33.7 ms (33.6 ms–33.8 ms) |
– |
46 ms |
6.82e8 |
3.00e10 (0.09 %) |
2.67e12 (80 %) |
350 MiB (10.2×) |
68.7 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
30.2 ms (30.2 ms–30.2 ms) |
– |
42.5 ms |
7.62e8 |
3.35e10 (0.1 %) |
3.36e12 (1e+02 %) |
422 MiB (12.3×) |
68.7 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
PyTorch torch.compile (reduce-overhead) |
8.69 ms (8.57 ms–8.83 ms) |
– |
20.8 ms |
2.65e9 |
1.16e11 (0.35 %) |
1.01e12 (30 %) |
242 MiB (7.05×) |
76.7 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
JAX jit + vmap(while_loop) |
4.95 ms (4.92 ms–4.99 ms) |
– |
12.8 ms |
4.65e9 |
2.04e11 (0.61 %) |
1.78e12 (53 %) |
158 MiB (4.6×) |
101 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
Warp per-thread kernel |
234 µs (231 µs–239 µs) |
– |
16 ms |
9.85e10 |
4.33e12 (13 %) |
2.74e11 (8.2 %) |
62 MiB (1.81×) |
68.7 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
CPU OpenMP |
21.1 ms (20.8 ms–21.3 ms) |
– |
30.1 ms |
1.09e9 |
4.81e10 (–) |
1.87e11 (–) |
– |
97.6 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (42.4 µs), then eagle.simulate (eagle picks the launch mode) (44 µs, 1.04× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (55.1 µs), then Warp per-thread kernel (59.2 µs, 1.08× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (82 µs), then Warp per-thread kernel (83.7 µs, 1.02× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (163 µs), then eagle.simulate (eagle picks the launch mode) (169 µs, 1.04× the time)
Spread stop steps (log-uniform in [10, 1000]), up to 1000 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
498 µs (488 µs–509 µs) |
– |
599 µs |
4.22e8 |
1.86e10 (0.055 %) |
1.31e10 (0.39 %) |
2 MiB (58.3×) |
2.04 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eagle graph + compaction |
3.9 ms (3.9 ms–3.9 ms) |
– |
4.04 ms |
5.39e7 |
2.37e9 (0.0071 %) |
4.05e9 (0.12 %) |
2 MiB (58.3×) |
2.04 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eagle graph + compaction + reorder |
4.12 ms (4.1 ms–4.12 ms) |
– |
4.26 ms |
5.11e7 |
2.25e9 (0.0067 %) |
4.01e9 (0.12 %) |
2 MiB (58.3×) |
2.14 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eagle graph, K fused steps per launch |
570 µs (565 µs–573 µs) |
– |
665 µs |
3.69e8 |
1.62e10 (0.049 %) |
4.32e9 (0.13 %) |
2 MiB (58.3×) |
2.03 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eagle graph, K fused steps per launch + compaction |
710 µs (706 µs–712 µs) |
– |
845 µs |
2.96e8 |
1.30e10 (0.039 %) |
2.01e9 (0.06 %) |
2 MiB (58.3×) |
2.03 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eagle auto (eagle picks the launch mode) |
94.9 µs (93.9 µs–96.3 µs) |
– |
207 µs |
2.21e9 |
9.74e10 (0.29 %) |
6.24e10 (1.9 %) |
0 MiB (0×) |
2.05 MiB |
– |
– |
– |
1980–1980 |
1,000 |
hawk + eagle persistent |
154 µs (150 µs–158 µs) |
– |
240 µs |
1.37e9 |
6.01e10 (0.18 %) |
3.84e10 (1.1 %) |
0 MiB (0×) |
2.04 MiB |
– |
0.22980196323115862 |
– |
1980–1980 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
103 µs (102 µs–103 µs) |
– |
221 µs |
2.05e9 |
9.00e10 (0.27 %) |
5.76e10 (1.7 %) |
0 MiB (0×) |
2.06 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eager launch loop |
18.7 ms (18.5 ms–18.7 ms) |
– |
18.8 ms |
1.13e7 |
4.96e8 (0.0015 %) |
2.08e9 (0.062 %) |
0 MiB (0×) |
0.0352 MiB |
– |
– |
– |
1980–1980 |
1,000 |
CuPy masked |
352 ms (349 ms–353 ms) |
– |
352 ms |
5.98e5 |
2.63e7 (7.9e-05 %) |
2.56e9 (0.076 %) |
0 MiB (0×) |
5.46 MiB |
– |
– |
– |
1980–1980 |
1,000 |
PyTorch masked |
244 ms (242 ms–245 ms) |
– |
244 ms |
8.61e5 |
3.79e7 (0.00011 %) |
3.69e9 (0.11 %) |
90 MiB (2.62e+03×) |
0.0703 MiB |
– |
– |
– |
1980–1980 |
1,000 |
PyTorch, CUDA graph of 16 steps |
70.1 ms (70.1 ms–70.1 ms) |
– |
70.3 ms |
3.00e6 |
1.32e8 (0.00039 %) |
1.30e10 (0.39 %) |
162 MiB (4.72e+03×) |
0.156 MiB |
– |
– |
– |
1980–1980 |
1,000 |
PyTorch torch.compile (reduce-overhead) |
65.7 ms (65.3 ms–66.1 ms) |
– |
65.9 ms |
3.20e6 |
1.41e8 (0.00042 %) |
1.34e9 (0.04 %) |
162 MiB (4.72e+03×) |
32.1 MiB |
– |
– |
– |
1980–1980 |
1,000 |
JAX jit + vmap(while_loop) |
21.8 ms (21.7 ms–21.8 ms) |
– |
23.5 ms |
9.65e6 |
4.25e8 (0.0013 %) |
4.04e9 (0.12 %) |
158 MiB (4.6e+03×) |
0.0234 MiB |
– |
– |
– |
1980–1980 |
1,000 |
Warp per-thread kernel |
112 µs (112 µs–112 µs) |
– |
300 µs |
1.88e9 |
8.27e10 (0.25 %) |
5.73e8 (0.017 %) |
30 MiB (874×) |
0.0117 MiB |
– |
– |
– |
1980–1980 |
1,000 |
CPU OpenMP |
406 µs (404 µs–409 µs) |
– |
417 µs |
5.18e8 |
2.28e10 (–) |
9.58e10 (–) |
– |
0.359 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
507 µs (506 µs–508 µs) |
– |
672 µs |
4.31e9 |
1.89e11 (0.57 %) |
1.33e11 (4 %) |
2 MiB (5.83×) |
2.44 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eagle graph + compaction |
4.06 ms (4.06 ms–4.07 ms) |
– |
4.26 ms |
5.37e8 |
2.36e10 (0.071 %) |
4.02e10 (1.2 %) |
2 MiB (5.83×) |
2.47 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eagle graph + compaction + reorder |
4.68 ms (4.68 ms–4.68 ms) |
– |
4.91 ms |
4.66e8 |
2.05e10 (0.061 %) |
3.61e10 (1.1 %) |
2 MiB (5.83×) |
3.02 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eagle graph, K fused steps per launch |
572 µs (570 µs–575 µs) |
– |
729 µs |
3.82e9 |
1.68e11 (0.5 %) |
4.33e10 (1.3 %) |
2 MiB (5.83×) |
2.44 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eagle graph, K fused steps per launch + compaction |
722 µs (721 µs–723 µs) |
– |
921 µs |
3.02e9 |
1.33e11 (0.4 %) |
2.02e10 (0.6 %) |
2 MiB (5.83×) |
2.45 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eagle auto (eagle picks the launch mode) |
106 µs (104 µs–109 µs) |
– |
283 µs |
2.06e10 |
9.07e11 (2.7 %) |
5.80e11 (17 %) |
0 MiB (0×) |
2.62 MiB |
– |
– |
– |
1980–1980 |
10,000 |
hawk + eagle persistent |
153 µs (148 µs–162 µs) |
– |
304 µs |
1.43e10 |
6.29e11 (1.9 %) |
4.03e11 (12 %) |
0 MiB (0×) |
2.57 MiB |
– |
0.2384722378329774 |
– |
1980–1980 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
118 µs (111 µs–127 µs) |
– |
317 µs |
1.85e10 |
8.13e11 (2.4 %) |
5.20e11 (16 %) |
0 MiB (0×) |
2.62 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eager launch loop |
19.3 ms (19.3 ms–19.4 ms) |
– |
19.5 ms |
1.13e8 |
4.96e9 (0.015 %) |
2.02e10 (0.6 %) |
0 MiB (0×) |
0.406 MiB |
– |
– |
– |
1980–1980 |
10,000 |
CuPy masked |
352 ms (350 ms–354 ms) |
– |
352 ms |
6.20e6 |
2.73e8 (0.00082 %) |
2.56e10 (0.76 %) |
0 MiB (0×) |
5.27 MiB |
– |
– |
– |
1980–1980 |
10,000 |
PyTorch masked |
250 ms (247 ms–253 ms) |
– |
250 ms |
8.74e6 |
3.85e8 (0.0012 %) |
3.61e10 (1.1 %) |
90 MiB (262×) |
0.684 MiB |
– |
– |
– |
1980–1980 |
10,000 |
PyTorch, CUDA graph of 16 steps |
70.8 ms (70.8 ms–70.9 ms) |
– |
71 ms |
3.08e7 |
1.36e9 (0.0041 %) |
1.29e11 (3.8 %) |
162 MiB (472×) |
0.766 MiB |
– |
– |
– |
1980–1980 |
10,000 |
PyTorch torch.compile (reduce-overhead) |
64.8 ms (64.6 ms–65.1 ms) |
– |
65.1 ms |
3.37e7 |
1.48e9 (0.0044 %) |
1.36e10 (0.4 %) |
162 MiB (472×) |
32.2 MiB |
– |
– |
– |
1980–1980 |
10,000 |
JAX jit + vmap(while_loop) |
24.4 ms (24.2 ms–24.5 ms) |
– |
26.3 ms |
8.95e7 |
3.94e9 (0.012 %) |
3.61e10 (1.1 %) |
158 MiB (460×) |
0.559 MiB |
– |
– |
– |
1980–1980 |
10,000 |
Warp per-thread kernel |
116 µs (115 µs–117 µs) |
– |
378 µs |
1.88e10 |
8.28e11 (2.5 %) |
5.52e9 (0.16 %) |
30 MiB (87.4×) |
0.691 MiB |
– |
– |
– |
1980–1980 |
10,000 |
CPU OpenMP |
2.26 ms (2.21 ms–2.35 ms) |
– |
2.3 ms |
9.66e8 |
4.25e10 (–) |
1.73e11 (–) |
– |
1.24 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
690 µs (685 µs–698 µs) |
– |
1.33 ms |
3.14e10 |
1.38e12 (4.1 %) |
9.70e11 (29 %) |
6 MiB (1.75×) |
7.88 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eagle graph + compaction |
5.08 ms (5.08 ms–5.08 ms) |
– |
5.78 ms |
4.26e9 |
1.88e11 (0.56 %) |
3.20e11 (9.5 %) |
6 MiB (1.75×) |
7.91 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eagle graph + compaction + reorder |
5.2 ms (5.2 ms–5.21 ms) |
– |
5.91 ms |
4.16e9 |
1.83e11 (0.55 %) |
3.23e11 (9.6 %) |
10 MiB (2.91×) |
8.54 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eagle graph, K fused steps per launch |
738 µs (737 µs–740 µs) |
– |
1.36 ms |
2.93e10 |
1.29e12 (3.9 %) |
3.35e11 (10 %) |
6 MiB (1.75×) |
7.87 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eagle graph, K fused steps per launch + compaction |
864 µs (859 µs–874 µs) |
– |
1.57 ms |
2.51e10 |
1.10e12 (3.3 %) |
1.68e11 (5 %) |
6 MiB (1.75×) |
7.88 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eagle auto (eagle picks the launch mode) |
197 µs (193 µs–200 µs) |
– |
860 µs |
1.10e11 |
4.84e12 (14 %) |
3.10e12 (92 %) |
4 MiB (1.17×) |
7.88 MiB |
– |
– |
– |
1980–1980 |
100,000 |
hawk + eagle persistent |
221 µs (218 µs–226 µs) |
– |
835 µs |
9.80e10 |
4.31e12 (13 %) |
2.76e12 (82 %) |
4 MiB (1.17×) |
7.82 MiB |
– |
0.5234178332680689 |
– |
1980–1980 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
199 µs (198 µs–200 µs) |
– |
850 µs |
1.09e11 |
4.79e12 (14 %) |
3.06e12 (91 %) |
4 MiB (1.17×) |
7.84 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eager launch loop |
20 ms (19.9 ms–20.1 ms) |
– |
20.6 ms |
1.08e9 |
4.77e10 (0.14 %) |
1.95e11 (5.8 %) |
4 MiB (1.17×) |
5.76 MiB |
– |
– |
– |
1980–1980 |
100,000 |
CuPy masked |
353 ms (352 ms–353 ms) |
– |
354 ms |
6.14e7 |
2.70e9 (0.0081 %) |
2.55e11 (7.6 %) |
22 MiB (6.41×) |
9.29 MiB |
– |
– |
– |
1980–1980 |
100,000 |
PyTorch masked |
265 ms (264 ms–266 ms) |
– |
266 ms |
8.18e7 |
3.60e9 (0.011 %) |
3.40e11 (10 %) |
114 MiB (33.2×) |
6.86 MiB |
– |
– |
– |
1980–1980 |
100,000 |
PyTorch, CUDA graph of 16 steps |
75.4 ms (75.4 ms–75.4 ms) |
– |
76.4 ms |
2.87e8 |
1.26e10 (0.038 %) |
1.21e12 (36 %) |
184 MiB (53.6×) |
6.87 MiB |
– |
– |
– |
1980–1980 |
100,000 |
PyTorch torch.compile (reduce-overhead) |
67.5 ms (66.2 ms–68.7 ms) |
– |
68.6 ms |
3.21e8 |
1.41e10 (0.042 %) |
1.30e11 (3.9 %) |
168 MiB (48.9×) |
36.1 MiB |
– |
– |
– |
1980–1980 |
100,000 |
JAX jit + vmap(while_loop) |
24.4 ms (24.1 ms–24.6 ms) |
– |
26.6 ms |
8.89e8 |
3.91e10 (0.12 %) |
3.61e11 (11 %) |
158 MiB (46×) |
10.1 MiB |
– |
– |
– |
1980–1980 |
100,000 |
Warp per-thread kernel |
237 µs (235 µs–238 µs) |
– |
1.29 ms |
9.15e10 |
4.03e12 (12 %) |
2.70e10 (0.81 %) |
30 MiB (8.74×) |
6.87 MiB |
– |
– |
– |
1980–1980 |
100,000 |
CPU OpenMP |
19.8 ms (19.6 ms–20.1 ms) |
– |
20.3 ms |
1.09e9 |
4.81e10 (–) |
1.97e11 (–) |
– |
10.1 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
2.13 ms (2.11 ms–2.14 ms) |
– |
6.75 ms |
1.02e11 |
4.47e12 (13 %) |
3.14e12 (94 %) |
42 MiB (1.22×) |
62.8 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eagle graph + compaction |
16 ms (16 ms–16 ms) |
– |
20.6 ms |
1.35e10 |
5.96e11 (1.8 %) |
1.02e12 (30 %) |
58 MiB (1.69×) |
62.9 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eagle graph + compaction + reorder |
10.7 ms (10.6 ms–10.7 ms) |
– |
15.3 ms |
2.03e10 |
8.93e11 (2.7 %) |
1.58e12 (47 %) |
86 MiB (2.5×) |
63.1 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eagle graph, K fused steps per launch |
2.38 ms (2.38 ms–2.38 ms) |
– |
6.99 ms |
9.09e10 |
4.00e12 (12 %) |
1.04e12 (31 %) |
42 MiB (1.22×) |
62.8 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
1.91 ms (1.9 ms–1.91 ms) |
– |
6.46 ms |
1.14e11 |
4.99e12 (15 %) |
7.61e11 (23 %) |
58 MiB (1.69×) |
62.8 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
601 µs (601 µs–601 µs) |
– |
5.24 ms |
3.60e11 |
1.58e13 (47 %) |
1.01e13 (3e+02 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
hawk + eagle persistent |
645 µs (644 µs–645 µs) |
– |
5.23 ms |
3.35e11 |
1.48e13 (44 %) |
9.44e12 (2.8e+02 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
0.7510729948982827 |
– |
1980–1980 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
613 µs (612 µs–614 µs) |
– |
5.34 ms |
3.53e11 |
1.55e13 (46 %) |
9.94e12 (3e+02 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eager launch loop |
32.3 ms (32 ms–32.7 ms) |
– |
36.9 ms |
6.70e9 |
2.95e11 (0.88 %) |
1.21e12 (36 %) |
40 MiB (1.17×) |
58.2 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
CuPy masked |
368 ms (367 ms–369 ms) |
– |
375 ms |
5.88e8 |
2.59e10 (0.077 %) |
2.45e12 (73 %) |
192 MiB (5.59×) |
68.8 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
PyTorch masked |
336 ms (334 ms–337 ms) |
– |
350 ms |
6.45e8 |
2.84e10 (0.085 %) |
2.69e12 (80 %) |
350 MiB (10.2×) |
68.7 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
272 ms (272 ms–272 ms) |
– |
286 ms |
7.94e8 |
3.50e10 (0.1 %) |
3.35e12 (1e+02 %) |
422 MiB (12.3×) |
68.7 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
PyTorch torch.compile (reduce-overhead) |
84.5 ms (84.1 ms–84.9 ms) |
– |
97.5 ms |
2.56e9 |
1.13e11 (0.34 %) |
1.04e12 (31 %) |
242 MiB (7.05×) |
76.9 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
JAX jit + vmap(while_loop) |
47 ms (46.8 ms–47 ms) |
– |
56 ms |
4.61e9 |
2.03e11 (0.61 %) |
1.87e12 (56 %) |
158 MiB (4.6×) |
101 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
Warp per-thread kernel |
1.54 ms (1.54 ms–1.55 ms) |
– |
17.2 ms |
1.40e11 |
6.17e12 (18 %) |
4.15e10 (1.2 %) |
62 MiB (1.81×) |
68.7 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
CPU OpenMP |
199 ms (198 ms–200 ms) |
– |
208 ms |
1.09e9 |
4.79e10 (–) |
1.97e11 (–) |
– |
97.6 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (94.9 µs), then eagle.simulate (eagle picks the launch mode) (103 µs, 1.08× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (106 µs), then Warp per-thread kernel (116 µs, 1.1× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (197 µs), then eagle.simulate (eagle picks the launch mode) (199 µs, 1.01× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (601 µs), then eagle.simulate (eagle picks the launch mode) (613 µs, 1.02× the time)
Uniform stop step: every sample runs 1000 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
483 µs (474 µs–491 µs) |
– |
593 µs |
2.07e9 |
9.11e10 (0.27 %) |
5.93e10 (1.8 %) |
2 MiB (58.3×) |
2.04 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eagle graph + compaction |
3.03 ms (3.02 ms–3.03 ms) |
– |
3.17 ms |
3.30e8 |
1.45e10 (0.043 %) |
2.16e10 (0.64 %) |
2 MiB (58.3×) |
2.07 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eagle graph + compaction + reorder |
3.05 ms (3.04 ms–3.05 ms) |
– |
3.2 ms |
3.28e8 |
1.44e10 (0.043 %) |
2.14e10 (0.64 %) |
2 MiB (58.3×) |
2.15 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eagle graph, K fused steps per launch |
568 µs (567 µs–571 µs) |
– |
667 µs |
1.76e9 |
7.74e10 (0.23 %) |
6.76e9 (0.2 %) |
2 MiB (58.3×) |
2.03 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eagle graph, K fused steps per launch + compaction |
480 µs (479 µs–483 µs) |
– |
610 µs |
2.08e9 |
9.16e10 (0.27 %) |
8.66e9 (0.26 %) |
2 MiB (58.3×) |
2.04 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eagle auto (eagle picks the launch mode) |
99.8 µs (98.3 µs–102 µs) |
– |
223 µs |
1.00e10 |
4.41e11 (1.3 %) |
2.81e11 (8.4 %) |
0 MiB (0×) |
2.05 MiB |
– |
– |
– |
1980–1980 |
1,000 |
hawk + eagle persistent |
153 µs (150 µs–156 µs) |
– |
244 µs |
6.56e9 |
2.88e11 (0.86 %) |
1.84e11 (5.5 %) |
0 MiB (0×) |
2.05 MiB |
– |
0.95367431640625 |
– |
1980–1980 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
101 µs (100 µs–103 µs) |
– |
215 µs |
9.85e9 |
4.34e11 (1.3 %) |
2.76e11 (8.2 %) |
0 MiB (0×) |
2.05 MiB |
– |
– |
– |
1980–1980 |
1,000 |
eager launch loop |
20.2 ms (20 ms–20.3 ms) |
– |
20.2 ms |
4.96e7 |
2.18e9 (0.0065 %) |
3.03e9 (0.09 %) |
0 MiB (0×) |
0.0352 MiB |
– |
– |
– |
1980–1980 |
1,000 |
CuPy masked |
358 ms (357 ms–360 ms) |
– |
358 ms |
2.79e6 |
1.23e8 (0.00037 %) |
2.52e9 (0.075 %) |
0 MiB (0×) |
5.01 MiB |
– |
– |
– |
1980–1980 |
1,000 |
PyTorch masked |
242 ms (241 ms–243 ms) |
– |
242 ms |
4.13e6 |
1.82e8 (0.00054 %) |
3.72e9 (0.11 %) |
90 MiB (2.62e+03×) |
0.0703 MiB |
– |
– |
– |
1980–1980 |
1,000 |
PyTorch, CUDA graph of 16 steps |
70.2 ms (70.2 ms–70.2 ms) |
– |
70.3 ms |
1.42e7 |
6.27e8 (0.0019 %) |
1.30e10 (0.39 %) |
162 MiB (4.72e+03×) |
0.16 MiB |
– |
– |
– |
1980–1980 |
1,000 |
PyTorch torch.compile (reduce-overhead) |
65.9 ms (65.4 ms–66.8 ms) |
– |
66.2 ms |
1.52e7 |
6.67e8 (0.002 %) |
1.33e9 (0.04 %) |
162 MiB (4.72e+03×) |
32.1 MiB |
– |
– |
– |
1980–1980 |
1,000 |
JAX jit + vmap(while_loop) |
22.3 ms (22.2 ms–22.4 ms) |
– |
24.1 ms |
4.48e7 |
1.97e9 (0.0059 %) |
3.94e9 (0.12 %) |
158 MiB (4.6e+03×) |
0.0117 MiB |
– |
– |
– |
1980–1980 |
1,000 |
Warp per-thread kernel |
118 µs (116 µs–119 µs) |
– |
313 µs |
8.49e9 |
3.74e11 (1.1 %) |
5.44e8 (0.016 %) |
30 MiB (874×) |
0.0156 MiB |
– |
– |
– |
1980–1980 |
1,000 |
CPU OpenMP |
369 µs (368 µs–371 µs) |
– |
383 µs |
2.71e9 |
1.19e11 (–) |
1.65e11 (–) |
– |
0.367 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
509 µs (508 µs–510 µs) |
– |
672 µs |
1.97e10 |
8.65e11 (2.6 %) |
5.63e11 (17 %) |
2 MiB (5.83×) |
2.43 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eagle graph + compaction |
3.02 ms (3.02 ms–3.02 ms) |
– |
3.22 ms |
3.31e9 |
1.46e11 (0.44 %) |
2.17e11 (6.5 %) |
2 MiB (5.83×) |
2.46 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eagle graph + compaction + reorder |
3.29 ms (3.29 ms–3.3 ms) |
– |
3.51 ms |
3.04e9 |
1.34e11 (0.4 %) |
1.98e11 (5.9 %) |
2 MiB (5.83×) |
2.99 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eagle graph, K fused steps per launch |
548 µs (546 µs–552 µs) |
– |
707 µs |
1.82e10 |
8.02e11 (2.4 %) |
7.01e10 (2.1 %) |
2 MiB (5.83×) |
2.43 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eagle graph, K fused steps per launch + compaction |
480 µs (473 µs–485 µs) |
– |
668 µs |
2.08e10 |
9.17e11 (2.7 %) |
8.68e10 (2.6 %) |
2 MiB (5.83×) |
2.44 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eagle auto (eagle picks the launch mode) |
111 µs (106 µs–116 µs) |
– |
295 µs |
8.99e10 |
3.96e12 (12 %) |
2.52e12 (75 %) |
0 MiB (0×) |
2.62 MiB |
– |
– |
– |
1980–1980 |
10,000 |
hawk + eagle persistent |
151 µs (150 µs–153 µs) |
– |
299 µs |
6.60e10 |
2.91e12 (8.7 %) |
1.85e12 (55 %) |
0 MiB (0×) |
2.57 MiB |
– |
0.9750024960063898 |
– |
1980–1980 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
112 µs (110 µs–114 µs) |
– |
291 µs |
8.91e10 |
3.92e12 (12 %) |
2.50e12 (75 %) |
0 MiB (0×) |
2.62 MiB |
– |
– |
– |
1980–1980 |
10,000 |
eager launch loop |
19.7 ms (19.5 ms–20 ms) |
– |
19.9 ms |
5.07e8 |
2.23e10 (0.067 %) |
3.09e10 (0.92 %) |
0 MiB (0×) |
0.414 MiB |
– |
– |
– |
1980–1980 |
10,000 |
CuPy masked |
350 ms (349 ms–350 ms) |
– |
350 ms |
2.86e7 |
1.26e9 (0.0038 %) |
2.57e10 (0.77 %) |
0 MiB (0×) |
5.28 MiB |
– |
– |
– |
1980–1980 |
10,000 |
PyTorch masked |
249 ms (248 ms–250 ms) |
– |
249 ms |
4.02e7 |
1.77e9 (0.0053 %) |
3.62e10 (1.1 %) |
90 MiB (262×) |
0.621 MiB |
– |
– |
– |
1980–1980 |
10,000 |
PyTorch, CUDA graph of 16 steps |
70.7 ms (70.7 ms–70.7 ms) |
– |
70.9 ms |
1.41e8 |
6.22e9 (0.019 %) |
1.29e11 (3.8 %) |
162 MiB (472×) |
0.762 MiB |
– |
– |
– |
1980–1980 |
10,000 |
PyTorch torch.compile (reduce-overhead) |
65.2 ms (64.9 ms–65.4 ms) |
– |
65.4 ms |
1.53e8 |
6.75e9 (0.02 %) |
1.35e10 (0.4 %) |
162 MiB (472×) |
32.8 MiB |
– |
– |
– |
1980–1980 |
10,000 |
JAX jit + vmap(while_loop) |
24.9 ms (24.7 ms–25.1 ms) |
– |
26.8 ms |
4.02e8 |
1.77e10 (0.053 %) |
3.54e10 (1.1 %) |
158 MiB (460×) |
0.664 MiB |
– |
– |
– |
1980–1980 |
10,000 |
Warp per-thread kernel |
121 µs (121 µs–123 µs) |
– |
400 µs |
8.23e10 |
3.62e12 (11 %) |
5.27e9 (0.16 %) |
30 MiB (87.4×) |
0.688 MiB |
– |
– |
– |
1980–1980 |
10,000 |
CPU OpenMP |
2.03 ms (2.01 ms–2.04 ms) |
– |
2.07 ms |
4.94e9 |
2.17e11 (–) |
3.01e11 (–) |
– |
1.23 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
710 µs (699 µs–722 µs) |
– |
1.38 ms |
1.41e11 |
6.20e12 (19 %) |
4.03e12 (1.2e+02 %) |
6 MiB (1.75×) |
7.66 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eagle graph + compaction |
3.75 ms (3.75 ms–3.75 ms) |
– |
4.48 ms |
2.67e10 |
1.17e12 (3.5 %) |
1.74e12 (52 %) |
6 MiB (1.75×) |
7.71 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eagle graph + compaction + reorder |
4.16 ms (4.15 ms–4.17 ms) |
– |
4.95 ms |
2.40e10 |
1.06e12 (3.2 %) |
1.57e12 (47 %) |
10 MiB (2.91×) |
8.36 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eagle graph, K fused steps per launch |
795 µs (792 µs–798 µs) |
– |
1.48 ms |
1.26e11 |
5.53e12 (17 %) |
4.83e11 (14 %) |
6 MiB (1.75×) |
7.66 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eagle graph, K fused steps per launch + compaction |
685 µs (682 µs–688 µs) |
– |
1.41 ms |
1.46e11 |
6.43e12 (19 %) |
6.08e11 (18 %) |
6 MiB (1.75×) |
7.66 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eagle auto (eagle picks the launch mode) |
252 µs (251 µs–254 µs) |
– |
966 µs |
3.97e11 |
1.75e13 (52 %) |
1.11e13 (3.3e+02 %) |
4 MiB (1.17×) |
7.88 MiB |
– |
– |
– |
1980–1980 |
100,000 |
hawk + eagle persistent |
367 µs (366 µs–369 µs) |
– |
1.04 ms |
2.72e11 |
1.20e13 (36 %) |
7.63e12 (2.3e+02 %) |
4 MiB (1.17×) |
7.83 MiB |
– |
0.9765625 |
– |
1980–1980 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
260 µs (258 µs–261 µs) |
– |
993 µs |
3.85e11 |
1.70e13 (51 %) |
1.08e13 (3.2e+02 %) |
4 MiB (1.17×) |
7.84 MiB |
– |
– |
– |
1980–1980 |
100,000 |
eager launch loop |
19.8 ms (19.5 ms–20.1 ms) |
– |
20.4 ms |
5.06e9 |
2.23e11 (0.67 %) |
3.09e11 (9.2 %) |
4 MiB (1.17×) |
5.76 MiB |
– |
– |
– |
1980–1980 |
100,000 |
CuPy masked |
368 ms (367 ms–369 ms) |
– |
369 ms |
2.72e8 |
1.20e10 (0.036 %) |
2.45e11 (7.3 %) |
22 MiB (6.41×) |
9.29 MiB |
– |
– |
– |
1980–1980 |
100,000 |
PyTorch masked |
265 ms (264 ms–265 ms) |
– |
266 ms |
3.78e8 |
1.66e10 (0.05 %) |
3.41e11 (10 %) |
114 MiB (33.2×) |
6.24 MiB |
– |
– |
– |
1980–1980 |
100,000 |
PyTorch, CUDA graph of 16 steps |
75.4 ms (75.4 ms–75.4 ms) |
– |
76.3 ms |
1.33e9 |
5.84e10 (0.17 %) |
1.21e12 (36 %) |
184 MiB (53.6×) |
6.32 MiB |
– |
– |
– |
1980–1980 |
100,000 |
PyTorch torch.compile (reduce-overhead) |
66.1 ms (65.5 ms–66.5 ms) |
– |
67.1 ms |
1.51e9 |
6.66e10 (0.2 %) |
1.33e11 (4 %) |
168 MiB (48.9×) |
36.5 MiB |
– |
– |
– |
1980–1980 |
100,000 |
JAX jit + vmap(while_loop) |
24.9 ms (24.7 ms–25.1 ms) |
– |
27.1 ms |
4.02e9 |
1.77e11 (0.53 %) |
3.54e11 (11 %) |
158 MiB (46×) |
10.1 MiB |
– |
– |
– |
1980–1980 |
100,000 |
Warp per-thread kernel |
245 µs (244 µs–245 µs) |
– |
1.3 ms |
4.08e11 |
1.80e13 (54 %) |
2.61e10 (0.78 %) |
30 MiB (8.74×) |
6.87 MiB |
– |
– |
– |
1980–1980 |
100,000 |
CPU OpenMP |
16.5 ms (16.4 ms–16.6 ms) |
– |
17 ms |
6.04e9 |
2.66e11 (–) |
3.69e11 (–) |
– |
10 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
2.32 ms (2.31 ms–2.33 ms) |
– |
6.97 ms |
4.30e11 |
1.89e13 (57 %) |
1.23e13 (3.7e+02 %) |
42 MiB (1.22×) |
60.1 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eagle graph + compaction |
22.9 ms (22.9 ms–22.9 ms) |
– |
28 ms |
4.37e10 |
1.92e12 (5.7 %) |
2.85e12 (85 %) |
58 MiB (1.69×) |
60.1 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eagle graph + compaction + reorder |
23.3 ms (23.3 ms–23.3 ms) |
– |
28.5 ms |
4.30e10 |
1.89e12 (5.7 %) |
2.81e12 (84 %) |
86 MiB (2.5×) |
60.6 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eagle graph, K fused steps per launch |
2.6 ms (2.6 ms–2.61 ms) |
– |
7.35 ms |
3.84e11 |
1.69e13 (51 %) |
1.48e12 (44 %) |
42 MiB (1.22×) |
60.1 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
2.55 ms (2.55 ms–2.56 ms) |
– |
7.23 ms |
3.92e11 |
1.72e13 (51 %) |
1.63e12 (49 %) |
58 MiB (1.69×) |
60.1 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
1.74 ms (1.74 ms–1.75 ms) |
– |
6.58 ms |
5.74e11 |
2.53e13 (75 %) |
1.61e13 (4.8e+02 %) |
40 MiB (1.17×) |
60.3 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
hawk + eagle persistent |
1.79 ms (1.79 ms–1.79 ms) |
– |
6.42 ms |
5.59e11 |
2.46e13 (73 %) |
1.57e13 (4.7e+02 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
0.9765625 |
– |
1980–1980 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
1.75 ms (1.75 ms–1.76 ms) |
– |
6.53 ms |
5.71e11 |
2.51e13 (75 %) |
1.60e13 (4.8e+02 %) |
40 MiB (1.17×) |
60.3 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
eager launch loop |
39.1 ms (39 ms–39.2 ms) |
– |
44.5 ms |
2.56e10 |
1.13e12 (3.4 %) |
1.56e12 (47 %) |
40 MiB (1.17×) |
58.2 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
CuPy masked |
365 ms (364 ms–366 ms) |
– |
372 ms |
2.74e9 |
1.21e11 (0.36 %) |
2.47e12 (74 %) |
192 MiB (5.59×) |
64.4 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
PyTorch masked |
337 ms (335 ms–338 ms) |
– |
351 ms |
2.97e9 |
1.31e11 (0.39 %) |
2.68e12 (80 %) |
350 MiB (10.2×) |
62.1 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
272 ms (272 ms–272 ms) |
– |
286 ms |
3.68e9 |
1.62e11 (0.48 %) |
3.36e12 (1e+02 %) |
422 MiB (12.3×) |
62.6 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
PyTorch torch.compile (reduce-overhead) |
83.6 ms (83.4 ms–83.7 ms) |
– |
97.1 ms |
1.20e10 |
5.27e11 (1.6 %) |
1.05e12 (31 %) |
242 MiB (7.05×) |
70.7 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
JAX jit + vmap(while_loop) |
48.1 ms (47.8 ms–48.3 ms) |
– |
57.9 ms |
2.08e10 |
9.15e11 (2.7 %) |
1.83e12 (55 %) |
158 MiB (4.6×) |
95 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
Warp per-thread kernel |
1.71 ms (1.71 ms–1.71 ms) |
– |
19.8 ms |
5.85e11 |
2.57e13 (77 %) |
3.74e10 (1.1 %) |
62 MiB (1.81×) |
68.7 MiB |
– |
– |
– |
1980–1980 |
1,000,000 |
CPU OpenMP |
170 ms (169 ms–170 ms) |
– |
178 ms |
5.90e9 |
2.59e11 (–) |
3.60e11 (–) |
– |
97.6 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (99.8 µs), then eagle.simulate (eagle picks the launch mode) (101 µs, 1.02× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (111 µs), then eagle.simulate (eagle picks the launch mode) (112 µs, 1.01× the time)
N = 100,000: Warp per-thread kernel (245 µs), then eagle auto (eagle picks the launch mode) (252 µs, 1.03× the time)
N = 1,000,000: Warp per-thread kernel (1.71 ms), then eagle auto (eagle picks the launch mode) (1.74 ms, 1.02× the time)
Compaction cadence sweep (isolates the active-set map)
the active-set map alone. For a given K, matched_cadence (the plain per-sample step, no map) and compact (eagle_graph_compact’s active-set step) check the loop’s stop guard at the SAME cadence, once every K steps; the only structural difference between the two rows at a given K is whether the step kernel reads the active-set map. The main table’s eagle_graph row checks its guard every step, so its difference from eagle_graph_compact there mixes the cadence change with the map – this sweep does not.
Config: Spread stop steps (log-uniform in [10, 1000]), up to 1000 steps.
N |
K (steps between guard checks) |
matched cadence, no map (range) |
eagle graph + compaction (range) |
ratio (matched/compact, |
|---|---|---|---|---|
100,000 |
8 |
2.96 ms (2.96 ms–2.97 ms) |
6.69 ms (6.69 ms–6.7 ms) |
0.443× |
100,000 |
16 |
2.76 ms (2.76 ms–2.76 ms) |
5.08 ms (5.08 ms–5.09 ms) |
0.543× |
100,000 |
32 |
2.63 ms (2.63 ms–2.63 ms) |
4.26 ms (4.26 ms–4.27 ms) |
0.617× |
1,000,000 |
8 |
12.7 ms (12.7 ms–12.7 ms) |
18.6 ms (18.5 ms–18.6 ms) |
0.687× |
1,000,000 |
16 |
12.6 ms (12.6 ms–12.6 ms) |
16 ms (16 ms–16 ms) |
0.788× |
1,000,000 |
32 |
12.4 ms (12.4 ms–12.4 ms) |
14.6 ms (14.6 ms–14.6 ms) |
0.85× |
FP32: eagle vs Warp (float32, S = 1000)
Every arm runs the same oscillator in float32: eagle.simulate with scalar_type=”float32”, NVIDIA Warp’s per-thread kernel and JAX’s vmapped loop, all over the same float32 inputs. Each cell runs 2 untimed runs per arm first (eagle’s automatic arm picks its entry over them: the size rule, then a measured run of the other entry, keeping the faster; the entry it settled on is in the last column), then each arm in its own block (no interleaving): an untimed ramp of back-to-back runs for 0.5 s, then 5 timed runs. Before any timing counts, eagle and JAX reach step counts identical to Warp’s and states within absolute 0.0001 of Warp’s (the largest differences are in the JSON). Each time is the mean of the middle 3 of the 5 runs (highest and lowest dropped), with the range of those 3; eagle / Warp is the ratio of these means: below 1 eagle is faster. Before each arm’s block the GPU is cooled down: the block starts once the temperature is within 3 °C of the idle 30 °C and the GPU is idle, or after 20 s. Right after each timed run the SM clock is read (its range is shown per arm); cycles are the wall time times that clock, and eagle / Warp (cycles) is their ratio.
Every arm runs each sample to its own stop and no further than max_steps (S): eagle and Warp enforce the cap per sample, JAX in its while-loop condition, PyTorch and CuPy by the outer step count (a batch-level cap); an arm that runs whole blocks of steps per launch or replay stops at the first block boundary at or after it.
What each arm needed to get there: Warp, a hand-written step counter and cap inside the kernel; JAX, a cap in the while_loop condition; eagle, the kernel source unchanged, with its loop shape, unrolling and device entry chosen by hawk + eagle (per device, per batch).
Cap fixture (checked before any timing counts): N = 4,096, 8 samples with stop step 2S = 256, S = 128; every arm (3) reported k = min(stop step, S) per sample and the same finished count (4,088): passed.
distribution |
N |
eagle.simulate (range) |
Warp (range) |
JAX (range) |
eagle / Warp |
eagle / Warp (cycles) |
eagle entry |
|---|---|---|---|---|---|---|---|
uniform |
10,000 |
98.7 µs (96.5 µs–100 µs), 1980–1980 MHz |
102 µs (100 µs–105 µs), 1980–1980 MHz |
24.9 ms (24.7 ms–25 ms), 1980–1980 MHz |
0.97× |
0.97× |
fused_one |
uniform |
100,000 |
181 µs (179 µs–184 µs), 1980–1980 MHz |
172 µs (172 µs–172 µs), 1980–1980 MHz |
24.3 ms (24.1 ms–24.4 ms), 1980–1980 MHz |
1.05× |
1.05× |
fused_one |
uniform |
1,000,000 |
1.03 ms (1.03 ms–1.04 ms), 1980–1980 MHz |
1.02 ms (1.02 ms–1.02 ms), 1980–1980 MHz |
24.8 ms (24.7 ms–24.9 ms), 1980–1980 MHz |
1.02× |
1.02× |
fused_one |
spread |
10,000 |
91 µs (89.9 µs–92.2 µs), 1980–1980 MHz |
93.6 µs (92.7 µs–94.2 µs), 1980–1980 MHz |
24.7 ms (24.5 ms–24.8 ms), 1980–1980 MHz |
0.971× |
0.971× |
fused_one |
spread |
100,000 |
170 µs (168 µs–172 µs), 1980–1980 MHz |
173 µs (171 µs–176 µs), 1980–1980 MHz |
24.2 ms (24.1 ms–24.3 ms), 1980–1980 MHz |
0.983× |
0.983× |
fused_one |
spread |
1,000,000 |
469 µs (466 µs–473 µs), 1980–1980 MHz |
923 µs (922 µs–924 µs), 1980–1980 MHz |
25.3 ms (25.2 ms–25.3 ms), 1980–1980 MHz |
0.509× |
0.509× |
persist |
Fixed overhead per run (N = 64, S = 100)
item 3: wall at a cell small enough that per-run overhead dominates (N=64, S=100), each arm’s own kernel-only vs wall where the main nsys pass covers this cell’s N (it does not by default, so kernel_only_s is usually null here)
arm |
wall (range) |
kernel-only |
|---|---|---|
eagle graph (device loop) |
321 µs (315 µs–327 µs) |
– |
eagle graph + compaction |
461 µs (454 µs–473 µs) |
– |
eagle graph + compaction + reorder |
527 µs (525 µs–530 µs) |
– |
eagle graph, K fused steps per launch |
171 µs (165 µs–179 µs) |
– |
eagle graph, K fused steps per launch + compaction |
199 µs (193 µs–208 µs) |
– |
eagle auto (eagle picks the launch mode) |
53.1 µs (52.4 µs–54.2 µs) |
– |
hawk + eagle persistent |
87 µs (85.3 µs–88.3 µs) |
– |
eagle.simulate (eagle picks the launch mode) |
56.4 µs (54.6 µs–57.7 µs) |
– |
eager launch loop |
2.01 ms (1.97 ms–2.04 ms) |
– |
CuPy masked |
34.6 ms (34.1 ms–34.9 ms) |
– |
PyTorch masked |
24.4 ms (24.3 ms–24.5 ms) |
– |
PyTorch, CUDA graph of 16 steps |
6.17 ms (6.16 ms–6.18 ms) |
– |
PyTorch torch.compile (reduce-overhead) |
6.74 ms (6.67 ms–6.82 ms) |
– |
JAX jit + vmap(while_loop) |
2.37 ms (2.36 ms–2.38 ms) |
– |
Warp per-thread kernel |
67.5 µs (66.8 µs–68.4 µs) |
– |
CPU OpenMP |
160 µs (159 µs–163 µs) |
– |
Where each tool fits
Each tool below is written the way its users write it, and each brings something of its own. Times are walls (mean of the middle 3 of 5 runs) at the largest N; ratios are against the eagle graph arm (below 1 = less time). The fastest arm per N is listed under each table above.
eagle graph (device loop). The step written once as a hawk kernel; the whole loop runs on the device as one CUDA graph with a device-side stop guard, so a run is one launch with no host round trip per step. On this card: N = 1,000,000, spread S=100: 573 µs; spread S=1000: 2.13 ms; uniform S=1000: 2.32 ms. Fastest in 0 of 12 cells.
eagle graph + compaction. Adds an active-set map: the launch covers only the samples still running, rebuilt every 16 steps; it is made for batches that thin out (the spread configurations). On this card: N = 1,000,000, spread S=100: 2.02 ms (3.53× the eagle graph time); spread S=1000: 16 ms (7.5× the eagle graph time); uniform S=1000: 22.9 ms (9.85× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph + compaction + reorder. Adds a physical reorder when the live samples spread thin over their warps, so the mapped reads stay contiguous; the final restore to sample order is inside its wall. On this card: N = 1,000,000, spread S=100: 2.15 ms (3.75× the eagle graph time); spread S=1000: 10.7 ms (5.01× the eagle graph time); uniform S=1000: 23.3 ms (10× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph, K fused steps per launch. The same step with one decorator argument, @hawk.kernel(steps=16): each launch runs that many steps per sample with the state kept in registers, as Warp’s per-thread loop does, and the launches, guard checks and plane reads shrink by the same factor; its results are bit-identical to the eagle graph arm’s in every cell. K is chosen by hand here; the eagle_graph_auto arm picks it at run time. On this card: N = 1,000,000, spread S=100: 419 µs (0.731× the eagle graph time); spread S=1000: 2.38 ms (1.12× the eagle graph time); uniform S=1000: 2.6 ms (1.12× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph, K fused steps per launch + compaction. The K-steps kernel over the active-set map: the register-resident steps of the arm above, and the launches cover only the samples still running, so it fits batches that thin out. On this card: N = 1,000,000, spread S=100: 457 µs (0.798× the eagle graph time); spread S=1000: 1.91 ms (0.896× the eagle graph time); uniform S=1000: 2.55 ms (1.1× the eagle graph time). Fastest in 0 of 12 cells.
eagle auto (eagle picks the launch mode). The same step with @hawk.kernel(steps=”auto”) on the active-set kind, and nothing to tune: it picks the steps per launch itself after every launch, on the device, so a batch whose samples all run to the end gets Warp-like fusion of up to 64 steps per launch, and a batch that thins out gets shorter launches with the active set compacted after each one; no sample takes more steps than the cap. On this card: N = 1,000,000, spread S=100: 163 µs (0.285× the eagle graph time); spread S=1000: 601 µs (0.282× the eagle graph time); uniform S=1000: 1.74 ms (0.75× the eagle graph time). Fastest in 10 of 12 cells.
hawk + eagle persistent. The plain (not active-set) automatic kernel, forced through hawk’s persist entry: one launch, no graph, no map, no policy kernel, no cap at 64 – a grid of SMs blocks of 256 lanes each steal the next unfinished sample off a counter when their own finishes, so idle warps refill without ever rebuilding an index map; made for batches too large for one resident wave. On this card: N = 1,000,000, spread S=100: 246 µs (0.429× the eagle graph time); spread S=1000: 645 µs (0.303× the eagle graph time); uniform S=1000: 1.79 ms (0.77× the eagle graph time). Fastest in 0 of 12 cells.
eagle.simulate (eagle picks the launch mode). The same automatic kernel through eagle.simulate: the model, its state and its parameters by name, with no plan or plane binding to write; eagle deploys the kernel and builds the same device loop. On this card: N = 1,000,000, spread S=100: 169 µs (0.295× the eagle graph time); spread S=1000: 613 µs (0.288× the eagle graph time); uniform S=1000: 1.75 ms (0.754× the eagle graph time). Fastest in 0 of 12 cells.
eager launch loop. The same kernels launched from Python one step at a time; the host sees the state after every step. On this card: N = 1,000,000, spread S=100: 3.52 ms (6.14× the eagle graph time); spread S=1000: 32.3 ms (15.2× the eagle graph time); uniform S=1000: 39.1 ms (16.8× the eagle graph time). Fastest in 0 of 12 cells.
CuPy masked. NumPy-like array code on GPUs with no kernel to write; every sample is computed every step, so it fits dense batches where all samples run to the end (the uniform configuration). On this card: N = 1,000,000, spread S=100: 36.4 ms (63.5× the eagle graph time); spread S=1000: 368 ms (173× the eagle graph time); uniform S=1000: 365 ms (157× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch masked. The same array code in PyTorch tensors; it fits where the computation sits next to a PyTorch model or needs autograd. On this card: N = 1,000,000, spread S=100: 33.7 ms (58.9× the eagle graph time); spread S=1000: 336 ms (158× the eagle graph time); uniform S=1000: 337 ms (145× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch, CUDA graph of 16 steps. The same PyTorch code with its launch overhead removed by hand: 16 steps captured as one CUDA graph and replayed, one host check per replay, the masking unchanged. On this card: N = 1,000,000, spread S=100: 30.2 ms (52.7× the eagle graph time); spread S=1000: 272 ms (128× the eagle graph time); uniform S=1000: 272 ms (117× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch torch.compile (reduce-overhead). The PyTorch step under torch.compile(mode=”reduce-overhead”): Inductor-fused kernels replayed as a CUDA graph. On this card: N = 1,000,000, spread S=100: 8.69 ms (15.2× the eagle graph time); spread S=1000: 84.5 ms (39.7× the eagle graph time); uniform S=1000: 83.6 ms (36× the eagle graph time). Fastest in 0 of 12 cells.
JAX jit + vmap(while_loop). The whole loop compiled once, a per-sample while loop batched with vmap; the same code runs on CPUs and GPUs and differentiates with jax.grad. On this card: N = 1,000,000, spread S=100: 4.95 ms (8.64× the eagle graph time); spread S=1000: 47 ms (22.1× the eagle graph time); uniform S=1000: 48.1 ms (20.7× the eagle graph time). Fastest in 0 of 12 cells.
Warp per-thread kernel. Explicit per-thread kernels written in Python, maintained by NVIDIA and differentiable through wp.Tape; each thread runs its own sample and stops at its own step, so finished samples cost nothing. On this card: N = 1,000,000, spread S=100: 234 µs (0.407× the eagle graph time); spread S=1000: 1.54 ms (0.725× the eagle graph time); uniform S=1000: 1.71 ms (0.736× the eagle graph time). Fastest in 2 of 12 cells.
CPU OpenMP. The same hawk kernel on the host through eagle’s OpenMP team, for machines without a GPU. On this card: N = 1,000,000, spread S=100: 21.1 ms (36.7× the eagle graph time); spread S=1000: 199 ms (93.4× the eagle graph time); uniform S=1000: 170 ms (73× the eagle graph time). Fastest in 0 of 12 cells.
Compile time
Its own pass, never part of any wall. Each measurement is a fresh process: the imports and the GPU context start-up run untimed, then the timer spans the first call to a ready kernel (trace + code generation + compile + module load, or the cache lookup). Cold: every cache empty – hawk’s compile cache (HAWK_CACHE_DIR), CuPy’s kernel cache (CUPY_CACHE_DIR), the CUDA driver’s PTX cache (CUDA_CACHE_PATH), PyTorch Inductor’s and Triton’s caches (TORCHINDUCTOR_CACHE_DIR, TRITON_CACHE_DIR) and JAX’s persistent compilation cache (minimum compile time and entry size 0), each a fresh directory. Warm: a new process on the caches the first cold run populated. eagle arms: eagle.deploy of the arm’s hawk kernel, which returns once the device build is done (C++/CUDA emission, nvcc to PTX, driver load; eagle’s own small CuPy kernels compile beside it) while the host compiler builds the host side on another thread, and the arm’s set-up on a 64-sample batch (eagle’s graph built, one step run). CuPy: one masked step on 64 samples (every elementwise and reduction kernel compiled on first use). Warp: the kernel module’s code generation, compile and load on the first launch (64 samples), cached in WARP_CACHE_PATH. PyTorch torch.compile, where it runs: three steps at N (the first call traces and compiles, the second records the CUDA graph, the third replays it), per N (dynamic=False). JAX: lower + compile from abstract shapes, per N (XLA specialises on the shape). CPU OpenMP: the same eagle.deploy as the eagle graph arm and one host run, which waits for the host build. Medians of 3 cold and 3 warm processes.
arm |
compiled |
cold (median of 3) |
warm (median of 3) |
|---|---|---|---|
eagle graph (device loop) |
eagle.deploy of the hawk step kernel (the device build; the host side builds in the background), the device-loop graph |
796 ms |
70.8 ms |
eagle graph + compaction |
eagle.deploy of the hawk active-set step kernel (the device build; the host side builds in the background), the compaction-loop graph |
619 ms |
61.1 ms |
eagle graph + compaction + reorder |
eagle.deploy of the hawk active-set step kernel (the device build; the host side builds in the background), the compaction + reorder loop graph |
690 ms |
61.9 ms |
eagle graph, K fused steps per launch |
eagle.deploy of the hawk K-steps kernel (the device build; the host side builds in the background), the device-loop graph |
633 ms |
68.8 ms |
eagle graph, K fused steps per launch + compaction |
eagle.deploy of the hawk active-set K-steps kernel (the device build; the host side builds in the background), the compaction-loop graph |
631 ms |
68.1 ms |
eagle auto (eagle picks the launch mode) |
eagle.deploy of the hawk active-set automatic kernel (the device build; the host side builds in the background), the policy kernel, the policy-loop graph |
858 ms |
67.9 ms |
hawk + eagle persistent |
eagle.deploy of the hawk automatic kernel (the device build; the host side builds in the background), the persistent launch |
672 ms |
69.7 ms |
eagle.simulate (eagle picks the launch mode) |
eagle.simulate’s deploy of the hawk active-set automatic kernel (the device build; the host side builds in the background), the policy kernel, the policy-loop graph |
875 ms |
70 ms |
eager launch loop |
eagle.deploy of the hawk single step kernel (the device build; the host side builds in the background) |
542 ms |
51.4 ms |
CuPy masked |
CuPy’s elementwise and reduction kernels of one masked step |
693 ms |
22.6 ms |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 1,000 |
3.38 s |
2.08 s |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 10,000 |
3.62 s |
2.11 s |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 100,000 |
3.73 s |
2.11 s |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 1,000,000 |
3.71 s |
2.11 s |
Warp per-thread kernel |
Warp kernel module (code generation, NVRTC compile, load) |
574 ms |
122 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 1,000 |
242 ms |
40 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 10,000 |
247 ms |
41.5 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 100,000 |
263 ms |
41 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 1,000,000 |
272 ms |
41.2 ms |
CPU OpenMP |
eagle.deploy of the hawk step kernel and its first host run, which waits for the host build |
1 s |
64.1 ms |
No compile step: PyTorch masked, PyTorch, CUDA graph of 16 steps (PyTorch’s eager kernels ship prebuilt; the CUDA-graph capture is per N and recorded per configuration as torch_graph_capture_s).
Memory method
A separate pass after the timing pass. Each (arm, N, configuration) runs in a fresh process: imports, the CUDA context created by one tiny device operation, then the baseline read, before the arm builds or allocates anything; then the arm’s kernels built and a warm-up run on a 64-sample batch (every compile happens here; JAX also compiles for shape N from abstract shapes), so every library’s pool reserved during the warm-up counts in the reading; CuPy’s pools freed and torch.cuda.empty_cache() called after the warm-up (Warp’s pool release threshold set to 0); then the batch’s inputs generated, the arm set up for N (eagle builds its graph, PyTorch captures its CUDA graph, whose private memory pool counts) and one full run with upload and download. Device memory = the memory the CUDA driver accounts to the process (NVML per-process used memory) after the run minus at the baseline. The caching allocators of CuPy, PyTorch and JAX are left at their defaults and are not trimmed after the run; each keeps the memory it reserved until trimmed, so the reading after the run is the allocator’s high-water reservation, the same measure for every arm. A disabled pool returns memory between operations, so its peak could only be caught by sampling, which misses short peaks; the held reservation needs no sampling. Caveats: the reading is what the process holds, so each allocator’s rounding and growth policy count (JAX’s allocator grows in regions that can exceed the request; PyTorch’s rounds blocks up; a captured CUDA graph keeps a private pool; Warp’s stream-ordered pool reserves in chunks of tens of MiB); the driver accounts in pages of about 2 MiB, so small-N rows read 0 or one page. The per-library counters (CuPy pool, torch.cuda.max_memory_reserved/allocated, JAX memory_stats) are recorded in the JSON as a cross-check. Host memory = peak resident set size above the host baseline (taken after the warm-up; /proc/self/clear_refs reset there, then VmHWM). Minimum = the state the workload needs, N × ((2 state + 2 float64 scalars: omega, zeta) × 8 + 4 bytes for the stop step as a 32-bit integer) = 36 bytes per sample; the factor is device memory / minimum.
Device memory the process already held at the baseline (CUDA context, loaded kernels and libraries, the warm-up’s leftovers; at N = 1,000,000, first configuration): eagle graph (device loop) 522 MiB; eagle graph + compaction 522 MiB; eagle graph + compaction + reorder 522 MiB; eagle graph, K fused steps per launch 522 MiB; eagle graph, K fused steps per launch + compaction 522 MiB; eagle auto (eagle picks the launch mode) 522 MiB; hawk + eagle persistent 522 MiB; eagle.simulate (eagle picks the launch mode) 522 MiB; eager launch loop 522 MiB; CuPy masked 522 MiB; PyTorch masked 522 MiB; PyTorch, CUDA graph of 16 steps 522 MiB; PyTorch torch.compile (reduce-overhead) 522 MiB; JAX jit + vmap(while_loop) 522 MiB; Warp per-thread kernel 522 MiB.
Run on: driver 580.178.04; CUDA runtime 12.9; host CPU Intel Xeon CPU @ 2.20GHz; compiler x86_64-conda-linux-gnu-g++ (conda-forge gcc 14.4.0-7) 14.4.0; Python 3.12.14; Warp 1.17.0, JAX 0.11.2, PyTorch 2.12.1+cu129, CuPy 14.2.0, NumPy 2.5.3.
N = 1,000,000
Log scale, from 2.95 ms to 254 ms.
N = 1,000,000
Log scale, from 15.8 ms to 2.56 s.
N = 1,000,000
Log scale, from 64.3 ms to 2.56 s.
Table of every arm and every N
Device: NVIDIA L4 (compute capability 8.9, 58 SMs). Peak FP64: 4.73e11 FLOP/s. Peak DRAM bandwidth: 3.00e11 B/s. Mean of the middle 3 of 5 runs of each arm (the highest and the lowest dropped), range of those 3 in parentheses.
Each arm’s block starts once the GPU is within 3 °C of its idle 46 °C and reads idle, or after 20 s; the SM clock is read right after each timed run and its range is in the last column.
The FP64 peak above is at the device’s reported maximum clock (2040 MHz). During the timed repetitions the SM clock read 2040 MHz; at that clock the FP64 peak is 4.73e11 FLOP/s, and every FP64 percentage below is 1× larger against it.
“% of issue peak” (the eagle arms only) counts DP instructions (DFMA/DMUL/DADD/DSETP/MUFU), not useful FLOPs: the step’s SASS has ? of them, against the device’s DP issue-slot throughput at the observed clock – a kernel that is already at the issue pipe’s limit can still read well under 100% of FP64 peak, because useful FLOPs/step is smaller than issued DP instructions/step.
eagle_graph checks its stop guard every step; eagle_graph_compact checks it once every 16 steps (the cadence its compaction runs on) and reads an active-set map built from it. The two rows below differ in both the cadence and the map; the compaction cadence sweep further down holds the cadence fixed at a few values and isolates the map alone.
PyTorch, CUDA graph of 16 steps captures 16 masked steps once as a torch.cuda.CUDAGraph over static buffers and replays it, with one host check of “any sample running” per replay: it removes the per-kernel launch overhead and keeps the masking (every sample is computed every step, finished ones held by torch.where, so the steps a replay runs past the last stop change nothing). JAX’s while loop is one compiled executable with no host check per step; batched by vmap, it runs until the slowest sample stops. Warp runs one thread per sample, each stopping at its own step.
Every arm runs each sample to its own stop and no further than max_steps (S): eagle and Warp enforce the cap per sample, JAX in its while-loop condition, PyTorch and CuPy by the outer step count (a batch-level cap); an arm that runs whole blocks of steps per launch or replay stops at the first block boundary at or after it.
What each arm needed to get there: Warp, a hand-written step counter and cap inside the kernel; JAX, a cap in the while_loop condition; eagle, the kernel source unchanged, with its loop shape, unrolling and device entry chosen by hawk + eagle (per device, per batch).
Cap fixture (checked before any timing counts): N = 4,096, 8 samples with stop step 2S = 256, S = 128; every arm (16) reported k = min(stop step, S) per sample and the same finished count (4,088): passed.
arm |
loop structure |
|---|---|
eagle graph (device loop) |
per launch over the batch, inside one CUDA graph: the plain decorator’s kernel, which finishes its own samples and runs the steps per launch eagle’s policy picks (8 to 64), guard checked on the device every launch |
eagle graph + compaction |
as the graph arm, the step launched over the active-set map only (recomputed every 16 steps) |
eagle graph + compaction + reorder |
as the compaction arm, plus a physical reorder of the per-sample planes when the live samples spread thin |
eagle graph, K fused steps per launch |
per launch over the batch, inside one CUDA graph: the step kernel runs up to K = 16 steps per sample with the state in registers, each sample leaving at its own stop step; guard checked on the device every launch |
eagle graph, K fused steps per launch + compaction |
as the K-steps arm, the launch over the active-set map only (recomputed every 64 steps = 4 launches) |
eagle auto (eagle picks the launch mode) |
the active-set kernel under eagle’s automatic policy, which picks the launch mode from the batch: one launch running every sample to its end for a small batch, the persistent launch above that, and the compaction loop (active-set map, steps per launch picked on the device) only when the caller asks for compaction or reordering; the row records the mode taken |
hawk + eagle persistent |
ONE launch, no graph, no map, no policy kernel: a grid sized from the SM count and the kernel’s occupancy, each lane fetching base + atomicAdd(counter, 1) whenever its sample is done, running the whole budget’s worth of steps per sample it picks up |
eagle.simulate (eagle picks the launch mode) |
as the automatic arm: eagle.simulate takes the kernel, its state and its parameters by name and eagle picks the launch mode the same way |
eager launch loop |
per step over the batch: one kernel launch from Python, a host read of the finished count every step |
CuPy masked |
per step over the batch: whole-batch array expressions, finished samples held by cupy.where, a host check every step; the cap is batch-level (the outer step loop runs at most max_steps) |
PyTorch masked |
per step over the batch: whole-batch tensor expressions, finished samples held by torch.where, a host check every step; the cap is batch-level (the outer step loop runs at most max_steps) |
PyTorch, CUDA graph of 16 steps |
as PyTorch masked, 16 steps captured as one CUDA graph over static buffers, replayed with one host check per replay |
PyTorch torch.compile (reduce-overhead) |
as PyTorch masked, the step compiled by Inductor and replayed as a CUDA graph, a host check every step |
JAX jit + vmap(while_loop) |
per step over the batch, inside one compiled executable: the vmapped while loop runs until the slowest sample stops or the step cap is reached, finished samples held by a select |
Warp per-thread kernel |
per sample: one launch, each thread loops over its own sample’s steps and stops at its own stop step or the step cap, whichever comes first |
CPU OpenMP |
per step over the batch on the host: one host-team launch per step, the kernel marks and counts its finished samples |
Spread stop steps (log-uniform in [1, 100]), up to 100 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
618 µs (614 µs–620 µs) |
– |
781 µs |
3.57e7 |
1.57e9 (0.33 %) |
1.18e9 (0.39 %) |
0 MiB (0×) |
2.14 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eagle graph + compaction |
674 µs (653 µs–693 µs) |
– |
925 µs |
3.27e7 |
1.44e9 (0.3 %) |
2.93e9 (0.98 %) |
0 MiB (0×) |
2.05 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eagle graph + compaction + reorder |
780 µs (778 µs–781 µs) |
– |
1.04 ms |
2.83e7 |
1.24e9 (0.26 %) |
3.07e9 (1 %) |
0 MiB (0×) |
2.1 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eagle graph, K fused steps per launch |
296 µs (294 µs–298 µs) |
– |
459 µs |
7.45e7 |
3.28e9 (0.69 %) |
9.67e8 (0.32 %) |
0 MiB (0×) |
2.03 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eagle graph, K fused steps per launch + compaction |
363 µs (354 µs–371 µs) |
– |
605 µs |
6.07e7 |
2.67e9 (0.56 %) |
7.60e8 (0.25 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eagle auto (eagle picks the launch mode) |
103 µs (102 µs–105 µs) |
– |
315 µs |
2.13e8 |
9.39e9 (2 %) |
6.31e9 (2.1 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
2040–2040 |
1,000 |
hawk + eagle persistent |
165 µs (162 µs–169 µs) |
– |
315 µs |
1.34e8 |
5.88e9 (1.2 %) |
3.95e9 (1.3 %) |
0 MiB (0×) |
2.04 MiB |
– |
0.2098712531887755 |
– |
2040–2040 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
116 µs (112 µs–122 µs) |
– |
332 µs |
1.91e8 |
8.39e9 (1.8 %) |
5.63e9 (1.9 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eager launch loop |
3.24 ms (3.22 ms–3.26 ms) |
– |
3.39 ms |
6.81e6 |
3.00e8 (0.063 %) |
1.21e9 (0.4 %) |
0 MiB (0×) |
0.0312 MiB |
– |
– |
– |
2040–2040 |
1,000 |
CuPy masked |
65.8 ms (65.7 ms–65.9 ms) |
– |
66 ms |
3.35e5 |
1.48e7 (0.0031 %) |
1.37e9 (0.46 %) |
0 MiB (0×) |
4.94 MiB |
– |
– |
– |
2040–2040 |
1,000 |
PyTorch masked |
41.2 ms (41.2 ms–41.3 ms) |
– |
41.6 ms |
5.35e5 |
2.35e7 (0.005 %) |
2.18e9 (0.73 %) |
28 MiB (816×) |
0.0742 MiB |
– |
– |
– |
2040–2040 |
1,000 |
PyTorch, CUDA graph of 16 steps |
7.29 ms (7.28 ms–7.3 ms) |
– |
7.6 ms |
3.02e6 |
1.33e8 (0.028 %) |
1.39e10 (4.6 %) |
56 MiB (1.63e+03×) |
0.164 MiB |
– |
– |
– |
2040–2040 |
1,000 |
PyTorch torch.compile (reduce-overhead) |
13.7 ms (13.6 ms–13.8 ms) |
– |
14 ms |
1.62e6 |
7.11e7 (0.015 %) |
6.44e8 (0.21 %) |
50 MiB (1.46e+03×) |
34.9 MiB |
– |
– |
– |
2040–2040 |
1,000 |
JAX jit + vmap(while_loop) |
2.14 ms (2.12 ms–2.15 ms) |
– |
3.98 ms |
1.03e7 |
4.54e8 (0.096 %) |
4.13e9 (1.4 %) |
134 MiB (3.9e+03×) |
0.0273 MiB |
– |
– |
– |
2040–2040 |
1,000 |
Warp per-thread kernel |
248 µs (247 µs–248 µs) |
– |
608 µs |
8.90e7 |
3.92e9 (0.83 %) |
2.58e8 (0.086 %) |
30 MiB (874×) |
0.0234 MiB |
– |
– |
– |
2040–2040 |
1,000 |
CPU OpenMP |
465 µs (463 µs–469 µs) |
– |
485 µs |
4.74e7 |
2.09e9 (–) |
8.42e9 (–) |
– |
0.195 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
695 µs (683 µs–707 µs) |
– |
1.02 ms |
3.29e8 |
1.45e10 (3.1 %) |
1.08e10 (3.6 %) |
0 MiB (0×) |
2.44 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eagle graph + compaction |
704 µs (699 µs–709 µs) |
– |
1.12 ms |
3.25e8 |
1.43e10 (3 %) |
2.88e10 (9.6 %) |
0 MiB (0×) |
2.45 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eagle graph + compaction + reorder |
1.12 ms (1.1 ms–1.13 ms) |
– |
1.56 ms |
2.05e8 |
9.00e9 (1.9 %) |
2.20e10 (7.3 %) |
0 MiB (0×) |
2.95 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eagle graph, K fused steps per launch |
366 µs (361 µs–376 µs) |
– |
686 µs |
6.24e8 |
2.75e10 (5.8 %) |
7.85e9 (2.6 %) |
0 MiB (0×) |
2.43 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eagle graph, K fused steps per launch + compaction |
413 µs (407 µs–425 µs) |
– |
815 µs |
5.53e8 |
2.44e10 (5.1 %) |
6.76e9 (2.3 %) |
0 MiB (0×) |
2.44 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eagle auto (eagle picks the launch mode) |
197 µs (193 µs–199 µs) |
– |
569 µs |
1.16e9 |
5.11e10 (11 %) |
3.42e10 (11 %) |
0 MiB (0×) |
2.36 MiB |
– |
– |
– |
2040–2040 |
10,000 |
hawk + eagle persistent |
256 µs (252 µs–260 µs) |
– |
557 µs |
8.94e8 |
3.93e10 (8.3 %) |
2.64e10 (8.8 %) |
0 MiB (0×) |
2.3 MiB |
– |
0.21156370013627354 |
– |
2040–2040 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
213 µs (209 µs–220 µs) |
– |
594 µs |
1.07e9 |
4.71e10 (10 %) |
3.16e10 (11 %) |
0 MiB (0×) |
2.34 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eager launch loop |
3.22 ms (3.21 ms–3.23 ms) |
– |
3.52 ms |
7.10e7 |
3.12e9 (0.66 %) |
1.22e10 (4.1 %) |
0 MiB (0×) |
0.121 MiB |
– |
– |
– |
2040–2040 |
10,000 |
CuPy masked |
65.6 ms (65.5 ms–65.6 ms) |
– |
66 ms |
3.49e6 |
1.53e8 (0.032 %) |
1.37e10 (4.6 %) |
0 MiB (0×) |
5.62 MiB |
– |
– |
– |
2040–2040 |
10,000 |
PyTorch masked |
41.3 ms (41.3 ms–41.4 ms) |
– |
41.8 ms |
5.53e6 |
2.43e8 (0.051 %) |
2.18e10 (7.3 %) |
28 MiB (81.6×) |
0.684 MiB |
– |
– |
– |
2040–2040 |
10,000 |
PyTorch, CUDA graph of 16 steps |
7.01 ms (7.01 ms–7.02 ms) |
– |
7.41 ms |
3.26e7 |
1.43e9 (0.3 %) |
1.45e11 (48 %) |
56 MiB (163×) |
0.719 MiB |
– |
– |
– |
2040–2040 |
10,000 |
PyTorch torch.compile (reduce-overhead) |
12.8 ms (12.8 ms–12.8 ms) |
– |
13.2 ms |
1.78e7 |
7.85e8 (0.17 %) |
6.87e9 (2.3 %) |
98 MiB (285×) |
90.8 MiB |
– |
– |
– |
2040–2040 |
10,000 |
JAX jit + vmap(while_loop) |
2.25 ms (2.23 ms–2.27 ms) |
– |
3.99 ms |
1.01e8 |
4.46e9 (0.94 %) |
3.92e10 (13 %) |
134 MiB (390×) |
0.703 MiB |
– |
– |
– |
2040–2040 |
10,000 |
Warp per-thread kernel |
256 µs (256 µs–256 µs) |
– |
775 µs |
8.93e8 |
3.93e10 (8.3 %) |
2.50e9 (0.83 %) |
30 MiB (87.4×) |
0.691 MiB |
– |
– |
– |
2040–2040 |
10,000 |
CPU OpenMP |
991 µs (984 µs–1 ms) |
– |
1.07 ms |
2.31e8 |
1.02e10 (–) |
3.98e10 (–) |
– |
1.1 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
1.66 ms (1.65 ms–1.67 ms) |
– |
3.27 ms |
1.38e9 |
6.07e10 (13 %) |
5.65e10 (19 %) |
4 MiB (1.17×) |
7.86 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eagle graph + compaction |
1.41 ms (1.4 ms–1.41 ms) |
– |
3.1 ms |
1.63e9 |
7.16e10 (15 %) |
1.44e11 (48 %) |
4 MiB (1.17×) |
7.88 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eagle graph + compaction + reorder |
1.95 ms (1.94 ms–1.96 ms) |
– |
3.69 ms |
1.17e9 |
5.16e10 (11 %) |
1.26e11 (42 %) |
8 MiB (2.33×) |
8.5 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eagle graph, K fused steps per launch |
1.28 ms (1.28 ms–1.28 ms) |
– |
2.88 ms |
1.78e9 |
7.85e10 (17 %) |
2.24e10 (7.5 %) |
4 MiB (1.17×) |
7.86 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eagle graph, K fused steps per launch + compaction |
1.22 ms (1.22 ms–1.23 ms) |
– |
2.89 ms |
1.87e9 |
8.22e10 (17 %) |
2.28e10 (7.6 %) |
4 MiB (1.17×) |
7.88 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eagle auto (eagle picks the launch mode) |
439 µs (436 µs–442 µs) |
– |
2.08 ms |
5.22e9 |
2.30e11 (48 %) |
1.54e11 (51 %) |
4 MiB (1.17×) |
7.59 MiB |
– |
– |
– |
2040–2040 |
100,000 |
hawk + eagle persistent |
692 µs (683 µs–696 µs) |
– |
2.26 ms |
3.31e9 |
1.46e11 (31 %) |
9.76e10 (33 %) |
4 MiB (1.17×) |
7.58 MiB |
– |
0.3936514623434334 |
– |
2040–2040 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
459 µs (449 µs–471 µs) |
– |
2.11 ms |
4.99e9 |
2.20e11 (46 %) |
1.47e11 (49 %) |
4 MiB (1.17×) |
7.59 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eager launch loop |
4.13 ms (4.1 ms–4.17 ms) |
– |
5.71 ms |
5.54e8 |
2.44e10 (5.1 %) |
9.54e10 (32 %) |
4 MiB (1.17×) |
5.38 MiB |
– |
– |
– |
2040–2040 |
100,000 |
CuPy masked |
65.7 ms (65.5 ms–65.9 ms) |
– |
67.7 ms |
3.48e7 |
1.53e9 (0.32 %) |
1.37e11 (46 %) |
22 MiB (6.41×) |
9.39 MiB |
– |
– |
– |
2040–2040 |
100,000 |
PyTorch masked |
43 ms (43 ms–43.1 ms) |
– |
45.4 ms |
5.32e7 |
2.34e9 (0.49 %) |
2.09e11 (70 %) |
52 MiB (15.1×) |
6.86 MiB |
– |
– |
– |
2040–2040 |
100,000 |
PyTorch, CUDA graph of 16 steps |
10.5 ms (10.5 ms–10.5 ms) |
– |
12.6 ms |
2.18e8 |
9.60e9 (2 %) |
9.66e11 (3.2e+02 %) |
78 MiB (22.7×) |
6.87 MiB |
– |
– |
– |
2040–2040 |
100,000 |
PyTorch torch.compile (reduce-overhead) |
13.4 ms (13.3 ms–13.4 ms) |
– |
15.5 ms |
1.71e8 |
7.53e9 (1.6 %) |
6.58e10 (22 %) |
108 MiB (31.5×) |
94.7 MiB |
– |
– |
– |
2040–2040 |
100,000 |
JAX jit + vmap(while_loop) |
4.81 ms (4.79 ms–4.86 ms) |
– |
7.81 ms |
4.76e8 |
2.09e10 (4.4 %) |
1.84e11 (61 %) |
134 MiB (39×) |
10.9 MiB |
– |
– |
– |
2040–2040 |
100,000 |
Warp per-thread kernel |
1.22 ms (1.22 ms–1.22 ms) |
– |
3.37 ms |
1.88e9 |
8.27e10 (17 %) |
5.25e9 (1.8 %) |
30 MiB (8.74×) |
6.87 MiB |
– |
– |
– |
2040–2040 |
100,000 |
CPU OpenMP |
3.82 ms (3.8 ms–3.83 ms) |
– |
4.53 ms |
5.99e8 |
2.64e10 (–) |
1.03e11 (–) |
– |
10 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
10.3 ms (10.3 ms–10.4 ms) |
– |
24 ms |
2.22e9 |
9.79e10 (21 %) |
1.05e11 (35 %) |
40 MiB (1.17×) |
62.8 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eagle graph + compaction |
5.45 ms (5.43 ms–5.46 ms) |
– |
19.1 ms |
4.22e9 |
1.86e11 (39 %) |
3.73e11 (1.2e+02 %) |
56 MiB (1.63×) |
62.8 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eagle graph + compaction + reorder |
6.73 ms (6.72 ms–6.74 ms) |
– |
20.4 ms |
3.42e9 |
1.50e11 (32 %) |
3.66e11 (1.2e+02 %) |
84 MiB (2.45×) |
63.1 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eagle graph, K fused steps per launch |
9.77 ms (9.74 ms–9.79 ms) |
– |
23.6 ms |
2.35e9 |
1.04e11 (22 %) |
2.94e10 (9.8 %) |
40 MiB (1.17×) |
62.8 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
7.6 ms (7.6 ms–7.61 ms) |
– |
21.2 ms |
3.02e9 |
1.33e11 (28 %) |
3.68e10 (12 %) |
56 MiB (1.63×) |
62.8 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
4.71 ms (4.7 ms–4.72 ms) |
– |
18.4 ms |
4.88e9 |
2.15e11 (45 %) |
1.44e11 (48 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
hawk + eagle persistent |
4.76 ms (4.75 ms–4.77 ms) |
– |
18.1 ms |
4.83e9 |
2.13e11 (45 %) |
1.42e11 (47 %) |
40 MiB (1.17×) |
59.9 MiB |
– |
0.493892157690208 |
– |
2040–2040 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
4.73 ms (4.73 ms–4.73 ms) |
– |
18.3 ms |
4.86e9 |
2.14e11 (45 %) |
1.43e11 (48 %) |
40 MiB (1.17×) |
59.9 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eager launch loop |
14.5 ms (14.5 ms–14.5 ms) |
– |
28.1 ms |
1.59e9 |
6.98e10 (15 %) |
2.72e11 (91 %) |
40 MiB (1.17×) |
57.7 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
CuPy masked |
159 ms (159 ms–159 ms) |
– |
175 ms |
1.45e8 |
6.37e9 (1.3 %) |
5.67e11 (1.9e+02 %) |
192 MiB (5.59×) |
68.7 MiB |
– |
– |
– |
1605–1680 |
1,000,000 |
PyTorch masked |
167 ms (165 ms–168 ms) |
– |
190 ms |
1.38e8 |
6.07e9 (1.3 %) |
5.41e11 (1.8e+02 %) |
288 MiB (8.39×) |
68.7 MiB |
– |
– |
– |
1770–1800 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
186 ms (186 ms–186 ms) |
– |
210 ms |
1.24e8 |
5.44e9 (1.1 %) |
5.45e11 (1.8e+02 %) |
316 MiB (9.2×) |
68.7 MiB |
– |
– |
– |
1695–1725 |
1,000,000 |
PyTorch torch.compile (reduce-overhead) |
25.6 ms (25.6 ms–25.6 ms) |
– |
49.1 ms |
8.98e8 |
3.95e10 (8.3 %) |
3.43e11 (1.1e+02 %) |
238 MiB (6.93×) |
146 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
JAX jit + vmap(while_loop) |
25 ms (24.9 ms–25.1 ms) |
– |
40.6 ms |
9.19e8 |
4.05e10 (8.5 %) |
3.53e11 (1.2e+02 %) |
134 MiB (3.9×) |
101 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
Warp per-thread kernel |
10.7 ms (10.7 ms–10.7 ms) |
– |
42.6 ms |
2.14e9 |
9.43e10 (20 %) |
5.96e9 (2 %) |
62 MiB (1.81×) |
68.7 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
CPU OpenMP |
33 ms (33 ms–33.1 ms) |
– |
47.7 ms |
6.96e8 |
3.06e10 (–) |
1.19e11 (–) |
– |
99.3 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (103 µs), then eagle.simulate (eagle picks the launch mode) (116 µs, 1.12× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (197 µs), then eagle.simulate (eagle picks the launch mode) (213 µs, 1.08× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (439 µs), then eagle.simulate (eagle picks the launch mode) (459 µs, 1.05× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (4.71 ms), then eagle.simulate (eagle picks the launch mode) (4.73 ms, 1× the time)
Spread stop steps (log-uniform in [10, 1000]), up to 1000 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
1.14 ms (1.14 ms–1.15 ms) |
– |
1.36 ms |
1.84e8 |
8.09e9 (1.7 %) |
5.70e9 (1.9 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eagle graph + compaction |
3.58 ms (3.57 ms–3.6 ms) |
– |
3.84 ms |
5.86e7 |
2.58e9 (0.55 %) |
4.41e9 (1.5 %) |
0 MiB (0×) |
2.05 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eagle graph + compaction + reorder |
3.79 ms (3.79 ms–3.79 ms) |
– |
4.07 ms |
5.54e7 |
2.44e9 (0.52 %) |
4.35e9 (1.5 %) |
0 MiB (0×) |
2.09 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eagle graph, K fused steps per launch |
893 µs (887 µs–903 µs) |
– |
1.07 ms |
2.35e8 |
1.04e10 (2.2 %) |
2.76e9 (0.92 %) |
0 MiB (0×) |
2.03 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eagle graph, K fused steps per launch + compaction |
1.04 ms (1.03 ms–1.04 ms) |
– |
1.29 ms |
2.03e8 |
8.91e9 (1.9 %) |
1.37e9 (0.46 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eagle auto (eagle picks the launch mode) |
430 µs (429 µs–433 µs) |
– |
641 µs |
4.89e8 |
2.15e10 (4.5 %) |
1.38e10 (4.6 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
2040–2040 |
1,000 |
hawk + eagle persistent |
723 µs (659 µs–821 µs) |
– |
876 µs |
2.91e8 |
1.28e10 (2.7 %) |
8.19e9 (2.7 %) |
0 MiB (0×) |
2.03 MiB |
– |
0.22980196323115862 |
– |
2040–2040 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
441 µs (440 µs–443 µs) |
– |
668 µs |
4.76e8 |
2.10e10 (4.4 %) |
1.34e10 (4.5 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eager launch loop |
31.7 ms (31.6 ms–31.8 ms) |
– |
32 ms |
6.62e6 |
2.91e8 (0.062 %) |
1.22e9 (0.41 %) |
0 MiB (0×) |
0.0312 MiB |
– |
– |
– |
2040–2040 |
1,000 |
CuPy masked |
656 ms (653 ms–662 ms) |
– |
656 ms |
3.20e5 |
1.41e7 (0.003 %) |
1.37e9 (0.46 %) |
0 MiB (0×) |
5.07 MiB |
– |
– |
– |
2040–2040 |
1,000 |
PyTorch masked |
409 ms (408 ms–409 ms) |
– |
409 ms |
5.14e5 |
2.26e7 (0.0048 %) |
2.20e9 (0.73 %) |
28 MiB (816×) |
0.0742 MiB |
– |
– |
– |
2040–2040 |
1,000 |
PyTorch, CUDA graph of 16 steps |
64.3 ms (64.3 ms–64.3 ms) |
– |
64.6 ms |
3.27e6 |
1.44e8 (0.03 %) |
1.42e10 (4.7 %) |
56 MiB (1.63e+03×) |
0.156 MiB |
– |
– |
– |
2040–2040 |
1,000 |
PyTorch torch.compile (reduce-overhead) |
129 ms (129 ms–129 ms) |
– |
129 ms |
1.63e6 |
7.18e7 (0.015 %) |
6.83e8 (0.23 %) |
50 MiB (1.46e+03×) |
32.5 MiB |
– |
– |
– |
2040–2040 |
1,000 |
JAX jit + vmap(while_loop) |
17.6 ms (17.4 ms–17.8 ms) |
– |
19.5 ms |
1.20e7 |
5.27e8 (0.11 %) |
5.02e9 (1.7 %) |
134 MiB (3.9e+03×) |
0.0117 MiB |
– |
– |
– |
2040–2040 |
1,000 |
Warp per-thread kernel |
1.64 ms (1.64 ms–1.65 ms) |
– |
2.01 ms |
1.28e8 |
5.63e9 (1.2 %) |
3.89e7 (0.013 %) |
30 MiB (874×) |
0.0234 MiB |
– |
– |
– |
2040–2040 |
1,000 |
CPU OpenMP |
982 µs (978 µs–986 µs) |
– |
1 ms |
2.14e8 |
9.42e9 (–) |
3.96e10 (–) |
– |
0.191 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
1.81 ms (1.8 ms–1.82 ms) |
– |
2.13 ms |
1.21e9 |
5.31e10 (11 %) |
3.72e10 (12 %) |
0 MiB (0×) |
2.43 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eagle graph + compaction |
4.13 ms (4.13 ms–4.14 ms) |
– |
4.53 ms |
5.28e8 |
2.32e10 (4.9 %) |
3.95e10 (13 %) |
0 MiB (0×) |
2.44 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eagle graph + compaction + reorder |
4.73 ms (4.72 ms–4.74 ms) |
– |
5.17 ms |
4.61e8 |
2.03e10 (4.3 %) |
3.57e10 (12 %) |
0 MiB (0×) |
2.91 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eagle graph, K fused steps per launch |
1.67 ms (1.67 ms–1.67 ms) |
– |
2 ms |
1.30e9 |
5.74e10 (12 %) |
1.48e10 (4.9 %) |
0 MiB (0×) |
2.43 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eagle graph, K fused steps per launch + compaction |
1.35 ms (1.34 ms–1.35 ms) |
– |
1.74 ms |
1.62e9 |
7.12e10 (15 %) |
1.08e10 (3.6 %) |
0 MiB (0×) |
2.45 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eagle auto (eagle picks the launch mode) |
1.19 ms (1.19 ms–1.19 ms) |
– |
1.55 ms |
1.83e9 |
8.06e10 (17 %) |
5.16e10 (17 %) |
0 MiB (0×) |
2.34 MiB |
– |
– |
– |
2040–2040 |
10,000 |
hawk + eagle persistent |
1.59 ms (1.59 ms–1.6 ms) |
– |
1.91 ms |
1.37e9 |
6.02e10 (13 %) |
3.85e10 (13 %) |
0 MiB (0×) |
2.3 MiB |
– |
0.2384722378329774 |
– |
2040–2040 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
1.2 ms (1.2 ms–1.21 ms) |
– |
1.58 ms |
1.81e9 |
7.98e10 (17 %) |
5.10e10 (17 %) |
0 MiB (0×) |
2.34 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eager launch loop |
31.7 ms (31.6 ms–31.9 ms) |
– |
32.1 ms |
6.88e7 |
3.03e9 (0.64 %) |
1.23e10 (4.1 %) |
0 MiB (0×) |
0.141 MiB |
– |
– |
– |
2040–2040 |
10,000 |
CuPy masked |
659 ms (656 ms–666 ms) |
– |
660 ms |
3.31e6 |
1.46e8 (0.031 %) |
1.37e10 (4.6 %) |
0 MiB (0×) |
5.62 MiB |
– |
– |
– |
2040–2040 |
10,000 |
PyTorch masked |
415 ms (410 ms–424 ms) |
– |
415 ms |
5.26e6 |
2.32e8 (0.049 %) |
2.17e10 (7.2 %) |
28 MiB (81.6×) |
0.688 MiB |
– |
– |
– |
2040–2040 |
10,000 |
PyTorch, CUDA graph of 16 steps |
62.4 ms (62.3 ms–62.4 ms) |
– |
62.9 ms |
3.50e7 |
1.54e9 (0.33 %) |
1.46e11 (49 %) |
56 MiB (163×) |
0.719 MiB |
– |
– |
– |
2040–2040 |
10,000 |
PyTorch torch.compile (reduce-overhead) |
129 ms (129 ms–129 ms) |
– |
130 ms |
1.69e7 |
7.44e8 (0.16 %) |
6.82e9 (2.3 %) |
50 MiB (146×) |
33.2 MiB |
– |
– |
– |
2040–2040 |
10,000 |
JAX jit + vmap(while_loop) |
20 ms (19.9 ms–20.1 ms) |
– |
22.2 ms |
1.09e8 |
4.81e9 (1 %) |
4.40e10 (15 %) |
134 MiB (390×) |
0.703 MiB |
– |
– |
– |
2040–2040 |
10,000 |
Warp per-thread kernel |
1.76 ms (1.76 ms–1.76 ms) |
– |
2.27 ms |
1.24e9 |
5.46e10 (12 %) |
3.64e8 (0.12 %) |
30 MiB (87.4×) |
0.691 MiB |
– |
– |
– |
2040–2040 |
10,000 |
CPU OpenMP |
3.91 ms (3.88 ms–3.94 ms) |
– |
4.01 ms |
5.58e8 |
2.45e10 (–) |
1.00e11 (–) |
– |
1.17 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
10.6 ms (10.6 ms–10.7 ms) |
– |
12.3 ms |
2.03e9 |
8.95e10 (19 %) |
6.46e10 (22 %) |
4 MiB (1.17×) |
7.86 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eagle graph + compaction |
10.5 ms (10.5 ms–10.5 ms) |
– |
12.3 ms |
2.06e9 |
9.06e10 (19 %) |
1.54e11 (51 %) |
4 MiB (1.17×) |
7.89 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eagle graph + compaction + reorder |
11.1 ms (11.1 ms–11.1 ms) |
– |
12.9 ms |
1.94e9 |
8.56e10 (18 %) |
1.51e11 (50 %) |
8 MiB (2.33×) |
8.5 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eagle graph, K fused steps per launch |
10.6 ms (10.6 ms–10.6 ms) |
– |
12.2 ms |
2.05e9 |
9.00e10 (19 %) |
2.34e10 (7.8 %) |
4 MiB (1.17×) |
7.86 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eagle graph, K fused steps per launch + compaction |
6.2 ms (6.19 ms–6.2 ms) |
– |
7.89 ms |
3.49e9 |
1.54e11 (32 %) |
2.34e10 (7.8 %) |
4 MiB (1.17×) |
7.88 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eagle auto (eagle picks the launch mode) |
3.35 ms (3.34 ms–3.35 ms) |
– |
5.01 ms |
6.47e9 |
2.85e11 (60 %) |
1.82e11 (61 %) |
4 MiB (1.17×) |
7.59 MiB |
– |
– |
– |
2040–2040 |
100,000 |
hawk + eagle persistent |
3.9 ms (3.89 ms–3.92 ms) |
– |
5.49 ms |
5.55e9 |
2.44e11 (52 %) |
1.56e11 (52 %) |
4 MiB (1.17×) |
7.58 MiB |
– |
0.6067670242066809 |
– |
2040–2040 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
3.34 ms (3.34 ms–3.35 ms) |
– |
4.96 ms |
6.48e9 |
2.85e11 (60 %) |
1.82e11 (61 %) |
4 MiB (1.17×) |
7.58 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eager launch loop |
40.1 ms (40 ms–40.3 ms) |
– |
41.9 ms |
5.39e8 |
2.37e10 (5 %) |
9.73e10 (32 %) |
4 MiB (1.17×) |
5.37 MiB |
– |
– |
– |
2040–2040 |
100,000 |
CuPy masked |
661 ms (659 ms–663 ms) |
– |
663 ms |
3.28e7 |
1.44e9 (0.3 %) |
1.36e11 (45 %) |
22 MiB (6.41×) |
9.39 MiB |
– |
– |
– |
2040–2040 |
100,000 |
PyTorch masked |
428 ms (428 ms–428 ms) |
– |
430 ms |
5.06e7 |
2.23e9 (0.47 %) |
2.11e11 (70 %) |
52 MiB (15.1×) |
6.86 MiB |
– |
– |
– |
2040–2040 |
100,000 |
PyTorch, CUDA graph of 16 steps |
93.5 ms (93.5 ms–93.6 ms) |
– |
95.8 ms |
2.32e8 |
1.02e10 (2.2 %) |
9.75e11 (3.3e+02 %) |
78 MiB (22.7×) |
6.87 MiB |
– |
– |
– |
2040–2040 |
100,000 |
PyTorch torch.compile (reduce-overhead) |
133 ms (132 ms–135 ms) |
– |
135 ms |
1.63e8 |
7.17e9 (1.5 %) |
6.62e10 (22 %) |
56 MiB (16.3×) |
36.7 MiB |
– |
– |
– |
2040–2040 |
100,000 |
JAX jit + vmap(while_loop) |
44.3 ms (44.3 ms–44.3 ms) |
– |
47.8 ms |
4.89e8 |
2.15e10 (4.5 %) |
1.99e11 (66 %) |
134 MiB (39×) |
10.9 MiB |
– |
– |
– |
2040–2040 |
100,000 |
Warp per-thread kernel |
11.1 ms (11.1 ms–11.1 ms) |
– |
13.3 ms |
1.95e9 |
8.56e10 (18 %) |
5.75e8 (0.19 %) |
30 MiB (8.74×) |
6.87 MiB |
– |
– |
– |
2040–2040 |
100,000 |
CPU OpenMP |
32 ms (32 ms–32.1 ms) |
– |
33.1 ms |
6.76e8 |
2.98e10 (–) |
1.22e11 (–) |
– |
10 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
94.4 ms (94.4 ms–94.4 ms) |
– |
108 ms |
2.29e9 |
1.01e11 (21 %) |
8.93e10 (30 %) |
40 MiB (1.17×) |
62.8 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eagle graph + compaction |
39.3 ms (39.1 ms–39.9 ms) |
– |
53.2 ms |
5.50e9 |
2.42e11 (51 %) |
4.12e11 (1.4e+02 %) |
56 MiB (1.63×) |
62.8 MiB |
– |
– |
– |
1950–2025 |
1,000,000 |
eagle graph + compaction + reorder |
39.3 ms (39.3 ms–39.3 ms) |
– |
53.4 ms |
5.50e9 |
2.42e11 (51 %) |
4.27e11 (1.4e+02 %) |
84 MiB (2.45×) |
63.1 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eagle graph, K fused steps per launch |
93.9 ms (93.8 ms–93.9 ms) |
– |
108 ms |
2.30e9 |
1.01e11 (21 %) |
2.63e10 (8.8 %) |
40 MiB (1.17×) |
62.8 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
28.5 ms (28.5 ms–28.5 ms) |
– |
42.3 ms |
7.59e9 |
3.34e11 (71 %) |
5.09e10 (17 %) |
56 MiB (1.63×) |
62.8 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
25.3 ms (25.3 ms–25.3 ms) |
– |
39.1 ms |
8.54e9 |
3.76e11 (79 %) |
2.40e11 (80 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
hawk + eagle persistent |
25.4 ms (25.4 ms–25.5 ms) |
– |
39.1 ms |
8.50e9 |
3.74e11 (79 %) |
2.39e11 (80 %) |
40 MiB (1.17×) |
59.9 MiB |
– |
0.8854809251857924 |
– |
2040–2040 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
25.3 ms (25.3 ms–25.4 ms) |
– |
39.1 ms |
8.53e9 |
3.75e11 (79 %) |
2.40e11 (80 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eager launch loop |
146 ms (144 ms–147 ms) |
– |
159 ms |
1.49e9 |
6.54e10 (14 %) |
2.68e11 (89 %) |
40 MiB (1.17×) |
57.7 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
CuPy masked |
1.6 s (1.6 s–1.6 s) |
– |
1.62 s |
1.35e8 |
5.94e9 (1.3 %) |
5.62e11 (1.9e+02 %) |
192 MiB (5.59×) |
68.7 MiB |
– |
– |
– |
1500–1590 |
1,000,000 |
PyTorch masked |
1.68 s (1.67 s–1.69 s) |
– |
1.7 s |
1.29e8 |
5.67e9 (1.2 %) |
5.37e11 (1.8e+02 %) |
288 MiB (8.39×) |
68.7 MiB |
– |
– |
– |
1650–1665 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
1.68 s (1.68 s–1.68 s) |
– |
1.71 s |
1.28e8 |
5.65e9 (1.2 %) |
5.42e11 (1.8e+02 %) |
316 MiB (9.2×) |
68.7 MiB |
– |
– |
– |
1575–1620 |
1,000,000 |
PyTorch torch.compile (reduce-overhead) |
251 ms (251 ms–251 ms) |
– |
274 ms |
8.62e8 |
3.79e10 (8 %) |
3.51e11 (1.2e+02 %) |
130 MiB (3.79×) |
77.3 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
JAX jit + vmap(while_loop) |
243 ms (242 ms–243 ms) |
– |
258 ms |
8.92e8 |
3.92e10 (8.3 %) |
3.63e11 (1.2e+02 %) |
134 MiB (3.9×) |
101 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
Warp per-thread kernel |
105 ms (105 ms–105 ms) |
– |
130 ms |
2.06e9 |
9.08e10 (19 %) |
6.10e8 (0.2 %) |
62 MiB (1.81×) |
68.7 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
CPU OpenMP |
310 ms (310 ms–311 ms) |
– |
322 ms |
6.97e8 |
3.07e10 (–) |
1.26e11 (–) |
– |
99.3 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (430 µs), then eagle.simulate (eagle picks the launch mode) (441 µs, 1.03× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (1.19 ms), then eagle.simulate (eagle picks the launch mode) (1.2 ms, 1.01× the time)
N = 100,000: eagle.simulate (eagle picks the launch mode) (3.34 ms), then eagle auto (eagle picks the launch mode) (3.35 ms, 1× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (25.3 ms), then eagle.simulate (eagle picks the launch mode) (25.3 ms, 1× the time)
Uniform stop step: every sample runs 1000 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
1.01 ms (1 ms–1.01 ms) |
– |
1.17 ms |
9.94e8 |
4.37e10 (9.2 %) |
2.85e10 (9.5 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eagle graph + compaction |
2.88 ms (2.87 ms–2.88 ms) |
– |
3.13 ms |
3.47e8 |
1.53e10 (3.2 %) |
2.27e10 (7.6 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eagle graph + compaction + reorder |
3.03 ms (3.03 ms–3.03 ms) |
– |
3.32 ms |
3.30e8 |
1.45e10 (3.1 %) |
2.16e10 (7.2 %) |
0 MiB (0×) |
2.1 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eagle graph, K fused steps per launch |
902 µs (897 µs–909 µs) |
– |
1.08 ms |
1.11e9 |
4.88e10 (10 %) |
4.26e9 (1.4 %) |
0 MiB (0×) |
2.03 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eagle graph, K fused steps per launch + compaction |
863 µs (857 µs–871 µs) |
– |
1.12 ms |
1.16e9 |
5.10e10 (11 %) |
4.82e9 (1.6 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eagle auto (eagle picks the launch mode) |
429 µs (428 µs–429 µs) |
– |
646 µs |
2.33e9 |
1.03e11 (22 %) |
6.54e10 (22 %) |
0 MiB (0×) |
2.03 MiB |
– |
– |
– |
2040–2040 |
1,000 |
hawk + eagle persistent |
695 µs (692 µs–697 µs) |
– |
851 µs |
1.44e9 |
6.33e10 (13 %) |
4.04e10 (13 %) |
0 MiB (0×) |
2.03 MiB |
– |
0.95367431640625 |
– |
2040–2040 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
440 µs (438 µs–441 µs) |
– |
665 µs |
2.27e9 |
1.00e11 (21 %) |
6.37e10 (21 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
2040–2040 |
1,000 |
eager launch loop |
31.6 ms (31.6 ms–31.7 ms) |
– |
31.8 ms |
3.16e7 |
1.39e9 (0.29 %) |
1.93e9 (0.64 %) |
0 MiB (0×) |
0.0352 MiB |
– |
– |
– |
2040–2040 |
1,000 |
CuPy masked |
665 ms (659 ms–670 ms) |
– |
666 ms |
1.50e6 |
6.61e7 (0.014 %) |
1.35e9 (0.45 %) |
0 MiB (0×) |
4.66 MiB |
– |
– |
– |
2040–2040 |
1,000 |
PyTorch masked |
414 ms (408 ms–426 ms) |
– |
414 ms |
2.41e6 |
1.06e8 (0.022 %) |
2.18e9 (0.73 %) |
28 MiB (816×) |
0.0703 MiB |
– |
– |
– |
2040–2040 |
1,000 |
PyTorch, CUDA graph of 16 steps |
63.9 ms (63.9 ms–63.9 ms) |
– |
64.2 ms |
1.57e7 |
6.89e8 (0.15 %) |
1.43e10 (4.8 %) |
56 MiB (1.63e+03×) |
0.16 MiB |
– |
– |
– |
2040–2040 |
1,000 |
PyTorch torch.compile (reduce-overhead) |
129 ms (128 ms–130 ms) |
– |
129 ms |
7.77e6 |
3.42e8 (0.072 %) |
6.84e8 (0.23 %) |
50 MiB (1.46e+03×) |
32.4 MiB |
– |
– |
– |
2040–2040 |
1,000 |
JAX jit + vmap(while_loop) |
17.3 ms (17.2 ms–17.4 ms) |
– |
19.1 ms |
5.78e7 |
2.54e9 (0.54 %) |
5.08e9 (1.7 %) |
134 MiB (3.9e+03×) |
0.00781 MiB |
– |
– |
– |
2040–2040 |
1,000 |
Warp per-thread kernel |
1.84 ms (1.84 ms–1.84 ms) |
– |
2.22 ms |
5.43e8 |
2.39e10 (5.1 %) |
3.48e7 (0.012 %) |
30 MiB (874×) |
0.0156 MiB |
– |
– |
– |
2040–2040 |
1,000 |
CPU OpenMP |
869 µs (863 µs–873 µs) |
– |
890 µs |
1.15e9 |
5.06e10 (–) |
7.02e10 (–) |
– |
0.184 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
1.82 ms (1.81 ms–1.83 ms) |
– |
2.15 ms |
5.49e9 |
2.41e11 (51 %) |
1.57e11 (52 %) |
0 MiB (0×) |
2.43 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eagle graph + compaction |
3.99 ms (3.99 ms–4 ms) |
– |
4.4 ms |
2.50e9 |
1.10e11 (23 %) |
1.64e11 (55 %) |
0 MiB (0×) |
2.43 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eagle graph + compaction + reorder |
4.36 ms (4.36 ms–4.37 ms) |
– |
4.81 ms |
2.29e9 |
1.01e11 (21 %) |
1.50e11 (50 %) |
0 MiB (0×) |
2.91 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eagle graph, K fused steps per launch |
1.72 ms (1.72 ms–1.73 ms) |
– |
2.05 ms |
5.81e9 |
2.56e11 (54 %) |
2.23e10 (7.4 %) |
0 MiB (0×) |
2.42 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eagle graph, K fused steps per launch + compaction |
1.69 ms (1.68 ms–1.69 ms) |
– |
2.09 ms |
5.93e9 |
2.61e11 (55 %) |
2.47e10 (8.2 %) |
0 MiB (0×) |
2.43 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eagle auto (eagle picks the launch mode) |
1.24 ms (1.24 ms–1.25 ms) |
– |
1.62 ms |
8.03e9 |
3.54e11 (75 %) |
2.25e11 (75 %) |
0 MiB (0×) |
2.48 MiB |
– |
– |
– |
2040–2040 |
10,000 |
hawk + eagle persistent |
1.66 ms (1.65 ms–1.66 ms) |
– |
1.97 ms |
6.04e9 |
2.66e11 (56 %) |
1.69e11 (56 %) |
0 MiB (0×) |
2.31 MiB |
– |
0.9750024960063898 |
– |
2040–2040 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
1.25 ms (1.25 ms–1.25 ms) |
– |
1.63 ms |
7.99e9 |
3.52e11 (74 %) |
2.24e11 (75 %) |
0 MiB (0×) |
2.34 MiB |
– |
– |
– |
2040–2040 |
10,000 |
eager launch loop |
32.4 ms (32.1 ms–32.7 ms) |
– |
32.7 ms |
3.09e8 |
1.36e10 (2.9 %) |
1.88e10 (6.3 %) |
0 MiB (0×) |
0.129 MiB |
– |
– |
– |
2040–2040 |
10,000 |
CuPy masked |
656 ms (654 ms–658 ms) |
– |
657 ms |
1.52e7 |
6.71e8 (0.14 %) |
1.37e10 (4.6 %) |
0 MiB (0×) |
5.62 MiB |
– |
– |
– |
2040–2040 |
10,000 |
PyTorch masked |
412 ms (411 ms–413 ms) |
– |
412 ms |
2.43e7 |
1.07e9 (0.23 %) |
2.19e10 (7.3 %) |
28 MiB (81.6×) |
0.625 MiB |
– |
– |
– |
2040–2040 |
10,000 |
PyTorch, CUDA graph of 16 steps |
62.4 ms (62.3 ms–62.4 ms) |
– |
62.9 ms |
1.60e8 |
7.06e9 (1.5 %) |
1.46e11 (49 %) |
56 MiB (163×) |
0.719 MiB |
– |
– |
– |
2040–2040 |
10,000 |
PyTorch torch.compile (reduce-overhead) |
132 ms (132 ms–132 ms) |
– |
132 ms |
7.60e7 |
3.34e9 (0.71 %) |
6.69e9 (2.2 %) |
50 MiB (146×) |
33.3 MiB |
– |
– |
– |
2040–2040 |
10,000 |
JAX jit + vmap(while_loop) |
19.2 ms (19.1 ms–19.3 ms) |
– |
21.2 ms |
5.20e8 |
2.29e10 (4.8 %) |
4.58e10 (15 %) |
134 MiB (390×) |
0.641 MiB |
– |
– |
– |
2040–2040 |
10,000 |
Warp per-thread kernel |
1.86 ms (1.85 ms–1.87 ms) |
– |
2.41 ms |
5.37e9 |
2.36e11 (50 %) |
3.44e8 (0.11 %) |
30 MiB (87.4×) |
0.691 MiB |
– |
– |
– |
2040–2040 |
10,000 |
CPU OpenMP |
4.18 ms (4.17 ms–4.19 ms) |
– |
4.27 ms |
2.39e9 |
1.05e11 (–) |
1.46e11 (–) |
– |
1.15 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
11.5 ms (11.4 ms–11.5 ms) |
– |
13.1 ms |
8.73e9 |
3.84e11 (81 %) |
2.50e11 (83 %) |
4 MiB (1.17×) |
7.66 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eagle graph + compaction |
16.3 ms (16.3 ms–16.3 ms) |
– |
18 ms |
6.15e9 |
2.70e11 (57 %) |
4.02e11 (1.3e+02 %) |
4 MiB (1.17×) |
7.69 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eagle graph + compaction + reorder |
16.6 ms (16.6 ms–16.7 ms) |
– |
18.4 ms |
6.02e9 |
2.65e11 (56 %) |
3.93e11 (1.3e+02 %) |
8 MiB (2.33×) |
8.32 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eagle graph, K fused steps per launch |
11.4 ms (11.4 ms–11.4 ms) |
– |
13.1 ms |
8.74e9 |
3.84e11 (81 %) |
3.36e10 (11 %) |
4 MiB (1.17×) |
7.66 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eagle graph, K fused steps per launch + compaction |
11.4 ms (11.4 ms–11.4 ms) |
– |
13.1 ms |
8.78e9 |
3.86e11 (82 %) |
3.65e10 (12 %) |
4 MiB (1.17×) |
7.68 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eagle auto (eagle picks the launch mode) |
10.8 ms (10.8 ms–10.8 ms) |
– |
12.5 ms |
9.27e9 |
4.08e11 (86 %) |
2.60e11 (87 %) |
4 MiB (1.17×) |
7.61 MiB |
– |
– |
– |
2040–2040 |
100,000 |
hawk + eagle persistent |
10.8 ms (10.8 ms–10.8 ms) |
– |
12.5 ms |
9.24e9 |
4.07e11 (86 %) |
2.59e11 (86 %) |
4 MiB (1.17×) |
7.52 MiB |
– |
0.9765625 |
– |
2040–2040 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
10.8 ms (10.8 ms–10.8 ms) |
– |
12.6 ms |
9.25e9 |
4.07e11 (86 %) |
2.59e11 (86 %) |
4 MiB (1.17×) |
7.59 MiB |
– |
– |
– |
2040–2040 |
100,000 |
eager launch loop |
40.8 ms (40.8 ms–40.8 ms) |
– |
42.5 ms |
2.45e9 |
1.08e11 (23 %) |
1.49e11 (50 %) |
4 MiB (1.17×) |
5.37 MiB |
– |
– |
– |
2040–2040 |
100,000 |
CuPy masked |
665 ms (660 ms–673 ms) |
– |
667 ms |
1.50e8 |
6.61e9 (1.4 %) |
1.35e11 (45 %) |
22 MiB (6.41×) |
9.39 MiB |
– |
– |
– |
2040–2040 |
100,000 |
PyTorch masked |
434 ms (427 ms–447 ms) |
– |
436 ms |
2.30e8 |
1.01e10 (2.1 %) |
2.08e11 (69 %) |
52 MiB (15.1×) |
6.25 MiB |
– |
– |
– |
2040–2040 |
100,000 |
PyTorch, CUDA graph of 16 steps |
93.6 ms (93.5 ms–93.7 ms) |
– |
95.9 ms |
1.07e9 |
4.70e10 (9.9 %) |
9.75e11 (3.2e+02 %) |
78 MiB (22.7×) |
6.33 MiB |
– |
– |
– |
2040–2040 |
100,000 |
PyTorch torch.compile (reduce-overhead) |
132 ms (131 ms–133 ms) |
– |
134 ms |
7.60e8 |
3.34e10 (7.1 %) |
6.69e10 (22 %) |
56 MiB (16.3×) |
36.8 MiB |
– |
– |
– |
2040–2040 |
100,000 |
JAX jit + vmap(while_loop) |
44.1 ms (44.1 ms–44.1 ms) |
– |
47.4 ms |
2.27e9 |
9.98e10 (21 %) |
2.00e11 (67 %) |
134 MiB (39×) |
10.4 MiB |
– |
– |
– |
2040–2040 |
100,000 |
Warp per-thread kernel |
12.3 ms (12.3 ms–12.3 ms) |
– |
14.6 ms |
8.10e9 |
3.57e11 (75 %) |
5.19e8 (0.17 %) |
30 MiB (8.74×) |
6.87 MiB |
– |
– |
– |
2040–2040 |
100,000 |
CPU OpenMP |
36 ms (35.9 ms–36 ms) |
– |
37 ms |
2.78e9 |
1.22e11 (–) |
1.70e11 (–) |
– |
9.92 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
105 ms (105 ms–105 ms) |
– |
119 ms |
9.51e9 |
4.18e11 (88 %) |
2.72e11 (91 %) |
40 MiB (1.17×) |
60.1 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eagle graph + compaction |
133 ms (133 ms–133 ms) |
– |
147 ms |
7.50e9 |
3.30e11 (70 %) |
4.90e11 (1.6e+02 %) |
56 MiB (1.63×) |
60.1 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eagle graph + compaction + reorder |
134 ms (134 ms–134 ms) |
– |
147 ms |
7.47e9 |
3.29e11 (69 %) |
4.88e11 (1.6e+02 %) |
84 MiB (2.45×) |
60.6 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eagle graph, K fused steps per launch |
106 ms (106 ms–106 ms) |
– |
120 ms |
9.43e9 |
4.15e11 (88 %) |
3.63e10 (12 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
106 ms (106 ms–106 ms) |
– |
120 ms |
9.43e9 |
4.15e11 (88 %) |
3.92e10 (13 %) |
56 MiB (1.63×) |
60.1 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
103 ms (103 ms–103 ms) |
– |
117 ms |
9.71e9 |
4.27e11 (90 %) |
2.72e11 (91 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
hawk + eagle persistent |
103 ms (103 ms–103 ms) |
– |
117 ms |
9.71e9 |
4.27e11 (90 %) |
2.72e11 (91 %) |
40 MiB (1.17×) |
60 MiB |
– |
0.9765625 |
– |
2040–2040 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
103 ms (103 ms–103 ms) |
– |
117 ms |
9.71e9 |
4.27e11 (90 %) |
2.72e11 (91 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
eager launch loop |
158 ms (158 ms–158 ms) |
– |
172 ms |
6.33e9 |
2.78e11 (59 %) |
3.86e11 (1.3e+02 %) |
40 MiB (1.17×) |
57.8 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
CuPy masked |
1.6 s (1.6 s–1.6 s) |
– |
1.62 s |
6.24e8 |
2.75e10 (5.8 %) |
5.63e11 (1.9e+02 %) |
192 MiB (5.59×) |
64.6 MiB |
– |
– |
– |
1530–1590 |
1,000,000 |
PyTorch masked |
1.68 s (1.67 s–1.69 s) |
– |
1.7 s |
5.95e8 |
2.62e10 (5.5 %) |
5.36e11 (1.8e+02 %) |
288 MiB (8.39×) |
62.1 MiB |
– |
– |
– |
1650–1680 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
1.68 s (1.68 s–1.68 s) |
– |
1.71 s |
5.94e8 |
2.61e10 (5.5 %) |
5.42e11 (1.8e+02 %) |
316 MiB (9.2×) |
62.2 MiB |
– |
– |
– |
1590–1635 |
1,000,000 |
PyTorch torch.compile (reduce-overhead) |
251 ms (250 ms–251 ms) |
– |
274 ms |
3.99e9 |
1.76e11 (37 %) |
3.51e11 (1.2e+02 %) |
130 MiB (3.79×) |
71.1 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
JAX jit + vmap(while_loop) |
242 ms (241 ms–242 ms) |
– |
257 ms |
4.13e9 |
1.82e11 (38 %) |
3.64e11 (1.2e+02 %) |
134 MiB (3.9×) |
95.1 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
Warp per-thread kernel |
119 ms (119 ms–119 ms) |
– |
150 ms |
8.41e9 |
3.70e11 (78 %) |
5.38e8 (0.18 %) |
62 MiB (1.81×) |
68.7 MiB |
– |
– |
– |
2040–2040 |
1,000,000 |
CPU OpenMP |
352 ms (352 ms–352 ms) |
– |
368 ms |
2.84e9 |
1.25e11 (–) |
1.73e11 (–) |
– |
97.5 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (429 µs), then eagle.simulate (eagle picks the launch mode) (440 µs, 1.03× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (1.24 ms), then eagle.simulate (eagle picks the launch mode) (1.25 ms, 1.01× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (10.8 ms), then eagle.simulate (eagle picks the launch mode) (10.8 ms, 1× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (103 ms), then eagle.simulate (eagle picks the launch mode) (103 ms, 1× the time)
Compaction cadence sweep (isolates the active-set map)
the active-set map alone. For a given K, matched_cadence (the plain per-sample step, no map) and compact (eagle_graph_compact’s active-set step) check the loop’s stop guard at the SAME cadence, once every K steps; the only structural difference between the two rows at a given K is whether the step kernel reads the active-set map. The main table’s eagle_graph row checks its guard every step, so its difference from eagle_graph_compact there mixes the cadence change with the map – this sweep does not.
Config: Spread stop steps (log-uniform in [10, 1000]), up to 1000 steps.
N |
K (steps between guard checks) |
matched cadence, no map (range) |
eagle graph + compaction (range) |
ratio (matched/compact, |
|---|---|---|---|---|
100,000 |
8 |
14.4 ms (14.4 ms–14.4 ms) |
11.9 ms (11.9 ms–11.9 ms) |
1.21× |
100,000 |
16 |
14.2 ms (14.2 ms–14.2 ms) |
10.5 ms (10.5 ms–10.5 ms) |
1.35× |
100,000 |
32 |
14.1 ms (14.1 ms–14.1 ms) |
9.86 ms (9.86 ms–9.88 ms) |
1.43× |
1,000,000 |
8 |
117 ms (117 ms–117 ms) |
42.2 ms (41.9 ms–42.5 ms) |
2.77× |
1,000,000 |
16 |
117 ms (117 ms–117 ms) |
39.1 ms (38.7 ms–39.5 ms) |
2.99× |
1,000,000 |
32 |
117 ms (117 ms–117 ms) |
37.6 ms (37.4 ms–37.9 ms) |
3.11× |
FP32: eagle vs Warp (float32, S = 1000)
Every arm runs the same oscillator in float32: eagle.simulate with scalar_type=”float32”, NVIDIA Warp’s per-thread kernel and JAX’s vmapped loop, all over the same float32 inputs. Each cell runs 2 untimed runs per arm first (eagle’s automatic arm picks its entry over them: the size rule, then a measured run of the other entry, keeping the faster; the entry it settled on is in the last column), then each arm in its own block (no interleaving): an untimed ramp of back-to-back runs for 0.5 s, then 5 timed runs. Before any timing counts, eagle and JAX reach step counts identical to Warp’s and states within absolute 0.0001 of Warp’s (the largest differences are in the JSON). Each time is the mean of the middle 3 of the 5 runs (highest and lowest dropped), with the range of those 3; eagle / Warp is the ratio of these means: below 1 eagle is faster. Before each arm’s block the GPU is cooled down: the block starts once the temperature is within 3 °C of the idle 46 °C and the GPU is idle, or after 20 s. Right after each timed run the SM clock is read (its range is shown per arm); cycles are the wall time times that clock, and eagle / Warp (cycles) is their ratio.
Every arm runs each sample to its own stop and no further than max_steps (S): eagle and Warp enforce the cap per sample, JAX in its while-loop condition, PyTorch and CuPy by the outer step count (a batch-level cap); an arm that runs whole blocks of steps per launch or replay stops at the first block boundary at or after it.
What each arm needed to get there: Warp, a hand-written step counter and cap inside the kernel; JAX, a cap in the while_loop condition; eagle, the kernel source unchanged, with its loop shape, unrolling and device entry chosen by hawk + eagle (per device, per batch).
Cap fixture (checked before any timing counts): N = 4,096, 8 samples with stop step 2S = 256, S = 128; every arm (3) reported k = min(stop step, S) per sample and the same finished count (4,088): passed.
distribution |
N |
eagle.simulate (range) |
Warp (range) |
JAX (range) |
eagle / Warp |
eagle / Warp (cycles) |
eagle entry |
|---|---|---|---|---|---|---|---|
uniform |
10,000 |
156 µs (149 µs–161 µs), 2040–2040 MHz |
137 µs (134 µs–142 µs), 2040–2040 MHz |
19.3 ms (19.3 ms–19.4 ms), 2040–2040 MHz |
1.14× |
1.14× |
fused_one |
uniform |
100,000 |
347 µs (346 µs–348 µs), 2040–2040 MHz |
340 µs (338 µs–342 µs), 2040–2040 MHz |
19.3 ms (19.3 ms–19.3 ms), 2040–2040 MHz |
1.02× |
1.02× |
fused_one |
uniform |
1,000,000 |
2.8 ms (2.8 ms–2.8 ms), 2040–2040 MHz |
2.44 ms (2.44 ms–2.44 ms), 2040–2040 MHz |
36.7 ms (36.6 ms–36.8 ms), 2040–2040 MHz |
1.15× |
1.15× |
persist |
spread |
10,000 |
141 µs (140 µs–142 µs), 2040–2040 MHz |
133 µs (131 µs–135 µs), 2040–2040 MHz |
19.4 ms (19.3 ms–19.7 ms), 2040–2040 MHz |
1.06× |
1.06× |
fused_one |
spread |
100,000 |
269 µs (265 µs–276 µs), 2040–2040 MHz |
306 µs (303 µs–311 µs), 2040–2040 MHz |
19.3 ms (19.2 ms–19.4 ms), 2040–2040 MHz |
0.881× |
0.881× |
persist |
spread |
1,000,000 |
969 µs (962 µs–978 µs), 2040–2040 MHz |
1.9 ms (1.89 ms–1.9 ms), 2040–2040 MHz |
36.7 ms (36.7 ms–36.8 ms), 2040–2040 MHz |
0.511× |
0.511× |
persist |
Fixed overhead per run (N = 64, S = 100)
item 3: wall at a cell small enough that per-run overhead dominates (N=64, S=100), each arm’s own kernel-only vs wall where the main nsys pass covers this cell’s N (it does not by default, so kernel_only_s is usually null here)
arm |
wall (range) |
kernel-only |
|---|---|---|
eagle graph (device loop) |
594 µs (585 µs–601 µs) |
– |
eagle graph + compaction |
561 µs (556 µs–566 µs) |
– |
eagle graph + compaction + reorder |
722 µs (718 µs–724 µs) |
– |
eagle graph, K fused steps per launch |
296 µs (286 µs–307 µs) |
– |
eagle graph, K fused steps per launch + compaction |
352 µs (334 µs–375 µs) |
– |
eagle auto (eagle picks the launch mode) |
107 µs (102 µs–109 µs) |
– |
hawk + eagle persistent |
148 µs (145 µs–150 µs) |
– |
eagle.simulate (eagle picks the launch mode) |
124 µs (119 µs–132 µs) |
– |
eager launch loop |
3.26 ms (3.24 ms–3.28 ms) |
– |
CuPy masked |
63.9 ms (63.8 ms–64.2 ms) |
– |
PyTorch masked |
40.9 ms (40.8 ms–41 ms) |
– |
PyTorch, CUDA graph of 16 steps |
5.64 ms (5.64 ms–5.65 ms) |
– |
PyTorch torch.compile (reduce-overhead) |
13.3 ms (13.3 ms–13.3 ms) |
– |
JAX jit + vmap(while_loop) |
2.26 ms (2.25 ms–2.26 ms) |
– |
Warp per-thread kernel |
133 µs (132 µs–135 µs) |
– |
CPU OpenMP |
413 µs (398 µs–436 µs) |
– |
Where each tool fits
Each tool below is written the way its users write it, and each brings something of its own. Times are walls (mean of the middle 3 of 5 runs) at the largest N; ratios are against the eagle graph arm (below 1 = less time). The fastest arm per N is listed under each table above.
eagle graph (device loop). The step written once as a hawk kernel; the whole loop runs on the device as one CUDA graph with a device-side stop guard, so a run is one launch with no host round trip per step. On this card: N = 1,000,000, spread S=100: 10.3 ms; spread S=1000: 94.4 ms; uniform S=1000: 105 ms. Fastest in 0 of 12 cells.
eagle graph + compaction. Adds an active-set map: the launch covers only the samples still running, rebuilt every 16 steps; it is made for batches that thin out (the spread configurations). On this card: N = 1,000,000, spread S=100: 5.45 ms (0.527× the eagle graph time); spread S=1000: 39.3 ms (0.417× the eagle graph time); uniform S=1000: 133 ms (1.27× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph + compaction + reorder. Adds a physical reorder when the live samples spread thin over their warps, so the mapped reads stay contiguous; the final restore to sample order is inside its wall. On this card: N = 1,000,000, spread S=100: 6.73 ms (0.651× the eagle graph time); spread S=1000: 39.3 ms (0.417× the eagle graph time); uniform S=1000: 134 ms (1.27× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph, K fused steps per launch. The same step with one decorator argument, @hawk.kernel(steps=16): each launch runs that many steps per sample with the state kept in registers, as Warp’s per-thread loop does, and the launches, guard checks and plane reads shrink by the same factor; its results are bit-identical to the eagle graph arm’s in every cell. K is chosen by hand here; the eagle_graph_auto arm picks it at run time. On this card: N = 1,000,000, spread S=100: 9.77 ms (0.945× the eagle graph time); spread S=1000: 93.9 ms (0.994× the eagle graph time); uniform S=1000: 106 ms (1.01× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph, K fused steps per launch + compaction. The K-steps kernel over the active-set map: the register-resident steps of the arm above, and the launches cover only the samples still running, so it fits batches that thin out. On this card: N = 1,000,000, spread S=100: 7.6 ms (0.735× the eagle graph time); spread S=1000: 28.5 ms (0.302× the eagle graph time); uniform S=1000: 106 ms (1.01× the eagle graph time). Fastest in 0 of 12 cells.
eagle auto (eagle picks the launch mode). The same step with @hawk.kernel(steps=”auto”) on the active-set kind, and nothing to tune: it picks the steps per launch itself after every launch, on the device, so a batch whose samples all run to the end gets Warp-like fusion of up to 64 steps per launch, and a batch that thins out gets shorter launches with the active set compacted after each one; no sample takes more steps than the cap. On this card: N = 1,000,000, spread S=100: 4.71 ms (0.456× the eagle graph time); spread S=1000: 25.3 ms (0.268× the eagle graph time); uniform S=1000: 103 ms (0.979× the eagle graph time). Fastest in 11 of 12 cells.
hawk + eagle persistent. The plain (not active-set) automatic kernel, forced through hawk’s persist entry: one launch, no graph, no map, no policy kernel, no cap at 64 – a grid of SMs blocks of 256 lanes each steal the next unfinished sample off a counter when their own finishes, so idle warps refill without ever rebuilding an index map; made for batches too large for one resident wave. On this card: N = 1,000,000, spread S=100: 4.76 ms (0.46× the eagle graph time); spread S=1000: 25.4 ms (0.269× the eagle graph time); uniform S=1000: 103 ms (0.98× the eagle graph time). Fastest in 0 of 12 cells.
eagle.simulate (eagle picks the launch mode). The same automatic kernel through eagle.simulate: the model, its state and its parameters by name, with no plan or plane binding to write; eagle deploys the kernel and builds the same device loop. On this card: N = 1,000,000, spread S=100: 4.73 ms (0.457× the eagle graph time); spread S=1000: 25.3 ms (0.269× the eagle graph time); uniform S=1000: 103 ms (0.979× the eagle graph time). Fastest in 1 of 12 cells.
eager launch loop. The same kernels launched from Python one step at a time; the host sees the state after every step. On this card: N = 1,000,000, spread S=100: 14.5 ms (1.4× the eagle graph time); spread S=1000: 146 ms (1.54× the eagle graph time); uniform S=1000: 158 ms (1.5× the eagle graph time). Fastest in 0 of 12 cells.
CuPy masked. NumPy-like array code on GPUs with no kernel to write; every sample is computed every step, so it fits dense batches where all samples run to the end (the uniform configuration). On this card: N = 1,000,000, spread S=100: 159 ms (15.4× the eagle graph time); spread S=1000: 1.6 s (17× the eagle graph time); uniform S=1000: 1.6 s (15.2× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch masked. The same array code in PyTorch tensors; it fits where the computation sits next to a PyTorch model or needs autograd. On this card: N = 1,000,000, spread S=100: 167 ms (16.1× the eagle graph time); spread S=1000: 1.68 s (17.8× the eagle graph time); uniform S=1000: 1.68 s (16× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch, CUDA graph of 16 steps. The same PyTorch code with its launch overhead removed by hand: 16 steps captured as one CUDA graph and replayed, one host check per replay, the masking unchanged. On this card: N = 1,000,000, spread S=100: 186 ms (18× the eagle graph time); spread S=1000: 1.68 s (17.8× the eagle graph time); uniform S=1000: 1.68 s (16× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch torch.compile (reduce-overhead). The PyTorch step under torch.compile(mode=”reduce-overhead”): Inductor-fused kernels replayed as a CUDA graph. On this card: N = 1,000,000, spread S=100: 25.6 ms (2.48× the eagle graph time); spread S=1000: 251 ms (2.66× the eagle graph time); uniform S=1000: 251 ms (2.38× the eagle graph time). Fastest in 0 of 12 cells.
JAX jit + vmap(while_loop). The whole loop compiled once, a per-sample while loop batched with vmap; the same code runs on CPUs and GPUs and differentiates with jax.grad. On this card: N = 1,000,000, spread S=100: 25 ms (2.42× the eagle graph time); spread S=1000: 243 ms (2.57× the eagle graph time); uniform S=1000: 242 ms (2.3× the eagle graph time). Fastest in 0 of 12 cells.
Warp per-thread kernel. Explicit per-thread kernels written in Python, maintained by NVIDIA and differentiable through wp.Tape; each thread runs its own sample and stops at its own step, so finished samples cost nothing. On this card: N = 1,000,000, spread S=100: 10.7 ms (1.04× the eagle graph time); spread S=1000: 105 ms (1.11× the eagle graph time); uniform S=1000: 119 ms (1.13× the eagle graph time). Fastest in 0 of 12 cells.
CPU OpenMP. The same hawk kernel on the host through eagle’s OpenMP team, for machines without a GPU. On this card: N = 1,000,000, spread S=100: 33 ms (3.2× the eagle graph time); spread S=1000: 310 ms (3.29× the eagle graph time); uniform S=1000: 352 ms (3.35× the eagle graph time). Fastest in 0 of 12 cells.
Compile time
Its own pass, never part of any wall. Each measurement is a fresh process: the imports and the GPU context start-up run untimed, then the timer spans the first call to a ready kernel (trace + code generation + compile + module load, or the cache lookup). Cold: every cache empty – hawk’s compile cache (HAWK_CACHE_DIR), CuPy’s kernel cache (CUPY_CACHE_DIR), the CUDA driver’s PTX cache (CUDA_CACHE_PATH), PyTorch Inductor’s and Triton’s caches (TORCHINDUCTOR_CACHE_DIR, TRITON_CACHE_DIR) and JAX’s persistent compilation cache (minimum compile time and entry size 0), each a fresh directory. Warm: a new process on the caches the first cold run populated. eagle arms: eagle.deploy of the arm’s hawk kernel, which returns once the device build is done (C++/CUDA emission, nvcc to PTX, driver load; eagle’s own small CuPy kernels compile beside it) while the host compiler builds the host side on another thread, and the arm’s set-up on a 64-sample batch (eagle’s graph built, one step run). CuPy: one masked step on 64 samples (every elementwise and reduction kernel compiled on first use). Warp: the kernel module’s code generation, compile and load on the first launch (64 samples), cached in WARP_CACHE_PATH. PyTorch torch.compile, where it runs: three steps at N (the first call traces and compiles, the second records the CUDA graph, the third replays it), per N (dynamic=False). JAX: lower + compile from abstract shapes, per N (XLA specialises on the shape). CPU OpenMP: the same eagle.deploy as the eagle graph arm and one host run, which waits for the host build. Medians of 3 cold and 3 warm processes.
arm |
compiled |
cold (median of 3) |
warm (median of 3) |
|---|---|---|---|
eagle graph (device loop) |
eagle.deploy of the hawk step kernel (the device build; the host side builds in the background), the device-loop graph |
1.26 s |
110 ms |
eagle graph + compaction |
eagle.deploy of the hawk active-set step kernel (the device build; the host side builds in the background), the compaction-loop graph |
989 ms |
95.9 ms |
eagle graph + compaction + reorder |
eagle.deploy of the hawk active-set step kernel (the device build; the host side builds in the background), the compaction + reorder loop graph |
1.1 s |
95.6 ms |
eagle graph, K fused steps per launch |
eagle.deploy of the hawk K-steps kernel (the device build; the host side builds in the background), the device-loop graph |
998 ms |
116 ms |
eagle graph, K fused steps per launch + compaction |
eagle.deploy of the hawk active-set K-steps kernel (the device build; the host side builds in the background), the compaction-loop graph |
1.01 s |
111 ms |
eagle auto (eagle picks the launch mode) |
eagle.deploy of the hawk active-set automatic kernel (the device build; the host side builds in the background), the policy kernel, the policy-loop graph |
1.07 s |
106 ms |
hawk + eagle persistent |
eagle.deploy of the hawk automatic kernel (the device build; the host side builds in the background), the persistent launch |
1.06 s |
109 ms |
eagle.simulate (eagle picks the launch mode) |
eagle.simulate’s deploy of the hawk active-set automatic kernel (the device build; the host side builds in the background), the policy kernel, the policy-loop graph |
1.08 s |
105 ms |
eager launch loop |
eagle.deploy of the hawk single step kernel (the device build; the host side builds in the background) |
868 ms |
90.6 ms |
CuPy masked |
CuPy’s elementwise and reduction kernels of one masked step |
1.05 s |
34.4 ms |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 1,000 |
6.31 s |
4.18 s |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 10,000 |
6.6 s |
4.16 s |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 100,000 |
6.69 s |
4.15 s |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 1,000,000 |
6.7 s |
4.14 s |
Warp per-thread kernel |
Warp kernel module (code generation, NVRTC compile, load) |
823 ms |
114 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 1,000 |
253 ms |
57.3 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 10,000 |
265 ms |
56.9 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 100,000 |
287 ms |
57.6 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 1,000,000 |
288 ms |
57.2 ms |
CPU OpenMP |
eagle.deploy of the hawk step kernel and its first host run, which waits for the host build |
1.78 s |
105 ms |
No compile step: PyTorch masked, PyTorch, CUDA graph of 16 steps (PyTorch’s eager kernels ship prebuilt; the CUDA-graph capture is per N and recorded per configuration as torch_graph_capture_s).
Memory method
A separate pass after the timing pass. Each (arm, N, configuration) runs in a fresh process: imports, the CUDA context created by one tiny device operation, then the baseline read, before the arm builds or allocates anything; then the arm’s kernels built and a warm-up run on a 64-sample batch (every compile happens here; JAX also compiles for shape N from abstract shapes), so every library’s pool reserved during the warm-up counts in the reading; CuPy’s pools freed and torch.cuda.empty_cache() called after the warm-up (Warp’s pool release threshold set to 0); then the batch’s inputs generated, the arm set up for N (eagle builds its graph, PyTorch captures its CUDA graph, whose private memory pool counts) and one full run with upload and download. Device memory = the memory the CUDA driver accounts to the process (NVML per-process used memory) after the run minus at the baseline. The caching allocators of CuPy, PyTorch and JAX are left at their defaults and are not trimmed after the run; each keeps the memory it reserved until trimmed, so the reading after the run is the allocator’s high-water reservation, the same measure for every arm. A disabled pool returns memory between operations, so its peak could only be caught by sampling, which misses short peaks; the held reservation needs no sampling. Caveats: the reading is what the process holds, so each allocator’s rounding and growth policy count (JAX’s allocator grows in regions that can exceed the request; PyTorch’s rounds blocks up; a captured CUDA graph keeps a private pool; Warp’s stream-ordered pool reserves in chunks of tens of MiB); the driver accounts in pages of about 2 MiB, so small-N rows read 0 or one page. The per-library counters (CuPy pool, torch.cuda.max_memory_reserved/allocated, JAX memory_stats) are recorded in the JSON as a cross-check. Host memory = peak resident set size above the host baseline (taken after the warm-up; /proc/self/clear_refs reset there, then VmHWM). Minimum = the state the workload needs, N × ((2 state + 2 float64 scalars: omega, zeta) × 8 + 4 bytes for the stop step as a 32-bit integer) = 36 bytes per sample; the factor is device memory / minimum.
Device memory the process already held at the baseline (CUDA context, loaded kernels and libraries, the warm-up’s leftovers; at N = 1,000,000, first configuration): eagle graph (device loop) 190 MiB; eagle graph + compaction 190 MiB; eagle graph + compaction + reorder 190 MiB; eagle graph, K fused steps per launch 190 MiB; eagle graph, K fused steps per launch + compaction 190 MiB; eagle auto (eagle picks the launch mode) 190 MiB; hawk + eagle persistent 190 MiB; eagle.simulate (eagle picks the launch mode) 190 MiB; eager launch loop 190 MiB; CuPy masked 190 MiB; PyTorch masked 190 MiB; PyTorch, CUDA graph of 16 steps 190 MiB; PyTorch torch.compile (reduce-overhead) 190 MiB; JAX jit + vmap(while_loop) 190 MiB; Warp per-thread kernel 190 MiB.
spread (up to 100 steps)
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (103 µs), then eagle.simulate (eagle picks the launch mode) (116 µs, 1.12× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (197 µs), then eagle.simulate (eagle picks the launch mode) (213 µs, 1.08× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (439 µs), then eagle.simulate (eagle picks the launch mode) (459 µs, 1.05× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (4.71 ms), then eagle.simulate (eagle picks the launch mode) (4.73 ms, 1× the time)
spread (up to 1000 steps)
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (430 µs), then eagle.simulate (eagle picks the launch mode) (441 µs, 1.03× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (1.19 ms), then eagle.simulate (eagle picks the launch mode) (1.2 ms, 1.01× the time)
N = 100,000: eagle.simulate (eagle picks the launch mode) (3.34 ms), then eagle auto (eagle picks the launch mode) (3.35 ms, 1× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (25.3 ms), then eagle.simulate (eagle picks the launch mode) (25.3 ms, 1× the time)
uniform (up to 1000 steps)
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (429 µs), then eagle.simulate (eagle picks the launch mode) (440 µs, 1.03× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (1.24 ms), then eagle.simulate (eagle picks the launch mode) (1.25 ms, 1.01× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (10.8 ms), then eagle.simulate (eagle picks the launch mode) (10.8 ms, 1× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (103 ms), then eagle.simulate (eagle picks the launch mode) (103 ms, 1× the time)
eagle_simulate against its hand-built twin eagle_graph_auto:
spread (up to 100 steps), N = 1,000: wall 1.12× the twin’s time
spread (up to 100 steps), N = 10,000: wall 1.08× the twin’s time
spread (up to 100 steps), N = 100,000: wall 1.05× the twin’s time
spread (up to 100 steps), N = 1,000,000: wall 1× the twin’s time
spread (up to 1000 steps), N = 1,000: wall 1.03× the twin’s time
spread (up to 1000 steps), N = 10,000: wall 1.01× the twin’s time
spread (up to 1000 steps), N = 100,000: wall 0.999× the twin’s time
spread (up to 1000 steps), N = 1,000,000: wall 1× the twin’s time
uniform (up to 1000 steps), N = 1,000: wall 1.03× the twin’s time
uniform (up to 1000 steps), N = 10,000: wall 1.01× the twin’s time
uniform (up to 1000 steps), N = 100,000: wall 1× the twin’s time
uniform (up to 1000 steps), N = 1,000,000: wall 1× the twin’s time
Run on: driver 580.178.04; CUDA runtime 12.9; host CPU Intel Xeon CPU @ 2.20GHz; compiler x86_64-conda-linux-gnu-g++ (conda-forge gcc 14.4.0-7) 14.4.0; Python 3.12.14; Warp 1.17.0, JAX 0.11.2, PyTorch 2.12.1+cu129, CuPy 14.2.0, NumPy 2.5.3.
N = 1,000,000
Log scale, from 256 µs to 110 ms.
N = 1,000,000
Log scale, from 935 µs to 1.08 s.
N = 1,000,000
Log scale, from 3.03 ms to 1.09 s.
Table of every arm and every N
Device: NVIDIA A100-SXM4-40GB (compute capability 8.0, 108 SMs). Peak FP64: 9.75e12 FLOP/s. Peak DRAM bandwidth: 1.56e12 B/s. Mean of the middle 3 of 5 runs of each arm (the highest and the lowest dropped), range of those 3 in parentheses.
Each arm’s block starts once the GPU is within 3 °C of its idle 33 °C and reads idle, or after 20 s; the SM clock is read right after each timed run and its range is in the last column.
The FP64 peak above is at the device’s reported maximum clock (1410 MHz). During the timed repetitions the SM clock read 1252 MHz; at that clock the FP64 peak is 8.66e12 FLOP/s, and every FP64 percentage below is 1.13× larger against it.
“% of issue peak” (the eagle arms only) counts DP instructions (DFMA/DMUL/DADD/DSETP/MUFU), not useful FLOPs: the step’s SASS has ? of them, against the device’s DP issue-slot throughput at the observed clock – a kernel that is already at the issue pipe’s limit can still read well under 100% of FP64 peak, because useful FLOPs/step is smaller than issued DP instructions/step.
eagle_graph checks its stop guard every step; eagle_graph_compact checks it once every 16 steps (the cadence its compaction runs on) and reads an active-set map built from it. The two rows below differ in both the cadence and the map; the compaction cadence sweep further down holds the cadence fixed at a few values and isolates the map alone.
PyTorch, CUDA graph of 16 steps captures 16 masked steps once as a torch.cuda.CUDAGraph over static buffers and replays it, with one host check of “any sample running” per replay: it removes the per-kernel launch overhead and keeps the masking (every sample is computed every step, finished ones held by torch.where, so the steps a replay runs past the last stop change nothing). JAX’s while loop is one compiled executable with no host check per step; batched by vmap, it runs until the slowest sample stops. Warp runs one thread per sample, each stopping at its own step.
Every arm runs each sample to its own stop and no further than max_steps (S): eagle and Warp enforce the cap per sample, JAX in its while-loop condition, PyTorch and CuPy by the outer step count (a batch-level cap); an arm that runs whole blocks of steps per launch or replay stops at the first block boundary at or after it.
What each arm needed to get there: Warp, a hand-written step counter and cap inside the kernel; JAX, a cap in the while_loop condition; eagle, the kernel source unchanged, with its loop shape, unrolling and device entry chosen by hawk + eagle (per device, per batch).
Cap fixture (checked before any timing counts): N = 4,096, 8 samples with stop step 2S = 256, S = 128; every arm (16) reported k = min(stop step, S) per sample and the same finished count (4,088): passed.
arm |
loop structure |
|---|---|
eagle graph (device loop) |
per launch over the batch, inside one CUDA graph: the plain decorator’s kernel, which finishes its own samples and runs the steps per launch eagle’s policy picks (8 to 64), guard checked on the device every launch |
eagle graph + compaction |
as the graph arm, the step launched over the active-set map only (recomputed every 16 steps) |
eagle graph + compaction + reorder |
as the compaction arm, plus a physical reorder of the per-sample planes when the live samples spread thin |
eagle graph, K fused steps per launch |
per launch over the batch, inside one CUDA graph: the step kernel runs up to K = 16 steps per sample with the state in registers, each sample leaving at its own stop step; guard checked on the device every launch |
eagle graph, K fused steps per launch + compaction |
as the K-steps arm, the launch over the active-set map only (recomputed every 64 steps = 4 launches) |
eagle auto (eagle picks the launch mode) |
the active-set kernel under eagle’s automatic policy, which picks the launch mode from the batch: one launch running every sample to its end for a small batch, the persistent launch above that, and the compaction loop (active-set map, steps per launch picked on the device) only when the caller asks for compaction or reordering; the row records the mode taken |
hawk + eagle persistent |
ONE launch, no graph, no map, no policy kernel: a grid sized from the SM count and the kernel’s occupancy, each lane fetching base + atomicAdd(counter, 1) whenever its sample is done, running the whole budget’s worth of steps per sample it picks up |
eagle.simulate (eagle picks the launch mode) |
as the automatic arm: eagle.simulate takes the kernel, its state and its parameters by name and eagle picks the launch mode the same way |
eager launch loop |
per step over the batch: one kernel launch from Python, a host read of the finished count every step |
CuPy masked |
per step over the batch: whole-batch array expressions, finished samples held by cupy.where, a host check every step; the cap is batch-level (the outer step loop runs at most max_steps) |
PyTorch masked |
per step over the batch: whole-batch tensor expressions, finished samples held by torch.where, a host check every step; the cap is batch-level (the outer step loop runs at most max_steps) |
PyTorch, CUDA graph of 16 steps |
as PyTorch masked, 16 steps captured as one CUDA graph over static buffers, replayed with one host check per replay |
PyTorch torch.compile (reduce-overhead) |
as PyTorch masked, the step compiled by Inductor and replayed as a CUDA graph, a host check every step |
JAX jit + vmap(while_loop) |
per step over the batch, inside one compiled executable: the vmapped while loop runs until the slowest sample stops or the step cap is reached, finished samples held by a select |
Warp per-thread kernel |
per sample: one launch, each thread loops over its own sample’s steps and stops at its own stop step or the step cap, whichever comes first |
CPU OpenMP |
per step over the batch on the host: one host-team launch per step, the kernel marks and counts its finished samples |
Spread stop steps (log-uniform in [1, 100]), up to 100 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
620 µs (595 µs–641 µs) |
– |
797 µs |
3.56e7 |
1.57e9 (0.016 %) |
1.18e9 (0.076 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
1095–1095 |
1,000 |
eagle graph + compaction |
757 µs (754 µs–759 µs) |
– |
1 ms |
2.91e7 |
1.28e9 (0.013 %) |
2.61e9 (0.17 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
1410–1410 |
1,000 |
eagle graph + compaction + reorder |
915 µs (910 µs–919 µs) |
– |
1.19 ms |
2.41e7 |
1.06e9 (0.011 %) |
2.62e9 (0.17 %) |
0 MiB (0×) |
2.1 MiB |
– |
– |
– |
1410–1410 |
1,000 |
eagle graph, K fused steps per launch |
276 µs (274 µs–279 µs) |
– |
447 µs |
7.99e7 |
3.52e9 (0.036 %) |
1.04e9 (0.067 %) |
0 MiB (0×) |
2.03 MiB |
– |
– |
– |
1410–1410 |
1,000 |
eagle graph, K fused steps per launch + compaction |
396 µs (385 µs–416 µs) |
– |
656 µs |
5.57e7 |
2.45e9 (0.025 %) |
6.97e8 (0.045 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
1095–1095 |
1,000 |
eagle auto (eagle picks the launch mode) |
90.8 µs (88.3 µs–92.5 µs) |
– |
303 µs |
2.43e8 |
1.07e10 (0.11 %) |
7.18e9 (0.46 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
1095–1095 |
1,000 |
hawk + eagle persistent |
155 µs (151 µs–159 µs) |
– |
306 µs |
1.43e8 |
6.27e9 (0.064 %) |
4.21e9 (0.27 %) |
0 MiB (0×) |
2.03 MiB |
– |
0.2098712531887755 |
– |
1095–1095 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
102 µs (100 µs–103 µs) |
– |
320 µs |
2.17e8 |
9.53e9 (0.098 %) |
6.40e9 (0.41 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
1095–1095 |
1,000 |
eager launch loop |
3.61 ms (3.59 ms–3.62 ms) |
– |
3.76 ms |
6.12e6 |
2.69e8 (0.0028 %) |
1.09e9 (0.07 %) |
0 MiB (0×) |
0.0312 MiB |
– |
– |
– |
1095–1095 |
1,000 |
CuPy masked |
67.7 ms (67.7 ms–67.7 ms) |
– |
67.9 ms |
3.26e5 |
1.43e7 (0.00015 %) |
1.33e9 (0.086 %) |
0 MiB (0×) |
5.01 MiB |
– |
– |
– |
1095–1095 |
1,000 |
PyTorch masked |
42.4 ms (42.3 ms–42.5 ms) |
– |
42.7 ms |
5.20e5 |
2.29e7 (0.00023 %) |
2.12e9 (0.14 %) |
74 MiB (2.16e+03×) |
0.0703 MiB |
– |
– |
– |
1185–1185 |
1,000 |
PyTorch, CUDA graph of 16 steps |
8.93 ms (8.92 ms–8.93 ms) |
– |
9.16 ms |
2.47e6 |
1.09e8 (0.0011 %) |
1.14e10 (0.73 %) |
102 MiB (2.97e+03×) |
0.164 MiB |
– |
– |
– |
1410–1410 |
1,000 |
PyTorch torch.compile (reduce-overhead) |
13.2 ms (13.2 ms–13.3 ms) |
– |
13.5 ms |
1.67e6 |
7.34e7 (0.00075 %) |
6.65e8 (0.043 %) |
96 MiB (2.8e+03×) |
32.8 MiB |
– |
– |
– |
1410–1410 |
1,000 |
JAX jit + vmap(while_loop) |
3.07 ms (3.05 ms–3.08 ms) |
– |
4.45 ms |
7.18e6 |
3.16e8 (0.0032 %) |
2.88e9 (0.18 %) |
134 MiB (3.9e+03×) |
0.0195 MiB |
– |
– |
– |
1410–1410 |
1,000 |
Warp per-thread kernel |
105 µs (104 µs–105 µs) |
– |
448 µs |
2.11e8 |
9.27e9 (0.095 %) |
6.11e8 (0.039 %) |
30 MiB (874×) |
0.0156 MiB |
– |
– |
– |
1095–1095 |
1,000 |
CPU OpenMP |
410 µs (406 µs–416 µs) |
– |
428 µs |
5.38e7 |
2.37e9 (–) |
9.56e9 (–) |
– |
0.25 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
645 µs (643 µs–647 µs) |
– |
966 µs |
3.54e8 |
1.56e10 (0.16 %) |
1.17e10 (0.75 %) |
0 MiB (0×) |
2.43 MiB |
– |
– |
– |
1095–1095 |
10,000 |
eagle graph + compaction |
798 µs (796 µs–800 µs) |
– |
1.2 ms |
2.86e8 |
1.26e10 (0.13 %) |
2.54e10 (1.6 %) |
0 MiB (0×) |
2.45 MiB |
– |
– |
– |
1410–1410 |
10,000 |
eagle graph + compaction + reorder |
1.26 ms (1.25 ms–1.28 ms) |
– |
1.71 ms |
1.81e8 |
7.97e9 (0.082 %) |
1.94e10 (1.3 %) |
0 MiB (0×) |
2.92 MiB |
– |
– |
– |
1410–1410 |
10,000 |
eagle graph, K fused steps per launch |
311 µs (302 µs–331 µs) |
– |
655 µs |
7.34e8 |
3.23e10 (0.33 %) |
9.23e9 (0.59 %) |
0 MiB (0×) |
2.43 MiB |
– |
– |
– |
1410–1410 |
10,000 |
eagle graph, K fused steps per launch + compaction |
384 µs (374 µs–402 µs) |
– |
797 µs |
5.96e8 |
2.62e10 (0.27 %) |
7.27e9 (0.47 %) |
0 MiB (0×) |
2.45 MiB |
– |
– |
– |
1410–1410 |
10,000 |
eagle auto (eagle picks the launch mode) |
117 µs (116 µs–117 µs) |
– |
495 µs |
1.96e9 |
8.63e10 (0.89 %) |
5.78e10 (3.7 %) |
0 MiB (0×) |
2.37 MiB |
– |
– |
– |
1095–1095 |
10,000 |
hawk + eagle persistent |
157 µs (154 µs–159 µs) |
– |
483 µs |
1.45e9 |
6.40e10 (0.66 %) |
4.29e10 (2.8 %) |
0 MiB (0×) |
2.31 MiB |
– |
0.21156370013627354 |
– |
1095–1095 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
129 µs (128 µs–131 µs) |
– |
521 µs |
1.77e9 |
7.80e10 (0.8 %) |
5.22e10 (3.4 %) |
0 MiB (0×) |
2.37 MiB |
– |
– |
– |
1095–1095 |
10,000 |
eager launch loop |
3.64 ms (3.62 ms–3.65 ms) |
– |
3.97 ms |
6.27e7 |
2.76e9 (0.028 %) |
1.08e10 (0.7 %) |
0 MiB (0×) |
0.121 MiB |
– |
– |
– |
1095–1095 |
10,000 |
CuPy masked |
67.5 ms (67.4 ms–67.6 ms) |
– |
68 ms |
3.39e6 |
1.49e8 (0.0015 %) |
1.33e10 (0.86 %) |
0 MiB (0×) |
5.62 MiB |
– |
– |
– |
1095–1095 |
10,000 |
PyTorch masked |
43.9 ms (43 ms–45.7 ms) |
– |
44.5 ms |
5.20e6 |
2.29e8 (0.0023 %) |
2.05e10 (1.3 %) |
74 MiB (216×) |
0.68 MiB |
– |
– |
– |
1125–1125 |
10,000 |
PyTorch, CUDA graph of 16 steps |
8.94 ms (8.94 ms–8.94 ms) |
– |
9.37 ms |
2.56e7 |
1.13e9 (0.012 %) |
1.13e11 (7.3 %) |
102 MiB (297×) |
0.719 MiB |
– |
– |
– |
1410–1410 |
10,000 |
PyTorch torch.compile (reduce-overhead) |
13.6 ms (13.5 ms–13.7 ms) |
– |
14.1 ms |
1.68e7 |
7.39e8 (0.0076 %) |
6.47e9 (0.42 %) |
96 MiB (280×) |
33.3 MiB |
– |
– |
– |
1410–1410 |
10,000 |
JAX jit + vmap(while_loop) |
3.44 ms (3.43 ms–3.44 ms) |
– |
5.34 ms |
6.65e7 |
2.93e9 (0.03 %) |
2.57e10 (1.7 %) |
134 MiB (390×) |
0.555 MiB |
– |
– |
– |
1095–1095 |
10,000 |
Warp per-thread kernel |
113 µs (113 µs–113 µs) |
– |
642 µs |
2.03e9 |
8.91e10 (0.91 %) |
5.67e9 (0.36 %) |
30 MiB (87.4×) |
0.688 MiB |
– |
– |
– |
1095–1095 |
10,000 |
CPU OpenMP |
798 µs (790 µs–811 µs) |
– |
875 µs |
2.86e8 |
1.26e10 (–) |
4.94e10 (–) |
– |
1.14 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
710 µs (704 µs–720 µs) |
– |
2.3 ms |
3.22e9 |
1.42e11 (1.5 %) |
1.06e11 (6.8 %) |
4 MiB (1.17×) |
7.86 MiB |
– |
– |
– |
1095–1095 |
100,000 |
eagle graph + compaction |
997 µs (995 µs–1 ms) |
– |
2.72 ms |
2.29e9 |
1.01e11 (1 %) |
2.03e11 (13 %) |
4 MiB (1.17×) |
7.88 MiB |
– |
– |
– |
1410–1410 |
100,000 |
eagle graph + compaction + reorder |
1.52 ms (1.51 ms–1.52 ms) |
– |
3.25 ms |
1.51e9 |
6.62e10 (0.68 %) |
1.61e11 (10 %) |
8 MiB (2.33×) |
8.5 MiB |
– |
– |
– |
1410–1410 |
100,000 |
eagle graph, K fused steps per launch |
343 µs (343 µs–344 µs) |
– |
1.97 ms |
6.67e9 |
2.93e11 (3 %) |
8.38e10 (5.4 %) |
4 MiB (1.17×) |
7.86 MiB |
– |
– |
– |
1410–1410 |
100,000 |
eagle graph, K fused steps per launch + compaction |
482 µs (476 µs–486 µs) |
– |
2.18 ms |
4.74e9 |
2.09e11 (2.1 %) |
5.79e10 (3.7 %) |
4 MiB (1.17×) |
7.88 MiB |
– |
– |
– |
1110–1110 |
100,000 |
eagle auto (eagle picks the launch mode) |
145 µs (145 µs–146 µs) |
– |
1.8 ms |
1.57e10 |
6.93e11 (7.1 %) |
4.64e11 (30 %) |
4 MiB (1.17×) |
7.59 MiB |
– |
– |
– |
1095–1095 |
100,000 |
hawk + eagle persistent |
204 µs (199 µs–207 µs) |
– |
1.82 ms |
1.12e10 |
4.94e11 (5.1 %) |
3.31e11 (21 %) |
4 MiB (1.17×) |
7.58 MiB |
– |
0.3725825992089435 |
– |
1095–1095 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
161 µs (158 µs–165 µs) |
– |
1.8 ms |
1.42e10 |
6.27e11 (6.4 %) |
4.20e11 (27 %) |
4 MiB (1.17×) |
7.61 MiB |
– |
– |
– |
1095–1095 |
100,000 |
eager launch loop |
3.68 ms (3.67 ms–3.69 ms) |
– |
5.29 ms |
6.22e8 |
2.74e10 (0.28 %) |
1.07e11 (6.9 %) |
4 MiB (1.17×) |
5.39 MiB |
– |
– |
– |
1095–1095 |
100,000 |
CuPy masked |
68 ms (67.8 ms–68.1 ms) |
– |
70.1 ms |
3.37e7 |
1.48e9 (0.015 %) |
1.33e11 (8.5 %) |
22 MiB (6.41×) |
9.39 MiB |
– |
– |
– |
1095–1095 |
100,000 |
PyTorch masked |
45.5 ms (45.4 ms–45.5 ms) |
– |
47.8 ms |
5.03e7 |
2.21e9 (0.023 %) |
1.98e11 (13 %) |
98 MiB (28.5×) |
6.86 MiB |
– |
– |
– |
1275–1275 |
100,000 |
PyTorch, CUDA graph of 16 steps |
10.3 ms (10.3 ms–10.3 ms) |
– |
12.4 ms |
2.23e8 |
9.80e9 (0.1 %) |
9.86e11 (63 %) |
124 MiB (36.1×) |
6.87 MiB |
– |
– |
– |
1410–1410 |
100,000 |
PyTorch torch.compile (reduce-overhead) |
13.8 ms (13.7 ms–13.8 ms) |
– |
15.9 ms |
1.66e8 |
7.32e9 (0.075 %) |
6.39e10 (4.1 %) |
102 MiB (29.7×) |
36.9 MiB |
– |
– |
– |
1410–1410 |
100,000 |
JAX jit + vmap(while_loop) |
3.54 ms (3.5 ms–3.57 ms) |
– |
6.21 ms |
6.47e8 |
2.85e10 (0.29 %) |
2.50e11 (16 %) |
134 MiB (39×) |
10.9 MiB |
– |
– |
– |
1410–1410 |
100,000 |
Warp per-thread kernel |
170 µs (170 µs–171 µs) |
– |
2.28 ms |
1.35e10 |
5.92e11 (6.1 %) |
3.76e10 (2.4 %) |
30 MiB (8.74×) |
6.87 MiB |
– |
– |
– |
1095–1095 |
100,000 |
CPU OpenMP |
3.42 ms (3.41 ms–3.43 ms) |
– |
4.18 ms |
6.68e8 |
2.94e10 (–) |
1.15e11 (–) |
– |
9.93 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
1.38 ms (1.36 ms–1.4 ms) |
– |
14.9 ms |
1.67e10 |
7.35e11 (7.5 %) |
6.03e11 (39 %) |
40 MiB (1.17×) |
62.8 MiB |
– |
– |
– |
1095–1095 |
1,000,000 |
eagle graph + compaction |
3.77 ms (3.76 ms–3.79 ms) |
– |
17.4 ms |
6.09e9 |
2.68e11 (2.8 %) |
5.39e11 (35 %) |
56 MiB (1.63×) |
62.8 MiB |
– |
– |
– |
1320–1410 |
1,000,000 |
eagle graph + compaction + reorder |
3.23 ms (3.22 ms–3.24 ms) |
– |
16.9 ms |
7.12e9 |
3.13e11 (3.2 %) |
7.61e11 (49 %) |
84 MiB (2.45×) |
63.1 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
eagle graph, K fused steps per launch |
880 µs (874 µs–890 µs) |
– |
14.3 ms |
2.61e10 |
1.15e12 (12 %) |
3.27e11 (21 %) |
40 MiB (1.17×) |
62.8 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
959 µs (939 µs–984 µs) |
– |
14.6 ms |
2.40e10 |
1.05e12 (11 %) |
2.91e11 (19 %) |
56 MiB (1.63×) |
62.8 MiB |
– |
– |
– |
1275–1290 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
390 µs (389 µs–391 µs) |
– |
14.1 ms |
5.89e10 |
2.59e12 (27 %) |
1.74e12 (1.1e+02 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
1095–1095 |
1,000,000 |
hawk + eagle persistent |
536 µs (534 µs–539 µs) |
– |
14 ms |
4.29e10 |
1.89e12 (19 %) |
1.26e12 (81 %) |
40 MiB (1.17×) |
59.9 MiB |
– |
0.47417527299096307 |
– |
1095–1095 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
410 µs (403 µs–414 µs) |
– |
13.9 ms |
5.61e10 |
2.47e12 (25 %) |
1.65e12 (1.1e+02 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
1095–1095 |
1,000,000 |
eager launch loop |
6.43 ms (6.42 ms–6.45 ms) |
– |
20 ms |
3.58e9 |
1.57e11 (1.6 %) |
6.13e11 (39 %) |
40 MiB (1.17×) |
57.8 MiB |
– |
– |
– |
1125–1155 |
1,000,000 |
CuPy masked |
68.7 ms (68.5 ms–69.2 ms) |
– |
84.5 ms |
3.35e8 |
1.47e10 (0.15 %) |
1.31e12 (84 %) |
192 MiB (5.59×) |
68.7 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
PyTorch masked |
60.7 ms (60.7 ms–60.7 ms) |
– |
83.7 ms |
3.79e8 |
1.67e10 (0.17 %) |
1.48e12 (95 %) |
334 MiB (9.73×) |
68.7 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
59.2 ms (59.2 ms–59.3 ms) |
– |
82.3 ms |
3.88e8 |
1.71e10 (0.18 %) |
1.71e12 (1.1e+02 %) |
362 MiB (10.5×) |
68.7 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
PyTorch torch.compile (reduce-overhead) |
17.2 ms (17.2 ms–17.2 ms) |
– |
40.3 ms |
1.34e9 |
5.89e10 (0.6 %) |
5.12e11 (33 %) |
176 MiB (5.13×) |
77.5 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
JAX jit + vmap(while_loop) |
9.5 ms (9.47 ms–9.54 ms) |
– |
24.3 ms |
2.42e9 |
1.07e11 (1.1 %) |
9.30e11 (60 %) |
134 MiB (3.9×) |
101 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
Warp per-thread kernel |
604 µs (603 µs–606 µs) |
– |
30.2 ms |
3.81e10 |
1.67e12 (17 %) |
1.06e11 (6.8 %) |
62 MiB (1.81×) |
68.7 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
CPU OpenMP |
29.8 ms (29.8 ms–29.8 ms) |
– |
45 ms |
7.73e8 |
3.40e10 (–) |
1.33e11 (–) |
– |
97.5 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (90.8 µs), then eagle.simulate (eagle picks the launch mode) (102 µs, 1.12× the time)
N = 10,000: Warp per-thread kernel (113 µs), then eagle auto (eagle picks the launch mode) (117 µs, 1.03× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (145 µs), then eagle.simulate (eagle picks the launch mode) (161 µs, 1.11× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (390 µs), then eagle.simulate (eagle picks the launch mode) (410 µs, 1.05× the time)
Spread stop steps (log-uniform in [10, 1000]), up to 1000 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
834 µs (825 µs–849 µs) |
– |
1 ms |
2.52e8 |
1.11e10 (0.11 %) |
7.81e9 (0.5 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
1410–1410 |
1,000 |
eagle graph + compaction |
4.59 ms (4.58 ms–4.6 ms) |
– |
4.84 ms |
4.58e7 |
2.02e9 (0.021 %) |
3.45e9 (0.22 %) |
0 MiB (0×) |
2.05 MiB |
– |
– |
– |
1410–1410 |
1,000 |
eagle graph + compaction + reorder |
4.79 ms (4.78 ms–4.8 ms) |
– |
5.07 ms |
4.39e7 |
1.93e9 (0.02 %) |
3.44e9 (0.22 %) |
0 MiB (0×) |
2.1 MiB |
– |
– |
– |
1410–1410 |
1,000 |
eagle graph, K fused steps per launch |
791 µs (790 µs–792 µs) |
– |
964 µs |
2.66e8 |
1.17e10 (0.12 %) |
3.11e9 (0.2 %) |
0 MiB (0×) |
2.03 MiB |
– |
– |
– |
1410–1410 |
1,000 |
eagle graph, K fused steps per launch + compaction |
973 µs (967 µs–983 µs) |
– |
1.23 ms |
2.16e8 |
9.50e9 (0.097 %) |
1.46e9 (0.094 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
1410–1410 |
1,000 |
eagle auto (eagle picks the launch mode) |
145 µs (144 µs–147 µs) |
– |
358 µs |
1.45e9 |
6.36e10 (0.65 %) |
4.07e10 (2.6 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
1410–1410 |
1,000 |
hawk + eagle persistent |
281 µs (272 µs–298 µs) |
– |
435 µs |
7.48e8 |
3.29e10 (0.34 %) |
2.11e10 (1.4 %) |
0 MiB (0×) |
2.03 MiB |
– |
0.22980196323115862 |
– |
1410–1410 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
156 µs (154 µs–159 µs) |
– |
373 µs |
1.35e9 |
5.92e10 (0.61 %) |
3.79e10 (2.4 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
1410–1410 |
1,000 |
eager launch loop |
35.5 ms (35.4 ms–35.7 ms) |
– |
35.7 ms |
5.92e6 |
2.61e8 (0.0027 %) |
1.10e9 (0.07 %) |
0 MiB (0×) |
0.0312 MiB |
– |
– |
– |
1095–1095 |
1,000 |
CuPy masked |
673 ms (673 ms–674 ms) |
– |
674 ms |
3.12e5 |
1.37e7 (0.00014 %) |
1.34e9 (0.086 %) |
0 MiB (0×) |
4.89 MiB |
– |
– |
– |
1095–1095 |
1,000 |
PyTorch masked |
424 ms (424 ms–424 ms) |
– |
424 ms |
4.96e5 |
2.18e7 (0.00022 %) |
2.12e9 (0.14 %) |
74 MiB (2.16e+03×) |
0.0703 MiB |
– |
– |
– |
1185–1185 |
1,000 |
PyTorch, CUDA graph of 16 steps |
79.1 ms (79.1 ms–79.1 ms) |
– |
79.4 ms |
2.66e6 |
1.17e8 (0.0012 %) |
1.15e10 (0.74 %) |
102 MiB (2.97e+03×) |
0.156 MiB |
– |
– |
– |
1410–1410 |
1,000 |
PyTorch torch.compile (reduce-overhead) |
132 ms (132 ms–132 ms) |
– |
132 ms |
1.59e6 |
7.00e7 (0.00072 %) |
6.66e8 (0.043 %) |
96 MiB (2.8e+03×) |
32.5 MiB |
– |
– |
– |
1410–1410 |
1,000 |
JAX jit + vmap(while_loop) |
28.3 ms (28.3 ms–28.4 ms) |
– |
30.3 ms |
7.42e6 |
3.26e8 (0.0033 %) |
3.11e9 (0.2 %) |
134 MiB (3.9e+03×) |
0.00781 MiB |
– |
– |
– |
1095–1095 |
1,000 |
Warp per-thread kernel |
272 µs (262 µs–279 µs) |
– |
632 µs |
7.74e8 |
3.40e10 (0.35 %) |
2.36e8 (0.015 %) |
30 MiB (874×) |
0.0117 MiB |
– |
– |
– |
1095–1095 |
1,000 |
CPU OpenMP |
795 µs (787 µs–800 µs) |
– |
815 µs |
2.64e8 |
1.16e10 (–) |
4.89e10 (–) |
– |
0.25 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
894 µs (893 µs–896 µs) |
– |
1.22 ms |
2.44e9 |
1.07e11 (1.1 %) |
7.54e10 (4.8 %) |
0 MiB (0×) |
2.43 MiB |
– |
– |
– |
1410–1410 |
10,000 |
eagle graph + compaction |
5.06 ms (5.06 ms–5.07 ms) |
– |
5.48 ms |
4.31e8 |
1.90e10 (0.19 %) |
3.23e10 (2.1 %) |
0 MiB (0×) |
2.45 MiB |
– |
– |
– |
1410–1410 |
10,000 |
eagle graph + compaction + reorder |
5.91 ms (5.91 ms–5.91 ms) |
– |
6.37 ms |
3.69e8 |
1.62e10 (0.17 %) |
2.86e10 (1.8 %) |
0 MiB (0×) |
2.91 MiB |
– |
– |
– |
1410–1410 |
10,000 |
eagle graph, K fused steps per launch |
838 µs (835 µs–842 µs) |
– |
1.18 ms |
2.60e9 |
1.15e11 (1.2 %) |
2.95e10 (1.9 %) |
0 MiB (0×) |
2.43 MiB |
– |
– |
– |
1410–1410 |
10,000 |
eagle graph, K fused steps per launch + compaction |
1.03 ms (1.03 ms–1.03 ms) |
– |
1.44 ms |
2.13e9 |
9.36e10 (0.96 %) |
1.42e10 (0.91 %) |
0 MiB (0×) |
2.44 MiB |
– |
– |
– |
1410–1410 |
10,000 |
eagle auto (eagle picks the launch mode) |
231 µs (229 µs–233 µs) |
– |
615 µs |
9.44e9 |
4.15e11 (4.3 %) |
2.66e11 (17 %) |
0 MiB (0×) |
2.38 MiB |
– |
– |
– |
1410–1410 |
10,000 |
hawk + eagle persistent |
292 µs (291 µs–294 µs) |
– |
606 µs |
7.47e9 |
3.29e11 (3.4 %) |
2.10e11 (14 %) |
0 MiB (0×) |
2.31 MiB |
– |
0.2384722378329774 |
– |
1350–1350 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
244 µs (242 µs–246 µs) |
– |
641 µs |
8.96e9 |
3.94e11 (4 %) |
2.52e11 (16 %) |
0 MiB (0×) |
2.37 MiB |
– |
– |
– |
1350–1350 |
10,000 |
eager launch loop |
35.9 ms (35.9 ms–36 ms) |
– |
36.3 ms |
6.07e7 |
2.67e9 (0.027 %) |
1.09e10 (0.7 %) |
0 MiB (0×) |
0.148 MiB |
– |
– |
– |
1095–1095 |
10,000 |
CuPy masked |
675 ms (672 ms–678 ms) |
– |
676 ms |
3.23e6 |
1.42e8 (0.0015 %) |
1.33e10 (0.86 %) |
0 MiB (0×) |
5.62 MiB |
– |
– |
– |
1095–1095 |
10,000 |
PyTorch masked |
426 ms (424 ms–427 ms) |
– |
426 ms |
5.13e6 |
2.26e8 (0.0023 %) |
2.12e10 (1.4 %) |
74 MiB (216×) |
0.688 MiB |
– |
– |
– |
1155–1155 |
10,000 |
PyTorch, CUDA graph of 16 steps |
79.6 ms (79.5 ms–79.6 ms) |
– |
80.1 ms |
2.74e7 |
1.21e9 (0.012 %) |
1.15e11 (7.4 %) |
102 MiB (297×) |
0.719 MiB |
– |
– |
– |
1410–1410 |
10,000 |
PyTorch torch.compile (reduce-overhead) |
131 ms (131 ms–131 ms) |
– |
132 ms |
1.67e7 |
7.33e8 (0.0075 %) |
6.71e9 (0.43 %) |
96 MiB (280×) |
33.3 MiB |
– |
– |
– |
1410–1410 |
10,000 |
JAX jit + vmap(while_loop) |
31.4 ms (31.4 ms–31.5 ms) |
– |
33.5 ms |
6.94e7 |
3.05e9 (0.031 %) |
2.80e10 (1.8 %) |
134 MiB (390×) |
0.469 MiB |
– |
– |
– |
1125–1125 |
10,000 |
Warp per-thread kernel |
269 µs (268 µs–271 µs) |
– |
816 µs |
8.11e9 |
3.57e11 (3.7 %) |
2.38e9 (0.15 %) |
30 MiB (87.4×) |
0.691 MiB |
– |
– |
– |
1125–1125 |
10,000 |
CPU OpenMP |
3.68 ms (3.67 ms–3.69 ms) |
– |
3.77 ms |
5.93e8 |
2.61e10 (–) |
1.06e11 (–) |
– |
1.17 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
1.31 ms (1.28 ms–1.33 ms) |
– |
2.91 ms |
1.65e10 |
7.26e11 (7.4 %) |
5.10e11 (33 %) |
4 MiB (1.17×) |
7.86 MiB |
– |
– |
– |
1410–1410 |
100,000 |
eagle graph + compaction |
7.12 ms (7.11 ms–7.13 ms) |
– |
8.84 ms |
3.04e9 |
1.34e11 (1.4 %) |
2.28e11 (15 %) |
4 MiB (1.17×) |
7.88 MiB |
– |
– |
– |
1410–1410 |
100,000 |
eagle graph + compaction + reorder |
6.92 ms (6.91 ms–6.92 ms) |
– |
8.67 ms |
3.13e9 |
1.38e11 (1.4 %) |
2.43e11 (16 %) |
8 MiB (2.33×) |
8.5 MiB |
– |
– |
– |
1410–1410 |
100,000 |
eagle graph, K fused steps per launch |
1.33 ms (1.32 ms–1.33 ms) |
– |
2.91 ms |
1.63e10 |
7.18e11 (7.4 %) |
1.86e11 (12 %) |
4 MiB (1.17×) |
7.86 MiB |
– |
– |
– |
1410–1410 |
100,000 |
eagle graph, K fused steps per launch + compaction |
1.34 ms (1.33 ms–1.34 ms) |
– |
3.01 ms |
1.62e10 |
7.12e11 (7.3 %) |
1.08e11 (7 %) |
4 MiB (1.17×) |
7.88 MiB |
– |
– |
– |
1410–1410 |
100,000 |
eagle auto (eagle picks the launch mode) |
350 µs (345 µs–354 µs) |
– |
2.02 ms |
6.18e10 |
2.72e12 (28 %) |
1.74e12 (1.1e+02 %) |
4 MiB (1.17×) |
7.62 MiB |
– |
– |
– |
1410–1410 |
100,000 |
hawk + eagle persistent |
391 µs (391 µs–391 µs) |
– |
1.97 ms |
5.54e10 |
2.44e12 (25 %) |
1.56e12 (1e+02 %) |
4 MiB (1.17×) |
7.56 MiB |
– |
0.5649979515277368 |
– |
1410–1410 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
434 µs (431 µs–439 µs) |
– |
2.1 ms |
4.99e10 |
2.20e12 (23 %) |
1.41e12 (90 %) |
4 MiB (1.17×) |
7.64 MiB |
– |
– |
– |
1095–1095 |
100,000 |
eager launch loop |
36.3 ms (36.2 ms–36.5 ms) |
– |
38.1 ms |
5.96e8 |
2.62e10 (0.27 %) |
1.08e11 (6.9 %) |
4 MiB (1.17×) |
5.38 MiB |
– |
– |
– |
1095–1095 |
100,000 |
CuPy masked |
679 ms (674 ms–683 ms) |
– |
681 ms |
3.19e7 |
1.40e9 (0.014 %) |
1.33e11 (8.5 %) |
22 MiB (6.41×) |
9.39 MiB |
– |
– |
– |
1095–1095 |
100,000 |
PyTorch masked |
448 ms (446 ms–449 ms) |
– |
450 ms |
4.84e7 |
2.13e9 (0.022 %) |
2.01e11 (13 %) |
98 MiB (28.5×) |
6.86 MiB |
– |
– |
– |
1320–1320 |
100,000 |
PyTorch, CUDA graph of 16 steps |
91.6 ms (91.6 ms–91.7 ms) |
– |
94 ms |
2.36e8 |
1.04e10 (0.11 %) |
9.96e11 (64 %) |
124 MiB (36.1×) |
6.87 MiB |
– |
– |
– |
1410–1410 |
100,000 |
PyTorch torch.compile (reduce-overhead) |
131 ms (130 ms–131 ms) |
– |
133 ms |
1.66e8 |
7.30e9 (0.075 %) |
6.74e10 (4.3 %) |
102 MiB (29.7×) |
36.9 MiB |
– |
– |
– |
1410–1410 |
100,000 |
JAX jit + vmap(while_loop) |
30.4 ms (30.2 ms–30.5 ms) |
– |
33.6 ms |
7.11e8 |
3.13e10 (0.32 %) |
2.89e11 (19 %) |
134 MiB (39×) |
10.8 MiB |
– |
– |
– |
1335–1365 |
100,000 |
Warp per-thread kernel |
659 µs (648 µs–671 µs) |
– |
2.82 ms |
3.28e10 |
1.45e12 (15 %) |
9.71e9 (0.62 %) |
30 MiB (8.74×) |
6.87 MiB |
– |
– |
– |
1365–1365 |
100,000 |
CPU OpenMP |
29 ms (28.9 ms–29 ms) |
– |
30 ms |
7.48e8 |
3.29e10 (–) |
1.35e11 (–) |
– |
9.93 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
5.33 ms (5.32 ms–5.33 ms) |
– |
18.7 ms |
4.06e10 |
1.79e12 (18 %) |
1.26e12 (81 %) |
40 MiB (1.17×) |
62.8 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
eagle graph + compaction |
32.6 ms (32.6 ms–32.7 ms) |
– |
46.3 ms |
6.63e9 |
2.92e11 (3 %) |
4.97e11 (32 %) |
56 MiB (1.63×) |
62.8 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
eagle graph + compaction + reorder |
17.2 ms (17.2 ms–17.2 ms) |
– |
30.7 ms |
1.26e10 |
5.52e11 (5.7 %) |
9.74e11 (63 %) |
84 MiB (2.45×) |
63.1 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
eagle graph, K fused steps per launch |
5.55 ms (5.55 ms–5.55 ms) |
– |
19 ms |
3.90e10 |
1.71e12 (18 %) |
4.45e11 (29 %) |
40 MiB (1.17×) |
62.8 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
3.54 ms (3.54 ms–3.55 ms) |
– |
17 ms |
6.10e10 |
2.69e12 (28 %) |
4.09e11 (26 %) |
56 MiB (1.63×) |
62.8 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
1.48 ms (1.48 ms–1.48 ms) |
– |
14.9 ms |
1.46e11 |
6.44e12 (66 %) |
4.12e12 (2.7e+02 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
hawk + eagle persistent |
1.53 ms (1.53 ms–1.53 ms) |
– |
14.9 ms |
1.41e11 |
6.22e12 (64 %) |
3.98e12 (2.6e+02 %) |
40 MiB (1.17×) |
59.9 MiB |
– |
0.8243109163213885 |
– |
1410–1410 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
1.5 ms (1.49 ms–1.5 ms) |
– |
14.9 ms |
1.45e11 |
6.36e12 (65 %) |
4.07e12 (2.6e+02 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
eager launch loop |
59.4 ms (59.3 ms–59.5 ms) |
– |
73 ms |
3.64e9 |
1.60e11 (1.6 %) |
6.58e11 (42 %) |
40 MiB (1.17×) |
57.8 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
CuPy masked |
673 ms (672 ms–673 ms) |
– |
688 ms |
3.22e8 |
1.41e10 (0.15 %) |
1.34e12 (86 %) |
192 MiB (5.59×) |
68.7 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
PyTorch masked |
608 ms (606 ms–611 ms) |
– |
631 ms |
3.56e8 |
1.57e10 (0.16 %) |
1.48e12 (95 %) |
334 MiB (9.73×) |
68.7 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
532 ms (532 ms–532 ms) |
– |
556 ms |
4.06e8 |
1.79e10 (0.18 %) |
1.71e12 (1.1e+02 %) |
362 MiB (10.5×) |
68.7 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
PyTorch torch.compile (reduce-overhead) |
167 ms (167 ms–167 ms) |
– |
190 ms |
1.29e9 |
5.69e10 (0.58 %) |
5.26e11 (34 %) |
176 MiB (5.13×) |
77.4 MiB |
– |
– |
– |
1095–1095 |
1,000,000 |
JAX jit + vmap(while_loop) |
90.7 ms (90.3 ms–91 ms) |
– |
106 ms |
2.39e9 |
1.05e11 (1.1 %) |
9.71e11 (62 %) |
134 MiB (3.9×) |
101 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
Warp per-thread kernel |
5.94 ms (5.94 ms–5.95 ms) |
– |
36.3 ms |
3.64e10 |
1.60e12 (16 %) |
1.08e10 (0.69 %) |
62 MiB (1.81×) |
68.7 MiB |
– |
– |
– |
1095–1095 |
1,000,000 |
CPU OpenMP |
281 ms (281 ms–281 ms) |
– |
293 ms |
7.70e8 |
3.39e10 (–) |
1.39e11 (–) |
– |
97.5 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (145 µs), then eagle.simulate (eagle picks the launch mode) (156 µs, 1.07× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (231 µs), then eagle.simulate (eagle picks the launch mode) (244 µs, 1.05× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (350 µs), then hawk + eagle persistent (391 µs, 1.12× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (1.48 ms), then eagle.simulate (eagle picks the launch mode) (1.5 ms, 1.01× the time)
Uniform stop step: every sample runs 1000 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
855 µs (853 µs–858 µs) |
– |
1.03 ms |
1.17e9 |
5.15e10 (0.53 %) |
3.35e10 (2.2 %) |
0 MiB (0×) |
2.03 MiB |
– |
– |
– |
1410–1410 |
1,000 |
eagle graph + compaction |
3.72 ms (3.72 ms–3.73 ms) |
– |
3.99 ms |
2.69e8 |
1.18e10 (0.12 %) |
1.75e10 (1.1 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
1410–1410 |
1,000 |
eagle graph + compaction + reorder |
3.7 ms (3.7 ms–3.7 ms) |
– |
3.99 ms |
2.70e8 |
1.19e10 (0.12 %) |
1.76e10 (1.1 %) |
0 MiB (0×) |
2.11 MiB |
– |
– |
– |
1410–1410 |
1,000 |
eagle graph, K fused steps per launch |
775 µs (766 µs–779 µs) |
– |
958 µs |
1.29e9 |
5.68e10 (0.58 %) |
4.96e9 (0.32 %) |
0 MiB (0×) |
2.03 MiB |
– |
– |
– |
1410–1410 |
1,000 |
eagle graph, K fused steps per launch + compaction |
691 µs (690 µs–691 µs) |
– |
944 µs |
1.45e9 |
6.37e10 (0.65 %) |
6.03e9 (0.39 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
1410–1410 |
1,000 |
eagle auto (eagle picks the launch mode) |
156 µs (154 µs–158 µs) |
– |
392 µs |
6.42e9 |
2.82e11 (2.9 %) |
1.80e11 (12 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
1410–1410 |
1,000 |
hawk + eagle persistent |
286 µs (282 µs–294 µs) |
– |
451 µs |
3.49e9 |
1.54e11 (1.6 %) |
9.80e10 (6.3 %) |
0 MiB (0×) |
2.03 MiB |
– |
0.95367431640625 |
– |
1410–1410 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
174 µs (163 µs–184 µs) |
– |
401 µs |
5.76e9 |
2.53e11 (2.6 %) |
1.61e11 (10 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
1410–1410 |
1,000 |
eager launch loop |
35.1 ms (35.1 ms–35.2 ms) |
– |
35.4 ms |
2.85e7 |
1.25e9 (0.013 %) |
1.74e9 (0.11 %) |
0 MiB (0×) |
0.0312 MiB |
– |
– |
– |
1410–1410 |
1,000 |
CuPy masked |
678 ms (676 ms–681 ms) |
– |
678 ms |
1.48e6 |
6.49e7 (0.00067 %) |
1.33e9 (0.086 %) |
0 MiB (0×) |
5.1 MiB |
– |
– |
– |
1095–1095 |
1,000 |
PyTorch masked |
425 ms (424 ms–425 ms) |
– |
425 ms |
2.36e6 |
1.04e8 (0.0011 %) |
2.12e9 (0.14 %) |
74 MiB (2.16e+03×) |
0.0703 MiB |
– |
– |
– |
1185–1185 |
1,000 |
PyTorch, CUDA graph of 16 steps |
79.7 ms (79.6 ms–79.7 ms) |
– |
80 ms |
1.26e7 |
5.52e8 (0.0057 %) |
1.15e10 (0.74 %) |
102 MiB (2.97e+03×) |
0.16 MiB |
– |
– |
– |
1410–1410 |
1,000 |
PyTorch torch.compile (reduce-overhead) |
129 ms (129 ms–130 ms) |
– |
130 ms |
7.72e6 |
3.40e8 (0.0035 %) |
6.80e8 (0.044 %) |
96 MiB (2.8e+03×) |
32.6 MiB |
– |
– |
– |
1410–1410 |
1,000 |
JAX jit + vmap(while_loop) |
28.3 ms (28.3 ms–28.3 ms) |
– |
30.2 ms |
3.53e7 |
1.55e9 (0.016 %) |
3.11e9 (0.2 %) |
134 MiB (3.9e+03×) |
0.0195 MiB |
– |
– |
– |
1095–1095 |
1,000 |
Warp per-thread kernel |
274 µs (272 µs–275 µs) |
– |
658 µs |
3.65e9 |
1.61e11 (1.6 %) |
2.34e8 (0.015 %) |
30 MiB (874×) |
0.0156 MiB |
– |
– |
– |
1095–1095 |
1,000 |
CPU OpenMP |
718 µs (707 µs–723 µs) |
– |
739 µs |
1.39e9 |
6.13e10 (–) |
8.50e10 (–) |
– |
0.242 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
902 µs (897 µs–905 µs) |
– |
1.23 ms |
1.11e10 |
4.88e11 (5 %) |
3.17e11 (20 %) |
0 MiB (0×) |
2.42 MiB |
– |
– |
– |
1410–1410 |
10,000 |
eagle graph + compaction |
3.78 ms (3.78 ms–3.78 ms) |
– |
4.2 ms |
2.64e9 |
1.16e11 (1.2 %) |
1.73e11 (11 %) |
0 MiB (0×) |
2.44 MiB |
– |
– |
– |
1410–1410 |
10,000 |
eagle graph + compaction + reorder |
4.4 ms (4.39 ms–4.41 ms) |
– |
4.86 ms |
2.27e9 |
1.00e11 (1 %) |
1.48e11 (9.5 %) |
0 MiB (0×) |
2.91 MiB |
– |
– |
– |
1410–1410 |
10,000 |
eagle graph, K fused steps per launch |
896 µs (891 µs–901 µs) |
– |
1.24 ms |
1.12e10 |
4.91e11 (5 %) |
4.29e10 (2.8 %) |
0 MiB (0×) |
2.42 MiB |
– |
– |
– |
1410–1410 |
10,000 |
eagle graph, K fused steps per launch + compaction |
810 µs (809 µs–812 µs) |
– |
1.22 ms |
1.23e10 |
5.43e11 (5.6 %) |
5.14e10 (3.3 %) |
0 MiB (0×) |
2.43 MiB |
– |
– |
– |
1410–1410 |
10,000 |
eagle auto (eagle picks the launch mode) |
239 µs (237 µs–242 µs) |
– |
647 µs |
4.19e10 |
1.84e12 (19 %) |
1.17e12 (76 %) |
0 MiB (0×) |
2.36 MiB |
– |
– |
– |
1410–1410 |
10,000 |
hawk + eagle persistent |
298 µs (296 µs–300 µs) |
– |
633 µs |
3.36e10 |
1.48e12 (15 %) |
9.42e11 (61 %) |
0 MiB (0×) |
2.31 MiB |
– |
0.9750024960063898 |
– |
1410–1410 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
248 µs (248 µs–249 µs) |
– |
636 µs |
4.03e10 |
1.77e12 (18 %) |
1.13e12 (73 %) |
0 MiB (0×) |
2.38 MiB |
– |
– |
– |
1410–1410 |
10,000 |
eager launch loop |
36.1 ms (36 ms–36.3 ms) |
– |
36.5 ms |
2.77e8 |
1.22e10 (0.13 %) |
1.69e10 (1.1 %) |
0 MiB (0×) |
0.137 MiB |
– |
– |
– |
1095–1095 |
10,000 |
CuPy masked |
672 ms (671 ms–673 ms) |
– |
672 ms |
1.49e7 |
6.55e8 (0.0067 %) |
1.34e10 (0.86 %) |
0 MiB (0×) |
5.62 MiB |
– |
– |
– |
1095–1095 |
10,000 |
PyTorch masked |
426 ms (425 ms–428 ms) |
– |
427 ms |
2.35e7 |
1.03e9 (0.011 %) |
2.11e10 (1.4 %) |
74 MiB (216×) |
0.625 MiB |
– |
– |
– |
1155–1155 |
10,000 |
PyTorch, CUDA graph of 16 steps |
79.6 ms (79.6 ms–79.7 ms) |
– |
80.2 ms |
1.26e8 |
5.53e9 (0.057 %) |
1.15e11 (7.4 %) |
102 MiB (297×) |
0.719 MiB |
– |
– |
– |
1410–1410 |
10,000 |
PyTorch torch.compile (reduce-overhead) |
130 ms (130 ms–131 ms) |
– |
131 ms |
7.69e7 |
3.38e9 (0.035 %) |
6.77e9 (0.44 %) |
96 MiB (280×) |
33.3 MiB |
– |
– |
– |
1410–1410 |
10,000 |
JAX jit + vmap(while_loop) |
31.4 ms (31.2 ms–31.6 ms) |
– |
33.8 ms |
3.18e8 |
1.40e10 (0.14 %) |
2.80e10 (1.8 %) |
134 MiB (390×) |
0.328 MiB |
– |
– |
– |
1125–1125 |
10,000 |
Warp per-thread kernel |
274 µs (271 µs–280 µs) |
– |
817 µs |
3.65e10 |
1.61e12 (16 %) |
2.33e9 (0.15 %) |
30 MiB (87.4×) |
0.691 MiB |
– |
– |
– |
1125–1125 |
10,000 |
CPU OpenMP |
3.3 ms (3.29 ms–3.32 ms) |
– |
3.39 ms |
3.03e9 |
1.33e11 (–) |
1.85e11 (–) |
– |
1.16 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
1.35 ms (1.33 ms–1.35 ms) |
– |
2.95 ms |
7.43e10 |
3.27e12 (34 %) |
2.13e12 (1.4e+02 %) |
4 MiB (1.17×) |
7.66 MiB |
– |
– |
– |
1410–1410 |
100,000 |
eagle graph + compaction |
5.05 ms (5.05 ms–5.05 ms) |
– |
6.78 ms |
1.98e10 |
8.71e11 (8.9 %) |
1.29e12 (83 %) |
4 MiB (1.17×) |
7.69 MiB |
– |
– |
– |
1410–1410 |
100,000 |
eagle graph + compaction + reorder |
5.48 ms (5.47 ms–5.49 ms) |
– |
7.24 ms |
1.82e10 |
8.03e11 (8.2 %) |
1.19e12 (77 %) |
8 MiB (2.33×) |
8.32 MiB |
– |
– |
– |
1410–1410 |
100,000 |
eagle graph, K fused steps per launch |
1.29 ms (1.28 ms–1.29 ms) |
– |
2.91 ms |
7.76e10 |
3.42e12 (35 %) |
2.98e11 (19 %) |
4 MiB (1.17×) |
7.65 MiB |
– |
– |
– |
1410–1410 |
100,000 |
eagle graph, K fused steps per launch + compaction |
1.25 ms (1.24 ms–1.25 ms) |
– |
2.96 ms |
8.03e10 |
3.53e12 (36 %) |
3.34e11 (21 %) |
4 MiB (1.17×) |
7.68 MiB |
– |
– |
– |
1410–1410 |
100,000 |
eagle auto (eagle picks the launch mode) |
650 µs (649 µs–652 µs) |
– |
2.32 ms |
1.54e11 |
6.77e12 (69 %) |
4.31e12 (2.8e+02 %) |
4 MiB (1.17×) |
7.61 MiB |
– |
– |
– |
1410–1410 |
100,000 |
hawk + eagle persistent |
778 µs (776 µs–781 µs) |
– |
2.41 ms |
1.28e11 |
5.65e12 (58 %) |
3.60e12 (2.3e+02 %) |
4 MiB (1.17×) |
7.56 MiB |
– |
0.9765625 |
– |
1410–1410 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
660 µs (660 µs–661 µs) |
– |
2.33 ms |
1.51e11 |
6.66e12 (68 %) |
4.25e12 (2.7e+02 %) |
4 MiB (1.17×) |
7.62 MiB |
– |
– |
– |
1410–1410 |
100,000 |
eager launch loop |
35.7 ms (35.6 ms–35.7 ms) |
– |
37.5 ms |
2.80e9 |
1.23e11 (1.3 %) |
1.71e11 (11 %) |
4 MiB (1.17×) |
5.39 MiB |
– |
– |
– |
1410–1410 |
100,000 |
CuPy masked |
674 ms (673 ms–676 ms) |
– |
676 ms |
1.48e8 |
6.53e9 (0.067 %) |
1.34e11 (8.6 %) |
22 MiB (6.41×) |
9.39 MiB |
– |
– |
– |
1095–1095 |
100,000 |
PyTorch masked |
448 ms (447 ms–449 ms) |
– |
450 ms |
2.23e8 |
9.83e9 (0.1 %) |
2.01e11 (13 %) |
98 MiB (28.5×) |
6.24 MiB |
– |
– |
– |
1305–1320 |
100,000 |
PyTorch, CUDA graph of 16 steps |
91.3 ms (91.3 ms–91.4 ms) |
– |
93.7 ms |
1.09e9 |
4.82e10 (0.49 %) |
9.99e11 (64 %) |
124 MiB (36.1×) |
6.33 MiB |
– |
– |
– |
1410–1410 |
100,000 |
PyTorch torch.compile (reduce-overhead) |
130 ms (130 ms–130 ms) |
– |
132 ms |
7.69e8 |
3.38e10 (0.35 %) |
6.77e10 (4.4 %) |
102 MiB (29.7×) |
37 MiB |
– |
– |
– |
1410–1410 |
100,000 |
JAX jit + vmap(while_loop) |
30.4 ms (30.3 ms–30.5 ms) |
– |
33.7 ms |
3.29e9 |
1.45e11 (1.5 %) |
2.89e11 (19 %) |
134 MiB (39×) |
10.4 MiB |
– |
– |
– |
1365–1380 |
100,000 |
Warp per-thread kernel |
679 µs (678 µs–679 µs) |
– |
2.86 ms |
1.47e11 |
6.48e12 (66 %) |
9.43e9 (0.61 %) |
30 MiB (8.74×) |
6.87 MiB |
– |
– |
– |
1380–1380 |
100,000 |
CPU OpenMP |
26.9 ms (26.8 ms–26.9 ms) |
– |
27.9 ms |
3.72e9 |
1.64e11 (–) |
2.27e11 (–) |
– |
9.92 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
5.87 ms (5.84 ms–5.9 ms) |
– |
19.3 ms |
1.70e11 |
7.49e12 (77 %) |
4.88e12 (3.1e+02 %) |
40 MiB (1.17×) |
60.1 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
eagle graph + compaction |
53.7 ms (53.6 ms–53.7 ms) |
– |
67.5 ms |
1.86e10 |
8.20e11 (8.4 %) |
1.22e12 (78 %) |
56 MiB (1.63×) |
60.1 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
eagle graph + compaction + reorder |
54.4 ms (54.3 ms–54.4 ms) |
– |
68.3 ms |
1.84e10 |
8.09e11 (8.3 %) |
1.20e12 (77 %) |
84 MiB (2.45×) |
60.6 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
eagle graph, K fused steps per launch |
6.19 ms (6.18 ms–6.2 ms) |
– |
19.6 ms |
1.62e11 |
7.11e12 (73 %) |
6.21e11 (40 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
6.07 ms (6.07 ms–6.08 ms) |
– |
19.6 ms |
1.65e11 |
7.24e12 (74 %) |
6.85e11 (44 %) |
56 MiB (1.63×) |
60.1 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
4.83 ms (4.82 ms–4.83 ms) |
– |
18.3 ms |
2.07e11 |
9.12e12 (94 %) |
5.81e12 (3.7e+02 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
hawk + eagle persistent |
5.07 ms (5.05 ms–5.09 ms) |
– |
18.4 ms |
1.97e11 |
8.68e12 (89 %) |
5.53e12 (3.6e+02 %) |
40 MiB (1.17×) |
60 MiB |
– |
0.9765625 |
– |
1410–1410 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
4.85 ms (4.85 ms–4.86 ms) |
– |
18.3 ms |
2.06e11 |
9.07e12 (93 %) |
5.78e12 (3.7e+02 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
eager launch loop |
76.9 ms (76.5 ms–77.1 ms) |
– |
90.7 ms |
1.30e10 |
5.72e11 (5.9 %) |
7.94e11 (51 %) |
40 MiB (1.17×) |
57.8 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
CuPy masked |
679 ms (676 ms–682 ms) |
– |
695 ms |
1.47e9 |
6.48e10 (0.66 %) |
1.33e12 (85 %) |
192 MiB (5.59×) |
65.3 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
PyTorch masked |
608 ms (606 ms–611 ms) |
– |
631 ms |
1.65e9 |
7.24e10 (0.74 %) |
1.48e12 (95 %) |
334 MiB (9.73×) |
62.1 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
532 ms (532 ms–532 ms) |
– |
556 ms |
1.88e9 |
8.27e10 (0.85 %) |
1.71e12 (1.1e+02 %) |
362 MiB (10.5×) |
62.2 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
PyTorch torch.compile (reduce-overhead) |
166 ms (165 ms–166 ms) |
– |
189 ms |
6.03e9 |
2.65e11 (2.7 %) |
5.31e11 (34 %) |
176 MiB (5.13×) |
71.3 MiB |
– |
– |
– |
1095–1095 |
1,000,000 |
JAX jit + vmap(while_loop) |
90.5 ms (90.4 ms–90.5 ms) |
– |
106 ms |
1.11e10 |
4.86e11 (5 %) |
9.73e11 (63 %) |
134 MiB (3.9×) |
95.1 MiB |
– |
– |
– |
1410–1410 |
1,000,000 |
Warp per-thread kernel |
6.67 ms (6.67 ms–6.68 ms) |
– |
38 ms |
1.50e11 |
6.60e12 (68 %) |
9.59e9 (0.62 %) |
62 MiB (1.81×) |
68.7 MiB |
– |
– |
– |
1095–1095 |
1,000,000 |
CPU OpenMP |
261 ms (261 ms–261 ms) |
– |
276 ms |
3.83e9 |
1.69e11 (–) |
2.34e11 (–) |
– |
97.5 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (156 µs), then eagle.simulate (eagle picks the launch mode) (174 µs, 1.11× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (239 µs), then eagle.simulate (eagle picks the launch mode) (248 µs, 1.04× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (650 µs), then eagle.simulate (eagle picks the launch mode) (660 µs, 1.02× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (4.83 ms), then eagle.simulate (eagle picks the launch mode) (4.85 ms, 1.01× the time)
Compaction cadence sweep (isolates the active-set map)
the active-set map alone. For a given K, matched_cadence (the plain per-sample step, no map) and compact (eagle_graph_compact’s active-set step) check the loop’s stop guard at the SAME cadence, once every K steps; the only structural difference between the two rows at a given K is whether the step kernel reads the active-set map. The main table’s eagle_graph row checks its guard every step, so its difference from eagle_graph_compact there mixes the cadence change with the map – this sweep does not.
Config: Spread stop steps (log-uniform in [10, 1000]), up to 1000 steps.
N |
K (steps between guard checks) |
matched cadence, no map (range) |
eagle graph + compaction (range) |
ratio (matched/compact, |
|---|---|---|---|---|
100,000 |
8 |
4.04 ms (4.03 ms–4.04 ms) |
8.79 ms (8.78 ms–8.81 ms) |
0.459× |
100,000 |
16 |
3.68 ms (3.68 ms–3.68 ms) |
6.98 ms (6.97 ms–6.99 ms) |
0.527× |
100,000 |
32 |
3.56 ms (3.55 ms–3.56 ms) |
5.98 ms (5.97 ms–5.99 ms) |
0.595× |
1,000,000 |
8 |
31.2 ms (31.2 ms–31.2 ms) |
36.4 ms (36.4 ms–36.4 ms) |
0.855× |
1,000,000 |
16 |
31 ms (31 ms–31 ms) |
32.5 ms (32.5 ms–32.6 ms) |
0.952× |
1,000,000 |
32 |
30.7 ms (30.7 ms–30.7 ms) |
30.6 ms (30.6 ms–30.6 ms) |
1× |
FP32: eagle vs Warp (float32, S = 1000)
Every arm runs the same oscillator in float32: eagle.simulate with scalar_type=”float32”, NVIDIA Warp’s per-thread kernel and JAX’s vmapped loop, all over the same float32 inputs. Each cell runs 2 untimed runs per arm first (eagle’s automatic arm picks its entry over them: the size rule, then a measured run of the other entry, keeping the faster; the entry it settled on is in the last column), then each arm in its own block (no interleaving): an untimed ramp of back-to-back runs for 0.5 s, then 5 timed runs. Before any timing counts, eagle and JAX reach step counts identical to Warp’s and states within absolute 0.0001 of Warp’s (the largest differences are in the JSON). Each time is the mean of the middle 3 of the 5 runs (highest and lowest dropped), with the range of those 3; eagle / Warp is the ratio of these means: below 1 eagle is faster. Before each arm’s block the GPU is cooled down: the block starts once the temperature is within 3 °C of the idle 33 °C and the GPU is idle, or after 20 s. Right after each timed run the SM clock is read (its range is shown per arm); cycles are the wall time times that clock, and eagle / Warp (cycles) is their ratio.
Every arm runs each sample to its own stop and no further than max_steps (S): eagle and Warp enforce the cap per sample, JAX in its while-loop condition, PyTorch and CuPy by the outer step count (a batch-level cap); an arm that runs whole blocks of steps per launch or replay stops at the first block boundary at or after it.
What each arm needed to get there: Warp, a hand-written step counter and cap inside the kernel; JAX, a cap in the while_loop condition; eagle, the kernel source unchanged, with its loop shape, unrolling and device entry chosen by hawk + eagle (per device, per batch).
Cap fixture (checked before any timing counts): N = 4,096, 8 samples with stop step 2S = 256, S = 128; every arm (3) reported k = min(stop step, S) per sample and the same finished count (4,088): passed.
distribution |
N |
eagle.simulate (range) |
Warp (range) |
JAX (range) |
eagle / Warp |
eagle / Warp (cycles) |
eagle entry |
|---|---|---|---|---|---|---|---|
uniform |
10,000 |
212 µs (210 µs–213 µs), 1095–1095 MHz |
198 µs (195 µs–201 µs), 1095–1095 MHz |
31.5 ms (31.5 ms–31.5 ms), 1095–1095 MHz |
1.07× |
1.07× |
fused_one |
uniform |
100,000 |
449 µs (442 µs–453 µs), 1140–1140 MHz |
447 µs (442 µs–453 µs), 1140–1140 MHz |
30.7 ms (30.6 ms–30.7 ms), 1260–1260 MHz |
1× |
1× |
fused_one |
uniform |
1,000,000 |
2.51 ms (2.51 ms–2.51 ms), 1410–1410 MHz |
2.59 ms (2.59 ms–2.59 ms), 1410–1410 MHz |
37.4 ms (36.2 ms–38.2 ms), 1410–1410 MHz |
0.97× |
0.97× |
fused_one |
spread |
10,000 |
194 µs (190 µs–201 µs), 1410–1410 MHz |
192 µs (191 µs–193 µs), 1095–1095 MHz |
31.6 ms (31.5 ms–31.6 ms), 1095–1095 MHz |
1.01× |
1.3× |
fused_one |
spread |
100,000 |
321 µs (314 µs–332 µs), 1095–1095 MHz |
444 µs (439 µs–450 µs), 1095–1095 MHz |
30.8 ms (30.7 ms–30.9 ms), 1215–1245 MHz |
0.725× |
0.725× |
persist |
spread |
1,000,000 |
994 µs (987 µs–999 µs), 1260–1260 MHz |
2.34 ms (2.33 ms–2.35 ms), 1350–1350 MHz |
36.8 ms (36.1 ms–38.1 ms), 1410–1410 MHz |
0.425× |
0.397× |
persist |
Fixed overhead per run (N = 64, S = 100)
item 3: wall at a cell small enough that per-run overhead dominates (N=64, S=100), each arm’s own kernel-only vs wall where the main nsys pass covers this cell’s N (it does not by default, so kernel_only_s is usually null here)
arm |
wall (range) |
kernel-only |
|---|---|---|
eagle graph (device loop) |
647 µs (639 µs–652 µs) |
– |
eagle graph + compaction |
674 µs (673 µs–674 µs) |
– |
eagle graph + compaction + reorder |
819 µs (811 µs–834 µs) |
– |
eagle graph, K fused steps per launch |
318 µs (317 µs–320 µs) |
– |
eagle graph, K fused steps per launch + compaction |
397 µs (391 µs–408 µs) |
– |
eagle auto (eagle picks the launch mode) |
119 µs (118 µs–121 µs) |
– |
hawk + eagle persistent |
175 µs (168 µs–185 µs) |
– |
eagle.simulate (eagle picks the launch mode) |
130 µs (127 µs–134 µs) |
– |
eager launch loop |
3.71 ms (3.7 ms–3.71 ms) |
– |
CuPy masked |
66 ms (65.9 ms–66 ms) |
– |
PyTorch masked |
42.6 ms (42.5 ms–42.6 ms) |
– |
PyTorch, CUDA graph of 16 steps |
6.47 ms (6.47 ms–6.48 ms) |
– |
PyTorch torch.compile (reduce-overhead) |
13.6 ms (13.5 ms–13.8 ms) |
– |
JAX jit + vmap(while_loop) |
2.25 ms (2.22 ms–2.31 ms) |
– |
Warp per-thread kernel |
136 µs (132 µs–143 µs) |
– |
CPU OpenMP |
384 µs (382 µs–386 µs) |
– |
Where each tool fits
Each tool below is written the way its users write it, and each brings something of its own. Times are walls (mean of the middle 3 of 5 runs) at the largest N; ratios are against the eagle graph arm (below 1 = less time). The fastest arm per N is listed under each table above.
eagle graph (device loop). The step written once as a hawk kernel; the whole loop runs on the device as one CUDA graph with a device-side stop guard, so a run is one launch with no host round trip per step. On this card: N = 1,000,000, spread S=100: 1.38 ms; spread S=1000: 5.33 ms; uniform S=1000: 5.87 ms. Fastest in 0 of 12 cells.
eagle graph + compaction. Adds an active-set map: the launch covers only the samples still running, rebuilt every 16 steps; it is made for batches that thin out (the spread configurations). On this card: N = 1,000,000, spread S=100: 3.77 ms (2.74× the eagle graph time); spread S=1000: 32.6 ms (6.13× the eagle graph time); uniform S=1000: 53.7 ms (9.14× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph + compaction + reorder. Adds a physical reorder when the live samples spread thin over their warps, so the mapped reads stay contiguous; the final restore to sample order is inside its wall. On this card: N = 1,000,000, spread S=100: 3.23 ms (2.35× the eagle graph time); spread S=1000: 17.2 ms (3.24× the eagle graph time); uniform S=1000: 54.4 ms (9.26× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph, K fused steps per launch. The same step with one decorator argument, @hawk.kernel(steps=16): each launch runs that many steps per sample with the state kept in registers, as Warp’s per-thread loop does, and the launches, guard checks and plane reads shrink by the same factor; its results are bit-identical to the eagle graph arm’s in every cell. K is chosen by hand here; the eagle_graph_auto arm picks it at run time. On this card: N = 1,000,000, spread S=100: 880 µs (0.639× the eagle graph time); spread S=1000: 5.55 ms (1.04× the eagle graph time); uniform S=1000: 6.19 ms (1.05× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph, K fused steps per launch + compaction. The K-steps kernel over the active-set map: the register-resident steps of the arm above, and the launches cover only the samples still running, so it fits batches that thin out. On this card: N = 1,000,000, spread S=100: 959 µs (0.697× the eagle graph time); spread S=1000: 3.54 ms (0.665× the eagle graph time); uniform S=1000: 6.07 ms (1.03× the eagle graph time). Fastest in 0 of 12 cells.
eagle auto (eagle picks the launch mode). The same step with @hawk.kernel(steps=”auto”) on the active-set kind, and nothing to tune: it picks the steps per launch itself after every launch, on the device, so a batch whose samples all run to the end gets Warp-like fusion of up to 64 steps per launch, and a batch that thins out gets shorter launches with the active set compacted after each one; no sample takes more steps than the cap. On this card: N = 1,000,000, spread S=100: 390 µs (0.284× the eagle graph time); spread S=1000: 1.48 ms (0.277× the eagle graph time); uniform S=1000: 4.83 ms (0.822× the eagle graph time). Fastest in 11 of 12 cells.
hawk + eagle persistent. The plain (not active-set) automatic kernel, forced through hawk’s persist entry: one launch, no graph, no map, no policy kernel, no cap at 64 – a grid of SMs blocks of 256 lanes each steal the next unfinished sample off a counter when their own finishes, so idle warps refill without ever rebuilding an index map; made for batches too large for one resident wave. On this card: N = 1,000,000, spread S=100: 536 µs (0.389× the eagle graph time); spread S=1000: 1.53 ms (0.287× the eagle graph time); uniform S=1000: 5.07 ms (0.863× the eagle graph time). Fastest in 0 of 12 cells.
eagle.simulate (eagle picks the launch mode). The same automatic kernel through eagle.simulate: the model, its state and its parameters by name, with no plan or plane binding to write; eagle deploys the kernel and builds the same device loop. On this card: N = 1,000,000, spread S=100: 410 µs (0.298× the eagle graph time); spread S=1000: 1.5 ms (0.281× the eagle graph time); uniform S=1000: 4.85 ms (0.826× the eagle graph time). Fastest in 0 of 12 cells.
eager launch loop. The same kernels launched from Python one step at a time; the host sees the state after every step. On this card: N = 1,000,000, spread S=100: 6.43 ms (4.67× the eagle graph time); spread S=1000: 59.4 ms (11.2× the eagle graph time); uniform S=1000: 76.9 ms (13.1× the eagle graph time). Fastest in 0 of 12 cells.
CuPy masked. NumPy-like array code on GPUs with no kernel to write; every sample is computed every step, so it fits dense batches where all samples run to the end (the uniform configuration). On this card: N = 1,000,000, spread S=100: 68.7 ms (49.9× the eagle graph time); spread S=1000: 673 ms (126× the eagle graph time); uniform S=1000: 679 ms (116× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch masked. The same array code in PyTorch tensors; it fits where the computation sits next to a PyTorch model or needs autograd. On this card: N = 1,000,000, spread S=100: 60.7 ms (44.1× the eagle graph time); spread S=1000: 608 ms (114× the eagle graph time); uniform S=1000: 608 ms (104× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch, CUDA graph of 16 steps. The same PyTorch code with its launch overhead removed by hand: 16 steps captured as one CUDA graph and replayed, one host check per replay, the masking unchanged. On this card: N = 1,000,000, spread S=100: 59.2 ms (43× the eagle graph time); spread S=1000: 532 ms (100× the eagle graph time); uniform S=1000: 532 ms (90.7× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch torch.compile (reduce-overhead). The PyTorch step under torch.compile(mode=”reduce-overhead”): Inductor-fused kernels replayed as a CUDA graph. On this card: N = 1,000,000, spread S=100: 17.2 ms (12.5× the eagle graph time); spread S=1000: 167 ms (31.4× the eagle graph time); uniform S=1000: 166 ms (28.2× the eagle graph time). Fastest in 0 of 12 cells.
JAX jit + vmap(while_loop). The whole loop compiled once, a per-sample while loop batched with vmap; the same code runs on CPUs and GPUs and differentiates with jax.grad. On this card: N = 1,000,000, spread S=100: 9.5 ms (6.9× the eagle graph time); spread S=1000: 90.7 ms (17× the eagle graph time); uniform S=1000: 90.5 ms (15.4× the eagle graph time). Fastest in 0 of 12 cells.
Warp per-thread kernel. Explicit per-thread kernels written in Python, maintained by NVIDIA and differentiable through wp.Tape; each thread runs its own sample and stops at its own step, so finished samples cost nothing. On this card: N = 1,000,000, spread S=100: 604 µs (0.439× the eagle graph time); spread S=1000: 5.94 ms (1.12× the eagle graph time); uniform S=1000: 6.67 ms (1.14× the eagle graph time). Fastest in 1 of 12 cells.
CPU OpenMP. The same hawk kernel on the host through eagle’s OpenMP team, for machines without a GPU. On this card: N = 1,000,000, spread S=100: 29.8 ms (21.6× the eagle graph time); spread S=1000: 281 ms (52.8× the eagle graph time); uniform S=1000: 261 ms (44.4× the eagle graph time). Fastest in 0 of 12 cells.
Compile time
Its own pass, never part of any wall. Each measurement is a fresh process: the imports and the GPU context start-up run untimed, then the timer spans the first call to a ready kernel (trace + code generation + compile + module load, or the cache lookup). Cold: every cache empty – hawk’s compile cache (HAWK_CACHE_DIR), CuPy’s kernel cache (CUPY_CACHE_DIR), the CUDA driver’s PTX cache (CUDA_CACHE_PATH), PyTorch Inductor’s and Triton’s caches (TORCHINDUCTOR_CACHE_DIR, TRITON_CACHE_DIR) and JAX’s persistent compilation cache (minimum compile time and entry size 0), each a fresh directory. Warm: a new process on the caches the first cold run populated. eagle arms: eagle.deploy of the arm’s hawk kernel, which returns once the device build is done (C++/CUDA emission, nvcc to PTX, driver load; eagle’s own small CuPy kernels compile beside it) while the host compiler builds the host side on another thread, and the arm’s set-up on a 64-sample batch (eagle’s graph built, one step run). CuPy: one masked step on 64 samples (every elementwise and reduction kernel compiled on first use). Warp: the kernel module’s code generation, compile and load on the first launch (64 samples), cached in WARP_CACHE_PATH. PyTorch torch.compile, where it runs: three steps at N (the first call traces and compiles, the second records the CUDA graph, the third replays it), per N (dynamic=False). JAX: lower + compile from abstract shapes, per N (XLA specialises on the shape). CPU OpenMP: the same eagle.deploy as the eagle graph arm and one host run, which waits for the host build. Medians of 3 cold and 3 warm processes.
arm |
compiled |
cold (median of 3) |
warm (median of 3) |
|---|---|---|---|
eagle graph (device loop) |
eagle.deploy of the hawk step kernel (the device build; the host side builds in the background), the device-loop graph |
1.22 s |
110 ms |
eagle graph + compaction |
eagle.deploy of the hawk active-set step kernel (the device build; the host side builds in the background), the compaction-loop graph |
978 ms |
92.9 ms |
eagle graph + compaction + reorder |
eagle.deploy of the hawk active-set step kernel (the device build; the host side builds in the background), the compaction + reorder loop graph |
1.09 s |
96.1 ms |
eagle graph, K fused steps per launch |
eagle.deploy of the hawk K-steps kernel (the device build; the host side builds in the background), the device-loop graph |
995 ms |
102 ms |
eagle graph, K fused steps per launch + compaction |
eagle.deploy of the hawk active-set K-steps kernel (the device build; the host side builds in the background), the compaction-loop graph |
998 ms |
104 ms |
eagle auto (eagle picks the launch mode) |
eagle.deploy of the hawk active-set automatic kernel (the device build; the host side builds in the background), the policy kernel, the policy-loop graph |
1.32 s |
110 ms |
hawk + eagle persistent |
eagle.deploy of the hawk automatic kernel (the device build; the host side builds in the background), the persistent launch |
1.06 s |
107 ms |
eagle.simulate (eagle picks the launch mode) |
eagle.simulate’s deploy of the hawk active-set automatic kernel (the device build; the host side builds in the background), the policy kernel, the policy-loop graph |
1.32 s |
113 ms |
eager launch loop |
eagle.deploy of the hawk single step kernel (the device build; the host side builds in the background) |
865 ms |
82.3 ms |
CuPy masked |
CuPy’s elementwise and reduction kernels of one masked step |
1.05 s |
33.5 ms |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 1,000 |
6.19 s |
4.08 s |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 10,000 |
6.53 s |
4.05 s |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 100,000 |
6.62 s |
4.05 s |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 1,000,000 |
6.63 s |
4.06 s |
Warp per-thread kernel |
Warp kernel module (code generation, NVRTC compile, load) |
936 ms |
250 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 1,000 |
250 ms |
56.9 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 10,000 |
235 ms |
56.4 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 100,000 |
258 ms |
56 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 1,000,000 |
274 ms |
57.2 ms |
CPU OpenMP |
eagle.deploy of the hawk step kernel and its first host run, which waits for the host build |
1.78 s |
102 ms |
No compile step: PyTorch masked, PyTorch, CUDA graph of 16 steps (PyTorch’s eager kernels ship prebuilt; the CUDA-graph capture is per N and recorded per configuration as torch_graph_capture_s).
Memory method
A separate pass after the timing pass. Each (arm, N, configuration) runs in a fresh process: imports, the CUDA context created by one tiny device operation, then the baseline read, before the arm builds or allocates anything; then the arm’s kernels built and a warm-up run on a 64-sample batch (every compile happens here; JAX also compiles for shape N from abstract shapes), so every library’s pool reserved during the warm-up counts in the reading; CuPy’s pools freed and torch.cuda.empty_cache() called after the warm-up (Warp’s pool release threshold set to 0); then the batch’s inputs generated, the arm set up for N (eagle builds its graph, PyTorch captures its CUDA graph, whose private memory pool counts) and one full run with upload and download. Device memory = the memory the CUDA driver accounts to the process (NVML per-process used memory) after the run minus at the baseline. The caching allocators of CuPy, PyTorch and JAX are left at their defaults and are not trimmed after the run; each keeps the memory it reserved until trimmed, so the reading after the run is the allocator’s high-water reservation, the same measure for every arm. A disabled pool returns memory between operations, so its peak could only be caught by sampling, which misses short peaks; the held reservation needs no sampling. Caveats: the reading is what the process holds, so each allocator’s rounding and growth policy count (JAX’s allocator grows in regions that can exceed the request; PyTorch’s rounds blocks up; a captured CUDA graph keeps a private pool; Warp’s stream-ordered pool reserves in chunks of tens of MiB); the driver accounts in pages of about 2 MiB, so small-N rows read 0 or one page. The per-library counters (CuPy pool, torch.cuda.max_memory_reserved/allocated, JAX memory_stats) are recorded in the JSON as a cross-check. Host memory = peak resident set size above the host baseline (taken after the warm-up; /proc/self/clear_refs reset there, then VmHWM). Minimum = the state the workload needs, N × ((2 state + 2 float64 scalars: omega, zeta) × 8 + 4 bytes for the stop step as a 32-bit integer) = 36 bytes per sample; the factor is device memory / minimum.
Device memory the process already held at the baseline (CUDA context, loaded kernels and libraries, the warm-up’s leftovers; at N = 1,000,000, first configuration): eagle graph (device loop) 420 MiB; eagle graph + compaction 420 MiB; eagle graph + compaction + reorder 420 MiB; eagle graph, K fused steps per launch 420 MiB; eagle graph, K fused steps per launch + compaction 420 MiB; eagle auto (eagle picks the launch mode) 420 MiB; hawk + eagle persistent 420 MiB; eagle.simulate (eagle picks the launch mode) 420 MiB; eager launch loop 420 MiB; CuPy masked 420 MiB; PyTorch masked 420 MiB; PyTorch, CUDA graph of 16 steps 420 MiB; PyTorch torch.compile (reduce-overhead) 420 MiB; JAX jit + vmap(while_loop) 420 MiB; Warp per-thread kernel 420 MiB.
spread (up to 100 steps)
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (90.8 µs), then eagle.simulate (eagle picks the launch mode) (102 µs, 1.12× the time)
N = 10,000: Warp per-thread kernel (113 µs), then eagle auto (eagle picks the launch mode) (117 µs, 1.03× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (145 µs), then eagle.simulate (eagle picks the launch mode) (161 µs, 1.11× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (390 µs), then eagle.simulate (eagle picks the launch mode) (410 µs, 1.05× the time)
spread (up to 1000 steps)
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (145 µs), then eagle.simulate (eagle picks the launch mode) (156 µs, 1.07× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (231 µs), then eagle.simulate (eagle picks the launch mode) (244 µs, 1.05× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (350 µs), then hawk + eagle persistent (391 µs, 1.12× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (1.48 ms), then eagle.simulate (eagle picks the launch mode) (1.5 ms, 1.01× the time)
uniform (up to 1000 steps)
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (156 µs), then eagle.simulate (eagle picks the launch mode) (174 µs, 1.11× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (239 µs), then eagle.simulate (eagle picks the launch mode) (248 µs, 1.04× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (650 µs), then eagle.simulate (eagle picks the launch mode) (660 µs, 1.02× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (4.83 ms), then eagle.simulate (eagle picks the launch mode) (4.85 ms, 1.01× the time)
eagle_simulate against its hand-built twin eagle_graph_auto:
spread (up to 100 steps), N = 1,000: wall 1.12× the twin’s time
spread (up to 100 steps), N = 10,000: wall 1.11× the twin’s time
spread (up to 100 steps), N = 100,000: wall 1.11× the twin’s time
spread (up to 100 steps), N = 1,000,000: wall 1.05× the twin’s time
spread (up to 1000 steps), N = 1,000: wall 1.07× the twin’s time
spread (up to 1000 steps), N = 10,000: wall 1.05× the twin’s time
spread (up to 1000 steps), N = 100,000: wall 1.24× the twin’s time
spread (up to 1000 steps), N = 1,000,000: wall 1.01× the twin’s time
uniform (up to 1000 steps), N = 1,000: wall 1.11× the twin’s time
uniform (up to 1000 steps), N = 10,000: wall 1.04× the twin’s time
uniform (up to 1000 steps), N = 100,000: wall 1.02× the twin’s time
uniform (up to 1000 steps), N = 1,000,000: wall 1.01× the twin’s time
Run on: driver 580.178.04; CUDA runtime 12.9; host CPU Intel Xeon CPU @ 2.00GHz; compiler x86_64-conda-linux-gnu-g++ (conda-forge gcc 14.4.0-7) 14.4.0; Python 3.12.14; Warp 1.17.0, JAX 0.11.2, PyTorch 2.12.1+cu129, CuPy 14.2.0, NumPy 2.5.3.
N = 1,000,000
Log scale, from 5.37 ms to 592 ms.
N = 1,000,000
Log scale, from 28.8 ms to 5.92 s.
N = 1,000,000
Log scale, from 120 ms to 5.91 s.
Table of every arm and every N
Device: Tesla T4 (compute capability 7.5, 40 SMs). Peak FP64: 2.54e11 FLOP/s. Peak DRAM bandwidth: 3.20e11 B/s. Mean of the middle 3 of 5 runs of each arm (the highest and the lowest dropped), range of those 3 in parentheses.
Each arm’s block starts once the GPU is within 3 °C of its idle 73 °C and reads idle, or after 20 s; the SM clock is read right after each timed run and its range is in the last column.
The FP64 peak above is at the device’s reported maximum clock (1590 MHz). During the timed repetitions the SM clock read 1568 MHz; at that clock the FP64 peak is 2.51e11 FLOP/s, and every FP64 percentage below is 1.01× larger against it.
“% of issue peak” (the eagle arms only) counts DP instructions (DFMA/DMUL/DADD/DSETP/MUFU), not useful FLOPs: the step’s SASS has 27 of them, against the device’s DP issue-slot throughput at the observed clock – a kernel that is already at the issue pipe’s limit can still read well under 100% of FP64 peak, because useful FLOPs/step is smaller than issued DP instructions/step.
eagle_graph checks its stop guard every step; eagle_graph_compact checks it once every 16 steps (the cadence its compaction runs on) and reads an active-set map built from it. The two rows below differ in both the cadence and the map; the compaction cadence sweep further down holds the cadence fixed at a few values and isolates the map alone.
PyTorch, CUDA graph of 16 steps captures 16 masked steps once as a torch.cuda.CUDAGraph over static buffers and replays it, with one host check of “any sample running” per replay: it removes the per-kernel launch overhead and keeps the masking (every sample is computed every step, finished ones held by torch.where, so the steps a replay runs past the last stop change nothing). JAX’s while loop is one compiled executable with no host check per step; batched by vmap, it runs until the slowest sample stops. Warp runs one thread per sample, each stopping at its own step.
Every arm runs each sample to its own stop and no further than max_steps (S): eagle and Warp enforce the cap per sample, JAX in its while-loop condition, PyTorch and CuPy by the outer step count (a batch-level cap); an arm that runs whole blocks of steps per launch or replay stops at the first block boundary at or after it.
What each arm needed to get there: Warp, a hand-written step counter and cap inside the kernel; JAX, a cap in the while_loop condition; eagle, the kernel source unchanged, with its loop shape, unrolling and device entry chosen by hawk + eagle (per device, per batch).
Cap fixture (checked before any timing counts): N = 4,096, 8 samples with stop step 2S = 256, S = 128; every arm (16) reported k = min(stop step, S) per sample and the same finished count (4,088): passed.
arm |
loop structure |
|---|---|
eagle graph (device loop) |
per launch over the batch, inside one CUDA graph: the plain decorator’s kernel, which finishes its own samples and runs the steps per launch eagle’s policy picks (8 to 64), guard checked on the device every launch |
eagle graph + compaction |
as the graph arm, the step launched over the active-set map only (recomputed every 16 steps) |
eagle graph + compaction + reorder |
as the compaction arm, plus a physical reorder of the per-sample planes when the live samples spread thin |
eagle graph, K fused steps per launch |
per launch over the batch, inside one CUDA graph: the step kernel runs up to K = 16 steps per sample with the state in registers, each sample leaving at its own stop step; guard checked on the device every launch |
eagle graph, K fused steps per launch + compaction |
as the K-steps arm, the launch over the active-set map only (recomputed every 64 steps = 4 launches) |
eagle auto (eagle picks the launch mode) |
the active-set kernel under eagle’s automatic policy, which picks the launch mode from the batch: one launch running every sample to its end for a small batch, the persistent launch above that, and the compaction loop (active-set map, steps per launch picked on the device) only when the caller asks for compaction or reordering; the row records the mode taken |
hawk + eagle persistent |
ONE launch, no graph, no map, no policy kernel: a grid sized from the SM count and the kernel’s occupancy, each lane fetching base + atomicAdd(counter, 1) whenever its sample is done, running the whole budget’s worth of steps per sample it picks up |
eagle.simulate (eagle picks the launch mode) |
as the automatic arm: eagle.simulate takes the kernel, its state and its parameters by name and eagle picks the launch mode the same way |
eager launch loop |
per step over the batch: one kernel launch from Python, a host read of the finished count every step |
CuPy masked |
per step over the batch: whole-batch array expressions, finished samples held by cupy.where, a host check every step; the cap is batch-level (the outer step loop runs at most max_steps) |
PyTorch masked |
per step over the batch: whole-batch tensor expressions, finished samples held by torch.where, a host check every step; the cap is batch-level (the outer step loop runs at most max_steps) |
PyTorch, CUDA graph of 16 steps |
as PyTorch masked, 16 steps captured as one CUDA graph over static buffers, replayed with one host check per replay |
PyTorch torch.compile (reduce-overhead) |
as PyTorch masked, the step compiled by Inductor and replayed as a CUDA graph, a host check every step |
JAX jit + vmap(while_loop) |
per step over the batch, inside one compiled executable: the vmapped while loop runs until the slowest sample stops or the step cap is reached, finished samples held by a select |
Warp per-thread kernel |
per sample: one launch, each thread loops over its own sample’s steps and stops at its own stop step or the step cap, whichever comes first |
CPU OpenMP |
per step over the batch on the host: one host-team launch per step, the kernel marks and counts its finished samples |
Spread stop steps (log-uniform in [1, 100]), up to 100 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
768 µs (760 µs–775 µs) |
– |
999 µs |
2.87e7 |
1.26e9 (0.5 %) |
9.50e8 (0.3 %) |
0 MiB (0×) |
2.19 MiB |
0.62 % |
– |
– |
585–585 |
1,000 |
eagle graph + compaction |
824 µs (822 µs–827 µs) |
– |
1.11 ms |
2.68e7 |
1.18e9 (0.46 %) |
2.40e9 (0.75 %) |
0 MiB (0×) |
2.07 MiB |
0.58 % |
– |
– |
1380–1380 |
1,000 |
eagle graph + compaction + reorder |
991 µs (969 µs–1.02 ms) |
– |
1.34 ms |
2.23e7 |
9.80e8 (0.39 %) |
2.42e9 (0.76 %) |
0 MiB (0×) |
2.16 MiB |
0.48 % |
– |
– |
1590–1590 |
1,000 |
eagle graph, K fused steps per launch |
346 µs (315 µs–371 µs) |
– |
558 µs |
6.38e7 |
2.81e9 (1.1 %) |
8.27e8 (0.26 %) |
0 MiB (0×) |
2.25 MiB |
– |
– |
– |
1590–1590 |
1,000 |
eagle graph, K fused steps per launch + compaction |
377 µs (376 µs–378 µs) |
– |
665 µs |
5.85e7 |
2.57e9 (1 %) |
7.32e8 (0.23 %) |
0 MiB (0×) |
2.06 MiB |
– |
– |
– |
1590–1590 |
1,000 |
eagle auto (eagle picks the launch mode) |
160 µs (152 µs–176 µs) |
– |
455 µs |
1.38e8 |
6.06e9 (2.4 %) |
4.07e9 (1.3 %) |
0 MiB (0×) |
2.2 MiB |
– |
– |
– |
855–885 |
1,000 |
hawk + eagle persistent |
280 µs (273 µs–285 µs) |
– |
484 µs |
7.88e7 |
3.47e9 (1.4 %) |
2.33e9 (0.73 %) |
0 MiB (0×) |
2.06 MiB |
– |
0.2098712531887755 |
– |
960–960 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
161 µs (152 µs–167 µs) |
– |
479 µs |
1.37e8 |
6.01e9 (2.4 %) |
4.04e9 (1.3 %) |
0 MiB (0×) |
2.03 MiB |
– |
– |
– |
960–960 |
1,000 |
eager launch loop |
3.47 ms (3.4 ms–3.58 ms) |
– |
3.67 ms |
6.36e6 |
2.80e8 (0.11 %) |
1.13e9 (0.35 %) |
0 MiB (0×) |
0.0664 MiB |
0.14 % |
– |
– |
600–960 |
1,000 |
CuPy masked |
67.9 ms (67.7 ms–68 ms) |
– |
68.1 ms |
3.25e5 |
1.43e7 (0.0056 %) |
1.33e9 (0.41 %) |
0 MiB (0×) |
166 MiB |
– |
– |
– |
585–585 |
1,000 |
PyTorch masked |
44.3 ms (39.9 ms–50 ms) |
– |
44.6 ms |
4.98e5 |
2.19e7 (0.0086 %) |
2.03e9 (0.64 %) |
14 MiB (408×) |
0.137 MiB |
– |
– |
– |
1035–1035 |
1,000 |
PyTorch, CUDA graph of 16 steps |
10.1 ms (10.1 ms–10.1 ms) |
– |
10.4 ms |
2.19e6 |
9.62e7 (0.038 %) |
1.00e10 (3.1 %) |
42 MiB (1.22e+03×) |
0.16 MiB |
– |
– |
– |
1590–1590 |
1,000 |
PyTorch torch.compile (reduce-overhead) |
14.5 ms (14.3 ms–14.8 ms) |
– |
14.9 ms |
1.52e6 |
6.67e7 (0.026 %) |
6.05e8 (0.19 %) |
36 MiB (1.05e+03×) |
35.2 MiB |
– |
– |
– |
1590–1590 |
1,000 |
JAX jit + vmap(while_loop) |
3 ms (2.95 ms–3.06 ms) |
– |
5.11 ms |
7.35e6 |
3.23e8 (0.13 %) |
2.94e9 (0.92 %) |
8 MiB (233×) |
0.0703 MiB |
– |
– |
– |
825–870 |
1,000 |
Warp per-thread kernel |
408 µs (396 µs–430 µs) |
– |
838 µs |
5.41e7 |
2.38e9 (0.94 %) |
1.57e8 (0.049 %) |
30 MiB (874×) |
0.0625 MiB |
– |
– |
– |
1065–1065 |
1,000 |
CPU OpenMP |
378 µs (376 µs–379 µs) |
– |
393 µs |
5.84e7 |
2.57e9 (–) |
1.04e10 (–) |
– |
0.234 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
1.02 ms (1.01 ms–1.02 ms) |
– |
1.43 ms |
2.25e8 |
9.88e9 (3.9 %) |
7.75e9 (2.4 %) |
0 MiB (0×) |
2.49 MiB |
4.8 % |
– |
– |
900–900 |
10,000 |
eagle graph + compaction |
929 µs (911 µs–947 µs) |
– |
1.38 ms |
2.46e8 |
1.08e10 (4.3 %) |
2.18e10 (6.8 %) |
0 MiB (0×) |
2.51 MiB |
5.3 % |
– |
– |
1590–1590 |
10,000 |
eagle graph + compaction + reorder |
1.31 ms (1.29 ms–1.34 ms) |
– |
1.8 ms |
1.75e8 |
7.68e9 (3 %) |
1.87e10 (5.9 %) |
0 MiB (0×) |
3.13 MiB |
3.8 % |
– |
– |
1590–1590 |
10,000 |
eagle graph, K fused steps per launch |
467 µs (466 µs–468 µs) |
– |
830 µs |
4.90e8 |
2.15e10 (8.5 %) |
6.16e9 (1.9 %) |
0 MiB (0×) |
2.55 MiB |
– |
– |
– |
1590–1590 |
10,000 |
eagle graph, K fused steps per launch + compaction |
529 µs (505 µs–550 µs) |
– |
1.03 ms |
4.32e8 |
1.90e10 (7.5 %) |
5.27e9 (1.6 %) |
0 MiB (0×) |
2.56 MiB |
– |
– |
– |
1590–1590 |
10,000 |
eagle auto (eagle picks the launch mode) |
297 µs (291 µs–305 µs) |
– |
749 µs |
7.69e8 |
3.39e10 (13 %) |
2.27e10 (7.1 %) |
0 MiB (0×) |
2.37 MiB |
– |
– |
– |
1590–1590 |
10,000 |
hawk + eagle persistent |
339 µs (327 µs–346 µs) |
– |
761 µs |
6.74e8 |
2.96e10 (12 %) |
1.99e10 (6.2 %) |
0 MiB (0×) |
2.44 MiB |
– |
0.21156370013627354 |
– |
1560–1560 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
330 µs (309 µs–343 µs) |
– |
783 µs |
6.92e8 |
3.05e10 (12 %) |
2.04e10 (6.4 %) |
0 MiB (0×) |
2.52 MiB |
– |
– |
– |
1560–1560 |
10,000 |
eager launch loop |
3.43 ms (3.4 ms–3.47 ms) |
– |
3.82 ms |
6.67e7 |
2.94e9 (1.2 %) |
1.15e10 (3.6 %) |
0 MiB (0×) |
0.336 MiB |
1.4 % |
– |
– |
1110–1560 |
10,000 |
CuPy masked |
68.9 ms (68.3 ms–69.3 ms) |
– |
69.3 ms |
3.32e6 |
1.46e8 (0.057 %) |
1.31e10 (4.1 %) |
0 MiB (0×) |
195 MiB |
– |
– |
– |
795–870 |
10,000 |
PyTorch masked |
42.8 ms (42.7 ms–42.8 ms) |
– |
43.3 ms |
5.35e6 |
2.35e8 (0.092 %) |
2.11e10 (6.6 %) |
14 MiB (40.8×) |
0.746 MiB |
– |
– |
– |
1095–1095 |
10,000 |
PyTorch, CUDA graph of 16 steps |
10.1 ms (10.1 ms–10.2 ms) |
– |
10.6 ms |
2.25e7 |
9.91e8 (0.39 %) |
9.99e10 (31 %) |
42 MiB (122×) |
0.781 MiB |
– |
– |
– |
1590–1590 |
10,000 |
PyTorch torch.compile (reduce-overhead) |
15.4 ms (15.3 ms–15.5 ms) |
– |
15.9 ms |
1.48e7 |
6.52e8 (0.26 %) |
5.70e9 (1.8 %) |
56 MiB (163×) |
80.5 MiB |
– |
– |
– |
1590–1590 |
10,000 |
JAX jit + vmap(while_loop) |
3.36 ms (3.32 ms–3.41 ms) |
– |
5.55 ms |
6.80e7 |
2.99e9 (1.2 %) |
2.63e10 (8.2 %) |
8 MiB (23.3×) |
0.453 MiB |
– |
– |
– |
1020–1065 |
10,000 |
Warp per-thread kernel |
458 µs (441 µs–490 µs) |
– |
1.07 ms |
4.99e8 |
2.20e10 (8.6 %) |
1.40e9 (0.44 %) |
30 MiB (87.4×) |
0.754 MiB |
– |
– |
– |
990–990 |
10,000 |
CPU OpenMP |
1.41 ms (1.39 ms–1.42 ms) |
– |
1.51 ms |
1.62e8 |
7.14e9 (–) |
2.80e10 (–) |
– |
1.18 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
2.52 ms (2.5 ms–2.54 ms) |
– |
4.35 ms |
9.08e8 |
4.00e10 (16 %) |
3.87e10 (12 %) |
4 MiB (1.17×) |
7.91 MiB |
20 % |
– |
– |
1590–1590 |
100,000 |
eagle graph + compaction |
2.12 ms (2.02 ms–2.29 ms) |
– |
3.94 ms |
1.08e9 |
4.74e10 (19 %) |
9.54e10 (30 %) |
4 MiB (1.17×) |
7.95 MiB |
23 % |
– |
– |
1320–1590 |
100,000 |
eagle graph + compaction + reorder |
2.75 ms (2.7 ms–2.8 ms) |
– |
4.66 ms |
8.34e8 |
3.67e10 (14 %) |
8.93e10 (28 %) |
8 MiB (2.33×) |
8.5 MiB |
18 % |
– |
– |
1590–1590 |
100,000 |
eagle graph, K fused steps per launch |
2.11 ms (2.1 ms–2.11 ms) |
– |
3.84 ms |
1.08e9 |
4.77e10 (19 %) |
1.36e10 (4.3 %) |
4 MiB (1.17×) |
7.93 MiB |
– |
– |
– |
1590–1590 |
100,000 |
eagle graph, K fused steps per launch + compaction |
2 ms (1.92 ms–2.06 ms) |
– |
3.88 ms |
1.15e9 |
5.04e10 (20 %) |
1.40e10 (4.4 %) |
4 MiB (1.17×) |
8.01 MiB |
– |
– |
– |
1425–1590 |
100,000 |
eagle auto (eagle picks the launch mode) |
1.05 ms (1.01 ms–1.14 ms) |
– |
2.88 ms |
2.18e9 |
9.58e10 (38 %) |
6.42e10 (20 %) |
4 MiB (1.17×) |
7.72 MiB |
– |
– |
– |
960–1125 |
100,000 |
hawk + eagle persistent |
1.33 ms (1.24 ms–1.38 ms) |
– |
3.09 ms |
1.72e9 |
7.58e10 (30 %) |
5.08e10 (16 %) |
4 MiB (1.17×) |
7.54 MiB |
– |
0.42353027924346015 |
– |
1260–1470 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
867 µs (864 µs–869 µs) |
– |
2.69 ms |
2.64e9 |
1.16e11 (46 %) |
7.78e10 (24 %) |
4 MiB (1.17×) |
7.64 MiB |
– |
– |
– |
1305–1305 |
100,000 |
eager launch loop |
5.81 ms (5.8 ms–5.83 ms) |
– |
7.6 ms |
3.94e8 |
1.73e10 (6.8 %) |
6.78e10 (21 %) |
4 MiB (1.17×) |
5.38 MiB |
8.5 % |
– |
– |
1275–1275 |
100,000 |
CuPy masked |
67.9 ms (67.2 ms–68.4 ms) |
– |
70 ms |
3.37e7 |
1.48e9 (0.58 %) |
1.33e11 (41 %) |
22 MiB (6.41×) |
9.35 MiB |
– |
– |
– |
1125–1125 |
100,000 |
PyTorch masked |
42 ms (40 ms–43.9 ms) |
– |
44.3 ms |
5.45e7 |
2.40e9 (0.94 %) |
2.14e11 (67 %) |
38 MiB (11.1×) |
6.92 MiB |
– |
– |
– |
1485–1560 |
100,000 |
PyTorch, CUDA graph of 16 steps |
30.7 ms (30.5 ms–30.9 ms) |
– |
33 ms |
7.45e7 |
3.28e9 (1.3 %) |
3.30e11 (1e+02 %) |
64 MiB (18.6×) |
6.87 MiB |
– |
– |
– |
1305–1350 |
100,000 |
PyTorch torch.compile (reduce-overhead) |
19.5 ms (18.8 ms–20.6 ms) |
– |
21.8 ms |
1.17e8 |
5.15e9 (2 %) |
4.50e10 (14 %) |
66 MiB (19.2×) |
87.1 MiB |
– |
– |
– |
660–660 |
100,000 |
JAX jit + vmap(while_loop) |
7.27 ms (7.22 ms–7.3 ms) |
– |
10.6 ms |
3.15e8 |
1.39e10 (5.4 %) |
1.22e11 (38 %) |
20 MiB (5.83×) |
10.9 MiB |
– |
– |
– |
1590–1590 |
100,000 |
Warp per-thread kernel |
2.13 ms (2.12 ms–2.15 ms) |
– |
4.51 ms |
1.07e9 |
4.72e10 (19 %) |
3.00e9 (0.94 %) |
30 MiB (8.74×) |
6.87 MiB |
– |
– |
– |
1590–1590 |
100,000 |
CPU OpenMP |
6.87 ms (6.73 ms–6.99 ms) |
– |
7.49 ms |
3.33e8 |
1.47e10 (–) |
5.74e10 (–) |
– |
9.98 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
18.3 ms (18.3 ms–18.3 ms) |
– |
32.2 ms |
1.26e9 |
5.54e10 (22 %) |
5.96e10 (19 %) |
40 MiB (1.17×) |
62.9 MiB |
27 % |
– |
– |
1590–1590 |
1,000,000 |
eagle graph + compaction |
23 ms (22.9 ms–23 ms) |
– |
36.6 ms |
1.00e9 |
4.41e10 (17 %) |
8.86e10 (28 %) |
56 MiB (1.63×) |
63 MiB |
22 % |
– |
– |
1305–1320 |
1,000,000 |
eagle graph + compaction + reorder |
16.1 ms (16 ms–16.3 ms) |
– |
30.1 ms |
1.43e9 |
6.27e10 (25 %) |
1.53e11 (48 %) |
84 MiB (2.45×) |
63.2 MiB |
31 % |
– |
– |
1425–1560 |
1,000,000 |
eagle graph, K fused steps per launch |
17.6 ms (17.6 ms–17.7 ms) |
– |
31.8 ms |
1.30e9 |
5.74e10 (23 %) |
1.63e10 (5.1 %) |
40 MiB (1.17×) |
62.9 MiB |
– |
– |
– |
1590–1590 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
13.9 ms (13.9 ms–14 ms) |
– |
28.2 ms |
1.65e9 |
7.26e10 (29 %) |
2.01e10 (6.3 %) |
56 MiB (1.63×) |
62.9 MiB |
– |
– |
– |
1590–1590 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
8.58 ms (8.57 ms–8.59 ms) |
– |
22.8 ms |
2.68e9 |
1.18e11 (46 %) |
7.90e10 (25 %) |
40 MiB (1.17×) |
60.1 MiB |
– |
– |
– |
1590–1590 |
1,000,000 |
hawk + eagle persistent |
8.64 ms (8.62 ms–8.65 ms) |
– |
22.3 ms |
2.66e9 |
1.17e11 (46 %) |
7.85e10 (25 %) |
40 MiB (1.17×) |
60 MiB |
– |
0.49831346553288575 |
– |
1590–1590 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
8.59 ms (8.59 ms–8.6 ms) |
– |
22.4 ms |
2.68e9 |
1.18e11 (46 %) |
7.89e10 (25 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
1590–1590 |
1,000,000 |
eager launch loop |
26.2 ms (26 ms–26.4 ms) |
– |
40.1 ms |
8.79e8 |
3.87e10 (15 %) |
1.51e11 (47 %) |
40 MiB (1.17×) |
57.9 MiB |
19 % |
– |
– |
1410–1500 |
1,000,000 |
CuPy masked |
370 ms (370 ms–370 ms) |
– |
387 ms |
6.22e7 |
2.74e9 (1.1 %) |
2.44e11 (76 %) |
192 MiB (5.59×) |
68.7 MiB |
– |
– |
– |
1065–1110 |
1,000,000 |
PyTorch masked |
387 ms (386 ms–387 ms) |
– |
412 ms |
5.95e7 |
2.62e9 (1 %) |
2.33e11 (73 %) |
274 MiB (7.98×) |
68.7 MiB |
– |
– |
– |
1005–1125 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
427 ms (427 ms–427 ms) |
– |
452 ms |
5.39e7 |
2.37e9 (0.93 %) |
2.37e11 (74 %) |
302 MiB (8.8×) |
68.7 MiB |
– |
– |
– |
975–1140 |
1,000,000 |
PyTorch torch.compile (reduce-overhead) |
51 ms (50.3 ms–51.3 ms) |
– |
75.2 ms |
4.51e8 |
1.99e10 (7.8 %) |
1.73e11 (54 %) |
136 MiB (3.96×) |
136 MiB |
– |
– |
– |
1305–1425 |
1,000,000 |
JAX jit + vmap(while_loop) |
59.6 ms (59.4 ms–59.8 ms) |
– |
76.2 ms |
3.86e8 |
1.70e10 (6.7 %) |
1.48e11 (46 %) |
136 MiB (3.96×) |
101 MiB |
– |
– |
– |
1245–1290 |
1,000,000 |
Warp per-thread kernel |
19.6 ms (19.6 ms–19.6 ms) |
– |
50.3 ms |
1.17e9 |
5.15e10 (20 %) |
3.26e9 (1 %) |
62 MiB (1.81×) |
68.7 MiB |
– |
– |
– |
1590–1590 |
1,000,000 |
CPU OpenMP |
61.2 ms (60.8 ms–62.1 ms) |
– |
74.6 ms |
3.76e8 |
1.65e10 (–) |
6.44e10 (–) |
– |
99.4 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (160 µs), then eagle.simulate (eagle picks the launch mode) (161 µs, 1.01× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (297 µs), then eagle.simulate (eagle picks the launch mode) (330 µs, 1.11× the time)
N = 100,000: eagle.simulate (eagle picks the launch mode) (867 µs), then eagle auto (eagle picks the launch mode) (1.05 ms, 1.21× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (8.58 ms), then eagle.simulate (eagle picks the launch mode) (8.59 ms, 1× the time)
Spread stop steps (log-uniform in [10, 1000]), up to 1000 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
1.26 ms (1.17 ms–1.31 ms) |
– |
1.5 ms |
1.67e8 |
7.34e9 (2.9 %) |
5.17e9 (1.6 %) |
0 MiB (0×) |
2.15 MiB |
3.6 % |
– |
– |
1590–1590 |
1,000 |
eagle graph + compaction |
4.62 ms (4.6 ms–4.65 ms) |
– |
4.92 ms |
4.55e7 |
2.00e9 (0.79 %) |
3.42e9 (1.1 %) |
0 MiB (0×) |
2.17 MiB |
0.98 % |
– |
– |
1590–1590 |
1,000 |
eagle graph + compaction + reorder |
4.93 ms (4.9 ms–4.95 ms) |
– |
5.27 ms |
4.26e7 |
1.88e9 (0.74 %) |
3.35e9 (1 %) |
0 MiB (0×) |
2.16 MiB |
0.92 % |
– |
– |
1590–1590 |
1,000 |
eagle graph, K fused steps per launch |
1.1 ms (1.07 ms–1.12 ms) |
– |
1.34 ms |
1.92e8 |
8.44e9 (3.3 %) |
2.25e9 (0.7 %) |
0 MiB (0×) |
2.16 MiB |
– |
– |
– |
1590–1590 |
1,000 |
eagle graph, K fused steps per launch + compaction |
1.25 ms (1.24 ms–1.28 ms) |
– |
1.56 ms |
1.68e8 |
7.38e9 (2.9 %) |
1.14e9 (0.36 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
1590–1590 |
1,000 |
eagle auto (eagle picks the launch mode) |
515 µs (512 µs–518 µs) |
– |
810 µs |
4.08e8 |
1.80e10 (7.1 %) |
1.15e10 (3.6 %) |
0 MiB (0×) |
2.07 MiB |
– |
– |
– |
1590–1590 |
1,000 |
hawk + eagle persistent |
1.1 ms (1.09 ms–1.11 ms) |
– |
1.35 ms |
1.91e8 |
8.39e9 (3.3 %) |
5.37e9 (1.7 %) |
0 MiB (0×) |
2.09 MiB |
– |
0.22980196323115862 |
– |
1590–1590 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
526 µs (513 µs–538 µs) |
– |
835 µs |
4.00e8 |
1.76e10 (6.9 %) |
1.13e10 (3.5 %) |
0 MiB (0×) |
2.09 MiB |
– |
– |
– |
1590–1590 |
1,000 |
eager launch loop |
33.9 ms (33.9 ms–34 ms) |
– |
34.2 ms |
6.19e6 |
2.72e8 (0.11 %) |
1.15e9 (0.36 %) |
0 MiB (0×) |
0.152 MiB |
0.13 % |
– |
– |
1590–1590 |
1,000 |
CuPy masked |
666 ms (655 ms–672 ms) |
– |
666 ms |
3.16e5 |
1.39e7 (0.0055 %) |
1.35e9 (0.42 %) |
0 MiB (0×) |
4.95 MiB |
– |
– |
– |
585–660 |
1,000 |
PyTorch masked |
439 ms (431 ms–447 ms) |
– |
440 ms |
4.78e5 |
2.10e7 (0.0083 %) |
2.05e9 (0.64 %) |
14 MiB (408×) |
0.0742 MiB |
– |
– |
– |
990–1035 |
1,000 |
PyTorch, CUDA graph of 16 steps |
89.6 ms (89.6 ms–89.6 ms) |
– |
89.9 ms |
2.35e6 |
1.03e8 (0.041 %) |
1.02e10 (3.2 %) |
42 MiB (1.22e+03×) |
0.281 MiB |
– |
– |
– |
1590–1590 |
1,000 |
PyTorch torch.compile (reduce-overhead) |
143 ms (141 ms–146 ms) |
– |
144 ms |
1.47e6 |
6.45e7 (0.025 %) |
6.14e8 (0.19 %) |
36 MiB (1.05e+03×) |
32.8 MiB |
– |
– |
– |
1590–1590 |
1,000 |
JAX jit + vmap(while_loop) |
26.6 ms (26.1 ms–26.9 ms) |
– |
28.9 ms |
7.91e6 |
3.48e8 (0.14 %) |
3.31e9 (1 %) |
8 MiB (233×) |
0.0703 MiB |
– |
– |
– |
945–945 |
1,000 |
Warp per-thread kernel |
2.1 ms (2.1 ms–2.1 ms) |
– |
2.58 ms |
1.00e8 |
4.40e9 (1.7 %) |
3.04e7 (0.0095 %) |
30 MiB (874×) |
0.0781 MiB |
– |
– |
– |
1590–1590 |
1,000 |
CPU OpenMP |
1.48 ms (1.48 ms–1.49 ms) |
– |
1.5 ms |
1.42e8 |
6.25e9 (–) |
2.63e10 (–) |
– |
0.168 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
2.67 ms (2.66 ms–2.68 ms) |
– |
3.08 ms |
8.18e8 |
3.60e10 (14 %) |
2.53e10 (7.9 %) |
0 MiB (0×) |
2.49 MiB |
18 % |
– |
– |
1590–1590 |
10,000 |
eagle graph + compaction |
5.75 ms (5.75 ms–5.76 ms) |
– |
6.26 ms |
3.79e8 |
1.67e10 (6.6 %) |
2.84e10 (8.9 %) |
0 MiB (0×) |
2.5 MiB |
8.2 % |
– |
– |
1590–1590 |
10,000 |
eagle graph + compaction + reorder |
6.41 ms (6.4 ms–6.41 ms) |
– |
7.02 ms |
3.41e8 |
1.50e10 (5.9 %) |
2.64e10 (8.3 %) |
0 MiB (0×) |
2.98 MiB |
7.3 % |
– |
– |
1590–1590 |
10,000 |
eagle graph, K fused steps per launch |
2.57 ms (2.56 ms–2.59 ms) |
– |
2.97 ms |
8.49e8 |
3.73e10 (15 %) |
9.62e9 (3 %) |
0 MiB (0×) |
2.44 MiB |
– |
– |
– |
1590–1590 |
10,000 |
eagle graph, K fused steps per launch + compaction |
1.97 ms (1.96 ms–1.98 ms) |
– |
2.41 ms |
1.11e9 |
4.88e10 (19 %) |
7.41e9 (2.3 %) |
0 MiB (0×) |
2.5 MiB |
– |
– |
– |
1590–1590 |
10,000 |
eagle auto (eagle picks the launch mode) |
2 ms (1.99 ms–2.01 ms) |
– |
2.44 ms |
1.09e9 |
4.81e10 (19 %) |
3.08e10 (9.6 %) |
0 MiB (0×) |
2.43 MiB |
– |
– |
– |
1590–1590 |
10,000 |
hawk + eagle persistent |
2.06 ms (2.04 ms–2.08 ms) |
– |
2.51 ms |
1.06e9 |
4.66e10 (18 %) |
2.98e10 (9.3 %) |
0 MiB (0×) |
2.31 MiB |
– |
0.2384722378329774 |
– |
1590–1590 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
2 ms (2 ms–2.01 ms) |
– |
2.53 ms |
1.09e9 |
4.79e10 (19 %) |
3.07e10 (9.6 %) |
0 MiB (0×) |
2.47 MiB |
– |
– |
– |
1590–1590 |
10,000 |
eager launch loop |
41.3 ms (41.2 ms–41.5 ms) |
– |
41.7 ms |
5.28e7 |
2.32e9 (0.91 %) |
9.46e9 (3 %) |
0 MiB (0×) |
0.191 MiB |
1.1 % |
– |
– |
585–585 |
10,000 |
CuPy masked |
674 ms (669 ms–680 ms) |
– |
675 ms |
3.24e6 |
1.42e8 (0.056 %) |
1.34e10 (4.2 %) |
0 MiB (0×) |
5.67 MiB |
– |
– |
– |
585–585 |
10,000 |
PyTorch masked |
428 ms (424 ms–434 ms) |
– |
428 ms |
5.10e6 |
2.24e8 (0.088 %) |
2.11e10 (6.6 %) |
14 MiB (40.8×) |
0.68 MiB |
– |
– |
– |
1035–1095 |
10,000 |
PyTorch, CUDA graph of 16 steps |
90 ms (89.9 ms–90 ms) |
– |
90.5 ms |
2.43e7 |
1.07e9 (0.42 %) |
1.01e11 (32 %) |
42 MiB (122×) |
0.719 MiB |
– |
– |
– |
1590–1590 |
10,000 |
PyTorch torch.compile (reduce-overhead) |
138 ms (131 ms–148 ms) |
– |
138 ms |
1.58e7 |
6.97e8 (0.27 %) |
6.38e9 (2 %) |
36 MiB (105×) |
33.5 MiB |
– |
– |
– |
585–585 |
10,000 |
JAX jit + vmap(while_loop) |
29.9 ms (29.6 ms–30 ms) |
– |
32.2 ms |
7.31e7 |
3.22e9 (1.3 %) |
2.95e10 (9.2 %) |
8 MiB (23.3×) |
0.695 MiB |
– |
– |
– |
1200–1305 |
10,000 |
Warp per-thread kernel |
2.25 ms (2.23 ms–2.28 ms) |
– |
2.82 ms |
9.72e8 |
4.28e10 (17 %) |
2.85e8 (0.089 %) |
30 MiB (87.4×) |
0.754 MiB |
– |
– |
– |
1590–1590 |
10,000 |
CPU OpenMP |
6.23 ms (6.17 ms–6.28 ms) |
– |
6.3 ms |
3.51e8 |
1.54e10 (–) |
6.28e10 (–) |
– |
1.29 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
18.8 ms (18.7 ms–18.8 ms) |
– |
20.5 ms |
1.15e9 |
5.08e10 (20 %) |
3.88e10 (12 %) |
4 MiB (1.17×) |
7.85 MiB |
25 % |
– |
– |
1590–1590 |
100,000 |
eagle graph + compaction |
15.7 ms (15.6 ms–15.8 ms) |
– |
17.5 ms |
1.38e9 |
6.07e10 (24 %) |
1.03e11 (32 %) |
4 MiB (1.17×) |
8.01 MiB |
30 % |
– |
– |
1500–1530 |
100,000 |
eagle graph + compaction + reorder |
14.6 ms (14.5 ms–14.7 ms) |
– |
16.5 ms |
1.48e9 |
6.53e10 (26 %) |
1.15e11 (36 %) |
8 MiB (2.33×) |
8.56 MiB |
32 % |
– |
– |
1590–1590 |
100,000 |
eagle graph, K fused steps per launch |
18.8 ms (18.7 ms–18.8 ms) |
– |
20.5 ms |
1.15e9 |
5.08e10 (20 %) |
1.32e10 (4.1 %) |
4 MiB (1.17×) |
7.99 MiB |
– |
– |
– |
1590–1590 |
100,000 |
eagle graph, K fused steps per launch + compaction |
7.96 ms (7.95 ms–7.97 ms) |
– |
9.68 ms |
2.72e9 |
1.20e11 (47 %) |
1.82e10 (5.7 %) |
4 MiB (1.17×) |
7.88 MiB |
– |
– |
– |
1590–1590 |
100,000 |
eagle auto (eagle picks the launch mode) |
5.85 ms (5.83 ms–5.88 ms) |
– |
7.67 ms |
3.70e9 |
1.63e11 (64 %) |
1.04e11 (33 %) |
4 MiB (1.17×) |
7.64 MiB |
– |
– |
– |
1590–1590 |
100,000 |
hawk + eagle persistent |
6.38 ms (6.36 ms–6.41 ms) |
– |
8.35 ms |
3.39e9 |
1.49e11 (59 %) |
9.55e10 (30 %) |
4 MiB (1.17×) |
7.57 MiB |
– |
0.6812188930842286 |
– |
1590–1590 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
5.86 ms (5.84 ms–5.89 ms) |
– |
7.53 ms |
3.70e9 |
1.63e11 (64 %) |
1.04e11 (33 %) |
4 MiB (1.17×) |
7.61 MiB |
– |
– |
– |
1590–1590 |
100,000 |
eager launch loop |
51.4 ms (50.8 ms–52.4 ms) |
– |
53.1 ms |
4.21e8 |
1.85e10 (7.3 %) |
7.60e10 (24 %) |
4 MiB (1.17×) |
5.37 MiB |
9.1 % |
– |
– |
1575–1590 |
100,000 |
CuPy masked |
678 ms (666 ms–693 ms) |
– |
680 ms |
3.20e7 |
1.41e9 (0.55 %) |
1.33e11 (42 %) |
22 MiB (6.41×) |
9.41 MiB |
– |
– |
– |
1320–1590 |
100,000 |
PyTorch masked |
434 ms (430 ms–436 ms) |
– |
436 ms |
4.99e7 |
2.20e9 (0.86 %) |
2.08e11 (65 %) |
38 MiB (11.1×) |
6.86 MiB |
– |
– |
– |
1455–1515 |
100,000 |
PyTorch, CUDA graph of 16 steps |
279 ms (279 ms–280 ms) |
– |
282 ms |
7.75e7 |
3.41e9 (1.3 %) |
3.27e11 (1e+02 %) |
64 MiB (18.6×) |
6.93 MiB |
– |
– |
– |
1275–1290 |
100,000 |
PyTorch torch.compile (reduce-overhead) |
193 ms (191 ms–196 ms) |
– |
195 ms |
1.12e8 |
4.94e9 (1.9 %) |
4.56e10 (14 %) |
42 MiB (12.2×) |
36.9 MiB |
– |
– |
– |
750–750 |
100,000 |
JAX jit + vmap(while_loop) |
71.8 ms (71.7 ms–72 ms) |
– |
75.1 ms |
3.02e8 |
1.33e10 (5.2 %) |
1.23e11 (38 %) |
20 MiB (5.83×) |
6.92 MiB |
– |
– |
– |
1515–1530 |
100,000 |
Warp per-thread kernel |
20.2 ms (20.2 ms–20.3 ms) |
– |
22.6 ms |
1.07e9 |
4.71e10 (19 %) |
3.16e8 (0.099 %) |
30 MiB (8.74×) |
6.93 MiB |
– |
– |
– |
1590–1590 |
100,000 |
CPU OpenMP |
58.2 ms (56.8 ms–59.5 ms) |
– |
58.8 ms |
3.72e8 |
1.64e10 (–) |
6.72e10 (–) |
– |
10 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
173 ms (173 ms–174 ms) |
– |
187 ms |
1.25e9 |
5.49e10 (22 %) |
5.11e10 (16 %) |
40 MiB (1.17×) |
62.9 MiB |
27 % |
– |
– |
1590–1590 |
1,000,000 |
eagle graph + compaction |
221 ms (221 ms–221 ms) |
– |
235 ms |
9.79e8 |
4.31e10 (17 %) |
7.34e10 (23 %) |
56 MiB (1.63×) |
62.8 MiB |
21 % |
– |
– |
1125–1170 |
1,000,000 |
eagle graph + compaction + reorder |
113 ms (113 ms–113 ms) |
– |
127 ms |
1.91e9 |
8.42e10 (33 %) |
1.49e11 (46 %) |
84 MiB (2.45×) |
63.1 MiB |
41 % |
– |
– |
1140–1245 |
1,000,000 |
eagle graph, K fused steps per launch |
172 ms (172 ms–172 ms) |
– |
186 ms |
1.26e9 |
5.53e10 (22 %) |
1.44e10 (4.5 %) |
40 MiB (1.17×) |
63 MiB |
– |
– |
– |
1590–1590 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
55.5 ms (55.1 ms–56.2 ms) |
– |
69.2 ms |
3.90e9 |
1.71e11 (67 %) |
2.61e10 (8.2 %) |
56 MiB (1.63×) |
62.9 MiB |
– |
– |
– |
1395–1500 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
46.1 ms (46.1 ms–46.1 ms) |
– |
60 ms |
4.69e9 |
2.06e11 (81 %) |
1.32e11 (41 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
1590–1590 |
1,000,000 |
hawk + eagle persistent |
46.2 ms (46.1 ms–46.2 ms) |
– |
59.7 ms |
4.68e9 |
2.06e11 (81 %) |
1.32e11 (41 %) |
40 MiB (1.17×) |
60 MiB |
– |
0.9002918367181713 |
– |
1590–1590 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
46.1 ms (46.1 ms–46.1 ms) |
– |
60.3 ms |
4.69e9 |
2.07e11 (81 %) |
1.32e11 (41 %) |
40 MiB (1.17×) |
60.1 MiB |
– |
– |
– |
1590–1590 |
1,000,000 |
eager launch loop |
294 ms (293 ms–294 ms) |
– |
308 ms |
7.36e8 |
3.24e10 (13 %) |
1.33e11 (42 %) |
40 MiB (1.17×) |
57.8 MiB |
16 % |
– |
– |
1410–1455 |
1,000,000 |
CuPy masked |
3.7 s (3.7 s–3.7 s) |
– |
3.72 s |
5.85e7 |
2.57e9 (1 %) |
2.44e11 (76 %) |
192 MiB (5.59×) |
68.9 MiB |
– |
– |
– |
1095–1110 |
1,000,000 |
PyTorch masked |
3.86 s (3.86 s–3.86 s) |
– |
3.89 s |
5.60e7 |
2.47e9 (0.97 %) |
2.33e11 (73 %) |
274 MiB (7.98×) |
68.7 MiB |
– |
– |
– |
1050–1275 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
3.84 s (3.84 s–3.84 s) |
– |
3.87 s |
5.63e7 |
2.48e9 (0.97 %) |
2.38e11 (74 %) |
302 MiB (8.8×) |
68.8 MiB |
– |
– |
– |
1020–1215 |
1,000,000 |
PyTorch torch.compile (reduce-overhead) |
530 ms (527 ms–536 ms) |
– |
555 ms |
4.08e8 |
1.80e10 (7.1 %) |
1.66e11 (52 %) |
116 MiB (3.38×) |
77.6 MiB |
– |
– |
– |
1320–1470 |
1,000,000 |
JAX jit + vmap(while_loop) |
613 ms (612 ms–613 ms) |
– |
630 ms |
3.53e8 |
1.55e10 (6.1 %) |
1.44e11 (45 %) |
136 MiB (3.96×) |
101 MiB |
– |
– |
– |
1215–1350 |
1,000,000 |
Warp per-thread kernel |
194 ms (193 ms–194 ms) |
– |
227 ms |
1.12e9 |
4.92e10 (19 %) |
3.31e8 (0.1 %) |
62 MiB (1.81×) |
68.7 MiB |
– |
– |
– |
1590–1590 |
1,000,000 |
CPU OpenMP |
594 ms (592 ms–599 ms) |
– |
609 ms |
3.64e8 |
1.60e10 (–) |
6.57e10 (–) |
– |
99.4 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (515 µs), then eagle.simulate (eagle picks the launch mode) (526 µs, 1.02× the time)
N = 10,000: eagle graph, K fused steps per launch + compaction (1.97 ms), then eagle auto (eagle picks the launch mode) (2 ms, 1.01× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (5.85 ms), then eagle.simulate (eagle picks the launch mode) (5.86 ms, 1× the time)
N = 1,000,000: eagle.simulate (eagle picks the launch mode) (46.1 ms), then eagle auto (eagle picks the launch mode) (46.1 ms, 1× the time)
Uniform stop step: every sample runs 1000 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
1.13 ms (1.12 ms–1.15 ms) |
– |
1.37 ms |
8.81e8 |
3.88e10 (15 %) |
2.52e10 (7.9 %) |
0 MiB (0×) |
2.02 MiB |
19 % |
– |
– |
1590–1590 |
1,000 |
eagle graph + compaction |
3.44 ms (3.43 ms–3.44 ms) |
– |
3.76 ms |
2.91e8 |
1.28e10 (5 %) |
1.90e10 (5.9 %) |
0 MiB (0×) |
2.18 MiB |
6.3 % |
– |
– |
1590–1590 |
1,000 |
eagle graph + compaction + reorder |
3.7 ms (3.69 ms–3.72 ms) |
– |
4.04 ms |
2.70e8 |
1.19e10 (4.7 %) |
1.77e10 (5.5 %) |
0 MiB (0×) |
2.22 MiB |
5.8 % |
– |
– |
1590–1590 |
1,000 |
eagle graph, K fused steps per launch |
1.13 ms (1.09 ms–1.18 ms) |
– |
1.4 ms |
8.89e8 |
3.91e10 (15 %) |
3.42e9 (1.1 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
1590–1590 |
1,000 |
eagle graph, K fused steps per launch + compaction |
1.07 ms (1.05 ms–1.08 ms) |
– |
1.38 ms |
9.37e8 |
4.12e10 (16 %) |
3.90e9 (1.2 %) |
0 MiB (0×) |
2.16 MiB |
– |
– |
– |
1590–1590 |
1,000 |
eagle auto (eagle picks the launch mode) |
520 µs (517 µs–525 µs) |
– |
855 µs |
1.92e9 |
8.46e10 (33 %) |
5.39e10 (17 %) |
0 MiB (0×) |
2.09 MiB |
– |
– |
– |
1590–1590 |
1,000 |
hawk + eagle persistent |
1.14 ms (1.14 ms–1.15 ms) |
– |
1.36 ms |
8.75e8 |
3.85e10 (15 %) |
2.45e10 (7.7 %) |
0 MiB (0×) |
2.09 MiB |
– |
0.95367431640625 |
– |
1590–1590 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
530 µs (524 µs–538 µs) |
– |
831 µs |
1.89e9 |
8.30e10 (33 %) |
5.29e10 (17 %) |
0 MiB (0×) |
2.09 MiB |
– |
– |
– |
1590–1590 |
1,000 |
eager launch loop |
30.3 ms (29.8 ms–31.3 ms) |
– |
30.5 ms |
3.30e7 |
1.45e9 (0.57 %) |
2.01e9 (0.63 %) |
0 MiB (0×) |
0.0859 MiB |
0.71 % |
– |
– |
1590–1590 |
1,000 |
CuPy masked |
655 ms (644 ms–668 ms) |
– |
655 ms |
1.53e6 |
6.72e7 (0.026 %) |
1.38e9 (0.43 %) |
0 MiB (0×) |
5.03 MiB |
– |
– |
– |
585–645 |
1,000 |
PyTorch masked |
427 ms (424 ms–428 ms) |
– |
427 ms |
2.34e6 |
1.03e8 (0.041 %) |
2.11e9 (0.66 %) |
14 MiB (408×) |
0.137 MiB |
– |
– |
– |
1020–1080 |
1,000 |
PyTorch, CUDA graph of 16 steps |
89.4 ms (89.1 ms–89.6 ms) |
– |
89.7 ms |
1.12e7 |
4.92e8 (0.19 %) |
1.02e10 (3.2 %) |
42 MiB (1.22e+03×) |
0.16 MiB |
– |
– |
– |
1590–1590 |
1,000 |
PyTorch torch.compile (reduce-overhead) |
148 ms (143 ms–153 ms) |
– |
148 ms |
6.77e6 |
2.98e8 (0.12 %) |
5.95e8 (0.19 %) |
36 MiB (1.05e+03×) |
32.9 MiB |
– |
– |
– |
600–1590 |
1,000 |
JAX jit + vmap(while_loop) |
27.2 ms (27.1 ms–27.5 ms) |
– |
29.4 ms |
3.67e7 |
1.62e9 (0.64 %) |
3.23e9 (1 %) |
8 MiB (233×) |
0.133 MiB |
– |
– |
– |
735–885 |
1,000 |
Warp per-thread kernel |
2.36 ms (2.35 ms–2.36 ms) |
– |
2.83 ms |
4.25e8 |
1.87e10 (7.3 %) |
2.72e7 (0.0085 %) |
30 MiB (874×) |
0.0156 MiB |
– |
– |
– |
1590–1590 |
1,000 |
CPU OpenMP |
1.28 ms (1.26 ms–1.3 ms) |
– |
1.31 ms |
7.81e8 |
3.44e10 (–) |
4.77e10 (–) |
– |
0.109 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
2.68 ms (2.65 ms–2.7 ms) |
– |
3.06 ms |
3.73e9 |
1.64e11 (65 %) |
1.07e11 (33 %) |
0 MiB (0×) |
2.47 MiB |
80 % |
– |
– |
1590–1590 |
10,000 |
eagle graph + compaction |
5.43 ms (5.41 ms–5.44 ms) |
– |
5.94 ms |
1.84e9 |
8.11e10 (32 %) |
1.20e11 (38 %) |
0 MiB (0×) |
2.56 MiB |
40 % |
– |
– |
1590–1590 |
10,000 |
eagle graph + compaction + reorder |
5.92 ms (5.9 ms–5.93 ms) |
– |
6.42 ms |
1.69e9 |
7.44e10 (29 %) |
1.10e11 (34 %) |
0 MiB (0×) |
3.04 MiB |
36 % |
– |
– |
1590–1590 |
10,000 |
eagle graph, K fused steps per launch |
2.67 ms (2.66 ms–2.68 ms) |
– |
3.08 ms |
3.75e9 |
1.65e11 (65 %) |
1.44e10 (4.5 %) |
0 MiB (0×) |
2.55 MiB |
– |
– |
– |
1590–1590 |
10,000 |
eagle graph, K fused steps per launch + compaction |
2.64 ms (2.63 ms–2.65 ms) |
– |
3.09 ms |
3.78e9 |
1.66e11 (65 %) |
1.58e10 (4.9 %) |
0 MiB (0×) |
2.55 MiB |
– |
– |
– |
1590–1590 |
10,000 |
eagle auto (eagle picks the launch mode) |
2.07 ms (2.06 ms–2.08 ms) |
– |
2.52 ms |
4.83e9 |
2.12e11 (84 %) |
1.35e11 (42 %) |
0 MiB (0×) |
2.33 MiB |
– |
– |
– |
1590–1590 |
10,000 |
hawk + eagle persistent |
2.15 ms (2.12 ms–2.16 ms) |
– |
2.53 ms |
4.66e9 |
2.05e11 (81 %) |
1.31e11 (41 %) |
0 MiB (0×) |
2.44 MiB |
– |
0.9750024960063898 |
– |
1590–1590 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
2.08 ms (2.08 ms–2.08 ms) |
– |
2.52 ms |
4.80e9 |
2.11e11 (83 %) |
1.35e11 (42 %) |
0 MiB (0×) |
2.33 MiB |
– |
– |
– |
1590–1590 |
10,000 |
eager launch loop |
41 ms (39.6 ms–42.3 ms) |
– |
41.4 ms |
2.44e8 |
1.07e10 (4.2 %) |
1.49e10 (4.7 %) |
0 MiB (0×) |
0.254 MiB |
5.3 % |
– |
– |
585–585 |
10,000 |
CuPy masked |
671 ms (664 ms–678 ms) |
– |
671 ms |
1.49e7 |
6.56e8 (0.26 %) |
1.34e10 (4.2 %) |
0 MiB (0×) |
5.61 MiB |
– |
– |
– |
585–585 |
10,000 |
PyTorch masked |
429 ms (425 ms–432 ms) |
– |
430 ms |
2.33e7 |
1.02e9 (0.4 %) |
2.10e10 (6.6 %) |
14 MiB (40.8×) |
0.625 MiB |
– |
– |
– |
1020–1065 |
10,000 |
PyTorch, CUDA graph of 16 steps |
90 ms (89.9 ms–90 ms) |
– |
90.4 ms |
1.11e8 |
4.89e9 (1.9 %) |
1.01e11 (32 %) |
42 MiB (122×) |
0.719 MiB |
– |
– |
– |
1590–1590 |
10,000 |
PyTorch torch.compile (reduce-overhead) |
145 ms (140 ms–150 ms) |
– |
146 ms |
6.89e7 |
3.03e9 (1.2 %) |
6.07e9 (1.9 %) |
36 MiB (105×) |
33.3 MiB |
– |
– |
– |
600–1590 |
10,000 |
JAX jit + vmap(while_loop) |
28.7 ms (28.4 ms–29.3 ms) |
– |
31 ms |
3.48e8 |
1.53e10 (6 %) |
3.07e10 (9.6 %) |
8 MiB (23.3×) |
0.77 MiB |
– |
– |
– |
1395–1395 |
10,000 |
Warp per-thread kernel |
2.38 ms (2.36 ms–2.4 ms) |
– |
3.05 ms |
4.21e9 |
1.85e11 (73 %) |
2.69e8 (0.084 %) |
30 MiB (87.4×) |
0.754 MiB |
– |
– |
– |
1590–1590 |
10,000 |
CPU OpenMP |
7.68 ms (7.55 ms–7.85 ms) |
– |
7.76 ms |
1.30e9 |
5.73e10 (–) |
7.94e10 (–) |
– |
1.26 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
20.4 ms (20.4 ms–20.4 ms) |
– |
22.1 ms |
4.90e9 |
2.16e11 (85 %) |
1.40e11 (44 %) |
4 MiB (1.17×) |
7.71 MiB |
1.1e+02 % |
– |
– |
1590–1590 |
100,000 |
eagle graph + compaction |
34 ms (33.8 ms–34.3 ms) |
– |
35.9 ms |
2.94e9 |
1.29e11 (51 %) |
1.92e11 (60 %) |
4 MiB (1.17×) |
7.69 MiB |
63 % |
– |
– |
1365–1410 |
100,000 |
eagle graph + compaction + reorder |
35.1 ms (34.9 ms–35.4 ms) |
– |
36.9 ms |
2.85e9 |
1.25e11 (49 %) |
1.86e11 (58 %) |
8 MiB (2.33×) |
8.3 MiB |
61 % |
– |
– |
1350–1380 |
100,000 |
eagle graph, K fused steps per launch |
20.7 ms (20.6 ms–20.7 ms) |
– |
22.5 ms |
4.84e9 |
2.13e11 (84 %) |
1.86e10 (5.8 %) |
4 MiB (1.17×) |
7.66 MiB |
– |
– |
– |
1590–1590 |
100,000 |
eagle graph, K fused steps per launch + compaction |
20.7 ms (20.6 ms–20.7 ms) |
– |
22.4 ms |
4.84e9 |
2.13e11 (84 %) |
2.02e10 (6.3 %) |
4 MiB (1.17×) |
7.74 MiB |
– |
– |
– |
1590–1590 |
100,000 |
eagle auto (eagle picks the launch mode) |
19.7 ms (19.7 ms–19.7 ms) |
– |
21.5 ms |
5.09e9 |
2.24e11 (88 %) |
1.43e11 (45 %) |
4 MiB (1.17×) |
7.64 MiB |
– |
– |
– |
1590–1590 |
100,000 |
hawk + eagle persistent |
19.7 ms (19.7 ms–19.7 ms) |
– |
21.5 ms |
5.08e9 |
2.23e11 (88 %) |
1.42e11 (44 %) |
4 MiB (1.17×) |
7.59 MiB |
– |
0.9765625 |
– |
1590–1590 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
19.7 ms (19.7 ms–19.7 ms) |
– |
21.4 ms |
5.08e9 |
2.24e11 (88 %) |
1.42e11 (45 %) |
4 MiB (1.17×) |
7.66 MiB |
– |
– |
– |
1590–1590 |
100,000 |
eager launch loop |
53 ms (51.9 ms–53.7 ms) |
– |
54.8 ms |
1.89e9 |
8.30e10 (33 %) |
1.15e11 (36 %) |
4 MiB (1.17×) |
5.51 MiB |
41 % |
– |
– |
1590–1590 |
100,000 |
CuPy masked |
667 ms (660 ms–671 ms) |
– |
669 ms |
1.50e8 |
6.60e9 (2.6 %) |
1.35e11 (42 %) |
22 MiB (6.41×) |
9.41 MiB |
– |
– |
– |
1590–1590 |
100,000 |
PyTorch masked |
439 ms (438 ms–441 ms) |
– |
442 ms |
2.28e8 |
1.00e10 (3.9 %) |
2.05e11 (64 %) |
38 MiB (11.1×) |
6.24 MiB |
– |
– |
– |
1500–1515 |
100,000 |
PyTorch, CUDA graph of 16 steps |
279 ms (279 ms–279 ms) |
– |
281 ms |
3.59e8 |
1.58e10 (6.2 %) |
3.27e11 (1e+02 %) |
64 MiB (18.6×) |
6.39 MiB |
– |
– |
– |
1290–1290 |
100,000 |
PyTorch torch.compile (reduce-overhead) |
207 ms (202 ms–210 ms) |
– |
210 ms |
4.83e8 |
2.12e10 (8.4 %) |
4.25e10 (13 %) |
42 MiB (12.2×) |
37 MiB |
– |
– |
– |
645–675 |
100,000 |
JAX jit + vmap(while_loop) |
71.7 ms (71.5 ms–71.9 ms) |
– |
75.3 ms |
1.39e9 |
6.13e10 (24 %) |
1.23e11 (38 %) |
20 MiB (5.83×) |
6.6 MiB |
– |
– |
– |
1530–1545 |
100,000 |
Warp per-thread kernel |
22.5 ms (22.5 ms–22.5 ms) |
– |
24.7 ms |
4.44e9 |
1.96e11 (77 %) |
2.84e8 (0.089 %) |
30 MiB (8.74×) |
6.87 MiB |
– |
– |
– |
1590–1590 |
100,000 |
CPU OpenMP |
67.6 ms (66.3 ms–68.6 ms) |
– |
68.2 ms |
1.48e9 |
6.51e10 (–) |
9.03e10 (–) |
– |
10 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
193 ms (193 ms–193 ms) |
– |
207 ms |
5.17e9 |
2.28e11 (89 %) |
1.48e11 (46 %) |
40 MiB (1.17×) |
60.2 MiB |
1.1e+02 % |
– |
– |
1590–1590 |
1,000,000 |
eagle graph + compaction |
345 ms (345 ms–346 ms) |
– |
360 ms |
2.90e9 |
1.27e11 (50 %) |
1.89e11 (59 %) |
56 MiB (1.63×) |
60.2 MiB |
62 % |
– |
– |
1125–1125 |
1,000,000 |
eagle graph + compaction + reorder |
345 ms (345 ms–345 ms) |
– |
360 ms |
2.90e9 |
1.28e11 (50 %) |
1.89e11 (59 %) |
84 MiB (2.45×) |
60.6 MiB |
62 % |
– |
– |
1125–1125 |
1,000,000 |
eagle graph, K fused steps per launch |
197 ms (197 ms–198 ms) |
– |
211 ms |
5.07e9 |
2.23e11 (88 %) |
1.95e10 (6.1 %) |
40 MiB (1.17×) |
60.1 MiB |
– |
– |
– |
1575–1575 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
198 ms (198 ms–198 ms) |
– |
212 ms |
5.05e9 |
2.22e11 (87 %) |
2.10e10 (6.6 %) |
56 MiB (1.63×) |
60.2 MiB |
– |
– |
– |
1575–1575 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
191 ms (191 ms–191 ms) |
– |
206 ms |
5.23e9 |
2.30e11 (90 %) |
1.47e11 (46 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
1590–1590 |
1,000,000 |
hawk + eagle persistent |
191 ms (191 ms–191 ms) |
– |
205 ms |
5.22e9 |
2.30e11 (90 %) |
1.46e11 (46 %) |
40 MiB (1.17×) |
59.9 MiB |
– |
0.9765625 |
– |
1590–1590 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
191 ms (191 ms–191 ms) |
– |
205 ms |
5.23e9 |
2.30e11 (90 %) |
1.47e11 (46 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
– |
1590–1590 |
1,000,000 |
eager launch loop |
350 ms (349 ms–351 ms) |
– |
365 ms |
2.86e9 |
1.26e11 (49 %) |
1.74e11 (54 %) |
40 MiB (1.17×) |
57.9 MiB |
62 % |
– |
– |
1215–1305 |
1,000,000 |
CuPy masked |
3.69 s (3.69 s–3.69 s) |
– |
3.71 s |
2.71e8 |
1.19e10 (4.7 %) |
2.44e11 (76 %) |
192 MiB (5.59×) |
65.3 MiB |
– |
– |
– |
1110–1110 |
1,000,000 |
PyTorch masked |
3.86 s (3.86 s–3.86 s) |
– |
3.89 s |
2.59e8 |
1.14e10 (4.5 %) |
2.33e11 (73 %) |
274 MiB (7.98×) |
62.1 MiB |
– |
– |
– |
1050–1305 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
3.84 s (3.84 s–3.84 s) |
– |
3.86 s |
2.61e8 |
1.15e10 (4.5 %) |
2.38e11 (74 %) |
302 MiB (8.8×) |
62.6 MiB |
– |
– |
– |
1020–1275 |
1,000,000 |
PyTorch torch.compile (reduce-overhead) |
532 ms (528 ms–537 ms) |
– |
557 ms |
1.88e9 |
8.28e10 (33 %) |
1.66e11 (52 %) |
116 MiB (3.38×) |
71.4 MiB |
– |
– |
– |
1305–1455 |
1,000,000 |
JAX jit + vmap(while_loop) |
610 ms (608 ms–613 ms) |
– |
629 ms |
1.64e9 |
7.21e10 (28 %) |
1.44e11 (45 %) |
136 MiB (3.96×) |
99.2 MiB |
– |
– |
– |
1215–1275 |
1,000,000 |
Warp per-thread kernel |
220 ms (220 ms–220 ms) |
– |
253 ms |
4.56e9 |
2.00e11 (79 %) |
2.92e8 (0.091 %) |
62 MiB (1.81×) |
68.8 MiB |
– |
– |
– |
1590–1590 |
1,000,000 |
CPU OpenMP |
668 ms (659 ms–673 ms) |
– |
683 ms |
1.50e9 |
6.59e10 (–) |
9.13e10 (–) |
– |
97.5 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (520 µs), then eagle.simulate (eagle picks the launch mode) (530 µs, 1.02× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (2.07 ms), then eagle.simulate (eagle picks the launch mode) (2.08 ms, 1.01× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (19.7 ms), then eagle.simulate (eagle picks the launch mode) (19.7 ms, 1× the time)
N = 1,000,000: eagle.simulate (eagle picks the launch mode) (191 ms), then eagle auto (eagle picks the launch mode) (191 ms, 1× the time)
Compaction cadence sweep (isolates the active-set map)
the active-set map alone. For a given K, matched_cadence (the plain per-sample step, no map) and compact (eagle_graph_compact’s active-set step) check the loop’s stop guard at the SAME cadence, once every K steps; the only structural difference between the two rows at a given K is whether the step kernel reads the active-set map. The main table’s eagle_graph row checks its guard every step, so its difference from eagle_graph_compact there mixes the cadence change with the map – this sweep does not.
Config: Spread stop steps (log-uniform in [10, 1000]), up to 1000 steps.
N |
K (steps between guard checks) |
matched cadence, no map (range) |
eagle graph + compaction (range) |
ratio (matched/compact, |
|---|---|---|---|---|
100,000 |
8 |
25.1 ms (25.1 ms–25.1 ms) |
17.9 ms (17.8 ms–18 ms) |
1.4× |
100,000 |
16 |
25.6 ms (25.6 ms–25.7 ms) |
15.5 ms (15.5 ms–15.5 ms) |
1.66× |
100,000 |
32 |
25.1 ms (25 ms–25.1 ms) |
14.6 ms (14.5 ms–14.7 ms) |
1.72× |
1,000,000 |
8 |
279 ms (278 ms–279 ms) |
238 ms (238 ms–238 ms) |
1.17× |
1,000,000 |
16 |
282 ms (282 ms–283 ms) |
227 ms (227 ms–227 ms) |
1.24× |
1,000,000 |
32 |
283 ms (283 ms–283 ms) |
221 ms (221 ms–221 ms) |
1.28× |
FP32: eagle vs Warp (float32, S = 1000)
Every arm runs the same oscillator in float32: eagle.simulate with scalar_type=”float32”, NVIDIA Warp’s per-thread kernel and JAX’s vmapped loop, all over the same float32 inputs. Each cell runs 2 untimed runs per arm first (eagle’s automatic arm picks its entry over them: the size rule, then a measured run of the other entry, keeping the faster; the entry it settled on is in the last column), then each arm in its own block (no interleaving): an untimed ramp of back-to-back runs for 0.5 s, then 5 timed runs. Before any timing counts, eagle and JAX reach step counts identical to Warp’s and states within absolute 0.0001 of Warp’s (the largest differences are in the JSON). Each time is the mean of the middle 3 of the 5 runs (highest and lowest dropped), with the range of those 3; eagle / Warp is the ratio of these means: below 1 eagle is faster. Before each arm’s block the GPU is cooled down: the block starts once the temperature is within 3 °C of the idle 73 °C and the GPU is idle, or after 20 s. Right after each timed run the SM clock is read (its range is shown per arm); cycles are the wall time times that clock, and eagle / Warp (cycles) is their ratio.
Every arm runs each sample to its own stop and no further than max_steps (S): eagle and Warp enforce the cap per sample, JAX in its while-loop condition, PyTorch and CuPy by the outer step count (a batch-level cap); an arm that runs whole blocks of steps per launch or replay stops at the first block boundary at or after it.
What each arm needed to get there: Warp, a hand-written step counter and cap inside the kernel; JAX, a cap in the while_loop condition; eagle, the kernel source unchanged, with its loop shape, unrolling and device entry chosen by hawk + eagle (per device, per batch).
Cap fixture (checked before any timing counts): N = 4,096, 8 samples with stop step 2S = 256, S = 128; every arm (3) reported k = min(stop step, S) per sample and the same finished count (4,088): passed.
distribution |
N |
eagle.simulate (range) |
Warp (range) |
JAX (range) |
eagle / Warp |
eagle / Warp (cycles) |
eagle entry |
|---|---|---|---|---|---|---|---|
uniform |
10,000 |
329 µs (323 µs–332 µs), 600–600 MHz |
273 µs (266 µs–286 µs), 600–600 MHz |
30.7 ms (30.7 ms–30.8 ms), 690–690 MHz |
1.2× |
1.2× |
persist |
uniform |
100,000 |
705 µs (700 µs–710 µs), 1560–1575 MHz |
704 µs (702 µs–707 µs), 1575–1575 MHz |
20.5 ms (19.7 ms–21.1 ms), 1590–1590 MHz |
1× |
0.999× |
fused_one |
uniform |
1,000,000 |
6.8 ms (6.69 ms–6.87 ms), 1350–1425 MHz |
6.75 ms (6.66 ms–6.79 ms), 1365–1410 MHz |
206 ms (206 ms–207 ms), 1200–1275 MHz |
1.01× |
1.01× |
persist |
spread |
10,000 |
179 µs (174 µs–186 µs), 1590–1590 MHz |
188 µs (179 µs–207 µs), 1590–1590 MHz |
30.2 ms (30 ms–30.3 ms), 900–915 MHz |
0.953× |
0.953× |
fused_one |
spread |
100,000 |
670 µs (666 µs–672 µs), 870–870 MHz |
731 µs (723 µs–741 µs), 1410–1410 MHz |
20.7 ms (20.5 ms–20.7 ms), 1590–1590 MHz |
0.917× |
0.566× |
persist |
spread |
1,000,000 |
1.77 ms (1.77 ms–1.78 ms), 1590–1590 MHz |
5.19 ms (5.19 ms–5.2 ms), 1590–1590 MHz |
206 ms (206 ms–207 ms), 1140–1230 MHz |
0.342× |
0.342× |
persist |
Fixed overhead per run (N = 64, S = 100)
item 3: wall at a cell small enough that per-run overhead dominates (N=64, S=100), each arm’s own kernel-only vs wall where the main nsys pass covers this cell’s N (it does not by default, so kernel_only_s is usually null here)
arm |
wall (range) |
kernel-only |
|---|---|---|
eagle graph (device loop) |
698 µs (688 µs–706 µs) |
– |
eagle graph + compaction |
754 µs (724 µs–772 µs) |
– |
eagle graph + compaction + reorder |
908 µs (893 µs–926 µs) |
– |
eagle graph, K fused steps per launch |
359 µs (344 µs–389 µs) |
– |
eagle graph, K fused steps per launch + compaction |
464 µs (458 µs–473 µs) |
– |
eagle auto (eagle picks the launch mode) |
219 µs (206 µs–233 µs) |
– |
hawk + eagle persistent |
300 µs (288 µs–314 µs) |
– |
eagle.simulate (eagle picks the launch mode) |
229 µs (219 µs–238 µs) |
– |
eager launch loop |
3.62 ms (3.46 ms–3.72 ms) |
– |
CuPy masked |
66.6 ms (65.5 ms–68.1 ms) |
– |
PyTorch masked |
42.4 ms (41.2 ms–43.2 ms) |
– |
PyTorch, CUDA graph of 16 steps |
7.89 ms (7.88 ms–7.91 ms) |
– |
PyTorch torch.compile (reduce-overhead) |
13.2 ms (12.9 ms–13.3 ms) |
– |
JAX jit + vmap(while_loop) |
2.13 ms (2.11 ms–2.15 ms) |
– |
Warp per-thread kernel |
236 µs (225 µs–250 µs) |
– |
CPU OpenMP |
341 µs (313 µs–391 µs) |
– |
Where each tool fits
Each tool below is written the way its users write it, and each brings something of its own. Times are walls (mean of the middle 3 of 5 runs) at the largest N; ratios are against the eagle graph arm (below 1 = less time). The fastest arm per N is listed under each table above.
eagle graph (device loop). The step written once as a hawk kernel; the whole loop runs on the device as one CUDA graph with a device-side stop guard, so a run is one launch with no host round trip per step. On this card: N = 1,000,000, spread S=100: 18.3 ms; spread S=1000: 173 ms; uniform S=1000: 193 ms. Fastest in 0 of 12 cells.
eagle graph + compaction. Adds an active-set map: the launch covers only the samples still running, rebuilt every 16 steps; it is made for batches that thin out (the spread configurations). On this card: N = 1,000,000, spread S=100: 23 ms (1.26× the eagle graph time); spread S=1000: 221 ms (1.27× the eagle graph time); uniform S=1000: 345 ms (1.79× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph + compaction + reorder. Adds a physical reorder when the live samples spread thin over their warps, so the mapped reads stay contiguous; the final restore to sample order is inside its wall. On this card: N = 1,000,000, spread S=100: 16.1 ms (0.882× the eagle graph time); spread S=1000: 113 ms (0.651× the eagle graph time); uniform S=1000: 345 ms (1.78× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph, K fused steps per launch. The same step with one decorator argument, @hawk.kernel(steps=16): each launch runs that many steps per sample with the state kept in registers, as Warp’s per-thread loop does, and the launches, guard checks and plane reads shrink by the same factor; its results are bit-identical to the eagle graph arm’s in every cell. K is chosen by hand here; the eagle_graph_auto arm picks it at run time. On this card: N = 1,000,000, spread S=100: 17.6 ms (0.965× the eagle graph time); spread S=1000: 172 ms (0.992× the eagle graph time); uniform S=1000: 197 ms (1.02× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph, K fused steps per launch + compaction. The K-steps kernel over the active-set map: the register-resident steps of the arm above, and the launches cover only the samples still running, so it fits batches that thin out. On this card: N = 1,000,000, spread S=100: 13.9 ms (0.762× the eagle graph time); spread S=1000: 55.5 ms (0.32× the eagle graph time); uniform S=1000: 198 ms (1.02× the eagle graph time). Fastest in 1 of 12 cells.
eagle auto (eagle picks the launch mode). The same step with @hawk.kernel(steps=”auto”) on the active-set kind, and nothing to tune: it picks the steps per launch itself after every launch, on the device, so a batch whose samples all run to the end gets Warp-like fusion of up to 64 steps per launch, and a batch that thins out gets shorter launches with the active set compacted after each one; no sample takes more steps than the cap. On this card: N = 1,000,000, spread S=100: 8.58 ms (0.469× the eagle graph time); spread S=1000: 46.1 ms (0.266× the eagle graph time); uniform S=1000: 191 ms (0.99× the eagle graph time). Fastest in 8 of 12 cells.
hawk + eagle persistent. The plain (not active-set) automatic kernel, forced through hawk’s persist entry: one launch, no graph, no map, no policy kernel, no cap at 64 – a grid of SMs blocks of 256 lanes each steal the next unfinished sample off a counter when their own finishes, so idle warps refill without ever rebuilding an index map; made for batches too large for one resident wave. On this card: N = 1,000,000, spread S=100: 8.64 ms (0.473× the eagle graph time); spread S=1000: 46.2 ms (0.266× the eagle graph time); uniform S=1000: 191 ms (0.99× the eagle graph time). Fastest in 0 of 12 cells.
eagle.simulate (eagle picks the launch mode). The same automatic kernel through eagle.simulate: the model, its state and its parameters by name, with no plan or plane binding to write; eagle deploys the kernel and builds the same device loop. On this card: N = 1,000,000, spread S=100: 8.59 ms (0.47× the eagle graph time); spread S=1000: 46.1 ms (0.266× the eagle graph time); uniform S=1000: 191 ms (0.99× the eagle graph time). Fastest in 3 of 12 cells.
eager launch loop. The same kernels launched from Python one step at a time; the host sees the state after every step. On this card: N = 1,000,000, spread S=100: 26.2 ms (1.43× the eagle graph time); spread S=1000: 294 ms (1.69× the eagle graph time); uniform S=1000: 350 ms (1.81× the eagle graph time). Fastest in 0 of 12 cells.
CuPy masked. NumPy-like array code on GPUs with no kernel to write; every sample is computed every step, so it fits dense batches where all samples run to the end (the uniform configuration). On this card: N = 1,000,000, spread S=100: 370 ms (20.2× the eagle graph time); spread S=1000: 3.7 s (21.3× the eagle graph time); uniform S=1000: 3.69 s (19.1× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch masked. The same array code in PyTorch tensors; it fits where the computation sits next to a PyTorch model or needs autograd. On this card: N = 1,000,000, spread S=100: 387 ms (21.1× the eagle graph time); spread S=1000: 3.86 s (22.2× the eagle graph time); uniform S=1000: 3.86 s (20× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch, CUDA graph of 16 steps. The same PyTorch code with its launch overhead removed by hand: 16 steps captured as one CUDA graph and replayed, one host check per replay, the masking unchanged. On this card: N = 1,000,000, spread S=100: 427 ms (23.4× the eagle graph time); spread S=1000: 3.84 s (22.1× the eagle graph time); uniform S=1000: 3.84 s (19.9× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch torch.compile (reduce-overhead). The PyTorch step under torch.compile(mode=”reduce-overhead”): Inductor-fused kernels replayed as a CUDA graph. On this card: N = 1,000,000, spread S=100: 51 ms (2.79× the eagle graph time); spread S=1000: 530 ms (3.05× the eagle graph time); uniform S=1000: 532 ms (2.75× the eagle graph time). Fastest in 0 of 12 cells.
JAX jit + vmap(while_loop). The whole loop compiled once, a per-sample while loop batched with vmap; the same code runs on CPUs and GPUs and differentiates with jax.grad. On this card: N = 1,000,000, spread S=100: 59.6 ms (3.26× the eagle graph time); spread S=1000: 613 ms (3.53× the eagle graph time); uniform S=1000: 610 ms (3.16× the eagle graph time). Fastest in 0 of 12 cells.
Warp per-thread kernel. Explicit per-thread kernels written in Python, maintained by NVIDIA and differentiable through wp.Tape; each thread runs its own sample and stops at its own step, so finished samples cost nothing. On this card: N = 1,000,000, spread S=100: 19.6 ms (1.07× the eagle graph time); spread S=1000: 194 ms (1.12× the eagle graph time); uniform S=1000: 220 ms (1.14× the eagle graph time). Fastest in 0 of 12 cells.
CPU OpenMP. The same hawk kernel on the host through eagle’s OpenMP team, for machines without a GPU. On this card: N = 1,000,000, spread S=100: 61.2 ms (3.35× the eagle graph time); spread S=1000: 594 ms (3.42× the eagle graph time); uniform S=1000: 668 ms (3.46× the eagle graph time). Fastest in 0 of 12 cells.
Compile time
Its own pass, never part of any wall. Each measurement is a fresh process: the imports and the GPU context start-up run untimed, then the timer spans the first call to a ready kernel (trace + code generation + compile + module load, or the cache lookup). Cold: every cache empty – hawk’s compile cache (HAWK_CACHE_DIR), CuPy’s kernel cache (CUPY_CACHE_DIR), the CUDA driver’s PTX cache (CUDA_CACHE_PATH), PyTorch Inductor’s and Triton’s caches (TORCHINDUCTOR_CACHE_DIR, TRITON_CACHE_DIR) and JAX’s persistent compilation cache (minimum compile time and entry size 0), each a fresh directory. Warm: a new process on the caches the first cold run populated. eagle arms: eagle.deploy of the arm’s hawk kernel, which returns once the device build is done (C++/CUDA emission, nvcc to PTX, driver load; eagle’s own small CuPy kernels compile beside it) while the host compiler builds the host side on another thread, and the arm’s set-up on a 64-sample batch (eagle’s graph built, one step run). CuPy: one masked step on 64 samples (every elementwise and reduction kernel compiled on first use). Warp: the kernel module’s code generation, compile and load on the first launch (64 samples), cached in WARP_CACHE_PATH. PyTorch torch.compile, where it runs: three steps at N (the first call traces and compiles, the second records the CUDA graph, the third replays it), per N (dynamic=False). JAX: lower + compile from abstract shapes, per N (XLA specialises on the shape). CPU OpenMP: the same eagle.deploy as the eagle graph arm and one host run, which waits for the host build. Medians of 3 cold and 3 warm processes.
arm |
compiled |
cold (median of 3) |
warm (median of 3) |
|---|---|---|---|
eagle graph (device loop) |
eagle.deploy of the hawk step kernel (the device build; the host side builds in the background), the device-loop graph |
1.44 s |
138 ms |
eagle graph + compaction |
eagle.deploy of the hawk active-set step kernel (the device build; the host side builds in the background), the compaction-loop graph |
1.36 s |
120 ms |
eagle graph + compaction + reorder |
eagle.deploy of the hawk active-set step kernel (the device build; the host side builds in the background), the compaction + reorder loop graph |
1.23 s |
120 ms |
eagle graph, K fused steps per launch |
eagle.deploy of the hawk K-steps kernel (the device build; the host side builds in the background), the device-loop graph |
1.28 s |
133 ms |
eagle graph, K fused steps per launch + compaction |
eagle.deploy of the hawk active-set K-steps kernel (the device build; the host side builds in the background), the compaction-loop graph |
1.36 s |
142 ms |
eagle auto (eagle picks the launch mode) |
eagle.deploy of the hawk active-set automatic kernel (the device build; the host side builds in the background), the policy kernel, the policy-loop graph |
1.18 s |
129 ms |
hawk + eagle persistent |
eagle.deploy of the hawk automatic kernel (the device build; the host side builds in the background), the persistent launch |
1.49 s |
129 ms |
eagle.simulate (eagle picks the launch mode) |
eagle.simulate’s deploy of the hawk active-set automatic kernel (the device build; the host side builds in the background), the policy kernel, the policy-loop graph |
1.36 s |
130 ms |
eager launch loop |
eagle.deploy of the hawk single step kernel (the device build; the host side builds in the background) |
1.07 s |
110 ms |
CuPy masked |
CuPy’s elementwise and reduction kernels of one masked step |
1.05 s |
35.5 ms |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 1,000 |
6.31 s |
4.17 s |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 10,000 |
6.61 s |
4.15 s |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 100,000 |
6.68 s |
4.15 s |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 1,000,000 |
6.85 s |
4.2 s |
Warp per-thread kernel |
Warp kernel module (code generation, NVRTC compile, load) |
869 ms |
196 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 1,000 |
261 ms |
58 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 10,000 |
232 ms |
57.6 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 100,000 |
275 ms |
53.8 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 1,000,000 |
284 ms |
58.3 ms |
CPU OpenMP |
eagle.deploy of the hawk step kernel and its first host run, which waits for the host build |
2.02 s |
129 ms |
No compile step: PyTorch masked, PyTorch, CUDA graph of 16 steps (PyTorch’s eager kernels ship prebuilt; the CUDA-graph capture is per N and recorded per configuration as torch_graph_capture_s).
Memory method
A separate pass after the timing pass. Each (arm, N, configuration) runs in a fresh process: imports, the CUDA context created by one tiny device operation, then the baseline read, before the arm builds or allocates anything; then the arm’s kernels built and a warm-up run on a 64-sample batch (every compile happens here; JAX also compiles for shape N from abstract shapes), so every library’s pool reserved during the warm-up counts in the reading; CuPy’s pools freed and torch.cuda.empty_cache() called after the warm-up (Warp’s pool release threshold set to 0); then the batch’s inputs generated, the arm set up for N (eagle builds its graph, PyTorch captures its CUDA graph, whose private memory pool counts) and one full run with upload and download. Device memory = the memory the CUDA driver accounts to the process (NVML per-process used memory) after the run minus at the baseline. The caching allocators of CuPy, PyTorch and JAX are left at their defaults and are not trimmed after the run; each keeps the memory it reserved until trimmed, so the reading after the run is the allocator’s high-water reservation, the same measure for every arm. A disabled pool returns memory between operations, so its peak could only be caught by sampling, which misses short peaks; the held reservation needs no sampling. Caveats: the reading is what the process holds, so each allocator’s rounding and growth policy count (JAX’s allocator grows in regions that can exceed the request; PyTorch’s rounds blocks up; a captured CUDA graph keeps a private pool; Warp’s stream-ordered pool reserves in chunks of tens of MiB); the driver accounts in pages of about 2 MiB, so small-N rows read 0 or one page. The per-library counters (CuPy pool, torch.cuda.max_memory_reserved/allocated, JAX memory_stats) are recorded in the JSON as a cross-check. Host memory = peak resident set size above the host baseline (taken after the warm-up; /proc/self/clear_refs reset there, then VmHWM). Minimum = the state the workload needs, N × ((2 state + 2 float64 scalars: omega, zeta) × 8 + 4 bytes for the stop step as a 32-bit integer) = 36 bytes per sample; the factor is device memory / minimum.
Device memory the process already held at the baseline (CUDA context, loaded kernels and libraries, the warm-up’s leftovers; at N = 1,000,000, first configuration): eagle graph (device loop) 106 MiB; eagle graph + compaction 106 MiB; eagle graph + compaction + reorder 106 MiB; eagle graph, K fused steps per launch 106 MiB; eagle graph, K fused steps per launch + compaction 106 MiB; eagle auto (eagle picks the launch mode) 106 MiB; hawk + eagle persistent 106 MiB; eagle.simulate (eagle picks the launch mode) 106 MiB; eager launch loop 106 MiB; CuPy masked 106 MiB; PyTorch masked 106 MiB; PyTorch, CUDA graph of 16 steps 106 MiB; PyTorch torch.compile (reduce-overhead) 106 MiB; JAX jit + vmap(while_loop) 106 MiB; Warp per-thread kernel 106 MiB.
spread (up to 100 steps)
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (160 µs), then eagle.simulate (eagle picks the launch mode) (161 µs, 1.01× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (297 µs), then eagle.simulate (eagle picks the launch mode) (330 µs, 1.11× the time)
N = 100,000: eagle.simulate (eagle picks the launch mode) (867 µs), then eagle auto (eagle picks the launch mode) (1.05 ms, 1.21× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (8.58 ms), then eagle.simulate (eagle picks the launch mode) (8.59 ms, 1× the time)
spread (up to 1000 steps)
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (515 µs), then eagle.simulate (eagle picks the launch mode) (526 µs, 1.02× the time)
N = 10,000: eagle graph, K fused steps per launch + compaction (1.97 ms), then eagle auto (eagle picks the launch mode) (2 ms, 1.01× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (5.85 ms), then eagle.simulate (eagle picks the launch mode) (5.86 ms, 1× the time)
N = 1,000,000: eagle.simulate (eagle picks the launch mode) (46.1 ms), then eagle auto (eagle picks the launch mode) (46.1 ms, 1× the time)
uniform (up to 1000 steps)
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (520 µs), then eagle.simulate (eagle picks the launch mode) (530 µs, 1.02× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (2.07 ms), then eagle.simulate (eagle picks the launch mode) (2.08 ms, 1.01× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (19.7 ms), then eagle.simulate (eagle picks the launch mode) (19.7 ms, 1× the time)
N = 1,000,000: eagle.simulate (eagle picks the launch mode) (191 ms), then eagle auto (eagle picks the launch mode) (191 ms, 1× the time)
eagle_simulate against its hand-built twin eagle_graph_auto:
spread (up to 100 steps), N = 1,000: wall 1.01× the twin’s time
spread (up to 100 steps), N = 10,000: wall 1.11× the twin’s time
spread (up to 100 steps), N = 100,000: wall 0.825× the twin’s time
spread (up to 100 steps), N = 1,000,000: wall 1× the twin’s time
spread (up to 1000 steps), N = 1,000: wall 1.02× the twin’s time
spread (up to 1000 steps), N = 10,000: wall 1× the twin’s time
spread (up to 1000 steps), N = 100,000: wall 1× the twin’s time
spread (up to 1000 steps), N = 1,000,000: wall 0.999× the twin’s time
uniform (up to 1000 steps), N = 1,000: wall 1.02× the twin’s time
uniform (up to 1000 steps), N = 10,000: wall 1.01× the twin’s time
uniform (up to 1000 steps), N = 100,000: wall 1× the twin’s time
uniform (up to 1000 steps), N = 1,000,000: wall 1× the twin’s time
Run on: driver 560.35.03; CUDA runtime 12.9; host CPU Intel Xeon W-2125 CPU @ 4.00GHz; compiler g++ (GCC) 11.5.0 20240719 (Red Hat 11.5.0-5.0.1); Python 3.12.14; Warp 1.17.0, JAX 0.11.2, PyTorch 2.12.1, CuPy 14.2.0, NumPy 2.5.3.
N = 1,000,000
Log scale, from 17.6 ms to 1.19 s.
N = 1,000,000
Log scale, from 93.6 ms to 11.8 s.
N = 1,000,000
Log scale, from 400 ms to 11.9 s.
Table of every arm and every N
Device: Quadro P2000 (compute capability 6.1, 8 SMs). Peak FP64: 9.48e10 FLOP/s. Peak DRAM bandwidth: 1.40e11 B/s. Mean of the middle 3 of 5 runs of each arm (the highest and the lowest dropped), range of those 3 in parentheses.
Each arm’s block starts once the GPU is within 3 °C of its idle 58 °C and reads idle, or after 20 s; the SM clock is read right after each timed run and its range is in the last column.
The FP64 peak above is at the device’s reported maximum clock (1480.5 MHz). During the timed repetitions the SM clock read 1075 MHz; at that clock the FP64 peak is 6.88e10 FLOP/s, and every FP64 percentage below is 1.38× larger against it.
“% of issue peak” (the eagle arms only) counts DP instructions (DFMA/DMUL/DADD/DSETP/MUFU), not useful FLOPs: the step’s SASS has 27 of them, against the device’s DP issue-slot throughput at the observed clock – a kernel that is already at the issue pipe’s limit can still read well under 100% of FP64 peak, because useful FLOPs/step is smaller than issued DP instructions/step.
eagle_graph checks its stop guard every step; eagle_graph_compact checks it once every 16 steps (the cadence its compaction runs on) and reads an active-set map built from it. The two rows below differ in both the cadence and the map; the compaction cadence sweep further down holds the cadence fixed at a few values and isolates the map alone.
PyTorch, CUDA graph of 16 steps captures 16 masked steps once as a torch.cuda.CUDAGraph over static buffers and replays it, with one host check of “any sample running” per replay: it removes the per-kernel launch overhead and keeps the masking (every sample is computed every step, finished ones held by torch.where, so the steps a replay runs past the last stop change nothing). JAX’s while loop is one compiled executable with no host check per step; batched by vmap, it runs until the slowest sample stops. Warp runs one thread per sample, each stopping at its own step.
PyTorch torch.compile (reduce-overhead): needs compute capability 7.0 or newer (Triton, Inductor’s GPU code generator); this GPU is 6.1, so it was not run.
Every arm runs each sample to its own stop and no further than max_steps (S): eagle and Warp enforce the cap per sample, JAX in its while-loop condition, PyTorch and CuPy by the outer step count (a batch-level cap); an arm that runs whole blocks of steps per launch or replay stops at the first block boundary at or after it.
What each arm needed to get there: Warp, a hand-written step counter and cap inside the kernel; JAX, a cap in the while_loop condition; eagle, the kernel source unchanged, with its loop shape, unrolling and device entry chosen by hawk + eagle (per device, per batch).
Cap fixture (checked before any timing counts): N = 4,096, 8 samples with stop step 2S = 256, S = 128; every arm (15) reported k = min(stop step, S) per sample and the same finished count (4,088): passed.
arm |
loop structure |
|---|---|
eagle graph (device loop) |
per launch over the batch, inside one CUDA graph: the plain decorator’s kernel, which finishes its own samples and runs the steps per launch eagle’s policy picks (8 to 64), guard checked on the device every launch |
eagle graph + compaction |
as the graph arm, the step launched over the active-set map only (recomputed every 16 steps) |
eagle graph + compaction + reorder |
as the compaction arm, plus a physical reorder of the per-sample planes when the live samples spread thin |
eagle graph, K fused steps per launch |
per launch over the batch, inside one CUDA graph: the step kernel runs up to K = 16 steps per sample with the state in registers, each sample leaving at its own stop step; guard checked on the device every launch |
eagle graph, K fused steps per launch + compaction |
as the K-steps arm, the launch over the active-set map only (recomputed every 64 steps = 4 launches) |
eagle auto (eagle picks the launch mode) |
the active-set kernel under eagle’s automatic policy, which picks the launch mode from the batch: one launch running every sample to its end for a small batch, the persistent launch above that, and the compaction loop (active-set map, steps per launch picked on the device) only when the caller asks for compaction or reordering; the row records the mode taken |
hawk + eagle persistent |
ONE launch, no graph, no map, no policy kernel: a grid sized from the SM count and the kernel’s occupancy, each lane fetching base + atomicAdd(counter, 1) whenever its sample is done, running the whole budget’s worth of steps per sample it picks up |
eagle.simulate (eagle picks the launch mode) |
as the automatic arm: eagle.simulate takes the kernel, its state and its parameters by name and eagle picks the launch mode the same way |
eager launch loop |
per step over the batch: one kernel launch from Python, a host read of the finished count every step |
CuPy masked |
per step over the batch: whole-batch array expressions, finished samples held by cupy.where, a host check every step; the cap is batch-level (the outer step loop runs at most max_steps) |
PyTorch masked |
per step over the batch: whole-batch tensor expressions, finished samples held by torch.where, a host check every step; the cap is batch-level (the outer step loop runs at most max_steps) |
PyTorch, CUDA graph of 16 steps |
as PyTorch masked, 16 steps captured as one CUDA graph over static buffers, replayed with one host check per replay |
JAX jit + vmap(while_loop) |
per step over the batch, inside one compiled executable: the vmapped while loop runs until the slowest sample stops or the step cap is reached, finished samples held by a select |
Warp per-thread kernel |
per sample: one launch, each thread loops over its own sample’s steps and stops at its own stop step or the step cap, whichever comes first |
CPU OpenMP |
per step over the batch on the host: one host-team launch per step, the kernel marks and counts its finished samples |
Spread stop steps (log-uniform in [1, 100]), up to 100 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
454 µs (451 µs–458 µs) |
121 µs |
572 µs |
4.86e7 |
2.14e9 (2.3 %) |
1.61e9 (1.1 %) |
0 MiB (0×) |
2 MiB |
3.8 % |
– |
2/8/40 |
1075–1075 |
1,000 |
eagle graph + compaction |
833 µs (832 µs–833 µs) |
617 µs |
989 µs |
2.65e7 |
1.17e9 (1.2 %) |
2.37e9 (1.7 %) |
0 MiB (0×) |
2 MiB |
2.1 % |
– |
100/2/18 |
1075–1075 |
1,000 |
eagle graph + compaction + reorder |
927 µs (925 µs–931 µs) |
642 µs |
1.1 ms |
2.38e7 |
1.05e9 (1.1 %) |
2.58e9 (1.8 %) |
0 MiB (0×) |
2 MiB |
1.9 % |
– |
100/3/26 |
1075–1075 |
1,000 |
eagle graph, K fused steps per launch |
279 µs (278 µs–280 µs) |
122 µs |
389 µs |
7.92e7 |
3.48e9 (3.7 %) |
1.03e9 (0.73 %) |
0 MiB (0×) |
2 MiB |
– |
– |
7/1/14 |
1075–1075 |
1,000 |
eagle graph, K fused steps per launch + compaction |
319 µs (317 µs–324 µs) |
146 µs |
472 µs |
6.90e7 |
3.04e9 (3.2 %) |
8.64e8 (0.62 %) |
0 MiB (0×) |
2 MiB |
– |
– |
7/2/18 |
1075–1075 |
1,000 |
eagle auto (eagle picks the launch mode) |
132 µs (132 µs–132 µs) |
– |
274 µs |
1.67e8 |
7.35e9 (7.8 %) |
4.94e9 (3.5 %) |
0 MiB (0×) |
2.12 MiB |
– |
– |
– |
1075–1075 |
1,000 |
hawk + eagle persistent |
193 µs (188 µs–203 µs) |
– |
307 µs |
1.14e8 |
5.02e9 (5.3 %) |
3.37e9 (2.4 %) |
0 MiB (0×) |
2.12 MiB |
– |
0.2098712531887755 |
– |
1075–1075 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
144 µs (140 µs–150 µs) |
– |
306 µs |
1.53e8 |
6.73e9 (7.1 %) |
4.52e9 (3.2 %) |
0 MiB (0×) |
2.12 MiB |
– |
– |
– |
1075–1075 |
1,000 |
eager launch loop |
1.83 ms (1.82 ms–1.84 ms) |
366 µs |
1.93 ms |
1.21e7 |
5.31e8 (0.56 %) |
2.14e9 (1.5 %) |
0 MiB (0×) |
0 MiB |
0.95 % |
– |
100/100/502 |
1075–1075 |
1,000 |
CuPy masked |
34.5 ms (34.3 ms–34.8 ms) |
7.99 ms |
34.6 ms |
6.40e5 |
2.82e7 (0.03 %) |
2.61e9 (1.9 %) |
0 MiB (0×) |
4.75 MiB |
– |
– |
4404/100/4705 |
1075–1075 |
1,000 |
PyTorch masked |
23.5 ms (23.3 ms–23.7 ms) |
11.7 ms |
23.7 ms |
9.40e5 |
4.13e7 (0.044 %) |
3.84e9 (2.7 %) |
4 MiB (117×) |
0.125 MiB |
– |
– |
100/100/4505 |
1075–1075 |
1,000 |
PyTorch, CUDA graph of 16 steps |
12.5 ms (12.4 ms–12.5 ms) |
13.2 ms |
12.6 ms |
1.77e6 |
7.79e7 (0.082 %) |
8.14e9 (5.8 %) |
32 MiB (932×) |
1.39 MiB |
– |
– |
100/28/26 |
1075–1075 |
1,000 |
JAX jit + vmap(while_loop) |
2.36 ms (2.26 ms–2.41 ms) |
617 µs |
3.7 ms |
9.33e6 |
4.11e8 (0.43 %) |
3.74e9 (2.7 %) |
8 MiB (233×) |
0.125 MiB |
– |
– |
100/103/418 |
1075–1075 |
1,000 |
Warp per-thread kernel |
198 µs (197 µs–202 µs) |
143 µs |
445 µs |
1.11e8 |
4.89e9 (5.2 %) |
3.23e8 (0.23 %) |
30 MiB (874×) |
0.125 MiB |
– |
– |
1/0/4 |
1075–1075 |
1,000 |
CPU OpenMP |
404 µs (342 µs–436 µs) |
– |
420 µs |
5.45e7 |
2.40e9 (–) |
9.69e9 (–) |
– |
0.125 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
1.01 ms (1.01 ms–1.01 ms) |
693 µs |
1.19 ms |
2.26e8 |
9.93e9 (10 %) |
8.52e9 (6.1 %) |
0 MiB (0×) |
2.38 MiB |
18 % |
– |
2/8/41 |
1075–1075 |
10,000 |
eagle graph + compaction |
1.14 ms (1.14 ms–1.14 ms) |
912 µs |
1.36 ms |
2.00e8 |
8.79e9 (9.3 %) |
1.77e10 (13 %) |
0 MiB (0×) |
2.5 MiB |
16 % |
– |
100/2/18 |
1075–1075 |
10,000 |
eagle graph + compaction + reorder |
1.45 ms (1.44 ms–1.45 ms) |
1.04 ms |
1.7 ms |
1.58e8 |
6.95e9 (7.3 %) |
1.69e10 (12 %) |
0 MiB (0×) |
2.88 MiB |
12 % |
– |
100/4/56 |
1075–1075 |
10,000 |
eagle graph, K fused steps per launch |
823 µs (818 µs–827 µs) |
667 µs |
999 µs |
2.78e8 |
1.22e10 (13 %) |
3.49e9 (2.5 %) |
0 MiB (0×) |
2.38 MiB |
– |
– |
7/1/14 |
1075–1075 |
10,000 |
eagle graph, K fused steps per launch + compaction |
728 µs (725 µs–731 µs) |
555 µs |
945 µs |
3.14e8 |
1.38e10 (15 %) |
3.83e9 (2.7 %) |
0 MiB (0×) |
2.64 MiB |
– |
– |
7/2/18 |
1075–1075 |
10,000 |
eagle auto (eagle picks the launch mode) |
318 µs (316 µs–321 µs) |
– |
541 µs |
7.18e8 |
3.16e10 (33 %) |
2.12e10 (15 %) |
0 MiB (0×) |
2.38 MiB |
– |
– |
– |
1075–1075 |
10,000 |
hawk + eagle persistent |
455 µs (448 µs–459 µs) |
– |
623 µs |
5.02e8 |
2.21e10 (23 %) |
1.48e10 (11 %) |
0 MiB (0×) |
2.38 MiB |
– |
0.3805129651292335 |
– |
1075–1075 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
331 µs (330 µs–333 µs) |
– |
554 µs |
6.90e8 |
3.04e10 (32 %) |
2.03e10 (15 %) |
0 MiB (0×) |
2.63 MiB |
– |
– |
– |
1075–1075 |
10,000 |
eager launch loop |
2.82 ms (2.77 ms–2.88 ms) |
1.05 ms |
3 ms |
8.12e7 |
3.57e9 (3.8 %) |
1.40e10 (10 %) |
0 MiB (0×) |
0.375 MiB |
6.4 % |
– |
100/100/502 |
1075–1075 |
10,000 |
CuPy masked |
37.4 ms (37.3 ms–37.4 ms) |
10.4 ms |
37.6 ms |
6.12e6 |
2.69e8 (0.28 %) |
2.41e10 (17 %) |
0 MiB (0×) |
5.12 MiB |
– |
– |
4404/100/4705 |
1075–1075 |
10,000 |
PyTorch masked |
24.1 ms (24.1 ms–24.1 ms) |
13.2 ms |
24.4 ms |
9.49e6 |
4.18e8 (0.44 %) |
3.74e10 (27 %) |
4 MiB (11.7×) |
0.75 MiB |
– |
– |
100/100/4505 |
1075–1075 |
10,000 |
PyTorch, CUDA graph of 16 steps |
15.7 ms (15.7 ms–15.7 ms) |
16.4 ms |
16 ms |
1.45e7 |
6.39e8 (0.67 %) |
6.44e10 (46 %) |
32 MiB (93.2×) |
1.88 MiB |
– |
– |
100/28/26 |
1075–1075 |
10,000 |
JAX jit + vmap(while_loop) |
3.54 ms (3.53 ms–3.54 ms) |
1.92 ms |
4.78 ms |
6.46e7 |
2.84e9 (3 %) |
2.50e10 (18 %) |
8 MiB (23.3×) |
0.625 MiB |
– |
– |
100/103/519 |
1075–1075 |
10,000 |
Warp per-thread kernel |
719 µs (715 µs–725 µs) |
667 µs |
1.05 ms |
3.18e8 |
1.40e10 (15 %) |
8.91e8 (0.64 %) |
30 MiB (87.4×) |
0.625 MiB |
– |
– |
1/0/4 |
1075–1075 |
10,000 |
CPU OpenMP |
993 µs (989 µs–996 µs) |
– |
1.05 ms |
2.30e8 |
1.01e10 (–) |
3.97e10 (–) |
– |
1 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
6.55 ms (6.54 ms–6.57 ms) |
6.16 ms |
7.49 ms |
3.49e8 |
1.54e10 (16 %) |
1.66e10 (12 %) |
4 MiB (1.17×) |
7.62 MiB |
27 % |
– |
2/8/41 |
1075–1075 |
100,000 |
eagle graph + compaction |
4.87 ms (4.87 ms–4.88 ms) |
4.86 ms |
5.93 ms |
4.69e8 |
2.07e10 (22 %) |
4.16e10 (30 %) |
4 MiB (1.17×) |
7.62 MiB |
37 % |
– |
100/2/18 |
1075–1075 |
100,000 |
eagle graph + compaction + reorder |
5.36 ms (5.34 ms–5.38 ms) |
5.2 ms |
6.39 ms |
4.27e8 |
1.88e10 (20 %) |
4.57e10 (33 %) |
8 MiB (2.33×) |
8.25 MiB |
33 % |
– |
100/4/58 |
1075–1075 |
100,000 |
eagle graph, K fused steps per launch |
6.15 ms (6.14 ms–6.17 ms) |
5.95 ms |
7.07 ms |
3.72e8 |
1.64e10 (17 %) |
4.68e9 (3.3 %) |
4 MiB (1.17×) |
7.62 MiB |
– |
– |
7/1/14 |
1075–1075 |
100,000 |
eagle graph, K fused steps per launch + compaction |
4.83 ms (4.83 ms–4.84 ms) |
4.66 ms |
5.77 ms |
4.73e8 |
2.08e10 (22 %) |
5.78e9 (4.1 %) |
4 MiB (1.17×) |
7.75 MiB |
– |
– |
7/2/18 |
1075–1075 |
100,000 |
eagle auto (eagle picks the launch mode) |
1.77 ms (1.77 ms–1.78 ms) |
– |
2.73 ms |
1.29e9 |
5.68e10 (60 %) |
3.81e10 (27 %) |
4 MiB (1.17×) |
7.88 MiB |
– |
– |
– |
1075–1075 |
100,000 |
hawk + eagle persistent |
3.01 ms (3 ms–3.01 ms) |
– |
3.94 ms |
7.61e8 |
3.35e10 (35 %) |
2.24e10 (16 %) |
4 MiB (1.17×) |
7.88 MiB |
– |
0.4879300888270548 |
– |
1075–1075 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
1.78 ms (1.78 ms–1.78 ms) |
– |
2.7 ms |
1.29e9 |
5.66e10 (60 %) |
3.79e10 (27 %) |
4 MiB (1.17×) |
7.75 MiB |
– |
– |
– |
1075–1075 |
100,000 |
eager launch loop |
9.37 ms (9.34 ms–9.4 ms) |
7.93 ms |
10.3 ms |
2.44e8 |
1.07e10 (11 %) |
4.21e10 (30 %) |
4 MiB (1.17×) |
5.5 MiB |
19 % |
– |
100/100/502 |
1075–1075 |
100,000 |
CuPy masked |
77.3 ms (77.3 ms–77.3 ms) |
76.2 ms |
78.4 ms |
2.96e7 |
1.30e9 (1.4 %) |
1.17e11 (83 %) |
22 MiB (6.41×) |
8.5 MiB |
– |
– |
4404/100/4705 |
1075–1075 |
100,000 |
PyTorch masked |
67.8 ms (67.8 ms–67.9 ms) |
65.5 ms |
69.3 ms |
3.37e7 |
1.48e9 (1.6 %) |
1.33e11 (95 %) |
28 MiB (8.16×) |
6.88 MiB |
– |
– |
100/100/4505 |
1075–1075 |
100,000 |
PyTorch, CUDA graph of 16 steps |
72.6 ms (72.5 ms–72.6 ms) |
72.7 ms |
74 ms |
3.15e7 |
1.39e9 (1.5 %) |
1.40e11 (1e+02 %) |
54 MiB (15.7×) |
7.62 MiB |
– |
– |
100/28/26 |
1075–1075 |
100,000 |
JAX jit + vmap(while_loop) |
17.1 ms (17.1 ms–17.1 ms) |
15.8 ms |
19.4 ms |
1.34e8 |
5.89e9 (6.2 %) |
5.16e10 (37 %) |
20 MiB (5.83×) |
7.01 MiB |
– |
– |
100/103/519 |
1075–1075 |
100,000 |
Warp per-thread kernel |
6.47 ms (6.43 ms–6.5 ms) |
6.4 ms |
7.95 ms |
3.54e8 |
1.56e10 (16 %) |
9.89e8 (0.71 %) |
30 MiB (8.74×) |
6.88 MiB |
– |
– |
1/0/4 |
1075–1075 |
100,000 |
CPU OpenMP |
8.01 ms (7.31 ms–8.76 ms) |
– |
8.69 ms |
2.86e8 |
1.26e10 (–) |
4.92e10 (–) |
– |
9.75 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
59.3 ms (59.2 ms–59.3 ms) |
59 ms |
66.2 ms |
3.88e8 |
1.71e10 (18 %) |
1.84e10 (13 %) |
40 MiB (1.17×) |
60 MiB |
30 % |
– |
2/8/41 |
1075–1075 |
1,000,000 |
eagle graph + compaction |
42 ms (41.9 ms–42 ms) |
44.4 ms |
49.1 ms |
5.48e8 |
2.41e10 (25 %) |
4.85e10 (35 %) |
56 MiB (1.63×) |
60.1 MiB |
43 % |
– |
100/2/18 |
1075–1075 |
1,000,000 |
eagle graph + compaction + reorder |
40.1 ms (40.1 ms–40.1 ms) |
43.2 ms |
47.2 ms |
5.73e8 |
2.52e10 (27 %) |
6.13e10 (44 %) |
84 MiB (2.45×) |
62.9 MiB |
45 % |
– |
100/4/58 |
1075–1075 |
1,000,000 |
eagle graph, K fused steps per launch |
58.2 ms (58.2 ms–58.2 ms) |
58 ms |
65.2 ms |
3.95e8 |
1.74e10 (18 %) |
4.95e9 (3.5 %) |
40 MiB (1.17×) |
60 MiB |
– |
– |
7/1/14 |
1075–1075 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
45 ms (45 ms–45 ms) |
44.9 ms |
52.3 ms |
5.11e8 |
2.25e10 (24 %) |
6.21e9 (4.4 %) |
56 MiB (1.63×) |
60 MiB |
– |
– |
7/2/18 |
1075–1075 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
28.1 ms (28.1 ms–28.1 ms) |
– |
35.4 ms |
8.18e8 |
3.60e10 (38 %) |
2.41e10 (17 %) |
40 MiB (1.17×) |
60.3 MiB |
– |
– |
– |
1075–1075 |
1,000,000 |
hawk + eagle persistent |
28.2 ms (28.1 ms–28.2 ms) |
– |
35.3 ms |
8.17e8 |
3.59e10 (38 %) |
2.41e10 (17 %) |
40 MiB (1.17×) |
60.3 MiB |
– |
0.5062860113174482 |
– |
1075–1075 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
28.1 ms (28.1 ms–28.1 ms) |
– |
35.5 ms |
8.18e8 |
3.60e10 (38 %) |
2.41e10 (17 %) |
40 MiB (1.17×) |
60.4 MiB |
– |
– |
– |
1075–1075 |
1,000,000 |
eager launch loop |
73 ms (72.9 ms–73.1 ms) |
72.7 ms |
80.3 ms |
3.15e8 |
1.39e10 (15 %) |
5.41e10 (39 %) |
40 MiB (1.17×) |
58.3 MiB |
25 % |
– |
100/100/502 |
1075–1075 |
1,000,000 |
CuPy masked |
739 ms (739 ms–739 ms) |
736 ms |
747 ms |
3.11e7 |
1.37e9 (1.4 %) |
1.22e11 (87 %) |
192 MiB (5.59×) |
69 MiB |
– |
– |
4404/100/4705 |
1075–1075 |
1,000,000 |
PyTorch masked |
741 ms (741 ms–741 ms) |
738 ms |
748 ms |
3.10e7 |
1.37e9 (1.4 %) |
1.22e11 (87 %) |
264 MiB (7.69×) |
68.8 MiB |
– |
– |
100/100/4605 |
1075–1075 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
827 ms (827 ms–827 ms) |
826 ms |
843 ms |
2.78e7 |
1.22e9 (1.3 %) |
1.23e11 (87 %) |
292 MiB (8.51×) |
68.7 MiB |
– |
– |
100/28/26 |
1075–1075 |
1,000,000 |
JAX jit + vmap(while_loop) |
138 ms (138 ms–138 ms) |
– |
152 ms |
1.66e8 |
7.32e9 (7.7 %) |
6.39e10 (46 %) |
136 MiB (3.96×) |
101 MiB |
– |
– |
– |
1075–1075 |
1,000,000 |
Warp per-thread kernel |
62.1 ms (62 ms–62.2 ms) |
62 ms |
81.4 ms |
3.70e8 |
1.63e10 (17 %) |
1.03e9 (0.74 %) |
62 MiB (1.81×) |
68.7 MiB |
– |
– |
1/0/4 |
1075–1075 |
1,000,000 |
CPU OpenMP |
64.8 ms (64.7 ms–64.9 ms) |
– |
72 ms |
3.55e8 |
1.56e10 (–) |
6.09e10 (–) |
– |
99.2 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (132 µs), then eagle.simulate (eagle picks the launch mode) (144 µs, 1.09× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (318 µs), then eagle.simulate (eagle picks the launch mode) (331 µs, 1.04× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (1.77 ms), then eagle.simulate (eagle picks the launch mode) (1.78 ms, 1× the time)
N = 1,000,000: eagle.simulate (eagle picks the launch mode) (28.1 ms), then eagle auto (eagle picks the launch mode) (28.1 ms, 1× the time)
Spread stop steps (log-uniform in [10, 1000]), up to 1000 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
1.31 ms (1.3 ms–1.31 ms) |
950 µs |
1.42 ms |
1.61e8 |
7.08e9 (7.5 %) |
4.99e9 (3.6 %) |
0 MiB (0×) |
2.12 MiB |
13 % |
– |
16/8/40 |
1075–1075 |
1,000 |
eagle graph + compaction |
6.31 ms (6.31 ms–6.31 ms) |
5.68 ms |
6.47 ms |
3.33e7 |
1.47e9 (1.5 %) |
2.51e9 (1.8 %) |
0 MiB (0×) |
2 MiB |
2.6 % |
– |
1000/2/18 |
1075–1075 |
1,000 |
eagle graph + compaction + reorder |
6.59 ms (6.58 ms–6.59 ms) |
5.87 ms |
6.77 ms |
3.19e7 |
1.40e9 (1.5 %) |
2.50e9 (1.8 %) |
0 MiB (0×) |
2.25 MiB |
2.5 % |
– |
1000/3/26 |
1075–1075 |
1,000 |
eagle graph, K fused steps per launch |
1.47 ms (1.47 ms–1.48 ms) |
1.11 ms |
1.59 ms |
1.43e8 |
6.28e9 (6.6 %) |
1.67e9 (1.2 %) |
0 MiB (0×) |
2 MiB |
– |
– |
63/1/14 |
1075–1075 |
1,000 |
eagle graph, K fused steps per launch + compaction |
1.54 ms (1.54 ms–1.54 ms) |
1.23 ms |
1.7 ms |
1.37e8 |
6.01e9 (6.3 %) |
9.25e8 (0.66 %) |
0 MiB (0×) |
2.12 MiB |
– |
– |
63/2/18 |
1075–1075 |
1,000 |
eagle auto (eagle picks the launch mode) |
811 µs (810 µs–812 µs) |
– |
960 µs |
2.59e8 |
1.14e10 (12 %) |
7.30e9 (5.2 %) |
0 MiB (0×) |
2.28 MiB |
– |
– |
– |
1075–1075 |
1,000 |
hawk + eagle persistent |
1.26 ms (1.26 ms–1.27 ms) |
– |
1.37 ms |
1.66e8 |
7.32e9 (7.7 %) |
4.68e9 (3.3 %) |
0 MiB (0×) |
2.12 MiB |
– |
0.22980196323115862 |
– |
1075–1075 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
822 µs (821 µs–823 µs) |
– |
970 µs |
2.56e8 |
1.13e10 (12 %) |
7.20e9 (5.1 %) |
0 MiB (0×) |
2.12 MiB |
– |
– |
– |
1075–1075 |
1,000 |
eager launch loop |
19.7 ms (19.4 ms–20 ms) |
3.64 ms |
19.8 ms |
1.07e7 |
4.70e8 (0.5 %) |
1.98e9 (1.4 %) |
0 MiB (0×) |
0.141 MiB |
0.84 % |
– |
1000/1000/5002 |
1075–1075 |
1,000 |
CuPy masked |
376 ms (375 ms–378 ms) |
79.8 ms |
377 ms |
5.58e5 |
2.46e7 (0.026 %) |
2.39e9 (1.7 %) |
0 MiB (0×) |
4.88 MiB |
– |
– |
44004/1000/47005 |
1075–1075 |
1,000 |
PyTorch masked |
243 ms (242 ms–243 ms) |
117 ms |
243 ms |
8.67e5 |
3.81e7 (0.04 %) |
3.71e9 (2.7 %) |
4 MiB (117×) |
0.125 MiB |
– |
– |
1000/1000/45005 |
1075–1075 |
1,000 |
PyTorch, CUDA graph of 16 steps |
112 ms (112 ms–112 ms) |
119 ms |
112 ms |
1.88e6 |
8.26e7 (0.087 %) |
8.15e9 (5.8 %) |
32 MiB (932×) |
1.25 MiB |
– |
– |
1000/252/194 |
1075–1075 |
1,000 |
JAX jit + vmap(while_loop) |
17.9 ms (17.8 ms–18 ms) |
6.11 ms |
19.2 ms |
1.18e7 |
5.18e8 (0.55 %) |
4.93e9 (3.5 %) |
8 MiB (233×) |
0 MiB |
– |
– |
1000/1003/4018 |
1075–1075 |
1,000 |
Warp per-thread kernel |
1.41 ms (1.39 ms–1.42 ms) |
1.34 ms |
1.73 ms |
1.49e8 |
6.55e9 (6.9 %) |
4.53e7 (0.032 %) |
30 MiB (874×) |
0.125 MiB |
– |
– |
1/0/4 |
1075–1075 |
1,000 |
CPU OpenMP |
1.25 ms (1.21 ms–1.32 ms) |
– |
1.28 ms |
1.68e8 |
7.38e9 (–) |
3.10e10 (–) |
– |
0.25 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
6.77 ms (6.75 ms–6.79 ms) |
6.39 ms |
6.96 ms |
3.23e8 |
1.42e10 (15 %) |
1.00e10 (7.1 %) |
0 MiB (0×) |
2.38 MiB |
25 % |
– |
16/8/41 |
1075–1075 |
10,000 |
eagle graph + compaction |
8.93 ms (8.92 ms–8.93 ms) |
8.26 ms |
9.15 ms |
2.44e8 |
1.08e10 (11 %) |
1.83e10 (13 %) |
0 MiB (0×) |
2.38 MiB |
19 % |
– |
1000/2/18 |
1075–1075 |
10,000 |
eagle graph + compaction + reorder |
9.55 ms (9.54 ms–9.57 ms) |
8.21 ms |
9.85 ms |
2.29e8 |
1.01e10 (11 %) |
1.77e10 (13 %) |
0 MiB (0×) |
2.75 MiB |
18 % |
– |
1000/4/56 |
1075–1075 |
10,000 |
eagle graph, K fused steps per launch |
6.95 ms (6.94 ms–6.95 ms) |
6.61 ms |
7.13 ms |
3.14e8 |
1.38e10 (15 %) |
3.56e9 (2.5 %) |
0 MiB (0×) |
2.5 MiB |
– |
– |
63/1/14 |
1075–1075 |
10,000 |
eagle graph, K fused steps per launch + compaction |
3.18 ms (3.18 ms–3.18 ms) |
2.89 ms |
3.4 ms |
6.87e8 |
3.02e10 (32 %) |
4.59e9 (3.3 %) |
0 MiB (0×) |
2.38 MiB |
– |
– |
63/2/18 |
1075–1075 |
10,000 |
eagle auto (eagle picks the launch mode) |
2.39 ms (2.38 ms–2.39 ms) |
– |
2.6 ms |
9.15e8 |
4.02e10 (42 %) |
2.58e10 (18 %) |
0 MiB (0×) |
2.5 MiB |
– |
– |
– |
1075–1075 |
10,000 |
hawk + eagle persistent |
2.75 ms (2.71 ms–2.77 ms) |
– |
2.92 ms |
7.95e8 |
3.50e10 (37 %) |
2.24e10 (16 %) |
0 MiB (0×) |
2.26 MiB |
– |
0.5806479200875753 |
– |
1075–1075 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
2.39 ms (2.38 ms–2.4 ms) |
– |
2.61 ms |
9.13e8 |
4.02e10 (42 %) |
2.57e10 (18 %) |
0 MiB (0×) |
2.38 MiB |
– |
– |
– |
1075–1075 |
10,000 |
eager launch loop |
29.4 ms (27.6 ms–30.6 ms) |
10.5 ms |
29.7 ms |
7.42e7 |
3.26e9 (3.4 %) |
1.33e10 (9.5 %) |
0 MiB (0×) |
0.375 MiB |
5.8 % |
– |
1000/1000/5002 |
1075–1075 |
10,000 |
CuPy masked |
384 ms (380 ms–386 ms) |
104 ms |
384 ms |
5.69e6 |
2.50e8 (0.26 %) |
2.35e10 (17 %) |
0 MiB (0×) |
5.39 MiB |
– |
– |
44004/1000/47005 |
1075–1075 |
10,000 |
PyTorch masked |
228 ms (227 ms–229 ms) |
132 ms |
229 ms |
9.56e6 |
4.21e8 (0.44 %) |
3.95e10 (28 %) |
4 MiB (11.7×) |
0.75 MiB |
– |
– |
1000/1000/45005 |
1075–1075 |
10,000 |
PyTorch, CUDA graph of 16 steps |
141 ms (141 ms–141 ms) |
147 ms |
141 ms |
1.55e7 |
6.81e8 (0.72 %) |
6.47e10 (46 %) |
32 MiB (93.2×) |
1.88 MiB |
– |
– |
1000/252/194 |
1075–1075 |
10,000 |
JAX jit + vmap(while_loop) |
28.4 ms (28.3 ms–28.5 ms) |
19.1 ms |
29.8 ms |
7.68e7 |
3.38e9 (3.6 %) |
3.10e10 (22 %) |
8 MiB (23.3×) |
0.625 MiB |
– |
– |
1000/1003/5019 |
1075–1075 |
10,000 |
Warp per-thread kernel |
6.68 ms (6.67 ms–6.71 ms) |
6.56 ms |
7.04 ms |
3.27e8 |
1.44e10 (15 %) |
9.58e7 (0.068 %) |
30 MiB (87.4×) |
0.625 MiB |
– |
– |
1/0/4 |
1075–1075 |
10,000 |
CPU OpenMP |
6.83 ms (6.43 ms–7.41 ms) |
– |
6.92 ms |
3.19e8 |
1.41e10 (–) |
5.72e10 (–) |
– |
1.12 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
61 ms (61 ms–61 ms) |
60.7 ms |
62 ms |
3.55e8 |
1.56e10 (16 %) |
1.27e10 (9 %) |
4 MiB (1.17×) |
7.62 MiB |
28 % |
– |
16/8/40 |
1075–1075 |
100,000 |
eagle graph + compaction |
41.4 ms (41.4 ms–41.4 ms) |
42.7 ms |
42.4 ms |
5.23e8 |
2.30e10 (24 %) |
3.92e10 (28 %) |
4 MiB (1.17×) |
7.62 MiB |
41 % |
– |
1000/2/18 |
1075–1075 |
100,000 |
eagle graph + compaction + reorder |
32.2 ms (32.2 ms–32.2 ms) |
34.1 ms |
33.3 ms |
6.73e8 |
2.96e10 (31 %) |
5.23e10 (37 %) |
8 MiB (2.33×) |
8 MiB |
53 % |
– |
1000/4/58 |
1075–1075 |
100,000 |
eagle graph, K fused steps per launch |
59.7 ms (59.7 ms–59.8 ms) |
59.4 ms |
60.7 ms |
3.63e8 |
1.60e10 (17 %) |
4.14e9 (3 %) |
4 MiB (1.17×) |
7.62 MiB |
– |
– |
63/1/14 |
1075–1075 |
100,000 |
eagle graph, K fused steps per launch + compaction |
18.7 ms (18.7 ms–18.7 ms) |
18.6 ms |
19.7 ms |
1.16e9 |
5.09e10 (54 %) |
7.75e9 (5.5 %) |
4 MiB (1.17×) |
7.62 MiB |
– |
– |
63/2/18 |
1075–1075 |
100,000 |
eagle auto (eagle picks the launch mode) |
15.8 ms (15.8 ms–15.8 ms) |
– |
16.9 ms |
1.37e9 |
6.04e10 (64 %) |
3.86e10 (28 %) |
4 MiB (1.17×) |
7.75 MiB |
– |
– |
– |
1075–1075 |
100,000 |
hawk + eagle persistent |
16.2 ms (16.2 ms–16.2 ms) |
– |
17.1 ms |
1.34e9 |
5.89e10 (62 %) |
3.77e10 (27 %) |
4 MiB (1.17×) |
7.89 MiB |
– |
0.8692150566422379 |
– |
1075–1075 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
15.7 ms (15.7 ms–15.7 ms) |
– |
16.7 ms |
1.38e9 |
6.06e10 (64 %) |
3.88e10 (28 %) |
4 MiB (1.17×) |
7.75 MiB |
– |
– |
– |
1075–1075 |
100,000 |
eager launch loop |
94.2 ms (93.8 ms–94.5 ms) |
79.3 ms |
95.2 ms |
2.30e8 |
1.01e10 (11 %) |
4.15e10 (30 %) |
4 MiB (1.17×) |
5.5 MiB |
18 % |
– |
1000/1000/5002 |
1075–1075 |
100,000 |
CuPy masked |
768 ms (767 ms–768 ms) |
761 ms |
769 ms |
2.82e7 |
1.24e9 (1.3 %) |
1.17e11 (84 %) |
22 MiB (6.41×) |
8.62 MiB |
– |
– |
44004/1000/47005 |
1075–1075 |
100,000 |
PyTorch masked |
676 ms (676 ms–676 ms) |
654 ms |
678 ms |
3.20e7 |
1.41e9 (1.5 %) |
1.33e11 (95 %) |
28 MiB (8.16×) |
6.88 MiB |
– |
– |
1000/1000/45005 |
1075–1075 |
100,000 |
PyTorch, CUDA graph of 16 steps |
651 ms (651 ms–652 ms) |
654 ms |
653 ms |
3.32e7 |
1.46e9 (1.5 %) |
1.40e11 (1e+02 %) |
54 MiB (15.7×) |
7.52 MiB |
– |
– |
1000/252/194 |
1075–1075 |
100,000 |
JAX jit + vmap(while_loop) |
167 ms (167 ms–167 ms) |
157 ms |
170 ms |
1.29e8 |
5.70e9 (6 %) |
5.26e10 (38 %) |
20 MiB (5.83×) |
7 MiB |
– |
– |
1000/1003/5019 |
1075–1075 |
100,000 |
Warp per-thread kernel |
63.8 ms (63.5 ms–64 ms) |
63.8 ms |
65.3 ms |
3.40e8 |
1.49e10 (16 %) |
1.00e8 (0.072 %) |
30 MiB (8.74×) |
6.88 MiB |
– |
– |
1/0/4 |
1075–1075 |
100,000 |
CPU OpenMP |
57.1 ms (56.4 ms–57.9 ms) |
– |
57.8 ms |
3.79e8 |
1.67e10 (–) |
6.84e10 (–) |
– |
9.75 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
586 ms (586 ms–586 ms) |
587 ms |
593 ms |
3.69e8 |
1.62e10 (17 %) |
1.82e10 (13 %) |
40 MiB (1.17×) |
60.1 MiB |
29 % |
– |
16/8/40 |
1075–1075 |
1,000,000 |
eagle graph + compaction |
374 ms (374 ms–374 ms) |
398 ms |
382 ms |
5.78e8 |
2.54e10 (27 %) |
4.34e10 (31 %) |
56 MiB (1.63×) |
60 MiB |
45 % |
– |
1000/2/18 |
1075–1075 |
1,000,000 |
eagle graph + compaction + reorder |
253 ms (253 ms–253 ms) |
284 ms |
261 ms |
8.53e8 |
3.75e10 (40 %) |
6.62e10 (47 %) |
84 MiB (2.45×) |
62.9 MiB |
67 % |
– |
1000/4/58 |
1075–1075 |
1,000,000 |
eagle graph, K fused steps per launch |
576 ms (575 ms–576 ms) |
576 ms |
583 ms |
3.76e8 |
1.65e10 (17 %) |
4.29e9 (3.1 %) |
40 MiB (1.17×) |
60.1 MiB |
– |
– |
63/1/14 |
1075–1075 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
171 ms (171 ms–171 ms) |
173 ms |
178 ms |
1.27e9 |
5.57e10 (59 %) |
8.49e9 (6.1 %) |
56 MiB (1.63×) |
60 MiB |
– |
– |
63/2/18 |
1075–1075 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
150 ms (150 ms–150 ms) |
– |
157 ms |
1.44e9 |
6.35e10 (67 %) |
4.07e10 (29 %) |
40 MiB (1.17×) |
60.3 MiB |
– |
– |
– |
1075–1075 |
1,000,000 |
hawk + eagle persistent |
150 ms (150 ms–150 ms) |
– |
157 ms |
1.44e9 |
6.35e10 (67 %) |
4.06e10 (29 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
0.9266072721524322 |
– |
1075–1075 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
150 ms (150 ms–150 ms) |
– |
157 ms |
1.44e9 |
6.35e10 (67 %) |
4.07e10 (29 %) |
40 MiB (1.17×) |
60.4 MiB |
– |
– |
– |
1075–1075 |
1,000,000 |
eager launch loop |
723 ms (723 ms–724 ms) |
723 ms |
731 ms |
2.99e8 |
1.32e10 (14 %) |
5.40e10 (39 %) |
40 MiB (1.17×) |
58.2 MiB |
23 % |
– |
1000/1000/5002 |
1075–1075 |
1,000,000 |
CuPy masked |
7.38 s (7.38 s–7.38 s) |
7.35 s |
7.39 s |
2.93e7 |
1.29e9 (1.4 %) |
1.22e11 (87 %) |
192 MiB (5.59×) |
68.8 MiB |
– |
– |
44004/1000/47005 |
1075–1075 |
1,000,000 |
PyTorch masked |
7.4 s (7.4 s–7.4 s) |
7.37 s |
7.41 s |
2.92e7 |
1.29e9 (1.4 %) |
1.22e11 (87 %) |
264 MiB (7.69×) |
68.8 MiB |
– |
– |
1000/1000/46005 |
1075–1075 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
7.44 s (7.44 s–7.44 s) |
7.43 s |
7.45 s |
2.91e7 |
1.28e9 (1.3 %) |
1.23e11 (87 %) |
292 MiB (8.51×) |
68.7 MiB |
– |
– |
1000/252/194 |
1075–1075 |
1,000,000 |
JAX jit + vmap(while_loop) |
1.37 s (1.37 s–1.37 s) |
1.36 s |
1.38 s |
1.58e8 |
6.96e9 (7.3 %) |
6.44e10 (46 %) |
136 MiB (3.96×) |
101 MiB |
– |
– |
1000/1003/5019 |
1075–1075 |
1,000,000 |
Warp per-thread kernel |
615 ms (615 ms–616 ms) |
616 ms |
626 ms |
3.51e8 |
1.55e10 (16 %) |
1.04e8 (0.074 %) |
62 MiB (1.81×) |
68.7 MiB |
– |
– |
1/0/4 |
1075–1075 |
1,000,000 |
CPU OpenMP |
578 ms (573 ms–581 ms) |
– |
585 ms |
3.74e8 |
1.65e10 (–) |
6.76e10 (–) |
– |
99.2 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (811 µs), then eagle.simulate (eagle picks the launch mode) (822 µs, 1.01× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (2.39 ms), then eagle.simulate (eagle picks the launch mode) (2.39 ms, 1× the time)
N = 100,000: eagle.simulate (eagle picks the launch mode) (15.7 ms), then eagle auto (eagle picks the launch mode) (15.8 ms, 1× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (150 ms), then eagle.simulate (eagle picks the launch mode) (150 ms, 1× the time)
Uniform stop step: every sample runs 1000 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
1.27 ms (1.26 ms–1.28 ms) |
917 µs |
1.37 ms |
7.90e8 |
3.48e10 (37 %) |
2.26e10 (16 %) |
0 MiB (0×) |
2 MiB |
62 % |
– |
16/8/41 |
1075–1075 |
1,000 |
eagle graph + compaction |
4.94 ms (4.93 ms–4.94 ms) |
4.65 ms |
5.09 ms |
2.03e8 |
8.91e9 (9.4 %) |
1.32e10 (9.4 %) |
0 MiB (0×) |
2 MiB |
16 % |
– |
1000/2/18 |
1075–1075 |
1,000 |
eagle graph + compaction + reorder |
5.03 ms (5.02 ms–5.05 ms) |
4.66 ms |
5.21 ms |
1.99e8 |
8.74e9 (9.2 %) |
1.30e10 (9.3 %) |
0 MiB (0×) |
2.13 MiB |
16 % |
– |
1000/3/26 |
1075–1075 |
1,000 |
eagle graph, K fused steps per launch |
1.48 ms (1.48 ms–1.48 ms) |
1.12 ms |
1.6 ms |
6.76e8 |
2.97e10 (31 %) |
2.60e9 (1.9 %) |
0 MiB (0×) |
2 MiB |
– |
– |
63/1/14 |
1075–1075 |
1,000 |
eagle graph, K fused steps per launch + compaction |
1.33 ms (1.32 ms–1.33 ms) |
1.06 ms |
1.48 ms |
7.54e8 |
3.32e10 (35 %) |
3.14e9 (2.2 %) |
0 MiB (0×) |
2.12 MiB |
– |
– |
63/2/18 |
1075–1075 |
1,000 |
eagle auto (eagle picks the launch mode) |
817 µs (813 µs–820 µs) |
– |
961 µs |
1.22e9 |
5.39e10 (57 %) |
3.43e10 (24 %) |
0 MiB (0×) |
2.25 MiB |
– |
– |
– |
1075–1075 |
1,000 |
hawk + eagle persistent |
1.4 ms (1.39 ms–1.41 ms) |
– |
1.54 ms |
7.13e8 |
3.14e10 (33 %) |
2.00e10 (14 %) |
0 MiB (0×) |
2.14 MiB |
– |
0.95367431640625 |
– |
1075–1075 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
824 µs (823 µs–825 µs) |
– |
976 µs |
1.21e9 |
5.34e10 (56 %) |
3.40e10 (24 %) |
0 MiB (0×) |
2 MiB |
– |
– |
– |
1075–1075 |
1,000 |
eager launch loop |
19.5 ms (19.4 ms–19.5 ms) |
3.68 ms |
19.6 ms |
5.14e7 |
2.26e9 (2.4 %) |
3.13e9 (2.2 %) |
0 MiB (0×) |
0 MiB |
4 % |
– |
1000/1000/5002 |
1075–1075 |
1,000 |
CuPy masked |
338 ms (337 ms–339 ms) |
79.8 ms |
338 ms |
2.96e6 |
1.30e8 (0.14 %) |
2.67e9 (1.9 %) |
0 MiB (0×) |
4.75 MiB |
– |
– |
44004/1000/47005 |
1075–1075 |
1,000 |
PyTorch masked |
253 ms (246 ms–260 ms) |
117 ms |
254 ms |
3.96e6 |
1.74e8 (0.18 %) |
3.57e9 (2.5 %) |
4 MiB (117×) |
0.125 MiB |
– |
– |
1000/1000/45005 |
1075–1075 |
1,000 |
PyTorch, CUDA graph of 16 steps |
112 ms (112 ms–112 ms) |
119 ms |
112 ms |
8.93e6 |
3.93e8 (0.41 %) |
8.15e9 (5.8 %) |
32 MiB (932×) |
1.38 MiB |
– |
– |
1000/252/194 |
1075–1075 |
1,000 |
JAX jit + vmap(while_loop) |
19.3 ms (19.2 ms–19.3 ms) |
6.11 ms |
20.9 ms |
5.18e7 |
2.28e9 (2.4 %) |
4.56e9 (3.3 %) |
8 MiB (233×) |
0 MiB |
– |
– |
1000/1003/4018 |
1075–1075 |
1,000 |
Warp per-thread kernel |
1.49 ms (1.49 ms–1.49 ms) |
1.43 ms |
1.72 ms |
6.73e8 |
2.96e10 (31 %) |
4.30e7 (0.031 %) |
30 MiB (874×) |
0.125 MiB |
– |
– |
1/0/4 |
1075–1075 |
1,000 |
CPU OpenMP |
1.73 ms (1.64 ms–1.82 ms) |
– |
1.75 ms |
5.77e8 |
2.54e10 (–) |
3.52e10 (–) |
– |
0.141 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
7.12 ms (7.12 ms–7.13 ms) |
6.77 ms |
7.3 ms |
1.40e9 |
6.18e10 (65 %) |
4.02e10 (29 %) |
0 MiB (0×) |
2.38 MiB |
1.1e+02 % |
– |
16/8/40 |
1075–1075 |
10,000 |
eagle graph + compaction |
12.3 ms (12.3 ms–12.3 ms) |
11.9 ms |
12.5 ms |
8.15e8 |
3.59e10 (38 %) |
5.32e10 (38 %) |
0 MiB (0×) |
2.38 MiB |
64 % |
– |
1000/2/18 |
1075–1075 |
10,000 |
eagle graph + compaction + reorder |
12.8 ms (12.8 ms–12.8 ms) |
12 ms |
13 ms |
7.81e8 |
3.44e10 (36 %) |
5.10e10 (36 %) |
0 MiB (0×) |
3 MiB |
61 % |
– |
1000/3/26 |
1075–1075 |
10,000 |
eagle graph, K fused steps per launch |
7.42 ms (7.41 ms–7.43 ms) |
7.05 ms |
7.63 ms |
1.35e9 |
5.93e10 (63 %) |
5.18e9 (3.7 %) |
0 MiB (0×) |
2.38 MiB |
– |
– |
63/1/14 |
1075–1075 |
10,000 |
eagle graph, K fused steps per launch + compaction |
7.24 ms (7.24 ms–7.24 ms) |
6.99 ms |
7.46 ms |
1.38e9 |
6.08e10 (64 %) |
5.75e9 (4.1 %) |
0 MiB (0×) |
2.38 MiB |
– |
– |
63/2/18 |
1075–1075 |
10,000 |
eagle auto (eagle picks the launch mode) |
6.62 ms (6.62 ms–6.62 ms) |
– |
6.84 ms |
1.51e9 |
6.64e10 (70 %) |
4.23e10 (30 %) |
0 MiB (0×) |
2.5 MiB |
– |
– |
– |
1075–1075 |
10,000 |
hawk + eagle persistent |
6.66 ms (6.65 ms–6.67 ms) |
– |
6.85 ms |
1.50e9 |
6.60e10 (70 %) |
4.21e10 (30 %) |
0 MiB (0×) |
2.5 MiB |
– |
0.9750024960063898 |
– |
1075–1075 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
6.63 ms (6.63 ms–6.63 ms) |
– |
6.85 ms |
1.51e9 |
6.63e10 (70 %) |
4.23e10 (30 %) |
0 MiB (0×) |
2.63 MiB |
– |
– |
– |
1075–1075 |
10,000 |
eager launch loop |
26 ms (25.9 ms–26.1 ms) |
11.2 ms |
26.2 ms |
3.85e8 |
1.69e10 (18 %) |
2.35e10 (17 %) |
0 MiB (0×) |
0.25 MiB |
30 % |
– |
1000/1000/5002 |
1075–1075 |
10,000 |
CuPy masked |
371 ms (357 ms–380 ms) |
104 ms |
371 ms |
2.70e7 |
1.19e9 (1.3 %) |
2.43e10 (17 %) |
0 MiB (0×) |
5 MiB |
– |
– |
44004/1000/47005 |
1075–1075 |
10,000 |
PyTorch masked |
225 ms (224 ms–226 ms) |
132 ms |
226 ms |
4.45e7 |
1.96e9 (2.1 %) |
4.01e10 (29 %) |
4 MiB (11.7×) |
0.75 MiB |
– |
– |
1000/1000/45005 |
1075–1075 |
10,000 |
PyTorch, CUDA graph of 16 steps |
140 ms (140 ms–140 ms) |
147 ms |
141 ms |
7.12e7 |
3.13e9 (3.3 %) |
6.50e10 (46 %) |
32 MiB (93.2×) |
1.75 MiB |
– |
– |
1000/252/194 |
1075–1075 |
10,000 |
JAX jit + vmap(while_loop) |
29.1 ms (29.1 ms–29.1 ms) |
19.1 ms |
30.6 ms |
3.44e8 |
1.51e10 (16 %) |
3.02e10 (22 %) |
8 MiB (23.3×) |
0.625 MiB |
– |
– |
1000/1003/5019 |
1075–1075 |
10,000 |
Warp per-thread kernel |
7.22 ms (7.22 ms–7.23 ms) |
7.16 ms |
7.61 ms |
1.38e9 |
6.09e10 (64 %) |
8.86e7 (0.063 %) |
30 MiB (87.4×) |
0.766 MiB |
– |
– |
1/0/4 |
1075–1075 |
10,000 |
CPU OpenMP |
11 ms (11 ms–11 ms) |
– |
11.1 ms |
9.07e8 |
3.99e10 (–) |
5.53e10 (–) |
– |
1.14 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
65.9 ms (65.8 ms–65.9 ms) |
65.3 ms |
66.9 ms |
1.52e9 |
6.68e10 (70 %) |
4.35e10 (31 %) |
4 MiB (1.17×) |
7.62 MiB |
1.2e+02 % |
– |
16/8/40 |
1075–1075 |
100,000 |
eagle graph + compaction |
88.9 ms (88.9 ms–89 ms) |
93.4 ms |
90 ms |
1.12e9 |
4.95e10 (52 %) |
7.35e10 (52 %) |
4 MiB (1.17×) |
7.75 MiB |
88 % |
– |
1000/2/18 |
1075–1075 |
100,000 |
eagle graph + compaction + reorder |
89.7 ms (89.6 ms–89.7 ms) |
92.9 ms |
90.8 ms |
1.12e9 |
4.91e10 (52 %) |
7.29e10 (52 %) |
8 MiB (2.33×) |
8.12 MiB |
88 % |
– |
1000/3/26 |
1075–1075 |
100,000 |
eagle graph, K fused steps per launch |
67 ms (66.9 ms–67 ms) |
66.8 ms |
68 ms |
1.49e9 |
6.57e10 (69 %) |
5.74e9 (4.1 %) |
4 MiB (1.17×) |
7.62 MiB |
– |
– |
63/1/14 |
1075–1075 |
100,000 |
eagle graph, K fused steps per launch + compaction |
66.8 ms (66.8 ms–66.8 ms) |
66.5 ms |
67.8 ms |
1.50e9 |
6.59e10 (70 %) |
6.23e9 (4.4 %) |
4 MiB (1.17×) |
7.75 MiB |
– |
– |
63/2/18 |
1075–1075 |
100,000 |
eagle auto (eagle picks the launch mode) |
65 ms (65 ms–65 ms) |
– |
66.2 ms |
1.54e9 |
6.77e10 (71 %) |
4.31e10 (31 %) |
4 MiB (1.17×) |
7.75 MiB |
– |
– |
– |
1075–1075 |
100,000 |
hawk + eagle persistent |
65 ms (65 ms–65 ms) |
– |
66 ms |
1.54e9 |
6.77e10 (71 %) |
4.31e10 (31 %) |
4 MiB (1.17×) |
7.75 MiB |
– |
0.9765625 |
– |
1075–1075 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
65 ms (64.9 ms–65 ms) |
– |
66.2 ms |
1.54e9 |
6.77e10 (71 %) |
4.32e10 (31 %) |
4 MiB (1.17×) |
7.89 MiB |
– |
– |
– |
1075–1075 |
100,000 |
eager launch loop |
102 ms (102 ms–103 ms) |
90.4 ms |
103 ms |
9.77e8 |
4.30e10 (45 %) |
5.96e10 (43 %) |
4 MiB (1.17×) |
5.5 MiB |
77 % |
– |
1000/1000/5002 |
1075–1075 |
100,000 |
CuPy masked |
768 ms (768 ms–768 ms) |
761 ms |
769 ms |
1.30e8 |
5.73e9 (6 %) |
1.17e11 (84 %) |
22 MiB (6.41×) |
8.59 MiB |
– |
– |
44004/1000/47005 |
1075–1075 |
100,000 |
PyTorch masked |
676 ms (676 ms–676 ms) |
654 ms |
678 ms |
1.48e8 |
6.51e9 (6.9 %) |
1.33e11 (95 %) |
28 MiB (8.16×) |
6.25 MiB |
– |
– |
1000/1000/45005 |
1075–1075 |
100,000 |
PyTorch, CUDA graph of 16 steps |
652 ms (652 ms–652 ms) |
655 ms |
653 ms |
1.53e8 |
6.75e9 (7.1 %) |
1.40e11 (1e+02 %) |
54 MiB (15.7×) |
7.5 MiB |
– |
– |
1000/252/194 |
1075–1075 |
100,000 |
JAX jit + vmap(while_loop) |
167 ms (167 ms–167 ms) |
157 ms |
170 ms |
5.98e8 |
2.63e10 (28 %) |
5.27e10 (38 %) |
20 MiB (5.83×) |
6.25 MiB |
– |
– |
1000/1003/5019 |
1075–1075 |
100,000 |
Warp per-thread kernel |
71.5 ms (71.5 ms–71.5 ms) |
71.4 ms |
73 ms |
1.40e9 |
6.16e10 (65 %) |
8.95e7 (0.064 %) |
30 MiB (8.74×) |
7.02 MiB |
– |
– |
1/0/4 |
1075–1075 |
100,000 |
CPU OpenMP |
108 ms (106 ms–112 ms) |
– |
109 ms |
9.23e8 |
4.06e10 (–) |
5.63e10 (–) |
– |
9.88 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
645 ms (645 ms–645 ms) |
645 ms |
652 ms |
1.55e9 |
6.82e10 (72 %) |
4.44e10 (32 %) |
40 MiB (1.17×) |
60.1 MiB |
1.2e+02 % |
– |
16/8/40 |
1075–1075 |
1,000,000 |
eagle graph + compaction |
803 ms (803 ms–803 ms) |
821 ms |
810 ms |
1.25e9 |
5.48e10 (58 %) |
8.14e10 (58 %) |
56 MiB (1.63×) |
60.2 MiB |
98 % |
– |
1000/2/18 |
1075–1075 |
1,000,000 |
eagle graph + compaction + reorder |
803 ms (803 ms–803 ms) |
821 ms |
812 ms |
1.24e9 |
5.48e10 (58 %) |
8.13e10 (58 %) |
84 MiB (2.45×) |
60.5 MiB |
98 % |
– |
1000/3/26 |
1075–1075 |
1,000,000 |
eagle graph, K fused steps per launch |
652 ms (652 ms–652 ms) |
653 ms |
659 ms |
1.53e9 |
6.75e10 (71 %) |
5.90e9 (4.2 %) |
40 MiB (1.17×) |
60.1 MiB |
– |
– |
63/1/14 |
1075–1075 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
652 ms (652 ms–652 ms) |
653 ms |
659 ms |
1.53e9 |
6.75e10 (71 %) |
6.38e9 (4.6 %) |
56 MiB (1.63×) |
60.1 MiB |
– |
– |
63/2/18 |
1075–1075 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
640 ms (640 ms–640 ms) |
– |
648 ms |
1.56e9 |
6.87e10 (73 %) |
4.38e10 (31 %) |
40 MiB (1.17×) |
60.3 MiB |
– |
– |
– |
1075–1075 |
1,000,000 |
hawk + eagle persistent |
641 ms (640 ms–641 ms) |
– |
648 ms |
1.56e9 |
6.87e10 (72 %) |
4.38e10 (31 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
0.9765625 |
– |
1075–1075 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
640 ms (640 ms–640 ms) |
– |
648 ms |
1.56e9 |
6.87e10 (73 %) |
4.38e10 (31 %) |
40 MiB (1.17×) |
60.3 MiB |
– |
– |
– |
1075–1075 |
1,000,000 |
eager launch loop |
821 ms (818 ms–825 ms) |
817 ms |
828 ms |
1.22e9 |
5.36e10 (57 %) |
7.43e10 (53 %) |
40 MiB (1.17×) |
58.2 MiB |
96 % |
– |
1000/1000/5002 |
1075–1075 |
1,000,000 |
CuPy masked |
7.41 s (7.4 s–7.43 s) |
7.35 s |
7.42 s |
1.35e8 |
5.94e9 (6.3 %) |
1.22e11 (87 %) |
192 MiB (5.59×) |
65.6 MiB |
– |
– |
44004/1000/47005 |
1075–1075 |
1,000,000 |
PyTorch masked |
7.42 s (7.42 s–7.43 s) |
7.37 s |
7.43 s |
1.35e8 |
5.93e9 (6.3 %) |
1.21e11 (87 %) |
264 MiB (7.69×) |
62.1 MiB |
– |
– |
1000/1000/46005 |
1075–1075 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
7.44 s (7.44 s–7.44 s) |
7.43 s |
7.45 s |
1.34e8 |
5.91e9 (6.2 %) |
1.23e11 (87 %) |
292 MiB (8.51×) |
63.1 MiB |
– |
– |
1000/252/194 |
1075–1075 |
1,000,000 |
JAX jit + vmap(while_loop) |
1.37 s (1.37 s–1.37 s) |
1.36 s |
1.38 s |
7.31e8 |
3.21e10 (34 %) |
6.43e10 (46 %) |
136 MiB (3.96×) |
99.1 MiB |
– |
– |
1000/1003/5019 |
1075–1075 |
1,000,000 |
Warp per-thread kernel |
699 ms (699 ms–699 ms) |
699 ms |
712 ms |
1.43e9 |
6.30e10 (66 %) |
9.16e7 (0.065 %) |
62 MiB (1.81×) |
68.6 MiB |
– |
– |
1/0/4 |
1075–1075 |
1,000,000 |
CPU OpenMP |
1.48 s (1.41 s–1.56 s) |
– |
1.49 s |
6.75e8 |
2.97e10 (–) |
4.12e10 (–) |
– |
97.4 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (817 µs), then eagle.simulate (eagle picks the launch mode) (824 µs, 1.01× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (6.62 ms), then eagle.simulate (eagle picks the launch mode) (6.63 ms, 1× the time)
N = 100,000: eagle.simulate (eagle picks the launch mode) (65 ms), then hawk + eagle persistent (65 ms, 1× the time)
N = 1,000,000: eagle.simulate (eagle picks the launch mode) (640 ms), then eagle auto (eagle picks the launch mode) (640 ms, 1× the time)
Compaction cadence sweep (isolates the active-set map)
the active-set map alone. For a given K, matched_cadence (the plain per-sample step, no map) and compact (eagle_graph_compact’s active-set step) check the loop’s stop guard at the SAME cadence, once every K steps; the only structural difference between the two rows at a given K is whether the step kernel reads the active-set map. The main table’s eagle_graph row checks its guard every step, so its difference from eagle_graph_compact there mixes the cadence change with the map – this sweep does not.
Config: Spread stop steps (log-uniform in [10, 1000]), up to 1000 steps.
N |
K (steps between guard checks) |
matched cadence, no map (range) |
eagle graph + compaction (range) |
ratio (matched/compact, |
|---|---|---|---|---|
100,000 |
8 |
77 ms (77 ms–77 ms) |
45.4 ms (45.3 ms–45.4 ms) |
1.7× |
100,000 |
16 |
76.5 ms (76.5 ms–76.5 ms) |
41.6 ms (41.6 ms–41.6 ms) |
1.84× |
100,000 |
32 |
76.6 ms (76.6 ms–76.6 ms) |
40.2 ms (40.2 ms–40.3 ms) |
1.9× |
1,000,000 |
8 |
706 ms (706 ms–706 ms) |
395 ms (395 ms–395 ms) |
1.79× |
1,000,000 |
16 |
706 ms (706 ms–706 ms) |
374 ms (374 ms–374 ms) |
1.89× |
1,000,000 |
32 |
706 ms (706 ms–706 ms) |
365 ms (365 ms–365 ms) |
1.93× |
FP32: eagle vs Warp (float32, S = 1000)
Every arm runs the same oscillator in float32: eagle.simulate with scalar_type=”float32”, NVIDIA Warp’s per-thread kernel and JAX’s vmapped loop, all over the same float32 inputs. Each cell runs 2 untimed runs per arm first (eagle’s automatic arm picks its entry over them: the size rule, then a measured run of the other entry, keeping the faster; the entry it settled on is in the last column), then each arm in its own block (no interleaving): an untimed ramp of back-to-back runs for 0.5 s, then 5 timed runs. Before any timing counts, eagle and JAX reach step counts identical to Warp’s and states within absolute 0.0001 of Warp’s (the largest differences are in the JSON). Each time is the mean of the middle 3 of the 5 runs (highest and lowest dropped), with the range of those 3; eagle / Warp is the ratio of these means: below 1 eagle is faster. Before each arm’s block the GPU is cooled down: the block starts once the temperature is within 3 °C of the idle 58 °C and the GPU is idle, or after 20 s. Right after each timed run the SM clock is read (its range is shown per arm); cycles are the wall time times that clock, and eagle / Warp (cycles) is their ratio.
Every arm runs each sample to its own stop and no further than max_steps (S): eagle and Warp enforce the cap per sample, JAX in its while-loop condition, PyTorch and CuPy by the outer step count (a batch-level cap); an arm that runs whole blocks of steps per launch or replay stops at the first block boundary at or after it.
What each arm needed to get there: Warp, a hand-written step counter and cap inside the kernel; JAX, a cap in the while_loop condition; eagle, the kernel source unchanged, with its loop shape, unrolling and device entry chosen by hawk + eagle (per device, per batch).
Cap fixture (checked before any timing counts): N = 4,096, 8 samples with stop step 2S = 256, S = 128; every arm (3) reported k = min(stop step, S) per sample and the same finished count (4,088): passed.
distribution |
N |
eagle.simulate (range) |
Warp (range) |
JAX (range) |
eagle / Warp |
eagle / Warp (cycles) |
eagle entry |
|---|---|---|---|---|---|---|---|
uniform |
10,000 |
292 µs (290 µs–294 µs), 1075–1075 MHz |
332 µs (331 µs–333 µs), 1075–1075 MHz |
21.3 ms (21.1 ms–21.6 ms), 1075–1075 MHz |
0.88× |
0.88× |
fused_one |
uniform |
100,000 |
2.38 ms (2.37 ms–2.38 ms), 1075–1075 MHz |
2.67 ms (2.66 ms–2.69 ms), 1075–1075 MHz |
50 ms (49.3 ms–50.4 ms), 1075–1075 MHz |
0.892× |
0.892× |
fused_one |
uniform |
1,000,000 |
27.4 ms (27.4 ms–27.4 ms), 1075–1075 MHz |
25.6 ms (25.5 ms–25.6 ms), 1075–1075 MHz |
372 ms (372 ms–372 ms), 1075–1075 MHz |
1.07× |
1.07× |
persist |
spread |
10,000 |
284 µs (280 µs–287 µs), 1075–1075 MHz |
317 µs (316 µs–317 µs), 1075–1075 MHz |
20.3 ms (19.9 ms–21 ms), 1075–1075 MHz |
0.898× |
0.898× |
fused_one |
spread |
100,000 |
965 µs (961 µs–972 µs), 1075–1075 MHz |
2.42 ms (2.41 ms–2.43 ms), 1075–1075 MHz |
50.8 ms (50.5 ms–51.1 ms), 1075–1075 MHz |
0.399× |
0.399× |
persist |
spread |
1,000,000 |
7.39 ms (7.39 ms–7.4 ms), 1075–1075 MHz |
22.7 ms (22.6 ms–22.7 ms), 1075–1075 MHz |
377 ms (375 ms–379 ms), 1075–1075 MHz |
0.326× |
0.326× |
persist |
Fixed overhead per run (N = 64, S = 100)
item 3: wall at a cell small enough that per-run overhead dominates (N=64, S=100), each arm’s own kernel-only vs wall where the main nsys pass covers this cell’s N (it does not by default, so kernel_only_s is usually null here)
arm |
wall (range) |
kernel-only |
|---|---|---|
eagle graph (device loop) |
415 µs (411 µs–419 µs) |
– |
eagle graph + compaction |
677 µs (677 µs–678 µs) |
– |
eagle graph + compaction + reorder |
782 µs (775 µs–787 µs) |
– |
eagle graph, K fused steps per launch |
279 µs (277 µs–282 µs) |
– |
eagle graph, K fused steps per launch + compaction |
300 µs (299 µs–302 µs) |
– |
eagle auto (eagle picks the launch mode) |
126 µs (115 µs–133 µs) |
– |
hawk + eagle persistent |
149 µs (147 µs–150 µs) |
– |
eagle.simulate (eagle picks the launch mode) |
125 µs (122 µs–127 µs) |
– |
eager launch loop |
2.01 ms (2.01 ms–2.01 ms) |
– |
CuPy masked |
37.7 ms (37.2 ms–38.4 ms) |
– |
PyTorch masked |
22.6 ms (22.5 ms–22.7 ms) |
– |
PyTorch, CUDA graph of 16 steps |
9.81 ms (9.8 ms–9.81 ms) |
– |
JAX jit + vmap(while_loop) |
2.05 ms (1.98 ms–2.12 ms) |
– |
Warp per-thread kernel |
140 µs (129 µs–147 µs) |
– |
CPU OpenMP |
231 µs (231 µs–231 µs) |
– |
Where each tool fits
Each tool below is written the way its users write it, and each brings something of its own. Times are walls (mean of the middle 3 of 5 runs) at the largest N; ratios are against the eagle graph arm (below 1 = less time). The fastest arm per N is listed under each table above.
eagle graph (device loop). The step written once as a hawk kernel; the whole loop runs on the device as one CUDA graph with a device-side stop guard, so a run is one launch with no host round trip per step. On this card: N = 1,000,000, spread S=100: 59.3 ms; spread S=1000: 586 ms; uniform S=1000: 645 ms. Fastest in 0 of 12 cells.
eagle graph + compaction. Adds an active-set map: the launch covers only the samples still running, rebuilt every 16 steps; it is made for batches that thin out (the spread configurations). On this card: N = 1,000,000, spread S=100: 42 ms (0.708× the eagle graph time); spread S=1000: 374 ms (0.639× the eagle graph time); uniform S=1000: 803 ms (1.24× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph + compaction + reorder. Adds a physical reorder when the live samples spread thin over their warps, so the mapped reads stay contiguous; the final restore to sample order is inside its wall. On this card: N = 1,000,000, spread S=100: 40.1 ms (0.677× the eagle graph time); spread S=1000: 253 ms (0.433× the eagle graph time); uniform S=1000: 803 ms (1.25× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph, K fused steps per launch. The same step with one decorator argument, @hawk.kernel(steps=16): each launch runs that many steps per sample with the state kept in registers, as Warp’s per-thread loop does, and the launches, guard checks and plane reads shrink by the same factor; its results are bit-identical to the eagle graph arm’s in every cell. K is chosen by hand here; the eagle_graph_auto arm picks it at run time. On this card: N = 1,000,000, spread S=100: 58.2 ms (0.982× the eagle graph time); spread S=1000: 576 ms (0.982× the eagle graph time); uniform S=1000: 652 ms (1.01× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph, K fused steps per launch + compaction. The K-steps kernel over the active-set map: the register-resident steps of the arm above, and the launches cover only the samples still running, so it fits batches that thin out. On this card: N = 1,000,000, spread S=100: 45 ms (0.759× the eagle graph time); spread S=1000: 171 ms (0.291× the eagle graph time); uniform S=1000: 652 ms (1.01× the eagle graph time). Fastest in 0 of 12 cells.
eagle auto (eagle picks the launch mode). The same step with @hawk.kernel(steps=”auto”) on the active-set kind, and nothing to tune: it picks the steps per launch itself after every launch, on the device, so a batch whose samples all run to the end gets Warp-like fusion of up to 64 steps per launch, and a batch that thins out gets shorter launches with the active set compacted after each one; no sample takes more steps than the cap. On this card: N = 1,000,000, spread S=100: 28.1 ms (0.475× the eagle graph time); spread S=1000: 150 ms (0.256× the eagle graph time); uniform S=1000: 640 ms (0.993× the eagle graph time). Fastest in 8 of 12 cells.
hawk + eagle persistent. The plain (not active-set) automatic kernel, forced through hawk’s persist entry: one launch, no graph, no map, no policy kernel, no cap at 64 – a grid of SMs blocks of 256 lanes each steal the next unfinished sample off a counter when their own finishes, so idle warps refill without ever rebuilding an index map; made for batches too large for one resident wave. On this card: N = 1,000,000, spread S=100: 28.2 ms (0.475× the eagle graph time); spread S=1000: 150 ms (0.256× the eagle graph time); uniform S=1000: 641 ms (0.993× the eagle graph time). Fastest in 0 of 12 cells.
eagle.simulate (eagle picks the launch mode). The same automatic kernel through eagle.simulate: the model, its state and its parameters by name, with no plan or plane binding to write; eagle deploys the kernel and builds the same device loop. On this card: N = 1,000,000, spread S=100: 28.1 ms (0.474× the eagle graph time); spread S=1000: 150 ms (0.256× the eagle graph time); uniform S=1000: 640 ms (0.993× the eagle graph time). Fastest in 4 of 12 cells.
eager launch loop. The same kernels launched from Python one step at a time; the host sees the state after every step. On this card: N = 1,000,000, spread S=100: 73 ms (1.23× the eagle graph time); spread S=1000: 723 ms (1.23× the eagle graph time); uniform S=1000: 821 ms (1.27× the eagle graph time). Fastest in 0 of 12 cells.
CuPy masked. NumPy-like array code on GPUs with no kernel to write; every sample is computed every step, so it fits dense batches where all samples run to the end (the uniform configuration). On this card: N = 1,000,000, spread S=100: 739 ms (12.5× the eagle graph time); spread S=1000: 7.38 s (12.6× the eagle graph time); uniform S=1000: 7.41 s (11.5× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch masked. The same array code in PyTorch tensors; it fits where the computation sits next to a PyTorch model or needs autograd. On this card: N = 1,000,000, spread S=100: 741 ms (12.5× the eagle graph time); spread S=1000: 7.4 s (12.6× the eagle graph time); uniform S=1000: 7.42 s (11.5× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch, CUDA graph of 16 steps. The same PyTorch code with its launch overhead removed by hand: 16 steps captured as one CUDA graph and replayed, one host check per replay, the masking unchanged. On this card: N = 1,000,000, spread S=100: 827 ms (14× the eagle graph time); spread S=1000: 7.44 s (12.7× the eagle graph time); uniform S=1000: 7.44 s (11.5× the eagle graph time). Fastest in 0 of 12 cells.
JAX jit + vmap(while_loop). The whole loop compiled once, a per-sample while loop batched with vmap; the same code runs on CPUs and GPUs and differentiates with jax.grad. On this card: N = 1,000,000, spread S=100: 138 ms (2.33× the eagle graph time); spread S=1000: 1.37 s (2.34× the eagle graph time); uniform S=1000: 1.37 s (2.12× the eagle graph time). Fastest in 0 of 12 cells.
Warp per-thread kernel. Explicit per-thread kernels written in Python, maintained by NVIDIA and differentiable through wp.Tape; each thread runs its own sample and stops at its own step, so finished samples cost nothing. On this card: N = 1,000,000, spread S=100: 62.1 ms (1.05× the eagle graph time); spread S=1000: 615 ms (1.05× the eagle graph time); uniform S=1000: 699 ms (1.08× the eagle graph time). Fastest in 0 of 12 cells.
CPU OpenMP. The same hawk kernel on the host through eagle’s OpenMP team, for machines without a GPU. On this card: N = 1,000,000, spread S=100: 64.8 ms (1.09× the eagle graph time); spread S=1000: 578 ms (0.986× the eagle graph time); uniform S=1000: 1.48 s (2.3× the eagle graph time). Fastest in 0 of 12 cells.
Compile time
Its own pass, never part of any wall. Each measurement is a fresh process: the imports and the GPU context start-up run untimed, then the timer spans the first call to a ready kernel (trace + code generation + compile + module load, or the cache lookup). Cold: every cache empty – hawk’s compile cache (HAWK_CACHE_DIR), CuPy’s kernel cache (CUPY_CACHE_DIR), the CUDA driver’s PTX cache (CUDA_CACHE_PATH), PyTorch Inductor’s and Triton’s caches (TORCHINDUCTOR_CACHE_DIR, TRITON_CACHE_DIR) and JAX’s persistent compilation cache (minimum compile time and entry size 0), each a fresh directory. Warm: a new process on the caches the first cold run populated. eagle arms: eagle.deploy of the arm’s hawk kernel, which returns once the device build is done (C++/CUDA emission, nvcc to PTX, driver load; eagle’s own small CuPy kernels compile beside it) while the host compiler builds the host side on another thread, and the arm’s set-up on a 64-sample batch (eagle’s graph built, one step run). CuPy: one masked step on 64 samples (every elementwise and reduction kernel compiled on first use). Warp: the kernel module’s code generation, compile and load on the first launch (64 samples), cached in WARP_CACHE_PATH. PyTorch torch.compile, where it runs: three steps at N (the first call traces and compiles, the second records the CUDA graph, the third replays it), per N (dynamic=False). JAX: lower + compile from abstract shapes, per N (XLA specialises on the shape). CPU OpenMP: the same eagle.deploy as the eagle graph arm and one host run, which waits for the host build. Medians of 3 cold and 3 warm processes.
arm |
compiled |
cold (median of 3) |
warm (median of 3) |
|---|---|---|---|
eagle graph (device loop) |
eagle.deploy of the hawk step kernel (the device build; the host side builds in the background), the device-loop graph |
1.74 s |
139 ms |
eagle graph + compaction |
eagle.deploy of the hawk active-set step kernel (the device build; the host side builds in the background), the compaction-loop graph |
1.54 s |
114 ms |
eagle graph + compaction + reorder |
eagle.deploy of the hawk active-set step kernel (the device build; the host side builds in the background), the compaction + reorder loop graph |
1.33 s |
107 ms |
eagle graph, K fused steps per launch |
eagle.deploy of the hawk K-steps kernel (the device build; the host side builds in the background), the device-loop graph |
1.33 s |
123 ms |
eagle graph, K fused steps per launch + compaction |
eagle.deploy of the hawk active-set K-steps kernel (the device build; the host side builds in the background), the compaction-loop graph |
1.2 s |
110 ms |
eagle auto (eagle picks the launch mode) |
eagle.deploy of the hawk active-set automatic kernel (the device build; the host side builds in the background), the policy kernel, the policy-loop graph |
1.2 s |
120 ms |
hawk + eagle persistent |
eagle.deploy of the hawk automatic kernel (the device build; the host side builds in the background), the persistent launch |
1.11 s |
133 ms |
eagle.simulate (eagle picks the launch mode) |
eagle.simulate’s deploy of the hawk active-set automatic kernel (the device build; the host side builds in the background), the policy kernel, the policy-loop graph |
1.12 s |
119 ms |
eager launch loop |
eagle.deploy of the hawk single step kernel (the device build; the host side builds in the background) |
1.02 s |
89.8 ms |
CuPy masked |
CuPy’s elementwise and reduction kernels of one masked step |
715 ms |
25 ms |
Warp per-thread kernel |
Warp kernel module (code generation, NVRTC compile, load) |
537 ms |
59 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 1,000 |
201 ms |
64.7 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 10,000 |
203 ms |
41.6 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 100,000 |
214 ms |
41.3 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 1,000,000 |
235 ms |
41.9 ms |
CPU OpenMP |
eagle.deploy of the hawk step kernel and its first host run, which waits for the host build |
1.29 s |
110 ms |
No compile step: PyTorch masked, PyTorch, CUDA graph of 16 steps (PyTorch’s eager kernels ship prebuilt; the CUDA-graph capture is per N and recorded per configuration as torch_graph_capture_s).
Memory method
A separate pass after the timing pass. Each (arm, N, configuration) runs in a fresh process: imports, the CUDA context created by one tiny device operation, then the baseline read, before the arm builds or allocates anything; then the arm’s kernels built and a warm-up run on a 64-sample batch (every compile happens here; JAX also compiles for shape N from abstract shapes), so every library’s pool reserved during the warm-up counts in the reading; CuPy’s pools freed and torch.cuda.empty_cache() called after the warm-up (Warp’s pool release threshold set to 0); then the batch’s inputs generated, the arm set up for N (eagle builds its graph, PyTorch captures its CUDA graph, whose private memory pool counts) and one full run with upload and download. Device memory = the memory the CUDA driver accounts to the process (NVML per-process used memory) after the run minus at the baseline. The caching allocators of CuPy, PyTorch and JAX are left at their defaults and are not trimmed after the run; each keeps the memory it reserved until trimmed, so the reading after the run is the allocator’s high-water reservation, the same measure for every arm. A disabled pool returns memory between operations, so its peak could only be caught by sampling, which misses short peaks; the held reservation needs no sampling. Caveats: the reading is what the process holds, so each allocator’s rounding and growth policy count (JAX’s allocator grows in regions that can exceed the request; PyTorch’s rounds blocks up; a captured CUDA graph keeps a private pool; Warp’s stream-ordered pool reserves in chunks of tens of MiB); the driver accounts in pages of about 2 MiB, so small-N rows read 0 or one page. The per-library counters (CuPy pool, torch.cuda.max_memory_reserved/allocated, JAX memory_stats) are recorded in the JSON as a cross-check. Host memory = peak resident set size above the host baseline (taken after the warm-up; /proc/self/clear_refs reset there, then VmHWM). Minimum = the state the workload needs, N × ((2 state + 2 float64 scalars: omega, zeta) × 8 + 4 bytes for the stop step as a 32-bit integer) = 36 bytes per sample; the factor is device memory / minimum.
Device memory the process already held at the baseline (CUDA context, loaded kernels and libraries, the warm-up’s leftovers; at N = 1,000,000, first configuration): eagle graph (device loop) 58 MiB; eagle graph + compaction 58 MiB; eagle graph + compaction + reorder 58 MiB; eagle graph, K fused steps per launch 58 MiB; eagle graph, K fused steps per launch + compaction 58 MiB; eagle auto (eagle picks the launch mode) 58 MiB; hawk + eagle persistent 58 MiB; eagle.simulate (eagle picks the launch mode) 58 MiB; eager launch loop 58 MiB; CuPy masked 58 MiB; PyTorch masked 58 MiB; PyTorch, CUDA graph of 16 steps 58 MiB; JAX jit + vmap(while_loop) 58 MiB; Warp per-thread kernel 58 MiB.
spread (up to 100 steps)
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (132 µs), then eagle.simulate (eagle picks the launch mode) (144 µs, 1.09× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (318 µs), then eagle.simulate (eagle picks the launch mode) (331 µs, 1.04× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (1.77 ms), then eagle.simulate (eagle picks the launch mode) (1.78 ms, 1× the time)
N = 1,000,000: eagle.simulate (eagle picks the launch mode) (28.1 ms), then eagle auto (eagle picks the launch mode) (28.1 ms, 1× the time)
spread (up to 1000 steps)
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (811 µs), then eagle.simulate (eagle picks the launch mode) (822 µs, 1.01× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (2.39 ms), then eagle.simulate (eagle picks the launch mode) (2.39 ms, 1× the time)
N = 100,000: eagle.simulate (eagle picks the launch mode) (15.7 ms), then eagle auto (eagle picks the launch mode) (15.8 ms, 1× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (150 ms), then eagle.simulate (eagle picks the launch mode) (150 ms, 1× the time)
uniform (up to 1000 steps)
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (817 µs), then eagle.simulate (eagle picks the launch mode) (824 µs, 1.01× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (6.62 ms), then eagle.simulate (eagle picks the launch mode) (6.63 ms, 1× the time)
N = 100,000: eagle.simulate (eagle picks the launch mode) (65 ms), then hawk + eagle persistent (65 ms, 1× the time)
N = 1,000,000: eagle.simulate (eagle picks the launch mode) (640 ms), then eagle auto (eagle picks the launch mode) (640 ms, 1× the time)
eagle_simulate against its hand-built twin eagle_graph_auto:
spread (up to 100 steps), N = 1,000: wall 1.09× the twin’s time
spread (up to 100 steps), N = 10,000: wall 1.04× the twin’s time
spread (up to 100 steps), N = 100,000: wall 1× the twin’s time
spread (up to 100 steps), N = 1,000,000: wall 1× the twin’s time
spread (up to 1000 steps), N = 1,000: wall 1.01× the twin’s time
spread (up to 1000 steps), N = 10,000: wall 1× the twin’s time
spread (up to 1000 steps), N = 100,000: wall 0.997× the twin’s time
spread (up to 1000 steps), N = 1,000,000: wall 1× the twin’s time
uniform (up to 1000 steps), N = 1,000: wall 1.01× the twin’s time
uniform (up to 1000 steps), N = 10,000: wall 1× the twin’s time
uniform (up to 1000 steps), N = 100,000: wall 0.999× the twin’s time
uniform (up to 1000 steps), N = 1,000,000: wall 1× the twin’s time
Run on: driver 580.178.04; CUDA runtime 12.9; host CPU AMD EPYC 9B45; compiler x86_64-conda-linux-gnu-g++ (conda-forge gcc 14.4.0-7) 14.4.0; Python 3.12.14; Warp 1.17.0, JAX 0.11.2, PyTorch 2.12.1+cu129, CuPy 14.2.0, NumPy 2.5.3.
N = 1,000,000
Log scale, from 529 µs to 48.8 ms.
N = 1,000,000
Log scale, from 4.63 ms to 480 ms.
N = 1,000,000
Log scale, from 17.2 ms to 474 ms.
Table of every arm and every N
Device: NVIDIA RTX PRO 6000 Blackwell Server Edition (compute capability 12.0, 188 SMs). Peak FP64: – FLOP/s. Peak DRAM bandwidth: 1.60e12 B/s. Mean of the middle 3 of 5 runs of each arm (the highest and the lowest dropped), range of those 3 in parentheses.
Each arm’s block starts once the GPU is within 3 °C of its idle 36 °C and reads idle, or after 20 s; the SM clock is read right after each timed run and its range is in the last column.
eagle_graph checks its stop guard every step; eagle_graph_compact checks it once every 16 steps (the cadence its compaction runs on) and reads an active-set map built from it. The two rows below differ in both the cadence and the map; the compaction cadence sweep further down holds the cadence fixed at a few values and isolates the map alone.
PyTorch, CUDA graph of 16 steps captures 16 masked steps once as a torch.cuda.CUDAGraph over static buffers and replays it, with one host check of “any sample running” per replay: it removes the per-kernel launch overhead and keeps the masking (every sample is computed every step, finished ones held by torch.where, so the steps a replay runs past the last stop change nothing). JAX’s while loop is one compiled executable with no host check per step; batched by vmap, it runs until the slowest sample stops. Warp runs one thread per sample, each stopping at its own step.
Every arm runs each sample to its own stop and no further than max_steps (S): eagle and Warp enforce the cap per sample, JAX in its while-loop condition, PyTorch and CuPy by the outer step count (a batch-level cap); an arm that runs whole blocks of steps per launch or replay stops at the first block boundary at or after it.
What each arm needed to get there: Warp, a hand-written step counter and cap inside the kernel; JAX, a cap in the while_loop condition; eagle, the kernel source unchanged, with its loop shape, unrolling and device entry chosen by hawk + eagle (per device, per batch).
Cap fixture (checked before any timing counts): N = 4,096, 8 samples with stop step 2S = 256, S = 128; every arm (16) reported k = min(stop step, S) per sample and the same finished count (4,088): passed.
arm |
loop structure |
|---|---|
eagle graph (device loop) |
per launch over the batch, inside one CUDA graph: the plain decorator’s kernel, which finishes its own samples and runs the steps per launch eagle’s policy picks (8 to 64), guard checked on the device every launch |
eagle graph + compaction |
as the graph arm, the step launched over the active-set map only (recomputed every 16 steps) |
eagle graph + compaction + reorder |
as the compaction arm, plus a physical reorder of the per-sample planes when the live samples spread thin |
eagle graph, K fused steps per launch |
per launch over the batch, inside one CUDA graph: the step kernel runs up to K = 16 steps per sample with the state in registers, each sample leaving at its own stop step; guard checked on the device every launch |
eagle graph, K fused steps per launch + compaction |
as the K-steps arm, the launch over the active-set map only (recomputed every 64 steps = 4 launches) |
eagle auto (eagle picks the launch mode) |
the active-set kernel under eagle’s automatic policy, which picks the launch mode from the batch: one launch running every sample to its end for a small batch, the persistent launch above that, and the compaction loop (active-set map, steps per launch picked on the device) only when the caller asks for compaction or reordering; the row records the mode taken |
hawk + eagle persistent |
ONE launch, no graph, no map, no policy kernel: a grid sized from the SM count and the kernel’s occupancy, each lane fetching base + atomicAdd(counter, 1) whenever its sample is done, running the whole budget’s worth of steps per sample it picks up |
eagle.simulate (eagle picks the launch mode) |
as the automatic arm: eagle.simulate takes the kernel, its state and its parameters by name and eagle picks the launch mode the same way |
eager launch loop |
per step over the batch: one kernel launch from Python, a host read of the finished count every step |
CuPy masked |
per step over the batch: whole-batch array expressions, finished samples held by cupy.where, a host check every step; the cap is batch-level (the outer step loop runs at most max_steps) |
PyTorch masked |
per step over the batch: whole-batch tensor expressions, finished samples held by torch.where, a host check every step; the cap is batch-level (the outer step loop runs at most max_steps) |
PyTorch, CUDA graph of 16 steps |
as PyTorch masked, 16 steps captured as one CUDA graph over static buffers, replayed with one host check per replay |
PyTorch torch.compile (reduce-overhead) |
as PyTorch masked, the step compiled by Inductor and replayed as a CUDA graph, a host check every step |
JAX jit + vmap(while_loop) |
per step over the batch, inside one compiled executable: the vmapped while loop runs until the slowest sample stops or the step cap is reached, finished samples held by a select |
Warp per-thread kernel |
per sample: one launch, each thread loops over its own sample’s steps and stops at its own stop step or the step cap, whichever comes first |
CPU OpenMP |
per step over the batch on the host: one host-team launch per step, the kernel marks and counts its finished samples |
Spread stop steps (log-uniform in [1, 100]), up to 100 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
244 µs (240 µs–246 µs) |
– |
305 µs |
9.04e7 |
3.98e9 (–) |
2.99e9 (0.19 %) |
2 MiB (58.3×) |
2.07 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eagle graph + compaction |
487 µs (485 µs–488 µs) |
– |
564 µs |
4.53e7 |
1.99e9 (–) |
4.05e9 (0.25 %) |
2 MiB (58.3×) |
2.07 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eagle graph + compaction + reorder |
530 µs (529 µs–531 µs) |
– |
624 µs |
4.16e7 |
1.83e9 (–) |
4.52e9 (0.28 %) |
2 MiB (58.3×) |
2.05 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eagle graph, K fused steps per launch |
157 µs (156 µs–158 µs) |
– |
217 µs |
1.41e8 |
6.19e9 (–) |
1.83e9 (0.11 %) |
2 MiB (58.3×) |
2.05 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eagle graph, K fused steps per launch + compaction |
197 µs (192 µs–204 µs) |
– |
274 µs |
1.12e8 |
4.93e9 (–) |
1.40e9 (0.088 %) |
2 MiB (58.3×) |
2.06 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eagle auto (eagle picks the launch mode) |
87.3 µs (86.5 µs–88.5 µs) |
– |
158 µs |
2.53e8 |
1.11e10 (–) |
7.47e9 (0.47 %) |
0 MiB (0×) |
2.08 MiB |
– |
– |
– |
2325–2325 |
1,000 |
hawk + eagle persistent |
156 µs (144 µs–162 µs) |
– |
215 µs |
1.41e8 |
6.22e9 (–) |
4.18e9 (0.26 %) |
0 MiB (0×) |
2.07 MiB |
– |
0.2098712531887755 |
– |
2325–2325 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
88.5 µs (88.1 µs–88.8 µs) |
– |
154 µs |
2.49e8 |
1.10e10 (–) |
7.36e9 (0.46 %) |
0 MiB (0×) |
2.05 MiB |
– |
– |
– |
2332–2332 |
1,000 |
eager launch loop |
1.61 ms (1.6 ms–1.61 ms) |
– |
1.66 ms |
1.37e7 |
6.05e8 (–) |
2.44e9 (0.15 %) |
0 MiB (0×) |
0.043 MiB |
– |
– |
– |
2325–2325 |
1,000 |
CuPy masked |
23 ms (22.9 ms–23 ms) |
– |
23 ms |
9.61e5 |
4.23e7 (–) |
3.92e9 (0.25 %) |
0 MiB (0×) |
166 MiB |
– |
– |
– |
2325–2325 |
1,000 |
PyTorch masked |
16.4 ms (16.4 ms–16.4 ms) |
– |
16.5 ms |
1.35e6 |
5.93e7 (–) |
5.50e9 (0.34 %) |
88 MiB (2.56e+03×) |
0.0703 MiB |
– |
– |
– |
2325–2325 |
1,000 |
PyTorch, CUDA graph of 16 steps |
7.32 ms (7.32 ms–7.32 ms) |
– |
7.4 ms |
3.01e6 |
1.33e8 (–) |
1.39e10 (0.87 %) |
160 MiB (4.66e+03×) |
0.16 MiB |
– |
– |
– |
2325–2325 |
1,000 |
PyTorch torch.compile (reduce-overhead) |
3.69 ms (3.68 ms–3.7 ms) |
– |
3.79 ms |
5.97e6 |
2.63e8 (–) |
2.38e9 (0.15 %) |
160 MiB (4.66e+03×) |
33.5 MiB |
– |
– |
– |
2325–2325 |
1,000 |
JAX jit + vmap(while_loop) |
1.57 ms (1.53 ms–1.59 ms) |
– |
2.09 ms |
1.41e7 |
6.20e8 (–) |
5.64e9 (0.35 %) |
158 MiB (4.6e+03×) |
0.0273 MiB |
– |
– |
– |
2325–2325 |
1,000 |
Warp per-thread kernel |
174 µs (174 µs–174 µs) |
– |
291 µs |
1.27e8 |
5.58e9 (–) |
3.68e8 (0.023 %) |
30 MiB (874×) |
0 MiB |
– |
– |
– |
2325–2325 |
1,000 |
CPU OpenMP |
113 µs (112 µs–113 µs) |
– |
120 µs |
1.96e8 |
8.62e9 (–) |
3.48e10 (–) |
– |
0.371 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
252 µs (249 µs–253 µs) |
– |
344 µs |
9.09e8 |
4.00e10 (–) |
2.99e10 (1.9 %) |
2 MiB (5.83×) |
2.46 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eagle graph + compaction |
496 µs (495 µs–498 µs) |
– |
602 µs |
4.60e8 |
2.03e10 (–) |
4.08e10 (2.6 %) |
2 MiB (5.83×) |
2.49 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eagle graph + compaction + reorder |
686 µs (682 µs–693 µs) |
– |
812 µs |
3.33e8 |
1.46e10 (–) |
3.57e10 (2.2 %) |
2 MiB (5.83×) |
3.04 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eagle graph, K fused steps per launch |
161 µs (161 µs–162 µs) |
– |
253 µs |
1.42e9 |
6.25e10 (–) |
1.79e10 (1.1 %) |
2 MiB (5.83×) |
2.46 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eagle graph, K fused steps per launch + compaction |
193 µs (192 µs–193 µs) |
– |
297 µs |
1.19e9 |
5.22e10 (–) |
1.45e10 (0.91 %) |
2 MiB (5.83×) |
2.45 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eagle auto (eagle picks the launch mode) |
87.4 µs (86.5 µs–88.9 µs) |
– |
186 µs |
2.61e9 |
1.15e11 (–) |
7.71e10 (4.8 %) |
0 MiB (0×) |
2.61 MiB |
– |
– |
– |
2325–2325 |
10,000 |
hawk + eagle persistent |
165 µs (164 µs–165 µs) |
– |
252 µs |
1.39e9 |
6.10e10 (–) |
4.09e10 (2.6 %) |
0 MiB (0×) |
2.6 MiB |
– |
0.21156370013627354 |
– |
2325–2325 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
90.6 µs (90.4 µs–90.8 µs) |
– |
189 µs |
2.52e9 |
1.11e11 (–) |
7.44e10 (4.7 %) |
0 MiB (0×) |
2.61 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eager launch loop |
1.64 ms (1.64 ms–1.65 ms) |
– |
1.73 ms |
1.39e8 |
6.12e9 (–) |
2.40e10 (1.5 %) |
0 MiB (0×) |
0.406 MiB |
– |
– |
– |
2325–2325 |
10,000 |
CuPy masked |
22.9 ms (22.9 ms–22.9 ms) |
– |
23 ms |
9.98e6 |
4.39e8 (–) |
3.93e10 (2.5 %) |
0 MiB (0×) |
211 MiB |
– |
– |
– |
2325–2332 |
10,000 |
PyTorch masked |
16.7 ms (16.7 ms–16.7 ms) |
– |
16.8 ms |
1.37e7 |
6.02e8 (–) |
5.39e10 (3.4 %) |
88 MiB (256×) |
0.68 MiB |
– |
– |
– |
2325–2325 |
10,000 |
PyTorch, CUDA graph of 16 steps |
7.4 ms (7.4 ms–7.4 ms) |
– |
7.53 ms |
3.09e7 |
1.36e9 (–) |
1.37e11 (8.6 %) |
160 MiB (466×) |
0.844 MiB |
– |
– |
– |
2325–2325 |
10,000 |
PyTorch torch.compile (reduce-overhead) |
3.39 ms (3.37 ms–3.41 ms) |
– |
3.52 ms |
6.74e7 |
2.97e9 (–) |
2.60e10 (1.6 %) |
288 MiB (839×) |
89.2 MiB |
– |
– |
– |
2325–2325 |
10,000 |
JAX jit + vmap(while_loop) |
1.58 ms (1.56 ms–1.6 ms) |
– |
2.11 ms |
1.44e8 |
6.36e9 (–) |
5.58e10 (3.5 %) |
158 MiB (460×) |
0.738 MiB |
– |
– |
– |
2325–2325 |
10,000 |
Warp per-thread kernel |
180 µs (179 µs–180 µs) |
– |
337 µs |
1.27e9 |
5.59e10 (–) |
3.56e9 (0.22 %) |
30 MiB (87.4×) |
0.691 MiB |
– |
– |
– |
2325–2325 |
10,000 |
CPU OpenMP |
168 µs (167 µs–169 µs) |
– |
209 µs |
1.36e9 |
5.99e10 (–) |
2.35e11 (–) |
– |
1.25 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
575 µs (571 µs–578 µs) |
– |
873 µs |
3.98e9 |
1.75e11 (–) |
1.44e11 (9 %) |
6 MiB (1.75×) |
7.89 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eagle graph + compaction |
706 µs (705 µs–707 µs) |
– |
1.01 ms |
3.24e9 |
1.43e11 (–) |
2.87e11 (18 %) |
6 MiB (1.75×) |
7.93 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eagle graph + compaction + reorder |
986 µs (984 µs–988 µs) |
– |
1.32 ms |
2.32e9 |
1.02e11 (–) |
2.49e11 (16 %) |
10 MiB (2.91×) |
8.54 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eagle graph, K fused steps per launch |
470 µs (470 µs–471 µs) |
– |
768 µs |
4.86e9 |
2.14e11 (–) |
6.11e10 (3.8 %) |
6 MiB (1.75×) |
7.88 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eagle graph, K fused steps per launch + compaction |
447 µs (445 µs–449 µs) |
– |
748 µs |
5.12e9 |
2.25e11 (–) |
6.24e10 (3.9 %) |
6 MiB (1.75×) |
7.89 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eagle auto (eagle picks the launch mode) |
182 µs (182 µs–182 µs) |
– |
477 µs |
1.26e10 |
5.53e11 (–) |
3.71e11 (23 %) |
4 MiB (1.17×) |
7.87 MiB |
– |
– |
– |
2325–2325 |
100,000 |
hawk + eagle persistent |
254 µs (254 µs–255 µs) |
– |
553 µs |
9.00e9 |
3.96e11 (–) |
2.65e11 (17 %) |
4 MiB (1.17×) |
7.86 MiB |
– |
0.3120814528420707 |
– |
2325–2325 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
186 µs (185 µs–187 µs) |
– |
487 µs |
1.23e10 |
5.41e11 (–) |
3.62e11 (23 %) |
4 MiB (1.17×) |
7.87 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eager launch loop |
2.47 ms (2.45 ms–2.49 ms) |
– |
2.77 ms |
9.25e8 |
4.07e10 (–) |
1.59e11 (10 %) |
4 MiB (1.17×) |
5.75 MiB |
– |
– |
– |
2325–2325 |
100,000 |
CuPy masked |
23.1 ms (23.1 ms–23.1 ms) |
– |
23.4 ms |
9.92e7 |
4.36e9 (–) |
3.91e11 (24 %) |
22 MiB (6.41×) |
9.48 MiB |
– |
– |
– |
2325–2325 |
100,000 |
PyTorch masked |
17.6 ms (17.6 ms–17.7 ms) |
– |
18 ms |
1.30e8 |
5.71e9 (–) |
5.11e11 (32 %) |
112 MiB (32.6×) |
6.86 MiB |
– |
– |
– |
2325–2325 |
100,000 |
PyTorch, CUDA graph of 16 steps |
7.58 ms (7.58 ms–7.59 ms) |
– |
7.96 ms |
3.02e8 |
1.33e10 (–) |
1.34e12 (84 %) |
182 MiB (53×) |
6.95 MiB |
– |
– |
– |
2325–2325 |
100,000 |
PyTorch torch.compile (reduce-overhead) |
4.86 ms (4.85 ms–4.87 ms) |
– |
5.25 ms |
4.71e8 |
2.07e10 (–) |
1.81e11 (11 %) |
298 MiB (86.8×) |
94 MiB |
– |
– |
– |
2325–2325 |
100,000 |
JAX jit + vmap(while_loop) |
2.64 ms (2.63 ms–2.65 ms) |
– |
3.43 ms |
8.68e8 |
3.82e10 (–) |
3.35e11 (21 %) |
158 MiB (46×) |
10.1 MiB |
– |
– |
– |
2325–2325 |
100,000 |
Warp per-thread kernel |
452 µs (452 µs–453 µs) |
– |
866 µs |
5.06e9 |
2.23e11 (–) |
1.41e10 (0.89 %) |
30 MiB (8.74×) |
6.87 MiB |
– |
– |
– |
2325–2325 |
100,000 |
CPU OpenMP |
495 µs (494 µs–496 µs) |
– |
921 µs |
4.62e9 |
2.03e11 (–) |
7.96e11 (–) |
– |
9.98 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
2.88 ms (2.88 ms–2.89 ms) |
– |
5.17 ms |
7.98e9 |
3.51e11 (–) |
3.39e11 (21 %) |
42 MiB (1.22×) |
62.8 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eagle graph + compaction |
1.8 ms (1.8 ms–1.8 ms) |
– |
4.15 ms |
1.28e10 |
5.61e11 (–) |
1.13e12 (71 %) |
58 MiB (1.69×) |
62.9 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eagle graph + compaction + reorder |
2.26 ms (2.26 ms–2.27 ms) |
– |
4.59 ms |
1.02e10 |
4.47e11 (–) |
1.09e12 (68 %) |
86 MiB (2.5×) |
63.1 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eagle graph, K fused steps per launch |
2.69 ms (2.68 ms–2.69 ms) |
– |
4.98 ms |
8.56e9 |
3.77e11 (–) |
1.07e11 (6.7 %) |
42 MiB (1.22×) |
62.8 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
2.13 ms (2.13 ms–2.13 ms) |
– |
4.45 ms |
1.08e10 |
4.75e11 (–) |
1.31e11 (8.2 %) |
58 MiB (1.69×) |
62.8 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
841 µs (838 µs–844 µs) |
– |
3.16 ms |
2.73e10 |
1.20e12 (–) |
8.06e11 (50 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
hawk + eagle persistent |
1.36 ms (1.36 ms–1.36 ms) |
– |
3.64 ms |
1.69e10 |
7.42e11 (–) |
4.97e11 (31 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
0.46421765372493845 |
– |
2325–2325 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
847 µs (845 µs–851 µs) |
– |
3.17 ms |
2.72e10 |
1.19e12 (–) |
8.01e11 (50 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eager launch loop |
4.93 ms (4.92 ms–4.95 ms) |
– |
7.2 ms |
4.66e9 |
2.05e11 (–) |
7.99e11 (50 %) |
40 MiB (1.17×) |
58.2 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
CuPy masked |
30.5 ms (30.4 ms–30.7 ms) |
– |
33.1 ms |
7.54e8 |
3.32e10 (–) |
2.95e12 (1.8e+02 %) |
192 MiB (5.59×) |
68.7 MiB |
– |
– |
– |
2317–2317 |
1,000,000 |
PyTorch masked |
26.7 ms (26.5 ms–26.8 ms) |
– |
32.2 ms |
8.63e8 |
3.80e10 (–) |
3.38e12 (2.1e+02 %) |
348 MiB (10.1×) |
68.7 MiB |
– |
– |
– |
2317–2325 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
23.4 ms (23.4 ms–23.4 ms) |
– |
28.8 ms |
9.83e8 |
4.32e10 (–) |
4.33e12 (2.7e+02 %) |
420 MiB (12.2×) |
68.7 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
PyTorch torch.compile (reduce-overhead) |
7.33 ms (7.31 ms–7.36 ms) |
– |
12.8 ms |
3.14e9 |
1.38e11 (–) |
1.20e12 (75 %) |
428 MiB (12.5×) |
145 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
JAX jit + vmap(while_loop) |
8.12 ms (8.11 ms–8.15 ms) |
– |
12.2 ms |
2.83e9 |
1.25e11 (–) |
1.09e12 (68 %) |
158 MiB (4.6×) |
101 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
Warp per-thread kernel |
2.97 ms (2.97 ms–2.98 ms) |
– |
10.2 ms |
7.74e9 |
3.40e11 (–) |
2.15e10 (1.3 %) |
62 MiB (1.81×) |
68.7 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
CPU OpenMP |
3.73 ms (3.72 ms–3.74 ms) |
– |
9.09 ms |
6.16e9 |
2.71e11 (–) |
1.06e12 (–) |
– |
99.4 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (87.3 µs), then eagle.simulate (eagle picks the launch mode) (88.5 µs, 1.01× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (87.4 µs), then eagle.simulate (eagle picks the launch mode) (90.6 µs, 1.04× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (182 µs), then eagle.simulate (eagle picks the launch mode) (186 µs, 1.02× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (841 µs), then eagle.simulate (eagle picks the launch mode) (847 µs, 1.01× the time)
Spread stop steps (log-uniform in [10, 1000]), up to 1000 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
928 µs (927 µs–929 µs) |
– |
995 µs |
2.27e8 |
9.97e9 (–) |
7.02e9 (0.44 %) |
2 MiB (58.3×) |
2.04 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eagle graph + compaction |
3.82 ms (3.82 ms–3.82 ms) |
– |
3.9 ms |
5.50e7 |
2.42e9 (–) |
4.14e9 (0.26 %) |
2 MiB (58.3×) |
2.07 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eagle graph + compaction + reorder |
3.94 ms (3.94 ms–3.94 ms) |
– |
4.04 ms |
5.34e7 |
2.35e9 (–) |
4.19e9 (0.26 %) |
2 MiB (58.3×) |
2.14 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eagle graph, K fused steps per launch |
998 µs (997 µs–999 µs) |
– |
1.06 ms |
2.11e8 |
9.26e9 (–) |
2.46e9 (0.15 %) |
2 MiB (58.3×) |
2.03 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eagle graph, K fused steps per launch + compaction |
1.11 ms (1.11 ms–1.11 ms) |
– |
1.19 ms |
1.90e8 |
8.34e9 (–) |
1.28e9 (0.08 %) |
2 MiB (58.3×) |
2.04 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eagle auto (eagle picks the launch mode) |
631 µs (629 µs–632 µs) |
– |
704 µs |
3.33e8 |
1.47e10 (–) |
9.39e9 (0.59 %) |
0 MiB (0×) |
2.05 MiB |
– |
– |
– |
2332–2332 |
1,000 |
hawk + eagle persistent |
1.23 ms (1.22 ms–1.23 ms) |
– |
1.29 ms |
1.71e8 |
7.53e9 (–) |
4.82e9 (0.3 %) |
0 MiB (0×) |
2.04 MiB |
– |
0.22980196323115862 |
– |
2325–2325 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
634 µs (634 µs–635 µs) |
– |
706 µs |
3.31e8 |
1.46e10 (–) |
9.33e9 (0.58 %) |
0 MiB (0×) |
2.05 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eager launch loop |
15.4 ms (15.4 ms–15.5 ms) |
– |
15.5 ms |
1.36e7 |
5.99e8 (–) |
2.52e9 (0.16 %) |
0 MiB (0×) |
0.0352 MiB |
– |
– |
– |
2325–2325 |
1,000 |
CuPy masked |
229 ms (229 ms–229 ms) |
– |
229 ms |
9.18e5 |
4.04e7 (–) |
3.94e9 (0.25 %) |
0 MiB (0×) |
5.5 MiB |
– |
– |
– |
2325–2325 |
1,000 |
PyTorch masked |
166 ms (166 ms–166 ms) |
– |
166 ms |
1.27e6 |
5.58e7 (–) |
5.44e9 (0.34 %) |
88 MiB (2.56e+03×) |
0.0703 MiB |
– |
– |
– |
2325–2325 |
1,000 |
PyTorch, CUDA graph of 16 steps |
65.7 ms (65.7 ms–65.7 ms) |
– |
65.8 ms |
3.20e6 |
1.41e8 (–) |
1.39e10 (0.87 %) |
160 MiB (4.66e+03×) |
0.16 MiB |
– |
– |
– |
2325–2325 |
1,000 |
PyTorch torch.compile (reduce-overhead) |
36.9 ms (36.5 ms–37 ms) |
– |
37 ms |
5.70e6 |
2.51e8 (–) |
2.39e9 (0.15 %) |
160 MiB (4.66e+03×) |
30.3 MiB |
– |
– |
– |
2325–2325 |
1,000 |
JAX jit + vmap(while_loop) |
14.6 ms (14.6 ms–14.6 ms) |
– |
15.3 ms |
1.44e7 |
6.34e8 (–) |
6.03e9 (0.38 %) |
158 MiB (4.6e+03×) |
0.0156 MiB |
– |
– |
– |
2325–2325 |
1,000 |
Warp per-thread kernel |
1.43 ms (1.42 ms–1.43 ms) |
– |
1.55 ms |
1.47e8 |
6.49e9 (–) |
4.49e7 (0.0028 %) |
30 MiB (874×) |
0.00391 MiB |
– |
– |
– |
2332–2332 |
1,000 |
CPU OpenMP |
193 µs (190 µs–195 µs) |
– |
200 µs |
1.09e9 |
4.78e10 (–) |
2.01e11 (–) |
– |
0.355 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
927 µs (922 µs–934 µs) |
– |
1.02 ms |
2.35e9 |
1.04e11 (–) |
7.27e10 (4.6 %) |
2 MiB (5.83×) |
2.46 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eagle graph + compaction |
3.92 ms (3.92 ms–3.92 ms) |
– |
4.03 ms |
5.57e8 |
2.45e10 (–) |
4.17e10 (2.6 %) |
2 MiB (5.83×) |
2.49 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eagle graph + compaction + reorder |
4.39 ms (4.39 ms–4.4 ms) |
– |
4.52 ms |
4.97e8 |
2.19e10 (–) |
3.85e10 (2.4 %) |
2 MiB (5.83×) |
3.03 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eagle graph, K fused steps per launch |
1.01 ms (1.01 ms–1.01 ms) |
– |
1.1 ms |
2.15e9 |
9.48e10 (–) |
2.44e10 (1.5 %) |
2 MiB (5.83×) |
2.45 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eagle graph, K fused steps per launch + compaction |
1.12 ms (1.11 ms–1.12 ms) |
– |
1.23 ms |
1.96e9 |
8.60e10 (–) |
1.31e10 (0.82 %) |
2 MiB (5.83×) |
2.46 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eagle auto (eagle picks the launch mode) |
633 µs (633 µs–633 µs) |
– |
733 µs |
3.45e9 |
1.52e11 (–) |
9.71e10 (6.1 %) |
0 MiB (0×) |
2.6 MiB |
– |
– |
– |
2325–2325 |
10,000 |
hawk + eagle persistent |
1.3 ms (1.3 ms–1.31 ms) |
– |
1.39 ms |
1.67e9 |
7.37e10 (–) |
4.71e10 (3 %) |
0 MiB (0×) |
2.6 MiB |
– |
0.2384722378329774 |
– |
2325–2325 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
636 µs (636 µs–636 µs) |
– |
737 µs |
3.43e9 |
1.51e11 (–) |
9.66e10 (6 %) |
0 MiB (0×) |
2.61 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eager launch loop |
15.3 ms (15.1 ms–15.5 ms) |
– |
15.4 ms |
1.43e8 |
6.29e9 (–) |
2.56e10 (1.6 %) |
0 MiB (0×) |
0.406 MiB |
– |
– |
– |
2325–2325 |
10,000 |
CuPy masked |
230 ms (229 ms–230 ms) |
– |
230 ms |
9.51e6 |
4.18e8 (–) |
3.93e10 (2.5 %) |
0 MiB (0×) |
5.87 MiB |
– |
– |
– |
2325–2325 |
10,000 |
PyTorch masked |
167 ms (167 ms–167 ms) |
– |
167 ms |
1.31e7 |
5.75e8 (–) |
5.39e10 (3.4 %) |
88 MiB (256×) |
0.684 MiB |
– |
– |
– |
2325–2325 |
10,000 |
PyTorch, CUDA graph of 16 steps |
66.5 ms (66.5 ms–66.5 ms) |
– |
66.6 ms |
3.28e7 |
1.44e9 (–) |
1.37e11 (8.6 %) |
160 MiB (466×) |
0.844 MiB |
– |
– |
– |
2325–2325 |
10,000 |
PyTorch torch.compile (reduce-overhead) |
35.1 ms (35.1 ms–35.2 ms) |
– |
35.3 ms |
6.21e7 |
2.73e9 (–) |
2.51e10 (1.6 %) |
160 MiB (466×) |
32.4 MiB |
– |
– |
– |
2325–2325 |
10,000 |
JAX jit + vmap(while_loop) |
14.6 ms (14.6 ms–14.6 ms) |
– |
15.3 ms |
1.50e8 |
6.58e9 (–) |
6.03e10 (3.8 %) |
158 MiB (460×) |
0.164 MiB |
– |
– |
– |
2325–2325 |
10,000 |
Warp per-thread kernel |
1.49 ms (1.49 ms–1.5 ms) |
– |
1.65 ms |
1.46e9 |
6.43e10 (–) |
4.28e8 (0.027 %) |
30 MiB (87.4×) |
0.691 MiB |
– |
– |
– |
2325–2325 |
10,000 |
CPU OpenMP |
525 µs (523 µs–528 µs) |
– |
576 µs |
4.16e9 |
1.83e11 (–) |
7.45e11 (–) |
– |
1.25 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
4.01 ms (4 ms–4.01 ms) |
– |
4.32 ms |
5.41e9 |
2.38e11 (–) |
1.67e11 (10 %) |
6 MiB (1.75×) |
7.89 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eagle graph + compaction |
5.79 ms (5.79 ms–5.79 ms) |
– |
6.1 ms |
3.74e9 |
1.65e11 (–) |
2.80e11 (18 %) |
6 MiB (1.75×) |
7.92 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eagle graph + compaction + reorder |
6 ms (5.99 ms–6 ms) |
– |
6.35 ms |
3.61e9 |
1.59e11 (–) |
2.80e11 (18 %) |
10 MiB (2.91×) |
8.55 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eagle graph, K fused steps per launch |
4.14 ms (4.14 ms–4.14 ms) |
– |
4.45 ms |
5.23e9 |
2.30e11 (–) |
5.97e10 (3.7 %) |
6 MiB (1.75×) |
7.88 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eagle graph, K fused steps per launch + compaction |
2.17 ms (2.17 ms–2.17 ms) |
– |
2.48 ms |
1.00e10 |
4.40e11 (–) |
6.70e10 (4.2 %) |
6 MiB (1.75×) |
7.9 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eagle auto (eagle picks the launch mode) |
1.47 ms (1.47 ms–1.47 ms) |
– |
1.77 ms |
1.47e10 |
6.47e11 (–) |
4.14e11 (26 %) |
4 MiB (1.17×) |
7.87 MiB |
– |
– |
– |
2325–2325 |
100,000 |
hawk + eagle persistent |
1.5 ms (1.5 ms–1.5 ms) |
– |
1.8 ms |
1.45e10 |
6.36e11 (–) |
4.07e11 (25 %) |
4 MiB (1.17×) |
7.86 MiB |
– |
0.43331045943964186 |
– |
2325–2325 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
1.48 ms (1.48 ms–1.49 ms) |
– |
1.78 ms |
1.46e10 |
6.44e11 (–) |
4.12e11 (26 %) |
4 MiB (1.17×) |
7.87 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eager launch loop |
24.4 ms (24.3 ms–24.5 ms) |
– |
24.7 ms |
8.87e8 |
3.90e10 (–) |
1.60e11 (10 %) |
4 MiB (1.17×) |
5.75 MiB |
– |
– |
– |
2325–2325 |
100,000 |
CuPy masked |
228 ms (228 ms–228 ms) |
– |
228 ms |
9.50e7 |
4.18e9 (–) |
3.95e11 (25 %) |
22 MiB (6.41×) |
9.48 MiB |
– |
– |
– |
2325–2325 |
100,000 |
PyTorch masked |
177 ms (177 ms–177 ms) |
– |
178 ms |
1.22e8 |
5.37e9 (–) |
5.08e11 (32 %) |
112 MiB (32.6×) |
6.86 MiB |
– |
– |
– |
2325–2325 |
100,000 |
PyTorch, CUDA graph of 16 steps |
68.2 ms (68.2 ms–68.2 ms) |
– |
68.6 ms |
3.18e8 |
1.40e10 (–) |
1.34e12 (84 %) |
182 MiB (53×) |
6.95 MiB |
– |
– |
– |
2325–2325 |
100,000 |
PyTorch torch.compile (reduce-overhead) |
47.5 ms (47.5 ms–47.6 ms) |
– |
47.9 ms |
4.56e8 |
2.01e10 (–) |
1.85e11 (12 %) |
166 MiB (48.4×) |
35.8 MiB |
– |
– |
– |
2325–2325 |
100,000 |
JAX jit + vmap(while_loop) |
24.7 ms (24.7 ms–24.8 ms) |
– |
25.6 ms |
8.76e8 |
3.86e10 (–) |
3.56e11 (22 %) |
158 MiB (46×) |
10.9 MiB |
– |
– |
– |
2325–2325 |
100,000 |
Warp per-thread kernel |
4.25 ms (4.25 ms–4.25 ms) |
– |
4.66 ms |
5.10e9 |
2.24e11 (–) |
1.51e9 (0.094 %) |
30 MiB (8.74×) |
6.87 MiB |
– |
– |
– |
2325–2325 |
100,000 |
CPU OpenMP |
3.47 ms (3.46 ms–3.49 ms) |
– |
3.91 ms |
6.23e9 |
2.74e11 (–) |
1.12e12 (–) |
– |
10.1 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
25.5 ms (25.5 ms–25.6 ms) |
– |
27.8 ms |
8.47e9 |
3.73e11 (–) |
2.88e11 (18 %) |
42 MiB (1.22×) |
62.8 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eagle graph + compaction |
13.1 ms (13.1 ms–13.1 ms) |
– |
15.4 ms |
1.65e10 |
7.26e11 (–) |
1.24e12 (77 %) |
58 MiB (1.69×) |
62.9 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eagle graph + compaction + reorder |
13.8 ms (13.8 ms–13.8 ms) |
– |
16.1 ms |
1.57e10 |
6.89e11 (–) |
1.22e12 (76 %) |
86 MiB (2.5×) |
63.1 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eagle graph, K fused steps per launch |
25.7 ms (25.6 ms–25.7 ms) |
– |
27.9 ms |
8.43e9 |
3.71e11 (–) |
9.63e10 (6 %) |
42 MiB (1.22×) |
62.8 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
8.48 ms (8.48 ms–8.48 ms) |
– |
10.8 ms |
2.55e10 |
1.12e12 (–) |
1.71e11 (11 %) |
58 MiB (1.69×) |
62.8 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
7.38 ms (7.37 ms–7.39 ms) |
– |
9.68 ms |
2.93e10 |
1.29e12 (–) |
8.25e11 (52 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
hawk + eagle persistent |
7.61 ms (7.61 ms–7.61 ms) |
– |
9.89 ms |
2.84e10 |
1.25e12 (–) |
8.00e11 (50 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
0.7960095207633673 |
– |
2325–2325 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
7.41 ms (7.41 ms–7.41 ms) |
– |
9.68 ms |
2.92e10 |
1.28e12 (–) |
8.22e11 (51 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eager launch loop |
48.9 ms (48.8 ms–49 ms) |
– |
51.2 ms |
4.42e9 |
1.94e11 (–) |
7.98e11 (50 %) |
40 MiB (1.17×) |
58.2 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
CuPy masked |
300 ms (299 ms–300 ms) |
– |
302 ms |
7.21e8 |
3.17e10 (–) |
3.00e12 (1.9e+02 %) |
192 MiB (5.59×) |
68.7 MiB |
– |
– |
– |
2415–2415 |
1,000,000 |
PyTorch masked |
265 ms (264 ms–267 ms) |
– |
271 ms |
8.15e8 |
3.59e10 (–) |
3.40e12 (2.1e+02 %) |
348 MiB (10.1×) |
68.7 MiB |
– |
– |
– |
2362–2422 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
210 ms (210 ms–210 ms) |
– |
216 ms |
1.03e9 |
4.52e10 (–) |
4.34e12 (2.7e+02 %) |
420 MiB (12.2×) |
68.7 MiB |
– |
– |
– |
2422–2422 |
1,000,000 |
PyTorch torch.compile (reduce-overhead) |
72.4 ms (72 ms–72.7 ms) |
– |
77.8 ms |
2.99e9 |
1.32e11 (–) |
1.22e12 (76 %) |
240 MiB (6.99×) |
76.5 MiB |
– |
– |
– |
2317–2317 |
1,000,000 |
JAX jit + vmap(while_loop) |
79.5 ms (79.5 ms–79.5 ms) |
– |
84.1 ms |
2.72e9 |
1.20e11 (–) |
1.11e12 (69 %) |
158 MiB (4.6×) |
101 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
Warp per-thread kernel |
28.9 ms (28.8 ms–28.9 ms) |
– |
36.2 ms |
7.49e9 |
3.30e11 (–) |
2.22e9 (0.14 %) |
62 MiB (1.81×) |
68.7 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
CPU OpenMP |
31.2 ms (31.2 ms–31.2 ms) |
– |
36.5 ms |
6.93e9 |
3.05e11 (–) |
1.25e12 (–) |
– |
99.4 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: CPU OpenMP (193 µs), then eagle auto (eagle picks the launch mode) (631 µs, 3.26× the time)
N = 10,000: CPU OpenMP (525 µs), then eagle auto (eagle picks the launch mode) (633 µs, 1.2× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (1.47 ms), then eagle.simulate (eagle picks the launch mode) (1.48 ms, 1.01× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (7.38 ms), then eagle.simulate (eagle picks the launch mode) (7.41 ms, 1× the time)
Uniform stop step: every sample runs 1000 steps
N |
arm |
wall (range) |
kernel-only |
end-to-end |
sample·steps/s |
useful FLOP/s (% peak) |
issued B/s (% peak) |
device memory (× minimum) |
host memory |
% issue peak |
slot fill |
launches (kernel/memcpy/API) |
SM clock (MHz) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1,000 |
eagle graph (device loop) |
909 µs (909 µs–910 µs) |
– |
979 µs |
1.10e9 |
4.84e10 (–) |
3.15e10 (2 %) |
2 MiB (58.3×) |
2.04 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eagle graph + compaction |
3.14 ms (3.14 ms–3.14 ms) |
– |
3.23 ms |
3.18e8 |
1.40e10 (–) |
2.08e10 (1.3 %) |
2 MiB (58.3×) |
2.06 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eagle graph + compaction + reorder |
3.17 ms (3.17 ms–3.17 ms) |
– |
3.27 ms |
3.15e8 |
1.39e10 (–) |
2.06e10 (1.3 %) |
2 MiB (58.3×) |
2.12 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eagle graph, K fused steps per launch |
1.01 ms (1.01 ms–1.01 ms) |
– |
1.08 ms |
9.90e8 |
4.35e10 (–) |
3.80e9 (0.24 %) |
2 MiB (58.3×) |
2.03 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eagle graph, K fused steps per launch + compaction |
938 µs (938 µs–939 µs) |
– |
1.02 ms |
1.07e9 |
4.69e10 (–) |
4.44e9 (0.28 %) |
2 MiB (58.3×) |
2.04 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eagle auto (eagle picks the launch mode) |
634 µs (634 µs–635 µs) |
– |
709 µs |
1.58e9 |
6.94e10 (–) |
4.42e10 (2.8 %) |
0 MiB (0×) |
2.05 MiB |
– |
– |
– |
2332–2332 |
1,000 |
hawk + eagle persistent |
1.36 ms (1.36 ms–1.36 ms) |
– |
1.42 ms |
7.36e8 |
3.24e10 (–) |
2.06e10 (1.3 %) |
0 MiB (0×) |
2.05 MiB |
– |
0.95367431640625 |
– |
2325–2325 |
1,000 |
eagle.simulate (eagle picks the launch mode) |
635 µs (634 µs–636 µs) |
– |
710 µs |
1.57e9 |
6.93e10 (–) |
4.41e10 (2.8 %) |
0 MiB (0×) |
2.04 MiB |
– |
– |
– |
2325–2325 |
1,000 |
eager launch loop |
15 ms (14.9 ms–15 ms) |
– |
15 ms |
6.68e7 |
2.94e9 (–) |
4.08e9 (0.26 %) |
0 MiB (0×) |
0.043 MiB |
– |
– |
– |
2325–2325 |
1,000 |
CuPy masked |
229 ms (228 ms–229 ms) |
– |
229 ms |
4.37e6 |
1.92e8 (–) |
3.94e9 (0.25 %) |
0 MiB (0×) |
5.5 MiB |
– |
– |
– |
2325–2325 |
1,000 |
PyTorch masked |
163 ms (163 ms–163 ms) |
– |
163 ms |
6.13e6 |
2.70e8 (–) |
5.53e9 (0.35 %) |
88 MiB (2.56e+03×) |
0.0703 MiB |
– |
– |
– |
2325–2325 |
1,000 |
PyTorch, CUDA graph of 16 steps |
65.7 ms (65.7 ms–65.7 ms) |
– |
65.8 ms |
1.52e7 |
6.70e8 (–) |
1.39e10 (0.87 %) |
160 MiB (4.66e+03×) |
0.16 MiB |
– |
– |
– |
2325–2325 |
1,000 |
PyTorch torch.compile (reduce-overhead) |
36.6 ms (36.6 ms–36.6 ms) |
– |
36.7 ms |
2.73e7 |
1.20e9 (–) |
2.40e9 (0.15 %) |
160 MiB (4.66e+03×) |
31.5 MiB |
– |
– |
– |
2325–2325 |
1,000 |
JAX jit + vmap(while_loop) |
14.6 ms (14.5 ms–14.6 ms) |
– |
15.2 ms |
6.87e7 |
3.02e9 (–) |
6.05e9 (0.38 %) |
158 MiB (4.6e+03×) |
0.0117 MiB |
– |
– |
– |
2325–2325 |
1,000 |
Warp per-thread kernel |
1.56 ms (1.56 ms–1.56 ms) |
– |
1.69 ms |
6.40e8 |
2.81e10 (–) |
4.09e7 (0.0026 %) |
30 MiB (874×) |
0.00781 MiB |
– |
– |
– |
2332–2332 |
1,000 |
CPU OpenMP |
210 µs (208 µs–211 µs) |
– |
217 µs |
4.76e9 |
2.10e11 (–) |
2.91e11 (–) |
– |
0.359 MiB |
– |
– |
– |
– |
10,000 |
eagle graph (device loop) |
924 µs (918 µs–930 µs) |
– |
1.03 ms |
1.08e10 |
4.76e11 (–) |
3.10e11 (19 %) |
2 MiB (5.83×) |
2.44 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eagle graph + compaction |
3.16 ms (3.16 ms–3.16 ms) |
– |
3.28 ms |
3.17e9 |
1.39e11 (–) |
2.07e11 (13 %) |
2 MiB (5.83×) |
2.48 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eagle graph + compaction + reorder |
3.45 ms (3.45 ms–3.46 ms) |
– |
3.59 ms |
2.89e9 |
1.27e11 (–) |
1.89e11 (12 %) |
2 MiB (5.83×) |
3 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eagle graph, K fused steps per launch |
1.02 ms (1.02 ms–1.02 ms) |
– |
1.12 ms |
9.81e9 |
4.32e11 (–) |
3.77e10 (2.4 %) |
2 MiB (5.83×) |
2.43 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eagle graph, K fused steps per launch + compaction |
944 µs (943 µs–944 µs) |
– |
1.06 ms |
1.06e10 |
4.66e11 (–) |
4.41e10 (2.8 %) |
2 MiB (5.83×) |
2.45 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eagle auto (eagle picks the launch mode) |
639 µs (638 µs–641 µs) |
– |
744 µs |
1.56e10 |
6.88e11 (–) |
4.39e11 (27 %) |
0 MiB (0×) |
2.6 MiB |
– |
– |
– |
2325–2325 |
10,000 |
hawk + eagle persistent |
1.36 ms (1.36 ms–1.36 ms) |
– |
1.46 ms |
7.34e9 |
3.23e11 (–) |
2.06e11 (13 %) |
0 MiB (0×) |
2.61 MiB |
– |
0.9750024960063898 |
– |
2325–2325 |
10,000 |
eagle.simulate (eagle picks the launch mode) |
643 µs (642 µs–644 µs) |
– |
749 µs |
1.56e10 |
6.84e11 (–) |
4.36e11 (27 %) |
0 MiB (0×) |
2.6 MiB |
– |
– |
– |
2325–2325 |
10,000 |
eager launch loop |
15.3 ms (15.3 ms–15.4 ms) |
– |
15.4 ms |
6.52e8 |
2.87e10 (–) |
3.98e10 (2.5 %) |
0 MiB (0×) |
0.414 MiB |
– |
– |
– |
2325–2325 |
10,000 |
CuPy masked |
230 ms (229 ms–230 ms) |
– |
230 ms |
4.36e7 |
1.92e9 (–) |
3.93e10 (2.5 %) |
0 MiB (0×) |
5.79 MiB |
– |
– |
– |
2325–2325 |
10,000 |
PyTorch masked |
165 ms (165 ms–165 ms) |
– |
165 ms |
6.07e7 |
2.67e9 (–) |
5.47e10 (3.4 %) |
88 MiB (256×) |
0.625 MiB |
– |
– |
– |
2325–2325 |
10,000 |
PyTorch, CUDA graph of 16 steps |
66.5 ms (66.5 ms–66.5 ms) |
– |
66.6 ms |
1.50e8 |
6.62e9 (–) |
1.37e11 (8.6 %) |
160 MiB (466×) |
0.844 MiB |
– |
– |
– |
2325–2325 |
10,000 |
PyTorch torch.compile (reduce-overhead) |
34.2 ms (34.1 ms–34.2 ms) |
– |
34.3 ms |
2.93e8 |
1.29e10 (–) |
2.57e10 (1.6 %) |
160 MiB (466×) |
32.5 MiB |
– |
– |
– |
2325–2325 |
10,000 |
JAX jit + vmap(while_loop) |
14.6 ms (14.6 ms–14.6 ms) |
– |
15.2 ms |
6.85e8 |
3.02e10 (–) |
6.03e10 (3.8 %) |
158 MiB (460×) |
0.684 MiB |
– |
– |
– |
2325–2325 |
10,000 |
Warp per-thread kernel |
1.57 ms (1.57 ms–1.57 ms) |
– |
1.74 ms |
6.37e9 |
2.80e11 (–) |
4.08e8 (0.026 %) |
30 MiB (87.4×) |
0.691 MiB |
– |
– |
– |
2325–2325 |
10,000 |
CPU OpenMP |
684 µs (682 µs–687 µs) |
– |
732 µs |
1.46e10 |
6.44e11 (–) |
8.92e11 (–) |
– |
1.23 MiB |
– |
– |
– |
– |
100,000 |
eagle graph (device loop) |
4.27 ms (4.26 ms–4.27 ms) |
– |
4.57 ms |
2.34e10 |
1.03e12 (–) |
6.71e11 (42 %) |
6 MiB (1.75×) |
7.67 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eagle graph + compaction |
7.57 ms (7.57 ms–7.57 ms) |
– |
7.89 ms |
1.32e10 |
5.82e11 (–) |
8.63e11 (54 %) |
6 MiB (1.75×) |
7.73 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eagle graph + compaction + reorder |
7.87 ms (7.87 ms–7.87 ms) |
– |
8.23 ms |
1.27e10 |
5.59e11 (–) |
8.30e11 (52 %) |
10 MiB (2.91×) |
8.37 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eagle graph, K fused steps per launch |
4.42 ms (4.42 ms–4.43 ms) |
– |
4.74 ms |
2.26e10 |
9.95e11 (–) |
8.69e10 (5.4 %) |
6 MiB (1.75×) |
7.66 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eagle graph, K fused steps per launch + compaction |
4.35 ms (4.35 ms–4.35 ms) |
– |
4.66 ms |
2.30e10 |
1.01e12 (–) |
9.56e10 (6 %) |
6 MiB (1.75×) |
7.71 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eagle auto (eagle picks the launch mode) |
3.27 ms (3.19 ms–3.31 ms) |
– |
3.57 ms |
3.06e10 |
1.35e12 (–) |
8.58e11 (54 %) |
4 MiB (1.17×) |
7.87 MiB |
– |
– |
– |
2325–2325 |
100,000 |
hawk + eagle persistent |
3.66 ms (3.66 ms–3.67 ms) |
– |
3.96 ms |
2.73e10 |
1.20e12 (–) |
7.66e11 (48 %) |
4 MiB (1.17×) |
7.86 MiB |
– |
0.9765625 |
– |
2325–2325 |
100,000 |
eagle.simulate (eagle picks the launch mode) |
3.23 ms (3.19 ms–3.32 ms) |
– |
3.54 ms |
3.09e10 |
1.36e12 (–) |
8.67e11 (54 %) |
4 MiB (1.17×) |
7.87 MiB |
– |
– |
– |
2325–2325 |
100,000 |
eager launch loop |
27.3 ms (27.2 ms–27.4 ms) |
– |
27.6 ms |
3.66e9 |
1.61e11 (–) |
2.24e11 (14 %) |
4 MiB (1.17×) |
5.76 MiB |
– |
– |
– |
2325–2325 |
100,000 |
CuPy masked |
228 ms (227 ms–228 ms) |
– |
228 ms |
4.39e8 |
1.93e10 (–) |
3.96e11 (25 %) |
22 MiB (6.41×) |
9.47 MiB |
– |
– |
– |
2325–2325 |
100,000 |
PyTorch masked |
174 ms (174 ms–174 ms) |
– |
175 ms |
5.74e8 |
2.53e10 (–) |
5.17e11 (32 %) |
112 MiB (32.6×) |
6.24 MiB |
– |
– |
– |
2325–2325 |
100,000 |
PyTorch, CUDA graph of 16 steps |
68.1 ms (68.1 ms–68.2 ms) |
– |
68.5 ms |
1.47e9 |
6.46e10 (–) |
1.34e12 (84 %) |
182 MiB (53×) |
6.33 MiB |
– |
– |
– |
2325–2325 |
100,000 |
PyTorch torch.compile (reduce-overhead) |
46.7 ms (46.6 ms–46.8 ms) |
– |
47.1 ms |
2.14e9 |
9.42e10 (–) |
1.88e11 (12 %) |
166 MiB (48.4×) |
35.5 MiB |
– |
– |
– |
2325–2325 |
100,000 |
JAX jit + vmap(while_loop) |
24.7 ms (24.6 ms–24.7 ms) |
– |
25.6 ms |
4.05e9 |
1.78e11 (–) |
3.57e11 (22 %) |
158 MiB (46×) |
10.4 MiB |
– |
– |
– |
2325–2325 |
100,000 |
Warp per-thread kernel |
4.62 ms (4.62 ms–4.62 ms) |
– |
5.05 ms |
2.16e10 |
9.52e11 (–) |
1.38e9 (0.087 %) |
30 MiB (8.74×) |
6.87 MiB |
– |
– |
– |
2325–2325 |
100,000 |
CPU OpenMP |
4.78 ms (4.75 ms–4.8 ms) |
– |
5.15 ms |
2.09e10 |
9.21e11 (–) |
1.28e12 (–) |
– |
10 MiB |
– |
– |
– |
– |
1,000,000 |
eagle graph (device loop) |
28 ms (28 ms–28 ms) |
– |
30.3 ms |
3.57e10 |
1.57e12 (–) |
1.02e12 (64 %) |
42 MiB (1.22×) |
60.1 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eagle graph + compaction |
35.9 ms (35.9 ms–35.9 ms) |
– |
38.2 ms |
2.79e10 |
1.23e12 (–) |
1.82e12 (1.1e+02 %) |
58 MiB (1.69×) |
60.1 MiB |
– |
– |
– |
2340–2422 |
1,000,000 |
eagle graph + compaction + reorder |
37.6 ms (37.6 ms–37.6 ms) |
– |
39.9 ms |
2.66e10 |
1.17e12 (–) |
1.74e12 (1.1e+02 %) |
86 MiB (2.5×) |
60.8 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eagle graph, K fused steps per launch |
28.4 ms (28.4 ms–28.4 ms) |
– |
30.7 ms |
3.52e10 |
1.55e12 (–) |
1.35e11 (8.5 %) |
42 MiB (1.22×) |
60.1 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eagle graph, K fused steps per launch + compaction |
28.4 ms (28.4 ms–28.4 ms) |
– |
30.7 ms |
3.52e10 |
1.55e12 (–) |
1.47e11 (9.2 %) |
58 MiB (1.69×) |
60.1 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eagle auto (eagle picks the launch mode) |
27.6 ms (27.6 ms–27.6 ms) |
– |
29.9 ms |
3.63e10 |
1.60e12 (–) |
1.02e12 (64 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
hawk + eagle persistent |
27.6 ms (27.6 ms–27.6 ms) |
– |
29.8 ms |
3.62e10 |
1.59e12 (–) |
1.02e12 (64 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
0.9765625 |
– |
2325–2325 |
1,000,000 |
eagle.simulate (eagle picks the launch mode) |
27.6 ms (27.6 ms–27.6 ms) |
– |
29.9 ms |
3.62e10 |
1.59e12 (–) |
1.02e12 (64 %) |
40 MiB (1.17×) |
60.2 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
eager launch loop |
52.2 ms (52.2 ms–52.3 ms) |
– |
54.5 ms |
1.91e10 |
8.42e11 (–) |
1.17e12 (73 %) |
40 MiB (1.17×) |
58.2 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
CuPy masked |
296 ms (296 ms–296 ms) |
– |
299 ms |
3.38e9 |
1.49e11 (–) |
3.04e12 (1.9e+02 %) |
192 MiB (5.59×) |
66.6 MiB |
– |
– |
– |
2415–2415 |
1,000,000 |
PyTorch masked |
265 ms (264 ms–267 ms) |
– |
271 ms |
3.77e9 |
1.66e11 (–) |
3.40e12 (2.1e+02 %) |
348 MiB (10.1×) |
62.1 MiB |
– |
– |
– |
2355–2415 |
1,000,000 |
PyTorch, CUDA graph of 16 steps |
209 ms (209 ms–209 ms) |
– |
215 ms |
4.77e9 |
2.10e11 (–) |
4.36e12 (2.7e+02 %) |
420 MiB (12.2×) |
62.2 MiB |
– |
– |
– |
2415–2422 |
1,000,000 |
PyTorch torch.compile (reduce-overhead) |
71.6 ms (71.5 ms–71.6 ms) |
– |
77.1 ms |
1.40e10 |
6.15e11 (–) |
1.23e12 (77 %) |
240 MiB (6.99×) |
69.8 MiB |
– |
– |
– |
2317–2317 |
1,000,000 |
JAX jit + vmap(while_loop) |
79.5 ms (79.5 ms–79.5 ms) |
– |
83.9 ms |
1.26e10 |
5.53e11 (–) |
1.11e12 (69 %) |
158 MiB (4.6×) |
95.1 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
Warp per-thread kernel |
32.1 ms (32.1 ms–32.1 ms) |
– |
39.2 ms |
3.11e10 |
1.37e12 (–) |
1.99e9 (0.12 %) |
62 MiB (1.81×) |
68.7 MiB |
– |
– |
– |
2325–2325 |
1,000,000 |
CPU OpenMP |
43.2 ms (43.2 ms–43.2 ms) |
– |
48.1 ms |
2.32e10 |
1.02e12 (–) |
1.41e12 (–) |
– |
97.6 MiB |
– |
– |
– |
– |
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: CPU OpenMP (210 µs), then eagle auto (eagle picks the launch mode) (634 µs, 3.02× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (639 µs), then eagle.simulate (eagle picks the launch mode) (643 µs, 1.01× the time)
N = 100,000: eagle.simulate (eagle picks the launch mode) (3.23 ms), then eagle auto (eagle picks the launch mode) (3.27 ms, 1.01× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (27.6 ms), then eagle.simulate (eagle picks the launch mode) (27.6 ms, 1× the time)
Compaction cadence sweep (isolates the active-set map)
the active-set map alone. For a given K, matched_cadence (the plain per-sample step, no map) and compact (eagle_graph_compact’s active-set step) check the loop’s stop guard at the SAME cadence, once every K steps; the only structural difference between the two rows at a given K is whether the step kernel reads the active-set map. The main table’s eagle_graph row checks its guard every step, so its difference from eagle_graph_compact there mixes the cadence change with the map – this sweep does not.
Config: Spread stop steps (log-uniform in [10, 1000]), up to 1000 steps.
N |
K (steps between guard checks) |
matched cadence, no map (range) |
eagle graph + compaction (range) |
ratio (matched/compact, |
|---|---|---|---|---|
100,000 |
8 |
6.54 ms (6.54 ms–6.54 ms) |
7.13 ms (7.12 ms–7.13 ms) |
0.917× |
100,000 |
16 |
6.33 ms (6.33 ms–6.33 ms) |
5.81 ms (5.81 ms–5.81 ms) |
1.09× |
100,000 |
32 |
6.2 ms (6.2 ms–6.2 ms) |
5.18 ms (5.18 ms–5.18 ms) |
1.2× |
1,000,000 |
8 |
33.2 ms (33.2 ms–33.2 ms) |
14.8 ms (14.8 ms–14.8 ms) |
2.24× |
1,000,000 |
16 |
33 ms (33 ms–33 ms) |
13.1 ms (13.1 ms–13.1 ms) |
2.51× |
1,000,000 |
32 |
32.9 ms (32.9 ms–32.9 ms) |
12.5 ms (12.5 ms–12.5 ms) |
2.63× |
FP32: eagle vs Warp (float32, S = 1000)
Every arm runs the same oscillator in float32: eagle.simulate with scalar_type=”float32”, NVIDIA Warp’s per-thread kernel and JAX’s vmapped loop, all over the same float32 inputs. Each cell runs 2 untimed runs per arm first (eagle’s automatic arm picks its entry over them: the size rule, then a measured run of the other entry, keeping the faster; the entry it settled on is in the last column), then each arm in its own block (no interleaving): an untimed ramp of back-to-back runs for 0.5 s, then 5 timed runs. Before any timing counts, eagle and JAX reach step counts identical to Warp’s and states within absolute 0.0001 of Warp’s (the largest differences are in the JSON). Each time is the mean of the middle 3 of the 5 runs (highest and lowest dropped), with the range of those 3; eagle / Warp is the ratio of these means: below 1 eagle is faster. Before each arm’s block the GPU is cooled down: the block starts once the temperature is within 3 °C of the idle 36 °C and the GPU is idle, or after 20 s. Right after each timed run the SM clock is read (its range is shown per arm); cycles are the wall time times that clock, and eagle / Warp (cycles) is their ratio.
Every arm runs each sample to its own stop and no further than max_steps (S): eagle and Warp enforce the cap per sample, JAX in its while-loop condition, PyTorch and CuPy by the outer step count (a batch-level cap); an arm that runs whole blocks of steps per launch or replay stops at the first block boundary at or after it.
What each arm needed to get there: Warp, a hand-written step counter and cap inside the kernel; JAX, a cap in the while_loop condition; eagle, the kernel source unchanged, with its loop shape, unrolling and device entry chosen by hawk + eagle (per device, per batch).
Cap fixture (checked before any timing counts): N = 4,096, 8 samples with stop step 2S = 256, S = 128; every arm (3) reported k = min(stop step, S) per sample and the same finished count (4,088): passed.
distribution |
N |
eagle.simulate (range) |
Warp (range) |
JAX (range) |
eagle / Warp |
eagle / Warp (cycles) |
eagle entry |
|---|---|---|---|---|---|---|---|
uniform |
10,000 |
62 µs (60.6 µs–63 µs), 2325–2325 MHz |
70.7 µs (70.6 µs–70.8 µs), 2325–2325 MHz |
14.6 ms (14.5 ms–14.6 ms), 2325–2325 MHz |
0.877× |
0.877× |
fused_one |
uniform |
100,000 |
128 µs (127 µs–129 µs), 2302–2302 MHz |
127 µs (126 µs–127 µs), 2302–2302 MHz |
14.5 ms (14.4 ms–14.5 ms), 2325–2325 MHz |
1.01× |
1.01× |
fused_one |
uniform |
1,000,000 |
614 µs (613 µs–615 µs), 2310–2310 MHz |
619 µs (618 µs–619 µs), 2310–2310 MHz |
25.7 ms (25.7 ms–25.7 ms), 2325–2325 MHz |
0.993× |
0.993× |
fused_one |
spread |
10,000 |
62 µs (61.7 µs–62.3 µs), 2325–2325 MHz |
70.2 µs (70.1 µs–70.2 µs), 2325–2325 MHz |
14.5 ms (14.5 ms–14.6 ms), 2325–2325 MHz |
0.884× |
0.884× |
fused_one |
spread |
100,000 |
102 µs (101 µs–102 µs), 2325–2325 MHz |
122 µs (122 µs–122 µs), 2317–2317 MHz |
14.4 ms (14.4 ms–14.4 ms), 2325–2325 MHz |
0.834× |
0.837× |
persist |
spread |
1,000,000 |
290 µs (289 µs–291 µs), 2317–2317 MHz |
567 µs (566 µs–568 µs), 2317–2317 MHz |
25.7 ms (25.7 ms–25.7 ms), 2325–2325 MHz |
0.511× |
0.511× |
persist |
Fixed overhead per run (N = 64, S = 100)
item 3: wall at a cell small enough that per-run overhead dominates (N=64, S=100), each arm’s own kernel-only vs wall where the main nsys pass covers this cell’s N (it does not by default, so kernel_only_s is usually null here)
arm |
wall (range) |
kernel-only |
|---|---|---|
eagle graph (device loop) |
261 µs (253 µs–272 µs) |
– |
eagle graph + compaction |
411 µs (407 µs–418 µs) |
– |
eagle graph + compaction + reorder |
444 µs (442 µs–446 µs) |
– |
eagle graph, K fused steps per launch |
162 µs (160 µs–164 µs) |
– |
eagle graph, K fused steps per launch + compaction |
185 µs (184 µs–186 µs) |
– |
eagle auto (eagle picks the launch mode) |
95.1 µs (94.4 µs–95.8 µs) |
– |
hawk + eagle persistent |
121 µs (121 µs–122 µs) |
– |
eagle.simulate (eagle picks the launch mode) |
97 µs (96.7 µs–97.6 µs) |
– |
eager launch loop |
1.49 ms (1.49 ms–1.49 ms) |
– |
CuPy masked |
22.4 ms (22.3 ms–22.4 ms) |
– |
PyTorch masked |
16.3 ms (16.3 ms–16.3 ms) |
– |
PyTorch, CUDA graph of 16 steps |
5.95 ms (5.95 ms–5.95 ms) |
– |
PyTorch torch.compile (reduce-overhead) |
3.6 ms (3.58 ms–3.62 ms) |
– |
JAX jit + vmap(while_loop) |
1.43 ms (1.42 ms–1.44 ms) |
– |
Warp per-thread kernel |
103 µs (103 µs–104 µs) |
– |
CPU OpenMP |
109 µs (107 µs–110 µs) |
– |
Where each tool fits
Each tool below is written the way its users write it, and each brings something of its own. Times are walls (mean of the middle 3 of 5 runs) at the largest N; ratios are against the eagle graph arm (below 1 = less time). The fastest arm per N is listed under each table above.
eagle graph (device loop). The step written once as a hawk kernel; the whole loop runs on the device as one CUDA graph with a device-side stop guard, so a run is one launch with no host round trip per step. On this card: N = 1,000,000, spread S=100: 2.88 ms; spread S=1000: 25.5 ms; uniform S=1000: 28 ms. Fastest in 0 of 12 cells.
eagle graph + compaction. Adds an active-set map: the launch covers only the samples still running, rebuilt every 16 steps; it is made for batches that thin out (the spread configurations). On this card: N = 1,000,000, spread S=100: 1.8 ms (0.625× the eagle graph time); spread S=1000: 13.1 ms (0.513× the eagle graph time); uniform S=1000: 35.9 ms (1.28× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph + compaction + reorder. Adds a physical reorder when the live samples spread thin over their warps, so the mapped reads stay contiguous; the final restore to sample order is inside its wall. On this card: N = 1,000,000, spread S=100: 2.26 ms (0.785× the eagle graph time); spread S=1000: 13.8 ms (0.541× the eagle graph time); uniform S=1000: 37.6 ms (1.34× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph, K fused steps per launch. The same step with one decorator argument, @hawk.kernel(steps=16): each launch runs that many steps per sample with the state kept in registers, as Warp’s per-thread loop does, and the launches, guard checks and plane reads shrink by the same factor; its results are bit-identical to the eagle graph arm’s in every cell. K is chosen by hand here; the eagle_graph_auto arm picks it at run time. On this card: N = 1,000,000, spread S=100: 2.69 ms (0.931× the eagle graph time); spread S=1000: 25.7 ms (1× the eagle graph time); uniform S=1000: 28.4 ms (1.01× the eagle graph time). Fastest in 0 of 12 cells.
eagle graph, K fused steps per launch + compaction. The K-steps kernel over the active-set map: the register-resident steps of the arm above, and the launches cover only the samples still running, so it fits batches that thin out. On this card: N = 1,000,000, spread S=100: 2.13 ms (0.738× the eagle graph time); spread S=1000: 8.48 ms (0.332× the eagle graph time); uniform S=1000: 28.4 ms (1.01× the eagle graph time). Fastest in 0 of 12 cells.
eagle auto (eagle picks the launch mode). The same step with @hawk.kernel(steps=”auto”) on the active-set kind, and nothing to tune: it picks the steps per launch itself after every launch, on the device, so a batch whose samples all run to the end gets Warp-like fusion of up to 64 steps per launch, and a batch that thins out gets shorter launches with the active set compacted after each one; no sample takes more steps than the cap. On this card: N = 1,000,000, spread S=100: 841 µs (0.292× the eagle graph time); spread S=1000: 7.38 ms (0.289× the eagle graph time); uniform S=1000: 27.6 ms (0.984× the eagle graph time). Fastest in 8 of 12 cells.
hawk + eagle persistent. The plain (not active-set) automatic kernel, forced through hawk’s persist entry: one launch, no graph, no map, no policy kernel, no cap at 64 – a grid of SMs blocks of 256 lanes each steal the next unfinished sample off a counter when their own finishes, so idle warps refill without ever rebuilding an index map; made for batches too large for one resident wave. On this card: N = 1,000,000, spread S=100: 1.36 ms (0.473× the eagle graph time); spread S=1000: 7.61 ms (0.298× the eagle graph time); uniform S=1000: 27.6 ms (0.984× the eagle graph time). Fastest in 0 of 12 cells.
eagle.simulate (eagle picks the launch mode). The same automatic kernel through eagle.simulate: the model, its state and its parameters by name, with no plan or plane binding to write; eagle deploys the kernel and builds the same device loop. On this card: N = 1,000,000, spread S=100: 847 µs (0.294× the eagle graph time); spread S=1000: 7.41 ms (0.29× the eagle graph time); uniform S=1000: 27.6 ms (0.984× the eagle graph time). Fastest in 1 of 12 cells.
eager launch loop. The same kernels launched from Python one step at a time; the host sees the state after every step. On this card: N = 1,000,000, spread S=100: 4.93 ms (1.71× the eagle graph time); spread S=1000: 48.9 ms (1.92× the eagle graph time); uniform S=1000: 52.2 ms (1.86× the eagle graph time). Fastest in 0 of 12 cells.
CuPy masked. NumPy-like array code on GPUs with no kernel to write; every sample is computed every step, so it fits dense batches where all samples run to the end (the uniform configuration). On this card: N = 1,000,000, spread S=100: 30.5 ms (10.6× the eagle graph time); spread S=1000: 300 ms (11.7× the eagle graph time); uniform S=1000: 296 ms (10.6× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch masked. The same array code in PyTorch tensors; it fits where the computation sits next to a PyTorch model or needs autograd. On this card: N = 1,000,000, spread S=100: 26.7 ms (9.25× the eagle graph time); spread S=1000: 265 ms (10.4× the eagle graph time); uniform S=1000: 265 ms (9.46× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch, CUDA graph of 16 steps. The same PyTorch code with its launch overhead removed by hand: 16 steps captured as one CUDA graph and replayed, one host check per replay, the masking unchanged. On this card: N = 1,000,000, spread S=100: 23.4 ms (8.12× the eagle graph time); spread S=1000: 210 ms (8.24× the eagle graph time); uniform S=1000: 209 ms (7.47× the eagle graph time). Fastest in 0 of 12 cells.
PyTorch torch.compile (reduce-overhead). The PyTorch step under torch.compile(mode=”reduce-overhead”): Inductor-fused kernels replayed as a CUDA graph. On this card: N = 1,000,000, spread S=100: 7.33 ms (2.54× the eagle graph time); spread S=1000: 72.4 ms (2.83× the eagle graph time); uniform S=1000: 71.6 ms (2.55× the eagle graph time). Fastest in 0 of 12 cells.
JAX jit + vmap(while_loop). The whole loop compiled once, a per-sample while loop batched with vmap; the same code runs on CPUs and GPUs and differentiates with jax.grad. On this card: N = 1,000,000, spread S=100: 8.12 ms (2.82× the eagle graph time); spread S=1000: 79.5 ms (3.11× the eagle graph time); uniform S=1000: 79.5 ms (2.84× the eagle graph time). Fastest in 0 of 12 cells.
Warp per-thread kernel. Explicit per-thread kernels written in Python, maintained by NVIDIA and differentiable through wp.Tape; each thread runs its own sample and stops at its own step, so finished samples cost nothing. On this card: N = 1,000,000, spread S=100: 2.97 ms (1.03× the eagle graph time); spread S=1000: 28.9 ms (1.13× the eagle graph time); uniform S=1000: 32.1 ms (1.15× the eagle graph time). Fastest in 0 of 12 cells.
CPU OpenMP. The same hawk kernel on the host through eagle’s OpenMP team, for machines without a GPU. On this card: N = 1,000,000, spread S=100: 3.73 ms (1.29× the eagle graph time); spread S=1000: 31.2 ms (1.22× the eagle graph time); uniform S=1000: 43.2 ms (1.54× the eagle graph time). Fastest in 3 of 12 cells.
Compile time
Its own pass, never part of any wall. Each measurement is a fresh process: the imports and the GPU context start-up run untimed, then the timer spans the first call to a ready kernel (trace + code generation + compile + module load, or the cache lookup). Cold: every cache empty – hawk’s compile cache (HAWK_CACHE_DIR), CuPy’s kernel cache (CUPY_CACHE_DIR), the CUDA driver’s PTX cache (CUDA_CACHE_PATH), PyTorch Inductor’s and Triton’s caches (TORCHINDUCTOR_CACHE_DIR, TRITON_CACHE_DIR) and JAX’s persistent compilation cache (minimum compile time and entry size 0), each a fresh directory. Warm: a new process on the caches the first cold run populated. eagle arms: eagle.deploy of the arm’s hawk kernel, which returns once the device build is done (C++/CUDA emission, nvcc to PTX, driver load; eagle’s own small CuPy kernels compile beside it) while the host compiler builds the host side on another thread, and the arm’s set-up on a 64-sample batch (eagle’s graph built, one step run). CuPy: one masked step on 64 samples (every elementwise and reduction kernel compiled on first use). Warp: the kernel module’s code generation, compile and load on the first launch (64 samples), cached in WARP_CACHE_PATH. PyTorch torch.compile, where it runs: three steps at N (the first call traces and compiles, the second records the CUDA graph, the third replays it), per N (dynamic=False). JAX: lower + compile from abstract shapes, per N (XLA specialises on the shape). CPU OpenMP: the same eagle.deploy as the eagle graph arm and one host run, which waits for the host build. Medians of 3 cold and 3 warm processes.
arm |
compiled |
cold (median of 3) |
warm (median of 3) |
|---|---|---|---|
eagle graph (device loop) |
eagle.deploy of the hawk step kernel (the device build; the host side builds in the background), the device-loop graph |
521 ms |
51.6 ms |
eagle graph + compaction |
eagle.deploy of the hawk active-set step kernel (the device build; the host side builds in the background), the compaction-loop graph |
402 ms |
46.2 ms |
eagle graph + compaction + reorder |
eagle.deploy of the hawk active-set step kernel (the device build; the host side builds in the background), the compaction + reorder loop graph |
449 ms |
45.1 ms |
eagle graph, K fused steps per launch |
eagle.deploy of the hawk K-steps kernel (the device build; the host side builds in the background), the device-loop graph |
406 ms |
50.8 ms |
eagle graph, K fused steps per launch + compaction |
eagle.deploy of the hawk active-set K-steps kernel (the device build; the host side builds in the background), the compaction-loop graph |
409 ms |
51.5 ms |
eagle auto (eagle picks the launch mode) |
eagle.deploy of the hawk active-set automatic kernel (the device build; the host side builds in the background), the policy kernel, the policy-loop graph |
440 ms |
48.7 ms |
hawk + eagle persistent |
eagle.deploy of the hawk automatic kernel (the device build; the host side builds in the background), the persistent launch |
439 ms |
49.4 ms |
eagle.simulate (eagle picks the launch mode) |
eagle.simulate’s deploy of the hawk active-set automatic kernel (the device build; the host side builds in the background), the policy kernel, the policy-loop graph |
440 ms |
47.7 ms |
eager launch loop |
eagle.deploy of the hawk single step kernel (the device build; the host side builds in the background) |
344 ms |
38.8 ms |
CuPy masked |
CuPy’s elementwise and reduction kernels of one masked step |
477 ms |
15.8 ms |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 1,000 |
2.45 s |
1.51 s |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 10,000 |
2.61 s |
1.52 s |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 100,000 |
2.65 s |
1.52 s |
PyTorch torch.compile (reduce-overhead) |
torch.compile (Inductor + Triton) and CUDA-graph record, N = 1,000,000 |
2.66 s |
1.52 s |
Warp per-thread kernel |
Warp kernel module (code generation, NVRTC compile, load) |
359 ms |
71.7 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 1,000 |
136 ms |
26.1 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 10,000 |
128 ms |
25.9 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 100,000 |
146 ms |
26.3 ms |
JAX jit + vmap(while_loop) |
XLA executable for N = 1,000,000 |
148 ms |
26.1 ms |
CPU OpenMP |
eagle.deploy of the hawk step kernel and its first host run, which waits for the host build |
660 ms |
48.7 ms |
No compile step: PyTorch masked, PyTorch, CUDA graph of 16 steps (PyTorch’s eager kernels ship prebuilt; the CUDA-graph capture is per N and recorded per configuration as torch_graph_capture_s).
Memory method
A separate pass after the timing pass. Each (arm, N, configuration) runs in a fresh process: imports, the CUDA context created by one tiny device operation, then the baseline read, before the arm builds or allocates anything; then the arm’s kernels built and a warm-up run on a 64-sample batch (every compile happens here; JAX also compiles for shape N from abstract shapes), so every library’s pool reserved during the warm-up counts in the reading; CuPy’s pools freed and torch.cuda.empty_cache() called after the warm-up (Warp’s pool release threshold set to 0); then the batch’s inputs generated, the arm set up for N (eagle builds its graph, PyTorch captures its CUDA graph, whose private memory pool counts) and one full run with upload and download. Device memory = the memory the CUDA driver accounts to the process (NVML per-process used memory) after the run minus at the baseline. The caching allocators of CuPy, PyTorch and JAX are left at their defaults and are not trimmed after the run; each keeps the memory it reserved until trimmed, so the reading after the run is the allocator’s high-water reservation, the same measure for every arm. A disabled pool returns memory between operations, so its peak could only be caught by sampling, which misses short peaks; the held reservation needs no sampling. Caveats: the reading is what the process holds, so each allocator’s rounding and growth policy count (JAX’s allocator grows in regions that can exceed the request; PyTorch’s rounds blocks up; a captured CUDA graph keeps a private pool; Warp’s stream-ordered pool reserves in chunks of tens of MiB); the driver accounts in pages of about 2 MiB, so small-N rows read 0 or one page. The per-library counters (CuPy pool, torch.cuda.max_memory_reserved/allocated, JAX memory_stats) are recorded in the JSON as a cross-check. Host memory = peak resident set size above the host baseline (taken after the warm-up; /proc/self/clear_refs reset there, then VmHWM). Minimum = the state the workload needs, N × ((2 state + 2 float64 scalars: omega, zeta) × 8 + 4 bytes for the stop step as a 32-bit integer) = 36 bytes per sample; the factor is device memory / minimum.
Device memory the process already held at the baseline (CUDA context, loaded kernels and libraries, the warm-up’s leftovers; at N = 1,000,000, first configuration): eagle graph (device loop) 554 MiB; eagle graph + compaction 554 MiB; eagle graph + compaction + reorder 554 MiB; eagle graph, K fused steps per launch 554 MiB; eagle graph, K fused steps per launch + compaction 554 MiB; eagle auto (eagle picks the launch mode) 554 MiB; hawk + eagle persistent 554 MiB; eagle.simulate (eagle picks the launch mode) 554 MiB; eager launch loop 554 MiB; CuPy masked 554 MiB; PyTorch masked 554 MiB; PyTorch, CUDA graph of 16 steps 554 MiB; PyTorch torch.compile (reduce-overhead) 554 MiB; JAX jit + vmap(while_loop) 554 MiB; Warp per-thread kernel 554 MiB.
spread (up to 100 steps)
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: eagle auto (eagle picks the launch mode) (87.3 µs), then eagle.simulate (eagle picks the launch mode) (88.5 µs, 1.01× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (87.4 µs), then eagle.simulate (eagle picks the launch mode) (90.6 µs, 1.04× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (182 µs), then eagle.simulate (eagle picks the launch mode) (186 µs, 1.02× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (841 µs), then eagle.simulate (eagle picks the launch mode) (847 µs, 1.01× the time)
spread (up to 1000 steps)
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: CPU OpenMP (193 µs), then eagle auto (eagle picks the launch mode) (631 µs, 3.26× the time)
N = 10,000: CPU OpenMP (525 µs), then eagle auto (eagle picks the launch mode) (633 µs, 1.2× the time)
N = 100,000: eagle auto (eagle picks the launch mode) (1.47 ms), then eagle.simulate (eagle picks the launch mode) (1.48 ms, 1.01× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (7.38 ms), then eagle.simulate (eagle picks the launch mode) (7.41 ms, 1× the time)
uniform (up to 1000 steps)
Fastest arm by mean wall time (middle 10 of 12 runs):
N = 1,000: CPU OpenMP (210 µs), then eagle auto (eagle picks the launch mode) (634 µs, 3.02× the time)
N = 10,000: eagle auto (eagle picks the launch mode) (639 µs), then eagle.simulate (eagle picks the launch mode) (643 µs, 1.01× the time)
N = 100,000: eagle.simulate (eagle picks the launch mode) (3.23 ms), then eagle auto (eagle picks the launch mode) (3.27 ms, 1.01× the time)
N = 1,000,000: eagle auto (eagle picks the launch mode) (27.6 ms), then eagle.simulate (eagle picks the launch mode) (27.6 ms, 1× the time)
eagle_simulate against its hand-built twin eagle_graph_auto:
spread (up to 100 steps), N = 1,000: wall 1.01× the twin’s time
spread (up to 100 steps), N = 10,000: wall 1.04× the twin’s time
spread (up to 100 steps), N = 100,000: wall 1.02× the twin’s time
spread (up to 100 steps), N = 1,000,000: wall 1.01× the twin’s time
spread (up to 1000 steps), N = 1,000: wall 1.01× the twin’s time
spread (up to 1000 steps), N = 10,000: wall 1× the twin’s time
spread (up to 1000 steps), N = 100,000: wall 1.01× the twin’s time
spread (up to 1000 steps), N = 1,000,000: wall 1× the twin’s time
uniform (up to 1000 steps), N = 1,000: wall 1× the twin’s time
uniform (up to 1000 steps), N = 10,000: wall 1.01× the twin’s time
uniform (up to 1000 steps), N = 100,000: wall 0.989× the twin’s time
uniform (up to 1000 steps), N = 1,000,000: wall 1× the twin’s time
device |
compute capability / CPU model |
date |
|---|---|---|
NVIDIA H100 80GB HBM3 |
9.0 |
2026-10-13 |
NVIDIA L4 |
8.9 |
2026-10-14 |
NVIDIA A100-SXM4-40GB |
8.0 |
2026-10-13 |
Tesla T4 |
7.5 |
2026-10-13 |
Quadro P2000 |
6.1 |
2026-10-13 |
NVIDIA RTX PRO 6000 Blackwell Server Edition |
12.0 |
2026-10-14 |
CPU#
Run on: host CPU AMD EPYC 9B45; compiler x86_64-conda-linux-gnu-g++ (conda-forge gcc 14.4.0-7) 14.4.0; Python 3.12.14.
N = 1,000,000
Log scale, from 8.98 ms to 2.62 s.
N = 1,000,000
Log scale, from 88.7 ms to 24.4 s.
N = 1,000,000
Log scale, from 117 ms to 21.4 s.
Table of every arm and every N
CPU: AMD EPYC 9B45 (8 cores, 16 logical CPUs, 8 float64 lanes per vector instruction). Peak FP64: – FLOP/s (8 cores × 2 FMA units × 8 float64 lanes × 2 FLOP per FMA × an unknown clock (the model name, lscpu and cpufreq name none), so no peak). Without SIMD the same cores peak at – FLOP/s. Medians of 5 interleaved repetitions; IQR in parentheses.
Same workload as the GPU card (damped oscillators, classical RK4, float64, every sample stops at its own step). “8 threads” uses all 16 logical CPUs (8 cores × 2 hardware threads). CPU busy = process CPU seconds / wall (all threads and worker processes; spin-waiting counts). Compile times are not in any wall; they have their own table below. 0.2 s pause before every timed repetition, so the previous arm’s thread pools stop spin-waiting first, then 0.05 s of untimed runs of the arm itself, so the timed block starts on a ramped clock (the machine’s busy cores during the pause and the clock at the timer start are recorded per row); a repetition whose run is shorter than 0.1 s repeats it back to back until the repetition spans at least 0.1 s.
The arms run in 3 groups, each interleaved on its own: (eagle_term8, eagle_term1, eagle_compact8, eagle_compact1, eagle_reorder8, eagle_reorder1, python_loops); (eagle_term8, torch_masked, mp_numpy); (eagle_term8, numpy_masked, numba_prange, jax_shard8, jax_vmap). eagle at 8 threads runs in every group as the common reference; its row below is from the first group. Its median wall differed between groups by at most 0.59 % in any cell (spread S=1000, N=1,000,000).
SIMD. eagle’s host build of the step (hawk host profile native-vector-math: -O3 -ffp-contract=fast -march=native -mprefer-vector-width=512 -fno-trapping-math -DAETHER_HOST_VECTOR_MATH -std=c++23 -shared -fPIC -DAETHER_CPP_MODE) is vectorised: its entry function holds scalar float64 instructions (vaddsd ×6, vfmadd132sd ×11, vfmadd231sd ×1, vfnmsub132sd ×3, vfnmsub231sd ×1, vmulsd ×12) and packed ones (vaddpd ×4, vfmadd132pd ×19, vfmadd213pd ×2, vfmadd231pd ×3, vfnmadd132pd ×2, vfnmadd231pd ×6, vmulpd ×8). The active-set build of the step (compaction and reorder arms) is not vectorised (packed float64 instructions: none). Numba’s compiled loop uses scalar float64 instructions only (each sample’s steps are a dependent chain). NumPy and PyTorch use their libraries’ own SIMD kernels for each array operation; JAX’s XLA code was not inspected.
Every arm runs each sample to its own stop, and the stop steps here never exceed max_steps (S); the arms that carry a cap enforce it as well: eagle per sample, JAX in its while-loop condition, the masked arms by their outer step count (a batch-level cap).
What each arm needed to get there: JAX, a cap in the while_loop condition; eagle, the kernel source unchanged, with its loop shape, unrolling and host entry chosen by hawk + eagle (per device, per batch).
arm |
loop structure |
threading |
|---|---|---|
eagle host, termination loop, up to 8 threads |
per sample: eagle.until_done, one host-team pass over dynamic tiles of samples, each sample stepped to its own stop |
OMP_NUM_THREADS=8 (eagle’s OpenMP host team: up to 8 threads; eagle uses one per physical core for a run under eagle._host_loop.SMT_MIN_WORK sample-steps) |
eagle host, termination loop, 1 thread |
per sample: eagle.until_done, one host-team pass over dynamic tiles of samples, each sample stepped to its own stop |
OMP_NUM_THREADS=1 (eagle’s OpenMP host team) |
eagle host, termination + compaction, up to 8 threads |
as the termination loop, the step launched over the active-set map only (recomputed every 16 steps) |
OMP_NUM_THREADS=8 (eagle’s OpenMP host team: up to 8 threads; eagle uses one per physical core for a run under eagle._host_loop.SMT_MIN_WORK sample-steps) |
eagle host, termination + compaction, 1 thread |
as the termination loop, the step launched over the active-set map only (recomputed every 16 steps) |
OMP_NUM_THREADS=1 (eagle’s OpenMP host team) |
eagle host, termination + compaction + reorder, up to 8 threads |
as the compaction loop, plus a reorder of the planes when the live samples spread thin, restored at the end |
OMP_NUM_THREADS=8 (eagle’s OpenMP host team: up to 8 threads; eagle uses one per physical core for a run under eagle._host_loop.SMT_MIN_WORK sample-steps) |
eagle host, termination + compaction + reorder, 1 thread |
as the compaction loop, plus a reorder of the planes when the live samples spread thin, restored at the end |
OMP_NUM_THREADS=1 (eagle’s OpenMP host team) |
Numba prange, 8 threads |
per sample: prange over samples, each runs its own steps to its stop step |
NUMBA_NUM_THREADS=8 and numba.set_num_threads(8) |
JAX sharded over 8 CPU devices |
per step over a shard: 8 shards, each a vmapped while loop running until the shard’s last sample stops or the step cap is reached |
XLA_FLAGS=–xla_force_host_platform_device_count=8, one shard per device |
JAX jit + vmap(while_loop), 1 device |
per step over the batch: the vmapped while loop runs until the last sample stops or the step cap is reached, finished samples held by a select |
XLA’s CPU client defaults, one device (CPU busy column shows the use) |
PyTorch masked, 8 threads |
per step over the batch: whole-batch tensor expressions, finished samples held by torch.where |
torch.set_num_threads(8), OMP_NUM_THREADS=8 |
NumPy masked |
per step over the batch: whole-batch array expressions, finished samples held by np.where |
single-threaded ufuncs; BLAS/OpenMP pools pinned to 1 |
multiprocessing, 8 x NumPy |
per step over a chunk: 8 processes, each the NumPy masked loop over its chunk, stopping when its chunk is done |
multiprocessing.Pool(8), fork; every process pinned to 1 thread |
plain Python loops |
per sample: a Python loop over samples, each runs its own steps to its stop step |
one interpreter thread |
Spread stop steps (log-uniform in [1, 100]), up to 100 steps
N |
arm |
threads |
wall (IQR) |
sample·steps/s |
useful FLOP/s (% peak) |
CPU busy |
peak memory (× minimum) |
|---|---|---|---|---|---|---|---|
1,000 |
eagle host, termination loop, up to 8 threads |
8 |
27.2 µs (59.7 ns) |
8.10e8 |
3.56e10 (–) |
8.2 |
–* |
1,000 |
eagle host, termination loop, 1 thread |
1 |
117 µs (46.3 ns) |
1.89e8 |
8.32e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
233 µs (4.43 µs) |
9.46e7 |
4.16e9 (–) |
8 |
–* |
1,000 |
eagle host, termination + compaction, 1 thread |
1 |
205 µs (1.68 µs) |
1.08e8 |
4.73e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
249 µs (2 µs) |
8.85e7 |
3.89e9 (–) |
8 |
–* |
1,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
228 µs (499 ns) |
9.68e7 |
4.26e9 (–) |
1 |
–* |
1,000 |
Numba prange, 8 threads |
8 |
39 µs (59.4 ns) |
5.65e8 |
2.49e10 (–) |
8.1 |
–* |
1,000 |
JAX sharded over 8 CPU devices |
8 |
85.2 µs (45 ns) |
2.59e8 |
1.14e10 (–) |
5.8 |
–* |
1,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
185 µs (756 ns) |
1.19e8 |
5.23e9 (–) |
1.1 |
–* |
1,000 |
PyTorch masked, 8 threads |
8 |
4.5 ms (9.06 µs) |
4.90e6 |
2.16e8 (–) |
1 |
–* |
1,000 |
NumPy masked |
1 |
2.3 ms (13.9 µs) |
9.59e6 |
4.22e8 (–) |
1 |
–* |
1,000 |
multiprocessing, 8 x NumPy |
8 |
1.84 ms (8.38 µs) |
1.20e7 |
5.28e8 (–) |
6.9 |
–* |
1,000 |
plain Python loops |
1 |
3.67 ms (5.04 µs) |
6.00e6 |
2.64e8 (–) |
1 |
–* |
10,000 |
eagle host, termination loop, up to 8 threads |
8 |
159 µs (156 ns) |
1.44e9 |
6.31e10 (–) |
8.2 |
–* |
10,000 |
eagle host, termination loop, 1 thread |
1 |
1.11 ms (2.7 µs) |
2.06e8 |
9.07e9 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
484 µs (2.07 µs) |
4.72e8 |
2.08e10 (–) |
8.1 |
–* |
10,000 |
eagle host, termination + compaction, 1 thread |
1 |
1.48 ms (2.4 µs) |
1.54e8 |
6.79e9 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
811 µs (5.76 µs) |
2.82e8 |
1.24e10 (–) |
8 |
–* |
10,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
1.84 ms (3.72 µs) |
1.25e8 |
5.48e9 (–) |
1 |
–* |
10,000 |
Numba prange, 8 threads |
8 |
338 µs (671 ns) |
6.76e8 |
2.98e10 (–) |
8 |
–* |
10,000 |
JAX sharded over 8 CPU devices |
8 |
450 µs (4.22 µs) |
5.08e8 |
2.23e10 (–) |
6.1 |
–* |
10,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
1.59 ms (5.1 µs) |
1.44e8 |
6.33e9 (–) |
2 |
–* |
10,000 |
PyTorch masked, 8 threads |
8 |
12.5 ms (57.3 µs) |
1.83e7 |
8.07e8 (–) |
1 |
–* |
10,000 |
NumPy masked |
1 |
9.2 ms (54.9 µs) |
2.49e7 |
1.09e9 (–) |
1 |
–* |
10,000 |
multiprocessing, 8 x NumPy |
8 |
3.07 ms (10.1 µs) |
7.45e7 |
3.28e9 (–) |
7 |
–* |
10,000 |
plain Python loops |
1 |
38.7 ms (1.57 ms) |
5.91e6 |
2.60e8 (–) |
1 |
–* |
100,000 |
eagle host, termination loop, up to 8 threads |
8 |
1.44 ms (12.5 µs) |
1.59e9 |
7.00e10 (–) |
8 |
–* |
100,000 |
eagle host, termination loop, 1 thread |
1 |
11.3 ms (16.2 µs) |
2.03e8 |
8.92e9 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
2.96 ms (4.14 µs) |
7.74e8 |
3.40e10 (–) |
8 |
–* |
100,000 |
eagle host, termination + compaction, 1 thread |
1 |
15.3 ms (38.5 µs) |
1.49e8 |
6.57e9 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
7.04 ms (27 µs) |
3.25e8 |
1.43e10 (–) |
8 |
–* |
100,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
19.1 ms (66.5 µs) |
1.20e8 |
5.26e9 (–) |
1 |
–* |
100,000 |
Numba prange, 8 threads |
8 |
3.27 ms (4.29 µs) |
7.01e8 |
3.08e10 (–) |
8 |
–* |
100,000 |
JAX sharded over 8 CPU devices |
8 |
4.1 ms (35.6 µs) |
5.58e8 |
2.46e10 (–) |
11 |
–* |
100,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
13.2 ms (12.9 µs) |
1.73e8 |
7.63e9 (–) |
2.1 |
–* |
100,000 |
PyTorch masked, 8 threads |
8 |
41.2 ms (3.85 ms) |
5.56e7 |
2.45e9 (–) |
8 |
–* |
100,000 |
NumPy masked |
1 |
123 ms (1.59 ms) |
1.86e7 |
8.18e8 (–) |
1 |
–* |
100,000 |
multiprocessing, 8 x NumPy |
8 |
17.8 ms (7.32 ms) |
1.29e8 |
5.66e9 (–) |
6 |
–* |
1,000,000 |
eagle host, termination loop, up to 8 threads |
8 |
14.4 ms (24 µs) |
1.60e9 |
7.05e10 (–) |
8 |
61.7 MiB (1.8×) |
1,000,000 |
eagle host, termination loop, 1 thread |
1 |
114 ms (138 µs) |
2.02e8 |
8.90e9 (–) |
1 |
61.7 MiB (1.8×) |
1,000,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
38.4 ms (225 µs) |
5.98e8 |
2.63e10 (–) |
8 |
65.4 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction, 1 thread |
1 |
164 ms (2.22 ms) |
1.40e8 |
6.17e9 (–) |
1 |
65.5 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
74.6 ms (710 µs) |
3.08e8 |
1.36e10 (–) |
5.2 |
94.6 MiB (2.76×) |
1,000,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
196 ms (2.47 ms) |
1.17e8 |
5.17e9 (–) |
1 |
94.7 MiB (2.76×) |
1,000,000 |
Numba prange, 8 threads |
8 |
32.4 ms (21.2 µs) |
7.09e8 |
3.12e10 (–) |
8 |
68.3 MiB (1.99×) |
1,000,000 |
JAX sharded over 8 CPU devices |
8 |
194 ms (8.01 ms) |
1.19e8 |
5.23e9 (–) |
14 |
198 MiB (5.76×) |
1,000,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
296 ms (10.8 ms) |
7.77e7 |
3.42e9 (–) |
3.7 |
152 MiB (4.41×) |
1,000,000 |
PyTorch masked, 8 threads |
8 |
1.06 s (30.4 ms) |
2.17e7 |
9.56e8 (–) |
8 |
267 MiB (7.79×) |
1,000,000 |
NumPy masked |
1 |
1.64 s (10.9 ms) |
1.41e7 |
6.19e8 (–) |
1 |
169 MiB (4.92×) |
1,000,000 |
multiprocessing, 8 x NumPy |
8 |
782 ms (15.5 ms) |
2.94e7 |
1.29e9 (–) |
7.7 |
234 MiB (6.82×) |
Fastest arm by median wall time:
N = 1,000: eagle host, termination loop, up to 8 threads (27.2 µs), then Numba prange, 8 threads (39 µs, 1.43× the time)
N = 10,000: eagle host, termination loop, up to 8 threads (159 µs), then Numba prange, 8 threads (338 µs, 2.12× the time)
N = 100,000: eagle host, termination loop, up to 8 threads (1.44 ms), then eagle host, termination + compaction, up to 8 threads (2.96 ms, 2.06× the time)
N = 1,000,000: eagle host, termination loop, up to 8 threads (14.4 ms), then Numba prange, 8 threads (32.4 ms, 2.26× the time)
Spread stop steps (log-uniform in [10, 1000]), up to 1000 steps
N |
arm |
threads |
wall (IQR) |
sample·steps/s |
useful FLOP/s (% peak) |
CPU busy |
peak memory (× minimum) |
|---|---|---|---|---|---|---|---|
1,000 |
eagle host, termination loop, up to 8 threads |
8 |
167 µs (685 ns) |
1.26e9 |
5.52e10 (–) |
8.1 |
–* |
1,000 |
eagle host, termination loop, 1 thread |
1 |
1.07 ms (1.96 µs) |
1.96e8 |
8.64e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
2.28 ms (29 µs) |
9.21e7 |
4.05e9 (–) |
8 |
–* |
1,000 |
eagle host, termination + compaction, 1 thread |
1 |
1.93 ms (3.02 µs) |
1.09e8 |
4.79e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
2.45 ms (46.5 µs) |
8.58e7 |
3.77e9 (–) |
8 |
–* |
1,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
2.11 ms (8.59 µs) |
9.94e7 |
4.37e9 (–) |
1 |
–* |
1,000 |
Numba prange, 8 threads |
8 |
368 µs (122 ns) |
5.71e8 |
2.51e10 (–) |
8 |
–* |
1,000 |
JAX sharded over 8 CPU devices |
8 |
491 µs (3.59 µs) |
4.28e8 |
1.88e10 (–) |
6.1 |
–* |
1,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
1.75 ms (8.57 µs) |
1.20e8 |
5.28e9 (–) |
1 |
–* |
1,000 |
PyTorch masked, 8 threads |
8 |
44.7 ms (33.4 µs) |
4.70e6 |
2.07e8 (–) |
1 |
–* |
1,000 |
NumPy masked |
1 |
23.2 ms (264 µs) |
9.04e6 |
3.98e8 (–) |
1 |
–* |
1,000 |
multiprocessing, 8 x NumPy |
8 |
15.6 ms (5.18 ms) |
1.35e7 |
5.94e8 (–) |
7.6 |
–* |
1,000 |
plain Python loops |
1 |
34.9 ms (43.5 µs) |
6.02e6 |
2.65e8 (–) |
1 |
–* |
10,000 |
eagle host, termination loop, up to 8 threads |
8 |
1.5 ms (5.27 µs) |
1.45e9 |
6.38e10 (–) |
8 |
–* |
10,000 |
eagle host, termination loop, 1 thread |
1 |
11.5 ms (26.4 µs) |
1.89e8 |
8.34e9 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
3.88 ms (28.6 µs) |
5.62e8 |
2.47e10 (–) |
8 |
–* |
10,000 |
eagle host, termination + compaction, 1 thread |
1 |
9.27 ms (36 µs) |
2.35e8 |
1.04e10 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
4.47 ms (29.8 µs) |
4.88e8 |
2.15e10 (–) |
8 |
–* |
10,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
9.92 ms (17.7 µs) |
2.20e8 |
9.68e9 (–) |
1 |
–* |
10,000 |
Numba prange, 8 threads |
8 |
3.46 ms (6.29 µs) |
6.31e8 |
2.78e10 (–) |
8 |
–* |
10,000 |
JAX sharded over 8 CPU devices |
8 |
4.08 ms (168 µs) |
5.35e8 |
2.36e10 (–) |
6.3 |
–* |
10,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
16.7 ms (291 µs) |
1.31e8 |
5.75e9 (–) |
2 |
–* |
10,000 |
PyTorch masked, 8 threads |
8 |
117 ms (1.13 ms) |
1.87e7 |
8.21e8 (–) |
1 |
–* |
10,000 |
NumPy masked |
1 |
83.8 ms (484 µs) |
2.61e7 |
1.15e9 (–) |
1 |
–* |
10,000 |
multiprocessing, 8 x NumPy |
8 |
38.9 ms (1.58 ms) |
5.62e7 |
2.47e9 (–) |
5.8 |
–* |
10,000 |
plain Python loops |
1 |
362 ms (470 µs) |
6.03e6 |
2.65e8 (–) |
1 |
–* |
100,000 |
eagle host, termination loop, up to 8 threads |
8 |
14.3 ms (26.8 µs) |
1.51e9 |
6.65e10 (–) |
8 |
–* |
100,000 |
eagle host, termination loop, 1 thread |
1 |
113 ms (162 µs) |
1.92e8 |
8.44e9 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
19.1 ms (49.7 µs) |
1.13e9 |
4.98e10 (–) |
8 |
–* |
100,000 |
eagle host, termination + compaction, 1 thread |
1 |
91.8 ms (209 µs) |
2.36e8 |
1.04e10 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
22.9 ms (58.5 µs) |
9.46e8 |
4.16e10 (–) |
8 |
–* |
100,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
87.3 ms (85 µs) |
2.48e8 |
1.09e10 (–) |
1 |
–* |
100,000 |
Numba prange, 8 threads |
8 |
33.2 ms (9.04 µs) |
6.52e8 |
2.87e10 (–) |
8 |
–* |
100,000 |
JAX sharded over 8 CPU devices |
8 |
40.5 ms (44 µs) |
5.35e8 |
2.35e10 (–) |
11 |
–* |
100,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
132 ms (1.58 ms) |
1.64e8 |
7.21e9 (–) |
2.1 |
–* |
100,000 |
PyTorch masked, 8 threads |
8 |
314 ms (1.98 ms) |
6.90e7 |
3.04e9 (–) |
8 |
–* |
100,000 |
NumPy masked |
1 |
1 s (1.01 ms) |
2.16e7 |
9.49e8 (–) |
1 |
–* |
100,000 |
multiprocessing, 8 x NumPy |
8 |
134 ms (16 ms) |
1.62e8 |
7.13e9 (–) |
6.8 |
–* |
1,000,000 |
eagle host, termination loop, up to 8 threads |
8 |
142 ms (360 µs) |
1.52e9 |
6.71e10 (–) |
8 |
61.5 MiB (1.79×) |
1,000,000 |
eagle host, termination loop, 1 thread |
1 |
1.13 s (742 µs) |
1.92e8 |
8.44e9 (–) |
1 |
61.7 MiB (1.8×) |
1,000,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
292 ms (2.78 ms) |
7.41e8 |
3.26e10 (–) |
8 |
65.5 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction, 1 thread |
1 |
1.03 s (4.1 ms) |
2.09e8 |
9.21e9 (–) |
1 |
65.6 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
242 ms (3.67 ms) |
8.94e8 |
3.93e10 (–) |
7 |
94.6 MiB (2.75×) |
1,000,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
877 ms (193 µs) |
2.47e8 |
1.09e10 (–) |
1 |
94.7 MiB (2.76×) |
1,000,000 |
Numba prange, 8 threads |
8 |
328 ms (162 µs) |
6.59e8 |
2.90e10 (–) |
8 |
68.4 MiB (1.99×) |
1,000,000 |
JAX sharded over 8 CPU devices |
8 |
1.86 s (39.8 ms) |
1.16e8 |
5.10e9 (–) |
14 |
198 MiB (5.78×) |
1,000,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
2.74 s (16.9 ms) |
7.90e7 |
3.48e9 (–) |
3.7 |
174 MiB (5.08×) |
1,000,000 |
PyTorch masked, 8 threads |
8 |
10.2 s (84.7 ms) |
2.11e7 |
9.29e8 (–) |
8 |
275 MiB (8.01×) |
1,000,000 |
NumPy masked |
1 |
15.3 s (73.6 ms) |
1.42e7 |
6.24e8 (–) |
1 |
169 MiB (4.92×) |
1,000,000 |
multiprocessing, 8 x NumPy |
8 |
7.72 s (33.5 ms) |
2.80e7 |
1.23e9 (–) |
7.9 |
234 MiB (6.82×) |
Fastest arm by median wall time:
N = 1,000: eagle host, termination loop, up to 8 threads (167 µs), then Numba prange, 8 threads (368 µs, 2.2× the time)
N = 10,000: eagle host, termination loop, up to 8 threads (1.5 ms), then Numba prange, 8 threads (3.46 ms, 2.3× the time)
N = 100,000: eagle host, termination loop, up to 8 threads (14.3 ms), then eagle host, termination + compaction, up to 8 threads (19.1 ms, 1.34× the time)
N = 1,000,000: eagle host, termination loop, up to 8 threads (142 ms), then eagle host, termination + compaction + reorder, up to 8 threads (242 ms, 1.71× the time)
Uniform stop step: every sample runs 1000 steps
N |
arm |
threads |
wall (IQR) |
sample·steps/s |
useful FLOP/s (% peak) |
CPU busy |
peak memory (× minimum) |
|---|---|---|---|---|---|---|---|
1,000 |
eagle host, termination loop, up to 8 threads |
8 |
209 µs (68.1 ns) |
4.77e9 |
2.10e11 (–) |
8.1 |
–* |
1,000 |
eagle host, termination loop, 1 thread |
1 |
1.52 ms (320 ns) |
6.60e8 |
2.90e10 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
2.52 ms (62.7 µs) |
3.96e8 |
1.74e10 (–) |
8 |
–* |
1,000 |
eagle host, termination + compaction, 1 thread |
1 |
4.51 ms (21.9 µs) |
2.22e8 |
9.75e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
2.53 ms (40.7 µs) |
3.95e8 |
1.74e10 (–) |
8 |
–* |
1,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
4.51 ms (24.6 µs) |
2.22e8 |
9.76e9 (–) |
1 |
–* |
1,000 |
Numba prange, 8 threads |
8 |
1.53 ms (1.16 µs) |
6.54e8 |
2.88e10 (–) |
8 |
–* |
1,000 |
JAX sharded over 8 CPU devices |
8 |
492 µs (2.28 µs) |
2.03e9 |
8.95e10 (–) |
6.2 |
–* |
1,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
1.74 ms (7.7 µs) |
5.73e8 |
2.52e10 (–) |
1 |
–* |
1,000 |
PyTorch masked, 8 threads |
8 |
44.4 ms (89.5 µs) |
2.25e7 |
9.92e8 (–) |
1 |
–* |
1,000 |
NumPy masked |
1 |
22.9 ms (217 µs) |
4.37e7 |
1.92e9 (–) |
1 |
–* |
1,000 |
multiprocessing, 8 x NumPy |
8 |
15.4 ms (54.4 µs) |
6.48e7 |
2.85e9 (–) |
7.8 |
–* |
1,000 |
plain Python loops |
1 |
166 ms (110 µs) |
6.03e6 |
2.65e8 (–) |
1 |
–* |
10,000 |
eagle host, termination loop, up to 8 threads |
8 |
1.94 ms (2.55 µs) |
5.16e9 |
2.27e11 (–) |
8 |
–* |
10,000 |
eagle host, termination loop, 1 thread |
1 |
14.9 ms (9.47 µs) |
6.69e8 |
2.95e10 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
6.91 ms (19.6 µs) |
1.45e9 |
6.37e10 (–) |
8 |
–* |
10,000 |
eagle host, termination + compaction, 1 thread |
1 |
35.8 ms (30 µs) |
2.80e8 |
1.23e10 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
6.93 ms (81.4 µs) |
1.44e9 |
6.35e10 (–) |
8 |
–* |
10,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
35.8 ms (102 µs) |
2.79e8 |
1.23e10 (–) |
1 |
–* |
10,000 |
Numba prange, 8 threads |
8 |
15.3 ms (8.19 µs) |
6.55e8 |
2.88e10 (–) |
8 |
–* |
10,000 |
JAX sharded over 8 CPU devices |
8 |
4.03 ms (63.1 µs) |
2.48e9 |
1.09e11 (–) |
6.5 |
–* |
10,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
15.2 ms (152 µs) |
6.60e8 |
2.90e10 (–) |
2 |
–* |
10,000 |
PyTorch masked, 8 threads |
8 |
114 ms (488 µs) |
8.81e7 |
3.88e9 (–) |
1 |
–* |
10,000 |
NumPy masked |
1 |
81.8 ms (147 µs) |
1.22e8 |
5.38e9 (–) |
1 |
–* |
10,000 |
multiprocessing, 8 x NumPy |
8 |
25.9 ms (10.9 µs) |
3.86e8 |
1.70e10 (–) |
7.7 |
–* |
100,000 |
eagle host, termination loop, up to 8 threads |
8 |
18.8 ms (146 µs) |
5.31e9 |
2.34e11 (–) |
8 |
–* |
100,000 |
eagle host, termination loop, 1 thread |
1 |
149 ms (359 µs) |
6.70e8 |
2.95e10 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
52.8 ms (73.9 µs) |
1.89e9 |
8.34e10 (–) |
8 |
–* |
100,000 |
eagle host, termination + compaction, 1 thread |
1 |
354 ms (410 µs) |
2.82e8 |
1.24e10 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
53 ms (178 µs) |
1.89e9 |
8.30e10 (–) |
8 |
–* |
100,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
356 ms (2.87 ms) |
2.81e8 |
1.24e10 (–) |
1 |
–* |
100,000 |
Numba prange, 8 threads |
8 |
153 ms (68.1 µs) |
6.55e8 |
2.88e10 (–) |
8 |
–* |
100,000 |
JAX sharded over 8 CPU devices |
8 |
39.9 ms (411 µs) |
2.50e9 |
1.10e11 (–) |
11 |
–* |
100,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
133 ms (1.57 ms) |
7.51e8 |
3.30e10 (–) |
2.1 |
–* |
100,000 |
PyTorch masked, 8 threads |
8 |
300 ms (4.26 ms) |
3.33e8 |
1.47e10 (–) |
8 |
–* |
100,000 |
NumPy masked |
1 |
792 ms (1.47 ms) |
1.26e8 |
5.56e9 (–) |
1 |
–* |
100,000 |
multiprocessing, 8 x NumPy |
8 |
145 ms (15.2 ms) |
6.92e8 |
3.04e10 (–) |
6.4 |
–* |
1,000,000 |
eagle host, termination loop, up to 8 threads |
8 |
187 ms (618 µs) |
5.34e9 |
2.35e11 (–) |
8 |
61.9 MiB (1.8×) |
1,000,000 |
eagle host, termination loop, 1 thread |
1 |
1.49 s (10 ms) |
6.71e8 |
2.95e10 (–) |
1 |
61.8 MiB (1.8×) |
1,000,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
744 ms (4.38 ms) |
1.34e9 |
5.92e10 (–) |
8 |
65.4 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction, 1 thread |
1 |
3.56 s (2.33 ms) |
2.81e8 |
1.24e10 (–) |
1 |
65.5 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
761 ms (4.3 ms) |
1.31e9 |
5.78e10 (–) |
8 |
73.2 MiB (2.13×) |
1,000,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
3.58 s (14.9 ms) |
2.79e8 |
1.23e10 (–) |
1 |
73.1 MiB (2.13×) |
1,000,000 |
Numba prange, 8 threads |
8 |
1.53 s (878 µs) |
6.55e8 |
2.88e10 (–) |
8 |
60.8 MiB (1.77×) |
1,000,000 |
JAX sharded over 8 CPU devices |
8 |
1.94 s (41.5 ms) |
5.14e8 |
2.26e10 (–) |
14 |
188 MiB (5.46×) |
1,000,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
2.88 s (26.7 ms) |
3.47e8 |
1.53e10 (–) |
3.7 |
167 MiB (4.86×) |
1,000,000 |
PyTorch masked, 8 threads |
8 |
10.2 s (103 ms) |
9.80e7 |
4.31e9 (–) |
8 |
260 MiB (7.57×) |
1,000,000 |
NumPy masked |
1 |
13.4 s (207 ms) |
7.48e7 |
3.29e9 (–) |
1 |
162 MiB (4.73×) |
1,000,000 |
multiprocessing, 8 x NumPy |
8 |
7.65 s (14.1 ms) |
1.31e8 |
5.75e9 (–) |
7.9 |
223 MiB (6.51×) |
Fastest arm by median wall time:
N = 1,000: eagle host, termination loop, up to 8 threads (209 µs), then JAX sharded over 8 CPU devices (492 µs, 2.35× the time)
N = 10,000: eagle host, termination loop, up to 8 threads (1.94 ms), then JAX sharded over 8 CPU devices (4.03 ms, 2.08× the time)
N = 100,000: eagle host, termination loop, up to 8 threads (18.8 ms), then JAX sharded over 8 CPU devices (39.9 ms, 2.12× the time)
N = 1,000,000: eagle host, termination loop, up to 8 threads (187 ms), then eagle host, termination + compaction, up to 8 threads (744 ms, 3.97× the time)
Cells not run
plain Python loops: plain Python runs only at N <= 10,000 with at most 2,500,000 sample-steps per run (spread S=100 N=100,000; spread S=100 N=1,000,000; spread S=1000 N=100,000; spread S=1000 N=1,000,000; uniform S=1000 N=10,000; uniform S=1000 N=100,000; uniform S=1000 N=1,000,000).
Where each tool fits
Each tool below is written the way its users write it. Two loop structures appear: per step over the batch (eagle’s termination loops, JAX, PyTorch, NumPy, multiprocessing) and per sample over its steps (Numba, plain Python). The per-sample structure touches each sample’s state once for all its steps; the per-step structure streams the batch through memory every step, which is the structure the GPU card’s kernels use. Ratios below are wall-time ratios at the largest N each arm ran (below 1 = less time than eagle at 8 threads).
eagle host, termination loop, up to 8 threads. The GPU card’s eagle graph arm, unchanged, on the host: the same hawk step kernel, eagle.deploy and eagle.until_done call, run by eagle’s OpenMP host team; one code base for GPUs and CPUs. On this card: spread S=100, N=1,000,000: 14.4 ms; spread S=1000, N=1,000,000: 142 ms; uniform S=1000, N=1,000,000: 187 ms. Fastest in 12 of 12 cells.
eagle host, termination loop, 1 thread. The same loop on one thread: the difference to the 8-thread row is the threading alone. On this card: spread S=100, N=1,000,000: 114 ms (7.91× eagle 8-thread time); spread S=1000, N=1,000,000: 1.13 s (7.95× eagle 8-thread time); uniform S=1000, N=1,000,000: 1.49 s (7.97× eagle 8-thread time). Fastest in 0 of 12 cells.
eagle host, termination + compaction, up to 8 threads. The GPU card’s compaction arm on the host: the launch covers only the samples still running, the map rebuilt every 16 steps when a sample finished. On this card: spread S=100, N=1,000,000: 38.4 ms (2.68× eagle 8-thread time); spread S=1000, N=1,000,000: 292 ms (2.06× eagle 8-thread time); uniform S=1000, N=1,000,000: 744 ms (3.97× eagle 8-thread time). Fastest in 0 of 12 cells.
eagle host, termination + compaction, 1 thread. The compaction loop on one thread: the difference to the 8-thread row is the threading alone. On this card: spread S=100, N=1,000,000: 164 ms (11.4× eagle 8-thread time); spread S=1000, N=1,000,000: 1.03 s (7.28× eagle 8-thread time); uniform S=1000, N=1,000,000: 3.56 s (19× eagle 8-thread time). Fastest in 0 of 12 cells.
eagle host, termination + compaction + reorder, up to 8 threads. The GPU card’s reorder arm on the host: the live samples moved to the front of every plane when they spread thin, restored to sample order at the end (inside the wall). On this card: spread S=100, N=1,000,000: 74.6 ms (5.2× eagle 8-thread time); spread S=1000, N=1,000,000: 242 ms (1.71× eagle 8-thread time); uniform S=1000, N=1,000,000: 761 ms (4.07× eagle 8-thread time). Fastest in 0 of 12 cells.
eagle host, termination + compaction + reorder, 1 thread. The reorder loop on one thread: the difference to the 8-thread row is the threading alone. On this card: spread S=100, N=1,000,000: 196 ms (13.6× eagle 8-thread time); spread S=1000, N=1,000,000: 877 ms (6.18× eagle 8-thread time); uniform S=1000, N=1,000,000: 3.58 s (19.1× eagle 8-thread time). Fastest in 0 of 12 cells.
Numba prange, 8 threads. Compiles a plain Python loop to machine code with no separate language; each sample keeps its state in registers across all of its steps and stops on its own, so finished samples cost nothing. On this card: spread S=100, N=1,000,000: 32.4 ms (2.26× eagle 8-thread time); spread S=1000, N=1,000,000: 328 ms (2.31× eagle 8-thread time); uniform S=1000, N=1,000,000: 1.53 s (8.16× eagle 8-thread time). Fastest in 0 of 12 cells.
JAX sharded over 8 CPU devices. The JAX arm spread over 8 CPU devices: the batch is sharded and each shard runs its own vmapped while loop, with no cross-shard step synchronisation. On this card: spread S=100, N=1,000,000: 194 ms (13.5× eagle 8-thread time); spread S=1000, N=1,000,000: 1.86 s (13.1× eagle 8-thread time); uniform S=1000, N=1,000,000: 1.94 s (10.4× eagle 8-thread time). Fastest in 0 of 12 cells.
JAX jit + vmap(while_loop), 1 device. Compiles the whole loop once and batches a per-sample while loop with vmap; the same code runs on GPUs and differentiates with jax.grad. On this card: spread S=100, N=1,000,000: 296 ms (20.6× eagle 8-thread time); spread S=1000, N=1,000,000: 2.74 s (19.3× eagle 8-thread time); uniform S=1000, N=1,000,000: 2.88 s (15.4× eagle 8-thread time). Fastest in 0 of 12 cells.
PyTorch masked, 8 threads. Runs the masked array version on a multi-threaded tensor library; it fits where the computation already lives next to a PyTorch model or needs autograd. On this card: spread S=100, N=1,000,000: 1.06 s (73.7× eagle 8-thread time); spread S=1000, N=1,000,000: 10.2 s (72.2× eagle 8-thread time); uniform S=1000, N=1,000,000: 10.2 s (54.5× eagle 8-thread time). Fastest in 0 of 12 cells.
NumPy masked. No compile step and no dependency beyond NumPy; masked array code computes every sample every step, so it fits dense batches where all samples run to the end (the uniform configuration). On this card: spread S=100, N=1,000,000: 1.64 s (114× eagle 8-thread time); spread S=1000, N=1,000,000: 15.3 s (107× eagle 8-thread time); uniform S=1000, N=1,000,000: 13.4 s (71.4× eagle 8-thread time). Fastest in 0 of 12 cells.
multiprocessing, 8 x NumPy. Spreads NumPy over processes with the standard library alone; each chunk stops once its own samples are done. On this card: spread S=100, N=1,000,000: 782 ms (54.5× eagle 8-thread time); spread S=1000, N=1,000,000: 7.72 s (54.4× eagle 8-thread time); uniform S=1000, N=1,000,000: 7.65 s (40.9× eagle 8-thread time). Fastest in 0 of 12 cells.
plain Python loops. Nothing to install and each line can be stepped in a debugger; run only at small N here. On this card: spread S=100, N=10,000: 38.7 ms (243× eagle 8-thread time); spread S=1000, N=10,000: 362 ms (241× eagle 8-thread time); uniform S=1000, N=1,000: 166 ms (792× eagle 8-thread time). Fastest in 0 of 12 cells.
Compile time
Its own pass, never part of any wall. Each measurement is a fresh process: the imports run untimed, then the timer spans the first call to a ready kernel (trace + code generation + compile, or the cache lookup). Cold: an empty cache (hawk’s compile cache pointed at a fresh directory through HAWK_CACHE_DIR; Numba with cache=True into a fresh NUMBA_CACHE_DIR; JAX with its persistent compilation cache in a fresh directory). Warm: a new process on the cache the first cold run populated (hawk cache hit; Numba’s on-disk cache; JAX’s persistent cache with the minimum compile time and entry size set to 0). hawk: eagle.deploy of the kernel (tracing, C++ emission, the g++ host build, eagle’s plan); the active-set variant is a second build of the same step. Numba: the first call on a 64-sample batch (compile plus a negligible run). JAX: lower + compile from abstract shapes, once per N (XLA specialises on the shape). Medians of 3 cold and 3 warm processes.
arm |
compiled |
cold (median of 3) |
warm (median of 3) |
|---|---|---|---|
eagle host, termination loop, up to 8 threads |
hawk step kernel (host build) |
667 ms |
20.9 ms |
eagle host, termination + compaction, up to 8 threads |
hawk active-set step kernel (host build) |
565 ms |
17.7 ms |
Numba prange, 8 threads |
Numba parallel loop (first call, 64 samples) |
449 ms |
135 ms |
JAX sharded over 8 CPU devices |
XLA executable for N = 1,000 |
73.9 ms |
27.6 ms |
JAX sharded over 8 CPU devices |
XLA executable for N = 10,000 |
78 ms |
27.6 ms |
JAX sharded over 8 CPU devices |
XLA executable for N = 100,000 |
77.6 ms |
28 ms |
JAX sharded over 8 CPU devices |
XLA executable for N = 1,000,000 |
82.1 ms |
27.6 ms |
JAX jit + vmap(while_loop), 1 device |
XLA executable for N = 1,000 |
69.6 ms |
28.5 ms |
JAX jit + vmap(while_loop), 1 device |
XLA executable for N = 10,000 |
71.6 ms |
28.6 ms |
JAX jit + vmap(while_loop), 1 device |
XLA executable for N = 100,000 |
74.1 ms |
28.9 ms |
JAX jit + vmap(while_loop), 1 device |
XLA executable for N = 1,000,000 |
71.5 ms |
29.1 ms |
No compile step: PyTorch masked, 8 threads, NumPy masked, multiprocessing, 8 x NumPy, plain Python loops.
Same build as another row: eagle host, termination loop, 1 thread (as eagle host, termination loop, up to 8 threads); eagle host, termination + compaction, 1 thread (as eagle host, termination + compaction, up to 8 threads); eagle host, termination + compaction + reorder, up to 8 threads (as eagle host, termination + compaction, up to 8 threads); eagle host, termination + compaction + reorder, 1 thread (as eagle host, termination + compaction, up to 8 threads).
Memory method
A separate pass after the timing pass. Each (arm, N, configuration) runs in a fresh process: imports, then a warm-up run on a 64-sample batch (every JIT compiles here; JAX also compiles for shape N from abstract shapes), then the baseline (resident set size, VmRSS) and a reset of the process’s peak mark (/proc/self/clear_refs, so ru_maxrss restarts from the baseline); then the batch’s inputs are generated and one full run done. Peak memory = ru_maxrss at the end minus the baseline; for multiprocessing, plus each pool process’s own peak above its baseline (shared pages counted once per process). Minimum = the state the workload needs, N × ((2 state + 2 float64 scalars: omega, zeta) × 8 + 4 bytes for the stop step as a 32-bit integer) = 36 bytes per sample; the factor is peak / minimum. Compile overhead = baseline minus the resident size before the arm’s imports-and-warm-up. Below about 1 MiB the allocator reuses memory already resident at the baseline, so small-N peaks can read below the minimum.
Compile/import overhead (resident memory added by the warm-up compile, measured once per arm at the largest N): eagle host, termination loop, up to 8 threads 30.7 MiB; eagle host, termination loop, 1 thread 30.6 MiB; eagle host, termination + compaction, up to 8 threads 30.7 MiB; eagle host, termination + compaction, 1 thread 30.6 MiB; eagle host, termination + compaction + reorder, up to 8 threads 30.8 MiB; eagle host, termination + compaction + reorder, 1 thread 30.6 MiB; Numba prange, 8 threads 192 MiB; JAX sharded over 8 CPU devices 283 MiB; JAX jit + vmap(while_loop), 1 device 280 MiB; PyTorch masked, 8 threads 289 MiB; NumPy masked 23 MiB; multiprocessing, 8 x NumPy 23.2 MiB; plain Python loops 20.9 MiB.
CPU peak memory is shown for N = 1,000,000 only, net of the benchmark harness’s own copies (36 bytes per sample on eagle’s arms). At smaller N this harness cannot resolve it: each measuring process inherits the launching process’s peak, so smaller readings show that floor rather than the workload. The cards keep the raw readings; the next card run measures every size directly.
Run on: host CPU AMD EPYC 7B13; compiler x86_64-conda-linux-gnu-g++ (conda-forge gcc 14.4.0-7) 14.4.0; Python 3.12.14.
N = 1,000,000
Log scale, from 24.9 ms to 3.52 s.
N = 1,000,000
Log scale, from 249 ms to 34.6 s.
N = 1,000,000
Log scale, from 433 ms to 31.4 s.
Table of every arm and every N
CPU: AMD EPYC 7B13 (8 cores, 16 logical CPUs, 4 float64 lanes per vector instruction). Peak FP64: – FLOP/s (8 cores × 2 FMA units × 4 float64 lanes × 2 FLOP per FMA × an unknown clock (the model name, lscpu and cpufreq name none), so no peak). Without SIMD the same cores peak at – FLOP/s. Medians of 5 interleaved repetitions; IQR in parentheses.
Same workload as the GPU card (damped oscillators, classical RK4, float64, every sample stops at its own step). “8 threads” uses all 16 logical CPUs (8 cores × 2 hardware threads). CPU busy = process CPU seconds / wall (all threads and worker processes; spin-waiting counts). Compile times are not in any wall; they have their own table below. 0.2 s pause before every timed repetition, so the previous arm’s thread pools stop spin-waiting first, then 0.05 s of untimed runs of the arm itself, so the timed block starts on a ramped clock (the machine’s busy cores during the pause and the clock at the timer start are recorded per row); a repetition whose run is shorter than 0.1 s repeats it back to back until the repetition spans at least 0.1 s.
The arms run in 3 groups, each interleaved on its own: (eagle_term8, eagle_term1, eagle_compact8, eagle_compact1, eagle_reorder8, eagle_reorder1, python_loops); (eagle_term8, torch_masked, mp_numpy); (eagle_term8, numpy_masked, numba_prange, jax_shard8, jax_vmap). eagle at 8 threads runs in every group as the common reference; its row below is from the first group. Its median wall differed between groups by at most 0.3 % in any cell (spread S=100, N=1,000).
SIMD. eagle’s host build of the step (hawk host profile native-vector-math: -O3 -ffp-contract=fast -march=native -mprefer-vector-width=512 -fno-trapping-math -DAETHER_HOST_VECTOR_MATH -std=c++23 -shared -fPIC -DAETHER_CPP_MODE) is vectorised: its entry function holds scalar float64 instructions (vaddsd ×5, vfmadd132sd ×11, vfmadd231sd ×1, vfnmsub132sd ×2, vfnmsub231sd ×2, vmulsd ×12) and packed ones (vaddpd ×4, vfmadd132pd ×16, vfmadd213pd ×6, vfmadd231pd ×2, vfnmadd132pd ×2, vfnmadd231pd ×6, vmulpd ×8). The active-set build of the step (compaction and reorder arms) is not vectorised (packed float64 instructions: none). Numba’s compiled loop uses scalar float64 instructions only (each sample’s steps are a dependent chain). NumPy and PyTorch use their libraries’ own SIMD kernels for each array operation; JAX’s XLA code was not inspected.
Every arm runs each sample to its own stop, and the stop steps here never exceed max_steps (S); the arms that carry a cap enforce it as well: eagle per sample, JAX in its while-loop condition, the masked arms by their outer step count (a batch-level cap).
What each arm needed to get there: JAX, a cap in the while_loop condition; eagle, the kernel source unchanged, with its loop shape, unrolling and host entry chosen by hawk + eagle (per device, per batch).
arm |
loop structure |
threading |
|---|---|---|
eagle host, termination loop, up to 8 threads |
per sample: eagle.until_done, one host-team pass over dynamic tiles of samples, each sample stepped to its own stop |
OMP_NUM_THREADS=8 (eagle’s OpenMP host team: up to 8 threads; eagle uses one per physical core for a run under eagle._host_loop.SMT_MIN_WORK sample-steps) |
eagle host, termination loop, 1 thread |
per sample: eagle.until_done, one host-team pass over dynamic tiles of samples, each sample stepped to its own stop |
OMP_NUM_THREADS=1 (eagle’s OpenMP host team) |
eagle host, termination + compaction, up to 8 threads |
as the termination loop, the step launched over the active-set map only (recomputed every 16 steps) |
OMP_NUM_THREADS=8 (eagle’s OpenMP host team: up to 8 threads; eagle uses one per physical core for a run under eagle._host_loop.SMT_MIN_WORK sample-steps) |
eagle host, termination + compaction, 1 thread |
as the termination loop, the step launched over the active-set map only (recomputed every 16 steps) |
OMP_NUM_THREADS=1 (eagle’s OpenMP host team) |
eagle host, termination + compaction + reorder, up to 8 threads |
as the compaction loop, plus a reorder of the planes when the live samples spread thin, restored at the end |
OMP_NUM_THREADS=8 (eagle’s OpenMP host team: up to 8 threads; eagle uses one per physical core for a run under eagle._host_loop.SMT_MIN_WORK sample-steps) |
eagle host, termination + compaction + reorder, 1 thread |
as the compaction loop, plus a reorder of the planes when the live samples spread thin, restored at the end |
OMP_NUM_THREADS=1 (eagle’s OpenMP host team) |
Numba prange, 8 threads |
per sample: prange over samples, each runs its own steps to its stop step |
NUMBA_NUM_THREADS=8 and numba.set_num_threads(8) |
JAX sharded over 8 CPU devices |
per step over a shard: 8 shards, each a vmapped while loop running until the shard’s last sample stops or the step cap is reached |
XLA_FLAGS=–xla_force_host_platform_device_count=8, one shard per device |
JAX jit + vmap(while_loop), 1 device |
per step over the batch: the vmapped while loop runs until the last sample stops or the step cap is reached, finished samples held by a select |
XLA’s CPU client defaults, one device (CPU busy column shows the use) |
PyTorch masked, 8 threads |
per step over the batch: whole-batch tensor expressions, finished samples held by torch.where |
torch.set_num_threads(8), OMP_NUM_THREADS=8 |
NumPy masked |
per step over the batch: whole-batch array expressions, finished samples held by np.where |
single-threaded ufuncs; BLAS/OpenMP pools pinned to 1 |
multiprocessing, 8 x NumPy |
per step over a chunk: 8 processes, each the NumPy masked loop over its chunk, stopping when its chunk is done |
multiprocessing.Pool(8), fork; every process pinned to 1 thread |
plain Python loops |
per sample: a Python loop over samples, each runs its own steps to its stop step |
one interpreter thread |
Spread stop steps (log-uniform in [1, 100]), up to 100 steps
N |
arm |
threads |
wall (IQR) |
sample·steps/s |
useful FLOP/s (% peak) |
CPU busy |
peak memory (× minimum) |
|---|---|---|---|---|---|---|---|
1,000 |
eagle host, termination loop, up to 8 threads |
8 |
83.1 µs (360 ns) |
2.65e8 |
1.17e10 (–) |
8.3 |
–* |
1,000 |
eagle host, termination loop, 1 thread |
1 |
344 µs (325 ns) |
6.41e7 |
2.82e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
618 µs (3.95 µs) |
3.57e7 |
1.57e9 (–) |
8.1 |
–* |
1,000 |
eagle host, termination + compaction, 1 thread |
1 |
390 µs (582 ns) |
5.65e7 |
2.49e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
720 µs (3.32 µs) |
3.07e7 |
1.35e9 (–) |
8.1 |
–* |
1,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
469 µs (3.71 µs) |
4.70e7 |
2.07e9 (–) |
1 |
–* |
1,000 |
Numba prange, 8 threads |
8 |
50.8 µs (36.5 ns) |
4.34e8 |
1.91e10 (–) |
8.3 |
–* |
1,000 |
JAX sharded over 8 CPU devices |
8 |
186 µs (304 ns) |
1.19e8 |
5.23e9 (–) |
5.3 |
–* |
1,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
324 µs (235 ns) |
6.81e7 |
3.00e9 (–) |
0.36 |
–* |
1,000 |
PyTorch masked, 8 threads |
8 |
7.55 ms (62 µs) |
2.92e6 |
1.29e8 (–) |
1 |
–* |
1,000 |
NumPy masked |
1 |
3.94 ms (27.5 µs) |
5.60e6 |
2.47e8 (–) |
1 |
–* |
1,000 |
multiprocessing, 8 x NumPy |
8 |
5.92 ms (191 µs) |
3.73e6 |
1.64e8 (–) |
6 |
–* |
1,000 |
plain Python loops |
1 |
6.33 ms (6.34 µs) |
3.48e6 |
1.53e8 (–) |
1 |
–* |
10,000 |
eagle host, termination loop, up to 8 threads |
8 |
468 µs (951 ns) |
4.88e8 |
2.15e10 (–) |
8.2 |
–* |
10,000 |
eagle host, termination loop, 1 thread |
1 |
3.24 ms (2.18 µs) |
7.05e7 |
3.10e9 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
1.23 ms (4.38 µs) |
1.86e8 |
8.18e9 (–) |
8.1 |
–* |
10,000 |
eagle host, termination + compaction, 1 thread |
1 |
1.94 ms (1.32 µs) |
1.18e8 |
5.18e9 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
2.15 ms (9.97 µs) |
1.07e8 |
4.69e9 (–) |
7.9 |
–* |
10,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
2.6 ms (2.59 µs) |
8.78e7 |
3.86e9 (–) |
1 |
–* |
10,000 |
Numba prange, 8 threads |
8 |
414 µs (135 ns) |
5.52e8 |
2.43e10 (–) |
8.1 |
–* |
10,000 |
JAX sharded over 8 CPU devices |
8 |
768 µs (2.81 µs) |
2.97e8 |
1.31e10 (–) |
6.1 |
–* |
10,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
6.3 ms (49.8 µs) |
3.63e7 |
1.60e9 (–) |
1.5 |
–* |
10,000 |
PyTorch masked, 8 threads |
8 |
21.3 ms (27.2 µs) |
1.07e7 |
4.72e8 (–) |
1 |
–* |
10,000 |
NumPy masked |
1 |
15.7 ms (45.6 µs) |
1.46e7 |
6.40e8 (–) |
1 |
–* |
10,000 |
multiprocessing, 8 x NumPy |
8 |
10.5 ms (283 µs) |
2.17e7 |
9.56e8 (–) |
6 |
–* |
10,000 |
plain Python loops |
1 |
66.4 ms (152 µs) |
3.44e6 |
1.51e8 (–) |
1 |
–* |
100,000 |
eagle host, termination loop, up to 8 threads |
8 |
4.18 ms (2.49 µs) |
5.48e8 |
2.41e10 (–) |
8 |
–* |
100,000 |
eagle host, termination loop, 1 thread |
1 |
32 ms (14.8 µs) |
7.14e7 |
3.14e9 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
6.49 ms (10.3 µs) |
3.53e8 |
1.55e10 (–) |
7.7 |
–* |
100,000 |
eagle host, termination + compaction, 1 thread |
1 |
17.5 ms (16.7 µs) |
1.31e8 |
5.75e9 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
17.9 ms (35.5 µs) |
1.28e8 |
5.62e9 (–) |
7.9 |
–* |
100,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
25.6 ms (48.4 µs) |
8.93e7 |
3.93e9 (–) |
1 |
–* |
100,000 |
Numba prange, 8 threads |
8 |
4.05 ms (7.84 µs) |
5.65e8 |
2.49e10 (–) |
8 |
–* |
100,000 |
JAX sharded over 8 CPU devices |
8 |
13.1 ms (9.29 µs) |
1.74e8 |
7.66e9 (–) |
9.9 |
–* |
100,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
47 ms (431 µs) |
4.86e7 |
2.14e9 (–) |
1.8 |
–* |
100,000 |
PyTorch masked, 8 threads |
8 |
95.2 ms (5.39 ms) |
2.40e7 |
1.06e9 (–) |
8 |
–* |
100,000 |
NumPy masked |
1 |
193 ms (319 µs) |
1.19e7 |
5.22e8 (–) |
1 |
–* |
100,000 |
multiprocessing, 8 x NumPy |
8 |
41.7 ms (268 µs) |
5.49e7 |
2.42e9 (–) |
6.8 |
–* |
1,000,000 |
eagle host, termination loop, up to 8 threads |
8 |
41.3 ms (8.79 µs) |
5.57e8 |
2.45e10 (–) |
8 |
61.7 MiB (1.8×) |
1,000,000 |
eagle host, termination loop, 1 thread |
1 |
321 ms (444 µs) |
7.16e7 |
3.15e9 (–) |
1 |
61.7 MiB (1.8×) |
1,000,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
64.5 ms (116 µs) |
3.57e8 |
1.57e10 (–) |
8 |
65.4 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction, 1 thread |
1 |
182 ms (303 µs) |
1.27e8 |
5.57e9 (–) |
1 |
65.5 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
164 ms (208 µs) |
1.40e8 |
6.18e9 (–) |
4.3 |
93.4 MiB (2.72×) |
1,000,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
251 ms (575 µs) |
9.18e7 |
4.04e9 (–) |
1 |
93.7 MiB (2.73×) |
1,000,000 |
Numba prange, 8 threads |
8 |
39.9 ms (75.1 µs) |
5.76e8 |
2.54e10 (–) |
8 |
68 MiB (1.98×) |
1,000,000 |
JAX sharded over 8 CPU devices |
8 |
217 ms (1.12 ms) |
1.06e8 |
4.67e9 (–) |
13 |
198 MiB (5.77×) |
1,000,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
360 ms (5.46 ms) |
6.39e7 |
2.81e9 (–) |
3.7 |
174 MiB (5.08×) |
1,000,000 |
PyTorch masked, 8 threads |
8 |
870 ms (18.5 ms) |
2.64e7 |
1.16e9 (–) |
8 |
267 MiB (7.79×) |
1,000,000 |
NumPy masked |
1 |
2.2 s (9.04 ms) |
1.05e7 |
4.60e8 (–) |
1 |
169 MiB (4.93×) |
1,000,000 |
multiprocessing, 8 x NumPy |
8 |
494 ms (27.2 ms) |
4.65e7 |
2.05e9 (–) |
6.3 |
234 MiB (6.81×) |
Fastest arm by median wall time:
N = 1,000: Numba prange, 8 threads (50.8 µs), then eagle host, termination loop, up to 8 threads (83.1 µs, 1.64× the time)
N = 10,000: Numba prange, 8 threads (414 µs), then eagle host, termination loop, up to 8 threads (468 µs, 1.13× the time)
N = 100,000: Numba prange, 8 threads (4.05 ms), then eagle host, termination loop, up to 8 threads (4.18 ms, 1.03× the time)
N = 1,000,000: Numba prange, 8 threads (39.9 ms), then eagle host, termination loop, up to 8 threads (41.3 ms, 1.03× the time)
Spread stop steps (log-uniform in [10, 1000]), up to 1000 steps
N |
arm |
threads |
wall (IQR) |
sample·steps/s |
useful FLOP/s (% peak) |
CPU busy |
peak memory (× minimum) |
|---|---|---|---|---|---|---|---|
1,000 |
eagle host, termination loop, up to 8 threads |
8 |
498 µs (567 ns) |
4.22e8 |
1.86e10 (–) |
8.1 |
–* |
1,000 |
eagle host, termination loop, 1 thread |
1 |
3.1 ms (335 ns) |
6.78e7 |
2.98e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
5.48 ms (24.4 µs) |
3.84e7 |
1.69e9 (–) |
8 |
–* |
1,000 |
eagle host, termination + compaction, 1 thread |
1 |
3.45 ms (22 µs) |
6.09e7 |
2.68e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
6.27 ms (39.5 µs) |
3.35e7 |
1.47e9 (–) |
8 |
–* |
1,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
4.1 ms (8.42 µs) |
5.13e7 |
2.26e9 (–) |
1 |
–* |
1,000 |
Numba prange, 8 threads |
8 |
448 µs (372 ns) |
4.69e8 |
2.06e10 (–) |
8 |
–* |
1,000 |
JAX sharded over 8 CPU devices |
8 |
847 µs (2.71 µs) |
2.48e8 |
1.09e10 (–) |
6.1 |
–* |
1,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
2.84 ms (4.19 µs) |
7.41e7 |
3.26e9 (–) |
0.87 |
–* |
1,000 |
PyTorch masked, 8 threads |
8 |
74.9 ms (325 µs) |
2.81e6 |
1.23e8 (–) |
1 |
–* |
1,000 |
NumPy masked |
1 |
39.5 ms (227 µs) |
5.31e6 |
2.34e8 (–) |
1 |
–* |
1,000 |
multiprocessing, 8 x NumPy |
8 |
46.4 ms (1.25 ms) |
4.53e6 |
1.99e8 (–) |
6.9 |
–* |
1,000 |
plain Python loops |
1 |
60.6 ms (255 µs) |
3.47e6 |
1.52e8 (–) |
1 |
–* |
10,000 |
eagle host, termination loop, up to 8 threads |
8 |
4.26 ms (3.37 µs) |
5.12e8 |
2.25e10 (–) |
8 |
–* |
10,000 |
eagle host, termination loop, 1 thread |
1 |
32.1 ms (9.23 µs) |
6.80e7 |
2.99e9 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
9.53 ms (25.5 µs) |
2.29e8 |
1.01e10 (–) |
8 |
–* |
10,000 |
eagle host, termination + compaction, 1 thread |
1 |
14.3 ms (71.8 µs) |
1.52e8 |
6.70e9 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
10.9 ms (16.3 µs) |
2.00e8 |
8.78e9 (–) |
7.9 |
–* |
10,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
15.5 ms (43.2 µs) |
1.41e8 |
6.20e9 (–) |
1 |
–* |
10,000 |
Numba prange, 8 threads |
8 |
4.18 ms (1.02 µs) |
5.23e8 |
2.30e10 (–) |
8 |
–* |
10,000 |
JAX sharded over 8 CPU devices |
8 |
6.39 ms (31.4 µs) |
3.42e8 |
1.50e10 (–) |
6.1 |
–* |
10,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
57.4 ms (3.83 ms) |
3.80e7 |
1.67e9 (–) |
1.5 |
–* |
10,000 |
PyTorch masked, 8 threads |
8 |
204 ms (354 µs) |
1.07e7 |
4.72e8 (–) |
1 |
–* |
10,000 |
NumPy masked |
1 |
151 ms (135 µs) |
1.44e7 |
6.34e8 (–) |
1 |
–* |
10,000 |
multiprocessing, 8 x NumPy |
8 |
82.5 ms (1.66 ms) |
2.65e7 |
1.16e9 (–) |
7 |
–* |
10,000 |
plain Python loops |
1 |
632 ms (2.13 ms) |
3.46e6 |
1.52e8 (–) |
1 |
–* |
100,000 |
eagle host, termination loop, up to 8 threads |
8 |
40.4 ms (45 µs) |
5.36e8 |
2.36e10 (–) |
8 |
–* |
100,000 |
eagle host, termination loop, 1 thread |
1 |
317 ms (83.9 µs) |
6.84e7 |
3.01e9 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
40.5 ms (47.6 µs) |
5.35e8 |
2.35e10 (–) |
8 |
–* |
100,000 |
eagle host, termination + compaction, 1 thread |
1 |
119 ms (178 µs) |
1.82e8 |
8.02e9 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
50.2 ms (111 µs) |
4.32e8 |
1.90e10 (–) |
8 |
–* |
100,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
124 ms (114 µs) |
1.74e8 |
7.66e9 (–) |
1 |
–* |
100,000 |
Numba prange, 8 threads |
8 |
40.1 ms (17.6 µs) |
5.40e8 |
2.38e10 (–) |
8 |
–* |
100,000 |
JAX sharded over 8 CPU devices |
8 |
126 ms (531 µs) |
1.72e8 |
7.55e9 (–) |
10 |
–* |
100,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
473 ms (6.55 ms) |
4.58e7 |
2.02e9 (–) |
1.7 |
–* |
100,000 |
PyTorch masked, 8 threads |
8 |
618 ms (2.35 ms) |
3.50e7 |
1.54e9 (–) |
8 |
–* |
100,000 |
NumPy masked |
1 |
1.28 s (915 µs) |
1.69e7 |
7.44e8 (–) |
1 |
–* |
100,000 |
multiprocessing, 8 x NumPy |
8 |
297 ms (5.11 ms) |
7.30e7 |
3.21e9 (–) |
7 |
–* |
1,000,000 |
eagle host, termination loop, up to 8 threads |
8 |
400 ms (339 µs) |
5.40e8 |
2.38e10 (–) |
8 |
61.6 MiB (1.8×) |
1,000,000 |
eagle host, termination loop, 1 thread |
1 |
3.16 s (550 µs) |
6.84e7 |
3.01e9 (–) |
1 |
61.6 MiB (1.8×) |
1,000,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
419 ms (825 µs) |
5.16e8 |
2.27e10 (–) |
8 |
65.5 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction, 1 thread |
1 |
1.27 s (522 µs) |
1.71e8 |
7.52e9 (–) |
1 |
65.5 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
470 ms (836 µs) |
4.60e8 |
2.03e10 (–) |
6.5 |
93.6 MiB (2.73×) |
1,000,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
1.23 s (245 µs) |
1.75e8 |
7.72e9 (–) |
1 |
93.7 MiB (2.73×) |
1,000,000 |
Numba prange, 8 threads |
8 |
398 ms (64.4 µs) |
5.43e8 |
2.39e10 (–) |
8 |
68.2 MiB (1.99×) |
1,000,000 |
JAX sharded over 8 CPU devices |
8 |
2.15 s (32.3 ms) |
1.01e8 |
4.44e9 (–) |
13 |
199 MiB (5.79×) |
1,000,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
3.42 s (5.68 ms) |
6.32e7 |
2.78e9 (–) |
3.8 |
182 MiB (5.31×) |
1,000,000 |
PyTorch masked, 8 threads |
8 |
8.19 s (435 ms) |
2.64e7 |
1.16e9 (–) |
8 |
275 MiB (8.01×) |
1,000,000 |
NumPy masked |
1 |
21.6 s (194 ms) |
9.99e6 |
4.40e8 (–) |
1 |
169 MiB (4.92×) |
1,000,000 |
multiprocessing, 8 x NumPy |
8 |
4.07 s (34.1 ms) |
5.31e7 |
2.34e9 (–) |
7.8 |
234 MiB (6.81×) |
Fastest arm by median wall time:
N = 1,000: Numba prange, 8 threads (448 µs), then eagle host, termination loop, up to 8 threads (498 µs, 1.11× the time)
N = 10,000: Numba prange, 8 threads (4.18 ms), then eagle host, termination loop, up to 8 threads (4.26 ms, 1.02× the time)
N = 100,000: Numba prange, 8 threads (40.1 ms), then eagle host, termination loop, up to 8 threads (40.4 ms, 1.01× the time)
N = 1,000,000: Numba prange, 8 threads (398 ms), then eagle host, termination loop, up to 8 threads (400 ms, 1.01× the time)
Uniform stop step: every sample runs 1000 steps
N |
arm |
threads |
wall (IQR) |
sample·steps/s |
useful FLOP/s (% peak) |
CPU busy |
peak memory (× minimum) |
|---|---|---|---|---|---|---|---|
1,000 |
eagle host, termination loop, up to 8 threads |
8 |
764 µs (260 ns) |
1.31e9 |
5.76e10 (–) |
8 |
–* |
1,000 |
eagle host, termination loop, 1 thread |
1 |
5.58 ms (2.35 µs) |
1.79e8 |
7.88e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
6.74 ms (114 µs) |
1.48e8 |
6.53e9 (–) |
8 |
–* |
1,000 |
eagle host, termination + compaction, 1 thread |
1 |
6.46 ms (7.49 µs) |
1.55e8 |
6.81e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
7.05 ms (119 µs) |
1.42e8 |
6.24e9 (–) |
8 |
–* |
1,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
6.53 ms (13.2 µs) |
1.53e8 |
6.74e9 (–) |
1 |
–* |
1,000 |
Numba prange, 8 threads |
8 |
1.86 ms (5.48 µs) |
5.38e8 |
2.37e10 (–) |
8 |
–* |
1,000 |
JAX sharded over 8 CPU devices |
8 |
880 µs (2.28 µs) |
1.14e9 |
5.00e10 (–) |
6.2 |
–* |
1,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
2.84 ms (2.2 µs) |
3.52e8 |
1.55e10 (–) |
0.9 |
–* |
1,000 |
PyTorch masked, 8 threads |
8 |
73.9 ms (210 µs) |
1.35e7 |
5.95e8 (–) |
1 |
–* |
1,000 |
NumPy masked |
1 |
39 ms (163 µs) |
2.57e7 |
1.13e9 (–) |
1 |
–* |
1,000 |
multiprocessing, 8 x NumPy |
8 |
48 ms (290 µs) |
2.08e7 |
9.17e8 (–) |
7.1 |
–* |
1,000 |
plain Python loops |
1 |
289 ms (1.42 ms) |
3.46e6 |
1.52e8 (–) |
1 |
–* |
10,000 |
eagle host, termination loop, up to 8 threads |
8 |
7.19 ms (3.65 µs) |
1.39e9 |
6.12e10 (–) |
8 |
–* |
10,000 |
eagle host, termination loop, 1 thread |
1 |
55.3 ms (51.8 µs) |
1.81e8 |
7.96e9 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
14.5 ms (44.9 µs) |
6.89e8 |
3.03e10 (–) |
8 |
–* |
10,000 |
eagle host, termination + compaction, 1 thread |
1 |
47.3 ms (38.9 µs) |
2.12e8 |
9.31e9 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
14.8 ms (26.8 µs) |
6.76e8 |
2.97e10 (–) |
8 |
–* |
10,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
47.5 ms (15.2 µs) |
2.11e8 |
9.26e9 (–) |
1 |
–* |
10,000 |
Numba prange, 8 threads |
8 |
18.5 ms (4.23 µs) |
5.41e8 |
2.38e10 (–) |
8 |
–* |
10,000 |
JAX sharded over 8 CPU devices |
8 |
6.38 ms (186 µs) |
1.57e9 |
6.90e10 (–) |
6.3 |
–* |
10,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
29.6 ms (9.63 ms) |
3.38e8 |
1.49e10 (–) |
1.9 |
–* |
10,000 |
PyTorch masked, 8 threads |
8 |
195 ms (75 µs) |
5.12e7 |
2.25e9 (–) |
1 |
–* |
10,000 |
NumPy masked |
1 |
145 ms (334 µs) |
6.90e7 |
3.04e9 (–) |
1 |
–* |
10,000 |
multiprocessing, 8 x NumPy |
8 |
80.9 ms (1.13 ms) |
1.24e8 |
5.44e9 (–) |
7.1 |
–* |
100,000 |
eagle host, termination loop, up to 8 threads |
8 |
69.8 ms (41.7 µs) |
1.43e9 |
6.30e10 (–) |
8 |
–* |
100,000 |
eagle host, termination loop, 1 thread |
1 |
552 ms (420 µs) |
1.81e8 |
7.97e9 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
99.5 ms (194 µs) |
1.01e9 |
4.42e10 (–) |
8 |
–* |
100,000 |
eagle host, termination + compaction, 1 thread |
1 |
453 ms (500 µs) |
2.21e8 |
9.71e9 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
103 ms (55.1 µs) |
9.71e8 |
4.27e10 (–) |
8 |
–* |
100,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
453 ms (763 µs) |
2.21e8 |
9.71e9 (–) |
1 |
–* |
100,000 |
Numba prange, 8 threads |
8 |
185 ms (35.3 µs) |
5.42e8 |
2.38e10 (–) |
8 |
–* |
100,000 |
JAX sharded over 8 CPU devices |
8 |
126 ms (444 µs) |
7.92e8 |
3.48e10 (–) |
10 |
–* |
100,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
465 ms (2.29 ms) |
2.15e8 |
9.45e9 (–) |
1.8 |
–* |
100,000 |
PyTorch masked, 8 threads |
8 |
558 ms (1.47 ms) |
1.79e8 |
7.89e9 (–) |
8 |
–* |
100,000 |
NumPy masked |
1 |
1.06 s (2.24 ms) |
9.45e7 |
4.16e9 (–) |
1 |
–* |
100,000 |
multiprocessing, 8 x NumPy |
8 |
286 ms (7.13 ms) |
3.49e8 |
1.54e10 (–) |
6.8 |
–* |
1,000,000 |
eagle host, termination loop, up to 8 threads |
8 |
693 ms (175 µs) |
1.44e9 |
6.35e10 (–) |
8 |
61.9 MiB (1.8×) |
1,000,000 |
eagle host, termination loop, 1 thread |
1 |
5.52 s (3.57 ms) |
1.81e8 |
7.98e9 (–) |
1 |
61.7 MiB (1.8×) |
1,000,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
1.1 s (9.16 ms) |
9.09e8 |
4.00e10 (–) |
8 |
65.4 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction, 1 thread |
1 |
4.58 s (1.73 ms) |
2.18e8 |
9.60e9 (–) |
1 |
65.5 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
1.08 s (1.85 ms) |
9.27e8 |
4.08e10 (–) |
8 |
73.1 MiB (2.13×) |
1,000,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
4.58 s (564 µs) |
2.18e8 |
9.61e9 (–) |
1 |
73.1 MiB (2.13×) |
1,000,000 |
Numba prange, 8 threads |
8 |
1.85 s (711 µs) |
5.41e8 |
2.38e10 (–) |
8 |
60.4 MiB (1.76×) |
1,000,000 |
JAX sharded over 8 CPU devices |
8 |
2.14 s (4.74 ms) |
4.66e8 |
2.05e10 (–) |
13 |
189 MiB (5.52×) |
1,000,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
3.47 s (18.9 ms) |
2.88e8 |
1.27e10 (–) |
3.7 |
144 MiB (4.19×) |
1,000,000 |
PyTorch masked, 8 threads |
8 |
8.37 s (239 ms) |
1.19e8 |
5.25e9 (–) |
8 |
260 MiB (7.57×) |
1,000,000 |
NumPy masked |
1 |
19.6 s (39.1 ms) |
5.09e7 |
2.24e9 (–) |
1 |
162 MiB (4.73×) |
1,000,000 |
multiprocessing, 8 x NumPy |
8 |
4.04 s (22.9 ms) |
2.48e8 |
1.09e10 (–) |
7.7 |
222 MiB (6.47×) |
Fastest arm by median wall time:
N = 1,000: eagle host, termination loop, up to 8 threads (764 µs), then JAX sharded over 8 CPU devices (880 µs, 1.15× the time)
N = 10,000: JAX sharded over 8 CPU devices (6.38 ms), then eagle host, termination loop, up to 8 threads (7.19 ms, 1.13× the time)
N = 100,000: eagle host, termination loop, up to 8 threads (69.8 ms), then eagle host, termination + compaction, up to 8 threads (99.5 ms, 1.43× the time)
N = 1,000,000: eagle host, termination loop, up to 8 threads (693 ms), then eagle host, termination + compaction + reorder, up to 8 threads (1.08 s, 1.56× the time)
Cells not run
plain Python loops: plain Python runs only at N <= 10,000 with at most 2,500,000 sample-steps per run (spread S=100 N=100,000; spread S=100 N=1,000,000; spread S=1000 N=100,000; spread S=1000 N=1,000,000; uniform S=1000 N=10,000; uniform S=1000 N=100,000; uniform S=1000 N=1,000,000).
Where each tool fits
Each tool below is written the way its users write it. Two loop structures appear: per step over the batch (eagle’s termination loops, JAX, PyTorch, NumPy, multiprocessing) and per sample over its steps (Numba, plain Python). The per-sample structure touches each sample’s state once for all its steps; the per-step structure streams the batch through memory every step, which is the structure the GPU card’s kernels use. Ratios below are wall-time ratios at the largest N each arm ran (below 1 = less time than eagle at 8 threads).
eagle host, termination loop, up to 8 threads. The GPU card’s eagle graph arm, unchanged, on the host: the same hawk step kernel, eagle.deploy and eagle.until_done call, run by eagle’s OpenMP host team; one code base for GPUs and CPUs. On this card: spread S=100, N=1,000,000: 41.3 ms; spread S=1000, N=1,000,000: 400 ms; uniform S=1000, N=1,000,000: 693 ms. Fastest in 3 of 12 cells.
eagle host, termination loop, 1 thread. The same loop on one thread: the difference to the 8-thread row is the threading alone. On this card: spread S=100, N=1,000,000: 321 ms (7.78× eagle 8-thread time); spread S=1000, N=1,000,000: 3.16 s (7.9× eagle 8-thread time); uniform S=1000, N=1,000,000: 5.52 s (7.96× eagle 8-thread time). Fastest in 0 of 12 cells.
eagle host, termination + compaction, up to 8 threads. The GPU card’s compaction arm on the host: the launch covers only the samples still running, the map rebuilt every 16 steps when a sample finished. On this card: spread S=100, N=1,000,000: 64.5 ms (1.56× eagle 8-thread time); spread S=1000, N=1,000,000: 419 ms (1.05× eagle 8-thread time); uniform S=1000, N=1,000,000: 1.1 s (1.59× eagle 8-thread time). Fastest in 0 of 12 cells.
eagle host, termination + compaction, 1 thread. The compaction loop on one thread: the difference to the 8-thread row is the threading alone. On this card: spread S=100, N=1,000,000: 182 ms (4.4× eagle 8-thread time); spread S=1000, N=1,000,000: 1.27 s (3.16× eagle 8-thread time); uniform S=1000, N=1,000,000: 4.58 s (6.61× eagle 8-thread time). Fastest in 0 of 12 cells.
eagle host, termination + compaction + reorder, up to 8 threads. The GPU card’s reorder arm on the host: the live samples moved to the front of every plane when they spread thin, restored to sample order at the end (inside the wall). On this card: spread S=100, N=1,000,000: 164 ms (3.97× eagle 8-thread time); spread S=1000, N=1,000,000: 470 ms (1.17× eagle 8-thread time); uniform S=1000, N=1,000,000: 1.08 s (1.56× eagle 8-thread time). Fastest in 0 of 12 cells.
eagle host, termination + compaction + reorder, 1 thread. The reorder loop on one thread: the difference to the 8-thread row is the threading alone. On this card: spread S=100, N=1,000,000: 251 ms (6.07× eagle 8-thread time); spread S=1000, N=1,000,000: 1.23 s (3.08× eagle 8-thread time); uniform S=1000, N=1,000,000: 4.58 s (6.61× eagle 8-thread time). Fastest in 0 of 12 cells.
Numba prange, 8 threads. Compiles a plain Python loop to machine code with no separate language; each sample keeps its state in registers across all of its steps and stops on its own, so finished samples cost nothing. On this card: spread S=100, N=1,000,000: 39.9 ms (0.967× eagle 8-thread time); spread S=1000, N=1,000,000: 398 ms (0.995× eagle 8-thread time); uniform S=1000, N=1,000,000: 1.85 s (2.67× eagle 8-thread time). Fastest in 8 of 12 cells.
JAX sharded over 8 CPU devices. The JAX arm spread over 8 CPU devices: the batch is sharded and each shard runs its own vmapped while loop, with no cross-shard step synchronisation. On this card: spread S=100, N=1,000,000: 217 ms (5.25× eagle 8-thread time); spread S=1000, N=1,000,000: 2.15 s (5.36× eagle 8-thread time); uniform S=1000, N=1,000,000: 2.14 s (3.09× eagle 8-thread time). Fastest in 1 of 12 cells.
JAX jit + vmap(while_loop), 1 device. Compiles the whole loop once and batches a per-sample while loop with vmap; the same code runs on GPUs and differentiates with jax.grad. On this card: spread S=100, N=1,000,000: 360 ms (8.72× eagle 8-thread time); spread S=1000, N=1,000,000: 3.42 s (8.55× eagle 8-thread time); uniform S=1000, N=1,000,000: 3.47 s (5.01× eagle 8-thread time). Fastest in 0 of 12 cells.
PyTorch masked, 8 threads. Runs the masked array version on a multi-threaded tensor library; it fits where the computation already lives next to a PyTorch model or needs autograd. On this card: spread S=100, N=1,000,000: 870 ms (21.1× eagle 8-thread time); spread S=1000, N=1,000,000: 8.19 s (20.5× eagle 8-thread time); uniform S=1000, N=1,000,000: 8.37 s (12.1× eagle 8-thread time). Fastest in 0 of 12 cells.
NumPy masked. No compile step and no dependency beyond NumPy; masked array code computes every sample every step, so it fits dense batches where all samples run to the end (the uniform configuration). On this card: spread S=100, N=1,000,000: 2.2 s (53.3× eagle 8-thread time); spread S=1000, N=1,000,000: 21.6 s (54.1× eagle 8-thread time); uniform S=1000, N=1,000,000: 19.6 s (28.3× eagle 8-thread time). Fastest in 0 of 12 cells.
multiprocessing, 8 x NumPy. Spreads NumPy over processes with the standard library alone; each chunk stops once its own samples are done. On this card: spread S=100, N=1,000,000: 494 ms (12× eagle 8-thread time); spread S=1000, N=1,000,000: 4.07 s (10.2× eagle 8-thread time); uniform S=1000, N=1,000,000: 4.04 s (5.83× eagle 8-thread time). Fastest in 0 of 12 cells.
plain Python loops. Nothing to install and each line can be stepped in a debugger; run only at small N here. On this card: spread S=100, N=10,000: 66.4 ms (142× eagle 8-thread time); spread S=1000, N=10,000: 632 ms (148× eagle 8-thread time); uniform S=1000, N=1,000: 289 ms (379× eagle 8-thread time). Fastest in 0 of 12 cells.
Compile time
Its own pass, never part of any wall. Each measurement is a fresh process: the imports run untimed, then the timer spans the first call to a ready kernel (trace + code generation + compile, or the cache lookup). Cold: an empty cache (hawk’s compile cache pointed at a fresh directory through HAWK_CACHE_DIR; Numba with cache=True into a fresh NUMBA_CACHE_DIR; JAX with its persistent compilation cache in a fresh directory). Warm: a new process on the cache the first cold run populated (hawk cache hit; Numba’s on-disk cache; JAX’s persistent cache with the minimum compile time and entry size set to 0). hawk: eagle.deploy of the kernel (tracing, C++ emission, the g++ host build, eagle’s plan); the active-set variant is a second build of the same step. Numba: the first call on a 64-sample batch (compile plus a negligible run). JAX: lower + compile from abstract shapes, once per N (XLA specialises on the shape). Medians of 3 cold and 3 warm processes.
arm |
compiled |
cold (median of 3) |
warm (median of 3) |
|---|---|---|---|
eagle host, termination loop, up to 8 threads |
hawk step kernel (host build) |
1.29 s |
35.3 ms |
eagle host, termination + compaction, up to 8 threads |
hawk active-set step kernel (host build) |
1.1 s |
30.2 ms |
Numba prange, 8 threads |
Numba parallel loop (first call, 64 samples) |
712 ms |
253 ms |
JAX sharded over 8 CPU devices |
XLA executable for N = 1,000 |
113 ms |
43.4 ms |
JAX sharded over 8 CPU devices |
XLA executable for N = 10,000 |
117 ms |
43.1 ms |
JAX sharded over 8 CPU devices |
XLA executable for N = 100,000 |
118 ms |
42.9 ms |
JAX sharded over 8 CPU devices |
XLA executable for N = 1,000,000 |
123 ms |
43.5 ms |
JAX jit + vmap(while_loop), 1 device |
XLA executable for N = 1,000 |
105 ms |
45.3 ms |
JAX jit + vmap(while_loop), 1 device |
XLA executable for N = 10,000 |
107 ms |
45.8 ms |
JAX jit + vmap(while_loop), 1 device |
XLA executable for N = 100,000 |
634 ms |
46 ms |
JAX jit + vmap(while_loop), 1 device |
XLA executable for N = 1,000,000 |
642 ms |
46 ms |
No compile step: PyTorch masked, 8 threads, NumPy masked, multiprocessing, 8 x NumPy, plain Python loops.
Same build as another row: eagle host, termination loop, 1 thread (as eagle host, termination loop, up to 8 threads); eagle host, termination + compaction, 1 thread (as eagle host, termination + compaction, up to 8 threads); eagle host, termination + compaction + reorder, up to 8 threads (as eagle host, termination + compaction, up to 8 threads); eagle host, termination + compaction + reorder, 1 thread (as eagle host, termination + compaction, up to 8 threads).
Memory method
A separate pass after the timing pass. Each (arm, N, configuration) runs in a fresh process: imports, then a warm-up run on a 64-sample batch (every JIT compiles here; JAX also compiles for shape N from abstract shapes), then the baseline (resident set size, VmRSS) and a reset of the process’s peak mark (/proc/self/clear_refs, so ru_maxrss restarts from the baseline); then the batch’s inputs are generated and one full run done. Peak memory = ru_maxrss at the end minus the baseline; for multiprocessing, plus each pool process’s own peak above its baseline (shared pages counted once per process). Minimum = the state the workload needs, N × ((2 state + 2 float64 scalars: omega, zeta) × 8 + 4 bytes for the stop step as a 32-bit integer) = 36 bytes per sample; the factor is peak / minimum. Compile overhead = baseline minus the resident size before the arm’s imports-and-warm-up. Below about 1 MiB the allocator reuses memory already resident at the baseline, so small-N peaks can read below the minimum.
Compile/import overhead (resident memory added by the warm-up compile, measured once per arm at the largest N): eagle host, termination loop, up to 8 threads 31.7 MiB; eagle host, termination loop, 1 thread 31.6 MiB; eagle host, termination + compaction, up to 8 threads 31.7 MiB; eagle host, termination + compaction, 1 thread 31.5 MiB; eagle host, termination + compaction + reorder, up to 8 threads 31.7 MiB; eagle host, termination + compaction + reorder, 1 thread 31.6 MiB; Numba prange, 8 threads 193 MiB; JAX sharded over 8 CPU devices 282 MiB; JAX jit + vmap(while_loop), 1 device 283 MiB; PyTorch masked, 8 threads 290 MiB; NumPy masked 22 MiB; multiprocessing, 8 x NumPy 23.2 MiB; plain Python loops 21.9 MiB.
CPU peak memory is shown for N = 1,000,000 only, net of the benchmark harness’s own copies (36 bytes per sample on eagle’s arms). At smaller N this harness cannot resolve it: each measuring process inherits the launching process’s peak, so smaller readings show that floor rather than the workload. The cards keep the raw readings; the next card run measures every size directly.
spread (up to 100 steps)
Fastest arm by median wall time:
N = 1,000: Numba prange, 8 threads (50.8 µs), then eagle host, termination loop, up to 8 threads (83.1 µs, 1.64× the time)
N = 10,000: Numba prange, 8 threads (414 µs), then eagle host, termination loop, up to 8 threads (468 µs, 1.13× the time)
N = 100,000: Numba prange, 8 threads (4.05 ms), then eagle host, termination loop, up to 8 threads (4.18 ms, 1.03× the time)
N = 1,000,000: Numba prange, 8 threads (39.9 ms), then eagle host, termination loop, up to 8 threads (41.3 ms, 1.03× the time)
spread (up to 1000 steps)
Fastest arm by median wall time:
N = 1,000: Numba prange, 8 threads (448 µs), then eagle host, termination loop, up to 8 threads (498 µs, 1.11× the time)
N = 10,000: Numba prange, 8 threads (4.18 ms), then eagle host, termination loop, up to 8 threads (4.26 ms, 1.02× the time)
N = 100,000: Numba prange, 8 threads (40.1 ms), then eagle host, termination loop, up to 8 threads (40.4 ms, 1.01× the time)
N = 1,000,000: Numba prange, 8 threads (398 ms), then eagle host, termination loop, up to 8 threads (400 ms, 1.01× the time)
uniform (up to 1000 steps)
Fastest arm by median wall time:
N = 1,000: eagle host, termination loop, up to 8 threads (764 µs), then JAX sharded over 8 CPU devices (880 µs, 1.15× the time)
N = 10,000: JAX sharded over 8 CPU devices (6.38 ms), then eagle host, termination loop, up to 8 threads (7.19 ms, 1.13× the time)
N = 100,000: eagle host, termination loop, up to 8 threads (69.8 ms), then eagle host, termination + compaction, up to 8 threads (99.5 ms, 1.43× the time)
N = 1,000,000: eagle host, termination loop, up to 8 threads (693 ms), then eagle host, termination + compaction + reorder, up to 8 threads (1.08 s, 1.56× the time)
Run on: host CPU aarch64; compiler aarch64-conda-linux-gnu-g++ (conda-forge gcc 14.4.0-7) 14.4.0; Python 3.12.14.
N = 1,000,000
Log scale, from 21.6 ms to 2.54 s.
N = 1,000,000
Log scale, from 207 ms to 26.2 s.
N = 1,000,000
Log scale, from 257 ms to 24.9 s.
Table of every arm and every N
CPU: aarch64 (16 cores, 16 logical CPUs, 2 float64 lanes per vector instruction). Peak FP64: – FLOP/s (16 cores × 2 FMA units × 2 float64 lanes × 1 FLOP per FMA × an unknown clock (the model name, lscpu and cpufreq name none), so no peak). Without SIMD the same cores peak at – FLOP/s. Medians of 5 interleaved repetitions; IQR in parentheses.
Same workload as the GPU card (damped oscillators, classical RK4, float64, every sample stops at its own step). “8 threads” uses all 16 logical CPUs (16 cores × 1 hardware threads). CPU busy = process CPU seconds / wall (all threads and worker processes; spin-waiting counts). Compile times are not in any wall; they have their own table below. 0.2 s pause before every timed repetition, so the previous arm’s thread pools stop spin-waiting first, then 0.05 s of untimed runs of the arm itself, so the timed block starts on a ramped clock (the machine’s busy cores during the pause and the clock at the timer start are recorded per row); a repetition whose run is shorter than 0.1 s repeats it back to back until the repetition spans at least 0.1 s.
The arms run in 3 groups, each interleaved on its own: (eagle_term8, eagle_term1, eagle_compact8, eagle_compact1, eagle_reorder8, eagle_reorder1, python_loops); (eagle_term8, torch_masked, mp_numpy); (eagle_term8, numpy_masked, numba_prange, jax_shard8, jax_vmap). eagle at 8 threads runs in every group as the common reference; its row below is from the first group. Its median wall differed between groups by at most 0.4 % in any cell (spread S=1000, N=1,000).
SIMD. eagle’s host build of the step (hawk host profile native: -O3 -ffp-contract=off -mcpu=native -std=c++23 -shared -fPIC -DAETHER_CPP_MODE, no -march) is not vectorised: its entry function holds scalar float64 instructions (none) and packed ones (none); the 8-thread speed-up over 1 thread comes from the OpenMP team. The active-set build of the step (compaction and reorder arms) is not vectorised (packed float64 instructions: none). Numba’s compiled loop uses scalar float64 instructions only (each sample’s steps are a dependent chain). NumPy and PyTorch use their libraries’ own SIMD kernels for each array operation; JAX’s XLA code was not inspected.
Every arm runs each sample to its own stop, and the stop steps here never exceed max_steps (S); the arms that carry a cap enforce it as well: eagle per sample, JAX in its while-loop condition, the masked arms by their outer step count (a batch-level cap).
What each arm needed to get there: JAX, a cap in the while_loop condition; eagle, the kernel source unchanged, with its loop shape, unrolling and host entry chosen by hawk + eagle (per device, per batch).
arm |
loop structure |
threading |
|---|---|---|
eagle host, termination loop, up to 8 threads |
per sample: eagle.until_done, one host-team pass over dynamic tiles of samples, each sample stepped to its own stop |
OMP_NUM_THREADS=8 (eagle’s OpenMP host team: up to 8 threads; eagle uses one per physical core for a run under eagle._host_loop.SMT_MIN_WORK sample-steps) |
eagle host, termination loop, 1 thread |
per sample: eagle.until_done, one host-team pass over dynamic tiles of samples, each sample stepped to its own stop |
OMP_NUM_THREADS=1 (eagle’s OpenMP host team) |
eagle host, termination + compaction, up to 8 threads |
as the termination loop, the step launched over the active-set map only (recomputed every 16 steps) |
OMP_NUM_THREADS=8 (eagle’s OpenMP host team: up to 8 threads; eagle uses one per physical core for a run under eagle._host_loop.SMT_MIN_WORK sample-steps) |
eagle host, termination + compaction, 1 thread |
as the termination loop, the step launched over the active-set map only (recomputed every 16 steps) |
OMP_NUM_THREADS=1 (eagle’s OpenMP host team) |
eagle host, termination + compaction + reorder, up to 8 threads |
as the compaction loop, plus a reorder of the planes when the live samples spread thin, restored at the end |
OMP_NUM_THREADS=8 (eagle’s OpenMP host team: up to 8 threads; eagle uses one per physical core for a run under eagle._host_loop.SMT_MIN_WORK sample-steps) |
eagle host, termination + compaction + reorder, 1 thread |
as the compaction loop, plus a reorder of the planes when the live samples spread thin, restored at the end |
OMP_NUM_THREADS=1 (eagle’s OpenMP host team) |
Numba prange, 8 threads |
per sample: prange over samples, each runs its own steps to its stop step |
NUMBA_NUM_THREADS=8 and numba.set_num_threads(8) |
JAX sharded over 8 CPU devices |
per step over a shard: 8 shards, each a vmapped while loop running until the shard’s last sample stops or the step cap is reached |
XLA_FLAGS=–xla_force_host_platform_device_count=8, one shard per device |
JAX jit + vmap(while_loop), 1 device |
per step over the batch: the vmapped while loop runs until the last sample stops or the step cap is reached, finished samples held by a select |
XLA’s CPU client defaults, one device (CPU busy column shows the use) |
PyTorch masked, 8 threads |
per step over the batch: whole-batch tensor expressions, finished samples held by torch.where |
torch.set_num_threads(8), OMP_NUM_THREADS=8 |
NumPy masked |
per step over the batch: whole-batch array expressions, finished samples held by np.where |
single-threaded ufuncs; BLAS/OpenMP pools pinned to 1 |
multiprocessing, 8 x NumPy |
per step over a chunk: 8 processes, each the NumPy masked loop over its chunk, stopping when its chunk is done |
multiprocessing.Pool(8), fork; every process pinned to 1 thread |
plain Python loops |
per sample: a Python loop over samples, each runs its own steps to its stop step |
one interpreter thread |
Spread stop steps (log-uniform in [1, 100]), up to 100 steps
N |
arm |
threads |
wall (IQR) |
sample·steps/s |
useful FLOP/s (% peak) |
CPU busy |
peak memory (× minimum) |
|---|---|---|---|---|---|---|---|
1,000 |
eagle host, termination loop, up to 8 threads |
8 |
59.4 µs (79.9 ns) |
3.72e8 |
1.63e10 (–) |
6.9 |
–* |
1,000 |
eagle host, termination loop, 1 thread |
1 |
277 µs (663 ns) |
7.97e7 |
3.51e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
469 µs (1.18 µs) |
4.70e7 |
2.07e9 (–) |
8 |
–* |
1,000 |
eagle host, termination + compaction, 1 thread |
1 |
335 µs (223 ns) |
6.58e7 |
2.90e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
530 µs (3.14 µs) |
4.16e7 |
1.83e9 (–) |
8 |
–* |
1,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
361 µs (1.25 µs) |
6.11e7 |
2.69e9 (–) |
1 |
–* |
1,000 |
Numba prange, 8 threads |
8 |
48.3 µs (18.9 ns) |
4.56e8 |
2.01e10 (–) |
8.1 |
–* |
1,000 |
JAX sharded over 8 CPU devices |
8 |
166 µs (495 ns) |
1.33e8 |
5.84e9 (–) |
5.9 |
–* |
1,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
544 µs (90.6 ns) |
4.06e7 |
1.79e9 (–) |
1.1 |
–* |
1,000 |
PyTorch masked, 8 threads |
8 |
6.2 ms (18.9 µs) |
3.56e6 |
1.56e8 (–) |
1 |
–* |
1,000 |
NumPy masked |
1 |
2.73 ms (4.69 µs) |
8.09e6 |
3.56e8 (–) |
1 |
–* |
1,000 |
multiprocessing, 8 x NumPy |
8 |
2.08 ms (9.07 µs) |
1.06e7 |
4.67e8 (–) |
6.6 |
–* |
1,000 |
plain Python loops |
1 |
5.81 ms (1.86 µs) |
3.80e6 |
1.67e8 (–) |
1 |
–* |
10,000 |
eagle host, termination loop, up to 8 threads |
8 |
382 µs (322 ns) |
5.98e8 |
2.63e10 (–) |
7.8 |
–* |
10,000 |
eagle host, termination loop, 1 thread |
1 |
2.76 ms (926 ns) |
8.29e7 |
3.65e9 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
1 ms (1.64 µs) |
2.28e8 |
1.00e10 (–) |
8 |
–* |
10,000 |
eagle host, termination + compaction, 1 thread |
1 |
2.12 ms (2.17 µs) |
1.08e8 |
4.73e9 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
1.63 ms (1.92 µs) |
1.41e8 |
6.18e9 (–) |
6 |
–* |
10,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
2.7 ms (2.11 µs) |
8.48e7 |
3.73e9 (–) |
1 |
–* |
10,000 |
Numba prange, 8 threads |
8 |
407 µs (252 ns) |
5.62e8 |
2.47e10 (–) |
8 |
–* |
10,000 |
JAX sharded over 8 CPU devices |
8 |
741 µs (1.72 µs) |
3.08e8 |
1.36e10 (–) |
7.5 |
–* |
10,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
3.64 ms (17.6 µs) |
6.27e7 |
2.76e9 (–) |
2 |
–* |
10,000 |
PyTorch masked, 8 threads |
8 |
20.1 ms (229 µs) |
1.13e7 |
4.99e8 (–) |
1 |
–* |
10,000 |
NumPy masked |
1 |
11 ms (30.5 µs) |
2.08e7 |
9.15e8 (–) |
1 |
–* |
10,000 |
multiprocessing, 8 x NumPy |
8 |
3.57 ms (6.31 µs) |
6.40e7 |
2.81e9 (–) |
7 |
–* |
10,000 |
plain Python loops |
1 |
60.5 ms (46.9 µs) |
3.78e6 |
1.66e8 (–) |
1 |
–* |
100,000 |
eagle host, termination loop, up to 8 threads |
8 |
3.51 ms (2.07 µs) |
6.52e8 |
2.87e10 (–) |
6.9 |
–* |
100,000 |
eagle host, termination loop, 1 thread |
1 |
27.3 ms (5.86 µs) |
8.37e7 |
3.68e9 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
5.75 ms (5.63 µs) |
3.98e8 |
1.75e10 (–) |
7.4 |
–* |
100,000 |
eagle host, termination + compaction, 1 thread |
1 |
20.4 ms (20 µs) |
1.12e8 |
4.93e9 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
13.5 ms (58.1 µs) |
1.70e8 |
7.46e9 (–) |
3.7 |
–* |
100,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
27.9 ms (22.3 µs) |
8.20e7 |
3.61e9 (–) |
1 |
–* |
100,000 |
Numba prange, 8 threads |
8 |
3.96 ms (2.56 µs) |
5.78e8 |
2.54e10 (–) |
7.9 |
–* |
100,000 |
JAX sharded over 8 CPU devices |
8 |
5.57 ms (6.46 µs) |
4.11e8 |
1.81e10 (–) |
12 |
–* |
100,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
24.3 ms (65 µs) |
9.40e7 |
4.14e9 (–) |
2.6 |
–* |
100,000 |
PyTorch masked, 8 threads |
8 |
54.7 ms (17.3 µs) |
4.18e7 |
1.84e9 (–) |
8 |
–* |
100,000 |
NumPy masked |
1 |
154 ms (1.9 ms) |
1.49e7 |
6.54e8 (–) |
1 |
–* |
100,000 |
multiprocessing, 8 x NumPy |
8 |
17.4 ms (84.6 µs) |
1.31e8 |
5.78e9 (–) |
6.6 |
–* |
1,000,000 |
eagle host, termination loop, up to 8 threads |
8 |
34.6 ms (10.9 µs) |
6.65e8 |
2.93e10 (–) |
7.9 |
61.9 MiB (1.8×) |
1,000,000 |
eagle host, termination loop, 1 thread |
1 |
275 ms (60.4 µs) |
8.36e7 |
3.68e9 (–) |
1 |
61.9 MiB (1.8×) |
1,000,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
52.1 ms (22.4 µs) |
4.41e8 |
1.94e10 (–) |
7.3 |
65.7 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction, 1 thread |
1 |
208 ms (447 µs) |
1.10e8 |
4.86e9 (–) |
1 |
65.8 MiB (1.92×) |
1,000,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
131 ms (466 µs) |
1.76e8 |
7.72e9 (–) |
3.5 |
93.9 MiB (2.73×) |
1,000,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
274 ms (1.23 ms) |
8.40e7 |
3.70e9 (–) |
1 |
93.9 MiB (2.73×) |
1,000,000 |
Numba prange, 8 threads |
8 |
39 ms (175 µs) |
5.89e8 |
2.59e10 (–) |
7.9 |
67.5 MiB (1.97×) |
1,000,000 |
JAX sharded over 8 CPU devices |
8 |
42.6 ms (489 µs) |
5.39e8 |
2.37e10 (–) |
14 |
194 MiB (5.65×) |
1,000,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
185 ms (1.18 ms) |
1.24e8 |
5.47e9 (–) |
3.3 |
182 MiB (5.3×) |
1,000,000 |
PyTorch masked, 8 threads |
8 |
304 ms (2.31 ms) |
7.57e7 |
3.33e9 (–) |
8 |
211 MiB (6.13×) |
1,000,000 |
NumPy masked |
1 |
1.59 s (11 ms) |
1.45e7 |
6.37e8 (–) |
1 |
169 MiB (4.93×) |
1,000,000 |
multiprocessing, 8 x NumPy |
8 |
244 ms (2.19 ms) |
9.44e7 |
4.15e9 (–) |
7 |
235 MiB (6.85×) |
Fastest arm by median wall time:
N = 1,000: Numba prange, 8 threads (48.3 µs), then eagle host, termination loop, up to 8 threads (59.4 µs, 1.23× the time)
N = 10,000: eagle host, termination loop, up to 8 threads (382 µs), then Numba prange, 8 threads (407 µs, 1.06× the time)
N = 100,000: eagle host, termination loop, up to 8 threads (3.51 ms), then Numba prange, 8 threads (3.96 ms, 1.13× the time)
N = 1,000,000: eagle host, termination loop, up to 8 threads (34.6 ms), then Numba prange, 8 threads (39 ms, 1.13× the time)
Spread stop steps (log-uniform in [10, 1000]), up to 1000 steps
N |
arm |
threads |
wall (IQR) |
sample·steps/s |
useful FLOP/s (% peak) |
CPU busy |
peak memory (× minimum) |
|---|---|---|---|---|---|---|---|
1,000 |
eagle host, termination loop, up to 8 threads |
8 |
407 µs (123 ns) |
5.16e8 |
2.27e10 (–) |
8 |
–* |
1,000 |
eagle host, termination loop, 1 thread |
1 |
2.57 ms (11.8 µs) |
8.18e7 |
3.60e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
4.52 ms (53.9 µs) |
4.65e7 |
2.04e9 (–) |
8 |
–* |
1,000 |
eagle host, termination + compaction, 1 thread |
1 |
3.14 ms (13.3 µs) |
6.69e7 |
2.94e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
5.23 ms (27.7 µs) |
4.02e7 |
1.77e9 (–) |
8 |
–* |
1,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
3.38 ms (5.88 µs) |
6.22e7 |
2.74e9 (–) |
1 |
–* |
1,000 |
Numba prange, 8 threads |
8 |
438 µs (60.3 ns) |
4.80e8 |
2.11e10 (–) |
8 |
–* |
1,000 |
JAX sharded over 8 CPU devices |
8 |
803 µs (1.34 µs) |
2.62e8 |
1.15e10 (–) |
7.4 |
–* |
1,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
5.07 ms (667 ns) |
4.14e7 |
1.82e9 (–) |
1 |
–* |
1,000 |
PyTorch masked, 8 threads |
8 |
62.9 ms (160 µs) |
3.34e6 |
1.47e8 (–) |
1 |
–* |
1,000 |
NumPy masked |
1 |
27.5 ms (75.6 µs) |
7.64e6 |
3.36e8 (–) |
1 |
–* |
1,000 |
multiprocessing, 8 x NumPy |
8 |
16.9 ms (7.54 µs) |
1.25e7 |
5.49e8 (–) |
7.6 |
–* |
1,000 |
plain Python loops |
1 |
55.8 ms (23.5 µs) |
3.77e6 |
1.66e8 (–) |
1 |
–* |
10,000 |
eagle host, termination loop, up to 8 threads |
8 |
3.49 ms (217 ns) |
6.25e8 |
2.75e10 (–) |
7.8 |
–* |
10,000 |
eagle host, termination loop, 1 thread |
1 |
26.7 ms (6.24 µs) |
8.18e7 |
3.60e9 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
8.92 ms (78.7 µs) |
2.45e8 |
1.08e10 (–) |
8 |
–* |
10,000 |
eagle host, termination + compaction, 1 thread |
1 |
18.1 ms (16.5 µs) |
1.21e8 |
5.31e9 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
9.2 ms (15.9 µs) |
2.37e8 |
1.04e10 (–) |
7.6 |
–* |
10,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
19.1 ms (10.9 µs) |
1.14e8 |
5.02e9 (–) |
1 |
–* |
10,000 |
Numba prange, 8 threads |
8 |
4.08 ms (484 ns) |
5.34e8 |
2.35e10 (–) |
7.8 |
–* |
10,000 |
JAX sharded over 8 CPU devices |
8 |
6.47 ms (1.87 µs) |
3.37e8 |
1.48e10 (–) |
7.9 |
–* |
10,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
35.7 ms (41.8 µs) |
6.11e7 |
2.69e9 (–) |
2 |
–* |
10,000 |
PyTorch masked, 8 threads |
8 |
192 ms (691 µs) |
1.14e7 |
5.00e8 (–) |
1 |
–* |
10,000 |
NumPy masked |
1 |
110 ms (563 µs) |
1.99e7 |
8.75e8 (–) |
1 |
–* |
10,000 |
multiprocessing, 8 x NumPy |
8 |
30 ms (35.9 µs) |
7.28e7 |
3.20e9 (–) |
7.9 |
–* |
10,000 |
plain Python loops |
1 |
580 ms (38.2 µs) |
3.77e6 |
1.66e8 (–) |
1 |
–* |
100,000 |
eagle host, termination loop, up to 8 threads |
8 |
33.5 ms (18.6 µs) |
6.47e8 |
2.85e10 (–) |
7.9 |
–* |
100,000 |
eagle host, termination loop, 1 thread |
1 |
264 ms (50.4 µs) |
8.21e7 |
3.61e9 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
34.5 ms (43.2 µs) |
6.29e8 |
2.77e10 (–) |
7.8 |
–* |
100,000 |
eagle host, termination + compaction, 1 thread |
1 |
169 ms (187 µs) |
1.28e8 |
5.65e9 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
42.8 ms (65.4 µs) |
5.06e8 |
2.23e10 (–) |
6.1 |
–* |
100,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
174 ms (87.7 µs) |
1.25e8 |
5.48e9 (–) |
1 |
–* |
100,000 |
Numba prange, 8 threads |
8 |
39.1 ms (985 ns) |
5.54e8 |
2.44e10 (–) |
7.9 |
–* |
100,000 |
JAX sharded over 8 CPU devices |
8 |
54 ms (278 µs) |
4.01e8 |
1.77e10 (–) |
12 |
–* |
100,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
242 ms (1.81 ms) |
8.96e7 |
3.94e9 (–) |
2.6 |
–* |
100,000 |
PyTorch masked, 8 threads |
8 |
536 ms (3 ms) |
4.04e7 |
1.78e9 (–) |
8 |
–* |
100,000 |
NumPy masked |
1 |
1.2 s (1.06 ms) |
1.80e7 |
7.93e8 (–) |
1 |
–* |
100,000 |
multiprocessing, 8 x NumPy |
8 |
137 ms (184 µs) |
1.58e8 |
6.94e9 (–) |
7.7 |
–* |
1,000,000 |
eagle host, termination loop, up to 8 threads |
8 |
330 ms (5.36 µs) |
6.54e8 |
2.88e10 (–) |
8 |
61.9 MiB (1.8×) |
1,000,000 |
eagle host, termination loop, 1 thread |
1 |
2.64 s (376 µs) |
8.21e7 |
3.61e9 (–) |
1 |
61.8 MiB (1.8×) |
1,000,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
301 ms (1.73 ms) |
7.19e8 |
3.16e10 (–) |
6.8 |
65.8 MiB (1.92×) |
1,000,000 |
eagle host, termination + compaction, 1 thread |
1 |
1.76 s (2.42 ms) |
1.23e8 |
5.41e9 (–) |
1 |
65.8 MiB (1.92×) |
1,000,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
397 ms (853 µs) |
5.45e8 |
2.40e10 (–) |
5 |
93.8 MiB (2.73×) |
1,000,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
1.72 s (1.2 ms) |
1.26e8 |
5.54e9 (–) |
1 |
94 MiB (2.74×) |
1,000,000 |
Numba prange, 8 threads |
8 |
388 ms (118 µs) |
5.58e8 |
2.45e10 (–) |
8 |
67.6 MiB (1.97×) |
1,000,000 |
JAX sharded over 8 CPU devices |
8 |
410 ms (6.01 ms) |
5.28e8 |
2.32e10 (–) |
15 |
196 MiB (5.72×) |
1,000,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
1.77 s (11.8 ms) |
1.22e8 |
5.39e9 (–) |
3.3 |
182 MiB (5.29×) |
1,000,000 |
PyTorch masked, 8 threads |
8 |
3.02 s (12.3 ms) |
7.15e7 |
3.15e9 (–) |
8 |
217 MiB (6.31×) |
1,000,000 |
NumPy masked |
1 |
16.4 s (41.8 ms) |
1.32e7 |
5.81e8 (–) |
1 |
169 MiB (4.93×) |
1,000,000 |
multiprocessing, 8 x NumPy |
8 |
2.22 s (17.3 ms) |
9.74e7 |
4.28e9 (–) |
7.7 |
236 MiB (6.87×) |
Fastest arm by median wall time:
N = 1,000: eagle host, termination loop, up to 8 threads (407 µs), then Numba prange, 8 threads (438 µs, 1.08× the time)
N = 10,000: eagle host, termination loop, up to 8 threads (3.49 ms), then Numba prange, 8 threads (4.08 ms, 1.17× the time)
N = 100,000: eagle host, termination loop, up to 8 threads (33.5 ms), then eagle host, termination + compaction, up to 8 threads (34.5 ms, 1.03× the time)
N = 1,000,000: eagle host, termination + compaction, up to 8 threads (301 ms), then eagle host, termination loop, up to 8 threads (330 ms, 1.1× the time)
Uniform stop step: every sample runs 1000 steps
N |
arm |
threads |
wall (IQR) |
sample·steps/s |
useful FLOP/s (% peak) |
CPU busy |
peak memory (× minimum) |
|---|---|---|---|---|---|---|---|
1,000 |
eagle host, termination loop, up to 8 threads |
8 |
699 µs (225 ns) |
1.43e9 |
6.30e10 (–) |
8 |
–* |
1,000 |
eagle host, termination loop, 1 thread |
1 |
5.22 ms (2.6 µs) |
1.92e8 |
8.44e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
5.53 ms (15.2 µs) |
1.81e8 |
7.96e9 (–) |
8 |
–* |
1,000 |
eagle host, termination + compaction, 1 thread |
1 |
8.59 ms (10.5 µs) |
1.16e8 |
5.12e9 (–) |
1 |
–* |
1,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
6.03 ms (28.9 µs) |
1.66e8 |
7.30e9 (–) |
8 |
–* |
1,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
8.58 ms (3.31 µs) |
1.17e8 |
5.13e9 (–) |
1 |
–* |
1,000 |
Numba prange, 8 threads |
8 |
1.81 ms (515 ns) |
5.52e8 |
2.43e10 (–) |
8 |
–* |
1,000 |
JAX sharded over 8 CPU devices |
8 |
809 µs (172 ns) |
1.24e9 |
5.44e10 (–) |
7.6 |
–* |
1,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
5.06 ms (3.16 µs) |
1.98e8 |
8.70e9 (–) |
1 |
–* |
1,000 |
PyTorch masked, 8 threads |
8 |
63.6 ms (177 µs) |
1.57e7 |
6.92e8 (–) |
1 |
–* |
1,000 |
NumPy masked |
1 |
27.5 ms (20.6 µs) |
3.64e7 |
1.60e9 (–) |
1 |
–* |
1,000 |
multiprocessing, 8 x NumPy |
8 |
16.9 ms (18.2 µs) |
5.93e7 |
2.61e9 (–) |
7.8 |
–* |
1,000 |
plain Python loops |
1 |
266 ms (47.5 µs) |
3.76e6 |
1.66e8 (–) |
1 |
–* |
10,000 |
eagle host, termination loop, up to 8 threads |
8 |
6.67 ms (2.43 µs) |
1.50e9 |
6.60e10 (–) |
7.8 |
–* |
10,000 |
eagle host, termination loop, 1 thread |
1 |
51.7 ms (21.5 µs) |
1.93e8 |
8.51e9 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
14.1 ms (44.5 µs) |
7.11e8 |
3.13e10 (–) |
8 |
–* |
10,000 |
eagle host, termination + compaction, 1 thread |
1 |
73.9 ms (28.5 µs) |
1.35e8 |
5.95e9 (–) |
1 |
–* |
10,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
14 ms (20.4 µs) |
7.15e8 |
3.15e10 (–) |
8 |
–* |
10,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
73.9 ms (4.59 µs) |
1.35e8 |
5.95e9 (–) |
1 |
–* |
10,000 |
Numba prange, 8 threads |
8 |
18 ms (2.26 µs) |
5.56e8 |
2.45e10 (–) |
8 |
–* |
10,000 |
JAX sharded over 8 CPU devices |
8 |
6.46 ms (2.1 µs) |
1.55e9 |
6.82e10 (–) |
7.9 |
–* |
10,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
35.6 ms (161 µs) |
2.81e8 |
1.24e10 (–) |
2 |
–* |
10,000 |
PyTorch masked, 8 threads |
8 |
191 ms (679 µs) |
5.23e7 |
2.30e9 (–) |
1 |
–* |
10,000 |
NumPy masked |
1 |
109 ms (1.25 ms) |
9.16e7 |
4.03e9 (–) |
1 |
–* |
10,000 |
multiprocessing, 8 x NumPy |
8 |
30 ms (18.9 µs) |
3.33e8 |
1.47e10 (–) |
7.9 |
–* |
100,000 |
eagle host, termination loop, up to 8 threads |
8 |
65 ms (255 µs) |
1.54e9 |
6.77e10 (–) |
7.9 |
–* |
100,000 |
eagle host, termination loop, 1 thread |
1 |
517 ms (2.23 ms) |
1.94e8 |
8.52e9 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
99.1 ms (11.5 µs) |
1.01e9 |
4.44e10 (–) |
8 |
–* |
100,000 |
eagle host, termination + compaction, 1 thread |
1 |
725 ms (3.43 ms) |
1.38e8 |
6.07e9 (–) |
1 |
–* |
100,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
99.7 ms (608 µs) |
1.00e9 |
4.41e10 (–) |
8 |
–* |
100,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
726 ms (3.54 ms) |
1.38e8 |
6.06e9 (–) |
1 |
–* |
100,000 |
Numba prange, 8 threads |
8 |
180 ms (13.3 µs) |
5.56e8 |
2.45e10 (–) |
8 |
–* |
100,000 |
JAX sharded over 8 CPU devices |
8 |
54.2 ms (484 µs) |
1.84e9 |
8.11e10 (–) |
12 |
–* |
100,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
241 ms (447 µs) |
4.15e8 |
1.83e10 (–) |
2.6 |
–* |
100,000 |
PyTorch masked, 8 threads |
8 |
520 ms (6.38 ms) |
1.92e8 |
8.46e9 (–) |
8 |
–* |
100,000 |
NumPy masked |
1 |
1.19 s (2.02 ms) |
8.38e7 |
3.69e9 (–) |
1 |
–* |
100,000 |
multiprocessing, 8 x NumPy |
8 |
140 ms (41.4 µs) |
7.16e8 |
3.15e10 (–) |
7.6 |
–* |
1,000,000 |
eagle host, termination loop, up to 8 threads |
8 |
648 ms (92 µs) |
1.54e9 |
6.79e10 (–) |
8 |
62 MiB (1.8×) |
1,000,000 |
eagle host, termination loop, 1 thread |
1 |
5.17 s (1.88 ms) |
1.94e8 |
8.51e9 (–) |
1 |
61.9 MiB (1.8×) |
1,000,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
957 ms (690 µs) |
1.05e9 |
4.60e10 (–) |
8 |
65.7 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction, 1 thread |
1 |
7.25 s (5.27 ms) |
1.38e8 |
6.07e9 (–) |
1 |
65.9 MiB (1.92×) |
1,000,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
953 ms (387 µs) |
1.05e9 |
4.62e10 (–) |
8 |
73.4 MiB (2.14×) |
1,000,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
7.26 s (17.2 ms) |
1.38e8 |
6.06e9 (–) |
1 |
73.5 MiB (2.14×) |
1,000,000 |
Numba prange, 8 threads |
8 |
1.8 s (155 µs) |
5.56e8 |
2.45e10 (–) |
8 |
60.4 MiB (1.76×) |
1,000,000 |
JAX sharded over 8 CPU devices |
8 |
411 ms (11.4 ms) |
2.43e9 |
1.07e11 (–) |
15 |
187 MiB (5.43×) |
1,000,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
1.76 s (6.55 ms) |
5.68e8 |
2.50e10 (–) |
3.3 |
166 MiB (4.85×) |
1,000,000 |
PyTorch masked, 8 threads |
8 |
2.76 s (5.66 ms) |
3.62e8 |
1.59e10 (–) |
8 |
209 MiB (6.1×) |
1,000,000 |
NumPy masked |
1 |
15.6 s (41.3 ms) |
6.43e7 |
2.83e9 (–) |
1 |
163 MiB (4.73×) |
1,000,000 |
multiprocessing, 8 x NumPy |
8 |
2.2 s (9.66 ms) |
4.54e8 |
2.00e10 (–) |
7.7 |
224 MiB (6.52×) |
Fastest arm by median wall time:
N = 1,000: eagle host, termination loop, up to 8 threads (699 µs), then JAX sharded over 8 CPU devices (809 µs, 1.16× the time)
N = 10,000: JAX sharded over 8 CPU devices (6.46 ms), then eagle host, termination loop, up to 8 threads (6.67 ms, 1.03× the time)
N = 100,000: JAX sharded over 8 CPU devices (54.2 ms), then eagle host, termination loop, up to 8 threads (65 ms, 1.2× the time)
N = 1,000,000: JAX sharded over 8 CPU devices (411 ms), then eagle host, termination loop, up to 8 threads (648 ms, 1.58× the time)
Cells not run
plain Python loops: plain Python runs only at N <= 10,000 with at most 2,500,000 sample-steps per run (spread S=100 N=100,000; spread S=100 N=1,000,000; spread S=1000 N=100,000; spread S=1000 N=1,000,000; uniform S=1000 N=10,000; uniform S=1000 N=100,000; uniform S=1000 N=1,000,000).
Where each tool fits
Each tool below is written the way its users write it. Two loop structures appear: per step over the batch (eagle’s termination loops, JAX, PyTorch, NumPy, multiprocessing) and per sample over its steps (Numba, plain Python). The per-sample structure touches each sample’s state once for all its steps; the per-step structure streams the batch through memory every step, which is the structure the GPU card’s kernels use. Ratios below are wall-time ratios at the largest N each arm ran (below 1 = less time than eagle at 8 threads).
eagle host, termination loop, up to 8 threads. The GPU card’s eagle graph arm, unchanged, on the host: the same hawk step kernel, eagle.deploy and eagle.until_done call, run by eagle’s OpenMP host team; one code base for GPUs and CPUs. On this card: spread S=100, N=1,000,000: 34.6 ms; spread S=1000, N=1,000,000: 330 ms; uniform S=1000, N=1,000,000: 648 ms. Fastest in 7 of 12 cells.
eagle host, termination loop, 1 thread. The same loop on one thread: the difference to the 8-thread row is the threading alone. On this card: spread S=100, N=1,000,000: 275 ms (7.96× eagle 8-thread time); spread S=1000, N=1,000,000: 2.64 s (7.98× eagle 8-thread time); uniform S=1000, N=1,000,000: 5.17 s (7.98× eagle 8-thread time). Fastest in 0 of 12 cells.
eagle host, termination + compaction, up to 8 threads. The GPU card’s compaction arm on the host: the launch covers only the samples still running, the map rebuilt every 16 steps when a sample finished. On this card: spread S=100, N=1,000,000: 52.1 ms (1.51× eagle 8-thread time); spread S=1000, N=1,000,000: 301 ms (0.91× eagle 8-thread time); uniform S=1000, N=1,000,000: 957 ms (1.48× eagle 8-thread time). Fastest in 1 of 12 cells.
eagle host, termination + compaction, 1 thread. The compaction loop on one thread: the difference to the 8-thread row is the threading alone. On this card: spread S=100, N=1,000,000: 208 ms (6.02× eagle 8-thread time); spread S=1000, N=1,000,000: 1.76 s (5.32× eagle 8-thread time); uniform S=1000, N=1,000,000: 7.25 s (11.2× eagle 8-thread time). Fastest in 0 of 12 cells.
eagle host, termination + compaction + reorder, up to 8 threads. The GPU card’s reorder arm on the host: the live samples moved to the front of every plane when they spread thin, restored to sample order at the end (inside the wall). On this card: spread S=100, N=1,000,000: 131 ms (3.79× eagle 8-thread time); spread S=1000, N=1,000,000: 397 ms (1.2× eagle 8-thread time); uniform S=1000, N=1,000,000: 953 ms (1.47× eagle 8-thread time). Fastest in 0 of 12 cells.
eagle host, termination + compaction + reorder, 1 thread. The reorder loop on one thread: the difference to the 8-thread row is the threading alone. On this card: spread S=100, N=1,000,000: 274 ms (7.92× eagle 8-thread time); spread S=1000, N=1,000,000: 1.72 s (5.2× eagle 8-thread time); uniform S=1000, N=1,000,000: 7.26 s (11.2× eagle 8-thread time). Fastest in 0 of 12 cells.
Numba prange, 8 threads. Compiles a plain Python loop to machine code with no separate language; each sample keeps its state in registers across all of its steps and stops on its own, so finished samples cost nothing. On this card: spread S=100, N=1,000,000: 39 ms (1.13× eagle 8-thread time); spread S=1000, N=1,000,000: 388 ms (1.17× eagle 8-thread time); uniform S=1000, N=1,000,000: 1.8 s (2.78× eagle 8-thread time). Fastest in 1 of 12 cells.
JAX sharded over 8 CPU devices. The JAX arm spread over 8 CPU devices: the batch is sharded and each shard runs its own vmapped while loop, with no cross-shard step synchronisation. On this card: spread S=100, N=1,000,000: 42.6 ms (1.23× eagle 8-thread time); spread S=1000, N=1,000,000: 410 ms (1.24× eagle 8-thread time); uniform S=1000, N=1,000,000: 411 ms (0.635× eagle 8-thread time). Fastest in 3 of 12 cells.
JAX jit + vmap(while_loop), 1 device. Compiles the whole loop once and batches a per-sample while loop with vmap; the same code runs on GPUs and differentiates with jax.grad. On this card: spread S=100, N=1,000,000: 185 ms (5.35× eagle 8-thread time); spread S=1000, N=1,000,000: 1.77 s (5.34× eagle 8-thread time); uniform S=1000, N=1,000,000: 1.76 s (2.72× eagle 8-thread time). Fastest in 0 of 12 cells.
PyTorch masked, 8 threads. Runs the masked array version on a multi-threaded tensor library; it fits where the computation already lives next to a PyTorch model or needs autograd. On this card: spread S=100, N=1,000,000: 304 ms (8.79× eagle 8-thread time); spread S=1000, N=1,000,000: 3.02 s (9.15× eagle 8-thread time); uniform S=1000, N=1,000,000: 2.76 s (4.26× eagle 8-thread time). Fastest in 0 of 12 cells.
NumPy masked. No compile step and no dependency beyond NumPy; masked array code computes every sample every step, so it fits dense batches where all samples run to the end (the uniform configuration). On this card: spread S=100, N=1,000,000: 1.59 s (46× eagle 8-thread time); spread S=1000, N=1,000,000: 16.4 s (49.5× eagle 8-thread time); uniform S=1000, N=1,000,000: 15.6 s (24× eagle 8-thread time). Fastest in 0 of 12 cells.
multiprocessing, 8 x NumPy. Spreads NumPy over processes with the standard library alone; each chunk stops once its own samples are done. On this card: spread S=100, N=1,000,000: 244 ms (7.05× eagle 8-thread time); spread S=1000, N=1,000,000: 2.22 s (6.72× eagle 8-thread time); uniform S=1000, N=1,000,000: 2.2 s (3.4× eagle 8-thread time). Fastest in 0 of 12 cells.
plain Python loops. Nothing to install and each line can be stepped in a debugger; run only at small N here. On this card: spread S=100, N=10,000: 60.5 ms (158× eagle 8-thread time); spread S=1000, N=10,000: 580 ms (166× eagle 8-thread time); uniform S=1000, N=1,000: 266 ms (381× eagle 8-thread time). Fastest in 0 of 12 cells.
Compile time
Its own pass, never part of any wall. Each measurement is a fresh process: the imports run untimed, then the timer spans the first call to a ready kernel (trace + code generation + compile, or the cache lookup). Cold: an empty cache (hawk’s compile cache pointed at a fresh directory through HAWK_CACHE_DIR; Numba with cache=True into a fresh NUMBA_CACHE_DIR; JAX with its persistent compilation cache in a fresh directory). Warm: a new process on the cache the first cold run populated (hawk cache hit; Numba’s on-disk cache; JAX’s persistent cache with the minimum compile time and entry size set to 0). hawk: eagle.deploy of the kernel (tracing, C++ emission, the g++ host build, eagle’s plan); the active-set variant is a second build of the same step. Numba: the first call on a 64-sample batch (compile plus a negligible run). JAX: lower + compile from abstract shapes, once per N (XLA specialises on the shape). Medians of 3 cold and 3 warm processes.
arm |
compiled |
cold (median of 3) |
warm (median of 3) |
|---|---|---|---|
eagle host, termination loop, up to 8 threads |
hawk step kernel (host build) |
435 ms |
17.2 ms |
eagle host, termination + compaction, up to 8 threads |
hawk active-set step kernel (host build) |
297 ms |
13.4 ms |
Numba prange, 8 threads |
Numba parallel loop (first call, 64 samples) |
520 ms |
159 ms |
JAX sharded over 8 CPU devices |
XLA executable for N = 1,000 |
83.7 ms |
31 ms |
JAX sharded over 8 CPU devices |
XLA executable for N = 10,000 |
87 ms |
31.4 ms |
JAX sharded over 8 CPU devices |
XLA executable for N = 100,000 |
88.7 ms |
31.4 ms |
JAX sharded over 8 CPU devices |
XLA executable for N = 1,000,000 |
92.3 ms |
31.1 ms |
JAX jit + vmap(while_loop), 1 device |
XLA executable for N = 1,000 |
78.1 ms |
33 ms |
JAX jit + vmap(while_loop), 1 device |
XLA executable for N = 10,000 |
78.9 ms |
33 ms |
JAX jit + vmap(while_loop), 1 device |
XLA executable for N = 100,000 |
84.7 ms |
33.2 ms |
JAX jit + vmap(while_loop), 1 device |
XLA executable for N = 1,000,000 |
83.7 ms |
33.1 ms |
No compile step: PyTorch masked, 8 threads, NumPy masked, multiprocessing, 8 x NumPy, plain Python loops.
Same build as another row: eagle host, termination loop, 1 thread (as eagle host, termination loop, up to 8 threads); eagle host, termination + compaction, 1 thread (as eagle host, termination + compaction, up to 8 threads); eagle host, termination + compaction + reorder, up to 8 threads (as eagle host, termination + compaction, up to 8 threads); eagle host, termination + compaction + reorder, 1 thread (as eagle host, termination + compaction, up to 8 threads).
Memory method
A separate pass after the timing pass. Each (arm, N, configuration) runs in a fresh process: imports, then a warm-up run on a 64-sample batch (every JIT compiles here; JAX also compiles for shape N from abstract shapes), then the baseline (resident set size, VmRSS) and a reset of the process’s peak mark (/proc/self/clear_refs, so ru_maxrss restarts from the baseline); then the batch’s inputs are generated and one full run done. Peak memory = ru_maxrss at the end minus the baseline; for multiprocessing, plus each pool process’s own peak above its baseline (shared pages counted once per process). Minimum = the state the workload needs, N × ((2 state + 2 float64 scalars: omega, zeta) × 8 + 4 bytes for the stop step as a 32-bit integer) = 36 bytes per sample; the factor is peak / minimum. Compile overhead = baseline minus the resident size before the arm’s imports-and-warm-up. Below about 1 MiB the allocator reuses memory already resident at the baseline, so small-N peaks can read below the minimum.
Compile/import overhead (resident memory added by the warm-up compile, measured once per arm at the largest N): eagle host, termination loop, up to 8 threads 28.7 MiB; eagle host, termination loop, 1 thread 29.2 MiB; eagle host, termination + compaction, up to 8 threads 29 MiB; eagle host, termination + compaction, 1 thread 29.6 MiB; eagle host, termination + compaction + reorder, up to 8 threads 28.9 MiB; eagle host, termination + compaction + reorder, 1 thread 30.1 MiB; Numba prange, 8 threads 168 MiB; JAX sharded over 8 CPU devices 274 MiB; JAX jit + vmap(while_loop), 1 device 252 MiB; PyTorch masked, 8 threads 265 MiB; NumPy masked 20.7 MiB; multiprocessing, 8 x NumPy 21.5 MiB; plain Python loops 19.3 MiB.
CPU peak memory is shown for N = 1,000,000 only, net of the benchmark harness’s own copies (36 bytes per sample on eagle’s arms). At smaller N this harness cannot resolve it: each measuring process inherits the launching process’s peak, so smaller readings show that floor rather than the workload. The cards keep the raw readings; the next card run measures every size directly.
spread (up to 100 steps)
Fastest arm by median wall time:
N = 1,000: Numba prange, 8 threads (48.3 µs), then eagle host, termination loop, up to 8 threads (59.4 µs, 1.23× the time)
N = 10,000: eagle host, termination loop, up to 8 threads (382 µs), then Numba prange, 8 threads (407 µs, 1.06× the time)
N = 100,000: eagle host, termination loop, up to 8 threads (3.51 ms), then Numba prange, 8 threads (3.96 ms, 1.13× the time)
N = 1,000,000: eagle host, termination loop, up to 8 threads (34.6 ms), then Numba prange, 8 threads (39 ms, 1.13× the time)
spread (up to 1000 steps)
Fastest arm by median wall time:
N = 1,000: eagle host, termination loop, up to 8 threads (407 µs), then Numba prange, 8 threads (438 µs, 1.08× the time)
N = 10,000: eagle host, termination loop, up to 8 threads (3.49 ms), then Numba prange, 8 threads (4.08 ms, 1.17× the time)
N = 100,000: eagle host, termination loop, up to 8 threads (33.5 ms), then eagle host, termination + compaction, up to 8 threads (34.5 ms, 1.03× the time)
N = 1,000,000: eagle host, termination + compaction, up to 8 threads (301 ms), then eagle host, termination loop, up to 8 threads (330 ms, 1.1× the time)
uniform (up to 1000 steps)
Fastest arm by median wall time:
N = 1,000: eagle host, termination loop, up to 8 threads (699 µs), then JAX sharded over 8 CPU devices (809 µs, 1.16× the time)
N = 10,000: JAX sharded over 8 CPU devices (6.46 ms), then eagle host, termination loop, up to 8 threads (6.67 ms, 1.03× the time)
N = 100,000: JAX sharded over 8 CPU devices (54.2 ms), then eagle host, termination loop, up to 8 threads (65 ms, 1.2× the time)
N = 1,000,000: JAX sharded over 8 CPU devices (411 ms), then eagle host, termination loop, up to 8 threads (648 ms, 1.58× the time)
Run on: host CPU INTEL XEON PLATINUM 8581C CPU @ 2.30GHz; compiler x86_64-conda-linux-gnu-g++ (conda-forge gcc 14.4.0-7) 14.4.0; Python 3.12.14.
N = 1,000,000
Log scale, from 11.2 ms to 5.16 s.
N = 1,000,000
Log scale, from 101 ms to 46.6 s.
N = 1,000,000
Log scale, from 76.9 ms to 45.4 s.
Table of every arm and every N
CPU: INTEL(R) XEON(R) PLATINUM 8581C CPU @ 2.30GHz (8 cores, 16 logical CPUs, 8 float64 lanes per vector instruction). Peak FP64: 5.89e11 FLOP/s (8 cores × 2 FMA units × 8 float64 lanes × 2 FLOP per FMA × 2.3 GHz base clock). Without SIMD the same cores peak at 7.36e10 FLOP/s. Medians of 5 interleaved repetitions; IQR in parentheses.
Same workload as the GPU card (damped oscillators, classical RK4, float64, every sample stops at its own step). “8 threads” uses all 16 logical CPUs (8 cores × 2 hardware threads). CPU busy = process CPU seconds / wall (all threads and worker processes; spin-waiting counts). Compile times are not in any wall; they have their own table below. 0.2 s pause before every timed repetition, so the previous arm’s thread pools stop spin-waiting first, then 0.05 s of untimed runs of the arm itself, so the timed block starts on a ramped clock (the machine’s busy cores during the pause and the clock at the timer start are recorded per row); a repetition whose run is shorter than 0.1 s repeats it back to back until the repetition spans at least 0.1 s.
The arms run in 3 groups, each interleaved on its own: (eagle_term8, eagle_term1, eagle_compact8, eagle_compact1, eagle_reorder8, eagle_reorder1, python_loops); (eagle_term8, torch_masked, mp_numpy); (eagle_term8, numpy_masked, numba_prange, jax_shard8, jax_vmap). eagle at 8 threads runs in every group as the common reference; its row below is from the first group. Its median wall differed between groups by at most 2.5 % in any cell (uniform S=1000, N=1,000).
SIMD. eagle’s host build of the step (hawk host profile native-vector-math: -O3 -ffp-contract=fast -march=native -mprefer-vector-width=512 -fno-trapping-math -DAETHER_HOST_VECTOR_MATH -std=c++23 -shared -fPIC -DAETHER_CPP_MODE) is vectorised: its entry function holds scalar float64 instructions (vaddsd ×6, vfmadd132sd ×11, vfmadd231sd ×1, vfnmsub132sd ×3, vfnmsub231sd ×1, vmulsd ×12) and packed ones (vaddpd ×4, vfmadd132pd ×18, vfmadd213pd ×2, vfmadd231pd ×4, vfnmadd132pd ×2, vfnmadd231pd ×6, vmulpd ×8). The active-set build of the step (compaction and reorder arms) is not vectorised (packed float64 instructions: none). Numba’s compiled loop uses scalar float64 instructions only (each sample’s steps are a dependent chain). NumPy and PyTorch use their libraries’ own SIMD kernels for each array operation; JAX’s XLA code was not inspected.
Every arm runs each sample to its own stop, and the stop steps here never exceed max_steps (S); the arms that carry a cap enforce it as well: eagle per sample, JAX in its while-loop condition, the masked arms by their outer step count (a batch-level cap).
What each arm needed to get there: JAX, a cap in the while_loop condition; eagle, the kernel source unchanged, with its loop shape, unrolling and host entry chosen by hawk + eagle (per device, per batch).
arm |
loop structure |
threading |
|---|---|---|
eagle host, termination loop, up to 8 threads |
per sample: eagle.until_done, one host-team pass over dynamic tiles of samples, each sample stepped to its own stop |
OMP_NUM_THREADS=8 (eagle’s OpenMP host team: up to 8 threads; eagle uses one per physical core for a run under eagle._host_loop.SMT_MIN_WORK sample-steps) |
eagle host, termination loop, 1 thread |
per sample: eagle.until_done, one host-team pass over dynamic tiles of samples, each sample stepped to its own stop |
OMP_NUM_THREADS=1 (eagle’s OpenMP host team) |
eagle host, termination + compaction, up to 8 threads |
as the termination loop, the step launched over the active-set map only (recomputed every 16 steps) |
OMP_NUM_THREADS=8 (eagle’s OpenMP host team: up to 8 threads; eagle uses one per physical core for a run under eagle._host_loop.SMT_MIN_WORK sample-steps) |
eagle host, termination + compaction, 1 thread |
as the termination loop, the step launched over the active-set map only (recomputed every 16 steps) |
OMP_NUM_THREADS=1 (eagle’s OpenMP host team) |
eagle host, termination + compaction + reorder, up to 8 threads |
as the compaction loop, plus a reorder of the planes when the live samples spread thin, restored at the end |
OMP_NUM_THREADS=8 (eagle’s OpenMP host team: up to 8 threads; eagle uses one per physical core for a run under eagle._host_loop.SMT_MIN_WORK sample-steps) |
eagle host, termination + compaction + reorder, 1 thread |
as the compaction loop, plus a reorder of the planes when the live samples spread thin, restored at the end |
OMP_NUM_THREADS=1 (eagle’s OpenMP host team) |
Numba prange, 8 threads |
per sample: prange over samples, each runs its own steps to its stop step |
NUMBA_NUM_THREADS=8 and numba.set_num_threads(8) |
JAX sharded over 8 CPU devices |
per step over a shard: 8 shards, each a vmapped while loop running until the shard’s last sample stops or the step cap is reached |
XLA_FLAGS=–xla_force_host_platform_device_count=8, one shard per device |
JAX jit + vmap(while_loop), 1 device |
per step over the batch: the vmapped while loop runs until the last sample stops or the step cap is reached, finished samples held by a select |
XLA’s CPU client defaults, one device (CPU busy column shows the use) |
PyTorch masked, 8 threads |
per step over the batch: whole-batch tensor expressions, finished samples held by torch.where |
torch.set_num_threads(8), OMP_NUM_THREADS=8 |
NumPy masked |
per step over the batch: whole-batch array expressions, finished samples held by np.where |
single-threaded ufuncs; BLAS/OpenMP pools pinned to 1 |
multiprocessing, 8 x NumPy |
per step over a chunk: 8 processes, each the NumPy masked loop over its chunk, stopping when its chunk is done |
multiprocessing.Pool(8), fork; every process pinned to 1 thread |
plain Python loops |
per sample: a Python loop over samples, each runs its own steps to its stop step |
one interpreter thread |
Spread stop steps (log-uniform in [1, 100]), up to 100 steps
N |
arm |
threads |
wall (IQR) |
sample·steps/s |
useful FLOP/s (% peak) |
CPU busy |
peak memory (× minimum) |
|---|---|---|---|---|---|---|---|
1,000 |
eagle host, termination loop, up to 8 threads |
8 |
42.5 µs (302 ns) |
5.20e8 |
2.29e10 (3.9 %) |
8.3 |
–* |
1,000 |
eagle host, termination loop, 1 thread |
1 |
141 µs (986 ns) |
1.57e8 |
6.90e9 (1.2 %) |
1 |
–* |
1,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
503 µs (10.5 µs) |
4.38e7 |
1.93e9 (0.33 %) |
8.1 |
–* |
1,000 |
eagle host, termination + compaction, 1 thread |
1 |
260 µs (6.23 µs) |
8.47e7 |
3.73e9 (0.63 %) |
1 |
–* |
1,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
525 µs (11 µs) |
4.21e7 |
1.85e9 (0.31 %) |
8 |
–* |
1,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
303 µs (4.08 µs) |
7.29e7 |
3.21e9 (0.54 %) |
1 |
–* |
1,000 |
Numba prange, 8 threads |
8 |
42.5 µs (42.9 ns) |
5.19e8 |
2.28e10 (3.9 %) |
8.1 |
–* |
1,000 |
JAX sharded over 8 CPU devices |
8 |
136 µs (312 ns) |
1.62e8 |
7.11e9 (1.2 %) |
5 |
–* |
1,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
239 µs (651 ns) |
9.23e7 |
4.06e9 (0.69 %) |
1.1 |
–* |
1,000 |
PyTorch masked, 8 threads |
8 |
4.52 ms (3.94 µs) |
4.88e6 |
2.15e8 (0.037 %) |
1 |
–* |
1,000 |
NumPy masked |
1 |
2.59 ms (8.78 µs) |
8.52e6 |
3.75e8 (0.064 %) |
1 |
–* |
1,000 |
multiprocessing, 8 x NumPy |
8 |
2.12 ms (19 µs) |
1.04e7 |
4.58e8 (0.078 %) |
6.3 |
–* |
1,000 |
plain Python loops |
1 |
3.74 ms (11.9 µs) |
5.89e6 |
2.59e8 (0.044 %) |
1 |
–* |
10,000 |
eagle host, termination loop, up to 8 threads |
8 |
213 µs (354 ns) |
1.07e9 |
4.73e10 (8 %) |
8.1 |
–* |
10,000 |
eagle host, termination loop, 1 thread |
1 |
1.3 ms (586 ns) |
1.76e8 |
7.73e9 (1.3 %) |
1 |
–* |
10,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
1.03 ms (3.63 µs) |
2.22e8 |
9.75e9 (1.7 %) |
8 |
–* |
10,000 |
eagle host, termination + compaction, 1 thread |
1 |
1.61 ms (1.6 µs) |
1.42e8 |
6.25e9 (1.1 %) |
1 |
–* |
10,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
1.45 ms (47.4 µs) |
1.58e8 |
6.96e9 (1.2 %) |
8 |
–* |
10,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
2.03 ms (3.31 µs) |
1.13e8 |
4.96e9 (0.84 %) |
1 |
–* |
10,000 |
Numba prange, 8 threads |
8 |
359 µs (70 ns) |
6.36e8 |
2.80e10 (4.8 %) |
8.1 |
–* |
10,000 |
JAX sharded over 8 CPU devices |
8 |
568 µs (1.48 µs) |
4.02e8 |
1.77e10 (3 %) |
6.1 |
–* |
10,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
4.37 ms (151 µs) |
5.23e7 |
2.30e9 (0.39 %) |
1.7 |
–* |
10,000 |
PyTorch masked, 8 threads |
8 |
16.8 ms (48.2 µs) |
1.36e7 |
5.99e8 (0.1 %) |
1 |
–* |
10,000 |
NumPy masked |
1 |
16 ms (58.7 µs) |
1.43e7 |
6.27e8 (0.11 %) |
1 |
–* |
10,000 |
multiprocessing, 8 x NumPy |
8 |
4.3 ms (105 µs) |
5.31e7 |
2.34e9 (0.4 %) |
6.4 |
–* |
10,000 |
plain Python loops |
1 |
39.5 ms (335 µs) |
5.79e6 |
2.55e8 (0.043 %) |
1 |
–* |
100,000 |
eagle host, termination loop, up to 8 threads |
8 |
1.85 ms (193 ns) |
1.24e9 |
5.45e10 (9.2 %) |
8.1 |
–* |
100,000 |
eagle host, termination loop, 1 thread |
1 |
12.9 ms (5.91 µs) |
1.77e8 |
7.78e9 (1.3 %) |
1 |
–* |
100,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
5.92 ms (11.8 µs) |
3.87e8 |
1.70e10 (2.9 %) |
8.1 |
–* |
100,000 |
eagle host, termination + compaction, 1 thread |
1 |
18.4 ms (57.7 µs) |
1.24e8 |
5.48e9 (0.93 %) |
1 |
–* |
100,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
11.4 ms (59.5 µs) |
2.01e8 |
8.84e9 (1.5 %) |
7.8 |
–* |
100,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
21.4 ms (73.5 µs) |
1.07e8 |
4.70e9 (0.8 %) |
1 |
–* |
100,000 |
Numba prange, 8 threads |
8 |
3.51 ms (917 ns) |
6.52e8 |
2.87e10 (4.9 %) |
8 |
–* |
100,000 |
JAX sharded over 8 CPU devices |
8 |
9.51 ms (67.2 µs) |
2.41e8 |
1.06e10 (1.8 %) |
11 |
–* |
100,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
40.6 ms (2.28 ms) |
5.63e7 |
2.48e9 (0.42 %) |
1.8 |
–* |
100,000 |
PyTorch masked, 8 threads |
8 |
79.5 ms (3.54 ms) |
2.88e7 |
1.27e9 (0.22 %) |
8 |
–* |
100,000 |
NumPy masked |
1 |
241 ms (420 µs) |
9.48e6 |
4.17e8 (0.071 %) |
1 |
–* |
100,000 |
multiprocessing, 8 x NumPy |
8 |
27.7 ms (2.07 ms) |
8.25e7 |
3.63e9 (0.62 %) |
6.5 |
–* |
1,000,000 |
eagle host, termination loop, up to 8 threads |
8 |
17.9 ms (53.5 µs) |
1.29e9 |
5.66e10 (9.6 %) |
7.7 |
61.8 MiB (1.8×) |
1,000,000 |
eagle host, termination loop, 1 thread |
1 |
129 ms (590 µs) |
1.78e8 |
7.83e9 (1.3 %) |
1 |
61.7 MiB (1.8×) |
1,000,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
57.9 ms (208 µs) |
3.97e8 |
1.75e10 (3 %) |
7.9 |
65.5 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction, 1 thread |
1 |
191 ms (840 µs) |
1.21e8 |
5.31e9 (0.9 %) |
1 |
65.6 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
127 ms (2.02 ms) |
1.81e8 |
7.95e9 (1.3 %) |
4.2 |
93.7 MiB (2.73×) |
1,000,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
232 ms (2.44 ms) |
9.91e7 |
4.36e9 (0.74 %) |
1 |
93.9 MiB (2.73×) |
1,000,000 |
Numba prange, 8 threads |
8 |
34.6 ms (110 µs) |
6.64e8 |
2.92e10 (5 %) |
8 |
68.1 MiB (1.98×) |
1,000,000 |
JAX sharded over 8 CPU devices |
8 |
108 ms (1.6 ms) |
2.14e8 |
9.40e9 (1.6 %) |
13 |
195 MiB (5.67×) |
1,000,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
244 ms (10.5 ms) |
9.41e7 |
4.14e9 (0.7 %) |
3.3 |
151 MiB (4.4×) |
1,000,000 |
PyTorch masked, 8 threads |
8 |
790 ms (26.4 ms) |
2.91e7 |
1.28e9 (0.22 %) |
8 |
266 MiB (7.76×) |
1,000,000 |
NumPy masked |
1 |
3.23 s (24.5 ms) |
7.13e6 |
3.14e8 (0.053 %) |
1 |
169 MiB (4.93×) |
1,000,000 |
multiprocessing, 8 x NumPy |
8 |
431 ms (7.17 ms) |
5.34e7 |
2.35e9 (0.4 %) |
6.7 |
233 MiB (6.78×) |
Fastest arm by median wall time:
N = 1,000: eagle host, termination loop, up to 8 threads (42.5 µs), then Numba prange, 8 threads (42.5 µs, 1× the time)
N = 10,000: eagle host, termination loop, up to 8 threads (213 µs), then Numba prange, 8 threads (359 µs, 1.69× the time)
N = 100,000: eagle host, termination loop, up to 8 threads (1.85 ms), then Numba prange, 8 threads (3.51 ms, 1.9× the time)
N = 1,000,000: eagle host, termination loop, up to 8 threads (17.9 ms), then Numba prange, 8 threads (34.6 ms, 1.94× the time)
Spread stop steps (log-uniform in [10, 1000]), up to 1000 steps
N |
arm |
threads |
wall (IQR) |
sample·steps/s |
useful FLOP/s (% peak) |
CPU busy |
peak memory (× minimum) |
|---|---|---|---|---|---|---|---|
1,000 |
eagle host, termination loop, up to 8 threads |
8 |
224 µs (804 ns) |
9.38e8 |
4.13e10 (7 %) |
8.1 |
–* |
1,000 |
eagle host, termination loop, 1 thread |
1 |
1.28 ms (4.31 µs) |
1.64e8 |
7.23e9 (1.2 %) |
1 |
–* |
1,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
4.78 ms (141 µs) |
4.40e7 |
1.94e9 (0.33 %) |
8 |
–* |
1,000 |
eagle host, termination + compaction, 1 thread |
1 |
2.07 ms (8.65 µs) |
1.01e8 |
4.46e9 (0.76 %) |
1 |
–* |
1,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
4.82 ms (24.1 µs) |
4.36e7 |
1.92e9 (0.33 %) |
8 |
–* |
1,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
2.39 ms (38.6 µs) |
8.79e7 |
3.87e9 (0.66 %) |
1 |
–* |
1,000 |
Numba prange, 8 threads |
8 |
397 µs (686 ns) |
5.30e8 |
2.33e10 (4 %) |
8 |
–* |
1,000 |
JAX sharded over 8 CPU devices |
8 |
601 µs (1.67 µs) |
3.50e8 |
1.54e10 (2.6 %) |
5.9 |
–* |
1,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
1.86 ms (10.6 µs) |
1.13e8 |
4.98e9 (0.85 %) |
0.95 |
–* |
1,000 |
PyTorch masked, 8 threads |
8 |
44.1 ms (116 µs) |
4.77e6 |
2.10e8 (0.036 %) |
1 |
–* |
1,000 |
NumPy masked |
1 |
25.7 ms (95.5 µs) |
8.18e6 |
3.60e8 (0.061 %) |
1 |
–* |
1,000 |
multiprocessing, 8 x NumPy |
8 |
15.9 ms (125 µs) |
1.32e7 |
5.83e8 (0.099 %) |
7.5 |
–* |
1,000 |
plain Python loops |
1 |
35.5 ms (182 µs) |
5.92e6 |
2.61e8 (0.044 %) |
1 |
–* |
10,000 |
eagle host, termination loop, up to 8 threads |
8 |
1.88 ms (3.34 µs) |
1.16e9 |
5.10e10 (8.7 %) |
8 |
–* |
10,000 |
eagle host, termination loop, 1 thread |
1 |
13.1 ms (37.7 µs) |
1.67e8 |
7.33e9 (1.2 %) |
1 |
–* |
10,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
8.36 ms (17.2 µs) |
2.61e8 |
1.15e10 (2 %) |
8 |
–* |
10,000 |
eagle host, termination + compaction, 1 thread |
1 |
9.88 ms (24 µs) |
2.21e8 |
9.72e9 (1.7 %) |
1 |
–* |
10,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
7.97 ms (29.4 µs) |
2.74e8 |
1.20e10 (2 %) |
8 |
–* |
10,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
10.8 ms (18.4 µs) |
2.03e8 |
8.92e9 (1.5 %) |
1 |
–* |
10,000 |
Numba prange, 8 threads |
8 |
3.7 ms (2.47 µs) |
5.89e8 |
2.59e10 (4.4 %) |
8 |
–* |
10,000 |
JAX sharded over 8 CPU devices |
8 |
4.84 ms (31.2 µs) |
4.51e8 |
1.98e10 (3.4 %) |
6.6 |
–* |
10,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
44.8 ms (3.66 ms) |
4.87e7 |
2.14e9 (0.36 %) |
1.7 |
–* |
10,000 |
PyTorch masked, 8 threads |
8 |
159 ms (263 µs) |
1.37e7 |
6.03e8 (0.1 %) |
1 |
–* |
10,000 |
NumPy masked |
1 |
150 ms (311 µs) |
1.45e7 |
6.40e8 (0.11 %) |
1 |
–* |
10,000 |
multiprocessing, 8 x NumPy |
8 |
34.6 ms (303 µs) |
6.30e7 |
2.77e9 (0.47 %) |
7.4 |
–* |
10,000 |
plain Python loops |
1 |
372 ms (1.61 ms) |
5.87e6 |
2.58e8 (0.044 %) |
1 |
–* |
100,000 |
eagle host, termination loop, up to 8 threads |
8 |
17.6 ms (14.4 µs) |
1.23e9 |
5.40e10 (9.2 %) |
8 |
–* |
100,000 |
eagle host, termination loop, 1 thread |
1 |
128 ms (227 µs) |
1.70e8 |
7.46e9 (1.3 %) |
1 |
–* |
100,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
36 ms (25.4 µs) |
6.02e8 |
2.65e10 (4.5 %) |
8 |
–* |
100,000 |
eagle host, termination + compaction, 1 thread |
1 |
132 ms (501 µs) |
1.64e8 |
7.23e9 (1.2 %) |
1 |
–* |
100,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
39.9 ms (119 µs) |
5.43e8 |
2.39e10 (4.1 %) |
8 |
–* |
100,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
94.1 ms (304 µs) |
2.30e8 |
1.01e10 (1.7 %) |
1 |
–* |
100,000 |
Numba prange, 8 threads |
8 |
34.9 ms (915 µs) |
6.21e8 |
2.73e10 (4.6 %) |
8 |
–* |
100,000 |
JAX sharded over 8 CPU devices |
8 |
90 ms (447 µs) |
2.41e8 |
1.06e10 (1.8 %) |
11 |
–* |
100,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
404 ms (75.2 ms) |
5.36e7 |
2.36e9 (0.4 %) |
1.8 |
–* |
100,000 |
PyTorch masked, 8 threads |
8 |
594 ms (1.45 ms) |
3.64e7 |
1.60e9 (0.27 %) |
8 |
–* |
100,000 |
NumPy masked |
1 |
2.15 s (5.02 ms) |
1.01e7 |
4.44e8 (0.075 %) |
1 |
–* |
100,000 |
multiprocessing, 8 x NumPy |
8 |
214 ms (258 µs) |
1.01e8 |
4.45e9 (0.76 %) |
7.6 |
–* |
1,000,000 |
eagle host, termination loop, up to 8 threads |
8 |
162 ms (5.69 ms) |
1.34e9 |
5.88e10 (10 %) |
8 |
61.8 MiB (1.8×) |
1,000,000 |
eagle host, termination loop, 1 thread |
1 |
1.27 s (7.23 ms) |
1.70e8 |
7.50e9 (1.3 %) |
1 |
61.8 MiB (1.8×) |
1,000,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
321 ms (588 µs) |
6.74e8 |
2.96e10 (5 %) |
8 |
65.5 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction, 1 thread |
1 |
1.37 s (1.74 ms) |
1.58e8 |
6.96e9 (1.2 %) |
1 |
65.6 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
362 ms (6.46 ms) |
5.97e8 |
2.63e10 (4.5 %) |
6.7 |
93.8 MiB (2.73×) |
1,000,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
933 ms (8.02 ms) |
2.32e8 |
1.02e10 (1.7 %) |
1 |
93.8 MiB (2.73×) |
1,000,000 |
Numba prange, 8 threads |
8 |
335 ms (9.31 ms) |
6.45e8 |
2.84e10 (4.8 %) |
8 |
68.3 MiB (1.99×) |
1,000,000 |
JAX sharded over 8 CPU devices |
8 |
1.03 s (6.38 ms) |
2.10e8 |
9.25e9 (1.6 %) |
14 |
195 MiB (5.67×) |
1,000,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
2.55 s (25 ms) |
8.49e7 |
3.74e9 (0.63 %) |
3.1 |
182 MiB (5.3×) |
1,000,000 |
PyTorch masked, 8 threads |
8 |
7.39 s (70.2 ms) |
2.93e7 |
1.29e9 (0.22 %) |
8 |
274 MiB (7.98×) |
1,000,000 |
NumPy masked |
1 |
29.1 s (350 ms) |
7.43e6 |
3.27e8 (0.056 %) |
1 |
169 MiB (4.93×) |
1,000,000 |
multiprocessing, 8 x NumPy |
8 |
3.69 s (11 ms) |
5.87e7 |
2.58e9 (0.44 %) |
7 |
234 MiB (6.83×) |
Fastest arm by median wall time:
N = 1,000: eagle host, termination loop, up to 8 threads (224 µs), then Numba prange, 8 threads (397 µs, 1.77× the time)
N = 10,000: eagle host, termination loop, up to 8 threads (1.88 ms), then Numba prange, 8 threads (3.7 ms, 1.97× the time)
N = 100,000: eagle host, termination loop, up to 8 threads (17.6 ms), then Numba prange, 8 threads (34.9 ms, 1.98× the time)
N = 1,000,000: eagle host, termination loop, up to 8 threads (162 ms), then eagle host, termination + compaction, up to 8 threads (321 ms, 1.98× the time)
Uniform stop step: every sample runs 1000 steps
N |
arm |
threads |
wall (IQR) |
sample·steps/s |
useful FLOP/s (% peak) |
CPU busy |
peak memory (× minimum) |
|---|---|---|---|---|---|---|---|
1,000 |
eagle host, termination loop, up to 8 threads |
8 |
165 µs (330 ns) |
6.07e9 |
2.67e11 (45 %) |
8.1 |
–* |
1,000 |
eagle host, termination loop, 1 thread |
1 |
963 µs (11.1 µs) |
1.04e9 |
4.57e10 (7.8 %) |
1 |
–* |
1,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
4.34 ms (56 µs) |
2.30e8 |
1.01e10 (1.7 %) |
8 |
–* |
1,000 |
eagle host, termination + compaction, 1 thread |
1 |
4.79 ms (116 µs) |
2.09e8 |
9.19e9 (1.6 %) |
1 |
–* |
1,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
4.07 ms (77.4 µs) |
2.46e8 |
1.08e10 (1.8 %) |
8 |
–* |
1,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
4.77 ms (26.7 µs) |
2.10e8 |
9.23e9 (1.6 %) |
1 |
–* |
1,000 |
Numba prange, 8 threads |
8 |
1.64 ms (2.29 µs) |
6.10e8 |
2.69e10 (4.6 %) |
8 |
–* |
1,000 |
JAX sharded over 8 CPU devices |
8 |
603 µs (2.49 µs) |
1.66e9 |
7.30e10 (12 %) |
6 |
–* |
1,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
1.88 ms (14.2 µs) |
5.32e8 |
2.34e10 (4 %) |
0.96 |
–* |
1,000 |
PyTorch masked, 8 threads |
8 |
43.8 ms (32.1 µs) |
2.29e7 |
1.01e9 (0.17 %) |
1 |
–* |
1,000 |
NumPy masked |
1 |
25.2 ms (125 µs) |
3.96e7 |
1.74e9 (0.3 %) |
1 |
–* |
1,000 |
multiprocessing, 8 x NumPy |
8 |
16 ms (82 µs) |
6.25e7 |
2.75e9 (0.47 %) |
7.7 |
–* |
1,000 |
plain Python loops |
1 |
169 ms (2.39 ms) |
5.91e6 |
2.60e8 (0.044 %) |
1 |
–* |
10,000 |
eagle host, termination loop, up to 8 threads |
8 |
1.31 ms (2.92 µs) |
7.65e9 |
3.36e11 (57 %) |
8 |
–* |
10,000 |
eagle host, termination loop, 1 thread |
1 |
9.17 ms (4.22 µs) |
1.09e9 |
4.80e10 (8.1 %) |
1 |
–* |
10,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
11.6 ms (178 µs) |
8.59e8 |
3.78e10 (6.4 %) |
8 |
–* |
10,000 |
eagle host, termination + compaction, 1 thread |
1 |
38.5 ms (93 µs) |
2.60e8 |
1.14e10 (1.9 %) |
1 |
–* |
10,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
11.5 ms (250 µs) |
8.73e8 |
3.84e10 (6.5 %) |
8 |
–* |
10,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
38.5 ms (26 µs) |
2.60e8 |
1.14e10 (1.9 %) |
1 |
–* |
10,000 |
Numba prange, 8 threads |
8 |
15.9 ms (223 µs) |
6.28e8 |
2.76e10 (4.7 %) |
8 |
–* |
10,000 |
JAX sharded over 8 CPU devices |
8 |
4.68 ms (41.5 µs) |
2.14e9 |
9.40e10 (16 %) |
6.5 |
–* |
10,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
43.9 ms (2.92 ms) |
2.28e8 |
1.00e10 (1.7 %) |
1.7 |
–* |
10,000 |
PyTorch masked, 8 threads |
8 |
145 ms (2.12 ms) |
6.91e7 |
3.04e9 (0.52 %) |
1 |
–* |
10,000 |
NumPy masked |
1 |
137 ms (2.58 ms) |
7.27e7 |
3.20e9 (0.54 %) |
1 |
–* |
10,000 |
multiprocessing, 8 x NumPy |
8 |
33 ms (633 µs) |
3.03e8 |
1.33e10 (2.3 %) |
7.4 |
–* |
100,000 |
eagle host, termination loop, up to 8 threads |
8 |
12.2 ms (260 µs) |
8.22e9 |
3.61e11 (61 %) |
8 |
–* |
100,000 |
eagle host, termination loop, 1 thread |
1 |
91.3 ms (327 µs) |
1.10e9 |
4.82e10 (8.2 %) |
1 |
–* |
100,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
95.4 ms (1.05 ms) |
1.05e9 |
4.61e10 (7.8 %) |
8 |
–* |
100,000 |
eagle host, termination + compaction, 1 thread |
1 |
381 ms (1.89 ms) |
2.62e8 |
1.15e10 (2 %) |
1 |
–* |
100,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
94.8 ms (547 µs) |
1.05e9 |
4.64e10 (7.9 %) |
8 |
–* |
100,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
382 ms (1.16 ms) |
2.62e8 |
1.15e10 (2 %) |
1 |
–* |
100,000 |
Numba prange, 8 threads |
8 |
160 ms (1.35 ms) |
6.24e8 |
2.75e10 (4.7 %) |
8 |
–* |
100,000 |
JAX sharded over 8 CPU devices |
8 |
89.4 ms (554 µs) |
1.12e9 |
4.92e10 (8.4 %) |
11 |
–* |
100,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
429 ms (26.1 ms) |
2.33e8 |
1.03e10 (1.7 %) |
1.8 |
–* |
100,000 |
PyTorch masked, 8 threads |
8 |
532 ms (2.99 ms) |
1.88e8 |
8.27e9 (1.4 %) |
8 |
–* |
100,000 |
NumPy masked |
1 |
1.96 s (5.07 ms) |
5.11e7 |
2.25e9 (0.38 %) |
1 |
–* |
100,000 |
multiprocessing, 8 x NumPy |
8 |
191 ms (465 µs) |
5.25e8 |
2.31e10 (3.9 %) |
7.7 |
–* |
1,000,000 |
eagle host, termination loop, up to 8 threads |
8 |
123 ms (1.17 ms) |
8.13e9 |
3.58e11 (61 %) |
8 |
61.8 MiB (1.8×) |
1,000,000 |
eagle host, termination loop, 1 thread |
1 |
915 ms (1.01 ms) |
1.09e9 |
4.81e10 (8.2 %) |
1 |
62 MiB (1.81×) |
1,000,000 |
eagle host, termination + compaction, up to 8 threads |
8 |
885 ms (3.53 ms) |
1.13e9 |
4.97e10 (8.4 %) |
8 |
65.3 MiB (1.9×) |
1,000,000 |
eagle host, termination + compaction, 1 thread |
1 |
3.77 s (4.13 ms) |
2.65e8 |
1.17e10 (2 %) |
1 |
65.6 MiB (1.91×) |
1,000,000 |
eagle host, termination + compaction + reorder, up to 8 threads |
8 |
919 ms (6.77 ms) |
1.09e9 |
4.79e10 (8.1 %) |
8 |
72.9 MiB (2.12×) |
1,000,000 |
eagle host, termination + compaction + reorder, 1 thread |
1 |
3.78 s (1.77 ms) |
2.64e8 |
1.16e10 (2 %) |
1 |
73.3 MiB (2.14×) |
1,000,000 |
Numba prange, 8 threads |
8 |
1.59 s (8.36 ms) |
6.27e8 |
2.76e10 (4.7 %) |
8 |
60.7 MiB (1.77×) |
1,000,000 |
JAX sharded over 8 CPU devices |
8 |
1.03 s (6.57 ms) |
9.75e8 |
4.29e10 (7.3 %) |
14 |
186 MiB (5.42×) |
1,000,000 |
JAX jit + vmap(while_loop), 1 device |
8 |
2.4 s (49.7 ms) |
4.16e8 |
1.83e10 (3.1 %) |
3.2 |
167 MiB (4.85×) |
1,000,000 |
PyTorch masked, 8 threads |
8 |
7.24 s (245 ms) |
1.38e8 |
6.07e9 (1 %) |
8 |
259 MiB (7.55×) |
1,000,000 |
NumPy masked |
1 |
28.4 s (1.2 s) |
3.52e7 |
1.55e9 (0.26 %) |
1 |
163 MiB (4.73×) |
1,000,000 |
multiprocessing, 8 x NumPy |
8 |
3.6 s (27.1 ms) |
2.78e8 |
1.22e10 (2.1 %) |
6.8 |
222 MiB (6.48×) |
Fastest arm by median wall time:
N = 1,000: eagle host, termination loop, up to 8 threads (165 µs), then JAX sharded over 8 CPU devices (603 µs, 3.66× the time)
N = 10,000: eagle host, termination loop, up to 8 threads (1.31 ms), then JAX sharded over 8 CPU devices (4.68 ms, 3.58× the time)
N = 100,000: eagle host, termination loop, up to 8 threads (12.2 ms), then JAX sharded over 8 CPU devices (89.4 ms, 7.34× the time)
N = 1,000,000: eagle host, termination loop, up to 8 threads (123 ms), then eagle host, termination + compaction, up to 8 threads (885 ms, 7.19× the time)
Cells not run
plain Python loops: plain Python runs only at N <= 10,000 with at most 2,500,000 sample-steps per run (spread S=100 N=100,000; spread S=100 N=1,000,000; spread S=1000 N=100,000; spread S=1000 N=1,000,000; uniform S=1000 N=10,000; uniform S=1000 N=100,000; uniform S=1000 N=1,000,000).
Where each tool fits
Each tool below is written the way its users write it. Two loop structures appear: per step over the batch (eagle’s termination loops, JAX, PyTorch, NumPy, multiprocessing) and per sample over its steps (Numba, plain Python). The per-sample structure touches each sample’s state once for all its steps; the per-step structure streams the batch through memory every step, which is the structure the GPU card’s kernels use. Ratios below are wall-time ratios at the largest N each arm ran (below 1 = less time than eagle at 8 threads).
eagle host, termination loop, up to 8 threads. The GPU card’s eagle graph arm, unchanged, on the host: the same hawk step kernel, eagle.deploy and eagle.until_done call, run by eagle’s OpenMP host team; one code base for GPUs and CPUs. On this card: spread S=100, N=1,000,000: 17.9 ms; spread S=1000, N=1,000,000: 162 ms; uniform S=1000, N=1,000,000: 123 ms. Fastest in 12 of 12 cells.
eagle host, termination loop, 1 thread. The same loop on one thread: the difference to the 8-thread row is the threading alone. On this card: spread S=100, N=1,000,000: 129 ms (7.23× eagle 8-thread time); spread S=1000, N=1,000,000: 1.27 s (7.84× eagle 8-thread time); uniform S=1000, N=1,000,000: 915 ms (7.44× eagle 8-thread time). Fastest in 0 of 12 cells.
eagle host, termination + compaction, up to 8 threads. The GPU card’s compaction arm on the host: the launch covers only the samples still running, the map rebuilt every 16 steps when a sample finished. On this card: spread S=100, N=1,000,000: 57.9 ms (3.24× eagle 8-thread time); spread S=1000, N=1,000,000: 321 ms (1.98× eagle 8-thread time); uniform S=1000, N=1,000,000: 885 ms (7.19× eagle 8-thread time). Fastest in 0 of 12 cells.
eagle host, termination + compaction, 1 thread. The compaction loop on one thread: the difference to the 8-thread row is the threading alone. On this card: spread S=100, N=1,000,000: 191 ms (10.7× eagle 8-thread time); spread S=1000, N=1,000,000: 1.37 s (8.44× eagle 8-thread time); uniform S=1000, N=1,000,000: 3.77 s (30.7× eagle 8-thread time). Fastest in 0 of 12 cells.
eagle host, termination + compaction + reorder, up to 8 threads. The GPU card’s reorder arm on the host: the live samples moved to the front of every plane when they spread thin, restored to sample order at the end (inside the wall). On this card: spread S=100, N=1,000,000: 127 ms (7.12× eagle 8-thread time); spread S=1000, N=1,000,000: 362 ms (2.24× eagle 8-thread time); uniform S=1000, N=1,000,000: 919 ms (7.47× eagle 8-thread time). Fastest in 0 of 12 cells.
eagle host, termination + compaction + reorder, 1 thread. The reorder loop on one thread: the difference to the 8-thread row is the threading alone. On this card: spread S=100, N=1,000,000: 232 ms (13× eagle 8-thread time); spread S=1000, N=1,000,000: 933 ms (5.76× eagle 8-thread time); uniform S=1000, N=1,000,000: 3.78 s (30.8× eagle 8-thread time). Fastest in 0 of 12 cells.
Numba prange, 8 threads. Compiles a plain Python loop to machine code with no separate language; each sample keeps its state in registers across all of its steps and stops on its own, so finished samples cost nothing. On this card: spread S=100, N=1,000,000: 34.6 ms (1.94× eagle 8-thread time); spread S=1000, N=1,000,000: 335 ms (2.07× eagle 8-thread time); uniform S=1000, N=1,000,000: 1.59 s (13× eagle 8-thread time). Fastest in 0 of 12 cells.
JAX sharded over 8 CPU devices. The JAX arm spread over 8 CPU devices: the batch is sharded and each shard runs its own vmapped while loop, with no cross-shard step synchronisation. On this card: spread S=100, N=1,000,000: 108 ms (6.02× eagle 8-thread time); spread S=1000, N=1,000,000: 1.03 s (6.35× eagle 8-thread time); uniform S=1000, N=1,000,000: 1.03 s (8.34× eagle 8-thread time). Fastest in 0 of 12 cells.
JAX jit + vmap(while_loop), 1 device. Compiles the whole loop once and batches a per-sample while loop with vmap; the same code runs on GPUs and differentiates with jax.grad. On this card: spread S=100, N=1,000,000: 244 ms (13.7× eagle 8-thread time); spread S=1000, N=1,000,000: 2.55 s (15.7× eagle 8-thread time); uniform S=1000, N=1,000,000: 2.4 s (19.5× eagle 8-thread time). Fastest in 0 of 12 cells.
PyTorch masked, 8 threads. Runs the masked array version on a multi-threaded tensor library; it fits where the computation already lives next to a PyTorch model or needs autograd. On this card: spread S=100, N=1,000,000: 790 ms (44.2× eagle 8-thread time); spread S=1000, N=1,000,000: 7.39 s (45.6× eagle 8-thread time); uniform S=1000, N=1,000,000: 7.24 s (58.9× eagle 8-thread time). Fastest in 0 of 12 cells.
NumPy masked. No compile step and no dependency beyond NumPy; masked array code computes every sample every step, so it fits dense batches where all samples run to the end (the uniform configuration). On this card: spread S=100, N=1,000,000: 3.23 s (181× eagle 8-thread time); spread S=1000, N=1,000,000: 29.1 s (180× eagle 8-thread time); uniform S=1000, N=1,000,000: 28.4 s (231× eagle 8-thread time). Fastest in 0 of 12 cells.
multiprocessing, 8 x NumPy. Spreads NumPy over processes with the standard library alone; each chunk stops once its own samples are done. On this card: spread S=100, N=1,000,000: 431 ms (24.1× eagle 8-thread time); spread S=1000, N=1,000,000: 3.69 s (22.8× eagle 8-thread time); uniform S=1000, N=1,000,000: 3.6 s (29.3× eagle 8-thread time). Fastest in 0 of 12 cells.
plain Python loops. Nothing to install and each line can be stepped in a debugger; run only at small N here. On this card: spread S=100, N=10,000: 39.5 ms (186× eagle 8-thread time); spread S=1000, N=10,000: 372 ms (197× eagle 8-thread time); uniform S=1000, N=1,000: 169 ms (1.03e+03× eagle 8-thread time). Fastest in 0 of 12 cells.
Compile time
Its own pass, never part of any wall. Each measurement is a fresh process: the imports run untimed, then the timer spans the first call to a ready kernel (trace + code generation + compile, or the cache lookup). Cold: an empty cache (hawk’s compile cache pointed at a fresh directory through HAWK_CACHE_DIR; Numba with cache=True into a fresh NUMBA_CACHE_DIR; JAX with its persistent compilation cache in a fresh directory). Warm: a new process on the cache the first cold run populated (hawk cache hit; Numba’s on-disk cache; JAX’s persistent cache with the minimum compile time and entry size set to 0). hawk: eagle.deploy of the kernel (tracing, C++ emission, the g++ host build, eagle’s plan); the active-set variant is a second build of the same step. Numba: the first call on a 64-sample batch (compile plus a negligible run). JAX: lower + compile from abstract shapes, once per N (XLA specialises on the shape). Medians of 3 cold and 3 warm processes.
arm |
compiled |
cold (median of 3) |
warm (median of 3) |
|---|---|---|---|
eagle host, termination loop, up to 8 threads |
hawk step kernel (host build) |
734 ms |
24.3 ms |
eagle host, termination + compaction, up to 8 threads |
hawk active-set step kernel (host build) |
631 ms |
20.2 ms |
Numba prange, 8 threads |
Numba parallel loop (first call, 64 samples) |
478 ms |
169 ms |
JAX sharded over 8 CPU devices |
XLA executable for N = 1,000 |
78.9 ms |
30.4 ms |
JAX sharded over 8 CPU devices |
XLA executable for N = 10,000 |
81.1 ms |
30.3 ms |
JAX sharded over 8 CPU devices |
XLA executable for N = 100,000 |
82.4 ms |
31 ms |
JAX sharded over 8 CPU devices |
XLA executable for N = 1,000,000 |
85.5 ms |
31 ms |
JAX jit + vmap(while_loop), 1 device |
XLA executable for N = 1,000 |
73.6 ms |
32.3 ms |
JAX jit + vmap(while_loop), 1 device |
XLA executable for N = 10,000 |
76.8 ms |
32.5 ms |
JAX jit + vmap(while_loop), 1 device |
XLA executable for N = 100,000 |
79.7 ms |
32.6 ms |
JAX jit + vmap(while_loop), 1 device |
XLA executable for N = 1,000,000 |
74.3 ms |
32.7 ms |
No compile step: PyTorch masked, 8 threads, NumPy masked, multiprocessing, 8 x NumPy, plain Python loops.
Same build as another row: eagle host, termination loop, 1 thread (as eagle host, termination loop, up to 8 threads); eagle host, termination + compaction, 1 thread (as eagle host, termination + compaction, up to 8 threads); eagle host, termination + compaction + reorder, up to 8 threads (as eagle host, termination + compaction, up to 8 threads); eagle host, termination + compaction + reorder, 1 thread (as eagle host, termination + compaction, up to 8 threads).
Memory method
A separate pass after the timing pass. Each (arm, N, configuration) runs in a fresh process: imports, then a warm-up run on a 64-sample batch (every JIT compiles here; JAX also compiles for shape N from abstract shapes), then the baseline (resident set size, VmRSS) and a reset of the process’s peak mark (/proc/self/clear_refs, so ru_maxrss restarts from the baseline); then the batch’s inputs are generated and one full run done. Peak memory = ru_maxrss at the end minus the baseline; for multiprocessing, plus each pool process’s own peak above its baseline (shared pages counted once per process). Minimum = the state the workload needs, N × ((2 state + 2 float64 scalars: omega, zeta) × 8 + 4 bytes for the stop step as a 32-bit integer) = 36 bytes per sample; the factor is peak / minimum. Compile overhead = baseline minus the resident size before the arm’s imports-and-warm-up. Below about 1 MiB the allocator reuses memory already resident at the baseline, so small-N peaks can read below the minimum.
Compile/import overhead (resident memory added by the warm-up compile, measured once per arm at the largest N): eagle host, termination loop, up to 8 threads 25 MiB; eagle host, termination loop, 1 thread 25 MiB; eagle host, termination + compaction, up to 8 threads 25.4 MiB; eagle host, termination + compaction, 1 thread 25.1 MiB; eagle host, termination + compaction + reorder, up to 8 threads 25.2 MiB; eagle host, termination + compaction + reorder, 1 thread 25.1 MiB; Numba prange, 8 threads 141 MiB; JAX sharded over 8 CPU devices 227 MiB; JAX jit + vmap(while_loop), 1 device 224 MiB; PyTorch masked, 8 threads 210 MiB; NumPy masked 15.6 MiB; multiprocessing, 8 x NumPy 16.6 MiB; plain Python loops 15.4 MiB.
CPU peak memory is shown for N = 1,000,000 only, net of the benchmark harness’s own copies (36 bytes per sample on eagle’s arms). At smaller N this harness cannot resolve it: each measuring process inherits the launching process’s peak, so smaller readings show that floor rather than the workload. The cards keep the raw readings; the next card run measures every size directly.
spread (up to 100 steps)
Fastest arm by median wall time:
N = 1,000: eagle host, termination loop, up to 8 threads (42.5 µs), then Numba prange, 8 threads (42.5 µs, 1× the time)
N = 10,000: eagle host, termination loop, up to 8 threads (213 µs), then Numba prange, 8 threads (359 µs, 1.69× the time)
N = 100,000: eagle host, termination loop, up to 8 threads (1.85 ms), then Numba prange, 8 threads (3.51 ms, 1.9× the time)
N = 1,000,000: eagle host, termination loop, up to 8 threads (17.9 ms), then Numba prange, 8 threads (34.6 ms, 1.94× the time)
spread (up to 1000 steps)
Fastest arm by median wall time:
N = 1,000: eagle host, termination loop, up to 8 threads (224 µs), then Numba prange, 8 threads (397 µs, 1.77× the time)
N = 10,000: eagle host, termination loop, up to 8 threads (1.88 ms), then Numba prange, 8 threads (3.7 ms, 1.97× the time)
N = 100,000: eagle host, termination loop, up to 8 threads (17.6 ms), then Numba prange, 8 threads (34.9 ms, 1.98× the time)
N = 1,000,000: eagle host, termination loop, up to 8 threads (162 ms), then eagle host, termination + compaction, up to 8 threads (321 ms, 1.98× the time)
uniform (up to 1000 steps)
Fastest arm by median wall time:
N = 1,000: eagle host, termination loop, up to 8 threads (165 µs), then JAX sharded over 8 CPU devices (603 µs, 3.66× the time)
N = 10,000: eagle host, termination loop, up to 8 threads (1.31 ms), then JAX sharded over 8 CPU devices (4.68 ms, 3.58× the time)
N = 100,000: eagle host, termination loop, up to 8 threads (12.2 ms), then JAX sharded over 8 CPU devices (89.4 ms, 7.34× the time)
N = 1,000,000: eagle host, termination loop, up to 8 threads (123 ms), then eagle host, termination + compaction, up to 8 threads (885 ms, 7.19× the time)
device |
compute capability / CPU model |
date |
|---|---|---|
AMD EPYC 9B45 |
AMD EPYC 9B45 |
2026-10-13 |
AMD EPYC 7B13 |
AMD EPYC 7B13 |
2026-10-13 |
Google Axion (Arm Neoverse-V2) |
aarch64 |
2026-10-13 |
Intel Xeon Platinum 8581C |
INTEL(R) XEON(R) PLATINUM 8581C CPU @ 2.30GHz |
2026-10-13 |