Benchmarks

Each kit against the Python package it ports, same inputs, on an Apple M-series Mac. Tables are copied from each kit's docs/benchmarks.md at its latest tag; where the kit is slower than Python, the table says so.

FormatoKit 0.9.0

Full file and method

First table, 2026-10-21 (0.9.0 working tree). Small and typical sizes run in the session; the large rows are the nightly review's (tools/benchmarks/bench.py run --large). Same machine, one case at a time, no other benchmark running.


Machine	Apple M5 Pro, arm64, macOS 27.0.1
Kit	Swift 6.4 (swiftlang-6.4.0.34.1), `swift build -c release`, FormatoKit working tree on `main` after 0.8.0
Reference	RosettaSciIO 0.14.1.dev96+g049e7d707 (the kit's pin), Python 3.14.7, NumPy 2.5.4, h5py 3.16.0, tifffile 2026.9.20, numba 0.68.0, sparse 0.19.2 (`References/rsciio-venv`)
Operation	open the file and have all its data in memory: `file_reader(lazy=False)` against the kit's signal `load()` (Velox SI: `VeloxSpectrumImageSignals`; HDF5 4D cube: h5py `dataset[...]`, the floor, py4DSTEM is not an rsciio plugin)
Protocol	one fresh process per side and case; one warm-up read, then five timed reads; median, with min to max in brackets; peak resident size of that process (includes the interpreter, NumPy and numba on the Python side)
Same work	both sides' data sums are compared (inputs are integer-valued, so the sums are exact)
Inputs	generated by `tools/benchmarks/bench.py` into a scratch folder (planted DM4, Velox, TIFF, MRC, MIB, EMPAD, HDF5 and MSA files); nothing of the owner's data

Format	Input	Size	Kit CPU: median (min to max)	RosettaSciIO: median (min to max)	Python / kit	Peak RSS kit / Python	Same data sum
DM4 image	512² f32	small	0.3 ms (0.3 ms to 0.4 ms)	0.2 ms (0.2 ms to 0.3 ms)	0.67x	15 / 106 MB	yes
DM4 image	2048² f32	typical	1.9 ms (1.7 ms to 2.3 ms)	1.0 ms (1.0 ms to 1.7 ms)	0.54x	85 / 169 MB	yes
DM4 image	4096² f32	large	nightly	nightly	nightly
DM4 4D	16²x64² u16	small	0.3 ms (0.3 ms to 0.4 ms)	0.4 ms (0.3 ms to 0.4 ms)	1.03x	22 / 115 MB	yes
DM4 4D	32²x128² u16	typical	3.3 ms (2.9 ms to 4.2 ms)	2.1 ms (1.9 ms to 3.1 ms)	0.63x	227 / 303 MB	yes
DM4 4D	128²x128² u16	large	nightly	nightly	nightly
Velox image	1024² x1 u16	small	1.3 ms (1.2 ms to 1.6 ms)	0.7 ms (0.7 ms to 1.1 ms)	0.57x	36 / 98 MB	yes
Velox image	2048² x5 u16	typical	9.9 ms (9.7 ms to 10.3 ms)	8.5 ms (8.4 ms to 8.8 ms)	0.86x	409 / 567 MB	yes
Velox image	4096² x10 u16	large	nightly	nightly	nightly
Velox spectrum	4096 ch u32	small	0.3 ms (0.2 ms to 0.4 ms)	0.3 ms (0.3 ms to 0.5 ms)	1.02x	14 / 72 MB	yes
Velox SI (EDS)	16² px, 4096 ch, 100 ev/px	small	0.7 ms (0.6 ms to 0.8 ms)	1.0 ms (0.9 ms to 1.5 ms)	1.39x	30 / 173 MB	yes
Velox SI (EDS)	128² px, 4096 ch, 100 ev/px	typical	4.0 ms (3.9 ms to 14.4 ms)	13.2 ms (12.1 ms to 15.0 ms)	3.33x	823 / 1508 MB	yes
Velox SI (EDS)	512² px, 4096 ch, 100 ev/px	large	nightly	nightly	nightly
TIFF	1024² u16	small	0.5 ms (0.5 ms to 0.6 ms)	0.3 ms (0.3 ms to 0.3 ms)	0.61x	20 / 62 MB	yes
TIFF	4096² u16	typical	4.3 ms (4.2 ms to 5.5 ms)	2.0 ms (1.7 ms to 3.0 ms)	0.46x	209 / 250 MB	yes
TIFF	8192² u16	large	nightly	nightly	nightly
MRC	512² x1 f32	small	0.2 ms (0.2 ms to 0.3 ms)	1.1 ms (0.9 ms to 1.4 ms)	4.87x	11 / 109 MB	yes
MRC	1024² x10 f32	typical	6.0 ms (5.3 ms to 6.9 ms)	8.8 ms (4.2 ms to 10.7 ms)	1.46x	196 / 311 MB	yes
MRC	2048² x20 f32	large	nightly	nightly	nightly
MIB	256² x64 U16	small	2.0 ms (1.8 ms to 2.4 ms)	3.0 ms (2.7 ms to 3.3 ms)	1.54x	61 / 200 MB	yes
MIB	256² x1024 U16	typical	35.0 ms (32.3 ms to 37.7 ms)	28.8 ms (28.5 ms to 29.6 ms)	0.82x	879 / 1190 MB	yes
MIB	256² x16384 U16	large	nightly	nightly	nightly
EMPAD	16² scan, 128x130 f32	small	1.5 ms (1.3 ms to 2.2 ms)	1.4 ms (1.2 ms to 1.9 ms)	0.88x	75 / 126 MB	yes
EMPAD	48² scan, 128x130 f32	typical	13.7 ms (12.0 ms to 19.6 ms)	24.4 ms (23.4 ms to 25.3 ms)	1.79x	611 / 799 MB	yes
EMPAD	128² scan, 128x130 f32	large	nightly	nightly	nightly
HDF5 4D cube	16²x64² f32	small	0.7 ms (0.7 ms to 0.9 ms)	0.8 ms (0.8 ms to 0.8 ms)	1.09x	41 / 70 MB	yes
HDF5 4D cube	32²x128² f32	typical	7.1 ms (6.3 ms to 9.1 ms)	8.2 ms (6.9 ms to 11.3 ms)	1.14x	336 / 377 MB	yes
HDF5 4D cube	64²x256² f32	large	nightly	nightly	nightly
MSA	4096 points	small	1.1 ms (1.1 ms to 1.1 ms)	0.9 ms (0.9 ms to 0.9 ms)	0.81x	9 / 43 MB	yes
MSA	65 536 points	typical	16.6 ms (16.5 ms to 17.0 ms)	13.6 ms (13.2 ms to 14.1 ms)	0.82x	21 / 50 MB	yes

Reading it: "Python / kit" above 1 is the kit faster. The Velox SI row is the stream decode (RosettaSciIO: numba-jitted, warm). The MIB and MRC rows are byte-swap and copy at memory speed; MIB reads were 934 ms at the first measurement (see below).

Backend decision per function (D16)

Readers stay on the CPU: they are bound by file reads and memory copies, there is no arithmetic for a GPU to take over, and host-to-device copies of a freshly read array cost more than the read. Measured break-even: none exists, no Metal path is a work item for any reader. Threading and mmap: only the 4D loads above 100 MB could split a read over cores; at the typical size the 4D rows are already within 1.1x of h5py and of RosettaSciIO, so the question belongs to the nightly large rows (open-items).

Function	Backend	Why
DM, TIFF, MRC, MIB, EMPAD, HDF5 4D, Velox images and spectra, MSA	CPU only	read-bound; no per-element arithmetic beyond byte swaps
Velox spectrum-image decode	CPU only	sequential stream with data-dependent branches (markers); the kit is already 3.3x RosettaSciIO's numba loop at 128²

Slower than RosettaSciIO at the typical size

Listed first in docs/open-items.md with the cause. Found and fixed in this session (diagnosed with sample, then measured): MIB 934 ms to 35 ms (an ISO8601DateFormatter built per frame header, 1024 times, plus a per-pixel Data.append); MSA 111 ms to 17 ms (per-line allocation in the first reader).

ImagoKit 0.6.0

Full file and method

Run 2026-10-21 on the audit session (ImagoKit 0.6.0 sources). Header. Apple M5 Pro, macOS (Darwin 27.0.0), Swift 6.4 (swiftlang-6.4.0.34.1), swift test -c release; Python 3.12.14 from refs/skimage with NumPy 2.5.4, SciPy 1.18.1, scikit-image 0.26.0 (single-threaded np.fft/ndimage). Method. The same function, dtype and input size on both sides (uniform random pixels 30 to 70); one warm-up call, then five timed calls; the table shows the median and (min–max) in milliseconds. FFTs reuse their plan (FFT2D, FFT2DDouble; NumPy caches its own); rfft2, butterworth, the shifts, registrations and filters are the public one-call API. Caveat. The five other kit audits ran on the machine at the same time (load average 11 to 25 during these runs, 6 to 8 before): absolute times are noisy, and both sides were measured in the same minutes so the ratios are comparable but not exact. The sizes are the small (128²) and typical (512²) ones plus one non-power-of-two (500²); masked registration, which costs eleven transforms of (2n−1)², is at 64² and 256². Large sizes (2048², 4096², 8192²) are nightly: IMAGO_BENCH_LARGE=1 and --large add them; they were not run here.

Reproduce (under the suite's memory cap, one benchmark at a time machine-wide):

```sh

IMAGO_BENCH=1 IMAGO_BENCH_OUT=/tmp/swift.json ../tools/memcap.sh -- swift test -c release --filter ImagoBenchmarks ../tools/memcap.sh -- ../refs/skimage/bin/python -I -B tools/benchmarks/bench.py /tmp/python.json python3 tools/benchmarks/report.py /tmp/swift.json /tmp/python.json

```

The harness is Tests/ImagoBenchmarks/Benchmarks.swift (its one test returns at once unless IMAGO_BENCH=1); tools/benchmarks/bench.py is the Python side. "kit / Python" below 1 means the kit is faster.

Kit CPU against Python

function	dtype	size	kit CPU ms: median (min–max)	Python ms: median (min–max)	kit / Python
butterworth	float32	128²	0.152 (0.151–0.158)	0.0788 (0.0755–0.102)	1.92
butterworth	float32	512²	2.61 (2.57–2.64)	1.12 (1.06–1.23)	2.34
butterworth	float64	128²	0.283 (0.28–0.311)	0.0993 (0.0956–0.121)	2.85
butterworth	float64	512²	6.44 (6.34–6.66)	2.27 (2.02–2.64)	2.83
difference_of_gaussians	float32	128²	0.286 (0.279–0.288)	0.262 (0.242–0.274)	1.09
difference_of_gaussians	float32	512²	4.46 (4.45–4.53)	5.29 (5.16–5.49)	0.84
fft2	float32	128²	0.0191 (0.0175–0.0612)	0.0867 (0.0855–0.0941)	0.22
fft2	float32	500²	1.89 (1.84–1.97)	3.11 (2.68–3.37)	0.61
fft2	float32	512²	0.537 (0.528–0.537)	3.61 (3.42–3.77)	0.15
fft2	float64	128²	0.0695 (0.0673–0.093)	0.0759 (0.0716–0.0761)	0.92
fft2	float64	500²	4.63 (4.55–4.74)	1.94 (1.78–1.99)	2.38
fft2	float64	512²	2.14 (2.06–2.23)	2.66 (2.45–2.74)	0.81
gaussian_filter sigma=3	float32	128²	0.139 (0.136–0.157)	0.127 (0.124–0.136)	1.09
gaussian_filter sigma=3	float32	512²	2.22 (2.18–2.23)	2.52 (2.51–2.61)	0.88
gaussian_filter sigma=3	float64	128²	0.125 (0.124–0.129)	0.145 (0.133–0.191)	0.86
gaussian_filter sigma=3	float64	512²	1.96 (1.94–2.01)	3.09 (2.96–3.18)	0.63
label connectivity=2	bool	128²	0.154 (0.126–0.231)	0.117 (0.0848–0.167)	1.32
label connectivity=2	bool	512²	4.37 (4.2–4.64)	2.06 (2.04–2.1)	2.13
masked phase_cross_correlation	float32	64²	1.66 (1.65–1.74)	0.704 (0.693–0.789)	2.36
masked phase_cross_correlation	float32	256²	29.6 (29.1–30.7)	12.4 (12.2–13)	2.38
masked phase_cross_correlation	float64	64²	3.39 (3.2–4.65)	1.03 (1.02–1.28)	3.27
masked phase_cross_correlation	float64	256²	66.5 (64.3–67.5)	19.1 (18–19.7)	3.48
phase_cross_correlation	float32	128²	0.136 (0.135–0.15)	0.177 (0.173–0.207)	0.77
phase_cross_correlation	float32	512²	2.62 (2.56–2.63)	3.45 (3.19–3.63)	0.76
phase_cross_correlation	float64	128²	0.294 (0.293–0.334)	0.273 (0.266–0.278)	1.08
phase_cross_correlation	float64	512²	7.57 (7.51–8.34)	6.07 (5.52–6.59)	1.25
phase_cross_correlation upsample=10	float32	128²	0.317 (0.306–0.318)	0.241 (0.237–0.251)	1.32
phase_cross_correlation upsample=10	float32	512²	5.25 (5.19–5.29)	4.18 (3.83–4.61)	1.26
profile_line order=1 width=5	float32	128²	0.0114 (0.0112–0.0148)	0.601 (0.594–0.634)	0.02
profile_line order=1 width=5	float32	512²	0.0449 (0.0444–0.0474)	2.43 (2.42–2.71)	0.02
profile_line order=3 width=5	float32	128²	0.516 (0.502–0.527)	0.801 (0.794–0.928)	0.64
profile_line order=3 width=5	float32	512²	5.33 (5.28–5.44)	7.44 (7.21–7.6)	0.72
rfft2	float32	128²	0.0329 (0.0311–0.035)	0.0405 (0.0401–0.0433)	0.81
rfft2	float32	512²	0.586 (0.509–0.598)	1.52 (1.29–1.86)	0.39
rfft2	float64	128²	0.0799 (0.0755–0.0865)	0.0313 (0.031–0.0322)	2.55
rfft2	float64	512²	2.12 (2.07–2.22)	0.791 (0.724–0.888)	2.68
shift order=1	float32	128²	0.0466 (0.0464–0.0478)	0.158 (0.154–0.159)	0.29
shift order=1	float32	512²	0.651 (0.587–0.895)	2.51 (2.21–2.52)	0.26
shift order=3	float32	128²	0.491 (0.479–0.496)	0.641 (0.605–0.655)	0.77
shift order=3	float32	512²	7.69 (7.67–7.79)	10.9 (10.6–11.1)	0.71
uniform_filter size=5	float32	128²	0.0745 (0.0723–0.0846)	0.0368 (0.0367–0.0381)	2.02
uniform_filter size=5	float32	512²	1.17 (1.17–1.18)	1.27 (1.26–1.29)	0.93
uniform_filter size=5	float64	128²	0.052 (0.0517–0.0572)	0.0466 (0.0455–0.0492)	1.11
uniform_filter size=5	float64	512²	0.778 (0.773–0.811)	1.8 (1.77–2.05)	0.43

Not in the table because there is no counterpart on the same inputs: PeakLocalMax.find (counterpart peak_local_max needs maximum_filter; not timed this session).

Backend decision per function (D16)

Today every function above is CPU (Accelerate, Float64 reference plus Float32 where it exists). No function has a Metal path, so no Metal break-even has been measured; the column says what the CPU table implies and what the nightly run must measure. A guess is not recorded as a measurement.

function	backend today	decision
`fft2` Float32/Float64	CPU	CPU only at 128² to 512² (0.02 to 2 ms: a Metal dispatch cannot win). Work item: a Metal Float32 FFT prototype, break-even to be measured nightly at 2048², 4096², 8192²
`rfft2`, `butterworth`, masked registration	CPU	CPU; first the real-input transform (open-items 1), then re-measure
`gaussian_filter`, `uniform_filter`, `difference_of_gaussians`	CPU	CPU only at 128² to 512² (1 to 4 ms, memory bound). Metal candidate above 2048² for large sigma; to be measured nightly
`shift` order 1	CPU	CPU only: 0.65 ms at 512²
`shift` order 3 (prefilter plus taps)	CPU	CPU; the prefilter is sequential along lines, the tap sum is the Metal-friendly part: candidate above 2048², not measured
`phase_cross_correlation`	CPU	CPU; FFT-bound, follows `fft2`
`profile_line`	CPU	CPU: small arrays stay on the CPU (D16)
`label`	CPU	CPU: union-find is not a Metal workload
`peak_local_max`	CPU	CPU; the window maximum is a Metal candidate only above 4096²

DiffractoKit 0.9.0

Full file and method

Measured 2026-10-21 on the first table of the kit audit (suite ROADMAP task 4, protocol step 2 and 3). Small and typical sizes only; the large rows are the nightly review's (DIFFRACTO_BENCH_LARGE=1). Best of five after one warm-up: the median is the figure, the spread (min-max) is in brackets. Same inputs on both sides (tools/benchmarks/gen_inputs.py writes them), Float64 unless the function is Float32 by design (ACOM, ellipse pattern pixels).


Machine	Apple M5 Pro, 64 GB, macOS 27.0.1 (Darwin 27.0.0)
Toolchain	Apple Swift 6.4, `swift build -c release` (separate package `tools/benchmarks`)
Python	3.12.14, NumPy 2.5.4, SciPy 1.18.1; py4DSTEM 0.14.19 (`refs/py4dstem`); pymatgen 2026.10.16 and orix 0.15.0 and spglib 2.8.0 (`refs/orix`)
Load	10.5 on a 12-core machine while five other kit sessions ran (uptime 17:39): every figure carries that noise, the single-core rows least. Re-run when the machine is quiet before quoting a ratio.
Pin note	`refs/orix` holds pymatgen 2026.10.16, the kit's parity pin is 2026.9.24; nothing here compares numbers, only time

Re-run: tools/benchmarks/gen_inputs.py (refs/orix Python), then ../../tools/memcap.sh --gb 8 -- tools/benchmarks/run.sh for the kit and tools/benchmarks/bench_python.py pymatgen|py4dstem for Python, merged by merge.py. The raw lines are tools/benchmarks/results/2026-10-21.jsonl.

Per-function table (ms per call; `acom_match_scan` is ms per pattern)

Function	Size	n	Kit	Python	Kit / Python (first Python row)
`niggli_reduce`	small	100	0.0679 ms (0.064-0.0839)	11.3 ms (9.96-11.7) [cpu]<br>0.293 ms (0.29-0.296) [spglib]	0.006
`niggli_reduce`	typical	1000	0.51 ms (0.508-0.512)	163 ms (149-267) [cpu]<br>3.13 ms (2.88-3.5) [spglib]	0.0031
`niggli_reduce`	large	10000	nightly	nightly [cpu]<br>nightly [spglib]	-
`reflections`	small	10	0.0606 ms (0.0581-0.065)	0.0261 ms (0.024-0.0353)	2.3
`reflections`	typical	20	0.753 ms (0.56-0.916)	0.115 ms (0.0952-0.191)	6.6
`reflections`	large	40	nightly	nightly	-
`kinematic_reflections`	small	10	0.98 ms (0.849-1.14)	4.45 ms (3.96-55.3)	0.22
`kinematic_reflections`	typical	20	7 ms (6.64-7.55)	12.7 ms (11.4-34.7)	0.55
`kinematic_reflections`	large	30	nightly	nightly	-
`symmetry_detection`	small	4	6.87 ms (6.45-7.47)	5.38 ms (4.78-5.69)	1.3
`symmetry_detection`	typical	32	361 ms (350-438)	13.1 ms (11.9-16.5)	28
`symmetry_detection`	large	108	nightly	nightly	-
`cif_import`	small	962	0.652 ms (0.632-0.713)	1.28 ms (1.27-1.31)	0.51
`cif_import`	typical	2364	351 ms (329-506)	2.52 ms (2.32-2.64)	1.4e+02
`cif_import`	large	6483	nightly	nightly	-
`orientation_reduce`	small	1000	1.88 ms (1.87-5.39)	22.1 ms (17.8-26)	0.085
`orientation_reduce`	typical	10000	19.3 ms (17.8-29.4)	48.2 ms (42.2-50.7)	0.4
`orientation_reduce`	large	100000	nightly	nightly	-
`ellipse_fit_1d`	small	128	1.25 ms (1.2-1.41)	1.34 ms (1.22-1.8)	0.94
`ellipse_fit_1d`	typical	256	5.86 ms (4.35-17.7)	5.26 ms (4.87-5.62)	1.1
`ellipse_fit_1d`	large	512	nightly	nightly	-
`acom_library`	small	100	7.81 ms (6.88-16.9)	13.7 ms (12.4-40.2)	0.57
`acom_library`	typical	600	60.6 ms (49.4-80.7)	62.5 ms (39-132)	0.97
`acom_library`	large	2400	nightly	nightly	-
`acom_match_scan`	small	64	1.85 ms (1.83-1.88) [cpu 1 worker]<br>0.245 ms (0.235-0.274) [cpu all cores]<br>0.294 ms (0.285-0.314) [metal]	25.7 ms (14.4-37.9)	0.072 / 0.0095 / 0.011
`acom_match_scan`	typical	1024	1.94 ms (1.91-1.98) [cpu 1 worker]<br>0.216 ms (0.211-0.236) [cpu all cores]<br>0.236 ms (0.215-0.285) [metal]	17.5 ms (10.4-21.5)	0.11 / 0.012 / 0.014
`acom_match_scan`	large	16384	nightly [cpu 1 worker]<br>nightly [cpu all cores]<br>nightly [metal]	-	-

What the rows compare, and what they do not

niggli_reduce: the kit's UnitCell.niggliReduced against pymatgen's Lattice.get_niggli_reduced_lattice (pure Python) and

spglib's C niggli_reduce; the kit is faster than both (the first Python row is pymatgen). Cells are built outside the timer.

reflections: not like for like. The kit's reflections(kMax:) also gives d, the absence flag, the rings and their

orbits; pymatgen's get_points_in_sphere returns the points only. The kit is 2.3 to 6.6 times slower on the points.

kinematic_reflections: the kit's kinematicReflections(kMax:retainingAll:false) against py4DSTEM's

Crystal.calculate_structure_factors on the same C2/m cell; the kit is 1.8 to 4.5 times faster. Both use the Lobato-Van Dyck table.

symmetry_detection and cif_import: the kit's CIFImport.read always detects the symmetry of the cell it reads;

pymatgen's CifParser only parses and SpacegroupAnalyzer is spglib in C. On a 32-atom P1 cell (an fcc supercell with 1 536 operations) the kit takes 361 ms against 13 ms, 28 times slower; on a four-atom cell they are level. Listed first in docs/open-items.md.

orientation_reduce: the kit reduces one matrix at a time (24 proper operators); orix reduces a vectorised batch

(Orientation.from_matrix + map_into_symmetry_reduced_zone). The kit is 2.5 to 12 times faster at these sizes.

ellipse_fit_1d: level with py4DSTEM's fit_ellipse_1D (0.94 and 1.1).

acom_library: level with orientation_plan for about 600 zone axes (0.97); py4DSTEM's count comes from its angular step

(630 at 1.6 degrees), the kit's from zoneAxisCount. 128 in-plane steps on both sides.

acom_match_scan: the kit's matchAll over the whole scan against py4DSTEM's match_single_pattern per pattern (one

core; Python's per-pattern time includes its own refinements, so this is a time, not a like-for-like algorithm). Per pattern the kit on one core is 14 times (8 x 8 scan) and 9 times (32 x 32) faster than py4DSTEM on one core, and on all cores 105 and 81 times.

Backend decision per function (D16)

Function	Backend	Why / break-even
`niggli_reduce`, `reflections`, `kinematic_reflections`, `symmetry_detection`, `cif_import`	CPU only	small arrays, branchy integer logic, readers (D16); the kit is already faster than Python except where `docs/open-items.md` lists it
`orientation_reduce`	CPU only	24 small matrix products per rotation; the batch (100 000 in 0.2 s) is bound by memory traffic, a GPU would pay the transfer for no gain at these sizes
`ellipse_fit_1d`	CPU only	one 1D least-squares fit per pattern; level with Python
`acom_library`	CPU only	built once per phase (60 ms for 600 axes), not per pattern
`acom_match_scan`	CPU all cores; Metal not above any measured N	Metal 0.294 ms per pattern at 64 patterns, 0.236 at 1 024, against 0.245 and 0.216 on all CPU cores: the break-even is not reached up to 1 024 patterns (the Metal path is 7 to 22 % slower); the 128 x 128 row (16 384 patterns) is the nightly's, and the Metal path stays an option (`ACOMBackend.metal`), not the default. A larger library (2 400 axes) is the other axis to measure there

Numbers in docs/parity.md are untouched by this table (time only).

SpectroKit 0.8.0

Full file and method

Written 2026-10-21 for 0.8.0. Harness: tools/benchmarks/run.sh [small,typical] (Swift: Tests/SpectroEDXTests/BenchmarkQuantificationTests.swift, Tests/SpectroEELSTests/BenchmarkThicknessTests.swift, built by swift test -c release, enabled by SPECTRO_BENCH_OUT; Python: tools/benchmarks/bench.py), under tools/memcap.sh (8 GB; measured Swift peak 1.3 GB at the typical size). Inputs are closed-form on both sides (formulas in the harness headers), float64. One warm-up, then five runs, median and (min-max).

Machine: Apple M5 Pro, 64 GB, macOS 27.0.1 (26A434), Swift 6.4 (swiftlang-6.4.0.34.1), Python 3.12.14, NumPy 2.5.4, HyperSpy 2.4.0, exSpy 0.3.2 (no Numba). Other sessions were running on the machine, so the Python spreads are wide; read ratios within about 30 %.

Results

function	size	pixels	Swift CPU, ms median (min-max)	Python, ms median (min-max)	Python / Swift
quantification maps: CL	small	64	0.03 (0.03-0.04)	24.25 (22.87-54.62)	739.62
quantification maps: CL+absorption	small	64	0.20 (0.19-0.20)	178.25 (160.09-233.19)	913.51
quantification maps: zeta	small	64	0.02 (0.02-0.02)	30.13 (26.23-77.22)	1409.82
quantification maps: zeta+absorption	small	64	0.11 (0.10-0.12)	107.16 (92.21-159.42)	1002.64
quantification maps: cross_section	small	64	0.02 (0.02-0.03)	61.30 (57.32-122.77)	2464.49
quantification maps: CL	typical	65536	9.02 (8.63-9.50)	312.23 (307.89-322.30)	34.60
quantification maps: CL+absorption	typical	65536	39.38 (38.00-40.36)	989.56 (933.26-1025.54)	25.13
quantification maps: zeta	typical	65536	6.73 (6.38-7.32)	38.62 (34.42-87.84)	5.74
quantification maps: zeta+absorption	typical	65536	29.90 (29.16-39.16)	129.60 (92.88-168.46)	4.33
quantification maps: cross_section	typical	65536	9.99 (9.64-10.59)	50.59 (43.01-90.63)	5.06
t/lambda log ratio: fixed 3 eV	small	64	0.15 (0.15-0.17)	3.62 (3.08-4.46)	23.43
t/lambda log ratio: estimated	small	64	0.25 (0.24-0.32)	12.10 (11.87-12.86)	47.61
t/lambda log ratio: fixed 3 eV	typical	4096	9.90 (9.53-10.19)	5.01 (4.71-8.40)	0.51
t/lambda log ratio: estimated	typical	4096	16.34 (16.29-17.02)	87.30 (82.73-123.58)	5.34

Sizes: quantification small 8 x 8, typical 256 x 256 (3 elements), large 1024 x 1024; thickness small 8 x 8, typical 64 x 64 (2048 channels), large 256 x 256. Large: nightly (not run in the session).

Backend decision per function (D16)

function	decision	why
quantification maps, all methods (the Python column converts to weight percent only; Swift returns both)	CPU only (parallel over pixels since 0.8.0)	9 to 40 ms for 65 536 pixels; per pixel the work is 10 to 500 ns of arithmetic, so a Metal dispatch pays only above about 1e6 pixels; the break-even is not measured (large size is nightly). Not a work item until the nightly shows a slow large image
t/lambda log ratio, fixed threshold	CPU only; slower than Python on the typical size (0.49x), first in `docs/open-items.md`	single-threaded scalar passes against NumPy's vectorised axis sums
t/lambda log ratio, estimated threshold	CPU only	5x faster than Python at the typical size (Python's per-pixel Savitzky-Golay search dominates)
EDX line fit, auto ID, EELS background fit, edge quantification	not benchmarked in 0.8.0	need a Velox spectrum image or a planted EELS image on both sides; listed in `docs/open-items.md`

CuboKit 0.8.0

Full file and method

Kit against py4DSTEM, same inputs, Float32 cubes, one run of tools/benchmarks on 2026-10-21. Harness: Tests/CuboVirtualTests/BenchmarkHarness.swift (Swift, built with swift test -c release, skipped unless CUBOKIT_BENCH_DIR is set) and tools/benchmarks/bench.py (the py4DSTEM side and the merge; the commands are in its docstring). Both read the same .npy cubes. Best of five after one warm-up; cells are best / median (spread = slowest - best) in milliseconds; "CPU vs py4DSTEM" is py4DSTEM's best divided by the kit CPU's best (above 1 the kit is faster). The cubes are held in memory (MemoryCube; DataCube(data=ndarray)). The Metal columns copy each tile to the GPU on every call; the _resident rows hold the cube in a ResidentCube (one Metal buffer) first.

Sizes (scan rows x columns x detector rows x columns): small 32x32x32x32 (1.0 M values), mid 64x64x32x32 (4.2 M), typical 100x100x64x64 (41.9 M, 168 MB). The large size (330x330x64x64, 1.8 GB) and the Bragg, strain, DPC, parallax and ptychography functions are the nightly review's; this first table is what a Mac can measure while five other sessions use it. The machine was shared during the run (load average 5 to 11 from other sessions' builds), so the spreads are wider than on a quiet Mac.

Machine: Apple M5 Pro, macOS 27.0.1; Swift 6.4; py4DSTEM 0.14.19, NumPy 2.5.4, Python 3.12.14. Float32 cubes, best of five after one warm-up; cells are best / median (spread) in ms.

function	size	kit CPU	kit Metal	py4DSTEM	CPU vs py4DSTEM
bin_Q_2	small	1.3 / 1.3 (0.2)	—	3.4 / 3.6 (0.8)	2.64x
bin_Q_2	mid	5.4 / 5.6 (0.9)	—	12.9 / 13.1 (0.2)	2.40x
bin_Q_2	typical	54.0 / 55.3 (2.7)	—	116.5 / 120.0 (4.2)	2.16x
bin_R_2	small	0.8 / 0.9 (0.1)	—	0.4 / 0.5 (0.0)	0.53x
bin_R_2	mid	3.2 / 3.4 (0.3)	—	0.7 / 0.8 (0.6)	0.22x
bin_R_2	typical	32.3 / 33.0 (1.5)	—	6.4 / 6.8 (2.0)	0.20x
center_of_mass	small	0.4 / 0.6 (0.5)	1.6 / 1.6 (0.5)	—	—
center_of_mass	mid	1.2 / 1.7 (0.6)	2.1 / 2.3 (1.7)	—	—
center_of_mass	typical	12.8 / 13.2 (0.8)	31.4 / 31.9 (0.9)	—	—
dp_max	small	0.1 / 0.1 (0.0)	1.5 / 1.6 (0.5)	0.2 / 0.2 (0.0)	2.55x
dp_max	mid	0.3 / 0.3 (0.0)	3.5 / 3.5 (0.2)	0.4 / 0.4 (0.0)	1.48x
dp_max	typical	5.9 / 5.9 (0.4)	28.7 / 29.1 (0.6)	2.8 / 3.1 (3.9)	0.48x
dp_mean	small	0.8 / 0.8 (0.4)	1.5 / 1.8 (0.7)	0.2 / 0.2 (0.1)	0.23x
dp_mean	mid	3.4 / 3.8 (0.7)	3.4 / 3.5 (0.2)	0.4 / 0.4 (0.1)	0.12x
dp_mean	typical	22.0 / 23.6 (1.8)	27.5 / 28.7 (3.1)	2.8 / 3.2 (1.2)	0.13x
dp_mean_resident	small	—	0.5 / 0.5 (0.5)	—	—
dp_mean_resident	mid	—	1.6 / 1.7 (0.1)	—	—
dp_mean_resident	typical	—	3.4 / 3.5 (0.2)	—	—
dp_median	small	8.6 / 9.1 (0.8)	—	13.8 / 14.2 (0.4)	1.61x
dp_median	mid	36.8 / 38.3 (4.9)	—	65.3 / 68.0 (5.1)	1.77x
dp_median	typical	233.4 / 251.8 (34.5)	—	785.1 / 821.1 (51.6)	3.36x
median_masked_pixels	small	0.5 / 0.5 (0.0)	—	0.5 / 0.8 (0.8)	1.07x
median_masked_pixels	mid	2.0 / 2.1 (0.2)	—	0.9 / 0.9 (0.3)	0.42x
pad_Q_2	small	0.8 / 0.8 (0.2)	—	2.1 / 2.3 (0.4)	2.72x
pad_Q_2	mid	2.9 / 3.0 (0.3)	—	8.3 / 8.8 (0.7)	2.86x
pad_Q_2	typical	34.6 / 46.3 (23.0)	—	98.3 / 100.5 (10.6)	2.84x
resample_Q_half	small	0.6 / 0.6 (0.2)	—	10.0 / 10.8 (1.1)	16.94x
resample_Q_half	mid	2.4 / 2.4 (0.1)	—	39.3 / 45.6 (16.0)	16.51x
selected_area_mean_mask	small	1.3 / 1.4 (0.1)	1.7 / 1.9 (0.3)	0.5 / 0.6 (0.5)	0.38x
selected_area_mean_mask	mid	4.5 / 5.1 (1.1)	3.7 / 3.9 (0.3)	1.3 / 1.4 (0.2)	0.29x
selected_area_mean_mask	typical	47.6 / 49.4 (2.4)	29.0 / 30.3 (2.4)	10.8 / 12.8 (2.8)	0.23x
selected_area_mean_mask_resident	small	—	1.2 / 1.3 (0.3)	—	—
selected_area_mean_mask_resident	mid	—	1.8 / 2.0 (0.3)	—	—
selected_area_mean_mask_resident	typical	—	4.7 / 6.2 (6.0)	—	—
selected_area_mean_weights	small	1.5 / 1.7 (0.2)	1.5 / 1.7 (0.4)	2.9 / 3.1 (0.6)	1.89x
selected_area_mean_weights	mid	5.8 / 6.2 (1.3)	3.5 / 3.6 (0.2)	11.8 / 12.1 (0.6)	2.03x
selected_area_mean_weights	typical	61.7 / 63.1 (3.5)	27.7 / 29.2 (2.3)	166.5 / 168.1 (3.6)	2.70x
selected_area_mean_weights_resident	small	—	1.1 / 1.1 (0.0)	—	—
selected_area_mean_weights_resident	mid	—	1.7 / 1.8 (0.3)	—	—
selected_area_mean_weights_resident	typical	—	3.6 / 3.8 (0.3)	—	—
thin_R_2	small	0.1 / 0.1 (0.1)	—	0.6 / 0.6 (0.4)	7.23x
thin_R_2	mid	0.3 / 0.3 (0.0)	—	0.8 / 0.8 (0.1)	3.02x
thin_R_2	typical	2.4 / 2.7 (0.4)	—	4.5 / 5.0 (4.4)	1.91x
virtual_image_circle	small	0.2 / 0.2 (0.0)	0.9 / 0.9 (0.4)	1.9 / 2.0 (0.2)	10.09x
virtual_image_circle	mid	0.7 / 0.7 (0.0)	2.2 / 2.3 (0.6)	7.0 / 7.1 (0.2)	9.88x
virtual_image_circle	typical	10.3 / 10.8 (1.0)	23.5 / 24.1 (1.9)	31.3 / 32.7 (2.8)	3.04x

Backend decision per function (D16)

The break-even is read off the three sizes above; between two measured sizes it is interpolated, and says so. The Mac was loaded (load average 10 to 11) during the final run, so a crossing near a tie moves with load.

function	decision	why
`VirtualDiffraction.max`	CPU always	block-parallel since 0.8.0: 5.9 ms at the typical size against NumPy's 2.8 and 29 ms with the host copy; a `vmax` pass is memory-bound, so a copy to the GPU never pays
`VirtualDiffraction.mean`, `meanAndMax`	CPU; resident Metal always	the compensated Double sum is compute-bound. Host-copy Metal ties the CPU at 4.2 M values (3.4 against 3.4 ms) and loses at 41.9 M on this loaded Mac (27.5 against 22.0), so it does not pay; the resident cube beats the CPU at every size (0.5 against 0.8 ms at 1.0 M, 3.4 against 22.0 at 41.9 M) and comes within 1.2x of NumPy (3.4 against 2.8 ms)
`VirtualDiffraction.pattern` with a mask	Metal above about 4 M values; resident Metal always	CPU 47.6 ms, host-copy Metal 29.0, resident Metal 4.6 at the typical size; level at 4.2 M (4.5 against 3.7)
`VirtualDiffraction.pattern` with weights	Metal above about 1 M values; resident Metal always	the CPU path pays Double products (62 ms at the typical size); host-copy Metal is 1.7x faster at 4.2 M (3.5 against 5.8) and 2.2x at 41.9 M; resident Metal 4.3 ms. The CPU path is already 2.7x faster than py4DSTEM
`VirtualDiffraction.median`	CPU only	no Metal path; 3.4x faster than py4DSTEM at the typical size. A GPU sort per pixel needs the whole scan axis resident; revisit with the resident cube
`VirtualImaging.image`	CPU only for a non-resident cube	the CPU dot product is 9.5 ms at the typical size against 21 ms with the host copy; resident Metal is not measured (open item)
`CenterOfMass.measure`	CPU only	CPU 11.0 ms against Metal 26.7 at the typical size; the weighted-moment kernel is one thread per position
`CubeEditing.binScan`, `thinScan`, `padDetector`, `resampleDetector`, `medianFilterMaskedPixels`, `ViewedCube` bin	CPU only	memory-bound copies and small per-pattern arithmetic; the copy to the GPU would cost more than the work. Faster than py4DSTEM except `binScan` and the masked-pixel median at 64 x 64 (open item 3)
`Calibration`, `CalibrationMeasurement`, `BraggMasking`	CPU only	scalars and one pattern at a time

SimulatoKit 0.3.0

Full file and method

Per-function timings of SimulatoKit against abTEM 1.0.10 on the same inputs (Al [001], 200 keV, a plane wave or a 20 mrad probe, 10 slices of 2.025 Å, about 0.1 Å per pixel, infinite projection). One warm-up, then five timed runs; each cell is the median in milliseconds with (min-max). Written by tools/benchmarks/run.sh and merge.py.

Machine: Apple M5 Pro, macOS 27.0.1, Apple Swift version 6.4 (swiftlang-6.4.0.34.1 clang-2100.3.34.1)

Python 3.12.14 (NumPy 2.5.4, SciPy 1.18.1), abTEM 1.0.10, NumPy/SciPy from refs/abtem (CPU, NumPy's FFT; no GPU)

Kit CPU is float64 on Accelerate (one thread per call); Metal is float32 on MPSGraph; abTEM is shown in float64 and float32

Run while other sessions used the machine (load average 6 to 11): the spread columns show it; treat ratios within 20% as equal

Sizes: 64² to 256² small and typical; 1024² and above: nightly (not run in the session that wrote this table)