Benchmark results: Go 0.5 against Rust 0.6
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Andy Dixon
Claude
@@ -0,0 +1,85 @@
|
||||
# Benchmarks: Go 0.5 against Rust 0.6
|
||||
|
||||
Measured on 2026-10-08 on one machine: Intel Core i7-7700 (4 cores, 8
|
||||
threads), 31 GiB RAM, ext4 on a two-disk RAID 1 (`/dev/md2`), Linux 7.0.
|
||||
Both binaries were built from this tree: the Go oracle (`oracle/go`, Go
|
||||
1.27.1, `CGO_ENABLED=0`) and the Rust release build (Rust 1.99.0, LTO).
|
||||
|
||||
**Limitation:** the machine was shared. Other users' Docker builds kept the
|
||||
load average between 4 and 12 throughout, so absolute times are noisy.
|
||||
Go and Rust were run alternately under the same conditions, and every
|
||||
figure is a median with its spread, but small differences mean nothing.
|
||||
Page caches were warm (dropping them needs root), so cold-cache numbers
|
||||
were not measured.
|
||||
|
||||
## Primitives
|
||||
|
||||
`cargo bench --bench micro` and
|
||||
`(cd oracle/go && go test -run XXX -bench Oracle -benchtime 2s ./ ./internal/pqcrypt ./internal/stream)`,
|
||||
run alternately twice (load 4.4 to 5.4). Medians in milliseconds; the Rust
|
||||
harness's p10 to p90 range is in brackets.
|
||||
|
||||
| Operation | Go | Rust |
|
||||
| --- | --- | --- |
|
||||
| Derive hybrid identity from seed | 5.6 | 3.0 (2.9 to 4.6) |
|
||||
| Wrap file key (both encapsulations) | 7.4 to 8.0 | 3.1 (3.0 to 4.8) |
|
||||
| Unwrap file key (both decapsulations) | 7.6 to 8.3 | 3.1 (3.0 to 3.6) |
|
||||
| Encrypt and decrypt one 4 MiB object | 19.9 to 20.1 | 19.0 to 19.5 |
|
||||
| STREAM seal and open, 64 MiB, streaming API | 89 to 92 | 187 to 188 |
|
||||
| Unlock key file (Argon2id for both slots) | 121 to 125 | 129 (125 to 147) |
|
||||
|
||||
- The FrodoKEM port is about twice as fast as CIRCL's Go code, mostly from
|
||||
multiplying A row by row instead of walking it by column.
|
||||
- ChaCha20-Poly1305 is slower: RustCrypto reaches about 1.2 GB/s on this CPU
|
||||
(`cargo bench --bench aead_probe`), Go's assembly about twice that. The
|
||||
64 MiB row uses the streaming reader and writer, which copy through
|
||||
64 KiB buffers. The vault itself uses the in-memory path (`seal_all`
|
||||
0.85 GB/s, `open_all` 1.4 GB/s), and per 4 MiB object the two are
|
||||
level because the KEM work is cheaper.
|
||||
- Unlock takes 180 ms with Argon2id lanes computed in turn. Computing the
|
||||
four lanes in parallel (argon2's `parallel` feature) brings it to 129 ms,
|
||||
close to Go. The parameters are unchanged.
|
||||
|
||||
## End to end
|
||||
|
||||
`RUNS=3 BIG_MB=1024 SMALL=500 scripts/bench.sh` (load 8 to 13). Every step
|
||||
includes unlocking the key. The 1 GiB file and the 500 small files (1 to
|
||||
16 KiB each) are each added to a fresh vault. The reads use one vault
|
||||
holding both, plus filler. Times are medians of 3 (min to max).
|
||||
|
||||
| Step | Go | Rust |
|
||||
| --- | --- | --- |
|
||||
| add 1 GiB file | 41.7 s (32.0 to 50.3) | 29.6 s (28.5 to 31.0) |
|
||||
| add 500 small files | 123.9 s (113.3 to 156.2) | 79.0 s (77.3 to 133.3) |
|
||||
| get 1 GiB file | 9.8 s (8.8 to 10.0) | 16.2 s (15.5 to 16.7) |
|
||||
| list (about 530 records) | 4.95 s (4.89 to 5.20) | 0.74 s (0.71 to 0.77) |
|
||||
| get one small file | 5.24 s (4.88 to 6.00) | 0.74 s (0.70 to 0.81) |
|
||||
| Peak RSS | 137 to 140 MiB | 132 MiB |
|
||||
| Peak threads | 18 | 17 |
|
||||
| Binary size | 11.5 MB | 3.5 MB (static musl: 3.4 MB) |
|
||||
|
||||
- **Catalogue reads** (`list`, and every command that looks a name up) are
|
||||
about 6 times faster. Records are decrypted in parallel on the worker pool
|
||||
(bounded at twice the worker count in flight), and each FrodoKEM
|
||||
decapsulation is cheaper.
|
||||
- **Adds** are faster. Both implementations fsync every object, its shard
|
||||
directory and every record. Rust overlaps up to eight objects' encryption
|
||||
and syncs on the pool.
|
||||
- **`get` of a large file is slower, on purpose.** Rust fsyncs the output
|
||||
file before giving it its name, so a crash just after `get` cannot leave
|
||||
the name pointing at an empty or partial file. Go did not. On an idle
|
||||
rerun (load about 8, output on the same disk), Go took 3.6 s and Rust
|
||||
15.5 s. Of Rust's time, 13.3 s was the single `fsync` of 1 GiB of
|
||||
freshly written data (measured with `strace -T`). The other 2 s or so is
|
||||
unlock, decryption and writing. This is the cost of the
|
||||
durability barrier on this RAID 1 array, and it scales with the disk's
|
||||
write speed.
|
||||
- Peak memory is dominated by Argon2id: 64 MiB for each of the two slots,
|
||||
computed at the same time in both implementations. Rust's object pipeline
|
||||
adds at most eight 4 MiB plaintext buffers and their ciphertexts. Peak
|
||||
threads: Rust has 8 workers, 8 Argon2 threads while unlocking, and the
|
||||
main thread.
|
||||
|
||||
Reproduce with `scripts/bench.sh` (it builds both binaries, creates
|
||||
synthetic data in `target/bench`, and prints the tables' numbers) and the
|
||||
two micro-benchmark commands above.
|
||||
+32
-15
@@ -5,9 +5,9 @@
|
||||
# RUNS=5 BIG_MB=2048 SMALL=5000 scripts/bench.sh
|
||||
#
|
||||
# For each implementation, in a fresh key and vault on the same filesystem:
|
||||
# add one BIG_MB file, get it back, add SMALL small files (1-16 KiB) as a
|
||||
# folder, list the catalogue (filler plus SMALL+1 records), and get one small
|
||||
# file. Each step is timed RUNS times (median, min, max), with peak RSS and
|
||||
# add one BIG_MB file, and add SMALL small files (1-16 KiB) as a folder, each
|
||||
# into a fresh vault; then, in one vault holding both, get the big file,
|
||||
# list the catalogue (filler plus SMALL+1 records), and get one small file. Each step is timed RUNS times (median, min, max), with peak RSS and
|
||||
# peak thread count. Every step includes unlocking the key (two Argon2id
|
||||
# runs). Both implementations fsync every object and record, so `add`
|
||||
# timings include durable writes. Page caches are warm: dropping them needs
|
||||
@@ -20,7 +20,7 @@ root=$PWD
|
||||
work=${1:-$root/target/bench}
|
||||
runs=${RUNS:-3}
|
||||
big_mb=${BIG_MB:-1024}
|
||||
small=${SMALL:-2000}
|
||||
small=${SMALL:-500}
|
||||
|
||||
mkdir -p -- "$work"
|
||||
work=$(cd -- "$work" && pwd -P)
|
||||
@@ -65,25 +65,42 @@ summarise() { # lines of "secs rss threads" -> median min max, max rss, max thre
|
||||
sort -n | awk '{t[NR]=$1; if($2>r)r=$2; if($3>th)th=$3} END {printf "%8.2f s (min %.2f, max %.2f) rss %6.0f MiB threads %d\n", t[int((NR+1)/2)], t[1], t[NR], r/1024, th}'
|
||||
}
|
||||
|
||||
fresh() { # fresh key and vault at $1
|
||||
rm -rf -- "$1"
|
||||
mkdir -p -- "$1"
|
||||
export VAULT_KEY=$1/key VAULT_DIR=$1/store
|
||||
"$bin" genkey 2>/dev/null
|
||||
"$bin" init 2>/dev/null
|
||||
}
|
||||
|
||||
for impl in go rust; do
|
||||
bin=$work/vault-$impl
|
||||
echo "== $impl ($(stat -c %s "$bin") bytes)"
|
||||
for step in add-big get-big add-small list get-small; do
|
||||
# Writes: a fresh vault for every run.
|
||||
for step in add-big add-small; do
|
||||
results=""
|
||||
for run in $(seq 1 "$runs"); do
|
||||
v=$work/run-$impl
|
||||
rm -rf -- "$v"
|
||||
mkdir -p -- "$v"
|
||||
export VAULT_KEY=$v/key VAULT_DIR=$v/store
|
||||
"$bin" genkey 2>/dev/null
|
||||
"$bin" init 2>/dev/null
|
||||
case $step in
|
||||
get-big | list | get-small) "$bin" add "$data/big" 2>/dev/null; "$bin" add "$data/small" 2>/dev/null ;;
|
||||
esac
|
||||
fresh "$work/run-$impl"
|
||||
case $step in
|
||||
add-big) r=$(measure "$bin" add "$data/big") ;;
|
||||
get-big) r=$(measure "$bin" get big -o "$v/big.out") ;;
|
||||
add-small) r=$(measure "$bin" add "$data/small") ;;
|
||||
esac
|
||||
results+="$r"$'\n'
|
||||
done
|
||||
printf ' %-10s ' "$step"
|
||||
printf '%s' "$results" | grep . | summarise
|
||||
done
|
||||
# Reads: one populated vault, read repeatedly.
|
||||
v=$work/run-$impl
|
||||
fresh "$v"
|
||||
"$bin" add "$data/big" 2>/dev/null
|
||||
"$bin" add "$data/small" 2>/dev/null
|
||||
for step in get-big list get-small; do
|
||||
results=""
|
||||
for run in $(seq 1 "$runs"); do
|
||||
rm -f -- "$v/big.out" "$v/small.out"
|
||||
case $step in
|
||||
get-big) r=$(measure "$bin" get big -o "$v/big.out") ;;
|
||||
list) r=$(measure "$bin" list) ;;
|
||||
get-small) r=$(measure "$bin" get "small/f$(printf "%0${#small}d" 1)" -o "$v/small.out") ;;
|
||||
esac
|
||||
|
||||
Reference in New Issue
Block a user