Add performance measurements, trash test and documentation

This commit is contained in:
Andy Dixon
2026-10-02 08:31:42 +01:00
parent 0e5b9f7cc3
commit b771a57bc4
8 changed files with 418 additions and 27 deletions
+93
View File
@@ -0,0 +1,93 @@
# Performance
All numbers below were measured, not estimated. Re-run them on your own
hardware before relying on them; filesystem caches dominate scan timings.
## Test machine (2026-10-02)
| | |
|---|---|
| CPU | Intel Core i7-7700 @ 3.60 GHz, 4 cores / 8 threads |
| RAM | 32 GiB |
| OS | Linux 7.0, Go 1.27.1 |
| Filesystem | ext4 on Linux software RAID (`/dev/md2`) over two 7200 rpm SATA HDDs |
## Scanner
Dataset: `~/go/pkg/mod` — 388,307 files + 65,518 directories (458,510
inventory nodes), 10.5 GiB allocated. **Warm cache only**: dropping the page
cache needs root, which was not available, so these measure CPU and syscall
cost, not disk seeks. Cold-cache behaviour on HDDs will be dominated by seek
time; expect fewer workers to be optimal there.
`mapsize --no-ui --workers N ~/go/pkg/mod`, wall time, 3 runs each
(`/usr/bin/time`):
| workers | run 1 | run 2 | run 3 | entries/s (best) | peak RSS |
|---:|---:|---:|---:|---:|---:|
| 1 | 1.77 s | 1.67 s | 1.65 s | ~278,000 | 117 MB |
| 2 | 1.06 s | 0.99 s | 1.00 s | ~463,000 | 120 MB |
| 4 | 0.59 s | 0.59 s | 0.62 s | ~777,000 | 117 MB |
| 8 (balanced default) | 0.48 s | 0.50 s | 0.51 s | ~955,000 | 121 MB |
| 16 | 0.45 s | 0.44 s | 0.43 s | ~1,066,000 | 121 MB |
| 32 | 0.43 s | 0.44 s | 0.42 s | ~1,092,000 | 122 MB |
For comparison, `du -s` on the same tree (warm): 1.59 s, 34 MB RSS.
CPU profile at 8 workers: ~60 % of samples in `statx`/`getdents` syscalls;
path reconstruction for job dispatch ~7 %; aggregation is negligible.
### Memory
`go test ./internal/scan -run TestMemoryPerNode` with
`MAPSIZE_MEM_ROOT=~/go/pkg/mod`: 458,510 nodes, 69.1 MiB live heap,
**~158 bytes per node** (node record ~112 bytes plus name storage and
directory side tables). Extrapolated, 10 M entries need ~1.6 GiB.
## UI
`go test ./internal/tui -bench Big` on a synthetic in-memory tree of
1,008,201 nodes (200 directories × 40 subdirectories × 100 files, plus one
flat directory with 200,000 files), 200×60 terminal:
| operation | time per op |
|---|---:|
| cold frame at root (sort children, LOD, layout incl. nested previews, paint) | 33.8 ms |
| resize to a new size (layout + paint, child lists cached) | 2.0 ms |
| arrow-key move + re-render | 1.5 ms |
| zoomed into the 200,000-entry directory, cold | 29.5 ms |
| render of a small tree (2.4 K nodes) at 200×60 | 0.66 ms |
| resize relayout of the small tree | 0.45 ms |
During a scan the size caches are invalidated at most every 400 ms, so a
cold frame (worst case above ~34 ms) costs under 10 % of one core; other
frames reuse caches. Resize events only record the new size; layout happens
once per rendered frame, so a resize storm cannot queue up work.
### Geometry
`go test ./internal/treemap -bench .`:
| operation | time |
|---|---:|
| squarified layout, 300 items | 89 µs |
| spatial neighbour lookup among 300 blocks | 2.1 µs |
### Filtering (1,008,201 nodes)
| query | time |
|---|---:|
| `size > 1MB` | 31 ms |
| `*.dat AND size > 500k` | 119 ms |
| `path contains sub3` (reconstructs every path) | 244 ms |
Filters run on a background goroutine with cancellation and a 120 ms
keystroke debounce; the UI never waits for them.
## Known costs / future work
* Path predicates rebuild each node's path; an incremental evaluation along
the tree would make them ~10× cheaper.
* Node records could shrink (owner/group/mode/link count are only needed for
display and some filters) if memory on 10 M+ entry trees matters.
* Cold-cache HDD benchmarks still need to be run with root access.