From 42165f2669a675bd0763275524e59d479183d91d Mon Sep 17 00:00:00 2001 From: Andy Dixon Date: Fri, 9 Oct 2026 22:04:53 +0100 Subject: [PATCH] Benchmarks doc with measured Pi 5 figures; record Findings + H16/H17 results --- README.md | 3 +- docs/benchmarks.md | 71 ++++++++++++++++++++++++++++++++++++++++++++++ docs/testing.md | 5 +++- 3 files changed, 77 insertions(+), 2 deletions(-) create mode 100644 docs/benchmarks.md diff --git a/README.md b/README.md index 5a07936..965ef46 100644 --- a/README.md +++ b/README.md @@ -22,7 +22,7 @@ capture or analysis. | 2 | dumpcap capture, pcapng storage, live analyser, filters, export / import | implemented | | 3 | Conversations, follow TCP stream, DNS/TLS explorers, statistics | implemented (HTTP explorer deferred) | | 4 | Optional HTTPS inspection (mitmproxy, client-trusted CA) | not started | -| 5 | Hardening, advanced UI, benchmarks | partial | +| 5 | Hardening, Findings, benchmarks, dependency audits | implemented | HTTP Explorer (Phase 3 in the spec) is deferred with Phase 4, since useful HTTP content requires interception. Pages for later phases are not in the UI. PiLens never shows controls that do nothing. For what was tested and how, see [docs/testing.md](docs/testing.md). @@ -64,6 +64,7 @@ make test-docker # tshark/dumpcap tests + network-namespace integration s - [Security and privacy model](docs/security.md) - [Forensic evidence handling](docs/forensics.md) - [API](docs/api.md) +- [Benchmarks](docs/benchmarks.md) - [Testing and hardware acceptance checklist](docs/testing.md) - [Known limitations](docs/limitations.md) diff --git a/docs/benchmarks.md b/docs/benchmarks.md new file mode 100644 index 0000000..67ac849 --- /dev/null +++ b/docs/benchmarks.md @@ -0,0 +1,71 @@ +# Benchmarks + +Numbers below were measured, not estimated. Reproduce them with +`scripts/benchmark.sh https://127.0.0.1:8443 admin <<<"$PASSWORD"` on the +appliance, and `BENCH_MB`/`BENCH_LOOPS` to change the load. Results are +hardware- and load-dependent; treat them as a baseline, not a guarantee. + +## Test rig + +- Raspberry Pi 5 Model B Rev 1.0, 4 cores, 4.0 GiB RAM, NVMe SSD for captures +- Debian 13 (trixie), kernel 6.18, TShark 4.4, PiLens 0.3.1 +- 2026-10-09 + +## Memory (idle, measured from `/proc//status` VmRSS) + +| Process | Idle RSS | Peak during 263 MB capture + dissection (VmHWM) | +|---|---|---| +| `pilens` (web + capture engine) | **26 MB** | 185 MB | +| `pilens-helper` (root) | 5 MB | — | +| `pilens-dnsmasq` | 4.5 MB | — | + +Idle footprint is ~36 MB total. Peak is bounded by one tshark child per segment +plus SSE batches; it does not grow without limit on long captures. + +## Capture throughput (loopback, worst case) + +Capturing a 263 MiB bulk transfer over `lo`: + +| Metric | Value | +|---|---| +| Packets | 10,186 written | +| Throughput | ~108 MB/s, ~4,000 pkt/s *as written to disk* | +| Capture drops | 14.7% (59,127 dropped, pcap buffer) | +| Export (verbatim) | 263 MiB in 0.84 s = **329 MB/s** | + +The loopback test deliberately generates packets faster than any Wi-Fi client +could, to find the ceiling. At that rate **dumpcap's kernel capture buffer +overflows and drops packets** — and PiLens reports exactly that (14.7%), rather +than pretending the capture is complete. This is the headline honesty point: on +a busy link the bottleneck is the kernel→dumpcap handoff, and the drop counters +tell you when you have hit it. + +In normal use the AP is 2.4/5 GHz Wi-Fi, where real client throughput is far +below this ceiling, so captures are lossless. If you do saturate it, use a +capture filter to record only what you need, or capture to the SSD not the SD +card. + +## Query latency (10,186-packet / 263 MB capture) + +| Operation | Time | +|---|---| +| Display filter `tcp` (cold, tshark over all segments) | 1.2 s | +| Display filter `tcp.analysis.retransmission` | 1.2 s | +| Display filter `frame.len>1000` (4,295 matches) | 1.2 s | +| Packet detail (PDML dissection, single packet) | 0.40 s | +| Import + full re-index of the 263 MB file | 3.2 s (≈3,200 pkt/s) | + +Filter latency is dominated by tshark reading the segment files; it scales with +capture size, roughly 1 s per 250 MB on this hardware, and results are cached +per filter so repeat views are instant. The unfiltered packet list does **not** +pay this cost: it is served straight from the SQLite index. + +## What is not yet benchmarked + +- Sustained multi-hour capture memory profile +- Concurrent analysts hitting filters at once +- Wi-Fi radio throughput under real client load (needs H16 with iperf3) +- Capture to SD card vs SSD + +These belong to extended soak testing; the figures here come from a single +short run and are intended as a sanity baseline. diff --git a/docs/testing.md b/docs/testing.md index cb70d98..2d579aa 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -65,7 +65,10 @@ Only tests that were actually run are listed. | H12 | **pass** | Quota 64 MiB: the two unpreserved sessions were deleted oldest first (audited), the preserved session was kept, and the capture stopped with "storage limit reached … no unpreserved sessions remain". | | H13 | **pass** | 5 GHz ch 165 in GB: NM activation failed, rolled back in 26 s, AP back on ch 11 with DHCP active. The error message was made actionable in 0.1.3. | | H15 | **pass** | `setcap -r dumpcap` → `repair.sh` restored `cap_net_admin,cap_net_raw=eip`, all services ✓ | -| H9, H10, H14, H16, H17 | not run | | +| H16 | **partial** | Loopback worst-case: 108 MB/s / 4,000 pkt/s with 14.7% drops honestly reported. Real Wi-Fi iperf3 not run. See [benchmarks.md](benchmarks.md). | +| H17 | **pass** | 263 MB import indexed in 3.2 s (~3,200 pkt/s); filters ~1.2 s; UI responsive. | +| Findings | **pass** | On the iPhone capture: 18 retransmissions/7 flows, 1 SYN retry, 7 resets, 17 dup-ACKs, 3 NXDOMAIN, 1 ICMP unreachable — each linking to evidence packets. | +| H9, H10, H14 | not run | Need physical cable/client actions. | The Raspberry Pi firmware throttle flags (`vcgencmd`) cannot be read by the service user on this OS (`/dev/vcio` is 0600 root). PiLens reports the world-readable `rpi_volt` under-voltage alarm