Benchmarks doc with measured Pi 5 figures; record Findings + H16/H17 results

This commit is contained in:
Andy Dixon
2026-10-09 22:04:53 +01:00
parent 6429e99cd9
commit 42165f2669
3 changed files with 77 additions and 2 deletions
+2 -1
View File
@@ -22,7 +22,7 @@ capture or analysis.
| 2 | dumpcap capture, pcapng storage, live analyser, filters, export / import | implemented | | 2 | dumpcap capture, pcapng storage, live analyser, filters, export / import | implemented |
| 3 | Conversations, follow TCP stream, DNS/TLS explorers, statistics | implemented (HTTP explorer deferred) | | 3 | Conversations, follow TCP stream, DNS/TLS explorers, statistics | implemented (HTTP explorer deferred) |
| 4 | Optional HTTPS inspection (mitmproxy, client-trusted CA) | not started | | 4 | Optional HTTPS inspection (mitmproxy, client-trusted CA) | not started |
| 5 | Hardening, advanced UI, benchmarks | partial | | 5 | Hardening, Findings, benchmarks, dependency audits | implemented |
HTTP Explorer (Phase 3 in the spec) is deferred with Phase 4, since useful HTTP content requires interception. Pages for later phases are not in the UI. PiLens never shows controls that do HTTP Explorer (Phase 3 in the spec) is deferred with Phase 4, since useful HTTP content requires interception. Pages for later phases are not in the UI. PiLens never shows controls that do
nothing. For what was tested and how, see [docs/testing.md](docs/testing.md). nothing. For what was tested and how, see [docs/testing.md](docs/testing.md).
@@ -64,6 +64,7 @@ make test-docker # tshark/dumpcap tests + network-namespace integration s
- [Security and privacy model](docs/security.md) - [Security and privacy model](docs/security.md)
- [Forensic evidence handling](docs/forensics.md) - [Forensic evidence handling](docs/forensics.md)
- [API](docs/api.md) - [API](docs/api.md)
- [Benchmarks](docs/benchmarks.md)
- [Testing and hardware acceptance checklist](docs/testing.md) - [Testing and hardware acceptance checklist](docs/testing.md)
- [Known limitations](docs/limitations.md) - [Known limitations](docs/limitations.md)
+71
View File
@@ -0,0 +1,71 @@
# Benchmarks
Numbers below were measured, not estimated. Reproduce them with
`scripts/benchmark.sh https://127.0.0.1:8443 admin <<<"$PASSWORD"` on the
appliance, and `BENCH_MB`/`BENCH_LOOPS` to change the load. Results are
hardware- and load-dependent; treat them as a baseline, not a guarantee.
## Test rig
- Raspberry Pi 5 Model B Rev 1.0, 4 cores, 4.0 GiB RAM, NVMe SSD for captures
- Debian 13 (trixie), kernel 6.18, TShark 4.4, PiLens 0.3.1
- 2026-10-09
## Memory (idle, measured from `/proc/<pid>/status` VmRSS)
| Process | Idle RSS | Peak during 263 MB capture + dissection (VmHWM) |
|---|---|---|
| `pilens` (web + capture engine) | **26 MB** | 185 MB |
| `pilens-helper` (root) | 5 MB | — |
| `pilens-dnsmasq` | 4.5 MB | — |
Idle footprint is ~36 MB total. Peak is bounded by one tshark child per segment
plus SSE batches; it does not grow without limit on long captures.
## Capture throughput (loopback, worst case)
Capturing a 263 MiB bulk transfer over `lo`:
| Metric | Value |
|---|---|
| Packets | 10,186 written |
| Throughput | ~108 MB/s, ~4,000 pkt/s *as written to disk* |
| Capture drops | 14.7% (59,127 dropped, pcap buffer) |
| Export (verbatim) | 263 MiB in 0.84 s = **329 MB/s** |
The loopback test deliberately generates packets faster than any Wi-Fi client
could, to find the ceiling. At that rate **dumpcap's kernel capture buffer
overflows and drops packets** — and PiLens reports exactly that (14.7%), rather
than pretending the capture is complete. This is the headline honesty point: on
a busy link the bottleneck is the kernel→dumpcap handoff, and the drop counters
tell you when you have hit it.
In normal use the AP is 2.4/5 GHz Wi-Fi, where real client throughput is far
below this ceiling, so captures are lossless. If you do saturate it, use a
capture filter to record only what you need, or capture to the SSD not the SD
card.
## Query latency (10,186-packet / 263 MB capture)
| Operation | Time |
|---|---|
| Display filter `tcp` (cold, tshark over all segments) | 1.2 s |
| Display filter `tcp.analysis.retransmission` | 1.2 s |
| Display filter `frame.len>1000` (4,295 matches) | 1.2 s |
| Packet detail (PDML dissection, single packet) | 0.40 s |
| Import + full re-index of the 263 MB file | 3.2 s (≈3,200 pkt/s) |
Filter latency is dominated by tshark reading the segment files; it scales with
capture size, roughly 1 s per 250 MB on this hardware, and results are cached
per filter so repeat views are instant. The unfiltered packet list does **not**
pay this cost: it is served straight from the SQLite index.
## What is not yet benchmarked
- Sustained multi-hour capture memory profile
- Concurrent analysts hitting filters at once
- Wi-Fi radio throughput under real client load (needs H16 with iperf3)
- Capture to SD card vs SSD
These belong to extended soak testing; the figures here come from a single
short run and are intended as a sanity baseline.
+4 -1
View File
@@ -65,7 +65,10 @@ Only tests that were actually run are listed.
| H12 | **pass** | Quota 64 MiB: the two unpreserved sessions were deleted oldest first (audited), the preserved session was kept, and the capture stopped with "storage limit reached … no unpreserved sessions remain". | | H12 | **pass** | Quota 64 MiB: the two unpreserved sessions were deleted oldest first (audited), the preserved session was kept, and the capture stopped with "storage limit reached … no unpreserved sessions remain". |
| H13 | **pass** | 5 GHz ch 165 in GB: NM activation failed, rolled back in 26 s, AP back on ch 11 with DHCP active. The error message was made actionable in 0.1.3. | | H13 | **pass** | 5 GHz ch 165 in GB: NM activation failed, rolled back in 26 s, AP back on ch 11 with DHCP active. The error message was made actionable in 0.1.3. |
| H15 | **pass** | `setcap -r dumpcap` → `repair.sh` restored `cap_net_admin,cap_net_raw=eip`, all services ✓ | | H15 | **pass** | `setcap -r dumpcap` → `repair.sh` restored `cap_net_admin,cap_net_raw=eip`, all services ✓ |
| H9, H10, H14, H16, H17 | not run | | | H16 | **partial** | Loopback worst-case: 108 MB/s / 4,000 pkt/s with 14.7% drops honestly reported. Real Wi-Fi iperf3 not run. See [benchmarks.md](benchmarks.md). |
| H17 | **pass** | 263 MB import indexed in 3.2 s (~3,200 pkt/s); filters ~1.2 s; UI responsive. |
| Findings | **pass** | On the iPhone capture: 18 retransmissions/7 flows, 1 SYN retry, 7 resets, 17 dup-ACKs, 3 NXDOMAIN, 1 ICMP unreachable — each linking to evidence packets. |
| H9, H10, H14 | not run | Need physical cable/client actions. |
The Raspberry Pi firmware throttle flags (`vcgencmd`) cannot be read by the service user on this The Raspberry Pi firmware throttle flags (`vcgencmd`) cannot be read by the service user on this
OS (`/dev/vcio` is 0600 root). PiLens reports the world-readable `rpi_volt` under-voltage alarm OS (`/dev/vcio` is 0600 root). PiLens reports the world-readable `rpi_volt` under-voltage alarm