diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..bd86fe2 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,47 @@ +# Changelog + +All notable changes. Dates are when the work was done on the reference Pi 5. + +## 0.3.1 — 2026-10-09 +- Benchmark harness (`scripts/benchmark.sh`, python3-only) and measured figures + in `docs/benchmarks.md`. +- `MemoryAccounting=yes` on `pilens.service`. + +## 0.3.0 — 2026-10-09 (Phase 5: hardening) +- **Findings** diagnostics page: observable TCP/DNS/TLS/ICMP events and capture + drops, each with evidence packets, a reproducing filter and benign-cause notes. + Explicitly a forensic aid, not an IDS. +- Pinned Go toolchain 1.27.2 — 0 called standard-library vulnerabilities + (govulncheck); `npm audit` clean. + +## 0.2.1 — 2026-10-09 +- TLS Explorer reports the true negotiated version from `supported_versions` + (and labels QUIC as TLS 1.3), instead of the compatibility record version. + Captures the QUIC server port. + +## 0.2.0 — 2026-10-09 (Phase 3: analysis) +- Conversations (TCP/UDP, cross-segment, per-direction bytes) from the index. +- Follow TCP Stream via tshark reassembly, labelled as transport bytes. +- DNS Explorer (cleartext), TLS Explorer (metadata only), Statistics (SVG charts). + +## 0.1.3 — 2026-10-09 +- Reindex interrupted sessions on restart and correct packet counts. +- Actionable access-point activation errors. + +## 0.1.2 — 2026-10-09 +- `KillMode=mixed` so shutdown/upgrade finalises a running capture as *stopped*. +- `dumpcap -F pcapng`; installer AP-mode check runs after `iw` is installed. + +## 0.1.1 — 2026-10-09 +- `rpi_volt` under-voltage fallback where `vcgencmd` is root-only. +- dnsmasq bound to the AP interface only (`except-interface=lo`). + +## 0.1.0 — 2026-10-09 (Phases 1–2) +- Installer/uninstaller/repair/diagnose, systemd units, transactional network + helper with rollback, Argon2id auth + TOTP + CSRF + source allow-list. +- Access point, DHCP/DNS, nftables firewall (default-deny, IPv6 blocked for + clients), management isolation. +- dumpcap capture to hashed pcapng segments, tshark dissection with verified + byte ranges, Wireshark display filters, export/import, crash recovery. +- React analyser (packet list / protocol tree / hex), setup wizard, device, + health, library and settings pages. diff --git a/README.md b/README.md index 965ef46..49bddc4 100644 --- a/README.md +++ b/README.md @@ -61,9 +61,12 @@ make test-docker # tshark/dumpcap tests + network-namespace integration s - [Architecture and design decisions](docs/architecture.md) - [Installation, first run, upgrade and recovery](docs/operations.md) - [Network topology and firewall policy](docs/network.md) +- [Encryption: what PiLens can and cannot see](docs/tls-and-encryption.md) - [Security and privacy model](docs/security.md) - [Forensic evidence handling](docs/forensics.md) - [API](docs/api.md) +- [Development guide](docs/development.md) +- [Troubleshooting](docs/troubleshooting.md) - [Benchmarks](docs/benchmarks.md) - [Testing and hardware acceptance checklist](docs/testing.md) - [Known limitations](docs/limitations.md) diff --git a/docs/api.md b/docs/api.md index 797b927..9093f81 100644 --- a/docs/api.md +++ b/docs/api.md @@ -39,3 +39,20 @@ to 1 MiB, and unknown fields are rejected. Tree nodes: `{label, field, show, pos, size, verified, children}`. `verified` means the frame bytes at `pos..pos+size` were checked against tshark's raw value, so highlighting them is accurate. + +## Analysis (Phase 3) and findings (Phase 5) + +| Method & path | Purpose | +|---|---| +| `GET /captures/{id}/conversations?transport=tcp\|udp` | Conversations from the index: canonical endpoint pair, per-direction bytes, protocols, seg+stream for Follow | +| `GET /captures/{id}/endpoints?limit=` | Top talkers (per-address tx/rx bytes, packets) | +| `GET /captures/{id}/statistics?buckets=` | Totals, protocol hierarchy, size distribution, timeline, TCP stream count | +| `GET /captures/{id}/follow?seg=&stream=` | Follow TCP stream: reassembled payload chunks per direction (hex), via tshark. Transport bytes, not decrypted content. | +| `GET /captures/{id}/dns?limit=` | Cleartext DNS messages: client, resolver, name, type, rcode, answers | +| `GET /captures/{id}/tls?limit=` | `{handshakes[], certificates[], note}`. Handshakes carry SNI, ALPN, negotiated version, cipher. Metadata only. | +| `GET /captures/{id}/findings?min=` | Observed diagnostics: rule, title, severity, count, flows, example packets, reproducing filter, benign-explanation note | + +All are GET, authenticated, and derived from already-captured data; none change +state or touch the network. Conversations and statistics are computed from the +SQLite index (fast, cross-segment); follow/dns/tls/findings call tshark over the +segment files on demand. diff --git a/docs/development.md b/docs/development.md new file mode 100644 index 0000000..1b2c05f --- /dev/null +++ b/docs/development.md @@ -0,0 +1,67 @@ +# Development guide + +## Layout + +``` +cmd/pilens service + CLI (serve, init, setup, reset-password, rlimit-exec) +cmd/pilens-helper the only root component: typed network operations over a socket +internal/config configuration + validation (shared by core and helper) +internal/netgen renders NetworkManager / dnsmasq / nftables (pure, unit-tested) +internal/helper core↔helper protocol + parsers +internal/store SQLite open + migrations +internal/capture capture engine, pcap/pcapng I/O, tshark, analysis, findings +internal/pcapfile bounds-checked pcap/pcapng reader/writer +internal/sysinfo unprivileged host/interface/health probes +internal/auth Argon2id, TOTP, session tokens, login throttling +internal/api HTTP API + static UI serving + SSE +web/ React + TypeScript + Vite UI (embedded via go:embed) +migrations/ ordered SQL, embedded +systemd/ scripts/ packaging/ docs/ tests/ +``` + +## Prerequisites + +Go ≥ 1.27 (the module pins toolchain 1.27.2), Node ≥ 20, Docker for the +integration suite. On a dev box you do not need tshark/dnsmasq/nftables unless +you run those tests directly. + +## Common tasks + +```sh +make test # go vet + go test + tsc --noEmit (no external tools needed) +make test-docker # tshark/dumpcap tests + netns integration suite (needs Docker) +make package # cross-compile arm64, bundle UI + docs + scripts -> dist/*.tar.gz +cd web && npm run dev # UI against a running appliance: PILENS_API=https://:8443 npm run dev +``` + +The dev loop used to build this on real hardware: `make package VERSION=x.y.z`, +`scp` the tarball to the Pi, run `install.sh --yes` (it upgrades in place, +keeping data), then drive the API or UI. `install.sh` backs up and rolls back. + +## Design rules (that reviewers should enforce) + +- **The core never runs as root.** Privileged actions go through the helper, + which has a fixed set of typed operations and re-validates everything. No + generic command execution, ever. +- **Never interpolate user input into a shell or SQL.** All execs use argv + lists; all queries use bound parameters. tshark children run via + `rlimit-exec` with resource limits and `no_new_privs`. +- **Original captures are immutable.** Packet blocks are copied byte-for-byte; + derived data (summaries, dissection) is regenerable and clearly separated. +- **Be honest in the UI.** No fabricated data, no controls that do nothing, no + claims the evidence doesn't support (e.g. TLS version, interception status, + drop vs. UI-drop). A feature that isn't built is absent, not faked. +- **Validation lives in `internal/config` and runs twice** — in the core and + again in the root helper, which does not trust the core. + +## Adding a migration + +Drop `migrations/NNN_name.sql`; it is applied in order on next start and +recorded in `schema_migrations`. Migrations run inside a transaction. + +## Releasing + +1. `make test && make test-docker` green. +2. `go run golang.org/x/vuln/cmd/govulncheck ./...` and `cd web && npm audit`. +3. `make package VERSION=x.y.z`; the tarball carries its own SHA-256. +4. Record hardware results in `docs/testing.md` only after actually running them. diff --git a/docs/tls-and-encryption.md b/docs/tls-and-encryption.md new file mode 100644 index 0000000..7dfe02e --- /dev/null +++ b/docs/tls-and-encryption.md @@ -0,0 +1,71 @@ +# Encryption: what PiLens can and cannot see + +Short version: in capture-only mode PiLens shows you **packets and metadata**, +never decrypted application content. This page explains why, because it is the +single most common misunderstanding about a tool like this. + +## Two completely separate kinds of TLS + +1. **Management TLS** — the HTTPS you use to reach the PiLens web UI on :8443. + PiLens generates its own self-signed certificate for this on first start + (ECDSA P-256, 5-year). It protects *your* connection to the appliance and is + unrelated to anything captured. The installer prints its SHA-256 fingerprint; + verify it the first time your browser warns about it. + +2. **Interception CA** — a separate authority that *would* let PiLens decrypt a + test device's HTTPS, by having that device explicitly trust it. **This is not + implemented in this version** (Phase 4). When it is, it will be opt-in, + per-device, and require you to install and trust the CA on the client + yourself. There is no way to do it silently. + +The two are kept independent on purpose: compromising or distributing an +interception CA must never affect the management channel. + +## Why HTTPS content is unavailable without client trust + +TLS exists precisely to stop a device in the middle from reading traffic. To +decrypt it, the middle device must present a certificate the client trusts — +which means the client must have been configured, by its owner, to trust +PiLens's CA. Without that explicit step, PiLens sees only: + +- The **TLS handshake** metadata sent in clear: SNI (the server name, unless + Encrypted ClientHello is used), offered/negotiated version, cipher, ALPN. +- The **certificate** the server sends — but only for TLS 1.2 and earlier. TLS + 1.3 encrypts the certificate, so the TLS Explorer's certificate tab is usually + empty for modern servers. That is expected. +- Everything after the handshake as **ciphertext**. "Follow TCP stream" will + show it, clearly labelled as encrypted transport bytes, not content. + +## Cannot be sent via DHCP + +A trusted root CA **cannot** be delivered or installed through DHCP, a captive +portal, or any network mechanism. No mainstream OS will trust a CA handed to it +by the network — doing so would defeat TLS entirely. Trust is always a manual, +explicit action by the device owner. This is a feature, not a limitation. + +## QUIC and HTTP/3 + +QUIC carries HTTP/3 inside TLS 1.3 over UDP. PiLens captures it and reads the +clear handshake fields (SNI, ALPN), and labels it `TLS 1.3 (QUIC)`. The payload +is encrypted. Many apps prefer QUIC, so a lot of "web" traffic appears as QUIC +rather than TCP/TLS. + +## Encrypted DNS (DoH / DoT) + +If a client uses DNS-over-HTTPS or DNS-over-TLS, its lookups are inside TLS to a +resolver and do **not** appear in the DNS Explorer — they look like HTTPS/QUIC to +that resolver's address. Only cleartext DNS through the appliance's resolver is +observable. PiLens never presents encrypted DNS as if it were cleartext. + +## Certificate pinning + +Some apps pin to a specific certificate or key and will refuse *any* substitute, +including a trusted interception CA. When Phase 4 lands, such connections will be +reported honestly as failed/pinned rather than silently bypassed — PiLens will +never weaken a client's TLS to force a connection through. + +## IPv6 + +IPv6 is not inspected, so it is disabled on the AP and dropped in the firewall +(see [network.md](network.md)). Clients fall back to IPv4, which is fully +captured. This avoids an uninspected escape path. diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md new file mode 100644 index 0000000..b8d3ebd --- /dev/null +++ b/docs/troubleshooting.md @@ -0,0 +1,22 @@ +# Troubleshooting + +Run `sudo /opt/pilens/diagnose.sh` first — it bundles logs and state with +secrets redacted. Then: + +| Symptom | Check | Fix | +|---|---|---| +| Web UI won't load | `systemctl status pilens`; is your source IP in the management subnet? | `sudo /opt/pilens/repair.sh`; widen the subnet from the console if locked out (see operations.md) | +| Browser certificate warning | Expected (self-signed). Compare the fingerprint the installer printed. | Trust it for this host, or import `/var/lib/pilens/tls/mgmt.crt` | +| Can't log in | — | `sudo -u pilens /opt/pilens/bin/pilens reset-password -user admin` | +| Locked out by `429` | Login throttling after failures | Wait the stated time, or restart `pilens` to clear the limiter | +| No clients get an address | `systemctl status pilens-dnsmasq`; another DHCP server on the AP link? | `repair.sh`; ensure no stray `dnsmasq`/`hostapd` is running | +| AP won't start | `journalctl -u NetworkManager`; radio AP-mode and band/country support | Try WPA2 / 2.4 GHz / a permitted channel; failed applies roll back automatically | +| Clients associate but no internet | Dashboard "forwarding" card; Ethernet default route up? | Check the uplink; `forwarding` must read "forwarding", not "blocked" | +| Capture shows drops | System Health / Findings; CPU or SD-card bound | Use a capture filter; capture to an SSD (operations.md) | +| "interrupted" sessions after power loss | Expected: segments are hashed and re-indexed on restart | Verify hashes in Capture Library → Integrity | +| High CPU / slow filters | Large capture; tshark re-reads segments | Filter narrows after the first run (cached); smaller segment size helps | +| Under-voltage warning on dashboard | `rpi_volt` alarm | Use the official PSU; under-voltage corrupts storage and drops packets | + +If a network change ever leaves the appliance unreachable, every apply keeps a +backup and rolls back on failure; the last-known-good firewall is also restored +at boot before the AP starts.