Docs + guide for Mode B HTTPS interception; record Pi 5 verification

Updated README/architecture/security/limitations/testing/api/CHANGELOG and the
LaTeX user guide (now 14pp) with the Certificates and HTTP Explorer screens.
0 called Go vulns, npm audit clean.

CHANGELOG.md

@@ -1,5 +1,16 @@
 # Changelog
 
+## 0.4.0 — 2026-10-10 (Phase 4: authorised HTTPS inspection, Mode B)
+- Toggleable transparent HTTPS interception via mitmdump in a venv. Off by default.
+- Own RSA-2048 CA generated in-app (`PiLens Interception CA`), downloadable public
+  cert with per-OS install guide + fingerprint; private key never leaves the box.
+- nftables redirect accepts only dnat'd connections (never an open proxy); the
+  proxy's own upstream never loops.
+- HTTP Explorer: transactions with headers and opt-in body viewers (JSON/text/hex/
+  image, sandboxed). Bodies off by default. Pinned/untrusting = reported errors.
+- Verified on the Pi 5: a CA-trusting client's HTTPS decrypted (TLS 1.3), an
+  untrusting client honestly reported as a TLS failure.
+
 All notable changes. Dates are when the work was done on the reference Pi 5.
 
 ## 0.3.1 — 2026-10-09

README.md

@@ -21,10 +21,10 @@ capture or analysis.
 | 1 | Installer, services, auth, AP / DHCP / DNS / firewall, health | implemented |
 | 2 | dumpcap capture, pcapng storage, live analyser, filters, export / import | implemented |
 | 3 | Conversations, follow TCP stream, DNS/TLS explorers, statistics | implemented (HTTP explorer deferred) |
-| 4 | Optional HTTPS inspection (mitmproxy, client-trusted CA) | not started |
+| 4 | Optional HTTPS inspection (mitmproxy, client-trusted CA) | implemented |
 | 5 | Hardening, Findings, benchmarks, dependency audits | implemented |
 
-HTTP Explorer (Phase 3 in the spec) is deferred with Phase 4, since useful HTTP content requires interception. Pages for later phases are not in the UI. PiLens never shows controls that do
+Pages for later phases are not in the UI. PiLens never shows controls that do
 nothing. For what was tested and how, see [docs/testing.md](docs/testing.md).
 
 ## Quick start

docs/api.md

@@ -56,3 +56,19 @@ All are GET, authenticated, and derived from already-captured data; none change
 state or touch the network. Conversations and statistics are computed from the
 SQLite index (fast, cross-segment); follow/dns/tls/findings call tshark over the
 segment files on demand.
+
+## Interception (Phase 4, Mode B)
+
+| Method & path | Purpose |
+|---|---|
+| `GET /interception` | `{config, ca, running, port, note}` |
+| `PUT /interception {enabled, capture_bodies, max_body_kb, exclude_hosts[]}` | Toggle/configure; applies firewall redirect + proxy via helper |
+| `GET /interception/ca.crt` | Download the CA **public** certificate (never the key) |
+| `POST /interception/ca/rotate` | Regenerate the CA (only while disabled) |
+| `POST /interception/clear` | Delete all intercepted HTTP flows + retained bodies |
+| `GET /http?q=&client=&limit=` | Decrypted HTTP transactions, newest first |
+| `GET /http/{id}` | One transaction (headers, timing, TLS, flags) |
+| `GET /http/{id}/body/{req\|resp}` | Stored body as an opaque download (nosniff, sandbox CSP); only if body capture was on |
+
+Interception is off by default. Live transactions also arrive on `/events` as
+`http` events.

docs/architecture.md

@@ -167,7 +167,19 @@ dumpcap -i IF [-f BPF] -w -  ──pcapng──▶ splitter ──▶ seg-000001
 
 ## Not yet implemented (phase 3+)
 
-Implemented since Phase 2: conversations, follow TCP stream, DNS explorer, TLS
+Implemented: conversations, follow TCP stream, DNS/TLS explorers, statistics,
-explorer (metadata only), statistics. Not in the navigation until they work:
+findings, and Mode B HTTPS interception (CA + mitmdump + HTTP explorer). Not yet
-HTTP explorer and TLS interception (modes B and C, deferred together), findings,
+built: Mode C strict fail-closed policy, and capture scheduling.
-and capture scheduling.
+
+## Interception (Mode B)
+
+`pilens-proxy.service` runs `mitmdump` (in `/opt/pilens/proxy-venv`) as the
+unprivileged `pilens` user, in transparent mode. An in-app RSA CA
+(`internal/proxy`) is written to the mitmproxy confdir; the public cert is
+downloadable, the key never is. When the operator toggles interception on, the
+core writes `proxy.env`, then the root helper re-renders the nft table with a
+`prerouting` redirect (`iifname <ap> tcp dport {80,443} redirect`) plus an
+`input` rule accepting only `ct status dnat` connections, and starts the service.
+A mitmdump addon writes each transaction as NDJSON; the core tails it into SQLite
+and the HTTP Explorer. Toggling off removes the redirect and stops the service.
+Interception is off by default and the appliance is pure capture-only until enabled.

docs/guide/pilens-guide.tex

@@ -189,8 +189,7 @@ ports, DNS, TLS handshakes, timing, the lot.
 scrambled. PiLens shows you \emph{who} talked to \emph{whom}, \emph{when}, over
 \emph{what} --- the server name, the TLS version, the certificate when it is sent
 in the clear --- but \textbf{not the contents}. That is TLS doing its job, not
-PiLens failing. Decryption needs a separate, explicit step on each device (a
+PiLens failing. Decryption needs a separate, explicit step on each device (Mode B, \S3.9), and \textbf{a trusted certificate can never be pushed down the
-future feature), and \textbf{a trusted certificate can never be pushed down the
 network} --- the device's owner must install it by hand. Anything that claimed
 otherwise would be lying to you.
 \end{truth}
@@ -201,7 +200,8 @@ otherwise would be lying to you.
     self-signed, so your browser warns once. Check the fingerprint the installer
     printed, trust it, move on.
   \item \textbf{The interception padlock} (decrypting a device's HTTPS) is a
-    \emph{separate}, opt-in, per-device thing that is not part of this version.
+    \emph{separate}, opt-in, per-device thing: a toggle on the Certificates page.
+    See \S3.9.
 \end{itemize}
 
 % ======================= 3. THE SCREENS =======================
@@ -327,6 +327,43 @@ spells out the innocent explanations. PiLens will never tell you ``you've been
 hacked'' from a retransmission.
 \end{truth}
 
+\subsection{HTTPS inspection (Mode B) --- the opt-in one}
+\begin{tldr}
+A toggle on \textbf{Certificates}. Download the appliance's CA, install and trust
+it on a device you own, flip the switch, and that device's HTTPS shows up decrypted
+in the \textbf{HTTP Explorer}. Off by default.
+\end{tldr}
+
+\screen{certificates}{Certificates \& HTTPS Inspection --- the toggle, the CA to download, per-device install steps, and the privacy options.}
+
+\begin{truth}
+This is the only part that can see inside HTTPS, and it only works \emph{with
+consent}: you must install the CA on each device by hand and turn trust on. It
+\textbf{cannot} be pushed over the network or via DHCP. Pinned apps, QUIC and
+private trust stores still can't be read --- they show up as honest errors, never
+silently bypassed. The CA's private key never leaves the Pi and is not
+downloadable.
+\end{truth}
+
+\textbf{To use it:} Certificates \textgreater\ \emph{Download CA certificate}
+\textgreater\ install it on your test device (the page has per-OS steps; iOS needs
+the extra ``Certificate Trust Settings'' toggle) \textgreater\ verify the
+fingerprint matches \textgreater\ flip \emph{Interception} on.
+
+\begin{heads}
+Turning interception on while a device has \emph{not} installed the CA will break
+that device's HTTPS (it correctly refuses the unknown certificate). Install first,
+toggle second --- and toggle it back off when you're done.
+\end{heads}
+
+\begin{tip}
+\textbf{Bodies are off by default.} The Explorer shows headers, status and timing
+for everything; request/response bodies are only stored if you opt in on the
+Certificates page, for a named investigation. Clear them when done.
+\end{tip}
+
+\screen{http}{HTTP Explorer --- decrypted transactions from trusting devices, with headers and (opt-in) body viewers.}
+
 \subsection{The housekeeping screens}
 \begin{description}[leftmargin=0pt, style=nextline, itemsep=3pt]
   \item[\textcolor{amber}{Capture Library}] Every saved session. Rename, tag,
@@ -334,6 +371,8 @@ hacked'' from a retransmission.
     integrity (SHA-256 per file + a manifest).
   \item[\textcolor{amber}{Devices}] Who has been on the Wi-Fi. Label them, add
     notes, jump to their packets. \mn{Randomised MACs show a badge; PiLens never pretends they identify hardware.}
+  \item[\textcolor{amber}{Certificates}] The HTTPS-inspection toggle, CA
+    download and device install guide (\S3.9).
   \item[\textcolor{amber}{Network Settings}] Change the access point, DHCP, DNS,
     forwarding and management access. Applied as one transaction with automatic
     rollback.

docs/limitations.md

@@ -3,12 +3,14 @@
 **Encryption.** PiLens captures packets; it does not decrypt them. HTTPS, QUIC/HTTP-3, TLS 1.3 with
 Encrypted ClientHello, DNS-over-HTTPS/TLS, and VPN traffic are visible only as encrypted
 packets plus whatever metadata the protocol exposes in clear (for TLS ≤1.3 without ECH: SNI,
-versions, ALPN; for TLS 1.2: certificates). The optional HTTPS inspection mode (client-installed
+versions, ALPN; for TLS 1.2: certificates). The optional HTTPS inspection mode (Mode B) **is** available: enable it on the Certificates page,
-CA via mitmproxy) is **not implemented** in this version, and even when it is, it will not work
+install the appliance CA on each authorised test device, and decrypted transactions appear in the
-against certificate pinning, private trust stores, mutual TLS, QUIC, or apps that bypass proxies.
+HTTP Explorer. It does **not** work against certificate pinning, private trust stores, mutual TLS,
+QUIC/HTTP-3, or apps that bypass the proxy — these are reported honestly as not intercepted, never
+bypassed. It is off by default; while off, PiLens is pure capture-only.
 
-**Not yet implemented:** HTTP explorer, findings/diagnostics rules, capture scheduling,
+**Not yet implemented:** the strict inspection mode (Mode C fail-closed policy), capture scheduling,
-the strict inspection mode, WireGuard, packet comparison. They are not shown in the UI.
+WireGuard, packet comparison. They are not shown in the UI.
 
 **TLS explorer shows metadata only.** SNI, ALPN, negotiated version and cipher, plus any
 certificate sent in the clear (TLS ≤1.2). TLS 1.3 encrypts the certificate, so the certificate

docs/security.md

@@ -28,12 +28,17 @@ sensitive system.
   and `systemctl` with fixed argv; and nothing else. There is no generic
   command execution.
 - **pilens-dnsmasq**: Debian's dnsmasq, which drops privileges itself.
+- **pilens-proxy** (Mode B, optional): `mitmdump` as the unprivileged `pilens`
+  user, sandboxed (`ProtectSystem=strict`, `NoNewPrivileges`). Started/stopped by
+  the helper on toggle. The interception CA key is 0600 and never downloadable;
+  only the root helper adds the firewall redirect.
 
 ## Secrets
 
 | Secret | Where | Exposure |
 |---|---|---|
 | Admin password | Argon2id hash in SQLite (0640 pilens) | Never returned |
+| Interception CA key | `proxy/ca/mitmproxy-ca.pem` (0600 pilens) | Never downloadable; only the public cert is served. Separate from the management CA. |
 | Session tokens | Only the SHA-256 is stored | Cookie only |
 | Wi-Fi passphrase | SQLite + helper state (0600 root) + NM keyfile (0600 root) | Write-only in the API. Never in a process argv. Redacted in diagnostics. |
 | Setup token | `/var/lib/pilens/setup-token` (0600) | Deleted when the first admin is created |
@@ -41,7 +46,9 @@ sensitive system.
 
 ## Data protection
 
-- PiLens currently stores no decrypted payloads, because it decrypts nothing.
+- With interception **off** (the default) PiLens decrypts nothing and stores no
+  payloads. With it on, HTTP metadata is stored; bodies are retained only when the
+  operator explicitly opts in, size-limited, and clearable, with views audited.
 - Captures are deleted by retention (oldest unpreserved first) or explicitly
   by an admin, who must confirm. Deleting a session removes its files and all
   index rows.

docs/testing.md

@@ -67,6 +67,7 @@ Only tests that were actually run are listed.
 | H15 | **pass** | `setcap -r dumpcap` → `repair.sh` restored `cap_net_admin,cap_net_raw=eip`, all services ✓ |
 | H16 | **partial** | Loopback worst-case: 108 MB/s / 4,000 pkt/s with 14.7% drops honestly reported. Real Wi-Fi iperf3 not run. See [benchmarks.md](benchmarks.md). |
 | H17 | **pass** | 263 MB import indexed in 3.2 s (~3,200 pkt/s); filters ~1.2 s; UI responsive. |
+| Interception (Mode B) | **pass** | On the Pi 5: CA generated + downloaded (public only, 0 key lines). Toggle on → proxy active on 8081, nft redirect + `ct status dnat` accept present. A CA-trusting client (curl `--cacert`) decrypted https://example.com and api.github.com (TLS 1.3); an untrusting client was refused and logged as a TLS failure. HTTP Explorer showed all three. Toggle off → redirect removed, proxy stopped, masquerade intact. Full device trust (phone) is the operator's manual step. |
 | Findings | **pass** | On the iPhone capture: 18 retransmissions/7 flows, 1 SYN retry, 7 resets, 17 dup-ACKs, 3 NXDOMAIN, 1 ICMP unreachable — each linking to evidence packets. |
 | H9, H10, H14 | not run | Need physical cable/client actions. |
 

docs/tls-and-encryption.md

@@ -12,15 +12,36 @@ single most common misunderstanding about a tool like this.
    unrelated to anything captured. The installer prints its SHA-256 fingerprint;
    verify it the first time your browser warns about it.
 
-2. **Interception CA** — a separate authority that *would* let PiLens decrypt a
+2. **Interception CA** — a separate authority that lets PiLens decrypt a test
-   test device's HTTPS, by having that device explicitly trust it. **This is not
+   device's HTTPS, *after* that device's owner explicitly installs and trusts it.
-   implemented in this version** (Phase 4). When it is, it will be opt-in,
+   Generated in-app (RSA-2048, `PiLens Interception CA`), downloadable from the
-   per-device, and require you to install and trust the CA on the client
+   Certificates page (public cert only — the key never leaves the appliance).
-   yourself. There is no way to do it silently.
+   Interception is a toggle, **off by default**. There is no way to make a device
+   trust it silently or over the network.
 
 The two are kept independent on purpose: compromising or distributing an
 interception CA must never affect the management channel.
 
+## Mode B: authorised interception (optional, off by default)
+
+Turn it on from **Certificates & HTTPS Inspection**. PiLens then transparently
+redirects AP clients' TCP 80/443 to a local mitmdump, which presents certificates
+signed by the appliance CA. A device that has installed and trusts that CA sees a
+valid chain, so its HTTPS is decrypted and the transactions appear in the **HTTP
+Explorer**. A device that has *not* installed it sees an untrusted certificate and
+its connections fail — which is why interception is useless (and harmless) until
+you do the manual install step on each device you own.
+
+- Redirect is `iifname <ap>` only, TCP 80/443. The proxy accepts only
+  connections the redirect created (`ct status dnat`), so it is never an open
+  proxy for clients.
+- **Excluded hosts** (pinned apps, privacy-sensitive sites) are passed straight
+  through, still encrypted.
+- **Bodies are not retained by default.** Enable it per-investigation; the data
+  is on the appliance disk until cleared, and views/exports are audited.
+- Pinned apps, private trust stores, mutual TLS and QUIC cannot be intercepted
+  and are reported as such — never bypassed, never downgraded.
+
 ## Why HTTPS content is unavailable without client trust
 
 TLS exists precisely to stop a device in the middle from reading traffic. To