v3 candidate

audits/20260616-1111-functionalityNextSteps.md

@@ -0,0 +1,81 @@
+# Functionality and Next Steps Report
+
+## 1. Executive Summary
+
+This fix pass moves Nexus closer to predictable large-file behavior: file-backed Leviathan documents stay file-backed for save/export, recordings parse incrementally, and unavoidable in-memory parsers now advertise and enforce clear limits.
+
+The most useful next work is to surface these limits in the Leviathan UI and add user-facing guidance when an operation needs a smaller selection or a streaming implementation.
+
+## 2. Current Functional Risks
+
+Leviathan spectrogram conversion still uses bounded in-memory audio/image buffers because the current encoder/decoder APIs operate on byte slices. Very large images, MP3s, WAVs, or decoded MP3 output will now be rejected with an error rather than processed.
+
+PDF, certificate, and key scanners still operate on bounded whole-document buffers for structural parsing. The cap avoids memory denial of service, but users working with very large firmware or document corpora may need range-based scans in future.
+
+Recording playback is now line-streamed, but the returned API shape still collects parsed events before sending them to the frontend. Very long recordings are still protected by the playback cap; true progressive playback would require an API contract change.
+
+## 3. Code Quality Improvements Completed
+
+Chunk-first behavior:
+
+- Leviathan working-copy save now writes with `Document.ForEachChunk`.
+- Leviathan case export now streams `working.bin` into the zip file on disk.
+- Recording playback parses line-by-line instead of reading and splitting the whole file.
+
+Bounded read behavior:
+
+- Spectrogram image/audio inputs reject oversize data before decode.
+- Image decode checks dimensions before allocating decoded pixels.
+- HTTP response helpers cap JSON and error bodies.
+- SSH key, VNC key, vault, public-key, PDF, and BinSnoop file reads have explicit limits.
+
+Responsiveness:
+
+- Heavy Leviathan spectrogram/export operations no longer hold the engine lock while processing or showing native save dialogs.
+
+## 4. Frontend Interaction and Error Handling
+
+Recommended UI follow-up:
+
+- Show limit-aware messages for spectrogram tools, such as image max size, audio max size, and decoded image dimension limits.
+- For PDF/cert/key tools, show “scan a smaller file or selected range” guidance when caps are hit.
+- Add a small status line in Leviathan for whether an operation is streaming, bounded-buffer, or selection-based.
+- For recording playback, consider progressive playback loading so long recordings do not need all parsed events returned at once.
+
+## 5. Testing Roadmap
+
+Completed in this pass:
+
+- Oversized spectrogram image dimensions are rejected before full decode.
+- Leviathan bounded document reads reject oversize documents.
+- Patched working-file writes preserve changed bytes.
+- Oversized GitHub/update responses are rejected.
+- Oversized SSH public keys, vault files, PDFs, and BinSnoop files are rejected.
+
+Recommended next tests:
+
+- Wails UI tests for Leviathan oversized image/audio/PDF error messages.
+- A case-export test that opens a sparse file-backed document and verifies the zip is produced without materializing `working.bin`.
+- A recording playback test with malformed lines and a long valid event stream.
+- Browser interaction tests for the Leviathan tab while a long analysis is running.
+
+## 6. Recommended New Functionality
+
+Add range-based versions of whole-document Leviathan tools:
+
+- `keys.scanRange`
+- `certs.scanRange`
+- `pdf.quickScan` for headers/xref/trailer without full structural recovery
+- `spectrogram.renderRange` for selected audio windows
+
+Add streaming media support later:
+
+- Decode audio from file/range readers instead of complete byte slices.
+- Stream WAV output directly to disk for spectrogram generation.
+- Stream image insertion output to disk when carrier audio is large.
+
+## 7. Caveats
+
+The new caps trade permissiveness for stability. Files above the limits now fail fast instead of attempting best-effort processing.
+
+The frontend build and regression checks passed, but I did not run a packaged Wails app manually in this pass.

audits/20260616-1111-securityAudit.md

@@ -0,0 +1,135 @@
+# Security Audit Report
+
+## 1. Executive Summary
+
+This fix-mode audit focused on file-reading and response-reading paths that could allocate complete files or unbounded remote bodies in memory.
+
+The main risk fixed was local denial of service from large user-selected files, imported Leviathan artefacts, recordings, SSH key files, vault files, and remote update/GitHub API responses. Where the codebase already had a chunked document model, saves and exports now use it. Where a parser still requires a complete byte slice, the code now rejects oversized inputs before allocation.
+
+Verification passed with targeted package tests, `go test ./...`, frontend regression tests, frontend build, and `git diff --check`.
+
+## 2. Scope Reviewed
+
+Reviewed and changed:
+
+- Leviathan file-backed document save, case export, spectrogram image/audio conversion, PDF parsing, and BinSnoop file helpers.
+- Wails API helpers that read SSH public keys and Leviathan image/audio inputs.
+- SSH and VNC tunnel private-key loading.
+- GitHub client and update-checker HTTP response handling.
+- Recording playback parsing.
+- Encrypted secrets vault loading.
+
+Not live-tested:
+
+- A packaged Wails desktop binary.
+- Real remote SSH/VNC/GitHub endpoints.
+- Very large real-world media corpora beyond automated guard tests.
+
+## 3. Methodology
+
+I searched for `ReadFile`, `ReadAll`, and whole-document `Range(0, size)` patterns, then classified each as chunked, bounded, embedded/test-only, or unsafe. Fixes were kept to low-risk local changes:
+
+- Prefer `Document.ForEachChunk` for writing/exporting working file bytes.
+- Stream line-oriented recordings with `bufio.Scanner`.
+- Limit HTTP response reads with `io.LimitReader`.
+- Add `os.Stat` size checks before unavoidable bounded `os.ReadFile` calls.
+- Add parser-specific caps before whole-document reads.
+
+## 4. Findings Summary
+
+| ID | Severity | Confidence | Title | Component | Status |
+|---|---:|---:|---|---|---|
+| SEC-20260616-001 | Medium | High | Leviathan save/export and spectrogram tools could materialize large files | Leviathan API, spectrogram | Fixed |
+| SEC-20260616-002 | Medium | High | Remote API/update responses were read without size limits | GitHub client, updater | Fixed |
+| SEC-20260616-003 | Medium | High | Local app data/key reads lacked size caps | SSH keys, VNC keys, vault, BinSnoop | Fixed |
+| SEC-20260616-004 | Low | High | Recording playback loaded the whole capped recording file | Recording playback | Fixed |
+| SEC-20260616-005 | Low | Medium | Some parsers still require complete bounded buffers | PDF, cert/key scanners, media decoders | Accepted with caps |
+
+## 5. Detailed Findings
+
+### SEC-20260616-001: Leviathan full-file allocation paths
+
+Leviathan save/export and spectrogram operations could duplicate entire working files in memory. Case export also built the zip archive in memory before writing it.
+
+Fixed in:
+
+- `internal/api/leviathan.go`
+- `internal/leviathan/spectrogram/spectrogram.go`
+- `internal/leviathan/pdf/pdf.go`
+
+Changes:
+
+- Working-copy save writes via `Document.ForEachChunk`.
+- Case export streams `working.bin` directly into `zip.Writer` backed by the destination file.
+- Spectrogram encode/view/insert rejects oversized image/audio documents before bounded decode.
+- Image decoding checks compressed byte size and decoded pixel dimensions before full image decode.
+- PDF parsing rejects files above 128 MiB before whole-document structural parsing.
+- Long-running spectrogram/export tools no longer hold the Leviathan engine lock while doing heavy work or native save dialogs.
+
+### SEC-20260616-002: Unbounded remote response reads
+
+GitHub and update-check response bodies could be read without an explicit maximum.
+
+Fixed in:
+
+- `internal/githubclient/githubclient.go`
+- `internal/updater/updater.go`
+
+Changes:
+
+- GitHub JSON responses are capped at 4 MiB.
+- GitHub error bodies are capped at 64 KiB.
+- Update release/manifest responses are capped at 1 MiB.
+- Repository archive downloads continue to stream to disk instead of memory.
+
+### SEC-20260616-003: Local key/vault/helper reads lacked caps
+
+Several local helper paths read files directly even though the expected files are small.
+
+Fixed in:
+
+- `internal/api/api.go`
+- `internal/sshclient/sshclient.go`
+- `internal/sessions/manager.go`
+- `internal/secrets/secrets.go`
+- `internal/leviathan/binsnoop/common.go`
+- `internal/leviathan/binsnoop/binsnoop.go`
+
+Changes:
+
+- SSH public keys are capped at 64 KiB.
+- SSH private keys for SSH and VNC tunnel auth are capped at 1 MiB.
+- Secret vault files are capped at 10 MiB.
+- BinSnoop direct file helpers are capped at 64 MiB; the UI path already uses a bounded buffer.
+
+### SEC-20260616-004: Recording playback whole-file read
+
+Recording playback had a 100 MiB file cap, but still read and split the full file in memory.
+
+Fixed in:
+
+- `internal/recording/recorder.go`
+
+Changes:
+
+- Playback now opens the recording and scans line-by-line.
+- Individual playback lines are capped at 8 MiB.
+- Existing malformed-event tolerance is preserved.
+
+## 6. Residual Risk
+
+Some operations still need bounded complete buffers because downstream parsers or decoders require byte slices: image/audio spectrogram conversion, PDF structural parsing, certificate/key scanning, and small SSH/vault/public-key parsing. These now fail before allocation when inputs exceed explicit caps.
+
+Truly streaming versions of every binary parser would be a larger Leviathan architecture project. The current fixes remove the unbounded allocation behavior while preserving tool behavior for normal files.
+
+## 7. Verification
+
+Passed:
+
+- `go test ./internal/api ./internal/leviathan/spectrogram ./internal/leviathan/pdf ./internal/leviathan/binsnoop ./internal/updater ./internal/githubclient ./internal/secrets ./internal/recording ./internal/sshclient ./internal/sessions -count=1`
+- `go test ./... -count=1`
+- `npm --prefix frontend test`
+- `npm --prefix frontend run build`
+- `git diff --check`
+
+New tests cover oversized GitHub/update bodies, oversized spectrogram images, bounded Leviathan document reads, chunked patched-file writes, oversized public keys, oversized vault files, oversized PDFs, and BinSnoop file caps.