diff --git a/audits/20260616-1111-functionalityNextSteps.md b/audits/20260616-1111-functionalityNextSteps.md new file mode 100644 index 0000000..9967154 --- /dev/null +++ b/audits/20260616-1111-functionalityNextSteps.md @@ -0,0 +1,81 @@ +# Functionality and Next Steps Report + +## 1. Executive Summary + +This fix pass moves Nexus closer to predictable large-file behavior: file-backed Leviathan documents stay file-backed for save/export, recordings parse incrementally, and unavoidable in-memory parsers now advertise and enforce clear limits. + +The most useful next work is to surface these limits in the Leviathan UI and add user-facing guidance when an operation needs a smaller selection or a streaming implementation. + +## 2. Current Functional Risks + +Leviathan spectrogram conversion still uses bounded in-memory audio/image buffers because the current encoder/decoder APIs operate on byte slices. Very large images, MP3s, WAVs, or decoded MP3 output will now be rejected with an error rather than processed. + +PDF, certificate, and key scanners still operate on bounded whole-document buffers for structural parsing. The cap avoids memory denial of service, but users working with very large firmware or document corpora may need range-based scans in future. + +Recording playback is now line-streamed, but the returned API shape still collects parsed events before sending them to the frontend. Very long recordings are still protected by the playback cap; true progressive playback would require an API contract change. + +## 3. Code Quality Improvements Completed + +Chunk-first behavior: + +- Leviathan working-copy save now writes with `Document.ForEachChunk`. +- Leviathan case export now streams `working.bin` into the zip file on disk. +- Recording playback parses line-by-line instead of reading and splitting the whole file. + +Bounded read behavior: + +- Spectrogram image/audio inputs reject oversize data before decode. +- Image decode checks dimensions before allocating decoded pixels. +- HTTP response helpers cap JSON and error bodies. +- SSH key, VNC key, vault, public-key, PDF, and BinSnoop file reads have explicit limits. + +Responsiveness: + +- Heavy Leviathan spectrogram/export operations no longer hold the engine lock while processing or showing native save dialogs. + +## 4. Frontend Interaction and Error Handling + +Recommended UI follow-up: + +- Show limit-aware messages for spectrogram tools, such as image max size, audio max size, and decoded image dimension limits. +- For PDF/cert/key tools, show “scan a smaller file or selected range” guidance when caps are hit. +- Add a small status line in Leviathan for whether an operation is streaming, bounded-buffer, or selection-based. +- For recording playback, consider progressive playback loading so long recordings do not need all parsed events returned at once. + +## 5. Testing Roadmap + +Completed in this pass: + +- Oversized spectrogram image dimensions are rejected before full decode. +- Leviathan bounded document reads reject oversize documents. +- Patched working-file writes preserve changed bytes. +- Oversized GitHub/update responses are rejected. +- Oversized SSH public keys, vault files, PDFs, and BinSnoop files are rejected. + +Recommended next tests: + +- Wails UI tests for Leviathan oversized image/audio/PDF error messages. +- A case-export test that opens a sparse file-backed document and verifies the zip is produced without materializing `working.bin`. +- A recording playback test with malformed lines and a long valid event stream. +- Browser interaction tests for the Leviathan tab while a long analysis is running. + +## 6. Recommended New Functionality + +Add range-based versions of whole-document Leviathan tools: + +- `keys.scanRange` +- `certs.scanRange` +- `pdf.quickScan` for headers/xref/trailer without full structural recovery +- `spectrogram.renderRange` for selected audio windows + +Add streaming media support later: + +- Decode audio from file/range readers instead of complete byte slices. +- Stream WAV output directly to disk for spectrogram generation. +- Stream image insertion output to disk when carrier audio is large. + +## 7. Caveats + +The new caps trade permissiveness for stability. Files above the limits now fail fast instead of attempting best-effort processing. + +The frontend build and regression checks passed, but I did not run a packaged Wails app manually in this pass. diff --git a/audits/20260616-1111-securityAudit.md b/audits/20260616-1111-securityAudit.md new file mode 100644 index 0000000..b2e3f09 --- /dev/null +++ b/audits/20260616-1111-securityAudit.md @@ -0,0 +1,135 @@ +# Security Audit Report + +## 1. Executive Summary + +This fix-mode audit focused on file-reading and response-reading paths that could allocate complete files or unbounded remote bodies in memory. + +The main risk fixed was local denial of service from large user-selected files, imported Leviathan artefacts, recordings, SSH key files, vault files, and remote update/GitHub API responses. Where the codebase already had a chunked document model, saves and exports now use it. Where a parser still requires a complete byte slice, the code now rejects oversized inputs before allocation. + +Verification passed with targeted package tests, `go test ./...`, frontend regression tests, frontend build, and `git diff --check`. + +## 2. Scope Reviewed + +Reviewed and changed: + +- Leviathan file-backed document save, case export, spectrogram image/audio conversion, PDF parsing, and BinSnoop file helpers. +- Wails API helpers that read SSH public keys and Leviathan image/audio inputs. +- SSH and VNC tunnel private-key loading. +- GitHub client and update-checker HTTP response handling. +- Recording playback parsing. +- Encrypted secrets vault loading. + +Not live-tested: + +- A packaged Wails desktop binary. +- Real remote SSH/VNC/GitHub endpoints. +- Very large real-world media corpora beyond automated guard tests. + +## 3. Methodology + +I searched for `ReadFile`, `ReadAll`, and whole-document `Range(0, size)` patterns, then classified each as chunked, bounded, embedded/test-only, or unsafe. Fixes were kept to low-risk local changes: + +- Prefer `Document.ForEachChunk` for writing/exporting working file bytes. +- Stream line-oriented recordings with `bufio.Scanner`. +- Limit HTTP response reads with `io.LimitReader`. +- Add `os.Stat` size checks before unavoidable bounded `os.ReadFile` calls. +- Add parser-specific caps before whole-document reads. + +## 4. Findings Summary + +| ID | Severity | Confidence | Title | Component | Status | +|---|---:|---:|---|---|---| +| SEC-20260616-001 | Medium | High | Leviathan save/export and spectrogram tools could materialize large files | Leviathan API, spectrogram | Fixed | +| SEC-20260616-002 | Medium | High | Remote API/update responses were read without size limits | GitHub client, updater | Fixed | +| SEC-20260616-003 | Medium | High | Local app data/key reads lacked size caps | SSH keys, VNC keys, vault, BinSnoop | Fixed | +| SEC-20260616-004 | Low | High | Recording playback loaded the whole capped recording file | Recording playback | Fixed | +| SEC-20260616-005 | Low | Medium | Some parsers still require complete bounded buffers | PDF, cert/key scanners, media decoders | Accepted with caps | + +## 5. Detailed Findings + +### SEC-20260616-001: Leviathan full-file allocation paths + +Leviathan save/export and spectrogram operations could duplicate entire working files in memory. Case export also built the zip archive in memory before writing it. + +Fixed in: + +- `internal/api/leviathan.go` +- `internal/leviathan/spectrogram/spectrogram.go` +- `internal/leviathan/pdf/pdf.go` + +Changes: + +- Working-copy save writes via `Document.ForEachChunk`. +- Case export streams `working.bin` directly into `zip.Writer` backed by the destination file. +- Spectrogram encode/view/insert rejects oversized image/audio documents before bounded decode. +- Image decoding checks compressed byte size and decoded pixel dimensions before full image decode. +- PDF parsing rejects files above 128 MiB before whole-document structural parsing. +- Long-running spectrogram/export tools no longer hold the Leviathan engine lock while doing heavy work or native save dialogs. + +### SEC-20260616-002: Unbounded remote response reads + +GitHub and update-check response bodies could be read without an explicit maximum. + +Fixed in: + +- `internal/githubclient/githubclient.go` +- `internal/updater/updater.go` + +Changes: + +- GitHub JSON responses are capped at 4 MiB. +- GitHub error bodies are capped at 64 KiB. +- Update release/manifest responses are capped at 1 MiB. +- Repository archive downloads continue to stream to disk instead of memory. + +### SEC-20260616-003: Local key/vault/helper reads lacked caps + +Several local helper paths read files directly even though the expected files are small. + +Fixed in: + +- `internal/api/api.go` +- `internal/sshclient/sshclient.go` +- `internal/sessions/manager.go` +- `internal/secrets/secrets.go` +- `internal/leviathan/binsnoop/common.go` +- `internal/leviathan/binsnoop/binsnoop.go` + +Changes: + +- SSH public keys are capped at 64 KiB. +- SSH private keys for SSH and VNC tunnel auth are capped at 1 MiB. +- Secret vault files are capped at 10 MiB. +- BinSnoop direct file helpers are capped at 64 MiB; the UI path already uses a bounded buffer. + +### SEC-20260616-004: Recording playback whole-file read + +Recording playback had a 100 MiB file cap, but still read and split the full file in memory. + +Fixed in: + +- `internal/recording/recorder.go` + +Changes: + +- Playback now opens the recording and scans line-by-line. +- Individual playback lines are capped at 8 MiB. +- Existing malformed-event tolerance is preserved. + +## 6. Residual Risk + +Some operations still need bounded complete buffers because downstream parsers or decoders require byte slices: image/audio spectrogram conversion, PDF structural parsing, certificate/key scanning, and small SSH/vault/public-key parsing. These now fail before allocation when inputs exceed explicit caps. + +Truly streaming versions of every binary parser would be a larger Leviathan architecture project. The current fixes remove the unbounded allocation behavior while preserving tool behavior for normal files. + +## 7. Verification + +Passed: + +- `go test ./internal/api ./internal/leviathan/spectrogram ./internal/leviathan/pdf ./internal/leviathan/binsnoop ./internal/updater ./internal/githubclient ./internal/secrets ./internal/recording ./internal/sshclient ./internal/sessions -count=1` +- `go test ./... -count=1` +- `npm --prefix frontend test` +- `npm --prefix frontend run build` +- `git diff --check` + +New tests cover oversized GitHub/update bodies, oversized spectrogram images, bounded Leviathan document reads, chunked patched-file writes, oversized public keys, oversized vault files, oversized PDFs, and BinSnoop file caps. diff --git a/frontend/index.html b/frontend/index.html index e6baa1e..b8ec7fb 100644 --- a/frontend/index.html +++ b/frontend/index.html @@ -23,6 +23,7 @@ +