andy/mapsize
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 4 Wiki Activity

Remove the Go implementation; build and release the Rust version for every OS #2

Merged
andydixon merged 12 commits from rust-port into master 2026-10-08 22:25:29 +02:00
Conversation 0 Commits 12 Files Changed 155
+5778 -14597
155 changed files with 5778 additions and 14597 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/actions/cross-tools/action.yml
+16
View File
@@ -0,0 +1,16 @@
name: cross-tools
description: Rust targets, zig, cargo-zigbuild and cross for scripts/build.sh
runs:
using: composite
steps:
- shell: bash
run: |
rustup target add x86_64-unknown-linux-musl aarch64-unknown-linux-musl i686-unknown-linux-musl \
armv7-unknown-linux-musleabihf riscv64gc-unknown-linux-musl x86_64-pc-windows-gnu \
aarch64-pc-windows-gnullvm x86_64-unknown-freebsd x86_64-unknown-netbsd
curl -sSLo zig.tar.xz https://ziglang.org/download/0.16.0/zig-x86_64-linux-0.16.0.tar.xz
echo "70e49664a74374b48b51e6f3fdfbf437f6395d42509050588bd49abe52ba3d00 zig.tar.xz" | sha256sum -c
mkdir -p "$HOME/zig" && tar -xJf zig.tar.xz -C "$HOME/zig" --strip-components=1 && rm zig.tar.xz
echo "$HOME/zig" >>"$GITHUB_PATH"
cargo install cargo-zigbuild --version 0.23.4 --locked
cargo install cross --version 0.2.5 --locked
.github/workflows/ci.yml
+26 -17
View File
@@ -4,29 +4,38 @@ on:
pull_request:
permissions:
contents: read
env:
CARGO_TERM_COLOR: always
jobs:
lint:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v5
- run: rustup component add rustfmt clippy
- run: cargo fmt --check
- run: cargo clippy --locked --all-targets -- -D warnings
test:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
os: [ubuntu-24.04, ubuntu-24.04-arm, macos-15, macos-15-intel, windows-2025]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod
- name: format
if: runner.os == 'Linux'
run: test -z "$(gofmt -l .)"
- run: go vet ./...
- run: go test ./...
- name: race
if: runner.os != 'Windows'
run: go test -race ./...
- name: cross-build
if: runner.os == 'Linux'
- uses: actions/checkout@v5
- run: cargo test --release --locked --all-targets
- run: cargo run --release --locked -- --no-ui .
cross:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v5
- uses: ./.github/actions/cross-tools
- name: every target but macOS (built natively by the test job)
run: TARGETS="linux/amd64 linux/arm64 linux/386 linux/arm linux/riscv64 windows/amd64 windows/arm64 freebsd/amd64 netbsd/amd64" scripts/build.sh
- name: static binaries run under qemu
run: |
for t in linux/arm64 linux/386 darwin/arm64 windows/amd64 freebsd/amd64; do
GOOS=${t%/*} GOARCH=${t#*/} CGO_ENABLED=0 go build -o /dev/null ./cmd/mapsize
sudo apt-get update -q && sudo apt-get install -yq qemu-user
for t in "arm64 qemu-aarch64" "arm qemu-arm" "riscv64 qemu-riscv64"; do
${t#* } dist/mapsize-linux-${t%% *} --no-ui src >/dev/null
done
.github/workflows/release.yml
+46
View File
@@ -0,0 +1,46 @@
# On a version tag: build every target, package, and publish a GitHub
# release whose notes are the tag's CHANGELOG.md section.
name: release
on:
push:
tags: ["v*"]
permissions:
contents: read
env:
CARGO_TERM_COLOR: always
jobs:
build-cross:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v5
- uses: ./.github/actions/cross-tools
- run: TARGETS="linux/amd64 linux/arm64 linux/386 linux/arm linux/riscv64 windows/amd64 windows/arm64 freebsd/amd64 netbsd/amd64" scripts/build.sh
- uses: actions/upload-artifact@v4
with: {name: bin-cross, path: dist/mapsize-*}
build-macos:
runs-on: macos-15
steps:
- uses: actions/checkout@v5
- run: rustup target add x86_64-apple-darwin aarch64-apple-darwin
- run: TARGETS="darwin/amd64 darwin/arm64" scripts/build.sh
- uses: actions/upload-artifact@v4
with: {name: bin-macos, path: dist/mapsize-*}
publish:
needs: [build-cross, build-macos]
runs-on: ubuntu-24.04
permissions:
contents: write
steps:
- uses: actions/checkout@v5
- uses: actions/download-artifact@v4
with: {path: dist, merge-multiple: true}
- run: scripts/release.sh "$GITHUB_REF_NAME" dist dist/release
- name: release
env:
GH_TOKEN: ${{ github.token }}
run: |
v=${GITHUB_REF_NAME#v}
awk -v h="## $v" 'index($0, h) == 1 {f=1; next} /^## /{f=0} f' CHANGELOG.md >notes.md
gh release create "$GITHUB_REF_NAME" --title "$GITHUB_REF_NAME" --notes-file notes.md dist/release/*
.gitignore
+3 -3
View File
@@ -1,5 +1,5 @@
/target
/dist
bin/
*.msz
*.prof
*.test
/mapsize
!src/snapshot/testdata/*.msz
ARCHITECTURE.md
+100 -73
View File
@@ -5,36 +5,40 @@ part relies on. It is kept in sync with the code; if they disagree, the code
is wrong or this file is stale — fix one of them.
```
cmd/mapsize CLI entry point, flag parsing, mode selection
cmd/treegen synthetic test-tree generator
internal/brand product name / file extensions (rename in one place)
internal/inventory flat node store, aggregation, indexes, categories
internal/scan bounded concurrent scanner (controller + workers)
internal/platform OS-specific metadata (build-tagged files)
internal/treemap squarified layout, LOD, spatial navigation (pure geometry)
internal/filter query lexer / parser / evaluator
internal/snapshot versioned binary snapshot format + comparison
internal/duplicate staged duplicate detection
internal/export JSON / CSV / text reports
internal/config settings, themes, JSON config file
internal/tui Bubble Tea model, canvas renderer, views, modals
internal/textutil display sanitising, width-aware truncation, units
src/main.rs CLI entry point, flag parsing
src/app.rs mode selection: TUI, reports, snapshots, comparison
src/brand.rs product name / file extensions (rename in one place)
src/inventory/ flat node store, aggregation, indexes, categories
src/scan.rs bounded concurrent scanner (controller + workers)
src/cancel.rs cancellation token shared by scans and background work
src/platform.rs OS-specific metadata and file operations (cfg blocks)
src/treemap/ squarified layout, LOD, spatial navigation (pure geometry)
src/filter/ query lexer / parser / evaluator
src/snapshot/ versioned binary snapshot format + comparison
src/duplicate.rs staged duplicate detection
src/export.rs JSON / CSV / text reports
src/config.rs settings, themes, JSON config file
src/tui/ event loop, model, canvas renderer, views, modals
src/textutil.rs display sanitising, width-aware truncation, units
examples/treegen synthetic test-tree generator
examples/bench in-process benchmarks
```
Dependency direction is strictly downward: `tui` → (`treemap`, `filter`,
`inventory`, `scan`, …); `scan` → (`inventory`, `platform`); `treemap` depends
on nothing but the standard library. The TUI framework (Bubble Tea v2) is only
imported by `internal/tui` and `cmd/mapsize`.
on nothing but the standard library. The terminal library (crossterm) is only
used by `tui`.
## Inventory representation
`inventory.Tree` is an append-only arena of `Node` records addressed by
`NodeID` (`uint32`). Nodes are stored in fixed-size chunks
(`[][]Node`, 65 536 nodes per chunk), so growing the arena never copies or
`NodeId` (`u32`). Nodes are stored in fixed-size chunks
(`Vec<Vec<Node>>`, 65 536 nodes per chunk), so growing the arena never copies or
moves existing nodes and peak memory never doubles during growth.
A node holds: parent ID, first-child / next-sibling IDs (an intrusive child
list — no per-directory slices), name (the single path component, not the full
list — no per-directory vectors), name (the single path component as raw
bytes, not the full
path), kind, flags, own logical size, own allocated size, mtime, mode,
uid/gid, link count, and aggregates (total logical, total allocated, file
count, directory count, error count). Directory-only data — per-category byte
@@ -42,7 +46,7 @@ totals used for colouring — lives in a side table indexed by a directory index
so files do not pay for it.
Full paths are never stored; `Tree.Path(id)` walks parents. Extensions are
interned into a small table and stored as a `uint16`. All sizes are `int64`.
interned into a small table and stored as a `u16`. All sizes are `i64`.
Node IDs are stable for the lifetime of a `Tree`: nodes are never removed or
reordered. A rescan produces a new `Tree`. This is what makes selection
@@ -51,11 +55,11 @@ persistence trivial (see below).
## Scanner worker architecture
```
+------------------ controller goroutine ------------------+
+------------------ controller thread ---------------------+
| owns: Tree (write lock), pending FIFO of dir NodeIDs, |
| hard-link table, visited-dir table |
+----------------------------------------------------------+
| jobs (bounded chan, cap = workers*2) ^ results (bounded chan)
| jobs (bounded channel, cap = workers*2) ^ results (bounded channel)
v |
worker 1 .. worker N (fixed pool, N from --workers / mode)
```
@@ -77,9 +81,9 @@ while the only consumer of that queue is also blocked. mapsize avoids it by
construction:
* Only the controller produces jobs. Workers only produce *results*.
* The controller's main loop is a single `select` that simultaneously offers
the next pending job to the bounded `jobs` channel *and* receives from the
bounded `results` channel. It never performs a blocking send without also
* The controller's main loop is a single `crossbeam_channel::Select` that
simultaneously offers the next pending job to the bounded `jobs` channel
*and* receives from the bounded `results` channel. It never performs a blocking send without also
being ready to receive. Therefore a worker blocked on `results <-` is always
eventually served, and the controller is never blocked on `jobs <-` alone.
* Discovered-but-unscheduled directories wait in the controller's pending FIFO
@@ -94,11 +98,12 @@ directory with 5 M entries neither balloons worker memory nor stalls the UI.
Completion: the controller tracks `inflight` (dispatched directories whose
final chunk has not arrived). The scan is finished when `pending` is empty and
`inflight == 0`. Cancellation (`context.Context`) stops dispatch immediately;
workers check the context between chunks and between entries, and the
controller drains outstanding results so no goroutine leaks.
`inflight == 0`. Cancellation (`cancel::Cancel`, which the controller also
selects on) stops dispatch immediately; workers check the token between
chunks, and the controller drains outstanding results and joins every
worker, so no thread leaks.
Stress tests (`internal/scan/scan_test.go`) run with 1 worker and job/result
Stress tests (`scan::tests`) run with 1 worker and job/result
channel capacity 1 over trees with thousands of directories to prove progress
under maximum backpressure.
@@ -112,26 +117,27 @@ for a 1 M-file tree at depth 20 that is ~20 K ancestor updates instead of
~20 M. There is no reconciliation pass: totals are exact at every moment for
the data seen so far.
The UI reads the tree under `Tree.RLock`; the controller applies batches of
results under `Tree.Lock`, holding it for at most a few milliseconds (it
The tree is shared as `Arc<RwLock<Tree>>`. The UI reads it under the read
lock; the controller applies batches of results under the write lock, holding it for at most a few milliseconds (it
releases the lock between batches). This is the only lock in the data path.
**Concurrency invariants**
1. Only the scan controller (or a snapshot loader before publication) mutates
a `Tree`.
2. Every read of `Tree` node data from another goroutine holds `RLock`.
2. Every read of `Tree` node data from another thread holds the read lock.
3. Nodes are never deleted; a `NodeID` once observed stays valid.
4. Workers share nothing mutable except the bounded channels and atomic
progress counters.
5. No goroutine ever takes `RLock` while already holding it. Go's
`RWMutex` blocks new readers once a writer waits, so a recursive read
lock can deadlock against the scan controller. The TUI takes the lock
once per entry point (`View`, `handleKey`, mouse handlers); helpers they
call assume it is held. Background commands (filter, duplicates,
snapshot save) take their own lock on their own goroutine.
6. After a scan finishes, the UI goroutine may become the writer (move to
trash updates aggregates under `Lock`). Trash is refused while scanning.
5. No thread ever takes the read lock while already holding it. The
standard `RwLock` may block new readers once a writer waits, so a
recursive read lock can deadlock against the scan controller. The TUI
takes the lock once per entry point (render, key and mouse handlers) and
passes `&Tree` to the helpers it calls. Background tasks (filter,
duplicates, snapshot save) take their own lock on their own thread.
6. After a scan finishes, the UI thread may become the writer (move to
trash updates aggregates under the write lock). Trash is refused while
scanning.
## Hard links and sizes
@@ -166,31 +172,41 @@ releases the lock between batches). This is the only lock in the data path.
## TUI event flow
```
scan controller ──(applies under Lock)──> Tree
scan controller ──(applies under write lock)──> Tree
│ atomic progress counters
▼
tea.Tick (10 Hz while scanning) ──> tickMsg ──> Update marks frame dirty
key/mouse/resize ──> Update ──> state change ──> frame dirty
View(): if dirty → RLock tree → layout (if layout dirty) → paint canvas → string
tick (10 Hz while scanning) ─────┐
crossterm events (input thread) ─┼─> select ─> Model::update ─> state change ─> frame dirty
background task results ─────────┘
render: if dirty → read-lock tree → layout (if layout dirty) → paint canvas → write changed lines
```
There is never one message per file: the scanner does not talk to the UI at
all except for a single `scanDoneMsg`. The UI polls at a fixed rate.
`tui::run` puts the terminal in raw mode on the alternate screen and selects
over three crossbeam channels: terminal events from a dedicated input thread,
a timer tick, and messages from background tasks (filter, duplicates,
snapshot save, scan done). There is never one message per file: the scanner
does not talk to the UI at all except for the scan-done notification. The UI
polls its progress counters at a fixed rate.
`View()` is called by Bubble Tea after *every* `Update`. The model therefore
caches the rendered frame and only repaints when something visible changed.
The model caches the rendered frame and only repaints when something
visible changed; several updates in a row cost at most one render.
Expensive per-directory work (sorting children by size) is cached per data
version. During a scan the version advances at most every 400 ms, so the
layout does not jitter at frame rate even though labels and counters update
at the 10 Hz refresh.
A panic hook restores the terminal (leaves the alternate screen, disables
raw mode and mouse reporting) before the panic is reported; SIGTERM quits
cleanly.
## Rendering: the canvas
All drawing goes to an in-memory `Canvas` of cells (grapheme, fg, bg, attrs).
Views and modals paint into the canvas; the canvas is serialised to a string
with minimal SGR changes. Bubble Tea's renderer diffs frames and downsamples
true-colour to the detected profile (256 / 16 / none), so the drawing code
Views and modals paint into the canvas; the canvas is serialised with minimal
SGR changes, converted to the detected colour profile (truecolour / 256 / 16 /
none), and only the changed span of each changed line is written, inside a
synchronized-output block where the terminal supports it. The drawing code
uses RGB throughout. For 256-colour terminals the canvas quantizes colours
itself (redmean distance over the 6×6×6 cube and grey ramp, never the
palette-dependent first 16 entries): the stock conversion maps dark tints to
@@ -226,7 +242,7 @@ visible. Groups are selectable and inspectable.
## Spatial navigation algorithm
`treemap.Neighbour(rects, selected, dir)` is pure geometry:
`treemap::neighbour(blocks, selected, dir)` is pure geometry:
1. Candidates must lie in the requested half-plane: for RIGHT, `c.X ≥
sel.X + sel.W − tolerance` (with the candidate's centre to the right of the
@@ -241,18 +257,18 @@ visible. Groups are selectable and inspectable.
## Terminal resize architecture
`tea.WindowSizeMsg` only records the new size and marks layout dirty. The next
`View()` recomputes panel geometry for the new size class (large / medium /
A resize event only records the new size and marks layout dirty. The next
render recomputes panel geometry for the new size class (large / medium /
small / too-small), reruns the treemap layout for the new treemap rectangle
(never stretches old coordinates), and repaints. Because work happens only in
`View()` and the frame cache absorbs repeated calls, a storm of resize events
(never stretches old coordinates), and repaints. Because work happens only
at render time and the frame cache absorbs repeated calls, a storm of resize events
costs at most one layout per rendered frame. Modal geometry is derived from
the current size every frame, clamped to the screen, and becomes scrollable
when content does not fit.
## Selection persistence
Selection is a `NodeID` (or, for the synthetic LOD group, `-1 - parentID`).
Selection is a `NodeId` (or, for the synthetic LOD group, `-1 - parent`).
Until the user moves the selection it follows the largest block, which
changes while a scan runs; after that it is sticky. After any relayout the renderer looks the ID up in the
new rectangle list. If it is absent (filtered out, collapsed into a group,
@@ -261,32 +277,42 @@ nearest visible ancestor, then the largest block.
## Modal architecture
A modal is a small interface: `Title()`, `Lines(width)`, `Footer()`, and
`HandleKey`. A generic frame renders any modal centred, clamped to the screen
A modal is a small trait: `title()`, `content(width)`, `footer()`, and
`key`. A generic frame renders any modal centred, clamped to the screen
(min 1-cell margin), with a scrollable viewport when content exceeds height.
The base screen is dimmed before the modal is painted. While a modal is open
it receives all keys; `Esc` always closes it.
## OS abstraction
`internal/platform` exposes `ReadDir(path, func(chunk []Entry) error)` and
`Meta` structs. Implementations:
`platform` exposes `read_dir(path, chunk, |entries| …)`, `stat`/`lstat`,
`Meta`, trash, delete, reveal and name lookups. `platform/unix.rs` and
`platform/windows.rs` hold the implementations:
* `readdir_linux.go` — `getdents` via `os.File.ReadDir` + `statx` relative to
the directory fd (one syscall per entry, gives birth time).
* `readdir_unix.go` — darwin/BSD: `fstatat` relative to the directory fd.
* `readdir_windows.go` — `os.File.ReadDir` + `Lstat`; reparse points
(junctions, symlinks) are never descended by default.
* `fs_*.go` — virtual-filesystem detection, owner name lookup.
* Linux — `readdir` on an `O_DIRECTORY` descriptor + `statx` (via the raw
system call, so static musl builds work) relative to that descriptor: one
system call per entry, birth time included; `fstatat` if the kernel lacks
`statx`. Virtual filesystems are detected with `statfs`. freedesktop.org
trash; `xdg-open` reveals.
* macOS, FreeBSD, NetBSD — the same descriptor-relative walk with
`fstatat`. macOS moves items to `~/.Trash` (or the volume's `.Trashes`)
and reveals with `open -R`; the BSDs have no trash or reveal, so `d` is
only offered as a permanent delete when headless.
* Windows — `std::fs::read_dir` and metadata. Allocated size is reported as
unknown and there is no inode identity, so hard links are not
deduplicated and symlinks and junctions are never followed. The Recycle
Bin (`SHFileOperationW`) and `explorer /select,` reveal; permanent delete
is not offered.
No other package switches on `runtime.GOOS`.
The terminal code has the only other `cfg` blocks (signals and writing the
restore sequence).
## Security handling for hostile filenames
Filenames and snapshot contents are untrusted.
* Every string that reaches the terminal passes through
`textutil.Sanitize`, which replaces C0/C1 control characters (including
`textutil::sanitize`, which replaces C0/C1 control characters (including
ESC, CSI, BEL, DEL) with visible escapes (`\x1b`), replaces invalid UTF-8
with U+FFFD, and neutralises bidi override characters. The canvas also
refuses control characters as a second line of defence. Internal names keep
@@ -299,9 +325,10 @@ Filenames and snapshot contents are untrusted.
(≤ 1 EiB) and node count (at most one node per byte of snapshot file, 1 Mi floor); caps the decompressed body at 64× the file size
(decompression bombs fail fast); and verifies a SHA-256 trailer.
Aggregates are recomputed with saturating arithmetic rather than trusted.
* No shell is ever invoked. "Open/reveal" and trash use direct `exec` with
argument vectors or native APIs; paths are passed as single arguments.
* No shell is ever invoked. "Open/reveal" and trash use `std::process::Command`
with argument vectors or native APIs; paths are passed as single arguments.
* Permanent delete (headless only) checks the item's owner and type against
the scan and removes it through an `os.Root` on the parent directory, so
a symlink swapped in since the scan cannot redirect it.
the scan and removes it through descriptors opened from the parent
directory without following symlinks (`openat`/`unlinkat` with
`O_NOFOLLOW`), so a symlink swapped in since the scan cannot redirect it.
* `--read-only` disables every mutating action in the UI.
CHANGELOG.md
+22
View File
@@ -1,5 +1,27 @@
# Changelog
## Unreleased
### Changed
- Rewritten in Rust. Flags, keys, views, themes, the configuration file and
the snapshot format are unchanged: snapshots from earlier releases open as
before, and older releases open new ones. Measured against 1.2.0 on the
same machine, scans are about 1.3× faster with a third less memory, cold
treemap frames about 3× faster, and filters up to 1.8× faster (see
PERFORMANCE.md).
- Release builds: Linux (x86-64, ARM64, 32-bit x86, ARMv7, RISC-V 64),
macOS (Intel, Apple silicon), Windows (x86-64, ARM64), FreeBSD and NetBSD
(x86-64), built and published by CI. OpenBSD and FreeBSD/ARM64 builds are
no longer produced.
- `--log` and `--log-level` are accepted but no log is written yet;
`--cpuprofile` and `--memprofile` are removed.
- Install from source with `cargo install`; the Homebrew formula builds with
Rust.
### Fixed
- A search containing certain non-ASCII text (for example `voilà`) no longer
hangs.
## 1.2.0 — 2026-10-03
### Changed
rust/Cargo.lock → Cargo.lock
Generated
+1
View File
@@ -194,6 +194,7 @@ dependencies = [
"sha2",
"unicode-segmentation",
"unicode-width",
"windows-sys 0.61.2",
]
[[package]]
rust/Cargo.toml → Cargo.toml
+4 -1
View File
@@ -3,7 +3,7 @@ name = "mapsize"
version = "1.2.0"
edition = "2021"
license = "GPL-3.0-or-later"
description = "Interactive terminal disk usage analyser (Rust port)"
description = "Interactive terminal disk usage analyser with a live, navigable treemap"
[dependencies]
libc = "0.2"
@@ -17,6 +17,9 @@ serde_json = "1"
unicode-segmentation = "1"
unicode-width = "0.2"
[target.'cfg(windows)'.dependencies]
windows-sys = { version = "=0.61.2", features = ["Win32_Foundation", "Win32_System_Time", "Win32_UI_Shell"] }
[profile.release]
lto = true
codegen-units = 1
Cross.toml
+11
View File
@@ -0,0 +1,11 @@
# The cross image for NetBSD lacks libexecinfo, which Rust's standard library
# links on NetBSD. Take it (and libelf, which it needs) from the official
# NetBSD 9.2 base set, matching the image's libc, checked against the
# release's published SHA512.
[target.x86_64-unknown-netbsd]
pre-build = [
"curl -fsSL -o /tmp/base.tar.xz https://archive.netbsd.org/pub/NetBSD-archive/NetBSD-9.2/amd64/binary/sets/base.tar.xz",
"echo '961511ec63f4803529aeb057632e2b8cf87e73bfdfed445d7521b607f573245359163b9f6b6a158fa80de7a994cf3727af384ec42ac8d9ccdd59a87197a41b55 /tmp/base.tar.xz' | sha512sum -c",
"tar -xJf /tmp/base.tar.xz -C /usr/local/x86_64-unknown-netbsd --strip-components=2 ./usr/lib/libexecinfo.so ./usr/lib/libexecinfo.so.0 ./usr/lib/libexecinfo.so.0.0 ./usr/lib/libelf.so ./usr/lib/libelf.so.2 ./usr/lib/libelf.so.2.0",
"rm /tmp/base.tar.xz",
]
DEVELOPMENT.md
+59 -44
View File
@@ -1,61 +1,78 @@
# Development
Requires Go (see `go.mod`). No CGO, no external tools.
Requires `rustup`: `rust-toolchain.toml` pins the compiler (so local builds
match CI), and `Cargo.lock` pins the dependencies. No C toolchain beyond what
Rust itself needs.
```sh
make build # bin/mapsize, bin/treegen
make test # go test ./...
make race # go test -race ./...
make vet fmt-check
make benchmark # go test -bench
make fuzz # short fuzz runs of snapshot / treemap / filter
make release # release archives and packages in bin/release
make build # target/release/mapsize
make test # cargo test --release
make lint # clippy, warnings are errors
make fmt-check # rustfmt
make benchmark # in-process benchmarks (layout, render, filter): examples/bench.rs
make treegen # synthetic test-tree generator: examples/treegen.rs
make release # local release assets in dist/release (see below)
```
CI (`.github/workflows/ci.yml`) runs rustfmt and clippy, the tests on Linux
(x86-64 and ARM64), macOS (Intel and Apple silicon) and Windows, and
cross-builds every release target, running the ARM and RISC-V Linux
binaries under qemu.
## Platforms
`scripts/build.sh` builds `dist/mapsize-OS-ARCH`:
| OS | Architectures | Rust target | Built with |
|---|---|---|---|
| Linux | amd64, arm64, 386, arm (v7), riscv64 | `*-unknown-linux-musl` (static) | cargo-zigbuild + zig |
| Windows | amd64, arm64 | `x86_64-pc-windows-gnu`, `aarch64-pc-windows-gnullvm` | cargo-zigbuild + zig |
| macOS | amd64, arm64 | `*-apple-darwin` | cargo on a Mac (or zigbuild) |
| FreeBSD, NetBSD | amd64 | `x86_64-unknown-{freebsd,netbsd}` | cross + Docker (`Cross.toml` adds NetBSD's libexecinfo) |
Platform differences live in `src/platform.rs` (and `cfg` blocks where the
terminal code needs them); nothing else switches on the OS.
## Releasing
Example for 1.2.0; substitute the real version throughout.
1. Update the version in `internal/brand/brand.go` and `docs/mapsize.1`,
and add a `## 1.2.0` section to CHANGELOG.md. Commit.
1. Update the version in `Cargo.toml` (then `cargo check` to refresh
`Cargo.lock`) and `docs/mapsize.1`, and turn `## Unreleased` in
CHANGELOG.md into `## 1.2.0 — DATE`. Commit.
2. Tag and push:
```sh
git tag -a v1.2.0 -m "mapsize 1.2.0"
git push origin master v1.2.0
```
3. Build everything (archives, .deb/.rpm/.apk/Arch packages, SHA256SUMS
into `bin/release`; the first run downloads `nfpm`):
```sh
make release VERSION=v1.2.0
```
4. Publish the GitHub release with the changelog section as notes:
```sh
awk '/^## 1.2.0/{f=1;next} /^## /{f=0} f' CHANGELOG.md > /tmp/notes.md
gh release create v1.2.0 --title v1.2.0 --notes-file /tmp/notes.md bin/release/*
```
3. The tag starts `.github/workflows/release.yml`, which builds every
target, packages them (archives, .deb/.rpm/.apk/Arch packages,
SHA256SUMS) and publishes the GitHub release with the changelog section
as its notes. Watch it with `gh run watch`. `make release` builds the
same assets locally into `dist/release`, for whichever targets the host
can build (`scripts/build.sh`, then `scripts/release.sh`).
4. Check the release page once the workflow has finished.
5. Update Homebrew. This needs the tag pushed (step 2), because it
checksums GitHub's source tarball for that tag:
```sh
make brew-formula VERSION=v1.2.0 # writes bin/homebrew/mapsize.rb
make brew-formula VERSION=v1.2.0 # writes dist/homebrew/mapsize.rb
git clone https://github.com/andydixon/homebrew-tap /tmp/tap
cp bin/homebrew/mapsize.rb /tmp/tap/Formula/mapsize.rb
cp dist/homebrew/mapsize.rb /tmp/tap/Formula/mapsize.rb
git -C /tmp/tap commit -am "mapsize 1.2.0" && git -C /tmp/tap push
```
Users get it with `brew upgrade mapsize`. Template:
`packaging/homebrew/mapsize.rb.in`.
6. Publish to the apt, dnf and pacman repository at repo.dixon.cx. This
builds signed .deb, .rpm and Arch packages from the tag and rebuilds the
repository indexes (needs docker, gpg, apt-ftparchive and the signing key
in `~/.config/repo.dixon.cx/gnupg`):
repository indexes (needs cargo-zigbuild and zig, docker, gpg,
apt-ftparchive and the signing key in `~/.config/repo.dixon.cx/gnupg`):
```sh
packaging/repo.sh v1.2.0
```
The same script, with its own project block, publishes vault.
Always pass `VERSION=`: otherwise it comes from `git describe`, which only
matches the tag when HEAD is exactly the tagged commit. `make release` and
the Homebrew formula stamp the version into the binary; `make build` and
`go install` use the value in `brand.go`, which is why step 1 updates it.
The binary reports the version in `Cargo.toml`; `scripts/release.sh`
refuses to package a tag that does not match it.
The snapshot format is stable at version 1: a format change must bump
`snapshot.Version` and keep the reader for version 1.
@@ -65,17 +82,16 @@ The snapshot format is stable at version 1: a format change must bump
`treegen` creates synthetic trees:
```sh
bin/treegen --out /tmp/t --directories 50000 --files 1000000 --max-depth 20 --sparse --unicode
bin/mapsize /tmp/t
cargo run --release --example treegen -- --out /tmp/t --directories 50000 --files 1000000 --max-depth 20 --sparse --unicode
target/release/mapsize /tmp/t
```
## Benchmarks
```sh
make benchmark # Go micro-benchmarks (layout, render, filter, scan)
make benchmark # in-process: layout, render, navigation, filters
scripts/bench-scan.sh /path 1 2 4 8 # wall time and peak RSS per worker count
sudo DROP_CACHES=1 scripts/bench-scan.sh /path # cold-cache runs
MAPSIZE_MEM_ROOT=/path go test ./internal/scan -run TestMemoryPerNode -v
```
Paste results into PERFORMANCE.md with the machine description. Never edit
@@ -84,26 +100,25 @@ numbers by hand.
## Terminal restoration checks
```sh
MAPSIZE_DEBUG_PANIC=ui bin/mapsize . # panics on the first key press
MAPSIZE_DEBUG_PANIC=scanner bin/mapsize / # panics in a scanner goroutine
MAPSIZE_DEBUG_PANIC=ui target/release/mapsize . # panics on the first key press
MAPSIZE_DEBUG_PANIC=scanner target/release/mapsize / # panics in a scanner thread (aborts)
stty -a # echo/icanon must be back on
```
## Profiling
```sh
bin/mapsize --no-ui --cpuprofile cpu.prof --memprofile mem.prof /path
go tool pprof -http=: cpu.prof
```
`--log FILE --log-level debug` writes a log without disturbing the TUI.
Paths are logged only at `debug`/`trace` level.
Build with symbols (`CARGO_PROFILE_RELEASE_STRIP=false
CARGO_PROFILE_RELEASE_DEBUG=1 cargo build --release`) and use the platform
profiler, for example `perf record -g target/release/mapsize --no-ui /path`
on Linux or Instruments on macOS. `examples/bench.rs` times the hot UI paths
in-process.
## Layout
See ARCHITECTURE.md. Rules of thumb:
* `internal/treemap` and `internal/filter` must not import UI packages.
* Only `internal/platform` may contain OS-specific code (build tags).
* All strings derived from filenames go through `textutil.Sanitize` before
* `treemap` and `filter` must not depend on `tui`.
* OS-specific code lives in `platform` (`cfg` attributes); the terminal code
has the only other `cfg` blocks.
* All strings derived from filenames go through `textutil::sanitize` before
reaching a terminal.
Makefile
+22 -48
View File
@@ -1,80 +1,54 @@
BIN := bin
PKG := ./...
VERSION ?= $(shell git describe --tags --always --dirty 2>/dev/null || echo dev)
LDFLAGS := -s -w
PREFIX ?= /usr/local
VERSION ?= $(shell sed -n 's/^version = "\(.*\)"/\1/p' Cargo.toml | head -1)
.PHONY: build install man brew-formula test race vet fmt fmt-check benchmark fuzz release clean treegen
.PHONY: build install man test lint fmt fmt-check benchmark treegen release brew-formula clean
build:
go build -trimpath -ldflags '$(LDFLAGS)' -o $(BIN)/mapsize ./cmd/mapsize
go build -trimpath -ldflags '$(LDFLAGS)' -o $(BIN)/treegen ./cmd/treegen
cargo build --release --locked
install: build
install -Dm755 $(BIN)/mapsize $(DESTDIR)$(PREFIX)/bin/mapsize
install -Dm755 target/release/mapsize $(DESTDIR)$(PREFIX)/bin/mapsize
install -Dm644 docs/mapsize.1 $(DESTDIR)$(PREFIX)/share/man/man1/mapsize.1
man:
man -l docs/mapsize.1
test:
go test $(PKG)
cargo test --release --locked
race:
go test -race $(PKG)
vet:
go vet $(PKG)
lint:
cargo clippy --locked --all-targets -- -D warnings
fmt:
gofmt -w .
cargo fmt
fmt-check:
@test -z "$$(gofmt -l .)" || (gofmt -l . && echo 'run make fmt' && exit 1)
cargo fmt --check
benchmark:
go test -run=NONE -bench=. -benchmem $(PKG)
cargo run --release --locked --example bench
fuzz:
go test -run=NONE -fuzz=FuzzLoad -fuzztime=30s ./internal/snapshot
go test -run=NONE -fuzz=FuzzSquarify -fuzztime=30s ./internal/treemap
go test -run=NONE -fuzz=FuzzParse -fuzztime=30s ./internal/filter
treegen:
cargo build --release --locked --example treegen
# Cross-built archives plus .deb/.rpm/.apk/Arch packages in $(BIN)/release.
REL := $(BIN)/release
NFPM := go run github.com/goreleaser/nfpm/v2/cmd/nfpm@v2.41.1
# Binaries for every target this host can build (see scripts/build.sh), then
# archives, Linux packages and SHA256SUMS in dist/release. Tagged releases are
# built on GitHub's runners by .github/workflows/release.yml instead.
release:
rm -rf $(REL) && mkdir -p $(REL)
@set -e; v=$(VERSION); v=$${v#v}; \
for t in linux/amd64 linux/arm64 linux/386 linux/arm darwin/amd64 darwin/arm64 \
windows/amd64 windows/arm64 freebsd/amd64 freebsd/arm64 openbsd/amd64 netbsd/amd64; do \
os=$${t%/*}; arch=$${t#*/}; ext=; [ $$os = windows ] && ext=.exe; \
name=mapsize-$$v-$$os-$$arch; d=$(REL)/$$name; echo "$$os/$$arch"; mkdir -p $$d; \
CGO_ENABLED=0 GOOS=$$os GOARCH=$$arch GOARM=7 go build -trimpath \
-ldflags '$(LDFLAGS) -X github.com/andydixon/mapsize/internal/brand.Version='$$v \
-o $$d/mapsize$$ext ./cmd/mapsize; \
cp README.md CHANGELOG.md LICENSE docs/mapsize.1 $$d/; \
if [ $$os = windows ]; then (cd $(REL) && zip -qr $$name.zip $$name); \
else tar -C $(REL) -czf $(REL)/$$name.tar.gz $$name; fi; \
if [ $$os = linux ]; then mkdir -p $(REL)/pkg && cp $$d/mapsize $(REL)/pkg/; \
a=$$arch; [ $$a = arm ] && a=arm7; \
for f in deb rpm apk archlinux; do \
VERSION=$$v ARCH=$$a $(NFPM) pkg -f nfpm.yaml -p $$f -t $(REL)/; done; rm -rf $(REL)/pkg; fi; \
rm -rf $$d; \
done; \
cd $(REL) && sha256sum * > SHA256SUMS
scripts/build.sh || true
scripts/release.sh $(VERSION) dist dist/release
# Homebrew formula for the tagged release. The tag must already be pushed:
# the checksum is of GitHub's source tarball for that tag.
BREW_REPO := https://github.com/andydixon/mapsize
brew-formula:
@set -e; v=$(VERSION); v=$${v#v}; \
case $$v in *-*) echo "brew-formula: HEAD is not exactly a release tag ($$v)"; exit 1;; esac; \
url=$(BREW_REPO)/archive/refs/tags/v$$v.tar.gz; \
sum=$$(curl -fsSL "$$url" | sha256sum | cut -d' ' -f1) || { echo "cannot fetch $$url (tag pushed?)"; exit 1; }; \
mkdir -p $(BIN)/homebrew; \
sed -e '/^#/d' -e "s/@VERSION@/$$v/g" -e "s/@SHA256@/$$sum/" packaging/homebrew/mapsize.rb.in > $(BIN)/homebrew/mapsize.rb; \
echo "wrote $(BIN)/homebrew/mapsize.rb (v$$v, sha256 $$sum)"
mkdir -p dist/homebrew; \
sed -e '/^#/d' -e "s/@VERSION@/$$v/g" -e "s/@SHA256@/$$sum/" packaging/homebrew/mapsize.rb.in > dist/homebrew/mapsize.rb; \
echo "wrote dist/homebrew/mapsize.rb (v$$v, sha256 $$sum)"
clean:
rm -rf $(BIN)
cargo clean
rm -rf dist
PERFORMANCE.md
+69 -61
View File
@@ -3,107 +3,115 @@
All numbers below were measured, not estimated. Re-run them on your own
hardware before relying on them; filesystem caches dominate scan timings.
## Test machine (2026-10-02)
## Test machine (2026-10-08)
| | |
|---|---|
| CPU | Intel Core i7-7700 @ 3.60 GHz, 4 cores / 8 threads |
| RAM | 32 GiB |
| OS | Linux 7.0, Go 1.27.1 |
| OS | Linux 7.0, Rust 1.91.1 (release profile: LTO, one codegen unit) |
| Filesystem | ext4 on Linux software RAID (`/dev/md2`) over two 7200 rpm SATA HDDs |
The host is shared; its load average was 5–9 during these runs. Each figure
is the median (and best) of 7 runs.
## Scanner
Dataset: `~/go/pkg/mod` — 388,307 files + 65,518 directories (458,510
inventory nodes), 10.5 GiB allocated. **Warm cache only**: dropping the page
cache needs root, which was not available, so these measure CPU and syscall
Dataset: a source-package cache directory: 388,307 files + 65,518
directories (458,510 inventory nodes), 10.5 GiB allocated. **Warm cache
only**: dropping the page cache needs root, so these measure CPU and syscall
cost, not disk seeks. Cold-cache behaviour on HDDs will be dominated by seek
time; expect fewer workers to be optimal there.
`mapsize --no-ui --workers N ~/go/pkg/mod`, wall time, 3 runs each
(`/usr/bin/time`):
`mapsize --no-ui --workers N PATH`:
| workers | run 1 | run 2 | run 3 | entries/s (best) | peak RSS |
|---:|---:|---:|---:|---:|---:|
| 1 | 1.77 s | 1.67 s | 1.65 s | ~278,000 | 117 MB |
| 2 | 1.06 s | 0.99 s | 1.00 s | ~463,000 | 120 MB |
| 4 | 0.59 s | 0.59 s | 0.62 s | ~777,000 | 117 MB |
| 8 (balanced default) | 0.48 s | 0.50 s | 0.51 s | ~955,000 | 121 MB |
| 16 | 0.45 s | 0.44 s | 0.43 s | ~1,066,000 | 121 MB |
| 32 | 0.43 s | 0.44 s | 0.42 s | ~1,092,000 | 122 MB |
| workers | median | best | peak RSS |
|---:|---:|---:|---:|
| 1 | 1.52 s | 1.44 s | 75 MB |
| 8 (balanced default) | 0.47 s | 0.43 s | 76 MB |
| 32 | 0.46 s | 0.43 s | 80 MB |
For comparison, `du -s` on the same tree (warm): 1.59 s, 34 MB RSS.
At 8 workers that is about 1,000,000 entries a second, in under 80 MB for
458k nodes. Most of the time is spent in the `statx` and `getdents` system
calls.
CPU profile at 8 workers: ~60 % of samples in `statx`/`getdents` syscalls;
path reconstruction for job dispatch ~7 %; aggregation is negligible.
### Memory
`go test ./internal/scan -run TestMemoryPerNode` with
`MAPSIZE_MEM_ROOT=~/go/pkg/mod`: 458,510 nodes, 69.1 MiB live heap,
**~158 bytes per node** (node record ~112 bytes plus name storage and
directory side tables). Extrapolated, 10 M entries need ~1.6 GiB.
| other runs over the same tree | median | peak RSS |
|---|---:|---:|
| scan + `--largest-files 50` | 0.60 s | 76 MB |
| scan + `--json` (streamed to /dev/null) | 0.86 s | 76 MB |
## Snapshots
Same 458,510-node tree:
| operation | median | peak RSS |
|---|---:|---:|
| scan + save | 0.75 s | 77 MB |
| load (`mapsize --no-ui file.msz`) | 0.23 s | 78 MB |
| load a snapshot written by mapsize 1.2.0 | 0.26 s | 78 MB |
| operation | result |
|---|---:|
| snapshot file size (gzip level 1) | 6.7 MB (~15 bytes/node) |
| scan + save | 1.03 s |
| load (`mapsize --no-ui file.msz`), 3 runs | 0.28 / 0.29 / 0.32 s, 81 MB RSS |
| warm-cache rescan for comparison | 0.55 s |
Loading decompresses and verifies SHA-256 on a separate goroutine, overlapping
with parsing. Against a warm cache the win is ~1.9×; against a cold cache or
a network filesystem, where scans are dominated by I/O latency, it is far
larger (not yet measured here).
Loading decompresses and verifies SHA-256 on one thread while another parses.
Against a warm cache that is about twice as fast as rescanning; against a
cold cache or a network filesystem, where scans are dominated by I/O
latency, the gain is far larger (not yet measured here).
## UI
`go test ./internal/tui -bench Big` on a synthetic in-memory tree of
1,008,201 nodes (200 directories × 40 subdirectories × 100 files, plus one
flat directory with 200,000 files), 200×60 terminal:
`make benchmark` (`examples/bench.rs`), best of 5 runs, on a synthetic
in-memory tree of 1,008,201 nodes (200 directories × 40 subdirectories × 100
files, plus one flat directory with 200,000 files), 200×60 terminal:
| operation | time per op |
|---|---:|
| cold frame at root (sort children, LOD, layout incl. nested previews, paint) | 33.8 ms |
| resize to a new size (layout + paint, child lists cached) | 2.0 ms |
| arrow-key move + re-render | 1.5 ms |
| zoomed into the 200,000-entry directory, cold | 29.5 ms |
| render of a small tree (2.4 K nodes) at 200×60 | 0.66 ms |
| resize relayout of the small tree | 0.45 ms |
| cold frame at root (sort children, LOD, layout incl. nested previews, paint) | 11.34 ms |
| resize to a new size (layout + paint, child lists cached) | 1.50 ms |
| arrow-key move + re-render | 1.31 ms |
| zoomed into the 200,000-entry directory, cold | 9.37 ms |
During a scan the size caches are invalidated at most every 400 ms, so a
cold frame (worst case above ~34 ms) costs under 10 % of one core; other
frames reuse caches. Resize events only record the new size; layout happens
once per rendered frame, so a resize storm cannot queue up work.
During a scan the size caches are invalidated at most every 400 ms, so even
a cold frame costs a few percent of one core. Resize events only record the
new size; layout happens once per rendered frame, so a resize storm cannot
queue up work.
### Geometry
`go test ./internal/treemap -bench .`:
| operation | time |
|---|---:|
| squarified layout, 300 items | 89 µs |
| spatial neighbour lookup among 300 blocks | 2.1 µs |
| squarified layout, 300 items | 8.4 µs |
| spatial neighbour lookup among 300 blocks | 1.1 µs |
### Filtering (1,008,201 nodes)
| query | time |
|---|---:|
| `size > 1MB` | 31 ms |
| `*.dat AND size > 500k` | 119 ms |
| `path contains sub3` | 112 ms |
| `size > 1MB` | 26.75 ms |
| `*.dat AND size > 500k` | 67.84 ms |
| `path contains sub3` | 63.80 ms |
Filters run on a background goroutine with cancellation and a 120 ms
keystroke debounce; the UI never waits for them.
Filters run on a background thread with cancellation and a 120 ms keystroke
debounce; the UI never waits for them. Path predicates build each path from
the cached parent directory path into a reused buffer, and `*.ext` globs are
a suffix test.
## Compared with mapsize 1.2.0
mapsize 1.2.0 was the last release of the previous implementation, written
in Go. Both were run alternately on the same machine, over the same tree, in
the same session:
| | 1.2.0 | current | |
|---|---:|---:|---|
| scan, 8 workers (median) | 0.63 s | 0.47 s | 1.34× faster |
| scan, 1 worker (median) | 1.89 s | 1.52 s | 1.24× faster |
| peak RSS while scanning | 117 MB | 76 MB | |
| scan + save snapshot | 1.22 s | 0.75 s | 1.63× faster |
| load snapshot | 0.34 s | 0.23 s | 1.48× faster |
| scan + JSON export | 1.27 s, 153 MB | 0.86 s, 76 MB | 1.48× faster |
| cold frame, 1M nodes | 36.01 ms | 11.34 ms | 3.18× faster |
| squarified layout, 300 items | 84.3 µs | 8.4 µs | 10.07× faster |
| arrow key + re-render | 1.50 ms | 1.31 ms | 1.14× faster |
| filter `*.dat AND size > 500k` | 121.40 ms | 67.84 ms | 1.79× faster |
## Known costs / future work
* Path predicates now build each path from the cached parent directory path
(244 ms → 112 ms); globs are dominated by `filepath.Match`.
* Node records could shrink (owner/group/mode/link count are only needed for
display and some filters) if memory on 10 M+ entry trees matters.
* Cold-cache HDD benchmarks still need to be run with root access.
README.md
+10 -9
View File
@@ -1,17 +1,18 @@
# mapsize
An interactive, full-terminal disk usage analyser built around a live,
navigable treemap. Think WinDirStat + ncdu + WizTree, in one Go binary that
runs over SSH.
navigable treemap. Think WinDirStat + ncdu + WizTree, in one Rust binary
that runs over SSH.
> `mapsize` is a working name; branding lives in `internal/brand`.
> `mapsize` is a working name; branding lives in `src/brand.rs`.
## Status
Version 1.0. See [CHANGELOG.md](CHANGELOG.md), [ROADMAP.md](ROADMAP.md) for
Version 1.2. See [CHANGELOG.md](CHANGELOG.md), [ROADMAP.md](ROADMAP.md) for
what comes next, and [PERFORMANCE.md](PERFORMANCE.md) for measured numbers.
Tested on Linux; macOS and Windows builds are produced but trash and
file-manager integration have not yet been tested there.
Releases cover Linux, macOS, Windows, FreeBSD and NetBSD. It is tested on
Linux; CI also runs the test suite on macOS and Windows, but trash and
file-manager integration have not yet been tried by hand there.
## What it does
@@ -70,9 +71,9 @@ the map or scrolls lists. Everything works without a mouse (`--no-mouse`).
## Install
```sh
go install github.com/andydixon/mapsize/cmd/mapsize@v1.0.0
# or
make build # → bin/mapsize
cargo install --locked --git https://github.com/andydixon/mapsize
# or, from a checkout
make build # → target/release/mapsize
sudo make install # binary + man page under /usr/local (PREFIX=… to change)
# Homebrew (macOS and Linux)
ROADMAP.md
+2 -2
View File
@@ -3,14 +3,14 @@
Status markers: `[x]` done, `[~]` partial, `[ ]` not started.
## 0.1 — inventory engine
- [x] Go project skeleton, Makefile, CI
- [x] project skeleton, Makefile, CI
- [x] portable filesystem scanner, bounded worker pool, no-deadlock scheduler
- [x] inventory node model, incremental aggregation
- [x] cancellation, progress counters, error accounting
- [x] CLI: `--top`, `--json`, `--csv`, `--no-ui`
## 0.2 — TUI shell
- [x] full-screen Bubble Tea v2 app, canvas renderer, frame cache
- [x] full-screen terminal app, canvas renderer, frame cache
- [x] responsive layout (large / medium / small / too-small)
- [x] resize handling, status bar, progressive display, directory list
cmd/mapsize/main.go
-200
View File
@@ -1,200 +0,0 @@
// Command mapsize is an interactive terminal disk usage analyser.
package main
import (
"flag"
"fmt"
"io"
"log/slog"
"os"
"runtime"
"runtime/pprof"
"strings"
"github.com/andydixon/mapsize/internal/app"
"github.com/andydixon/mapsize/internal/brand"
"github.com/andydixon/mapsize/internal/config"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/scan"
"github.com/andydixon/mapsize/internal/snapshot"
"github.com/andydixon/mapsize/internal/textutil"
)
type multiFlag []string
func (m *multiFlag) String() string { return strings.Join(*m, ",") }
func (m *multiFlag) Set(s string) error { *m = append(*m, s); return nil }
func main() {
os.Exit(run(os.Args[1:]))
}
func run(args []string) int {
settings, cfgErr := config.Load()
fs := flag.NewFlagSet(brand.Name, flag.ContinueOnError)
fs.Usage = func() { usage(fs) }
var (
excludes multiFlag
workers = fs.String("workers", settings.Workers, "concurrency: conservative, balanced, aggressive or a number")
follow = fs.String("follow-symlinks", settings.FollowSymlinks, "none, same-filesystem or all")
oneFS = fs.Bool("one-file-system", settings.OneFileSystem, "do not cross filesystem boundaries")
oneFSx = fs.Bool("x", false, "short for --one-file-system")
readOnly = fs.Bool("read-only", false, "disable all modifying actions")
noUI = fs.Bool("no-ui", false, "scan without the interactive interface and print a summary")
jsonOut = fs.Bool("json", false, "print the inventory as JSON")
csvOut = fs.Bool("csv", false, "print the inventory as CSV")
depth = fs.Int("depth", -1, "maximum depth for --json/--csv (-1 = unlimited)")
top = fs.Int("top", 0, "print the N largest entries of the root")
largest = fs.Int("largest-files", 0, "print the N largest files")
dups = fs.Bool("duplicates", false, "find duplicate files (non-interactive)")
save = fs.String("save", "", "save a snapshot to `FILE` after scanning")
snap = fs.String("snapshot", "", "alias for --save")
load = fs.String("load", "", "open a saved snapshot `FILE` instead of scanning")
scanPath = fs.String("scan", "", "path to scan (alternative to a positional argument)")
compare = fs.Bool("compare", false, "compare two snapshots: --compare OLD NEW")
si = fs.Bool("si", settings.Units == "si", "use SI units (kB, MB) instead of IEC (KiB, MiB)")
logical = fs.Bool("apparent", settings.SizeMode == "logical", "size by logical (apparent) size instead of disk usage")
theme = fs.String("theme", settings.Theme, "colour theme: default, dark, high-contrast, mono")
colorMode = fs.String("color", "auto", "colour support: auto, truecolor, 256, 16, none")
noMouse = fs.Bool("no-mouse", !settings.Mouse, "disable mouse support")
logFile = fs.String("log", "", "write a debug log to `FILE`")
logLevel = fs.String("log-level", "info", "log level: error, warn, info, debug, trace")
cpuProfile = fs.String("cpuprofile", "", "write a CPU profile to `FILE`")
memProfile = fs.String("memprofile", "", "write a heap profile to `FILE` on exit")
version = fs.Bool("version", false, "print version and exit")
)
fs.Var(&excludes, "exclude", "exclude `PATTERN` (glob on name, or path prefix if it contains a separator); repeatable")
var positional []string
for {
if err := fs.Parse(args); err != nil {
if err == flag.ErrHelp {
return 0
}
return 2
}
args = fs.Args()
if len(args) == 0 {
break
}
positional = append(positional, args[0])
args = args[1:]
}
if *version {
fmt.Printf("%s %s (%s/%s)\n", brand.Name, brand.Version, runtime.GOOS, runtime.GOARCH)
return 0
}
if cfgErr != nil {
fmt.Fprintf(os.Stderr, "%s: warning: config: %v (using defaults)\n", brand.Name, cfgErr)
}
textutil.SI = *si
if *logFile != "" {
closeLog, err := setupLog(*logFile, *logLevel)
if err != nil {
fmt.Fprintf(os.Stderr, "%s: %v\n", brand.Name, err)
return 2
}
defer closeLog()
} else {
slog.SetDefault(slog.New(slog.NewTextHandler(io.Discard, nil)))
}
if *cpuProfile != "" {
f, err := os.Create(*cpuProfile)
if err == nil {
pprof.StartCPUProfile(f)
defer pprof.StopCPUProfile()
}
}
if *memProfile != "" {
defer func() {
if f, err := os.Create(*memProfile); err == nil {
runtime.GC()
pprof.WriteHeapProfile(f)
f.Close()
}
}()
}
cfg := &app.Config{
NoUI: *noUI, JSON: *jsonOut, CSV: *csvOut, Depth: *depth, Top: *top, LargestFiles: *largest,
Duplicates: *dups, ReadOnly: *readOnly, Theme: *theme, Mouse: !*noMouse, Color: *colorMode, Load: *load,
Settings: settings,
}
if *logical {
cfg.SizeMode = inventory.SizeLogical
}
cfg.Save = *save
if cfg.Save == "" {
cfg.Save = *snap
}
if *scanPath != "" {
positional = append([]string{*scanPath}, positional...)
}
if *compare {
if len(positional) != 2 {
fmt.Fprintf(os.Stderr, "%s: --compare needs exactly two snapshot files\n", brand.Name)
return 2
}
cfg.Compare = [2]string{positional[0], positional[1]}
} else {
if len(positional) > 1 {
fmt.Fprintf(os.Stderr, "%s: only one path may be scanned at a time\n", brand.Name)
return 2
}
if len(positional) == 1 && cfg.Load == "" && snapshot.IsSnapshot(positional[0]) {
cfg.Load = positional[0]
} else {
cfg.Paths = positional
}
}
w, err := scan.Workers(*workers)
if err != nil {
fmt.Fprintf(os.Stderr, "%s: %v\n", brand.Name, err)
return 2
}
fm, err := scan.ParseFollow(*follow)
if err != nil {
fmt.Fprintf(os.Stderr, "%s: %v\n", brand.Name, err)
return 2
}
cfg.ScanOpts = scan.Options{Workers: w, Follow: fm, OneFileSystem: *oneFS || *oneFSx,
Excludes: append(append([]string(nil), settings.Excludes...), excludes...)}
if err := app.Run(cfg); err != nil {
fmt.Fprintf(os.Stderr, "%s: %v\n", brand.Name, textutil.Sanitize(err.Error()))
return 1
}
return 0
}
func setupLog(path, level string) (func(), error) {
lv := map[string]slog.Level{"error": slog.LevelError, "warn": slog.LevelWarn, "info": slog.LevelInfo,
"debug": slog.LevelDebug, "trace": slog.LevelDebug - 4}
l, ok := lv[level]
if !ok {
return nil, fmt.Errorf("invalid --log-level %q", level)
}
f, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o600)
if err != nil {
return nil, err
}
slog.SetDefault(slog.New(slog.NewTextHandler(f, &slog.HandlerOptions{Level: l})))
return func() { f.Close() }, nil
}
func usage(fs *flag.FlagSet) {
w := fs.Output()
fmt.Fprintf(w, `%[1]s — interactive terminal disk usage analyser
Usage:
%[1]s [flags] [PATH] scan PATH (default .) interactively
%[1]s [flags] SNAPSHOT%[2]s open a saved snapshot
%[1]s --compare OLD NEW compare two snapshots
%[1]s PATH --top 50 | --json | --csv | --no-ui --save FILE
Flags:
`, brand.Name, brand.SnapshotExt)
fs.PrintDefaults()
}
cmd/treegen/main.go
-130
View File
@@ -1,130 +0,0 @@
// Command treegen creates synthetic directory trees for testing and
// benchmarking mapsize.
package main
import (
"flag"
"fmt"
"math"
"math/rand"
"os"
"path/filepath"
"time"
)
func main() {
var (
out = flag.String("out", "", "output directory (required, must not exist or be empty)")
dirs = flag.Int("directories", 1000, "number of directories")
files = flag.Int("files", 10000, "number of files")
depth = flag.Int("max-depth", 8, "maximum directory depth")
sparse = flag.Bool("sparse", false, "make some large files sparse")
unicode = flag.Bool("unicode", false, "use Unicode and unusual (but safe) names")
hostile = flag.Bool("hostile", false, "include names with control characters/escape sequences (Unix only)")
maxSize = flag.Int64("max-size", 1<<20, "largest regular file in bytes (sizes are log-uniform; content is written)")
seed = flag.Int64("seed", 1, "random seed (deterministic output)")
hotspots = flag.Int("hotspots", 3, "directories that receive a large share of files")
)
flag.Parse()
if *out == "" {
flag.Usage()
os.Exit(2)
}
if err := os.MkdirAll(*out, 0o755); err != nil {
fatal(err)
}
if ents, _ := os.ReadDir(*out); len(ents) > 0 {
fatal(fmt.Errorf("%s is not empty", *out))
}
r := rand.New(rand.NewSource(*seed))
start := time.Now()
type dir struct {
path string
depth int
}
all := []dir{{*out, 0}}
byDepth := [][]int{{0}}
for len(all) < *dirs+1 {
// Pick a depth first, then a directory at that depth, so the tree is
// broad near the top instead of one giant preferential-attachment chain.
lv := byDepth[r.Intn(len(byDepth))]
p := all[lv[r.Intn(len(lv))]]
if p.depth >= *depth {
continue
}
d := dir{filepath.Join(p.path, name(r, len(all), *unicode, *hostile)), p.depth + 1}
if err := os.Mkdir(d.path, 0o755); err != nil {
continue // name collision; try again
}
all = append(all, d)
if d.depth == len(byDepth) {
byDepth = append(byDepth, nil)
}
byDepth[d.depth] = append(byDepth[d.depth], len(all)-1)
}
hot := make([]int, *hotspots)
for i := range hot {
hot[i] = r.Intn(len(all))
}
var bytes int64
buf := make([]byte, *maxSize)
for i := range buf {
buf[i] = byte(r.Intn(256)) // incompressible, so allocation is real everywhere
}
for i := 0; i < *files; i++ {
var d dir
if len(hot) > 0 && r.Intn(3) == 0 {
d = all[hot[r.Intn(len(hot))]]
} else {
d = all[r.Intn(len(all))]
}
p := filepath.Join(d.path, name(r, i, *unicode, *hostile)+ext(r))
// Log-uniform sizes: many tiny files, a few large ones.
size := int64(math.Exp(r.Float64() * math.Log(float64(*maxSize))))
if r.Intn(20) == 0 {
size = 0
}
f, err := os.OpenFile(p, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o644)
if err != nil {
continue
}
if *sparse && r.Intn(200) == 0 {
// Sparse: a large logical size with almost nothing allocated.
f.Truncate(int64(1+r.Intn(8)) << 30)
f.WriteAt([]byte("x"), 0)
} else if size > 0 {
if _, err := f.Write(buf[:size]); err != nil {
fatal(err)
}
bytes += size
}
f.Close()
if i%100000 == 0 && i > 0 {
fmt.Fprintf(os.Stderr, "%d files…\n", i)
}
}
fmt.Fprintf(os.Stderr, "created %d directories, %d files (%d logical bytes) in %s\n",
len(all)-1, *files, bytes, time.Since(start).Round(time.Millisecond))
}
var words = []string{"data", "cache", "backup", "photos", "src", "build", "logs", "tmp", "vm", "media", "docs", "lib", "node_modules", "archive"}
var exts = []string{".txt", ".log", ".jpg", ".png", ".mp4", ".iso", ".qcow2", ".go", ".c", ".pdf", ".zip", ".tar.gz", ".db", ".so", "", ".json", ".mkv", ".flac"}
var uni = []string{"日本語", "Ünïcödé", "emoji 🎉", "Ελληνικά", "кириллица", "space name", "quote'name", "dash-name"}
func name(r *rand.Rand, i int, unicode, hostile bool) string {
switch {
case hostile && r.Intn(50) == 0:
return fmt.Sprintf("evil\x1b[31m%d\x1b]0;pwned\x07", i)
case unicode && r.Intn(5) == 0:
return fmt.Sprintf("%s-%d", uni[r.Intn(len(uni))], i)
}
return fmt.Sprintf("%s-%d", words[r.Intn(len(words))], i)
}
func ext(r *rand.Rand) string { return exts[r.Intn(len(exts))] }
func fatal(err error) {
fmt.Fprintln(os.Stderr, "treegen:", err)
os.Exit(1)
}
docs/mapsize.1
+4 -12
View File
@@ -142,18 +142,10 @@ colouring, or prints the most significant changes with \fB\-\-no\-ui\fR
(\fB\-\-top\fR sets how many; default 30).
.SS Diagnostics
.TP
.BI \-\-log " FILE"
Append a log to \fIFILE\fR without disturbing the interface. Full paths are
logged only at \fBdebug\fR level and above.
.TP
.BI \-\-log\-level " LEVEL"
.BR error ", " warn ", " info " (default), " debug " or " trace .
.TP
.BI \-\-cpuprofile " FILE"
Write a Go CPU profile.
.TP
.BI \-\-memprofile " FILE"
Write a Go heap profile on exit.
.BI \-\-log " FILE" "\fR, \fP\-\-log\-level " LEVEL
Accepted for compatibility with earlier releases; currently no log is
written. \fILEVEL\fR must still be one of
.BR error ", " warn ", " info ", " debug " or " trace .
.TP
.B \-\-version
Print the version and exit.
rust/examples/bench.rs → examples/bench.rs
+27 -7
View File
@@ -1,5 +1,5 @@
//! In-process benchmarks mirroring the Go `go test -bench` suite
//! (internal/tui/bench_test.go, internal/treemap/treemap_test.go):
//! In-process benchmarks of the TUI model (render, navigation, filtering)
//! and the treemap layout:
//!
//! cargo run --release --example bench
//!
@@ -25,25 +25,45 @@ fn bench<R>(name: &str, mut f: impl FnMut(u64) -> R) {
}
fn main() {
let items: Vec<Item> = (0..300).map(|i| Item { id: i, size: (1 + i * i) as f64 }).collect();
let bounds = Rect { x: 0, y: 0, w: 200, h: 60 };
let items: Vec<Item> = (0..300)
.map(|i| Item {
id: i,
size: (1 + i * i) as f64,
})
.collect();
let bounds = Rect {
x: 0,
y: 0,
w: 200,
h: 60,
};
bench("treemap/Squarify(300)", |_| {
black_box(treemap::squarify(black_box(&items), bounds));
});
let blocks = treemap::squarify(&items, bounds);
bench("treemap/Neighbour(300)", |_| {
black_box(treemap::neighbour(black_box(&blocks), 150, Direction::Right));
black_box(treemap::neighbour(
black_box(&blocks),
150,
Direction::Right,
));
});
let t0 = Instant::now();
let tree = Arc::new(RwLock::new(big_tree()));
println!("big tree: {} nodes built in {:.2?}", tree.read().unwrap().len(), t0.elapsed());
println!(
"big tree: {} nodes built in {:.2?}",
tree.read().unwrap().len(),
t0.elapsed()
);
let mut m = BenchModel::new(tree.clone(), 200, 60);
bench("tui/BigColdFrame", |_| m.cold_frame());
let mut m = BenchModel::new(tree.clone(), 200, 60);
m.cold_frame();
bench("tui/BigResize", |i| m.resize(150 + (i % 100) as u16, 40 + (i % 30) as u16));
bench("tui/BigResize", |i| {
m.resize(150 + (i % 100) as u16, 40 + (i % 30) as u16)
});
let mut m = BenchModel::new(tree.clone(), 200, 60);
m.zoom_to_child_named(b"flat");
bench("tui/BigFlatDirCold", |_| m.cold_frame());
examples/treegen.rs
+210
View File
@@ -0,0 +1,210 @@
//! Synthetic directory-tree generator for testing and benchmarking mapsize:
//!
//! cargo run --release --example treegen -- --out DIR [flags]
//!
//! Flags (`-flag` or `--flag`, value as `=V` or a separate argument):
//! --out DIR output directory (required, must not exist or be empty)
//! --directories N number of directories (1000)
//! --files N number of files (10000)
//! --max-depth N maximum directory depth (8)
//! --sparse make some large files sparse
//! --unicode use Unicode and unusual (but safe) names
//! --hostile include names with control characters/escape sequences (Unix only)
//! --max-size N largest regular file in bytes; sizes are log-uniform, content is written (1048576)
//! --seed N random seed; output is deterministic per seed (1)
//! --hotspots N directories that receive a large share of files (3)
use std::fs::{self, OpenOptions};
use std::io::Write;
use std::path::PathBuf;
use std::process::exit;
use std::time::Instant;
const WORDS: &[&str] = &[
"data",
"cache",
"backup",
"photos",
"src",
"build",
"logs",
"tmp",
"vm",
"media",
"docs",
"lib",
"node_modules",
"archive",
];
const EXTS: &[&str] = &[
".txt", ".log", ".jpg", ".png", ".mp4", ".iso", ".qcow2", ".go", ".c", ".pdf", ".zip",
".tar.gz", ".db", ".so", "", ".json", ".mkv", ".flac",
];
const UNI: &[&str] = &[
"日本語",
"Ünïcödé",
"emoji 🎉",
"Ελληνικά",
"кириллица",
"space name",
"quote'name",
"dash-name",
];
const BOOLS: &[&str] = &["sparse", "unicode", "hostile"];
/// SplitMix64: tiny, fast and good enough for synthetic data.
struct Rng(u64);
impl Rng {
fn next(&mut self) -> u64 {
self.0 = self.0.wrapping_add(0x9e37_79b9_7f4a_7c15);
let mut z = self.0;
z = (z ^ (z >> 30)).wrapping_mul(0xbf58_476d_1ce4_e5b9);
z = (z ^ (z >> 27)).wrapping_mul(0x94d0_49bb_1331_11eb);
z ^ (z >> 31)
}
fn n(&mut self, n: usize) -> usize {
(self.next() % n as u64) as usize
}
fn f64(&mut self) -> f64 {
(self.next() >> 11) as f64 / (1u64 << 53) as f64
}
fn pick<'a>(&mut self, s: &[&'a str]) -> &'a str {
s[self.n(s.len())]
}
}
fn fatal(msg: impl std::fmt::Display) -> ! {
eprintln!("treegen: {msg}");
exit(1);
}
fn usage() -> ! {
eprintln!("usage: treegen --out DIR [--directories N] [--files N] [--max-depth N] [--sparse] [--unicode] [--hostile] [--max-size N] [--seed N] [--hotspots N]");
exit(2);
}
fn name(r: &mut Rng, i: usize, unicode: bool, hostile: bool) -> String {
if hostile && r.n(50) == 0 {
format!("evil\x1b[31m{i}\x1b]0;pwned\x07")
} else if unicode && r.n(5) == 0 {
format!("{}-{i}", r.pick(UNI))
} else {
format!("{}-{i}", r.pick(WORDS))
}
}
fn main() {
let mut out = None;
let (mut dirs, mut files, mut depth, mut max_size, mut seed, mut hotspots) =
(1000usize, 10000usize, 8usize, 1usize << 20, 1u64, 3usize);
let (mut sparse, mut unicode, mut hostile) = (false, false, false);
let mut args = std::env::args().skip(1);
while let Some(a) = args.next() {
let Some(flag) = a.strip_prefix("--").or_else(|| a.strip_prefix('-')) else {
usage()
};
let (key, inline) = match flag.split_once('=') {
Some((k, v)) => (k, Some(v.to_string())),
None => (flag, None),
};
if BOOLS.contains(&key) {
let on = match inline.as_deref() {
None | Some("true" | "1") => true,
Some("false" | "0") => false,
Some(v) => fatal(format!("invalid value {v:?} for -{key}")),
};
*match key {
"sparse" => &mut sparse,
"unicode" => &mut unicode,
_ => &mut hostile,
} = on;
continue;
}
let v = inline
.or_else(|| args.next())
.unwrap_or_else(|| fatal(format!("flag needs an argument: -{key}")));
let num = || {
v.parse::<u64>()
.unwrap_or_else(|_| fatal(format!("invalid value {v:?} for -{key}")))
};
match key {
"out" => out = Some(PathBuf::from(&v)),
"directories" => dirs = num() as usize,
"files" => files = num() as usize,
"max-depth" => depth = num() as usize,
"max-size" => max_size = num() as usize,
"seed" => seed = num(),
"hotspots" => hotspots = num() as usize,
_ => usage(),
}
}
let Some(out) = out else { usage() };
hostile &= cfg!(unix);
fs::create_dir_all(&out).unwrap_or_else(|e| fatal(e));
if fs::read_dir(&out).is_ok_and(|mut d| d.next().is_some()) {
fatal(format!("{} is not empty", out.display()));
}
let mut r = Rng(seed);
let start = Instant::now();
// Pick a depth first, then a directory at that depth, so the tree is broad
// near the top instead of one giant preferential-attachment chain.
let mut all = vec![(out.clone(), 0usize)];
let mut by_depth = vec![vec![0usize]];
while all.len() < dirs + 1 {
let lv = &by_depth[r.n(by_depth.len())];
let (parent, d) = all[lv[r.n(lv.len())]].clone();
if d >= depth {
continue;
}
let p = parent.join(name(&mut r, all.len(), unicode, hostile));
if fs::create_dir(&p).is_err() {
continue; // name collision; try again
}
all.push((p, d + 1));
if d + 1 == by_depth.len() {
by_depth.push(Vec::new());
}
by_depth[d + 1].push(all.len() - 1);
}
let hot: Vec<usize> = (0..hotspots).map(|_| r.n(all.len())).collect();
// Incompressible, so allocation is real everywhere.
let buf: Vec<u8> = (0..max_size).map(|_| r.next() as u8).collect();
let (mut bytes, mut created) = (0u64, 0usize);
for i in 0..files {
let d = if !hot.is_empty() && r.n(3) == 0 {
&all[hot[r.n(hot.len())]].0
} else {
&all[r.n(all.len())].0
};
let n = name(&mut r, i, unicode, hostile);
let p = d.join(n + r.pick(EXTS));
// Log-uniform sizes: many tiny files, a few large ones.
let mut size = (r.f64() * (max_size as f64).ln()).exp() as usize;
if r.n(20) == 0 {
size = 0;
}
let Ok(mut f) = OpenOptions::new().write(true).create_new(true).open(&p) else {
continue;
};
if sparse && r.n(200) == 0 {
// Sparse: a large logical size with almost nothing allocated.
f.write_all(b"x")
.and_then(|_| f.set_len((1 + r.n(8) as u64) << 30))
.unwrap_or_else(|e| fatal(e));
} else if size > 0 {
f.write_all(&buf[..size]).unwrap_or_else(|e| fatal(e));
bytes += size as u64;
}
created += 1;
if i % 100_000 == 0 && i > 0 {
eprintln!("{i} files…");
}
}
eprintln!(
"created {} directories, {created} files ({bytes} logical bytes) in {:.2?}",
all.len() - 1,
start.elapsed()
);
}
go.mod
-25
View File
@@ -1,25 +0,0 @@
module github.com/andydixon/mapsize
go 1.26.0
require (
github.com/rivo/uniseg v0.4.7
golang.org/x/sys v0.48.0
)
require (
charm.land/bubbletea/v2 v2.0.10 // indirect
github.com/charmbracelet/colorprofile v0.4.3 // indirect
github.com/charmbracelet/ultraviolet v0.0.0-20260703014108-f5a850f9c2b7 // indirect
github.com/charmbracelet/x/ansi v0.11.8 // indirect
github.com/charmbracelet/x/term v0.2.2 // indirect
github.com/charmbracelet/x/termios v0.1.1 // indirect
github.com/charmbracelet/x/windows v0.2.2 // indirect
github.com/clipperhouse/displaywidth v0.11.0 // indirect
github.com/clipperhouse/uax29/v2 v2.7.0 // indirect
github.com/lucasb-eyer/go-colorful v1.4.0 // indirect
github.com/mattn/go-runewidth v0.0.24 // indirect
github.com/muesli/cancelreader v0.2.2 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
golang.org/x/sync v0.21.0 // indirect
)
go.sum
-32
View File
@@ -1,32 +0,0 @@
charm.land/bubbletea/v2 v2.0.10 h1:oolvo20VBpI0PfqE7iFjkZ1bx0WpmXGfnKz5Yldjq5o=
charm.land/bubbletea/v2 v2.0.10/go.mod h1:QOatcnhOjYIfxzUSTz6raF7Ex4R/rIuHa3SnBdCCpMc=
github.com/charmbracelet/colorprofile v0.4.3 h1:QPa1IWkYI+AOB+fE+mg/5/4HRMZcaXex9t5KX76i20Q=
github.com/charmbracelet/colorprofile v0.4.3/go.mod h1:/zT4BhpD5aGFpqQQqw7a+VtHCzu+zrQtt1zhMt9mR4Q=
github.com/charmbracelet/ultraviolet v0.0.0-20260703014108-f5a850f9c2b7 h1:3FmWoGNWK4STvqg0O0Aeav2T7rodWJAPeF0QpH+8gFw=
github.com/charmbracelet/ultraviolet v0.0.0-20260703014108-f5a850f9c2b7/go.mod h1:f/jRa757WUmaOZrbPspXymbg/GnbF+rwe4OLsG7aXYo=
github.com/charmbracelet/x/ansi v0.11.8 h1:JMFwp0CgDC2+jcOB162HH5k7I3FVbgFSMMYg7dSPBQQ=
github.com/charmbracelet/x/ansi v0.11.8/go.mod h1:ZNN+3mXny/516oTQPLMPIBeSINvNJJQ8uQXDgbeJxY0=
github.com/charmbracelet/x/term v0.2.2 h1:xVRT/S2ZcKdhhOuSP4t5cLi5o+JxklsoEObBSgfgZRk=
github.com/charmbracelet/x/term v0.2.2/go.mod h1:kF8CY5RddLWrsgVwpw4kAa6TESp6EB5y3uxGLeCqzAI=
github.com/charmbracelet/x/termios v0.1.1 h1:o3Q2bT8eqzGnGPOYheoYS8eEleT5ZVNYNy8JawjaNZY=
github.com/charmbracelet/x/termios v0.1.1/go.mod h1:rB7fnv1TgOPOyyKRJ9o+AsTU/vK5WHJ2ivHeut/Pcwo=
github.com/charmbracelet/x/windows v0.2.2 h1:IofanmuvaxnKHuV04sC0eBy/smG6kIKrWG2/jYn2GuM=
github.com/charmbracelet/x/windows v0.2.2/go.mod h1:/8XtdKZzedat74NQFn0NGlGL4soHB0YQZrETF96h75k=
github.com/clipperhouse/displaywidth v0.11.0 h1:lBc6kY44VFw+TDx4I8opi/EtL9m20WSEFgwIwO+UVM8=
github.com/clipperhouse/displaywidth v0.11.0/go.mod h1:bkrFNkf81G8HyVqmKGxsPufD3JhNl3dSqnGhOoSD/o0=
github.com/clipperhouse/uax29/v2 v2.7.0 h1:+gs4oBZ2gPfVrKPthwbMzWZDaAFPGYK72F0NJv2v7Vk=
github.com/clipperhouse/uax29/v2 v2.7.0/go.mod h1:EFJ2TJMRUaplDxHKj1qAEhCtQPW2tJSwu5BF98AuoVM=
github.com/lucasb-eyer/go-colorful v1.4.0 h1:UtrWVfLdarDgc44HcS7pYloGHJUjHV/4FwW4TvVgFr4=
github.com/lucasb-eyer/go-colorful v1.4.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
github.com/mattn/go-runewidth v0.0.24 h1:cpokDiIn0MGnhdHwuWnJBITySJ20QyNGnY2kR/ay2DU=
github.com/mattn/go-runewidth v0.0.24/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
github.com/muesli/cancelreader v0.2.2 h1:3I4Kt4BQjOR54NavqnDogx/MIoWBFa0StPA8ELUXHmA=
github.com/muesli/cancelreader v0.2.2/go.mod h1:3XuTXfFS2VjM+HTLZY9Ak0l6eUKfijIfMUZ4EgX0QYo=
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
internal/app/app.go
-171
View File
@@ -1,171 +0,0 @@
// Package app wires the command-line modes together: interactive TUI,
// non-interactive reports, snapshots and comparisons. TUI and CLI share the
// same scanner and inventory packages.
package app
import (
"context"
"fmt"
"io"
"log/slog"
"os"
"os/signal"
"time"
"github.com/andydixon/mapsize/internal/config"
"github.com/andydixon/mapsize/internal/export"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/scan"
"github.com/andydixon/mapsize/internal/textutil"
)
// Config is the fully-resolved command line.
type Config struct {
Paths []string
ScanOpts scan.Options
NoUI bool
JSON, CSV bool
Depth int
Top int
LargestFiles int
Save string
Load string
Compare [2]string
Duplicates bool
ReadOnly bool
Theme string
SizeMode inventory.SizeMode
Mouse bool
Color string
Settings config.Settings
Stdout io.Writer
Stderr io.Writer
}
// Report reports whether any non-interactive output was requested.
func (c *Config) Report() bool {
return c.NoUI || c.JSON || c.CSV || c.Top > 0 || c.LargestFiles > 0 || c.Duplicates
}
// Run executes the configured mode.
func Run(cfg *Config) error {
if cfg.Stdout == nil {
cfg.Stdout = os.Stdout
}
if cfg.Stderr == nil {
cfg.Stderr = os.Stderr
}
if cfg.Compare[0] != "" {
return runCompare(cfg)
}
if !cfg.Report() {
return runTUI(cfg)
}
return runReport(cfg)
}
// startSource begins a scan (or loads a snapshot). The scanner is nil when
// the tree came from a snapshot.
func startSource(ctx context.Context, cfg *Config) (*scan.Scanner, *inventory.Tree, error) {
if cfg.Load != "" {
t, err := loadSnapshot(cfg.Load)
return nil, t, err
}
root := "."
if len(cfg.Paths) > 0 {
root = cfg.Paths[0]
}
slog.Info("scan start", "workers", cfg.ScanOpts.Workers, "follow", cfg.ScanOpts.Follow.String())
s, err := scan.Start(ctx, root, cfg.ScanOpts)
if err != nil {
return nil, nil, err
}
return s, s.Tree, nil
}
func runReport(cfg *Config) error {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt)
defer stop()
s, t, err := startSource(ctx, cfg)
if err != nil {
return err
}
if s != nil {
waitWithProgress(s, cfg.Stderr)
}
if cfg.Duplicates && !cfg.JSON && !cfg.CSV {
// Find takes its own read lock; never nest RLock.
if err := runDuplicates(ctx, cfg, t); err != nil {
return err
}
}
t.RLock()
defer t.RUnlock()
if s != nil && cfg.Save != "" {
if err := saveSnapshot(t, cfg.Save); err != nil {
return err
}
fmt.Fprintf(cfg.Stderr, "Saved snapshot %s\n", cfg.Save)
}
out := cfg.Stdout
switch {
case cfg.JSON:
return export.JSON(out, t, t.Root(), cfg.Depth)
case cfg.CSV:
return export.CSV(out, t, t.Root(), cfg.Depth)
}
if cfg.Top > 0 {
ids := t.SortedChildren(t.Root(), cfg.SizeMode)
if len(ids) > cfg.Top {
ids = ids[:cfg.Top]
}
export.Table(out, t, ids, cfg.SizeMode)
fmt.Fprintln(out)
}
if cfg.LargestFiles > 0 {
ids := t.Top(t.Root(), cfg.LargestFiles,
func(n *inventory.Node) bool {
return n.Kind == inventory.KindFile && n.Flags&inventory.FlagHardlinkDup == 0
},
func(n *inventory.Node) int64 { return n.Own(cfg.SizeMode) })
export.Table(out, t, ids, cfg.SizeMode)
fmt.Fprintln(out)
}
export.Summary(out, t)
return nil
}
// waitWithProgress blocks until the scan finishes, printing a progress line
// to w if it is a terminal.
func waitWithProgress(s *scan.Scanner, w io.Writer) {
tty := isTerminal(w)
tick := time.NewTicker(250 * time.Millisecond)
defer tick.Stop()
start := time.Now()
for {
select {
case <-s.Done():
if tty {
fmt.Fprint(w, "\r\033[K")
}
return
case <-tick.C:
if tty {
p := s.Progress()
el := time.Since(start)
fmt.Fprintf(w, "\r\033[Kscanning… %s entries %s/s queue %d %s",
textutil.Count(p.Entries), textutil.Count(int64(float64(p.Entries)/el.Seconds())),
p.Pending, textutil.Duration(el))
}
}
}
}
func isTerminal(w io.Writer) bool {
f, ok := w.(*os.File)
if !ok {
return false
}
fi, err := f.Stat()
return err == nil && fi.Mode()&os.ModeCharDevice != 0
}
internal/app/app_test.go
-105
View File
@@ -1,105 +0,0 @@
package app
import (
"bytes"
"encoding/csv"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"github.com/andydixon/mapsize/internal/scan"
)
func fixture(t *testing.T) string {
root := t.TempDir()
os.MkdirAll(filepath.Join(root, "big"), 0o755)
os.MkdirAll(filepath.Join(root, "small"), 0o755)
os.WriteFile(filepath.Join(root, "big", "a.iso"), make([]byte, 300_000), 0o644)
os.WriteFile(filepath.Join(root, "big", "b.iso"), make([]byte, 300_000), 0o644)
os.WriteFile(filepath.Join(root, "small", "c.txt"), []byte("hello"), 0o644)
return root
}
func run(t *testing.T, cfg *Config) string {
t.Helper()
var out bytes.Buffer
cfg.Stdout, cfg.Stderr = &out, &bytes.Buffer{}
cfg.ScanOpts = scan.Options{Workers: 2}
if err := Run(cfg); err != nil {
t.Fatalf("Run: %v", err)
}
return out.String()
}
func TestTopAndLargest(t *testing.T) {
root := fixture(t)
out := run(t, &Config{Paths: []string{root}, Top: 5, LargestFiles: 2})
if !strings.Contains(out, filepath.Join(root, "big")) || !strings.Contains(out, "a.iso") {
t.Fatalf("report missing entries:\n%s", out)
}
if strings.Index(out, filepath.Join(root, "big")) > strings.Index(out, filepath.Join(root, "small")) {
t.Fatalf("not sorted by size:\n%s", out)
}
}
func TestJSONIsValid(t *testing.T) {
root := fixture(t)
out := run(t, &Config{Paths: []string{root}, JSON: true, Depth: -1})
var v struct {
Root string
Complete bool
Tree struct {
Size int64
Children []struct{ Name string }
}
}
if err := json.Unmarshal([]byte(out), &v); err != nil {
t.Fatalf("invalid JSON: %v\n%s", err, out)
}
if !v.Complete || v.Tree.Size < 600_005 || len(v.Tree.Children) != 2 {
t.Fatalf("unexpected JSON: %+v", v)
}
}
func TestCSV(t *testing.T) {
root := fixture(t)
out := run(t, &Config{Paths: []string{root}, CSV: true, Depth: -1})
rows, err := csv.NewReader(strings.NewReader(out)).ReadAll()
if err != nil {
t.Fatal(err)
}
if len(rows) != 1+1+2+3 { // header, root, 2 dirs, 3 files
t.Fatalf("%d rows:\n%s", len(rows), out)
}
}
func TestSnapshotSaveLoadCompare(t *testing.T) {
root := fixture(t)
dir := t.TempDir()
a, b := filepath.Join(dir, "a.msz"), filepath.Join(dir, "b.msz")
run(t, &Config{Paths: []string{root}, NoUI: true, Save: a})
os.WriteFile(filepath.Join(root, "small", "grown.bin"), make([]byte, 500_000), 0o644)
os.Remove(filepath.Join(root, "big", "b.iso"))
run(t, &Config{Paths: []string{root}, NoUI: true, Save: b})
out := run(t, &Config{Load: a, Top: 3})
if !strings.Contains(out, "big") {
t.Fatalf("loaded snapshot report:\n%s", out)
}
out = run(t, &Config{Compare: [2]string{a, b}, NoUI: true})
for _, want := range []string{"new", "grown.bin", "removed", "b.iso"} {
if !strings.Contains(out, want) {
t.Fatalf("compare output missing %q:\n%s", want, out)
}
}
}
func TestDuplicatesReport(t *testing.T) {
root := fixture(t)
out := run(t, &Config{Paths: []string{root}, Duplicates: true})
if !strings.Contains(out, "1 verified duplicate groups") {
t.Fatalf("duplicates:\n%s", out)
}
}
internal/app/modes.go
-80
View File
@@ -1,80 +0,0 @@
package app
import (
"context"
"fmt"
"github.com/andydixon/mapsize/internal/duplicate"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/scan"
"github.com/andydixon/mapsize/internal/snapshot"
"github.com/andydixon/mapsize/internal/textutil"
"github.com/andydixon/mapsize/internal/tui"
)
func runTUI(cfg *Config) error {
return tui.Run(tui.Options{
ReadOnly: cfg.ReadOnly, Theme: cfg.Theme, SizeMode: cfg.SizeMode, Mouse: cfg.Mouse, Color: cfg.Color,
Save: cfg.Save, Settings: cfg.Settings,
Start: func(ctx context.Context) (*scan.Scanner, *inventory.Tree, error) { return startSource(ctx, cfg) },
})
}
func loadSnapshot(path string) (*inventory.Tree, error) { return snapshot.LoadFile(path) }
func saveSnapshot(t *inventory.Tree, path string) error { return snapshot.SaveFile(path, t) }
func runCompare(cfg *Config) error {
old, err := snapshot.LoadFile(cfg.Compare[0])
if err != nil {
return fmt.Errorf("%s: %w", cfg.Compare[0], err)
}
nw, err := snapshot.LoadFile(cfg.Compare[1])
if err != nil {
return fmt.Errorf("%s: %w", cfg.Compare[1], err)
}
d := snapshot.Compare(old, nw)
if !cfg.Report() {
return tui.Run(tui.Options{
ReadOnly: true, Theme: cfg.Theme, SizeMode: cfg.SizeMode, Mouse: cfg.Mouse, Color: cfg.Color,
Settings: cfg.Settings, Diff: d,
Start: func(context.Context) (*scan.Scanner, *inventory.Tree, error) { return nil, nw, nil },
})
}
k := cfg.Top
if k <= 0 {
k = 30
}
out := cfg.Stdout
ro, rn := old.Node(0), nw.Node(0)
fmt.Fprintf(out, "%s → %s\n", textutil.Sanitize(old.Stats.FromSnapshot), textutil.Sanitize(nw.Stats.FromSnapshot))
fmt.Fprintf(out, "Total %s → %s (%s)\n\n", textutil.Size(ro.Total(cfg.SizeMode)), textutil.Size(rn.Total(cfg.SizeMode)),
textutil.SignedSize(rn.Total(cfg.SizeMode)-ro.Total(cfg.SizeMode)))
for _, c := range d.Changes(k, cfg.SizeMode) {
fmt.Fprintf(out, "%-10s %14s %s\n", c.Status, textutil.SignedSize(c.Delta), textutil.Sanitize(c.Path))
}
return nil
}
func runDuplicates(ctx context.Context, cfg *Config, t *inventory.Tree) error {
var f duplicate.Finder
groups, err := f.Find(ctx, t, duplicate.Options{MinSize: 1})
if err != nil {
return err
}
out := cfg.Stdout
var wasted int64
t.RLock()
defer t.RUnlock()
for _, g := range groups {
wasted += g.Wasted()
fmt.Fprintf(out, "%s × %d (sha256 %x…)\n", textutil.Size(g.Size), len(g.Files), g.Hash[:6])
for _, id := range g.Files {
fmt.Fprintf(out, " %s\n", textutil.Sanitize(t.Path(id)))
}
}
p := f.Progress()
fmt.Fprintf(out, "\n%d verified duplicate groups, %s reclaimable (%d candidates by size, %d unreadable/changed)\n\n",
len(groups), textutil.Size(wasted), p.Candidates, p.Skipped)
return nil
}
internal/brand/brand.go
-11
View File
@@ -1,11 +0,0 @@
// Package brand isolates the product name so it can be changed in one place.
package brand
const (
Name = "mapsize"
SnapshotExt = ".msz"
SnapshotMagic = "MAPSIZE\x00"
)
// Version is stamped at release time via -ldflags -X.
var Version = "1.2.0"
internal/config/config.go
-72
View File
@@ -1,72 +0,0 @@
// Package config loads user settings from a JSON file in the OS-specific
// user configuration directory. It knows nothing about rendering.
package config
import (
"encoding/json"
"errors"
"io/fs"
"os"
"path/filepath"
"github.com/andydixon/mapsize/internal/brand"
)
// Settings are user preferences. Zero values are never used directly; Load
// starts from Defaults and overlays the file.
type Settings struct {
Workers string `json:"workers"` // conservative|balanced|aggressive|N
FollowSymlinks string `json:"follow_symlinks"` // none|same-filesystem|all
OneFileSystem bool `json:"one_file_system"`
Units string `json:"units"` // iec|si
SizeMode string `json:"size_mode"` // allocated|logical
Theme string `json:"theme"`
Mouse bool `json:"mouse"`
Excludes []string `json:"excludes"`
// ExtensionColors maps an extension (without dot) to "#rrggbb".
ExtensionColors map[string]string `json:"extension_colors"`
// CategoryColors maps a category name to "#rrggbb".
CategoryColors map[string]string `json:"category_colors"`
// RefreshHz is the UI refresh rate while scanning.
RefreshHz int `json:"refresh_hz"`
}
// Defaults returns the built-in settings.
func Defaults() Settings {
return Settings{
Workers: "balanced", FollowSymlinks: "none", Units: "iec", SizeMode: "allocated",
Theme: "default", Mouse: true, RefreshHz: 10,
}
}
// Path returns the config file location.
func Path() (string, error) {
dir, err := os.UserConfigDir()
if err != nil {
return "", err
}
return filepath.Join(dir, brand.Name, "config.json"), nil
}
// Load reads the config file if present. A missing file is not an error.
func Load() (Settings, error) {
s := Defaults()
p, err := Path()
if err != nil {
return s, nil
}
b, err := os.ReadFile(p)
if errors.Is(err, fs.ErrNotExist) {
return s, nil
}
if err != nil {
return s, err
}
if err := json.Unmarshal(b, &s); err != nil {
return Defaults(), err
}
if s.RefreshHz < 1 || s.RefreshHz > 60 {
s.RefreshHz = 10
}
return s, nil
}
internal/duplicate/duplicate.go
-286
View File
@@ -1,286 +0,0 @@
// Package duplicate finds files with identical content in stages:
//
// 1. group by exact size (inventory only, no I/O)
// 2. hash three 16 KiB samples (start, middle, end) of each candidate
// 3. full SHA-256 of files whose samples still collide
//
// Only stage 3 establishes identity. Files are never declared duplicates by
// name or size alone. Reading files may update their access times.
package duplicate
import (
"context"
"crypto/sha256"
"io"
"os"
"sort"
"sync"
"sync/atomic"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/platform"
)
// Group is a set of files verified to have identical content.
type Group struct {
Size int64
Hash [32]byte
Files []inventory.NodeID
}
// Wasted returns the bytes reclaimable by keeping one copy.
func (g *Group) Wasted() int64 { return g.Size * int64(len(g.Files)-1) }
// Progress reports activity.
type Progress struct {
Stage string
Candidates int64 // files sharing a size with another file
BytesToHash int64
BytesHashed int64
Skipped int64 // unreadable or changed during hashing
VerifiedGroups int64
CandidateGroups int64
}
// Options tunes the search.
type Options struct {
MinSize int64 // ignore files smaller than this (default 1 byte)
Workers int // concurrent readers (default 4)
}
type cand struct {
id inventory.NodeID
path string
size int64
}
const sampleSize = 16 << 10
// Finder runs a search and exposes live progress.
type Finder struct {
hashed, toHash, skipped atomic.Int64
mu sync.Mutex
p Progress
}
// Progress returns a copy of the current progress.
func (f *Finder) Progress() Progress {
f.mu.Lock()
p := f.p
f.mu.Unlock()
p.BytesHashed, p.BytesToHash, p.Skipped = f.hashed.Load(), f.toHash.Load(), f.skipped.Load()
return p
}
func (f *Finder) stage(s string, fn func(*Progress)) {
f.mu.Lock()
f.p.Stage = s
if fn != nil {
fn(&f.p)
}
f.mu.Unlock()
}
// Find searches t (read-locked internally only while collecting candidates).
func (f *Finder) Find(ctx context.Context, t *inventory.Tree, o Options) ([]Group, error) {
if o.MinSize < 1 {
o.MinSize = 1
}
if o.Workers < 1 {
o.Workers = 4
}
f.stage("grouping by size", nil)
bySize := map[int64][]cand{}
t.RLock()
t.Walk(t.Root(), func(id inventory.NodeID, n *inventory.Node) bool {
if n.Kind == inventory.KindFile && n.Flags&inventory.FlagHardlinkDup == 0 && n.Size >= o.MinSize {
bySize[n.Size] = append(bySize[n.Size], cand{id: id, size: n.Size})
}
return true
})
var groups [][]cand
var ncand int64
for _, g := range bySize {
if len(g) > 1 {
for i := range g {
g[i].path = t.Path(g[i].id)
}
groups = append(groups, g)
ncand += int64(len(g))
}
}
t.RUnlock()
f.stage("sampling", func(p *Progress) { p.Candidates = ncand; p.CandidateGroups = int64(len(groups)) })
// Stage 2: sample hashes.
var sampled [][]cand
for _, g := range groups {
if ctx.Err() != nil {
return nil, ctx.Err()
}
if g[0].size <= 3*sampleSize {
sampled = append(sampled, g) // a sample would be the whole file
continue
}
sampled = append(sampled, f.split(ctx, g, o.Workers, sampleHash)...)
}
// Stage 3: full hashes.
var total int64
for _, g := range sampled {
total += g[0].size * int64(len(g))
}
f.toHash.Store(total)
f.stage("hashing", nil)
var out []Group
for _, g := range sampled {
if ctx.Err() != nil {
return nil, ctx.Err()
}
for _, sub := range f.splitHash(ctx, g, o.Workers) {
grp := Group{Size: sub.size, Hash: sub.hash}
for _, c := range sub.files {
grp.Files = append(grp.Files, c.id)
}
out = append(out, grp)
}
}
sort.Slice(out, func(i, j int) bool {
if a, b := out[i].Wasted(), out[j].Wasted(); a != b {
return a > b
}
return out[i].Files[0] < out[j].Files[0]
})
f.stage("done", func(p *Progress) { p.VerifiedGroups = int64(len(out)) })
return out, ctx.Err()
}
type hashed struct {
c cand
sum [32]byte
ok bool
}
// hashAll hashes every candidate with a bounded worker pool.
func (f *Finder) hashAll(ctx context.Context, g []cand, workers int, fn func(*Finder, cand) ([32]byte, bool)) []hashed {
res := make([]hashed, len(g))
var wg sync.WaitGroup
sem := make(chan struct{}, workers)
for i, c := range g {
if ctx.Err() != nil {
break
}
sem <- struct{}{}
wg.Add(1)
go func() {
defer wg.Done()
defer func() { <-sem }()
sum, ok := fn(f, c)
if !ok {
f.skipped.Add(1)
}
res[i] = hashed{c, sum, ok}
}()
}
wg.Wait()
return res
}
func (f *Finder) split(ctx context.Context, g []cand, workers int, fn func(*Finder, cand) ([32]byte, bool)) [][]cand {
by := map[[32]byte][]cand{}
for _, h := range f.hashAll(ctx, g, workers, fn) {
if h.ok {
by[h.sum] = append(by[h.sum], h.c)
}
}
var out [][]cand
for _, v := range by {
if len(v) > 1 {
out = append(out, v)
}
}
return out
}
type fullGroup struct {
size int64
hash [32]byte
files []cand
}
func (f *Finder) splitHash(ctx context.Context, g []cand, workers int) []fullGroup {
by := map[[32]byte][]cand{}
for _, h := range f.hashAll(ctx, g, workers, fullHash) {
if h.ok {
by[h.sum] = append(by[h.sum], h.c)
}
}
var out []fullGroup
for sum, v := range by {
if len(v) > 1 {
sort.Slice(v, func(i, j int) bool { return v[i].id < v[j].id })
out = append(out, fullGroup{size: v[0].size, hash: sum, files: v})
}
}
return out
}
// open opens a candidate and checks it still has the inventoried size.
func open(c cand) (*os.File, bool) {
fh, err := platform.OpenContent(c.path)
if err != nil {
return nil, false
}
fi, err := fh.Stat()
if err != nil || !fi.Mode().IsRegular() || fi.Size() != c.size {
fh.Close()
return nil, false
}
return fh, true
}
func sampleHash(_ *Finder, c cand) ([32]byte, bool) {
fh, ok := open(c)
if !ok {
return [32]byte{}, false
}
defer fh.Close()
h := sha256.New()
buf := make([]byte, sampleSize)
for _, off := range []int64{0, c.size/2 - sampleSize/2, c.size - sampleSize} {
if _, err := fh.ReadAt(buf, off); err != nil && err != io.EOF {
return [32]byte{}, false
}
h.Write(buf)
}
var s [32]byte
h.Sum(s[:0])
return s, true
}
func fullHash(f *Finder, c cand) ([32]byte, bool) {
fh, ok := open(c)
if !ok {
return [32]byte{}, false
}
defer fh.Close()
h := sha256.New()
n, err := io.Copy(h, &countingReader{fh, &f.hashed})
if err != nil || n != c.size {
return [32]byte{}, false
}
var s [32]byte
h.Sum(s[:0])
return s, true
}
type countingReader struct {
r io.Reader
n *atomic.Int64
}
func (c *countingReader) Read(p []byte) (int, error) {
n, err := c.r.Read(p)
c.n.Add(int64(n))
return n, err
}
internal/duplicate/duplicate_test.go
-61
View File
@@ -1,61 +0,0 @@
package duplicate
import (
"context"
"os"
"path/filepath"
"testing"
"github.com/andydixon/mapsize/internal/scan"
)
func TestFind(t *testing.T) {
root := t.TempDir()
big := make([]byte, 200_000)
for i := range big {
big[i] = byte(i * 7)
}
w := func(name string, b []byte) {
if err := os.WriteFile(filepath.Join(root, name), b, 0o644); err != nil {
t.Fatal(err)
}
}
w("a.bin", big)
w("b.bin", big)
// Same size, same samples, different middle-of-nowhere byte: must not match.
c := append([]byte(nil), big...)
c[20_000] ^= 1
w("c.bin", c)
w("small1", []byte("hello"))
w("small2", []byte("hello"))
w("small3", []byte("world")) // same size, different content
s, err := scan.Start(context.Background(), root, scan.Options{Workers: 2})
if err != nil {
t.Fatal(err)
}
<-s.Done()
var f Finder
groups, err := f.Find(context.Background(), s.Tree, Options{})
if err != nil {
t.Fatal(err)
}
if len(groups) != 2 {
t.Fatalf("groups=%d want 2", len(groups))
}
names := func(g Group) (out []string) {
for _, id := range g.Files {
out = append(out, s.Tree.Node(id).Name)
}
return
}
if g := names(groups[0]); len(g) != 2 || groups[0].Size != 200_000 {
t.Fatalf("big group %v", g)
}
if g := names(groups[1]); len(g) != 2 {
t.Fatalf("small group %v", g)
}
if p := f.Progress(); p.Candidates != 6 || p.VerifiedGroups != 2 {
t.Fatalf("progress %+v", p)
}
}
internal/duplicate/fifo_unix_test.go
-51
View File
@@ -1,51 +0,0 @@
//go:build unix
package duplicate
import (
"context"
"os"
"path/filepath"
"syscall"
"testing"
"time"
"github.com/andydixon/mapsize/internal/scan"
)
// A candidate replaced by a FIFO (or a symlink to one) after the scan must
// be skipped, not block the search forever in open(2).
func TestFindSkipsSwappedFIFO(t *testing.T) {
root := t.TempDir()
for _, n := range []string{"a", "b", "c"} {
os.WriteFile(filepath.Join(root, n), []byte("same"), 0o644)
}
s, err := scan.Start(context.Background(), root, scan.Options{Workers: 1})
if err != nil {
t.Fatal(err)
}
<-s.Done()
os.Remove(filepath.Join(root, "b"))
if err := syscall.Mkfifo(filepath.Join(root, "b"), 0o644); err != nil {
t.Skip("mkfifo:", err)
}
fifo2 := filepath.Join(t.TempDir(), "p")
syscall.Mkfifo(fifo2, 0o644)
os.Remove(filepath.Join(root, "c"))
os.Symlink(fifo2, filepath.Join(root, "c"))
done := make(chan error, 1)
var f Finder
go func() { _, err := f.Find(context.Background(), s.Tree, Options{}); done <- err }()
select {
case err := <-done:
if err != nil {
t.Fatal(err)
}
case <-time.After(5 * time.Second):
t.Fatal("duplicate search blocked on a FIFO")
}
if p := f.Progress(); p.Skipped != 2 {
t.Fatalf("skipped=%d want 2", p.Skipped)
}
}
internal/export/export.go
-221
View File
@@ -1,221 +0,0 @@
// Package export writes inventories as JSON, CSV or plain-text reports.
// Callers must hold the tree's read lock (or own the tree exclusively).
package export
import (
"bufio"
"encoding/csv"
"encoding/json"
"fmt"
"io"
"path/filepath"
"strconv"
"strings"
"time"
"github.com/andydixon/mapsize/internal/brand"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/textutil"
)
type jsonNode struct {
Name string `json:"name"`
Type string `json:"type"`
Size int64 `json:"size"`
Allocated int64 `json:"allocated"`
Files uint32 `json:"files,omitempty"`
Dirs uint32 `json:"dirs,omitempty"`
Errors uint32 `json:"errors,omitempty"`
Modified string `json:"modified,omitempty"`
Flags []string `json:"flags,omitempty"`
Children []*jsonNode `json:"children,omitempty"`
}
var flagNames = []struct {
f inventory.Flags
n string
}{
{inventory.FlagHardlinkDup, "hardlink-duplicate"}, {inventory.FlagMountPoint, "mount-point"},
{inventory.FlagVirtualFS, "virtual-fs-skipped"}, {inventory.FlagSkippedFS, "other-fs-skipped"},
{inventory.FlagSparse, "sparse"}, {inventory.FlagError, "error"}, {inventory.FlagIncomplete, "incomplete"},
{inventory.FlagLoop, "loop-skipped"}, {inventory.FlagFollowed, "followed-symlink"},
{inventory.FlagAllocUnknown, "allocated-unknown"}, {inventory.FlagBrokenLink, "broken-symlink"},
{inventory.FlagHardlinked, "hardlinked"},
}
// FlagList returns human-readable names for a node's flags.
func FlagList(f inventory.Flags) []string {
var out []string
for _, fn := range flagNames {
if f&fn.f != 0 {
out = append(out, fn.n)
}
}
return out
}
// JSON writes the tree beneath root as nested JSON, down to maxDepth levels
// (maxDepth < 0 means unlimited).
func JSON(w io.Writer, t *inventory.Tree, root inventory.NodeID, maxDepth int) error {
bw := bufio.NewWriter(w)
st := &t.Stats
head := map[string]any{
"generator": brand.Name + " " + brand.Version,
"root": t.Path(root),
"complete": !st.Incomplete(),
"cancelled": st.Cancelled,
"started": st.Start.Format(time.RFC3339),
"elapsed_s": st.Elapsed().Seconds(),
"errors": errorCounts(st),
"excluded": st.Excluded,
"excludes": st.Excludes,
"hardlink_duplicates": st.HardlinkDups,
}
hb, err := json.Marshal(head)
if err != nil {
return err
}
bw.Write(hb[:len(hb)-1])
bw.WriteString(`,"tree":`)
if err := writeNode(bw, t, root, maxDepth); err != nil {
return err
}
bw.WriteString("}\n")
return bw.Flush()
}
func errorCounts(st *inventory.Stats) map[string]int64 {
m := map[string]int64{}
for k, c := range st.ErrCounts {
if c > 0 {
m[inventory.ErrKind(k).String()] = c
}
}
return m
}
// writeNode streams one node; children are written recursively so memory
// stays proportional to depth, not tree size.
func writeNode(bw *bufio.Writer, t *inventory.Tree, id inventory.NodeID, depth int) error {
n := t.Node(id)
jn := jsonNode{Name: n.Name, Type: kindName(n.Kind), Size: n.TotSize, Allocated: n.TotAlloc,
Files: n.Files, Dirs: n.Dirs, Errors: n.Errors, Flags: FlagList(n.Flags)}
if n.MTime != 0 {
jn.Modified = time.Unix(0, n.MTime).UTC().Format(time.RFC3339)
}
b, err := json.Marshal(jn)
if err != nil {
return err
}
if n.FirstChild == inventory.NoNode || depth == 0 {
_, err = bw.Write(b)
return err
}
bw.Write(b[:len(b)-1])
bw.WriteString(`,"children":[`)
first := true
for _, c := range t.SortedChildren(id, inventory.SizeAllocated) {
if !first {
bw.WriteByte(',')
}
first = false
if err := writeNode(bw, t, c, depth-1); err != nil {
return err
}
}
_, err = bw.WriteString("]}")
return err
}
func kindName(k inventory.Kind) string {
return [...]string{"file", "dir", "symlink", "other"}[k&3]
}
// CSV writes one row per node beneath root (depth-limited). Paths are
// sanitised so the output is safe to print to a terminal.
func CSV(w io.Writer, t *inventory.Tree, root inventory.NodeID, maxDepth int) error {
cw := csv.NewWriter(w)
cw.Write([]string{"path", "type", "size", "allocated", "files", "dirs", "errors", "modified", "flags"})
var rec func(id inventory.NodeID, path string, depth int) error
rec = func(id inventory.NodeID, path string, depth int) error {
n := t.Node(id)
mod := ""
if n.MTime != 0 {
mod = time.Unix(0, n.MTime).UTC().Format(time.RFC3339)
}
flags := ""
for i, f := range FlagList(n.Flags) {
if i > 0 {
flags += "|"
}
flags += f
}
if err := cw.Write([]string{textutil.Sanitize(path), kindName(n.Kind),
strconv.FormatInt(n.TotSize, 10), strconv.FormatInt(n.TotAlloc, 10),
strconv.FormatUint(uint64(n.Files), 10), strconv.FormatUint(uint64(n.Dirs), 10),
strconv.FormatUint(uint64(n.Errors), 10), mod, flags}); err != nil {
return err
}
if depth == 0 {
return nil
}
for _, c := range t.SortedChildren(id, inventory.SizeAllocated) {
if err := rec(c, filepath.Join(path, t.Node(c).Name), depth-1); err != nil {
return err
}
}
return nil
}
if err := rec(root, t.Path(root), maxDepth); err != nil {
return err
}
cw.Flush()
return cw.Error()
}
// Table writes "size % files path" rows for the given nodes.
func Table(w io.Writer, t *inventory.Tree, ids []inventory.NodeID, m inventory.SizeMode) {
total := t.Node(t.Root()).Total(m)
fmt.Fprintf(w, "%12s %6s %12s %s\n", "SIZE", "%", "FILES", "PATH")
for _, id := range ids {
n := t.Node(id)
files := ""
if n.IsDir() {
files = textutil.Count(int64(n.Files))
}
fmt.Fprintf(w, "%12s %6s %12s %s\n", textutil.Size(n.Total(m)), textutil.Percent(n.Total(m), total),
files, textutil.Sanitize(t.Path(id)))
}
}
// Summary writes the scan summary, making incompleteness explicit.
func Summary(w io.Writer, t *inventory.Tree) {
st := &t.Stats
r := t.Node(t.Root())
fmt.Fprintf(w, "Root %s\n", textutil.Sanitize(st.Root))
fmt.Fprintf(w, "Files %s\n", textutil.Count(st.Files+st.Symlinks+st.Others))
fmt.Fprintf(w, "Directories %s\n", textutil.Count(st.Dirs))
fmt.Fprintf(w, "Logical size %s (%d bytes)\n", textutil.Size(r.TotSize), r.TotSize)
fmt.Fprintf(w, "Allocated size %s (%d bytes)\n", textutil.Size(r.TotAlloc), r.TotAlloc)
fmt.Fprintf(w, "Elapsed %s\n", st.Elapsed().Round(time.Millisecond))
for k, c := range st.ErrCounts {
if c > 0 {
fmt.Fprintf(w, "%-15s %s\n", inventory.ErrKind(k).String(), textutil.Count(c))
}
}
if st.Excluded > 0 {
fmt.Fprintf(w, "Excluded %s entries (%s)\n", textutil.Count(st.Excluded), textutil.Sanitize(strings.Join(st.Excludes, ", ")))
}
if st.SkippedMounts > 0 {
fmt.Fprintf(w, "Other FS skipped %s\n", textutil.Count(st.SkippedMounts))
}
if st.VirtualSkipped > 0 {
fmt.Fprintf(w, "Virtual FS skipped %s\n", textutil.Count(st.VirtualSkipped))
}
if st.HardlinkDups > 0 {
fmt.Fprintf(w, "Hard-link dups %s (counted once)\n", textutil.Count(st.HardlinkDups))
}
if st.Incomplete() {
fmt.Fprintln(w, "WARNING: totals are incomplete — some data could not be inspected.")
}
}
internal/export/export_test.go
-74
View File
@@ -1,74 +0,0 @@
package export
import (
"bytes"
"encoding/json"
"strings"
"testing"
"github.com/andydixon/mapsize/internal/inventory"
)
const hostile = "evil\x1b]0;pwned\x07\x1b[2J"
func tree() *inventory.Tree {
t := inventory.New("/r", inventory.KindDir)
d := t.Add(0, "d", inventory.KindDir)
f := t.Add(d, hostile, inventory.KindFile)
t.Node(f).Size, t.Node(f).Alloc = 10, 4096
t.Recompute()
t.Stats.Complete = true
return t
}
func noControl(t *testing.T, s string) {
t.Helper()
if strings.ContainsAny(s, "\x1b\x07") {
t.Fatalf("raw control characters in output: %q", s)
}
}
func TestOutputsAreTerminalSafe(t *testing.T) {
tr := tree()
var b bytes.Buffer
if err := CSV(&b, tr, 0, -1); err != nil {
t.Fatal(err)
}
noControl(t, b.String())
if !strings.Contains(b.String(), `evil\x1b]0;pwned\x07`) {
t.Fatalf("CSV should show the escaped name: %s", b.String())
}
b.Reset()
Table(&b, tr, []inventory.NodeID{2}, inventory.SizeAllocated)
Summary(&b, tr)
noControl(t, b.String())
b.Reset()
if err := JSON(&b, tr, 0, -1); err != nil {
t.Fatal(err)
}
noControl(t, b.String())
var v struct {
Tree struct {
Children []struct {
Children []struct{ Name string }
}
}
}
if err := json.Unmarshal(b.Bytes(), &v); err != nil {
t.Fatal(err)
}
// JSON keeps the exact name (escaped by the encoder, decoded back).
if got := v.Tree.Children[0].Children[0].Name; got != hostile {
t.Fatalf("JSON name %q", got)
}
}
func TestJSONDepthLimit(t *testing.T) {
var b bytes.Buffer
JSON(&b, tree(), 0, 1)
if strings.Contains(b.String(), "evil") {
t.Fatal("depth 1 should not include grandchildren")
}
}
internal/filter/apply.go
-78
View File
@@ -1,78 +0,0 @@
package filter
import (
"context"
"time"
"github.com/andydixon/mapsize/internal/inventory"
)
// Result holds per-node filtered sizes: the bytes of matching content beneath
// each node. A matching directory contributes its whole subtree (once, even
// if descendants match too). Matched marks every node satisfying the query. Nodes added
// after the result was computed (live scan) have no entry and are treated as
// not matching until the filter is re-applied.
type Result struct {
Query *Query
Mode inventory.SizeMode
Sizes []int64 // per NodeID; len = tree size at evaluation time
Matched []bool // node itself matched
Count int64 // number of matching nodes
Total int64
Elapsed time.Duration
}
// Size returns the filtered size of id (0 if out of range).
func (r *Result) Size(id inventory.NodeID) int64 {
if int(id) < len(r.Sizes) {
return r.Sizes[id]
}
return 0
}
// IsMatch reports whether id itself matched.
func (r *Result) IsMatch(id inventory.NodeID) bool {
return int(id) < len(r.Matched) && r.Matched[id]
}
// Apply evaluates q over the whole tree. The tree must be read-locked by the
// caller. It relies on parents having smaller IDs than their children.
func Apply(ctx context.Context, t *inventory.Tree, q *Query, m inventory.SizeMode) (*Result, error) {
start := time.Now()
n := t.Len()
r := &Result{Query: q, Mode: m, Sizes: make([]int64, n), Matched: make([]bool, n)}
covered := make([]bool, n)
c := &Ctx{Tree: t, Now: time.Now().UnixNano()}
if q.UsesPath {
c.dirPaths = map[inventory.NodeID]string{}
}
for i := range n {
if i&0xffff == 0 && ctx.Err() != nil {
return nil, ctx.Err()
}
id := inventory.NodeID(i)
nd := t.Node(id)
c.ID, c.Node, c.path = id, nd, ""
if c.dirPaths != nil && nd.IsDir() {
c.path = c.lowerPath()
c.dirPaths[id] = c.path
}
if i > 0 && q.Match(c) {
r.Matched[i] = true
covered[i] = true
r.Count++
}
if i > 0 && covered[nd.Parent] {
covered[i] = true
}
if covered[i] && nd.Flags&inventory.FlagHardlinkDup == 0 {
r.Sizes[i] = nd.Own(m)
}
}
for i := n - 1; i > 0; i-- {
r.Sizes[t.Node(inventory.NodeID(i)).Parent] += r.Sizes[i]
}
r.Total = r.Sizes[0]
r.Elapsed = time.Since(start)
return r, nil
}
internal/filter/eval.go
-176
View File
@@ -1,176 +0,0 @@
package filter
import (
"path/filepath"
"regexp"
"strings"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/platform"
)
// Ctx is the evaluation context for one node. The tree must be read-locked.
type Ctx struct {
Tree *inventory.Tree
ID inventory.NodeID
Node *inventory.Node
Now int64 // unix nanoseconds
path string
// dirPaths caches lower-cased directory paths during Apply, which visits
// parents before children, so a node's path is one concatenation.
dirPaths map[inventory.NodeID]string
}
func (c *Ctx) str(f Field) string {
n := c.Node
switch f {
case FName:
return strings.ToLower(n.Name)
case FPath:
if c.path == "" {
c.path = c.lowerPath()
}
return c.path
case FExt:
return c.Tree.ExtName(n)
case FType:
return strings.ToLower(n.Kind.String())
case FCategory:
if n.IsDir() {
return "directory"
}
return strings.ToLower(n.Cat.String())
case FOwner:
return strings.ToLower(platform.UserName(n.UID))
case FGroup:
return strings.ToLower(platform.GroupName(n.GID))
}
return ""
}
func (c *Ctx) num(f Field) int64 {
n := c.Node
switch f {
case FSize:
return n.TotSize
case FAllocated:
return n.TotAlloc
case FAge:
return c.Now - n.MTime
case FModified:
return n.MTime
case FFiles:
return int64(n.Files)
}
return 0
}
func (c *Ctx) lowerPath() string {
if c.dirPaths != nil {
if p, ok := c.dirPaths[c.ID]; ok {
return p
}
if pp, ok := c.dirPaths[c.Node.Parent]; ok {
sep := string(filepath.Separator)
if strings.HasSuffix(pp, sep) {
sep = ""
}
return pp + sep + strings.ToLower(c.Node.Name)
}
}
return strings.ToLower(c.Tree.Path(c.ID))
}
type node interface{ eval(*Ctx) bool }
type andNode struct{ l, r node }
type orNode struct{ l, r node }
type notNode struct{ n node }
func (n andNode) eval(c *Ctx) bool { return n.l.eval(c) && n.r.eval(c) }
func (n orNode) eval(c *Ctx) bool { return n.l.eval(c) || n.r.eval(c) }
func (n notNode) eval(c *Ctx) bool { return !n.n.eval(c) }
type containsNode struct {
field Field
sub string
}
func (n containsNode) eval(c *Ctx) bool { return strings.Contains(c.str(n.field), n.sub) }
type globNode struct {
field Field
pat string
}
func (n globNode) eval(c *Ctx) bool {
ok, _ := filepath.Match(n.pat, c.str(n.field))
return ok
}
type reNode struct {
field Field
re *regexp.Regexp
}
func (n reNode) eval(c *Ctx) bool { return n.re.MatchString(c.str(n.field)) }
type eqNode struct {
field Field
set []string
}
func (n eqNode) eval(c *Ctx) bool {
if n.field == FFlag {
for _, s := range n.set {
if c.Node.Flags&flagNames[s] != 0 {
return true
}
}
return false
}
v := c.str(n.field)
for _, s := range n.set {
if v == s {
return true
}
}
return false
}
type numNode struct {
field Field
op string
vals []int64
}
func (n numNode) eval(c *Ctx) bool {
if (n.field == FAge || n.field == FModified) && c.Node.MTime == 0 {
return false // unknown time never matches
}
v := c.num(n.field)
for _, x := range n.vals {
var ok bool
switch n.op {
case "=":
ok = v == x
case "!=":
ok = v != x
case ">":
ok = v > x
case ">=":
ok = v >= x
case "<":
ok = v < x
case "<=":
ok = v <= x
}
if ok {
return true
}
}
return false
}
// Match evaluates the query against one node.
func (q *Query) Match(c *Ctx) bool { return q.root.eval(c) }
internal/filter/filter_test.go
-122
View File
@@ -1,122 +0,0 @@
package filter
import (
"context"
"testing"
"time"
"github.com/andydixon/mapsize/internal/inventory"
)
func tree() *inventory.Tree {
t := inventory.New("/srv", inventory.KindDir)
vms := t.Add(0, "vms", inventory.KindDir)
cache := t.Add(0, "cache", inventory.KindDir)
add := func(p inventory.NodeID, name string, size int64, age time.Duration) inventory.NodeID {
id := t.Add(p, name, inventory.KindFile)
n := t.Node(id)
n.Size, n.Alloc = size, size
n.MTime = time.Now().Add(-age).UnixNano()
n.Cat = inventory.CategoryFor(inventory.Ext(name), false, false)
return id
}
add(vms, "ubuntu.qcow2", 80<<30, 400*24*time.Hour)
add(vms, "win.vmdk", 40<<30, time.Hour)
add(vms, "notes.txt", 1000, time.Hour)
add(cache, "blob.iso", 700<<20, 10*24*time.Hour)
add(cache, "x.tmp", 10, time.Hour)
t.Recompute()
return t
}
func names(t *testing.T, tr *inventory.Tree, q string) map[string]bool {
t.Helper()
pq, err := Parse(q, time.Now())
if err != nil {
t.Fatalf("Parse(%q): %v", q, err)
}
r, err := Apply(context.Background(), tr, pq, inventory.SizeLogical)
if err != nil {
t.Fatal(err)
}
out := map[string]bool{}
for i := range tr.Len() {
if r.IsMatch(inventory.NodeID(i)) {
out[tr.Node(inventory.NodeID(i)).Name] = true
}
}
return out
}
func TestQueries(t *testing.T) {
tr := tree()
cases := map[string][]string{
"ubuntu": {"ubuntu.qcow2"},
"*.iso": {"blob.iso"},
"size > 1GB": {"vms", "ubuntu.qcow2", "win.vmdk"},
"size > 1GB AND type = file": {"ubuntu.qcow2", "win.vmdk"},
"ext = qcow2": {"ubuntu.qcow2"},
"size > 500MB AND ext IN (iso,qcow2,vmdk)": {"ubuntu.qcow2", "win.vmdk", "blob.iso"},
"path contains cache": {"cache", "blob.iso", "x.tmp"},
"age > 365d": {"ubuntu.qcow2"},
"NOT type = dir AND size < 1k": {"notes.txt", "x.tmp"},
"(ext = iso OR ext = tmp) type=file": {"blob.iso", "x.tmp"},
"category = vm": {"ubuntu.qcow2", "win.vmdk", "blob.iso"},
"name matches '^w.*k$'": {"win.vmdk"},
"ext != qcow2 && type = file && size > 1GB": {"win.vmdk"},
"name = 'notes.txt'": {"notes.txt"},
}
for q, want := range cases {
got := names(t, tr, q)
if len(got) != len(want) {
t.Errorf("%q: got %v want %v", q, got, want)
continue
}
for _, w := range want {
if !got[w] {
t.Errorf("%q: got %v want %v", q, got, want)
}
}
}
}
func TestFilteredSizesNoDoubleCount(t *testing.T) {
tr := tree()
q, _ := Parse("name = vms OR ext = qcow2", time.Now())
r, _ := Apply(context.Background(), tr, q, inventory.SizeLogical)
if want := tr.Node(1).TotSize; r.Total != want {
t.Fatalf("total %d want %d (matching dir must not double count matching child)", r.Total, want)
}
}
func TestParseErrors(t *testing.T) {
for _, q := range []string{"", "size >", "size > banana", "(ext = iso", "ext = iso)", "size contains 5",
"name matches '['", "type = blob", "'unterminated", "a & b", "category = nope"} {
if _, err := Parse(q, time.Now()); err == nil {
t.Errorf("Parse(%q) should fail", q)
}
}
}
func TestParseSize(t *testing.T) {
cases := map[string]int64{"1GB": 1e9, "1GiB": 1 << 30, "1.5k": 1536, "500MB": 5e8, "10": 10}
for in, want := range cases {
if got, err := ParseSize(in); err != nil || got != want {
t.Errorf("ParseSize(%q)=%d,%v want %d", in, got, err, want)
}
}
}
func FuzzParse(f *testing.F) {
for _, s := range []string{"size > 1GB", "(a OR b) AND NOT c", "ext IN (a,b)", "'x", "name matches '.*'"} {
f.Add(s)
}
tr := tree()
f.Fuzz(func(t *testing.T, s string) {
q, err := Parse(s, time.Now())
if err != nil {
return
}
Apply(context.Background(), tr, q, inventory.SizeAllocated)
})
}
internal/filter/lexer.go
-103
View File
@@ -1,103 +0,0 @@
// Package filter implements the search/filter query language:
//
// query := or
// or := and { OR and }
// and := unary { [AND] unary } (juxtaposition means AND)
// unary := NOT unary | "(" query ")" | cmp | word
// cmp := field op value
// op := = | != | > | >= | < | <= | contains | matches | in | ~
// value := word | "quoted string" | "(" value { "," value } ")"
//
// A bare word matches the name: substring (case-insensitive), or a glob if it
// contains * ? or [. Parsing is independent of the UI and fully testable.
package filter
import (
"fmt"
"strings"
"unicode"
)
type tokKind int
const (
tEOF tokKind = iota
tWord
tString
tOp
tLParen
tRParen
tComma
)
type token struct {
kind tokKind
val string
pos int
}
func lex(s string) ([]token, error) {
var out []token
i := 0
for i < len(s) {
c := s[i]
switch {
case c == ' ' || c == '\t' || c == '\n' || c == '\r':
i++
case c == '(':
out = append(out, token{tLParen, "(", i})
i++
case c == ')':
out = append(out, token{tRParen, ")", i})
i++
case c == ',':
out = append(out, token{tComma, ",", i})
i++
case c == '"' || c == '\'':
q := c
start := i
i++
var b strings.Builder
for i < len(s) && s[i] != q {
if s[i] == '\\' && i+1 < len(s) {
i++
}
b.WriteByte(s[i])
i++
}
if i >= len(s) {
return nil, fmt.Errorf("unterminated string at %d", start+1)
}
i++
out = append(out, token{tString, b.String(), start})
case strings.ContainsRune("=!<>~&|", rune(c)):
start := i
op := string(c)
if i+1 < len(s) && (s[i+1] == '=' || (c == '&' && s[i+1] == '&') || (c == '|' && s[i+1] == '|')) {
op += string(s[i+1])
}
i += len(op)
switch op {
case "&&":
out = append(out, token{tWord, "AND", start})
case "||":
out = append(out, token{tWord, "OR", start})
case "!":
out = append(out, token{tWord, "NOT", start})
case "&", "|":
return nil, fmt.Errorf("unexpected %q at %d", op, start+1)
case "==":
out = append(out, token{tOp, "=", start})
default:
out = append(out, token{tOp, op, start})
}
default:
start := i
for i < len(s) && !unicode.IsSpace(rune(s[i])) && !strings.ContainsRune("()=!<>~,\"'&|", rune(s[i])) {
i++
}
out = append(out, token{tWord, s[start:i], start})
}
}
return append(out, token{tEOF, "", len(s)}), nil
}
internal/filter/parser.go
-382
View File
@@ -1,382 +0,0 @@
package filter
import (
"fmt"
"path/filepath"
"regexp"
"strconv"
"strings"
"time"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/textutil"
)
// Field is a filterable attribute.
type Field int
const (
FName Field = iota
FPath
FExt
FType
FCategory
FSize
FAllocated
FAge
FModified
FOwner
FGroup
FFiles
FFlag
)
var fieldNames = map[string]Field{
"name": FName, "path": FPath, "ext": FExt, "extension": FExt, "type": FType, "kind": FType,
"category": FCategory, "cat": FCategory, "size": FSize, "allocated": FAllocated, "alloc": FAllocated,
"age": FAge, "modified": FModified, "mtime": FModified, "owner": FOwner, "user": FOwner, "group": FGroup,
"files": FFiles, "flag": FFlag, "is": FFlag,
}
// Fields lists the field names for help text.
const Fields = "name path ext type category size allocated age modified owner group files flag"
// Query is a compiled filter.
type Query struct {
src string
root node
UsesPath bool
}
func (q *Query) String() string { return q.src }
type parser struct {
toks []token
i int
q *Query
now time.Time
}
// Parse compiles a query. now anchors relative ages (pass time.Now()).
func Parse(s string, now time.Time) (*Query, error) {
toks, err := lex(s)
if err != nil {
return nil, err
}
q := &Query{src: s}
p := &parser{toks: toks, q: q, now: now}
if p.peek().kind == tEOF {
return nil, fmt.Errorf("empty query")
}
q.root, err = p.or()
if err != nil {
return nil, err
}
if t := p.peek(); t.kind != tEOF {
return nil, fmt.Errorf("unexpected %q at %d", t.val, t.pos+1)
}
return q, nil
}
func (p *parser) peek() token { return p.toks[p.i] }
func (p *parser) next() token {
t := p.toks[p.i]
if t.kind != tEOF {
p.i++
}
return t
}
func isKw(t token, kw string) bool { return t.kind == tWord && strings.EqualFold(t.val, kw) }
func (p *parser) or() (node, error) {
l, err := p.and()
if err != nil {
return nil, err
}
for isKw(p.peek(), "OR") {
p.next()
r, err := p.and()
if err != nil {
return nil, err
}
l = orNode{l, r}
}
return l, nil
}
func (p *parser) and() (node, error) {
l, err := p.unary()
if err != nil {
return nil, err
}
for {
t := p.peek()
if isKw(t, "AND") {
p.next()
} else if t.kind == tEOF || t.kind == tRParen || isKw(t, "OR") {
return l, nil
}
r, err := p.unary()
if err != nil {
return nil, err
}
l = andNode{l, r}
}
}
func (p *parser) unary() (node, error) {
t := p.peek()
switch {
case isKw(t, "NOT"):
p.next()
n, err := p.unary()
if err != nil {
return nil, err
}
return notNode{n}, nil
case t.kind == tLParen:
p.next()
n, err := p.or()
if err != nil {
return nil, err
}
if p.next().kind != tRParen {
return nil, fmt.Errorf("missing ) for ( at %d", t.pos+1)
}
return n, nil
case t.kind == tWord || t.kind == tString:
p.next()
if f, ok := fieldNames[strings.ToLower(t.val)]; ok && t.kind == tWord && p.isOp(p.peek()) {
return p.cmp(f, p.next())
}
return nameMatch(t.val), nil
}
return nil, fmt.Errorf("unexpected %q at %d", t.val, t.pos+1)
}
func (p *parser) isOp(t token) bool {
return t.kind == tOp || isKw(t, "contains") || isKw(t, "matches") || isKw(t, "in")
}
func nameMatch(v string) node {
lv := strings.ToLower(v)
if strings.ContainsAny(v, "*?[") {
return globNode{field: FName, pat: lv}
}
return containsNode{field: FName, sub: lv}
}
func (p *parser) values() ([]token, error) {
if p.peek().kind != tLParen {
t := p.next()
if t.kind != tWord && t.kind != tString {
return nil, fmt.Errorf("expected value at %d", t.pos+1)
}
return []token{t}, nil
}
p.next()
var out []token
for {
t := p.next()
if t.kind != tWord && t.kind != tString {
return nil, fmt.Errorf("expected value at %d", t.pos+1)
}
out = append(out, t)
switch n := p.next(); n.kind {
case tComma:
case tRParen:
return out, nil
default:
return nil, fmt.Errorf("expected , or ) at %d", n.pos+1)
}
}
}
func (p *parser) cmp(f Field, opTok token) (node, error) {
op := strings.ToLower(opTok.val)
if op == "~" {
op = "matches"
}
vals, err := p.values()
if err != nil {
return nil, err
}
if op != "in" && len(vals) != 1 {
return nil, fmt.Errorf("operator %s takes one value", op)
}
if f == FPath {
p.q.UsesPath = true
}
switch f {
case FSize, FAllocated, FAge, FModified, FFiles:
if op == "contains" || op == "matches" {
return nil, fmt.Errorf("%s cannot be used with numeric field", op)
}
var nums []int64
for _, v := range vals {
n, err := p.number(f, v.val)
if err != nil {
return nil, err
}
nums = append(nums, n)
}
if op == "in" {
op = "="
}
return numNode{field: f, op: op, vals: nums}, nil
}
// String-like fields.
switch op {
case "contains":
return containsNode{field: f, sub: strings.ToLower(vals[0].val)}, nil
case "matches":
re, err := regexp.Compile("(?i)" + vals[0].val)
if err != nil {
return nil, fmt.Errorf("bad regular expression: %v", err)
}
return reNode{field: f, re: re}, nil
case "=", "!=", "in":
var n node
var set []string
for _, v := range vals {
s := strings.ToLower(v.val)
if f == FExt {
s = strings.TrimPrefix(s, ".")
}
if f == FCategory {
c, ok := inventory.ParseCategory(s)
if !ok {
return nil, fmt.Errorf("unknown category %q", v.val)
}
s = strings.ToLower(c.String())
}
if f == FType {
s, err = normType(s)
if err != nil {
return nil, err
}
}
if f == FFlag {
if _, ok := flagNames[s]; !ok {
return nil, fmt.Errorf("unknown flag %q (sparse, hardlink, error, mount, incomplete, followed, broken)", v.val)
}
}
set = append(set, s)
}
if len(set) == 1 && strings.ContainsAny(set[0], "*?[") {
if _, err := filepath.Match(set[0], ""); err != nil {
return nil, fmt.Errorf("bad pattern %q", vals[0].val)
}
n = globNode{field: f, pat: set[0]}
} else {
n = eqNode{field: f, set: set}
}
if op == "!=" {
n = notNode{n}
}
return n, nil
}
return nil, fmt.Errorf("operator %s not valid for %s", op, opTok.val)
}
func normType(s string) (string, error) {
switch s {
case "f", "file":
return "file", nil
case "d", "dir", "directory":
return "directory", nil
case "l", "link", "symlink":
return "symlink", nil
case "other", "special":
return "special", nil
}
return "", fmt.Errorf("unknown type %q (file, dir, symlink, other)", s)
}
var flagNames = map[string]inventory.Flags{
"sparse": inventory.FlagSparse, "hardlink": inventory.FlagHardlinked, "hardlinked": inventory.FlagHardlinked,
"error": inventory.FlagError, "mount": inventory.FlagMountPoint, "incomplete": inventory.FlagIncomplete,
"followed": inventory.FlagFollowed, "broken": inventory.FlagBrokenLink,
}
// number parses sizes ("1.5GiB", "500MB", "10k"), ages ("365d", "2w") and
// dates ("2024-01-31"). Ages and dates are converted to modification times
// (unix nanoseconds) relative to p.now.
func (p *parser) number(f Field, s string) (int64, error) {
switch f {
case FSize, FAllocated:
return ParseSize(s)
case FFiles:
return strconv.ParseInt(s, 10, 64)
case FAge:
d, err := ParseAge(s)
return int64(d), err
case FModified:
for _, layout := range []string{"2006-01-02", "2006-01-02T15:04", "2006-01-02 15:04", time.RFC3339} {
if t, err := time.ParseInLocation(layout, s, time.Local); err == nil {
return t.UnixNano(), nil
}
}
if d, err := ParseAge(s); err == nil {
return p.now.Add(-d).UnixNano(), nil
}
return 0, fmt.Errorf("bad date %q (use YYYY-MM-DD or an age like 30d)", s)
}
return 0, fmt.Errorf("not numeric")
}
// ParseSize parses a size with optional unit. kB/MB/GB/TB are SI (powers of
// 1000); KiB/MiB/GiB/TiB are IEC (1024). Bare K/M/G/T follow the display
// setting (textutil.SI).
func ParseSize(s string) (int64, error) {
i := 0
for i < len(s) && (s[i] >= '0' && s[i] <= '9' || s[i] == '.') {
i++
}
if i == 0 {
return 0, fmt.Errorf("bad size %q", s)
}
v, err := strconv.ParseFloat(s[:i], 64)
if err != nil {
return 0, fmt.Errorf("bad size %q", s)
}
unit := strings.ToLower(strings.TrimSpace(s[i:]))
bare := 1024.0
if textutil.SI {
bare = 1000
}
mult := map[string]float64{"": 1, "b": 1,
"k": bare, "m": bare * bare, "g": bare * bare * bare, "t": bare * bare * bare * bare, "p": bare * bare * bare * bare * bare,
"kb": 1e3, "mb": 1e6, "gb": 1e9, "tb": 1e12, "pb": 1e15,
"kib": 1 << 10, "mib": 1 << 20, "gib": 1 << 30, "tib": 1 << 40, "pib": 1 << 50}
m, ok := mult[unit]
if !ok {
return 0, fmt.Errorf("unknown size unit %q", s[i:])
}
r := v * m
if r > 9e18 {
return 0, fmt.Errorf("size %q too large", s)
}
return int64(r), nil
}
// ParseAge parses durations like 90s, 30min, 12h, 7d, 2w, 6mo, 1y.
func ParseAge(s string) (time.Duration, error) {
i := 0
for i < len(s) && (s[i] >= '0' && s[i] <= '9' || s[i] == '.') {
i++
}
v, err := strconv.ParseFloat(s[:i], 64)
if err != nil {
return 0, fmt.Errorf("bad age %q", s)
}
day := 24 * time.Hour
units := map[string]time.Duration{"s": time.Second, "sec": time.Second, "min": time.Minute,
"h": time.Hour, "d": day, "": day, "w": 7 * day, "m": 30 * day, "mo": 30 * day, "y": 365 * day}
u, ok := units[strings.ToLower(s[i:])]
if !ok {
return 0, fmt.Errorf("unknown age unit %q (s, min, h, d, w, mo, y)", s[i:])
}
return time.Duration(v * float64(u)), nil
}
internal/inventory/category.go
-117
View File
@@ -1,117 +0,0 @@
package inventory
import "strings"
// Category is a coarse file classification used for colouring and stats.
type Category uint8
const (
CatOther Category = iota
CatImage
CatVideo
CatAudio
CatArchive
CatDiskImage
CatDatabase
CatCode
CatDocument
CatExecutable
CatLog
CatCache
NumCategories
)
var categoryNames = [NumCategories]string{
"Other", "Image", "Video", "Audio", "Archive", "Disk image", "Database",
"Source code", "Document", "Executable", "Log", "Cache",
}
func (c Category) String() string {
if c < NumCategories {
return categoryNames[c]
}
return "Other"
}
// ParseCategory resolves a category by (case-insensitive) name or prefix.
func ParseCategory(s string) (Category, bool) {
s = strings.ToLower(strings.TrimSpace(s))
for i, n := range categoryNames {
ln := strings.ToLower(n)
if ln == s || strings.ReplaceAll(ln, " ", "") == s || strings.HasPrefix(ln, s) && len(s) >= 3 {
return Category(i), true
}
}
switch s {
case "vm", "virtual machine", "disk":
return CatDiskImage, true
case "db":
return CatDatabase, true
case "src", "source":
return CatCode, true
case "exe", "binary":
return CatExecutable, true
}
return 0, false
}
var extCategory = map[string]Category{}
func init() {
add := func(c Category, exts string) {
for _, e := range strings.Fields(exts) {
extCategory[e] = c
}
}
add(CatImage, "jpg jpeg png gif bmp tif tiff webp heic heif raw cr2 cr3 nef arw dng svg ico psd xcf avif jxl orf rw2")
add(CatVideo, "mp4 mkv mov avi wmv flv webm m4v mpg mpeg ts m2ts vob 3gp ogv mts")
add(CatAudio, "mp3 flac wav ogg oga opus m4a aac wma aiff aif alac mid midi")
add(CatArchive, "zip tar gz tgz bz2 tbz xz txz zst zstd 7z rar lz lz4 lzma cab deb rpm apk jar war cpio pkg snap whl gem")
add(CatDiskImage, "iso img qcow qcow2 vmdk vdi vhd vhdx hdd ova ovf dmg raw vmem vmsn nvram sav")
add(CatDatabase, "db sqlite sqlite3 db3 mdb accdb frm ibd myd myi ldf mdf ndf dbf wal kdbx realm lmdb rdb aof")
add(CatCode, "c h cc cpp cxx hpp hh go rs py pyc js mjs cjs ts tsx jsx java kt kts scala rb php pl pm sh bash zsh fish lua swift m mm cs fs vb r jl hs ml ex exs erl clj dart sql css scss sass less html htm vue svelte json yaml yml toml xml proto gradle cmake mk ipynb o a lib obj class")
add(CatDocument, "pdf doc docx odt rtf txt md rst tex xls xlsx ods csv tsv ppt pptx odp epub mobi djvu pages numbers key org adoc")
add(CatExecutable, "exe dll so dylib bin msi app elf ko sys efi appimage com")
add(CatLog, "log journal")
add(CatCache, "cache tmp temp swp bak old part crdownload")
}
// CategoryFor classifies a file by its (lower-case) extension and by
// whether it lives beneath a cache or log directory.
func CategoryFor(ext string, inCache, inLog bool) Category {
c, ok := extCategory[ext]
switch {
case inCache && (!ok || c == CatOther || c == CatCode):
return CatCache
case inLog && (!ok || c == CatArchive || c == CatOther):
return CatLog
case ok:
return c
}
return CatOther
}
// PathHints reports whether a directory path contains a cache or log
// component. It is evaluated once per directory, not per file.
func PathHints(path string) (inCache, inLog bool) {
lp := strings.ToLower(path)
for _, part := range strings.FieldsFunc(lp, func(r rune) bool { return r == '/' || r == '\\' }) {
switch part {
case "cache", ".cache", "caches", "__pycache__", ".npm", ".gradle", "inetcache", "temp", "tmp":
inCache = true
case "log", "logs", ".logs", "journal":
inLog = true
}
}
return
}
// Ext returns the lower-case extension of name without the dot ("" if none,
// or if the name is a dotfile like ".bashrc").
func Ext(name string) string {
i := strings.LastIndexByte(name, '.')
if i <= 0 || i == len(name)-1 || len(name)-i > 16 {
return ""
}
return strings.ToLower(name[i+1:])
}
internal/inventory/query.go
-176
View File
@@ -1,176 +0,0 @@
package inventory
import (
"container/heap"
"sort"
)
// SizeMode selects which size drives layout and sorting.
type SizeMode uint8
const (
SizeAllocated SizeMode = iota // disk usage (default)
SizeLogical // apparent size
)
func (m SizeMode) String() string {
if m == SizeLogical {
return "logical"
}
return "allocated"
}
// Total returns the node's aggregate size in the given mode.
func (n *Node) Total(m SizeMode) int64 {
if m == SizeLogical {
return n.TotSize
}
return n.TotAlloc
}
// Own returns the node's own size in the given mode.
func (n *Node) Own(m SizeMode) int64 {
if m == SizeLogical {
return n.Size
}
return n.Alloc
}
type ranked struct {
id NodeID
key int64
}
type minHeap []ranked
func (h minHeap) Len() int { return len(h) }
func (h minHeap) Less(i, j int) bool {
if h[i].key != h[j].key {
return h[i].key < h[j].key
}
return h[i].id > h[j].id
}
func (h minHeap) Swap(i, j int) { h[i], h[j] = h[j], h[i] }
func (h *minHeap) Push(x any) { *h = append(*h, x.(ranked)) }
func (h *minHeap) Pop() any {
old := *h
x := old[len(old)-1]
*h = old[:len(old)-1]
return x
}
// TopK keeps the k entries with the largest keys; O(n log k).
type TopK struct {
k int
h minHeap
}
// NewTopK creates a collector for the k largest keys.
func NewTopK(k int) *TopK { return &TopK{k: k} }
// Offer considers one candidate.
func (t *TopK) Offer(id NodeID, key int64) {
if t.k <= 0 {
return
}
if len(t.h) < t.k {
heap.Push(&t.h, ranked{id, key})
return
}
if r := t.h[0]; key > r.key || key == r.key && id < r.id {
t.h[0] = ranked{id, key}
heap.Fix(&t.h, 0)
}
}
// Sorted returns the collected IDs, largest key first (ties by ID).
func (t *TopK) Sorted() []NodeID {
rs := append([]ranked(nil), t.h...)
sort.Slice(rs, func(i, j int) bool {
if rs[i].key != rs[j].key {
return rs[i].key > rs[j].key
}
return rs[i].id < rs[j].id
})
out := make([]NodeID, len(rs))
for i, r := range rs {
out[i] = r.id
}
return out
}
// Walk visits id and all its descendants depth-first (iteratively).
func (t *Tree) Walk(id NodeID, fn func(NodeID, *Node) bool) {
stack := []NodeID{id}
for len(stack) > 0 {
cur := stack[len(stack)-1]
stack = stack[:len(stack)-1]
n := t.Node(cur)
if !fn(cur, n) {
continue
}
for c := n.FirstChild; c != NoNode; c = t.Node(c).NextSibling {
stack = append(stack, c)
}
}
}
// Top returns the k nodes beneath (and including) root with the largest key,
// considering only nodes accepted by keep.
func (t *Tree) Top(root NodeID, k int, keep func(*Node) bool, key func(*Node) int64) []NodeID {
tk := NewTopK(k)
t.Walk(root, func(id NodeID, n *Node) bool {
if keep(n) {
tk.Offer(id, key(n))
}
return true
})
return tk.Sorted()
}
// SortedChildren returns the children of id ordered by size descending.
func (t *Tree) SortedChildren(id NodeID, m SizeMode) []NodeID {
var out []NodeID
t.Children(id, func(c NodeID, _ *Node) { out = append(out, c) })
sort.Slice(out, func(i, j int) bool {
a, b := t.Node(out[i]).Total(m), t.Node(out[j]).Total(m)
if a != b {
return a > b
}
return out[i] < out[j]
})
return out
}
// ExtStat summarises one extension.
type ExtStat struct {
Ext string
Cat Category
Count int64
Size int64 // logical
Alloc int64
}
// ExtStats aggregates file statistics by extension beneath root.
func (t *Tree) ExtStats(root NodeID) []ExtStat {
stats := make([]ExtStat, t.ExtCount())
t.Walk(root, func(_ NodeID, n *Node) bool {
if n.Kind == KindFile && n.Flags&FlagHardlinkDup == 0 {
s := &stats[n.Ext]
s.Count++
s.Size += n.Size
s.Alloc += n.Alloc
s.Cat = n.Cat
}
return true
})
out := stats[:0]
for i, s := range stats {
if s.Count > 0 {
s.Ext = t.exts[i]
s.Cat = CategoryFor(s.Ext, false, false)
out = append(out, s)
}
}
return out
}
internal/inventory/query_test.go
-42
View File
@@ -1,42 +0,0 @@
package inventory
import "testing"
func TestTopKAndRecompute(t *testing.T) {
tr := New("/r", KindDir)
d := tr.Add(0, "d", KindDir)
for i := range 100 {
id := tr.Add(d, "f", KindFile)
n := tr.Node(id)
n.Size, n.Alloc = int64(i), int64(i)
}
tr.Recompute()
if got := tr.Node(0).TotSize; got != 4950 {
t.Fatalf("total %d", got)
}
if tr.Node(0).Files != 100 || tr.Node(0).Dirs != 1 {
t.Fatalf("counts %d %d", tr.Node(0).Files, tr.Node(0).Dirs)
}
top := tr.Top(0, 3, func(n *Node) bool { return n.Kind == KindFile }, func(n *Node) int64 { return n.Size })
if len(top) != 3 || tr.Node(top[0]).Size != 99 || tr.Node(top[2]).Size != 97 {
t.Fatalf("top %v", top)
}
if p := tr.Path(top[0]); p != "/r/d/f" {
t.Fatalf("path %q", p)
}
}
func TestExtAndCategory(t *testing.T) {
cases := map[string]string{"a.QCOW2": "qcow2", ".bashrc": "", "noext": "", "x.tar.gz": "gz", "trail.": ""}
for in, want := range cases {
if got := Ext(in); got != want {
t.Errorf("Ext(%q)=%q want %q", in, got, want)
}
}
if CategoryFor("qcow2", false, false) != CatDiskImage || CategoryFor("bin", true, false) != CatExecutable {
t.Fatal("category mapping")
}
if CategoryFor("", true, false) != CatCache {
t.Fatal("cache hint")
}
}
internal/inventory/stats.go
-106
View File
@@ -1,106 +0,0 @@
package inventory
import (
"errors"
"io/fs"
"syscall"
"time"
)
// ErrKind classifies scan errors for reporting.
type ErrKind uint8
const (
ErrPermission ErrKind = iota
ErrVanished
ErrIO
ErrLoop
ErrOther
NumErrKinds
)
func (k ErrKind) String() string {
return [...]string{"Permission denied", "Vanished during scan", "I/O error", "Symlink loop", "Other error"}[k]
}
// Classify maps an OS error to an ErrKind.
func Classify(err error) ErrKind {
switch {
case errors.Is(err, fs.ErrPermission):
return ErrPermission
case errors.Is(err, fs.ErrNotExist):
return ErrVanished
case errors.Is(err, syscall.ELOOP):
return ErrLoop
case errors.Is(err, syscall.EIO):
return ErrIO
}
return ErrOther
}
// ErrorRecord is one recorded scan error.
type ErrorRecord struct {
Node NodeID // node the error is attached to (the directory for listing failures)
Name string // entry name when no node was created (e.g. vanished)
Kind ErrKind
Msg string
}
// MaxErrorRecords caps the detailed error list; counts are always exact.
const MaxErrorRecords = 100_000
// Stats describes a scan as a whole.
type Stats struct {
Root string
Start, End time.Time
Complete bool // scan finished (possibly with errors)
Cancelled bool
Files, Dirs, Symlinks, Others int64
ErrCounts [NumErrKinds]int64
Errors []ErrorRecord
Excluded int64
SkippedMounts int64
VirtualSkipped int64
LoopsSkipped int64
BrokenLinks int64
HardlinkDups int64
Unscanned int64 // directories never read because the scan was cancelled
Excludes []string
OneFileSystem bool
Follow string
Workers int
FromSnapshot string
}
// TotalErrors sums all error kinds.
func (s *Stats) TotalErrors() int64 {
var t int64
for _, c := range s.ErrCounts {
t += c
}
return t
}
// AddError records an error (the detailed list is capped).
func (s *Stats) AddError(r ErrorRecord) {
s.ErrCounts[r.Kind]++
if len(s.Errors) < MaxErrorRecords {
s.Errors = append(s.Errors, r)
}
}
// Elapsed is the scan duration so far (or total once finished).
func (s *Stats) Elapsed() time.Duration {
if !s.End.IsZero() {
return s.End.Sub(s.Start)
}
return time.Since(s.Start)
}
// Incomplete reports whether totals may under-count real usage.
func (s *Stats) Incomplete() bool {
return !s.Complete || s.Cancelled || s.TotalErrors() > 0 || s.Unscanned > 0
}
internal/inventory/tree.go
-338
View File
@@ -1,338 +0,0 @@
// Package inventory is the in-memory filesystem model: an append-only arena
// of nodes addressed by stable NodeIDs, with incrementally maintained
// directory aggregates.
//
// Concurrency: a Tree has exactly one writer (the scan controller or a
// snapshot loader). Other goroutines must hold RLock while reading node data.
// Nodes are never removed, so a NodeID stays valid for the Tree's lifetime.
package inventory
import (
"math"
"path/filepath"
"slices"
"sync"
"time"
)
// NodeID addresses a node within one Tree. The root is always 0.
type NodeID uint32
// NoNode is the nil NodeID.
const NoNode NodeID = ^NodeID(0)
// Kind is the node type.
type Kind uint8
const (
KindFile Kind = iota
KindDir
KindSymlink
KindOther // devices, sockets, fifos, reparse points
)
func (k Kind) String() string {
return [...]string{"File", "Directory", "Symlink", "Special"}[k&3]
}
// Flags record per-node facts.
type Flags uint16
const (
FlagHardlinkDup Flags = 1 << iota // additional link to an inode counted elsewhere
FlagMountPoint // directory on a different device than its parent
FlagVirtualFS // virtual filesystem, not descended
FlagSkippedFS // other filesystem, not descended (--one-file-system)
FlagSparse // allocated < logical (sparse or compressed)
FlagError // stat or read failed
FlagIncomplete // directory contents not (fully) read
FlagScanned // directory fully read
FlagLoop // directory already visited via another path
FlagFollowed // reached through a followed symlink
FlagAllocUnknown // platform cannot report allocated size
FlagBrokenLink // symlink target does not exist
FlagHardlinked // regular file with link count > 1
FlagDeleted // moved to trash from the UI
)
// Node is one filesystem object. Sizes are own sizes; Tot* fields are
// aggregates including the node itself and (for directories) everything
// beneath it.
type Node struct {
Parent, FirstChild, NextSibling NodeID
dir uint32 // index into dir side table (directories only)
Name string // single path component; the root holds the full root path
Size, Alloc int64 // own logical / allocated bytes
TotSize, TotAlloc int64 // aggregates (hard-link duplicates contribute 0)
MTime int64 // unix nanoseconds
Files, Dirs uint32 // recursive counts beneath (dirs excludes self)
Errors uint32 // recursive error count beneath and including self
Mode uint32 // fs.FileMode bits
UID, GID uint32
Nlink uint32
Ext uint16
Flags Flags
Kind Kind
Cat Category
}
// IsDir reports whether the node is a directory.
func (n *Node) IsDir() bool { return n.Kind == KindDir }
// Delta is an aggregate change applied to a node and all its ancestors.
type Delta struct {
Size, Alloc int64
Files, Dirs, Errors int64
Cat [NumCategories]int64
}
const chunkBits = 16
const chunkSize = 1 << chunkBits
// chunked is a growable array that never moves existing elements.
type chunked[T any] struct {
c [][]T
n int
}
func (c *chunked[T]) add(v T) uint32 {
if c.n&(chunkSize-1) == 0 && c.n>>chunkBits == len(c.c) {
c.c = append(c.c, make([]T, 0, chunkSize))
}
ch := &c.c[c.n>>chunkBits]
*ch = append(*ch, v)
c.n++
return uint32(c.n - 1)
}
func (c *chunked[T]) at(i uint32) *T { return &c.c[i>>chunkBits][i&(chunkSize-1)] }
// Tree is the inventory for one scan root.
type Tree struct {
sync.RWMutex
nodes chunked[Node]
dirs chunked[[NumCategories]int64]
exts []string
extIdx map[string]uint16
Stats Stats
}
// New creates a tree whose root has the given full path and kind.
func New(rootPath string, kind Kind) *Tree {
t := &Tree{exts: []string{""}, extIdx: map[string]uint16{"": 0}}
t.Stats.Root = rootPath
t.Stats.Start = time.Now()
t.Add(NoNode, rootPath, kind)
return t
}
// Len returns the number of nodes.
func (t *Tree) Len() int { return t.nodes.n }
// Node returns the node for id. The pointer stays valid; reading its fields
// concurrently with the writer requires RLock.
func (t *Tree) Node(id NodeID) *Node { return t.nodes.at(uint32(id)) }
// Root returns the root NodeID (always 0).
func (t *Tree) Root() NodeID { return 0 }
// Add appends a child node and links it under parent. Only the writer may
// call it. It does not touch aggregates; use Propagate.
func (t *Tree) Add(parent NodeID, name string, kind Kind) NodeID {
n := Node{Parent: parent, FirstChild: NoNode, NextSibling: NoNode, Name: name, Kind: kind}
if kind == KindDir {
n.dir = t.dirs.add([NumCategories]int64{})
} else {
n.Ext = t.internExt(Ext(name))
}
id := NodeID(t.nodes.add(n))
if parent != NoNode {
p := t.Node(parent)
t.Node(id).NextSibling = p.FirstChild
p.FirstChild = id
}
return id
}
func (t *Tree) internExt(e string) uint16 {
if i, ok := t.extIdx[e]; ok {
return i
}
if len(t.exts) >= 1<<16-1 {
return 0
}
t.exts = append(t.exts, e)
t.extIdx[e] = uint16(len(t.exts) - 1)
return uint16(len(t.exts) - 1)
}
// ExtName returns the extension string for a node ("" if none).
func (t *Tree) ExtName(n *Node) string { return t.exts[n.Ext] }
// ExtCount returns the number of interned extensions (including "").
func (t *Tree) ExtCount() int { return len(t.exts) }
// ExtByIndex returns an interned extension.
func (t *Tree) ExtByIndex(i int) string { return t.exts[i] }
// CatSizes returns the per-category allocated bytes beneath a directory.
func (t *Tree) CatSizes(id NodeID) *[NumCategories]int64 {
n := t.Node(id)
if n.Kind != KindDir {
return nil
}
return t.dirs.at(n.dir)
}
// Propagate adds d to id and every ancestor. Cost is O(depth); callers batch
// many entries into one Delta.
func (t *Tree) Propagate(id NodeID, d Delta) {
for id != NoNode {
n := t.Node(id)
n.TotSize += d.Size
n.TotAlloc += d.Alloc
n.Files = addU32(n.Files, d.Files)
n.Dirs = addU32(n.Dirs, d.Dirs)
n.Errors = addU32(n.Errors, d.Errors)
if n.Kind == KindDir {
cs := t.dirs.at(n.dir)
for i, v := range d.Cat {
cs[i] += v
}
}
id = n.Parent
}
}
// satAdd adds non-negative int64s, saturating at the maximum.
func satAdd(a, b int64) int64 {
if b > 0 && a > math.MaxInt64-b {
return math.MaxInt64
}
return a + b
}
func addU32(a uint32, d int64) uint32 {
v := int64(a) + d
if v < 0 {
return 0
}
if v > 1<<32-1 {
return 1<<32 - 1
}
return uint32(v)
}
// Children calls fn for each child of id (in reverse insertion order).
func (t *Tree) Children(id NodeID, fn func(NodeID, *Node)) {
for c := t.Node(id).FirstChild; c != NoNode; {
n := t.Node(c)
fn(c, n)
c = n.NextSibling
}
}
// ChildCount returns the number of direct children.
func (t *Tree) ChildCount(id NodeID) int {
k := 0
for c := t.Node(id).FirstChild; c != NoNode; c = t.Node(c).NextSibling {
k++
}
return k
}
// Path reconstructs the full path of id.
func (t *Tree) Path(id NodeID) string {
var parts []string
for id != NoNode {
n := t.Node(id)
parts = append(parts, n.Name)
id = n.Parent
}
slices.Reverse(parts)
return filepath.Join(parts...)
}
// Ancestors returns the chain root..id inclusive.
func (t *Tree) Ancestors(id NodeID) []NodeID {
var out []NodeID
for id != NoNode {
out = append(out, id)
id = t.Node(id).Parent
}
slices.Reverse(out)
return out
}
// IsAncestor reports whether a is id or an ancestor of id.
func (t *Tree) IsAncestor(a, id NodeID) bool {
for id != NoNode {
if id == a {
return true
}
id = t.Node(id).Parent
}
return false
}
// Depth returns the number of ancestors of id.
func (t *Tree) Depth(id NodeID) int {
d := 0
for id = t.Node(id).Parent; id != NoNode; id = t.Node(id).Parent {
d++
}
return d
}
// Recompute rebuilds all aggregates bottom-up. It relies on the invariant
// that a parent's ID is smaller than its children's, which both the scanner
// and the snapshot loader guarantee. Used after loading a snapshot.
func (t *Tree) Recompute() {
for i := 0; i < t.Len(); i++ {
n := t.Node(NodeID(i))
n.TotSize, n.TotAlloc, n.Files, n.Dirs = 0, 0, 0, 0
n.Errors = 0
if n.Flags&FlagError != 0 {
n.Errors = 1
}
if n.Flags&FlagHardlinkDup == 0 {
n.TotSize, n.TotAlloc = n.Size, n.Alloc
}
if n.Kind == KindDir {
*t.dirs.at(n.dir) = [NumCategories]int64{}
}
}
// Sums saturate: snapshot data is untrusted and must not wrap to
// negative totals.
for i := t.Len() - 1; i > 0; i-- {
n := t.Node(NodeID(i))
p := t.Node(n.Parent)
p.TotSize = satAdd(p.TotSize, n.TotSize)
p.TotAlloc = satAdd(p.TotAlloc, n.TotAlloc)
p.Errors = addU32(p.Errors, int64(n.Errors))
cs := t.dirs.at(p.dir)
if n.Kind == KindDir {
p.Files = addU32(p.Files, int64(n.Files))
p.Dirs = addU32(p.Dirs, int64(n.Dirs)+1)
for c, v := range t.dirs.at(n.dir) {
cs[c] = satAdd(cs[c], v)
}
} else {
p.Files = addU32(p.Files, 1)
if n.Flags&FlagHardlinkDup == 0 {
cs[n.Cat] = satAdd(cs[n.Cat], n.Alloc)
}
}
}
}
// ExtIndexer returns a function interning extension strings; used by
// loaders that rebuild a tree from external data.
func (t *Tree) ExtIndexer() func(string) uint16 { return t.internExt }
internal/platform/delete_unix.go
-35
View File
@@ -1,35 +0,0 @@
//go:build unix
package platform
import (
"errors"
"io/fs"
"os"
"path/filepath"
"syscall"
)
// Delete permanently removes path (recursively for a directory) if it still
// matches want. The check and the removal both go through an os.Root on the
// parent directory, so a symlink swapped in since the scan cannot redirect
// the removal outside it.
func Delete(path string, want Expect) error {
if err := checkAbs(path); err != nil {
return err
}
dir, base := filepath.Dir(path), filepath.Base(path)
root, err := os.OpenRoot(dir)
if err != nil {
return err
}
defer root.Close()
fi, err := root.Lstat(base)
if err != nil {
return err
}
if fi.Sys().(*syscall.Stat_t).Uid != want.UID || fi.Mode()&fs.ModeType != want.Mode&fs.ModeType {
return errors.New("item changed since the scan (owner or type differs); rescan first")
}
return root.RemoveAll(base)
}
internal/platform/mode_unix.go
-36
View File
@@ -1,36 +0,0 @@
//go:build unix
package platform
import (
"io/fs"
"syscall"
)
func modeFromUnix(m uint32) fs.FileMode {
mode := fs.FileMode(m & 0o777)
switch m & syscall.S_IFMT {
case syscall.S_IFDIR:
mode |= fs.ModeDir
case syscall.S_IFLNK:
mode |= fs.ModeSymlink
case syscall.S_IFIFO:
mode |= fs.ModeNamedPipe
case syscall.S_IFSOCK:
mode |= fs.ModeSocket
case syscall.S_IFCHR:
mode |= fs.ModeDevice | fs.ModeCharDevice
case syscall.S_IFBLK:
mode |= fs.ModeDevice
}
if m&syscall.S_ISUID != 0 {
mode |= fs.ModeSetuid
}
if m&syscall.S_ISGID != 0 {
mode |= fs.ModeSetgid
}
if m&syscall.S_ISVTX != 0 {
mode |= fs.ModeSticky
}
return mode
}
internal/platform/open_other.go
-8
View File
@@ -1,8 +0,0 @@
//go:build !unix
package platform
import "os"
// OpenContent opens a file whose content is about to be read.
func OpenContent(path string) (*os.File, error) { return os.Open(path) }
internal/platform/open_unix.go
-15
View File
@@ -1,15 +0,0 @@
//go:build unix
package platform
import (
"os"
"syscall"
)
// OpenContent opens a file whose content is about to be read. It does not
// follow a final symlink, and O_NONBLOCK keeps a FIFO swapped in since the
// scan from blocking open(2) forever; callers still check the file type.
func OpenContent(path string) (*os.File, error) {
return os.OpenFile(path, os.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK, 0)
}
internal/platform/ops.go
-40
View File
@@ -1,40 +0,0 @@
package platform
import (
"errors"
"io/fs"
"os"
"os/exec"
"path/filepath"
)
// startDetached runs a helper program with an argument vector (never a
// shell), detached from the terminal so it cannot disturb the TUI.
func startDetached(name string, args ...string) error {
cmd := exec.Command(name, args...)
cmd.Stdin, cmd.Stdout, cmd.Stderr = nil, nil, nil
if err := cmd.Start(); err != nil {
return err
}
go cmd.Wait()
return nil
}
// Expect is what the scan recorded for an item about to be trashed. Trash
// refuses an item whose owner or file type no longer matches.
type Expect struct {
UID uint32
Mode fs.FileMode // only the type bits are compared
}
// checkAbs guards helper invocations: an absolute path can never be parsed
// as an option by the helper program.
func checkAbs(path string) error {
if !filepath.IsAbs(path) {
return errors.New("refusing relative path")
}
if _, err := os.Lstat(path); err != nil {
return err
}
return nil
}
internal/platform/ops_darwin.go
-81
View File
@@ -1,81 +0,0 @@
package platform
import (
"errors"
"fmt"
"os"
"path/filepath"
"syscall"
)
// TrashName is the user-facing name of the trash.
const TrashName = "Trash"
// Reveal selects the item in Finder.
func Reveal(path string) error {
if err := checkAbs(path); err != nil {
return err
}
return startDetached("open", "-R", path)
}
// Trash moves path into ~/.Trash (same volume) or the volume's
// .Trashes/<uid> directory. It never copies and never deletes.
func Trash(path string, want Expect) error {
if err := checkAbs(path); err != nil {
return err
}
home, err := os.UserHomeDir()
if err != nil {
return err
}
dirs := []string{filepath.Join(home, ".Trash")}
if vol := volumeOf(path); vol != "" {
dirs = append(dirs, filepath.Join(vol, ".Trashes", fmt.Sprint(os.Getuid())))
}
base := filepath.Base(path)
for vi, dir := range dirs {
if err := os.MkdirAll(filepath.Dir(dir), 0o700); err != nil {
continue
}
// A volume trash is on a shared filesystem: another user could plant
// a symlink or an open directory there, so insist on a private one.
if err := ensurePrivateDir(dir, vi > 0); err != nil {
continue
}
for i := 1; i < 10000; i++ {
name := base
if i > 1 {
name = fmt.Sprintf("%s %d", base, i)
}
dst := filepath.Join(dir, name)
if _, err := os.Lstat(dst); err == nil {
continue
}
err := renameChecked(path, dst, want)
if errors.Is(err, syscall.EXDEV) {
break // try the next trash location
}
return err
}
}
return errors.New("trash: item is on a volume without a usable trash")
}
func volumeOf(path string) string {
const prefix = "/Volumes/"
if len(path) > len(prefix) && path[:len(prefix)] == prefix {
rest := path[len(prefix):]
for i := 0; i < len(rest); i++ {
if rest[i] == '/' {
return prefix + rest[:i]
}
}
return path
}
return ""
}
// Headless is always false: macOS has Finder and the Trash even without
// $DISPLAY (which it does not use).
func Headless() bool { return false }
internal/platform/ops_linux.go
-132
View File
@@ -1,132 +0,0 @@
package platform
import (
"errors"
"fmt"
"net/url"
"os"
"path/filepath"
"strconv"
"syscall"
"time"
)
// TrashName is the user-facing name of the trash.
const TrashName = "trash"
// Reveal opens the item's directory in the desktop file manager.
func Reveal(path string) error {
if err := checkAbs(path); err != nil {
return err
}
fi, err := os.Lstat(path)
if err == nil && !fi.IsDir() {
path = filepath.Dir(path)
}
return startDetached("xdg-open", path)
}
// Trash moves path to the freedesktop.org trash: the home trash when on the
// same filesystem, otherwise $topdir/.Trash-$uid on the item's filesystem.
// It never copies across filesystems and never deletes.
func Trash(path string, want Expect) error {
if err := checkAbs(path); err != nil {
return err
}
st, err := os.Lstat(path)
if err != nil {
return err
}
dev := st.Sys().(*syscall.Stat_t).Dev
dataHome := os.Getenv("XDG_DATA_HOME")
if dataHome == "" {
home, err := os.UserHomeDir()
if err != nil {
return err
}
dataHome = filepath.Join(home, ".local", "share")
}
home := filepath.Join(dataHome, "Trash")
var trash, infoPath string
if hs, err := os.Stat(dataHome); err == nil && hs.Sys().(*syscall.Stat_t).Dev == dev {
trash, infoPath = home, path
} else {
top := mountTop(path, dev)
trash = filepath.Join(top, ".Trash-"+strconv.Itoa(os.Getuid()))
rel, err := filepath.Rel(top, path)
if err != nil {
return err
}
infoPath = rel
}
files, info := filepath.Join(trash, "files"), filepath.Join(trash, "info")
if trash == home {
if err := os.MkdirAll(trash, 0o700); err != nil {
return err
}
} else if err := ensurePrivateDir(trash, true); err != nil {
// On a shared filesystem another user could plant a symlink or a
// world-writable directory here; refuse rather than follow it.
return fmt.Errorf("trash directory %s is unsafe: %w", trash, err)
}
for _, d := range []string{files, info} {
if err := ensurePrivateDir(d, trash != home); err != nil {
return fmt.Errorf("trash directory %s is unsafe: %w", d, err)
}
}
base := filepath.Base(path)
for i := 1; i < 10000; i++ {
name := base
if i > 1 {
name = fmt.Sprintf("%s.%d", base, i)
}
dst := filepath.Join(files, name)
if _, err := os.Lstat(dst); err == nil {
continue // never overwrite an earlier trashed item (even without .trashinfo)
}
ip := filepath.Join(info, name+".trashinfo")
f, err := os.OpenFile(ip, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o600)
if errors.Is(err, os.ErrExist) {
continue
}
if err != nil {
return err
}
u := (&url.URL{Path: infoPath}).EscapedPath()
_, werr := fmt.Fprintf(f, "[Trash Info]\nPath=%s\nDeletionDate=%s\n", u, time.Now().Format("2006-01-02T15:04:05"))
cerr := f.Close()
if werr != nil || cerr != nil {
os.Remove(ip)
return errors.Join(werr, cerr)
}
if err := renameChecked(path, dst, want); err != nil {
os.Remove(ip)
return err
}
return nil
}
return errors.New("trash: too many items with the same name")
}
// mountTop returns the top directory of the filesystem containing path.
func mountTop(path string, dev uint64) string {
cur := filepath.Dir(path)
for {
parent := filepath.Dir(cur)
if parent == cur {
return cur
}
st, err := os.Stat(parent)
if err != nil || st.Sys().(*syscall.Stat_t).Dev != dev {
return cur
}
cur = parent
}
}
// Headless reports whether there is no graphical session (no X or Wayland
// display), so there is no file manager to open and no desktop trash anyone
// will empty.
func Headless() bool {
return os.Getenv("DISPLAY") == "" && os.Getenv("WAYLAND_DISPLAY") == ""
}
internal/platform/ops_linux_test.go
-155
View File
@@ -1,155 +0,0 @@
package platform
import (
"os"
"path/filepath"
"strings"
"testing"
)
func TestTrashFreedesktop(t *testing.T) {
data := t.TempDir()
t.Setenv("XDG_DATA_HOME", data)
dir := filepath.Join(data, "work")
os.Mkdir(dir, 0o755)
victim := filepath.Join(dir, "weird name ;rm -rf ~;")
if err := os.WriteFile(victim, []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
if err := Trash(victim, want(t, victim)); err != nil {
t.Fatal(err)
}
if _, err := os.Lstat(victim); !os.IsNotExist(err) {
t.Fatal("file still present")
}
trashed := filepath.Join(data, "Trash", "files", "weird name ;rm -rf ~;")
if _, err := os.Lstat(trashed); err != nil {
t.Fatalf("not in trash: %v", err)
}
info, err := os.ReadFile(filepath.Join(data, "Trash", "info", "weird name ;rm -rf ~;.trashinfo"))
if err != nil || !strings.Contains(string(info), "Path="+strings.ReplaceAll(dir, " ", "%20")) {
t.Fatalf("bad trashinfo: %q %v", info, err)
}
// A second item with the same name gets a distinct trash name.
os.WriteFile(victim, []byte("y"), 0o644)
if err := Trash(victim, want(t, victim)); err != nil {
t.Fatal(err)
}
if _, err := os.Lstat(trashed + ".2"); err != nil {
t.Fatal("collision not handled")
}
if err := Trash("relative/path", Expect{}); err == nil {
t.Fatal("relative paths must be refused")
}
}
func TestEnsurePrivateDirRejectsSymlink(t *testing.T) {
base := t.TempDir()
target := filepath.Join(base, "attacker")
os.Mkdir(target, 0o700)
link := filepath.Join(base, ".Trash-1000")
os.Symlink(target, link)
if err := ensurePrivateDir(link, false); err == nil {
t.Fatal("symlinked trash dir accepted")
}
open := filepath.Join(base, "open")
os.Mkdir(open, 0o777)
os.Chmod(open, 0o777)
if err := ensurePrivateDir(open, true); err == nil {
t.Fatal("world-accessible trash dir accepted")
}
if err := ensurePrivateDir(filepath.Join(base, "fresh"), true); err != nil {
t.Fatal(err)
}
}
func want(t *testing.T, path string) Expect {
t.Helper()
m, err := Lstat(path)
if err != nil {
t.Fatal(err)
}
return Expect{UID: m.UID, Mode: m.Mode}
}
// A parent swapped for a symlink after the scan must not redirect the move
// to whatever the symlink points at.
func TestTrashRefusesSwappedItem(t *testing.T) {
data := t.TempDir()
t.Setenv("XDG_DATA_HOME", data)
dir := filepath.Join(data, "work")
os.Mkdir(dir, 0o755)
victim := filepath.Join(dir, "big.iso")
os.WriteFile(victim, []byte("x"), 0o644)
scanned := want(t, victim)
// Item replaced by a symlink to a file elsewhere: type differs.
other := filepath.Join(data, "precious")
os.WriteFile(other, []byte("keep"), 0o644)
os.Remove(victim)
os.Symlink(other, victim)
scannedDir := Expect{UID: scanned.UID, Mode: os.ModeDir}
if err := Trash(victim, scanned); err == nil {
t.Fatal("symlink accepted where a regular file was scanned")
}
if err := Trash(victim, scannedDir); err == nil {
t.Fatal("type mismatch accepted")
}
if _, err := os.Lstat(victim); err != nil {
t.Fatal("refused item was moved")
}
// Owner differs from the scan.
os.Remove(victim)
os.WriteFile(victim, []byte("x"), 0o644)
if err := Trash(victim, Expect{UID: scanned.UID + 1, Mode: scanned.Mode}); err == nil {
t.Fatal("owner mismatch accepted")
}
entries, _ := os.ReadDir(filepath.Join(data, "Trash", "info"))
if len(entries) != 0 {
t.Fatalf("refused trash left %d .trashinfo files", len(entries))
}
if err := Trash(victim, scanned); err != nil {
t.Fatal(err)
}
}
func TestDeletePermanently(t *testing.T) {
base := t.TempDir()
dir := filepath.Join(base, "S01")
os.MkdirAll(filepath.Join(dir, "extras"), 0o755)
os.WriteFile(filepath.Join(dir, "extras", "E01.mkv"), []byte("x"), 0o644)
// A symlink inside must be removed, not followed.
keep := filepath.Join(base, "keep")
os.WriteFile(keep, []byte("k"), 0o644)
os.Symlink(keep, filepath.Join(dir, "link"))
w := want(t, dir)
if err := Delete(dir, Expect{UID: w.UID, Mode: 0}); err == nil {
t.Fatal("type mismatch accepted")
}
if err := Delete(dir, w); err != nil {
t.Fatal(err)
}
if _, err := os.Lstat(dir); !os.IsNotExist(err) {
t.Fatal("directory still present")
}
if _, err := os.Lstat(keep); err != nil {
t.Fatal("symlink target was deleted")
}
if err := Delete("relative/path", Expect{}); err == nil {
t.Fatal("relative paths must be refused")
}
}
func TestHeadless(t *testing.T) {
t.Setenv("DISPLAY", "")
t.Setenv("WAYLAND_DISPLAY", "")
if !Headless() {
t.Fatal("no display should be headless")
}
t.Setenv("WAYLAND_DISPLAY", "wayland-0")
if Headless() {
t.Fatal("wayland session reported headless")
}
}
internal/platform/ops_other.go
-24
View File
@@ -1,24 +0,0 @@
//go:build !linux && !darwin && !windows
package platform
import (
"errors"
"os"
)
// TrashName is the user-facing name of the trash.
const TrashName = "trash"
// Reveal is unsupported on this platform.
func Reveal(string) error { return errors.New("not supported on this platform") }
// Trash is unsupported on this platform.
func Trash(string, Expect) error { return errors.New("trash not supported on this platform") }
// Headless reports whether there is no graphical session (no X or Wayland
// display), so there is no file manager to open and no desktop trash anyone
// will empty.
func Headless() bool {
return os.Getenv("DISPLAY") == "" && os.Getenv("WAYLAND_DISPLAY") == ""
}
internal/platform/ops_windows.go
-78
View File
@@ -1,78 +0,0 @@
package platform
import (
"errors"
"strconv"
"syscall"
"unsafe"
)
// TrashName is the user-facing name of the trash.
const TrashName = "Recycle Bin"
// Reveal selects the item in Explorer.
func Reveal(path string) error {
if err := checkAbs(path); err != nil {
return err
}
return startDetached("explorer.exe", "/select,", path)
}
var (
shell32 = syscall.NewLazyDLL("shell32.dll")
procSHFileOpertn = shell32.NewProc("SHFileOperationW")
)
// shFileOpStruct mirrors SHFILEOPSTRUCTW (64-bit natural alignment).
type shFileOpStruct struct {
hwnd uintptr
wFunc uint32
pFrom *uint16
pTo *uint16
fFlags uint16
fAnyOperationsAborted int32
hNameMappings uintptr
lpszProgressTitle *uint16
}
const (
foDelete = 0x3
fofSilent = 0x4
fofNoConfirmation = 0x10
fofAllowUndo = 0x40
fofNoErrorUI = 0x400
fofNoConfirmMkdir = 0x200
)
// Trash moves path to the Recycle Bin via SHFileOperationW with
// FOF_ALLOWUNDO. It never deletes permanently. want is not checked here:
// Windows exposes no owner or inode identity through the scan.
func Trash(path string, _ Expect) error {
if err := checkAbs(path); err != nil {
return err
}
if unsafe.Sizeof(uintptr(0)) != 8 {
return errors.New("recycle bin support requires a 64-bit build")
}
u, err := syscall.UTF16FromString(path)
if err != nil {
return err
}
u = append(u, 0) // double NUL terminated list
op := shFileOpStruct{wFunc: foDelete, pFrom: &u[0],
fFlags: fofAllowUndo | fofNoConfirmation | fofSilent | fofNoErrorUI | fofNoConfirmMkdir}
r, _, _ := procSHFileOpertn.Call(uintptr(unsafe.Pointer(&op)))
if r != 0 {
return errors.New("SHFileOperation failed: code " + strconv.Itoa(int(r)))
}
if op.fAnyOperationsAborted != 0 {
return errors.New("operation aborted")
}
return nil
}
// Headless is always false: Windows always has Explorer and the Recycle Bin.
func Headless() bool { return false }
// Delete is not offered on Windows; items go to the Recycle Bin.
func Delete(string, Expect) error { return errors.New("permanent delete not supported on Windows") }
internal/platform/platform.go
-151
View File
@@ -1,151 +0,0 @@
// Package platform hides OS-specific filesystem metadata behind a small
// common API. Files are split by build tag; nothing else in the program
// switches on GOOS.
package platform
import (
"errors"
"io"
"io/fs"
"os"
"os/user"
"strconv"
"sync"
)
// Meta is the metadata of one directory entry.
type Meta struct {
Name string
Mode fs.FileMode
Size int64
Alloc int64
AllocKnown bool
MTime int64 // unix nanoseconds
BTime int64 // birth time, 0 if unknown
UID, GID uint32
Nlink uint32
Dev, Ino uint64
HasIno bool // Dev/Ino are meaningful
Err error // stat failed; only Name is valid
}
// ChunkFunc receives successive batches of entries. The slice is owned by
// the callee. Returning an error stops the read.
type ChunkFunc func([]Meta) error
// Lstat returns metadata for path without following a final symlink.
func Lstat(path string) (Meta, error) { return statPath(path, false) }
// Stat returns metadata for path, following symlinks.
func Stat(path string) (Meta, error) { return statPath(path, true) }
var (
nameMu sync.Mutex
userNames = map[uint32]string{}
groupNames = map[uint32]string{}
)
// UserName resolves a uid to a name (cached; falls back to the number).
func UserName(uid uint32) string {
return lookupName(userNames, uid, func(s string) (string, error) {
u, err := user.LookupId(s)
if err != nil {
return "", err
}
return u.Username, nil
})
}
// GroupName resolves a gid to a name (cached; falls back to the number).
func GroupName(gid uint32) string {
return lookupName(groupNames, gid, func(s string) (string, error) {
g, err := user.LookupGroupId(s)
if err != nil {
return "", err
}
return g.Name, nil
})
}
func lookupName(cache map[uint32]string, id uint32, look func(string) (string, error)) string {
nameMu.Lock()
defer nameMu.Unlock()
if n, ok := cache[id]; ok {
return n
}
s := strconv.FormatUint(uint64(id), 10)
if n, err := look(s); err == nil {
s = n
}
cache[id] = s
return s
}
// fromFileInfo fills the portable fields of Meta.
func fromFileInfo(fi fs.FileInfo) Meta {
return Meta{
Name: fi.Name(),
Mode: fi.Mode(),
Size: fi.Size(),
Alloc: fi.Size(),
MTime: fi.ModTime().UnixNano(),
Nlink: 1,
}
}
// readDirGeneric is the portable fallback: os.ReadDir in chunks plus Lstat
// per entry by path.
func readDirGeneric(path string, chunk int, fn ChunkFunc, stat func(string) (Meta, error)) error {
f, err := os.Open(path)
if err != nil {
return err
}
defer f.Close()
for {
des, err := f.ReadDir(chunk)
if len(des) > 0 {
out := make([]Meta, len(des))
for i, de := range des {
m, serr := stat(joinPath(path, de.Name()))
if serr != nil {
m = Meta{Err: serr}
}
m.Name = de.Name()
out[i] = m
}
if ferr := fn(out); ferr != nil {
return ferr
}
}
if err != nil {
if errors.Is(err, io.EOF) {
return nil
}
return err
}
}
}
func joinPath(dir, name string) string {
if len(dir) > 0 && os.IsPathSeparator(dir[len(dir)-1]) {
return dir + name
}
return dir + string(os.PathSeparator) + name
}
// JoinPath joins a directory and an entry name without cleaning.
func JoinPath(dir, name string) string { return joinPath(dir, name) }
func baseName(path string) string {
i := len(path) - 1
for i > 0 && os.IsPathSeparator(path[i]) {
i--
}
path = path[:i+1]
for j := len(path) - 1; j >= 0; j-- {
if os.IsPathSeparator(path[j]) {
return path[j+1:]
}
}
return path
}
internal/platform/readdir_bsd.go
-82
View File
@@ -1,82 +0,0 @@
//go:build darwin || freebsd
package platform
import (
"errors"
"io"
"os"
"golang.org/x/sys/unix"
)
// ReadDir streams the entries of a directory in chunks, stat-ing each entry
// relative to the directory fd.
func ReadDir(path string, chunk int, fn ChunkFunc) error {
fd, err := unix.Open(path, unix.O_RDONLY|unix.O_DIRECTORY|unix.O_CLOEXEC, 0)
if err != nil {
return &os.PathError{Op: "open", Path: path, Err: err}
}
f := os.NewFile(uintptr(fd), path)
defer f.Close()
for {
des, err := f.ReadDir(chunk)
if len(des) > 0 {
out := make([]Meta, len(des))
for i, de := range des {
var st unix.Stat_t
if serr := unix.Fstatat(fd, de.Name(), &st, unix.AT_SYMLINK_NOFOLLOW); serr != nil {
out[i] = Meta{Name: de.Name(), Err: &os.PathError{Op: "stat", Path: de.Name(), Err: serr}}
continue
}
out[i] = fromStat(&st)
out[i].Name = de.Name()
}
if ferr := fn(out); ferr != nil {
return ferr
}
}
if err != nil {
if errors.Is(err, io.EOF) {
return nil
}
return err
}
}
}
func fromStat(st *unix.Stat_t) Meta {
return Meta{
Mode: modeFromUnix(uint32(st.Mode)),
Size: st.Size,
Alloc: st.Blocks * 512,
AllocKnown: true,
MTime: st.Mtim.Nano(),
BTime: st.Btim.Nano(),
UID: st.Uid,
GID: st.Gid,
Nlink: uint32(st.Nlink),
Dev: uint64(st.Dev),
Ino: uint64(st.Ino),
HasIno: true,
}
}
func statPath(path string, follow bool) (Meta, error) {
var st unix.Stat_t
var err error
if follow {
err = unix.Stat(path, &st)
} else {
err = unix.Lstat(path, &st)
}
if err != nil {
return Meta{}, &os.PathError{Op: "stat", Path: path, Err: err}
}
m := fromStat(&st)
m.Name = baseName(path)
return m, nil
}
// IsVirtualFS is Linux-specific; elsewhere nothing is skipped.
func IsVirtualFS(string) bool { return false }
internal/platform/readdir_linux.go
-138
View File
@@ -1,138 +0,0 @@
package platform
import (
"errors"
"io"
"os"
"sync/atomic"
"golang.org/x/sys/unix"
)
// noStatx is set if the kernel lacks statx (pre-4.11); we fall back to fstatat.
var noStatx atomic.Bool
const statxMask = unix.STATX_BASIC_STATS | unix.STATX_BTIME
// ReadDir streams the entries of a directory in chunks, stat-ing each entry
// relative to the directory fd (no repeated path resolution).
func ReadDir(path string, chunk int, fn ChunkFunc) error {
fd, err := unix.Open(path, unix.O_RDONLY|unix.O_DIRECTORY|unix.O_CLOEXEC, 0)
if err != nil {
return &os.PathError{Op: "open", Path: path, Err: err}
}
f := os.NewFile(uintptr(fd), path)
defer f.Close()
for {
des, err := f.ReadDir(chunk)
if len(des) > 0 {
out := make([]Meta, len(des))
for i, de := range des {
m, serr := statAt(fd, de.Name(), unix.AT_SYMLINK_NOFOLLOW)
if serr != nil {
m = Meta{Err: &os.PathError{Op: "stat", Path: de.Name(), Err: serr}}
}
m.Name = de.Name()
out[i] = m
}
if ferr := fn(out); ferr != nil {
return ferr
}
}
if err != nil {
if errors.Is(err, io.EOF) {
return nil
}
return err
}
}
}
func statAt(dirfd int, name string, flags int) (Meta, error) {
if !noStatx.Load() {
var st unix.Statx_t
err := unix.Statx(dirfd, name, flags|unix.AT_STATX_DONT_SYNC, statxMask, &st)
if err == nil {
return fromStatx(&st), nil
}
if !errors.Is(err, unix.ENOSYS) {
return Meta{}, err
}
noStatx.Store(true)
}
var st unix.Stat_t
if err := unix.Fstatat(dirfd, name, &st, flags); err != nil {
return Meta{}, err
}
return Meta{
Mode: modeFromUnix(st.Mode), Size: st.Size, Alloc: st.Blocks * 512, AllocKnown: true,
MTime: st.Mtim.Nano(), UID: st.Uid, GID: st.Gid, Nlink: uint32(st.Nlink),
Dev: st.Dev, Ino: st.Ino, HasIno: true,
}, nil
}
func fromStatx(st *unix.Statx_t) Meta {
m := Meta{
Mode: modeFromUnix(uint32(st.Mode)),
Size: int64(st.Size),
Alloc: int64(st.Blocks) * 512,
AllocKnown: true,
MTime: st.Mtime.Sec*1e9 + int64(st.Mtime.Nsec),
UID: st.Uid,
GID: st.Gid,
Nlink: st.Nlink,
Dev: unix.Mkdev(st.Dev_major, st.Dev_minor),
Ino: st.Ino,
HasIno: true,
}
if st.Mask&unix.STATX_BTIME != 0 {
m.BTime = st.Btime.Sec*1e9 + int64(st.Btime.Nsec)
}
return m
}
func statPath(path string, follow bool) (Meta, error) {
flags := unix.AT_SYMLINK_NOFOLLOW
if follow {
flags = 0
}
m, err := statAt(unix.AT_FDCWD, path, flags)
if err != nil {
return Meta{}, &os.PathError{Op: "stat", Path: path, Err: err}
}
m.Name = baseName(path)
return m, nil
}
// Linux pseudo-filesystems whose contents are not disk usage.
var virtualFS = map[int64]bool{
0x9fa0: true, // proc
0x62656572: true, // sysfs
0x1cd1: true, // devpts
0x27e0eb: true, // cgroup
0x63677270: true, // cgroup2
0x64626720: true, // debugfs
0x74726163: true, // tracefs
0x73636673: true, // securityfs
0x6165676c: true, // pstore
0xcafe4a11: true, // bpf
0x62656570: true, // configfs
0x65735543: true, // fusectl
0x19800202: true, // mqueue
0xde5e81e4: true, // efivarfs
0x42494e4d: true, // binfmt_misc
0xf97cff8c: true, // selinuxfs
0x958458f6: true, // hugetlbfs
0x6e736673: true, // nsfs
0x50495045: true, // pipefs
}
// IsVirtualFS reports whether path is the root of a pseudo filesystem
// (proc, sysfs, cgroup, …) that should not be scanned as disk usage.
func IsVirtualFS(path string) bool {
var st unix.Statfs_t
if err := unix.Statfs(path, &st); err != nil {
return false
}
return virtualFS[int64(st.Type)]
}
internal/platform/readdir_other.go
-31
View File
@@ -1,31 +0,0 @@
//go:build !linux && !darwin && !freebsd
package platform
import "os"
// ReadDir streams the entries of a directory using the portable os API.
// Allocated size, inode identity and link counts are not available here.
func ReadDir(path string, chunk int, fn ChunkFunc) error {
return readDirGeneric(path, chunk, fn, func(p string) (Meta, error) { return statPath(p, false) })
}
func statPath(path string, follow bool) (Meta, error) {
var fi os.FileInfo
var err error
if follow {
fi, err = os.Stat(path)
} else {
fi, err = os.Lstat(path)
}
if err != nil {
return Meta{}, err
}
m := fromFileInfo(fi)
m.Name = baseName(path)
fillSys(&m, fi)
return m, nil
}
// IsVirtualFS is Linux-specific; elsewhere nothing is skipped.
func IsVirtualFS(string) bool { return false }
internal/platform/sys_nowindows.go
-7
View File
@@ -1,7 +0,0 @@
//go:build !windows && !linux && !darwin && !freebsd
package platform
import "os"
func fillSys(*Meta, os.FileInfo) {}
internal/platform/sys_windows.go
-12
View File
@@ -1,12 +0,0 @@
package platform
import (
"os"
"syscall"
)
func fillSys(m *Meta, fi os.FileInfo) {
if d, ok := fi.Sys().(*syscall.Win32FileAttributeData); ok {
m.BTime = d.CreationTime.Nanoseconds()
}
}
internal/platform/trash_unix.go
-59
View File
@@ -1,59 +0,0 @@
//go:build linux || darwin
package platform
import (
"errors"
"io/fs"
"os"
"path/filepath"
"syscall"
"golang.org/x/sys/unix"
)
// renameChecked moves path to dst if it still matches want. The parent
// directory is opened once and both the check and the rename are relative to
// that descriptor, so a parent swapped for a symlink since the scan (or
// between check and rename) cannot redirect the move to another user's file.
func renameChecked(path, dst string, want Expect) error {
dir, base := filepath.Dir(path), filepath.Base(path)
dfd, err := unix.Open(dir, unix.O_RDONLY|unix.O_DIRECTORY|unix.O_CLOEXEC, 0)
if err != nil {
return &os.PathError{Op: "open", Path: dir, Err: err}
}
defer unix.Close(dfd)
var st unix.Stat_t
if err := unix.Fstatat(dfd, base, &st, unix.AT_SYMLINK_NOFOLLOW); err != nil {
return &os.PathError{Op: "stat", Path: path, Err: err}
}
if st.Uid != want.UID || modeFromUnix(uint32(st.Mode))&fs.ModeType != want.Mode&fs.ModeType {
return errors.New("item changed since the scan (owner or type differs); rescan first")
}
return unix.Renameat(dfd, base, unix.AT_FDCWD, dst)
}
// ensurePrivateDir creates dir (mode 0700) if missing; an existing entry must
// be a real directory (not a symlink) owned by us. With strict (trash on a
// shared filesystem) it must also be inaccessible to others, as the
// freedesktop.org spec requires; home trash directories made by desktop
// environments are sometimes 0755, which is harmless there.
func ensurePrivateDir(dir string, strict bool) error {
if err := os.Mkdir(dir, 0o700); err != nil && !errors.Is(err, os.ErrExist) {
return err
}
fi, err := os.Lstat(dir)
if err != nil {
return err
}
st := fi.Sys().(*syscall.Stat_t)
switch {
case !fi.IsDir():
return errors.New("not a directory (symlink?)")
case int(st.Uid) != os.Getuid():
return errors.New("owned by another user")
case strict && fi.Mode().Perm()&0o077 != 0:
return errors.New("accessible to other users")
}
return nil
}
internal/scan/mem_test.go
-35
View File
@@ -1,35 +0,0 @@
package scan
import (
"context"
"os"
"runtime"
"testing"
"time"
)
// TestMemoryPerNode reports heap use per inventory node for a real tree.
// Set MAPSIZE_MEM_ROOT to a directory to run it.
func TestMemoryPerNode(t *testing.T) {
root := os.Getenv("MAPSIZE_MEM_ROOT")
if root == "" {
t.Skip("set MAPSIZE_MEM_ROOT")
}
var before, after runtime.MemStats
runtime.GC()
runtime.ReadMemStats(&before)
start := time.Now()
s, err := Start(context.Background(), root, Options{})
if err != nil {
t.Fatal(err)
}
<-s.Done()
el := time.Since(start)
runtime.GC()
runtime.ReadMemStats(&after)
n := s.Tree.Len()
heap := after.HeapAlloc - before.HeapAlloc
t.Logf("nodes=%d scan=%s heap=%.1f MiB (%.0f bytes/node)", n, el.Round(time.Millisecond),
float64(heap)/(1<<20), float64(heap)/float64(n))
runtime.KeepAlive(s)
}
internal/scan/panic.go
-14
View File
@@ -1,14 +0,0 @@
package scan
import "runtime/debug"
// OnPanic is called if a scanner goroutine panics. The default re-panics
// (crashing the process); the TUI replaces it so it can restore the terminal
// before reporting. Programmer bugs are never silently swallowed.
var OnPanic = func(v any, stack []byte) { panic(v) }
func guard() {
if v := recover(); v != nil {
OnPanic(v, debug.Stack())
}
}
internal/scan/scan.go
-542
View File
@@ -1,542 +0,0 @@
// Package scan implements the bounded concurrent filesystem scanner.
//
// Architecture (see ARCHITECTURE.md): a single controller goroutine owns the
// inventory.Tree and a FIFO of pending directories. A fixed pool of workers
// performs directory I/O and sends result chunks back. The controller's
// select loop offers jobs and accepts results at the same time, so bounded
// channels can never deadlock: workers never enqueue work themselves.
package scan
import (
"context"
"errors"
"fmt"
"log/slog"
"os"
"path/filepath"
"runtime"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/platform"
)
// FollowMode controls symlink traversal.
type FollowMode int
const (
FollowNone FollowMode = iota
FollowSameFS
FollowAll
)
func (f FollowMode) String() string {
return [...]string{"none", "same-filesystem", "all"}[f]
}
// ParseFollow parses a --follow-symlinks value.
func ParseFollow(s string) (FollowMode, error) {
switch s {
case "none", "":
return FollowNone, nil
case "same-filesystem", "same-fs":
return FollowSameFS, nil
case "all":
return FollowAll, nil
}
return 0, fmt.Errorf("invalid --follow-symlinks %q (want none, same-filesystem or all)", s)
}
// Workers resolves a concurrency mode name or number to a worker count.
func Workers(mode string) (int, error) {
cpu := runtime.NumCPU()
switch mode {
case "conservative":
return 2, nil
case "balanced", "":
return min(8, max(2, 2*cpu)), nil
case "aggressive":
return min(64, max(4, 8*cpu)), nil
}
var n int
if _, err := fmt.Sscan(mode, &n); err != nil || n < 1 || n > 1024 {
return 0, fmt.Errorf("invalid workers %q (conservative, balanced, aggressive or 1-1024)", mode)
}
return n, nil
}
// Options configures a scan.
type Options struct {
Workers int
Follow FollowMode
OneFileSystem bool
Excludes []string
// Tuning knobs, mostly for tests. Zero means default.
ChunkSize int // entries per result chunk
JobBuffer int // capacity of the job channel
ResultBuf int // capacity of the result channel
ApplyBatch int // max results applied per write-lock hold
}
func (o *Options) defaults() {
if o.Workers <= 0 {
o.Workers, _ = Workers("balanced")
}
if o.ChunkSize <= 0 {
o.ChunkSize = 1024
}
if o.JobBuffer <= 0 {
o.JobBuffer = o.Workers * 2
}
if o.ResultBuf <= 0 {
o.ResultBuf = o.Workers * 4
}
if o.ApplyBatch <= 0 {
o.ApplyBatch = 64
}
}
// Progress is a cheap, lock-free view of scan activity.
type Progress struct {
Entries int64
Pending int64 // directories discovered but not yet dispatched
Inflight int64 // directories being read
Workers int
}
// Scanner runs one scan. Tree is safe to read under Tree.RLock while the
// scan runs.
type Scanner struct {
Tree *inventory.Tree
opts Options
entries, pending, inflight atomic.Int64
done chan struct{}
}
type job struct {
id inventory.NodeID
path string
dev uint64
// hints derived once per directory
inCache, inLog bool
}
type result struct {
job *job
entries []platform.Meta
final bool
err error
}
type inodeKey struct{ dev, ino uint64 }
// Start stats root, creates the tree and begins scanning in the background.
func Start(ctx context.Context, root string, opts Options) (*Scanner, error) {
opts.defaults()
// Path-style exclusions are matched against absolute paths.
excl := make([]string, len(opts.Excludes))
for i, p := range opts.Excludes {
excl[i] = p
if strings.ContainsAny(p, `/\`) {
if a, err := filepath.Abs(p); err == nil {
excl[i] = a
}
}
}
opts.Excludes = excl
abs, err := filepath.Abs(root)
if err != nil {
return nil, err
}
m, err := platform.Stat(abs)
if err != nil {
return nil, err
}
kind := kindOf(m)
t := inventory.New(abs, kind)
t.Stats.Workers = opts.Workers
t.Stats.Excludes = opts.Excludes
t.Stats.OneFileSystem = opts.OneFileSystem
t.Stats.Follow = opts.Follow.String()
r := t.Node(t.Root())
setMeta(r, m)
r.TotSize, r.TotAlloc = m.Size, m.Alloc
s := &Scanner{Tree: t, opts: opts, done: make(chan struct{})}
if kind != inventory.KindDir {
t.Stats.Files = 1
t.Stats.Complete = true
t.Stats.End = time.Now()
close(s.done)
return s, nil
}
t.Stats.Dirs = 1
c := &controller{s: s, t: t, opts: opts, links: map[inodeKey]struct{}{}}
// Loop detection needs inode identity; without it (Windows and other
// portable-fallback platforms) symlinks are not followed at all.
if opts.Follow != FollowNone && !m.HasIno {
c.opts.Follow = FollowNone
t.Stats.Follow = "none (no inode identity on this platform)"
}
if c.opts.Follow != FollowNone {
c.visited = map[inodeKey]struct{}{{m.Dev, m.Ino}: {}}
}
c.rootDev = m.Dev
c.pending = []pendingDir{{id: t.Root(), dev: m.Dev}}
s.pending.Store(1)
go c.run(ctx)
return s, nil
}
// Done is closed when the scan has finished or been cancelled and drained.
func (s *Scanner) Done() <-chan struct{} { return s.done }
// Progress returns live counters without locking the tree.
func (s *Scanner) Progress() Progress {
return Progress{Entries: s.entries.Load(), Pending: s.pending.Load(), Inflight: s.inflight.Load(), Workers: s.opts.Workers}
}
// maxEntriesPerLock bounds how long the controller holds the write lock, so
// the UI's read lock is never starved for more than a few milliseconds.
const maxEntriesPerLock = 8192
type pendingDir struct {
id inventory.NodeID
dev uint64
}
type controller struct {
s *Scanner
t *inventory.Tree
opts Options
rootDev uint64
pending []pendingDir // FIFO; head index advances, compacted occasionally
head int
links map[inodeKey]struct{}
visited map[inodeKey]struct{} // only when following symlinks
}
func (c *controller) run(ctx context.Context) {
defer guard()
defer close(c.s.done)
jobs := make(chan *job, c.opts.JobBuffer)
results := make(chan result, c.opts.ResultBuf)
var wg sync.WaitGroup
for range c.opts.Workers {
wg.Add(1)
go func() {
defer wg.Done()
defer guard()
worker(ctx, jobs, results, c.opts.ChunkSize)
}()
}
inflight := 0
ctxDone := ctx.Done()
cancelled := false
var next *job
batch := make([]result, 0, c.opts.ApplyBatch)
for {
if next == nil && !cancelled && c.head < len(c.pending) {
next = c.makeJob(c.pending[c.head])
}
if inflight == 0 && next == nil {
break
}
var jobCh chan<- *job
if next != nil {
jobCh = jobs
}
select {
case jobCh <- next:
next = nil
c.head++
if c.head > 4096 && c.head*2 > len(c.pending) {
c.pending = append(c.pending[:0], c.pending[c.head:]...)
c.head = 0
}
inflight++
c.s.inflight.Store(int64(inflight))
c.s.pending.Store(int64(len(c.pending) - c.head))
case r := <-results:
batch = append(batch[:0], r)
n := len(r.entries)
drain:
for len(batch) < cap(batch) && n < maxEntriesPerLock {
select {
case r := <-results:
batch = append(batch, r)
n += len(r.entries)
default:
break drain
}
}
c.t.Lock()
for i := range batch {
if c.apply(&batch[i]) {
inflight--
}
batch[i] = result{}
}
c.t.Unlock()
c.s.inflight.Store(int64(inflight))
c.s.pending.Store(int64(len(c.pending) - c.head))
case <-ctxDone:
cancelled = true
ctxDone = nil
next = nil
}
}
close(jobs)
wg.Wait()
c.t.Lock()
st := &c.t.Stats
if cancelled {
st.Cancelled = true
st.Unscanned += int64(len(c.pending) - c.head)
for _, p := range c.pending[c.head:] {
c.t.Node(p.id).Flags |= inventory.FlagIncomplete
}
}
st.Complete = true
st.End = time.Now()
c.t.Unlock()
c.s.pending.Store(0)
c.s.inflight.Store(0)
}
func (c *controller) makeJob(p pendingDir) *job {
path := c.t.Path(p.id)
j := &job{id: p.id, path: path, dev: p.dev}
j.inCache, j.inLog = inventory.PathHints(path)
return j
}
func worker(ctx context.Context, jobs <-chan *job, results chan<- result, chunk int) {
for j := range jobs {
var err error
if err = ctx.Err(); err == nil {
err = platform.ReadDir(j.path, chunk, func(ms []platform.Meta) error {
if err := ctx.Err(); err != nil {
return err
}
results <- result{job: j, entries: ms}
return nil
})
}
results <- result{job: j, final: true, err: err}
}
}
func kindOf(m platform.Meta) inventory.Kind {
switch {
case m.Mode.IsDir():
return inventory.KindDir
case m.Mode.IsRegular():
return inventory.KindFile
case m.Mode&os.ModeSymlink != 0:
return inventory.KindSymlink
}
return inventory.KindOther
}
func setMeta(n *inventory.Node, m platform.Meta) {
n.Size, n.Alloc = m.Size, m.Alloc
n.MTime = m.MTime
n.Mode = uint32(m.Mode)
n.UID, n.GID = m.UID, m.GID
n.Nlink = m.Nlink
if !m.AllocKnown {
n.Flags |= inventory.FlagAllocUnknown
}
}
// apply merges one result into the tree. It reports whether the result
// completed its directory. Called with the tree write-locked.
func (c *controller) apply(r *result) bool {
t := c.t
j := r.job
dir := j.id
var d inventory.Delta
c.s.entries.Add(int64(len(r.entries)))
for _, m := range r.entries {
c.addEntry(j, m, &d)
}
if r.final {
dn := t.Node(dir)
switch {
case r.err == nil:
dn.Flags |= inventory.FlagScanned
case errors.Is(r.err, context.Canceled) || errors.Is(r.err, context.DeadlineExceeded):
dn.Flags |= inventory.FlagIncomplete
t.Stats.Unscanned++
default:
slog.Debug("read directory failed", "path", j.path, "err", r.err)
dn.Flags |= inventory.FlagError | inventory.FlagIncomplete
t.Stats.AddError(inventory.ErrorRecord{Node: dir, Kind: inventory.Classify(r.err), Msg: errMsg(r.err)})
d.Errors++
}
}
if d != (inventory.Delta{}) {
t.Propagate(dir, d)
}
return r.final
}
func errMsg(err error) string {
var pe *os.PathError
if errors.As(err, &pe) {
return pe.Op + ": " + pe.Err.Error()
}
return err.Error()
}
func (c *controller) excluded(path, name string) bool {
for _, p := range c.opts.Excludes {
if strings.ContainsAny(p, `/\`) {
pp := filepath.Clean(p)
if path == pp || strings.HasPrefix(path, pp+string(os.PathSeparator)) {
return true
}
if ok, _ := filepath.Match(pp, path); ok {
return true
}
} else if ok, _ := filepath.Match(p, name); ok {
return true
}
}
return false
}
func (c *controller) addEntry(j *job, m platform.Meta, d *inventory.Delta) {
t := c.t
st := &t.Stats
if m.Err != nil {
k := inventory.Classify(m.Err)
if k == inventory.ErrVanished {
st.AddError(inventory.ErrorRecord{Node: j.id, Name: m.Name, Kind: k, Msg: "vanished between listing and stat"})
d.Errors++
return
}
id := t.Add(j.id, m.Name, inventory.KindOther)
n := t.Node(id)
n.Flags |= inventory.FlagError
n.Errors = 1
st.AddError(inventory.ErrorRecord{Node: id, Kind: k, Msg: errMsg(m.Err)})
d.Errors++
d.Files++
st.Others++
return
}
var path string
if len(c.opts.Excludes) > 0 {
path = platform.JoinPath(j.path, m.Name)
if c.excluded(path, m.Name) {
st.Excluded++
return
}
}
kind := kindOf(m)
var target platform.Meta
followed, broken := false, false
if kind == inventory.KindSymlink {
if path == "" {
path = platform.JoinPath(j.path, m.Name)
}
tm, err := platform.Stat(path)
switch {
case err != nil:
st.BrokenLinks++
broken = true
case tm.Mode.IsDir() && tm.HasIno && (c.opts.Follow == FollowAll || c.opts.Follow == FollowSameFS && tm.Dev == j.dev):
target, followed = tm, true
kind = inventory.KindDir
}
}
id := t.Add(j.id, m.Name, kind)
n := t.Node(id)
setMeta(n, m)
if broken {
n.Flags |= inventory.FlagBrokenLink
}
switch kind {
case inventory.KindDir:
st.Dirs++
d.Dirs++
d.Size += n.Size
d.Alloc += n.Alloc
n.TotSize, n.TotAlloc = n.Size, n.Alloc
dev := m.Dev
if followed {
n.Flags |= inventory.FlagFollowed
dev = target.Dev
}
if c.visited != nil {
key := inodeKey{m.Dev, m.Ino}
if followed {
key = inodeKey{target.Dev, target.Ino}
}
if _, seen := c.visited[key]; seen {
n.Flags |= inventory.FlagLoop
st.LoopsSkipped++
return
}
c.visited[key] = struct{}{}
}
if m.HasIno && dev != j.dev {
n.Flags |= inventory.FlagMountPoint
if c.opts.OneFileSystem {
n.Flags |= inventory.FlagSkippedFS
st.SkippedMounts++
return
}
if path == "" {
path = platform.JoinPath(j.path, m.Name)
}
if platform.IsVirtualFS(path) {
n.Flags |= inventory.FlagVirtualFS
st.VirtualSkipped++
return
}
}
c.pending = append(c.pending, pendingDir{id: id, dev: dev})
case inventory.KindFile:
st.Files++
d.Files++
if m.Nlink > 1 && m.HasIno {
n.Flags |= inventory.FlagHardlinked
key := inodeKey{m.Dev, m.Ino}
if _, seen := c.links[key]; seen {
n.Flags |= inventory.FlagHardlinkDup
st.HardlinkDups++
return
}
c.links[key] = struct{}{}
}
if m.AllocKnown && m.Alloc < m.Size && m.Size-m.Alloc >= 4096 {
n.Flags |= inventory.FlagSparse
}
n.Cat = inventory.CategoryFor(t.ExtName(n), j.inCache, j.inLog)
n.TotSize, n.TotAlloc = n.Size, n.Alloc
d.Size += n.Size
d.Alloc += n.Alloc
d.Cat[n.Cat] += n.Alloc
default:
if kind == inventory.KindSymlink {
st.Symlinks++
} else {
st.Others++
}
d.Files++
n.TotSize, n.TotAlloc = n.Size, n.Alloc
d.Size += n.Size
d.Alloc += n.Alloc
}
}
internal/scan/scan_test.go
-288
View File
@@ -1,288 +0,0 @@
package scan
import (
"context"
"fmt"
"os"
"path/filepath"
"runtime"
"testing"
"time"
"github.com/andydixon/mapsize/internal/inventory"
)
// mkTree creates dirs×files under root: root/d0/d1/... fan-out tree.
func mkTree(t testing.TB, root string, fanout, depth, files int) (nfiles, ndirs int64, bytes int64) {
var rec func(dir string, d int)
rec = func(dir string, d int) {
for i := range files {
sz := (i + 1) * 100
if err := os.WriteFile(filepath.Join(dir, fmt.Sprintf("f%d.txt", i)), make([]byte, sz), 0o644); err != nil {
t.Fatal(err)
}
nfiles++
bytes += int64(sz)
}
if d == depth {
return
}
for i := range fanout {
sub := filepath.Join(dir, fmt.Sprintf("d%d", i))
if err := os.Mkdir(sub, 0o755); err != nil {
t.Fatal(err)
}
ndirs++
rec(sub, d+1)
}
}
rec(root, 0)
return
}
func runScan(t testing.TB, root string, o Options) *inventory.Tree {
s, err := Start(context.Background(), root, o)
if err != nil {
t.Fatal(err)
}
select {
case <-s.Done():
case <-time.After(60 * time.Second):
t.Fatal("scan did not finish (deadlock?)")
}
return s.Tree
}
func TestScanCountsAndAggregates(t *testing.T) {
root := t.TempDir()
nf, nd, bytes := mkTree(t, root, 3, 3, 4)
tr := runScan(t, root, Options{Workers: 4})
r := tr.Node(tr.Root())
if int64(r.Files) != nf || int64(r.Dirs) != nd {
t.Fatalf("files=%d dirs=%d, want %d %d", r.Files, r.Dirs, nf, nd)
}
// Logical total = file bytes + directory entries' own sizes.
var dirBytes int64
for i := 0; i < tr.Len(); i++ {
if n := tr.Node(inventory.NodeID(i)); n.IsDir() {
dirBytes += n.Size
}
}
if r.TotSize != bytes+dirBytes {
t.Fatalf("TotSize=%d want %d", r.TotSize, bytes+dirBytes)
}
if !tr.Stats.Complete || tr.Stats.Incomplete() {
t.Fatalf("stats: %+v", tr.Stats)
}
// Every directory's aggregate equals the sum of its children.
checkAggregates(t, tr)
// Incremental aggregation must match a full recompute.
before := snapshotTotals(tr)
tr.Recompute()
if after := snapshotTotals(tr); fmt.Sprint(before) != fmt.Sprint(after) {
t.Fatal("incremental aggregates differ from recompute")
}
}
func snapshotTotals(tr *inventory.Tree) [][4]int64 {
out := make([][4]int64, tr.Len())
for i := range out {
n := tr.Node(inventory.NodeID(i))
out[i] = [4]int64{n.TotSize, n.TotAlloc, int64(n.Files), int64(n.Dirs)}
}
return out
}
func checkAggregates(t *testing.T, tr *inventory.Tree) {
t.Helper()
for i := 0; i < tr.Len(); i++ {
id := inventory.NodeID(i)
n := tr.Node(id)
if !n.IsDir() {
continue
}
sum := n.Size
tr.Children(id, func(_ inventory.NodeID, c *inventory.Node) { sum += c.TotSize })
if sum != n.TotSize {
t.Fatalf("%s: TotSize %d != own+children %d", tr.Path(id), n.TotSize, sum)
}
}
}
// TestBackpressureNoDeadlock forces maximum contention: one worker, channel
// capacities of one, tiny chunks, and thousands of directories.
func TestBackpressureNoDeadlock(t *testing.T) {
root := t.TempDir()
nf, nd, _ := mkTree(t, root, 6, 4, 3) // 1554 dirs
for _, w := range []int{1, 2, 16} {
tr := runScan(t, root, Options{Workers: w, JobBuffer: 1, ResultBuf: 1, ChunkSize: 1, ApplyBatch: 1})
r := tr.Node(tr.Root())
if int64(r.Files) != nf || int64(r.Dirs) != nd {
t.Fatalf("workers=%d: files=%d dirs=%d want %d %d", w, r.Files, r.Dirs, nf, nd)
}
}
}
func TestCancel(t *testing.T) {
root := t.TempDir()
mkTree(t, root, 8, 3, 2)
ctx, cancel := context.WithCancel(context.Background())
s, err := Start(ctx, root, Options{Workers: 2, JobBuffer: 1, ResultBuf: 1, ChunkSize: 1})
if err != nil {
t.Fatal(err)
}
cancel()
select {
case <-s.Done():
case <-time.After(10 * time.Second):
t.Fatal("cancel did not stop scan")
}
tr := s.Tree
tr.RLock()
defer tr.RUnlock()
if !tr.Stats.Cancelled || !tr.Stats.Incomplete() {
t.Fatalf("expected cancelled+incomplete: %+v", tr.Stats)
}
// Workers must have exited: give the runtime a moment and check goroutines.
time.Sleep(50 * time.Millisecond)
if n := runtime.NumGoroutine(); n > 10 {
t.Logf("goroutines after cancel: %d", n)
}
}
func TestSymlinkLoops(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("symlinks need privileges on Windows")
}
root := t.TempDir()
a := filepath.Join(root, "a")
os.MkdirAll(filepath.Join(a, "b"), 0o755)
os.WriteFile(filepath.Join(a, "b", "x"), make([]byte, 5000), 0o644)
os.Symlink("..", filepath.Join(a, "b", "up")) // loop
os.Symlink("../a", filepath.Join(a, "self")) // loop
os.Symlink("nowhere", filepath.Join(root, "broken"))
for _, f := range []FollowMode{FollowNone, FollowSameFS, FollowAll} {
tr := runScan(t, root, Options{Workers: 4, Follow: f})
r := tr.Node(tr.Root())
if r.Files == 0 || r.TotSize <= 5000 {
t.Fatalf("follow=%v: bad totals %+v", f, r)
}
if tr.Stats.BrokenLinks != 1 {
t.Fatalf("follow=%v: broken links %d", f, tr.Stats.BrokenLinks)
}
if f != FollowNone && tr.Stats.LoopsSkipped == 0 {
t.Fatalf("follow=%v: expected loops to be detected", f)
}
// The 5000-byte file must be counted exactly once.
var count int
for i := 0; i < tr.Len(); i++ {
if n := tr.Node(inventory.NodeID(i)); n.Name == "x" {
count++
}
}
if count != 1 {
t.Fatalf("follow=%v: file x seen %d times", f, count)
}
}
}
func TestHardlinksCountedOnce(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("hard-link identity not available on Windows")
}
root := t.TempDir()
p := filepath.Join(root, "orig")
os.WriteFile(p, make([]byte, 100000), 0o644)
os.Mkdir(filepath.Join(root, "d"), 0o755)
if err := os.Link(p, filepath.Join(root, "d", "link")); err != nil {
t.Skip(err)
}
tr := runScan(t, root, Options{Workers: 2})
r := tr.Node(tr.Root())
if tr.Stats.HardlinkDups != 1 {
t.Fatalf("dups=%d", tr.Stats.HardlinkDups)
}
if r.TotSize >= 200000 {
t.Fatalf("hard link double counted: %d", r.TotSize)
}
}
func TestPermissionErrorsSurface(t *testing.T) {
if runtime.GOOS == "windows" || os.Geteuid() == 0 {
t.Skip("needs non-root unix")
}
root := t.TempDir()
locked := filepath.Join(root, "locked")
os.Mkdir(locked, 0o755)
os.WriteFile(filepath.Join(locked, "f"), []byte("x"), 0o644)
os.Chmod(locked, 0)
defer os.Chmod(locked, 0o755)
tr := runScan(t, root, Options{Workers: 2})
if tr.Stats.ErrCounts[inventory.ErrPermission] != 1 || !tr.Stats.Incomplete() {
t.Fatalf("expected one permission error: %+v", tr.Stats.ErrCounts)
}
if tr.Node(tr.Root()).Errors != 1 {
t.Fatal("error not aggregated to root")
}
}
func TestExcludes(t *testing.T) {
root := t.TempDir()
os.MkdirAll(filepath.Join(root, "keep"), 0o755)
os.MkdirAll(filepath.Join(root, "skip"), 0o755)
os.WriteFile(filepath.Join(root, "keep", "a.tmp"), []byte("x"), 0o644)
os.WriteFile(filepath.Join(root, "keep", "b.txt"), []byte("x"), 0o644)
os.WriteFile(filepath.Join(root, "skip", "c.txt"), []byte("x"), 0o644)
tr := runScan(t, root, Options{Workers: 2, Excludes: []string{"*.tmp", filepath.Join(root, "skip")}})
if tr.Stats.Excluded != 2 || tr.Node(0).Files != 1 {
t.Fatalf("excluded=%d files=%d", tr.Stats.Excluded, tr.Node(0).Files)
}
// Relative path patterns resolve against the working directory.
t.Chdir(root)
tr = runScan(t, root, Options{Workers: 2, Excludes: []string{"./skip"}})
if tr.Stats.Excluded != 1 {
t.Fatalf("relative exclude: excluded=%d", tr.Stats.Excluded)
}
}
func TestUnicodeAndHostileNames(t *testing.T) {
root := t.TempDir()
names := []string{"日本語.txt", "emoji-🎉.png", "esc\x1b[31mred", "nl\nname", "sp ace"}
if runtime.GOOS == "windows" {
names = names[:2]
}
for _, n := range names {
if err := os.WriteFile(filepath.Join(root, n), []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
}
tr := runScan(t, root, Options{Workers: 2})
got := map[string]bool{}
tr.Children(0, func(_ inventory.NodeID, n *inventory.Node) { got[n.Name] = true })
for _, n := range names {
if !got[n] {
t.Errorf("missing %q (raw names must be preserved)", n)
}
}
}
func TestSingleFileRoot(t *testing.T) {
p := filepath.Join(t.TempDir(), "file")
os.WriteFile(p, make([]byte, 1234), 0o644)
tr := runScan(t, p, Options{})
if tr.Node(0).TotSize != 1234 || tr.Node(0).Kind != inventory.KindFile {
t.Fatalf("%+v", tr.Node(0))
}
}
func BenchmarkScan(b *testing.B) {
root := b.TempDir()
mkTree(b, root, 5, 4, 20)
for _, w := range []int{1, 4, 16} {
b.Run(fmt.Sprintf("workers=%d", w), func(b *testing.B) {
for b.Loop() {
runScan(b, root, Options{Workers: w})
}
})
}
}
internal/snapshot/compare.go
-177
View File
@@ -1,177 +0,0 @@
package snapshot
import (
"sort"
"github.com/andydixon/mapsize/internal/inventory"
)
// Diff relates two inventories by path. Both trees must not be mutated
// while a Diff is in use (snapshots and finished scans satisfy this).
type Diff struct {
Old, New *inventory.Tree
// OldOf maps a New NodeID to the matching Old NodeID, or NoNode if the
// entry is new.
OldOf []inventory.NodeID
// Removed lists the topmost Old nodes with no counterpart in New.
Removed []inventory.NodeID
// RemovedUnder sums, per New directory, the sizes of removed children
// (in both size modes) so growth colouring can account for them.
removedUnder map[inventory.NodeID][2]int64
}
// Compare matches nodes by name, directory by directory, from the roots.
func Compare(old, new *inventory.Tree) *Diff {
d := &Diff{Old: old, New: new, OldOf: make([]inventory.NodeID, new.Len()),
removedUnder: map[inventory.NodeID][2]int64{}}
for i := range d.OldOf {
d.OldOf[i] = inventory.NoNode
}
type pair struct{ o, n inventory.NodeID }
stack := []pair{{old.Root(), new.Root()}}
for len(stack) > 0 {
p := stack[len(stack)-1]
stack = stack[:len(stack)-1]
d.OldOf[p.n] = p.o
on, nn := old.Node(p.o), new.Node(p.n)
if !on.IsDir() || !nn.IsDir() {
continue
}
byName := map[string]inventory.NodeID{}
old.Children(p.o, func(id inventory.NodeID, n *inventory.Node) { byName[n.Name] = id })
new.Children(p.n, func(id inventory.NodeID, n *inventory.Node) {
if oid, ok := byName[n.Name]; ok {
delete(byName, n.Name)
stack = append(stack, pair{oid, id})
}
})
for _, oid := range byName {
d.Removed = append(d.Removed, oid)
o := old.Node(oid)
r := d.removedUnder[p.n]
r[0] += o.TotAlloc
r[1] += o.TotSize
d.removedUnder[p.n] = r
}
}
sort.Slice(d.Removed, func(i, j int) bool { return d.Removed[i] < d.Removed[j] })
return d
}
// OldSize returns the matching old size of a New node and whether it existed.
func (d *Diff) OldSize(id inventory.NodeID, m inventory.SizeMode) (int64, bool) {
o := d.OldOf[id]
if o == inventory.NoNode {
return 0, false
}
return d.Old.Node(o).Total(m), true
}
// Delta returns new minus old size for a New node (full size if new).
func (d *Diff) Delta(id inventory.NodeID, m inventory.SizeMode) int64 {
old, _ := d.OldSize(id, m)
return d.New.Node(id).Total(m) - old
}
// Status classifies a node.
type Status uint8
const (
Unchanged Status = iota
Added
Removed
Grew
Shrank
)
func (s Status) String() string {
return [...]string{"unchanged", "new", "removed", "increased", "decreased"}[s]
}
// StatusOf classifies a New node.
func (d *Diff) StatusOf(id inventory.NodeID, m inventory.SizeMode) Status {
if d.OldOf[id] == inventory.NoNode {
return Added
}
switch delta := d.Delta(id, m); {
case delta > 0:
return Grew
case delta < 0:
return Shrank
}
return Unchanged
}
// Change is one row in a change report.
type Change struct {
Path string
Status Status
Delta int64
Old int64
New int64
Node inventory.NodeID // in New (NoNode for removed)
}
// Changes returns the k most significant changes. A directory is listed
// only if its change is not almost entirely explained by a single child, so
// the report points at /var/lib/postgresql rather than /, /var and /var/lib.
func (d *Diff) Changes(k int, m inventory.SizeMode) []Change {
var out []Change
for i := range d.New.Len() {
id := inventory.NodeID(i)
delta := d.Delta(id, m)
if delta == 0 {
continue
}
n := d.New.Node(id)
if d.OldOf[id] != inventory.NoNode && n.IsDir() && explainedByChild(d, id, delta, m) {
continue
}
if p := n.Parent; p != inventory.NoNode && d.OldOf[p] == inventory.NoNode {
continue // inside a wholly new directory; the directory is listed
}
old, _ := d.OldSize(id, m)
out = append(out, Change{Path: d.New.Path(id), Status: d.StatusOf(id, m), Delta: delta, Old: old, New: n.Total(m), Node: id})
}
for _, oid := range d.Removed {
o := d.Old.Node(oid)
out = append(out, Change{Path: d.Old.Path(oid), Status: Removed, Delta: -o.Total(m), Old: o.Total(m), Node: inventory.NoNode})
}
sort.Slice(out, func(i, j int) bool {
a, b := abs(out[i].Delta), abs(out[j].Delta)
if a != b {
return a > b
}
return out[i].Path < out[j].Path
})
if k > 0 && len(out) > k {
out = out[:k]
}
return out
}
func explainedByChild(d *Diff, id inventory.NodeID, delta int64, m inventory.SizeMode) bool {
explained := false
d.New.Children(id, func(c inventory.NodeID, _ *inventory.Node) {
if cd := d.Delta(c, m); (cd > 0) == (delta > 0) && abs(cd)*10 >= abs(delta)*8 {
explained = true
}
})
if r, ok := d.removedUnder[id]; ok && !explained {
v := r[0]
if m == inventory.SizeLogical {
v = r[1]
}
if delta < 0 && v*10 >= abs(delta)*8 {
explained = true
}
}
return explained
}
func abs(v int64) int64 {
if v < 0 {
return -v
}
return v
}
internal/snapshot/hostile_test.go
-160
View File
@@ -1,160 +0,0 @@
package snapshot
import (
"bytes"
"compress/gzip"
"crypto/sha256"
"encoding/binary"
"io"
"os"
"path/filepath"
"strings"
"testing"
"github.com/andydixon/mapsize/internal/brand"
"github.com/andydixon/mapsize/internal/export"
"github.com/andydixon/mapsize/internal/inventory"
)
// craft builds a well-formed (correctly checksummed) but hostile snapshot.
func craft(n int, meta string, rec func(i int, w *writer)) []byte {
var out bytes.Buffer
out.WriteString(brand.SnapshotMagic)
out.Write(binary.LittleEndian.AppendUint16(nil, Version))
out.Write(binary.LittleEndian.AppendUint16(nil, flagGzipped))
gz, _ := gzip.NewWriterLevel(&out, gzip.BestCompression)
h := sha256.New()
w := &writer{w: io.MultiWriter(gz, h)}
w.uvarint(uint64(len(meta)))
w.bytes([]byte(meta))
w.uvarint(1)
w.str("")
w.uvarint(uint64(n))
for i := range n {
rec(i, w)
}
w.uvarint(0)
gz.Close()
out.Write(h.Sum(nil))
return out.Bytes()
}
func node(w *writer, pd uint64, kind inventory.Kind, name string, size int64) {
w.uvarint(pd)
w.bytes([]byte{byte(kind), 0})
w.uvarint(0)
w.str(name)
w.varint(size)
w.varint(size)
w.varint(0)
for range 5 {
w.uvarint(0)
}
}
func chain(n int) []byte {
return craft(n, `{"root":"/r"}`, func(i int, w *writer) {
if i == 0 {
node(w, 0, inventory.KindDir, "", 0)
} else {
node(w, 1, inventory.KindDir, "a", 0)
}
})
}
func TestDeepChainRejected(t *testing.T) {
if _, err := Load(bytes.NewReader(chain(MaxDepth + 1))); err != nil {
t.Fatalf("depth at the limit must load: %v", err)
}
_, err := Load(bytes.NewReader(chain(MaxDepth + 2)))
if err == nil || !strings.Contains(err.Error(), "deeper") {
t.Fatalf("over-deep chain accepted: %v", err)
}
}
func TestDecompressionBombLimited(t *testing.T) {
flat := craft(100_000, `{"root":"/r"}`, func(i int, w *writer) {
if i == 0 {
node(w, 0, inventory.KindDir, "", 0)
} else {
node(w, uint64(i), inventory.KindFile, "f", 0)
}
})
_, err := LoadLimit(bytes.NewReader(flat), 64<<10)
if err == nil || !strings.Contains(err.Error(), "limit") {
t.Fatalf("bomb not limited: %v", err)
}
if got := bodyLimit(int64(len(flat))); got < 64<<20 {
t.Fatalf("bodyLimit floor: %d", got)
}
}
func TestHugeSizesDoNotWrap(t *testing.T) {
data := craft(3, `{"root":"/r"}`, func(i int, w *writer) {
if i == 0 {
node(w, 0, inventory.KindDir, "", 0)
} else {
node(w, uint64(i), inventory.KindFile, "f", MaxSize)
}
})
tr, err := Load(bytes.NewReader(data))
if err != nil {
t.Fatal(err)
}
if tr.Node(0).TotSize < 0 {
t.Fatal("aggregate wrapped negative")
}
over := craft(2, `{"root":"/r"}`, func(i int, w *writer) {
if i == 0 {
node(w, 0, inventory.KindDir, "", 0)
} else {
node(w, 1, inventory.KindFile, "f", MaxSize+1)
}
})
if _, err := Load(bytes.NewReader(over)); err == nil {
t.Fatal("size above MaxSize accepted")
}
}
func TestHostileMetadataSanitisedInSummary(t *testing.T) {
data := craft(1, `{"root":"/r\u001b[2J","excludes":["\u001b]0;PWNED\u0007"],"excluded":1}`, func(i int, w *writer) {
node(w, 0, inventory.KindDir, "", 0)
})
tr, err := Load(bytes.NewReader(data))
if err != nil {
t.Fatal(err)
}
tr.Stats.Excluded = 1
var b bytes.Buffer
export.Summary(&b, tr)
if strings.ContainsAny(b.String(), "\x1b\x07") {
t.Fatalf("raw control characters in summary: %q", b.String())
}
}
// Blocks of one directory plus K-1 files repeat byte for byte, so a small
// file describes millions of nodes; LoadFile must refuse before allocating.
func TestNodeAmplificationLimited(t *testing.T) {
const k, n = 1000, 1_200_001
data := craft(n, `{"root":"/r"}`, func(i int, w *writer) {
switch {
case i == 0:
node(w, 0, inventory.KindDir, "", 0)
case i == 1:
node(w, 1, inventory.KindDir, "d", 0)
case (i-1)%k == 0:
node(w, k, inventory.KindDir, "d", 0)
default:
node(w, uint64((i-1)%k), inventory.KindFile, "f", 0)
}
})
if _, err := Load(bytes.NewReader(data)); err != nil {
t.Fatalf("crafted tree is otherwise valid: %v", err)
}
p := filepath.Join(t.TempDir(), "amp.msz")
os.WriteFile(p, data, 0o644)
_, err := LoadFile(p)
if err == nil || !strings.Contains(err.Error(), "node count") {
t.Fatalf("%d nodes from a %d-byte file accepted: %v", n, len(data), err)
}
}
internal/snapshot/snapshot.go
-528
View File
@@ -1,528 +0,0 @@
// Package snapshot implements the versioned, checksummed binary inventory
// format and snapshot comparison.
//
// Layout (all integers little-endian / varint):
//
// magic 8 bytes brand.SnapshotMagic
// version uint16 currently 1
// flags uint16 bit 0: body is gzip-compressed (always set by Save)
// body gzip stream of:
// meta uvarint length + JSON (root, timestamps, counters, options)
// exts uvarint count, then count × (uvarint length + bytes)
// nodes uvarint count, then per node:
// uvarint parentDelta (id-parent; 0 only for the root)
// byte kind, byte category, uvarint flags
// uvarint nameLen + name
// varint size, varint alloc, varint mtime
// uvarint mode, uid, gid, nlink, ext
// errors uvarint count, then per record: uvarint node, byte kind,
// uvarint len + name, uvarint len + message
// sha256 32 bytes over the uncompressed body
//
// Nodes are written in ID order; a parent always precedes its children, so a
// reader can rebuild the tree in one pass and malformed input cannot create
// cycles. Aggregates are recomputed on load, never trusted.
package snapshot
import (
"bufio"
"bytes"
"compress/gzip"
"crypto/sha256"
"encoding/binary"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"strings"
"time"
"github.com/andydixon/mapsize/internal/brand"
"github.com/andydixon/mapsize/internal/inventory"
)
// Version is the current format version.
const Version = 1
// Limits applied when reading untrusted snapshots.
const (
MaxNodes = 1<<31 - 1
MaxDepth = 4096 // far beyond any real path; bounds recursion in exporters
MaxSize = 1 << 60 // per-node size cap (1 EiB); keeps aggregates from overflowing
// DefaultMaxBody caps the decompressed body for Load. LoadFile uses a
// limit proportional to the file size instead (see bodyLimit).
DefaultMaxBody = 4 << 30
MaxNameLen = 1 << 16
MaxMetaLen = 16 << 20
MaxExtLen = 64
MaxMsgLen = 4096
maxExts = 1<<16 - 1
flagGzipped = 1
)
type meta struct {
Root string `json:"root"`
Created time.Time `json:"created"`
Start time.Time `json:"start"`
End time.Time `json:"end"`
Complete bool `json:"complete"`
Cancelled bool `json:"cancelled"`
Files int64 `json:"files"`
Dirs int64 `json:"dirs"`
Symlinks int64 `json:"symlinks"`
Others int64 `json:"others"`
ErrCounts []int64 `json:"err_counts"`
Excluded int64 `json:"excluded"`
SkippedMounts int64 `json:"skipped_mounts"`
VirtualSkipped int64 `json:"virtual_skipped"`
LoopsSkipped int64 `json:"loops_skipped"`
BrokenLinks int64 `json:"broken_links"`
HardlinkDups int64 `json:"hardlink_dups"`
Unscanned int64 `json:"unscanned"`
Excludes []string `json:"excludes"`
OneFileSystem bool `json:"one_file_system"`
Follow string `json:"follow"`
Workers int `json:"workers"`
Host string `json:"host"`
Generator string `json:"generator"`
}
// IsSnapshot reports whether path starts with the snapshot magic.
func IsSnapshot(path string) bool {
f, err := os.Open(path)
if err != nil {
return false
}
defer f.Close()
b := make([]byte, len(brand.SnapshotMagic))
_, err = io.ReadFull(f, b)
return err == nil && string(b) == brand.SnapshotMagic
}
type writer struct {
w io.Writer
buf [binary.MaxVarintLen64]byte
err error
}
func (w *writer) bytes(b []byte) {
if w.err == nil {
_, w.err = w.w.Write(b)
}
}
func (w *writer) uvarint(v uint64) { w.bytes(w.buf[:binary.PutUvarint(w.buf[:], v)]) }
func (w *writer) varint(v int64) { w.bytes(w.buf[:binary.PutVarint(w.buf[:], v)]) }
func (w *writer) str(s string) { w.uvarint(uint64(len(s))); w.bytes([]byte(s)) }
// Save writes t to w. The caller must hold t's read lock.
func Save(out io.Writer, t *inventory.Tree) error {
bw := bufio.NewWriterSize(out, 1<<20)
hdr := make([]byte, 0, 12)
hdr = append(hdr, brand.SnapshotMagic...)
hdr = binary.LittleEndian.AppendUint16(hdr, Version)
hdr = binary.LittleEndian.AppendUint16(hdr, flagGzipped)
bw.Write(hdr)
gz, _ := gzip.NewWriterLevel(bw, gzip.BestSpeed)
h := sha256.New()
w := &writer{w: io.MultiWriter(gz, h)}
st := &t.Stats
host, _ := os.Hostname()
m := meta{Root: st.Root, Created: time.Now().UTC(), Start: st.Start, End: st.End, Complete: st.Complete,
Cancelled: st.Cancelled, Files: st.Files, Dirs: st.Dirs, Symlinks: st.Symlinks, Others: st.Others,
ErrCounts: st.ErrCounts[:], Excluded: st.Excluded, SkippedMounts: st.SkippedMounts,
VirtualSkipped: st.VirtualSkipped, LoopsSkipped: st.LoopsSkipped, BrokenLinks: st.BrokenLinks,
HardlinkDups: st.HardlinkDups, Unscanned: st.Unscanned, Excludes: st.Excludes,
OneFileSystem: st.OneFileSystem, Follow: st.Follow, Workers: st.Workers, Host: host,
Generator: brand.Name + " " + brand.Version}
mb, err := json.Marshal(m)
if err != nil {
return err
}
w.uvarint(uint64(len(mb)))
w.bytes(mb)
w.uvarint(uint64(t.ExtCount()))
for i := range t.ExtCount() {
w.str(t.ExtByIndex(i))
}
w.uvarint(uint64(t.Len()))
for i := range t.Len() {
id := inventory.NodeID(i)
n := t.Node(id)
if i == 0 {
w.uvarint(0)
} else {
w.uvarint(uint64(id - n.Parent))
}
w.bytes([]byte{byte(n.Kind), byte(n.Cat)})
w.uvarint(uint64(n.Flags))
w.str(n.Name)
w.varint(n.Size)
w.varint(n.Alloc)
w.varint(n.MTime)
w.uvarint(uint64(n.Mode))
w.uvarint(uint64(n.UID))
w.uvarint(uint64(n.GID))
w.uvarint(uint64(n.Nlink))
w.uvarint(uint64(n.Ext))
}
w.uvarint(uint64(len(st.Errors)))
for _, e := range st.Errors {
w.uvarint(uint64(e.Node))
w.bytes([]byte{byte(e.Kind)})
w.str(e.Name)
w.str(e.Msg)
}
if w.err != nil {
return w.err
}
if err := gz.Close(); err != nil {
return err
}
bw.Write(h.Sum(nil))
return bw.Flush()
}
// SaveFile writes a snapshot atomically (temp file + rename).
func SaveFile(path string, t *inventory.Tree) error {
tmp, err := os.CreateTemp(dirOf(path), ".mapsize-*.tmp")
if err != nil {
return err
}
defer os.Remove(tmp.Name())
if err := Save(tmp, t); err != nil {
tmp.Close()
return err
}
if err := tmp.Close(); err != nil {
return err
}
return os.Rename(tmp.Name(), path)
}
func dirOf(p string) string {
i := strings.LastIndexAny(p, `/\`)
if i < 0 {
return "."
}
if i == 0 {
return p[:1]
}
return p[:i]
}
type reader struct {
r *bufio.Reader
err error
}
func (r *reader) fail(format string, a ...any) {
if r.err == nil {
r.err = fmt.Errorf("snapshot: "+format, a...)
}
}
func (r *reader) byte1() byte {
if r.err != nil {
return 0
}
b, err := r.r.ReadByte()
if err != nil {
r.fail("truncated: %v", err)
}
return b
}
func (r *reader) uvarint(max uint64, what string) uint64 {
if r.err != nil {
return 0
}
v, err := readUvarint(r.r)
if err != nil {
r.fail("bad %s: %v", what, err)
return 0
}
if v > max {
r.fail("%s %d exceeds limit %d", what, v, max)
return 0
}
return v
}
func (r *reader) varint(what string) int64 {
if r.err != nil {
return 0
}
ux, err := readUvarint(r.r)
if err != nil {
r.fail("bad %s: %v", what, err)
}
v := int64(ux >> 1) // zig-zag, as encoding/binary.PutVarint
if ux&1 != 0 {
v = ^v
}
return v
}
var errOverflow = errors.New("varint overflows 64 bits")
// readUvarint is encoding/binary.ReadUvarint specialised to *bufio.Reader
// (no interface dispatch per byte; this is the loader's hot loop).
func readUvarint(br *bufio.Reader) (uint64, error) {
var x uint64
var s uint
for i := 0; i < binary.MaxVarintLen64; i++ {
b, err := br.ReadByte()
if err != nil {
if i > 0 && err == io.EOF {
err = io.ErrUnexpectedEOF
}
return x, err
}
if b < 0x80 {
if i == binary.MaxVarintLen64-1 && b > 1 {
return x, errOverflow
}
return x | uint64(b)<<s, nil
}
x |= uint64(b&0x7f) << s
s += 7
}
return x, errOverflow
}
func (r *reader) bytesN(n uint64) []byte {
if r.err != nil {
return nil
}
b := make([]byte, n)
if _, err := io.ReadFull(r.r, b); err != nil {
r.fail("truncated: %v", err)
return nil
}
return b
}
func (r *reader) str(max uint64, what string) string {
return string(r.bytesN(r.uvarint(max, what+" length")))
}
// limitedReader fails (rather than silently truncating) past n bytes, so a
// decompression bomb is reported as such.
type limitedReader struct {
r io.Reader
n int64
}
func (l *limitedReader) Read(p []byte) (int, error) {
if l.n <= 0 {
return 0, errors.New("snapshot: decompressed size exceeds limit (possible decompression bomb)")
}
if int64(len(p)) > l.n {
p = p[:l.n]
}
n, err := l.r.Read(p)
l.n -= int64(n)
return n, err
}
// bodyLimit allows decompressed bodies up to 64× the compressed file size
// (real snapshots compress ~6×), with a 64 MiB floor for tiny files.
func bodyLimit(fileSize int64) int64 { return max(64<<20, fileSize*64) }
// nodeLimit caps the node count for a file of fileSize bytes. Real
// snapshots spend ~15 compressed bytes per node; a crafted one describes a
// node in a fraction of a byte, so without this a 1 MB file can demand
// gigabytes of memory while staying within bodyLimit.
func nodeLimit(fileSize int64) uint64 { return uint64(max(1<<20, fileSize)) }
// Load reads and validates a snapshot, allowing up to DefaultMaxBody bytes
// of decompressed data.
func Load(in io.Reader) (*inventory.Tree, error) { return LoadLimit(in, DefaultMaxBody) }
// LoadLimit is Load with an explicit decompressed-size limit.
func LoadLimit(in io.Reader, maxBody int64) (*inventory.Tree, error) {
return load(in, maxBody, MaxNodes)
}
func load(in io.Reader, maxBody int64, maxNodes uint64) (*inventory.Tree, error) {
br := bufio.NewReaderSize(in, 1<<20)
hdr := make([]byte, len(brand.SnapshotMagic)+4)
if _, err := io.ReadFull(br, hdr); err != nil {
return nil, errors.New("snapshot: file too short")
}
if string(hdr[:8]) != brand.SnapshotMagic {
return nil, errors.New("snapshot: not a " + brand.Name + " snapshot")
}
if v := binary.LittleEndian.Uint16(hdr[8:]); v != Version {
return nil, fmt.Errorf("snapshot: unsupported version %d (this build reads %d)", v, Version)
}
if binary.LittleEndian.Uint16(hdr[10:])&flagGzipped == 0 {
return nil, errors.New("snapshot: uncompressed bodies are not supported")
}
gz, err := gzip.NewReader(br)
if err != nil {
return nil, fmt.Errorf("snapshot: %w", err)
}
gz.Multistream(false)
// Decompression and hashing run on their own goroutine, overlapping
// with parsing. The pipe's close orders the hash and br accesses: the
// parser only sees EOF after the copier has finished with both.
h := sha256.New()
pr, pw := io.Pipe()
defer pr.Close() // unblocks the copier if we bail out early
go func() {
_, err := io.CopyBuffer(pw, io.TeeReader(&limitedReader{gz, maxBody}, h), make([]byte, 256<<10))
pw.CloseWithError(err)
}()
r := &reader{r: bufio.NewReaderSize(pr, 1<<16)}
var m meta
if mb := r.bytesN(r.uvarint(MaxMetaLen, "metadata length")); r.err == nil {
if err := json.Unmarshal(mb, &m); err != nil {
return nil, fmt.Errorf("snapshot: metadata: %w", err)
}
}
if r.err != nil {
return nil, r.err
}
t := inventory.New(m.Root, inventory.KindDir)
nExt := r.uvarint(maxExts, "extension count")
exts := make([]string, 0, min(nExt, 1024))
for range nExt {
exts = append(exts, r.str(MaxExtLen, "extension"))
}
if r.err != nil {
return nil, r.err
}
extIndex := t.ExtIndexer()
nNodes := r.uvarint(maxNodes, "node count")
depth := []uint16{0} // per node, for the depth limit; parents precede children
if r.err != nil {
return nil, r.err
}
if nNodes == 0 {
return nil, errors.New("snapshot: no root node")
}
for i := uint64(0); i < nNodes && r.err == nil; i++ {
pd := r.uvarint(MaxNodes, "parent")
kind := inventory.Kind(r.byte1())
cat := inventory.Category(r.byte1())
flags := inventory.Flags(r.uvarint(1<<16-1, "flags"))
name := r.str(MaxNameLen, "name")
size, alloc, mtime := r.varint("size"), r.varint("alloc"), r.varint("mtime")
mode := r.uvarint(1<<32-1, "mode")
uid := r.uvarint(1<<32-1, "uid")
gid := r.uvarint(1<<32-1, "gid")
nlink := r.uvarint(1<<32-1, "nlink")
ext := r.uvarint(nExt, "ext")
if r.err != nil {
break
}
if kind > inventory.KindOther || cat >= inventory.NumCategories {
r.fail("node %d: invalid kind/category", i)
break
}
if size < 0 || alloc < 0 || size > MaxSize || alloc > MaxSize {
r.fail("node %d: size out of range", i)
break
}
var id inventory.NodeID
if i == 0 {
if pd != 0 || kind != inventory.KindDir && nNodes > 1 {
r.fail("invalid root")
break
}
id = 0
rn := t.Node(0)
rn.Kind = kind
} else {
if pd == 0 || pd > i {
r.fail("node %d: invalid parent reference", i)
break
}
parent := inventory.NodeID(i - pd)
if t.Node(parent).Kind != inventory.KindDir {
r.fail("node %d: parent is not a directory", i)
break
}
if !validName(name) {
r.fail("node %d: invalid name %q", i, name)
break
}
if d := depth[parent] + 1; d > MaxDepth {
r.fail("node %d: deeper than %d levels", i, MaxDepth)
break
}
id = t.Add(parent, name, kind)
depth = append(depth, depth[parent]+1)
}
n := t.Node(id)
n.Cat, n.Flags = cat, flags
n.Size, n.Alloc, n.MTime = size, alloc, mtime
n.Mode, n.UID, n.GID, n.Nlink = uint32(mode), uint32(uid), uint32(gid), uint32(nlink)
if kind != inventory.KindDir && ext < uint64(len(exts)) {
n.Ext = extIndex(exts[ext])
}
}
nErr := r.uvarint(inventory.MaxErrorRecords, "error count")
for range nErr {
node := r.uvarint(nNodes-1, "error node")
kind := inventory.ErrKind(r.byte1())
name := r.str(MaxNameLen, "error name")
msg := r.str(MaxMsgLen, "error message")
if r.err != nil {
break
}
if kind >= inventory.NumErrKinds {
kind = inventory.ErrOther
}
t.Stats.Errors = append(t.Stats.Errors, inventory.ErrorRecord{Node: inventory.NodeID(node), Kind: kind, Name: name, Msg: msg})
}
if r.err != nil {
return nil, r.err
}
// The body must end here, then the checksum follows the gzip stream.
if _, err := r.r.ReadByte(); err != io.EOF {
return nil, errors.New("snapshot: trailing data in body")
}
sum := make([]byte, sha256.Size)
if _, err := io.ReadFull(br, sum); err != nil {
return nil, errors.New("snapshot: missing checksum")
}
if !bytes.Equal(sum, h.Sum(nil)) {
return nil, errors.New("snapshot: checksum mismatch (file corrupt)")
}
t.Recompute()
st := &t.Stats
st.Start, st.End, st.Complete, st.Cancelled = m.Start, m.End, true, m.Cancelled
st.Files, st.Dirs, st.Symlinks, st.Others = m.Files, m.Dirs, m.Symlinks, m.Others
copy(st.ErrCounts[:], m.ErrCounts)
st.Excluded, st.SkippedMounts, st.VirtualSkipped = m.Excluded, m.SkippedMounts, m.VirtualSkipped
st.LoopsSkipped, st.BrokenLinks, st.HardlinkDups, st.Unscanned = m.LoopsSkipped, m.BrokenLinks, m.HardlinkDups, m.Unscanned
st.Excludes, st.OneFileSystem, st.Follow, st.Workers = m.Excludes, m.OneFileSystem, m.Follow, m.Workers
st.FromSnapshot = fmt.Sprintf("%s (%s, %s)", m.Created.Local().Format("2006-01-02 15:04"), m.Host, m.Generator)
return t, nil
}
func validName(s string) bool {
return s != "" && s != "." && s != ".." && !strings.ContainsAny(s, "/\x00")
}
// LoadFile opens and loads a snapshot file.
func LoadFile(path string) (*inventory.Tree, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
limit, nodes := int64(DefaultMaxBody), uint64(MaxNodes)
if fi, err := f.Stat(); err == nil {
limit, nodes = bodyLimit(fi.Size()), nodeLimit(fi.Size())
}
return load(f, limit, nodes)
}
internal/snapshot/snapshot_test.go
-139
View File
@@ -1,139 +0,0 @@
package snapshot
import (
"bufio"
"bytes"
"encoding/binary"
"testing"
"github.com/andydixon/mapsize/internal/inventory"
)
func sample() *inventory.Tree {
t := inventory.New("/srv", inventory.KindDir)
a := t.Add(0, "a", inventory.KindDir)
b := t.Add(0, "b", inventory.KindDir)
f1 := t.Add(a, "x.iso", inventory.KindFile)
t.Node(f1).Size, t.Node(f1).Alloc = 1000, 1024
f2 := t.Add(b, "y.log", inventory.KindFile)
t.Node(f2).Size, t.Node(f2).Alloc = 50, 4096
t.Node(f2).Flags = inventory.FlagSparse
t.Recompute()
t.Stats.Files, t.Stats.Dirs = 2, 3
t.Stats.AddError(inventory.ErrorRecord{Node: b, Kind: inventory.ErrPermission, Msg: "denied"})
return t
}
func TestRoundTrip(t *testing.T) {
src := sample()
var buf bytes.Buffer
if err := Save(&buf, src); err != nil {
t.Fatal(err)
}
got, err := Load(bytes.NewReader(buf.Bytes()))
if err != nil {
t.Fatal(err)
}
if got.Len() != src.Len() {
t.Fatalf("len %d want %d", got.Len(), src.Len())
}
for i := range src.Len() {
a, b := src.Node(inventory.NodeID(i)), got.Node(inventory.NodeID(i))
if a.Name != b.Name || a.TotSize != b.TotSize || a.TotAlloc != b.TotAlloc || a.Flags != b.Flags ||
src.Path(inventory.NodeID(i)) != got.Path(inventory.NodeID(i)) || src.ExtName(a) != got.ExtName(b) {
t.Fatalf("node %d differs: %+v vs %+v", i, a, b)
}
}
if got.Stats.ErrCounts[inventory.ErrPermission] != 1 || len(got.Stats.Errors) != 1 {
t.Fatalf("errors not restored: %+v", got.Stats)
}
}
func TestCorruptionDetected(t *testing.T) {
var buf bytes.Buffer
Save(&buf, sample())
data := buf.Bytes()
// Flip a byte in the checksum trailer.
bad := append([]byte(nil), data...)
bad[len(bad)-1] ^= 0xff
if _, err := Load(bytes.NewReader(bad)); err == nil {
t.Fatal("checksum corruption not detected")
}
// Every truncation must fail cleanly.
for i := 0; i < len(data)-1; i += 7 {
if _, err := Load(bytes.NewReader(data[:i])); err == nil {
t.Fatalf("truncation at %d accepted", i)
}
}
}
func FuzzLoad(f *testing.F) {
var buf bytes.Buffer
Save(&buf, sample())
f.Add(buf.Bytes())
f.Add([]byte("MAPSIZE\x00\x01\x00\x01\x00"))
f.Fuzz(func(t *testing.T, data []byte) {
tr, err := Load(bytes.NewReader(data))
if err != nil {
return
}
// A successfully loaded tree must be internally consistent.
for i := 1; i < tr.Len(); i++ {
if p := tr.Node(inventory.NodeID(i)).Parent; p >= inventory.NodeID(i) {
t.Fatalf("parent %d >= child %d", p, i)
}
}
_ = tr.Path(inventory.NodeID(tr.Len() - 1))
})
}
func TestCompare(t *testing.T) {
old := sample()
nw := inventory.New("/srv", inventory.KindDir)
a := nw.Add(0, "a", inventory.KindDir)
f1 := nw.Add(a, "x.iso", inventory.KindFile)
nw.Node(f1).Size, nw.Node(f1).Alloc = 5000, 5120
c := nw.Add(0, "c", inventory.KindDir)
f3 := nw.Add(c, "z", inventory.KindFile)
nw.Node(f3).Size, nw.Node(f3).Alloc = 10, 4096
nw.Recompute()
d := Compare(old, nw)
if d.StatusOf(f1, inventory.SizeAllocated) != Grew || d.Delta(f1, inventory.SizeAllocated) != 4096 {
t.Fatalf("x.iso: %v %d", d.StatusOf(f1, 0), d.Delta(f1, 0))
}
if d.StatusOf(c, 0) != Added {
t.Fatal("c should be new")
}
if len(d.Removed) != 1 || old.Node(d.Removed[0]).Name != "b" {
t.Fatalf("removed %v", d.Removed)
}
ch := d.Changes(10, inventory.SizeAllocated)
paths := map[string]Status{}
for _, c := range ch {
paths[c.Path] = c.Status
}
if paths["/srv/a/x.iso"] != Grew || paths["/srv/b"] != Removed || paths["/srv/c"] != Added {
t.Fatalf("changes %+v", ch)
}
if _, ok := paths["/srv/a"]; ok {
t.Fatal("/srv/a should be explained by its child")
}
if _, ok := paths["/srv/c/z"]; ok {
t.Fatal("contents of new dirs should not be listed separately")
}
}
func TestReadUvarintMatchesStdlib(t *testing.T) {
for _, v := range []uint64{0, 1, 127, 128, 300, 1 << 35, 1<<64 - 1} {
buf := binary.AppendUvarint(nil, v)
got, err := readUvarint(bufio.NewReader(bytes.NewReader(buf)))
if err != nil || got != v {
t.Fatalf("%d: got %d %v", v, got, err)
}
}
bad := bytes.Repeat([]byte{0xff}, 11)
if _, err := readUvarint(bufio.NewReader(bytes.NewReader(bad))); err == nil {
t.Fatal("overflow not detected")
}
}
internal/textutil/text.go
-160
View File
@@ -1,160 +0,0 @@
// Package textutil holds display helpers: sanitising untrusted strings for
// the terminal, width-aware truncation, and human-readable units.
package textutil
import (
"fmt"
"strings"
"unicode/utf8"
"github.com/charmbracelet/x/ansi"
"github.com/rivo/uniseg"
)
// Sanitize makes an untrusted string (a filename, snapshot data) safe to
// write to a terminal. Control characters (C0, DEL, C1) and bidi overrides are
// replaced by visible escapes; invalid UTF-8 becomes U+FFFD. The original
// string should be kept for filesystem operations.
func Sanitize(s string) string {
clean := true
for i := 0; i < len(s); i++ {
if c := s[i]; c < 0x20 || c >= 0x7f {
clean = false
break
}
}
if clean {
return s
}
var b strings.Builder
b.Grow(len(s) + 8)
for i := 0; i < len(s); {
r, size := utf8.DecodeRuneInString(s[i:])
if r == utf8.RuneError && size <= 1 {
b.WriteRune(utf8.RuneError)
i++
continue
}
i += size
switch {
case r < 0x20 || r == 0x7f || (r >= 0x80 && r <= 0x9f):
fmt.Fprintf(&b, `\x%02x`, r)
case isBidiControl(r):
fmt.Fprintf(&b, `\u%04x`, r)
default:
b.WriteRune(r)
}
}
return b.String()
}
func isBidiControl(r rune) bool {
return (r >= 0x202a && r <= 0x202e) || (r >= 0x2066 && r <= 0x2069) ||
r == 0x200e || r == 0x200f || r == 0x061c
}
// Clusters calls fn for each grapheme cluster of s with its display width.
// Clusters whose width terminals disagree on (per-rune wcwidth versus
// grapheme width, e.g. emoji ZWJ sequences) are replaced by U+FFFD with
// width 1, and zero-width clusters are dropped, so that measuring and
// drawing always agree with the terminal. fn returns false to stop.
func Clusters(s string, fn func(c string, w int) bool) {
state := -1
for s != "" {
var c string
var w int
c, s, w, state = uniseg.FirstGraphemeClusterInString(s, state)
if w == 0 {
continue
}
if len(c) > 1 && ansi.WcWidth.StringWidth(c) != w {
c, w = "\uFFFD", 1
}
if !fn(c, w) {
return
}
}
}
// Width returns the display width of s in terminal cells.
func Width(s string) int {
n := 0
for i := 0; i < len(s); i++ {
if s[i] < 0x20 || s[i] >= 0x7f {
Clusters(s, func(_ string, w int) bool { n += w; return true })
return n
}
}
return len(s)
}
// Truncate clips s to at most w cells, appending "…" if anything was cut.
func Truncate(s string, w int) string {
if w <= 0 {
return ""
}
if Width(s) <= w {
return s
}
return clip(s, w-1) + "…"
}
// TruncateLeft clips from the left, prefixing "…" (used for paths and
// breadcrumbs where the tail is the informative part).
func TruncateLeft(s string, w int) string {
if w <= 0 {
return ""
}
total := Width(s)
if total <= w {
return s
}
skip := total - (w - 1)
var b strings.Builder
Clusters(s, func(c string, cw int) bool {
if skip > 0 {
skip -= cw
if skip < 0 {
// A wide cluster overshot by one cell; pad to keep width exact.
b.WriteString(strings.Repeat(" ", -skip))
}
return true
}
b.WriteString(c)
return true
})
return "…" + b.String()
}
// clip returns the longest prefix of s that fits in w cells.
func clip(s string, w int) string {
var b strings.Builder
used := 0
Clusters(s, func(c string, cw int) bool {
if used+cw > w {
return false
}
used += cw
b.WriteString(c)
return true
})
return b.String()
}
// PadRight pads s with spaces to exactly w cells (truncating if longer).
func PadRight(s string, w int) string {
s = Truncate(s, w)
if d := w - Width(s); d > 0 {
s += strings.Repeat(" ", d)
}
return s
}
// PadLeft right-aligns s in w cells.
func PadLeft(s string, w int) string {
s = Truncate(s, w)
if d := w - Width(s); d > 0 {
s = strings.Repeat(" ", d) + s
}
return s
}
internal/textutil/text_test.go
-74
View File
@@ -1,74 +0,0 @@
package textutil
import (
"strings"
"testing"
)
func TestSanitize(t *testing.T) {
cases := map[string]string{
"plain.txt": "plain.txt",
"evil\x1b[2Jname": `evil\x1b[2Jname`,
"bell\a": `bell\x07`,
"c1\u009bx": `c1\x9bx`,
"bad\xffutf8": "bad�utf8",
"rtl\u202egnp.exe": `rtl\u202egnp.exe`,
"日本語": "日本語",
"tab\tnew\nline\rdel\x7f": `tab\x09new\x0aline\x0ddel\x7f`,
}
for in, want := range cases {
if got := Sanitize(in); got != want {
t.Errorf("Sanitize(%q) = %q, want %q", in, got, want)
}
if strings.ContainsAny(Sanitize(in), "\x1b\a\u009b") {
t.Errorf("control byte survived in %q", in)
}
}
}
func TestTruncate(t *testing.T) {
if got := Truncate("hello world", 5); got != "hell…" {
t.Errorf("got %q", got)
}
if got := Truncate("日本語テキスト", 5); Width(got) > 5 {
t.Errorf("wide truncation overflow: %q width %d", got, Width(got))
}
if got := Truncate("abc", 3); got != "abc" {
t.Errorf("got %q", got)
}
if got := Truncate("abc", 0); got != "" {
t.Errorf("got %q", got)
}
if got := TruncateLeft("/a/very/long/path", 8); Width(got) != 8 || !strings.HasSuffix(got, "path") {
t.Errorf("TruncateLeft got %q", got)
}
if got := TruncateLeft("ab日本語", 4); Width(got) != 4 {
t.Errorf("TruncateLeft wide got %q (%d)", got, Width(got))
}
}
func TestSize(t *testing.T) {
cases := map[int64]string{0: "0 B", 1023: "1023 B", 1024: "1.00 KiB", 1536: "1.50 KiB",
181 << 30: "181 GiB", 1 << 62: "4.00 EiB"}
for in, want := range cases {
if got := Size(in); got != want {
t.Errorf("Size(%d)=%q want %q", in, got, want)
}
}
if got := Count(2184392); got != "2,184,392" {
t.Errorf("Count got %q", got)
}
if got := SignedSize(-2048); got != "-2.00 KiB" {
t.Errorf("SignedSize got %q", got)
}
}
func TestClustersAgreeWithWidth(t *testing.T) {
for _, s := range []string{"abc", "日本", "é", "👨‍👩‍👧", "🎉x", "​"} {
n := 0
Clusters(s, func(_ string, w int) bool { n += w; return true })
if n != Width(s) {
t.Errorf("%q: clusters %d width %d", s, n, Width(s))
}
}
}
internal/textutil/units.go
-125
View File
@@ -1,125 +0,0 @@
package textutil
import (
"fmt"
"strconv"
"time"
)
// SI selects decimal units (kB, MB, …). The default is IEC (KiB, MiB, …).
// It is set once at start-up, before any goroutines format sizes.
var SI bool
var (
iecUnits = []string{"B", "KiB", "MiB", "GiB", "TiB", "PiB", "EiB"}
siUnits = []string{"B", "kB", "MB", "GB", "TB", "PB", "EB"}
)
func scaled(n int64) (float64, string) {
base, units := 1024.0, iecUnits
if SI {
base, units = 1000.0, siUnits
}
neg := n < 0
v := float64(n)
if neg {
v = -v
}
i := 0
for v >= base && i < len(units)-1 {
v /= base
i++
}
if neg {
v = -v
}
return v, units[i]
}
func num(v float64, unit string) string {
a := v
if a < 0 {
a = -a
}
switch {
case unit == "B":
return strconv.FormatFloat(v, 'f', 0, 64)
case a < 10:
return strconv.FormatFloat(v, 'f', 2, 64)
case a < 100:
return strconv.FormatFloat(v, 'f', 1, 64)
default:
return strconv.FormatFloat(v, 'f', 0, 64)
}
}
// Size formats a byte count, e.g. "86.4 GiB".
func Size(n int64) string {
v, u := scaled(n)
return num(v, u) + " " + u
}
// SizeCompact formats a byte count in at most ~7 cells, e.g. "86GiB".
func SizeCompact(n int64) string {
v, u := scaled(n)
a := v
if a < 0 {
a = -a
}
if u != "B" && a < 10 {
return strconv.FormatFloat(v, 'f', 1, 64) + u
}
return strconv.FormatFloat(v, 'f', 0, 64) + u
}
// SignedSize formats a delta with an explicit sign.
func SignedSize(n int64) string {
if n > 0 {
return "+" + Size(n)
}
if n < 0 {
return "-" + Size(-n)
}
return "±0 B"
}
// Count formats an integer with thousands separators.
func Count(n int64) string {
s := strconv.FormatInt(n, 10)
neg := false
if n < 0 {
neg, s = true, s[1:]
}
out := make([]byte, 0, len(s)+len(s)/3)
for i := range len(s) {
if i > 0 && (len(s)-i)%3 == 0 {
out = append(out, ',')
}
out = append(out, s[i])
}
if neg {
return "-" + string(out)
}
return string(out)
}
// Duration formats an elapsed time as mm:ss or h:mm:ss.
func Duration(d time.Duration) string {
s := int64(d.Seconds())
if s >= 3600 {
return fmt.Sprintf("%d:%02d:%02d", s/3600, s/60%60, s%60)
}
return fmt.Sprintf("%02d:%02d", s/60, s%60)
}
// Percent formats part/whole as a percentage.
func Percent(part, whole int64) string {
if whole <= 0 {
return "–"
}
p := float64(part) * 100 / float64(whole)
if p < 10 {
return strconv.FormatFloat(p, 'f', 1, 64) + "%"
}
return strconv.FormatFloat(p, 'f', 0, 64) + "%"
}
internal/treemap/nav.go
-106
View File
@@ -1,106 +0,0 @@
package treemap
// Direction is an arrow-key direction.
type Direction int
const (
Left Direction = iota
Right
Up
Down
)
// Neighbour finds the block a user expects to reach by pressing an arrow
// key from the block with ID sel:
//
// 1. Only blocks entirely beyond the selected block's edge in the
// direction of travel are candidates.
// 2. Blocks whose span overlaps the selected block's orthogonal span are
// strongly preferred over all others.
// 3. Then the smallest gap along the direction of travel wins.
// 4. Then the largest overlap with the selected block's span.
// 5. Then the block whose orthogonal span is closest to the selected
// block's centre line (0 if it contains it).
// 6. Ties break by position (top, then left) and finally ID.
//
// It returns false if there is no block in that direction.
func Neighbour(blocks []Block, sel int64, d Direction) (int64, bool) {
var s Block
found := false
for _, b := range blocks {
if b.ID == sel {
s, found = b, true
break
}
}
if !found {
return 0, false
}
type score struct {
tier, gap, ovl, off, y, x int
id int64
}
less := func(a, b score) bool {
switch {
case a.tier != b.tier:
return a.tier < b.tier
case a.gap != b.gap:
return a.gap < b.gap
case a.ovl != b.ovl:
return a.ovl > b.ovl
case a.off != b.off:
return a.off < b.off
case a.y != b.y:
return a.y < b.y
case a.x != b.x:
return a.x < b.x
}
return a.id < b.id
}
var best score
have := false
for _, c := range blocks {
if c.ID == sel {
continue
}
var gap, lo, hi, clo, chi int
var ok bool
switch d {
case Right:
ok, gap = c.X >= s.X+s.W, c.X-(s.X+s.W)
lo, hi, clo, chi = s.Y, s.Y+s.H, c.Y, c.Y+c.H
case Left:
ok, gap = c.X+c.W <= s.X, s.X-(c.X+c.W)
lo, hi, clo, chi = s.Y, s.Y+s.H, c.Y, c.Y+c.H
case Down:
ok, gap = c.Y >= s.Y+s.H, c.Y-(s.Y+s.H)
lo, hi, clo, chi = s.X, s.X+s.W, c.X, c.X+c.W
case Up:
ok, gap = c.Y+c.H <= s.Y, s.Y-(c.Y+c.H)
lo, hi, clo, chi = s.X, s.X+s.W, c.X, c.X+c.W
}
if !ok {
continue
}
tier, ovl := 1, min(hi, chi)-max(lo, clo)
if ovl > 0 {
tier = 0
} else {
ovl = 0
}
// Distance (doubled, to stay integral) from the selected centre
// line to the candidate's span; 0 if the span contains it.
centre := lo + hi
off := 0
if centre < 2*clo {
off = 2*clo - centre
} else if centre > 2*chi {
off = centre - 2*chi
}
sc := score{tier, gap, ovl, off, c.Y, c.X, c.ID}
if !have || less(sc, best) {
best, have = sc, true
}
}
return best.id, have
}
internal/treemap/treemap.go
-189
View File
@@ -1,189 +0,0 @@
// Package treemap is pure geometry: squarified layout on a cell grid,
// level-of-detail selection and spatial neighbour search. It has no
// knowledge of filesystems or terminals.
package treemap
import (
"math"
"sort"
)
// Rect is an integer cell rectangle.
type Rect struct{ X, Y, W, H int }
// Empty reports whether r has no cells.
func (r Rect) Empty() bool { return r.W <= 0 || r.H <= 0 }
// Area returns W*H.
func (r Rect) Area() int { return max(r.W, 0) * max(r.H, 0) }
// Contains reports whether the cell (x, y) is inside r.
func (r Rect) Contains(x, y int) bool { return x >= r.X && x < r.X+r.W && y >= r.Y && y < r.Y+r.H }
// Inset shrinks r by n cells on each side.
func (r Rect) Inset(n int) Rect {
r = Rect{r.X + n, r.Y + n, r.W - 2*n, r.H - 2*n}
if r.W < 0 {
r.W = 0
}
if r.H < 0 {
r.H = 0
}
return r
}
// Item is something to lay out. Size must be > 0 to receive space.
type Item struct {
ID int64
Size float64
}
// Block is a laid-out item.
type Block struct {
ID int64
Rect
}
// Squarify lays items out inside bounds using the squarified treemap
// algorithm (Bruls, Huizing, van Wijk). Items with non-positive size are
// ignored. The output partitions bounds exactly: rectangle edges are snapped
// to the grid by rounding shared float edges, so there are no gaps and no
// overlaps. Items that round to zero width or height are omitted. The result
// is deterministic for a given input set regardless of input order.
func Squarify(items []Item, bounds Rect) []Block {
if bounds.Empty() {
return nil
}
its := make([]Item, 0, len(items))
total := 0.0
for _, it := range items {
if it.Size > 0 && !math.IsInf(it.Size, 0) && !math.IsNaN(it.Size) {
its = append(its, it)
total += it.Size
}
}
if len(its) == 0 || total <= 0 {
return nil
}
sort.SliceStable(its, func(i, j int) bool {
if its[i].Size != its[j].Size {
return its[i].Size > its[j].Size
}
return its[i].ID < its[j].ID
})
scale := float64(bounds.W) * float64(bounds.H) / total
areas := make([]float64, len(its))
for i, it := range its {
areas[i] = it.Size * scale
}
frs := layout(areas, float64(bounds.X), float64(bounds.Y), float64(bounds.W), float64(bounds.H))
out := make([]Block, 0, len(its))
for i, f := range frs {
x0, y0 := int(math.Round(f[0])), int(math.Round(f[1]))
x1, y1 := int(math.Round(f[0]+f[2])), int(math.Round(f[1]+f[3]))
x1 = min(x1, bounds.X+bounds.W)
y1 = min(y1, bounds.Y+bounds.H)
if x1 > x0 && y1 > y0 {
out = append(out, Block{ID: its[i].ID, Rect: Rect{x0, y0, x1 - x0, y1 - y0}})
}
}
return out
}
// worst returns the worst aspect ratio of a row with the given sum, largest
// and smallest areas laid along a side of length side.
func worst(sum, largest, smallest, side float64) float64 {
s2, w2 := sum*sum, side*side
return math.Max(w2*largest/s2, s2/(w2*smallest))
}
// layout returns float rectangles [x, y, w, h] for areas sorted descending.
func layout(areas []float64, x, y, w, h float64) [][4]float64 {
out := make([][4]float64, len(areas))
n := len(areas)
for i := 0; i < n; {
short := math.Min(w, h)
j := i + 1
sum := areas[i]
cur := worst(sum, areas[i], areas[i], short)
for j < n {
s2 := sum + areas[j]
if nw := worst(s2, areas[i], areas[j], short); nw > cur {
break
} else {
cur = nw
}
sum = s2
j++
}
last := j == n
if w >= h {
thick := sum / h
if last || thick > w {
thick = w
}
yy := y
for k := i; k < j; k++ {
hh := areas[k] / sum * h
if k == j-1 {
hh = y + h - yy
}
out[k] = [4]float64{x, yy, thick, hh}
yy += hh
}
x += thick
w -= thick
} else {
thick := sum / w
if last || thick > h {
thick = h
}
xx := x
for k := i; k < j; k++ {
ww := areas[k] / sum * w
if k == j-1 {
ww = x + w - xx
}
out[k] = [4]float64{xx, y, ww, thick}
xx += ww
}
y += thick
h -= thick
}
i = j
}
return out
}
// Visible decides how many leading items of a size-sorted (descending)
// list deserve their own block when area cells represent total: items are
// kept while each would get at least minArea cells and fewer than maxItems
// have been kept. The remainder should be merged into one group block.
func Visible(sizes []float64, total float64, area, minArea, maxItems int) int {
if total <= 0 || area <= 0 {
return 0
}
k := 0
for k < len(sizes) && k < maxItems {
if sizes[k]/total*float64(area) < float64(minArea) {
break
}
k++
}
// Grouping a single leftover item gains nothing; show it directly if it
// gets at least one cell.
if k == len(sizes)-1 && k < maxItems && sizes[k]/total*float64(area) >= 1 {
k++
}
return k
}
// At returns the block containing the cell (x, y).
func At(blocks []Block, x, y int) (int64, bool) {
for _, b := range blocks {
if b.Contains(x, y) {
return b.ID, true
}
}
return 0, false
}
internal/treemap/treemap_test.go
-247
View File
@@ -1,247 +0,0 @@
package treemap
import (
"math/rand"
"testing"
)
func checkPartition(t *testing.T, blocks []Block, b Rect, exact bool) {
t.Helper()
grid := make([]int, b.W*b.H)
for _, bl := range blocks {
if bl.W <= 0 || bl.H <= 0 {
t.Fatalf("non-positive block %+v", bl)
}
if bl.X < b.X || bl.Y < b.Y || bl.X+bl.W > b.X+b.W || bl.Y+bl.H > b.Y+b.H {
t.Fatalf("block %+v outside bounds %+v", bl, b)
}
for y := bl.Y; y < bl.Y+bl.H; y++ {
for x := bl.X; x < bl.X+bl.W; x++ {
i := (y-b.Y)*b.W + (x - b.X)
if grid[i] != 0 {
t.Fatalf("overlap at %d,%d", x, y)
}
grid[i] = 1
}
}
}
if exact {
for i, v := range grid {
if v == 0 {
t.Fatalf("gap at %d,%d", b.X+i%b.W, b.Y+i/b.W)
}
}
}
}
func TestSquarifyBasic(t *testing.T) {
b := Rect{2, 3, 60, 20}
items := []Item{{1, 6}, {2, 6}, {3, 4}, {4, 3}, {5, 2}, {6, 2}, {7, 1}}
blocks := Squarify(items, b)
if len(blocks) != len(items) {
t.Fatalf("got %d blocks", len(blocks))
}
checkPartition(t, blocks, b, true)
// Areas roughly proportional.
for _, bl := range blocks {
want := float64(b.Area()) * items[bl.ID-1].Size / 24
if got := float64(bl.Area()); got < want*0.6-4 || got > want*1.4+4 {
t.Errorf("block %d area %v want ~%v", bl.ID, got, want)
}
}
// Deterministic regardless of input order.
rev := append([]Item(nil), items...)
for i, j := 0, len(rev)-1; i < j; i, j = i+1, j-1 {
rev[i], rev[j] = rev[j], rev[i]
}
b2 := Squarify(rev, b)
for i := range blocks {
if blocks[i] != b2[i] {
t.Fatal("layout depends on input order")
}
}
}
func TestSquarifyEdgeCases(t *testing.T) {
if Squarify(nil, Rect{0, 0, 10, 10}) != nil {
t.Fatal("nil items")
}
if Squarify([]Item{{1, 0}, {2, -5}}, Rect{0, 0, 10, 10}) != nil {
t.Fatal("zero sizes should produce nothing")
}
for _, b := range []Rect{{0, 0, 0, 0}, {0, 0, 1, 1}, {0, 0, 1, 50}, {0, 0, 50, 1}, {0, 0, -3, 4}} {
bl := Squarify([]Item{{1, 5}, {2, 3}, {3, 1}}, b)
if !b.Empty() {
checkPartition(t, bl, b, true)
}
}
// One enormous item and many tiny ones.
items := []Item{{0, 1e15}}
for i := 1; i < 500; i++ {
items = append(items, Item{int64(i), 1})
}
bl := Squarify(items, Rect{0, 0, 80, 24})
checkPartition(t, bl, Rect{0, 0, 80, 24}, true)
}
func TestResizeRecomputes(t *testing.T) {
items := []Item{{1, 50}, {2, 30}, {3, 20}}
for _, b := range []Rect{{0, 0, 80, 24}, {0, 0, 120, 40}, {0, 0, 300, 80}, {0, 0, 70, 20}, {0, 0, 180, 50}} {
bl := Squarify(items, b)
checkPartition(t, bl, b, true)
if len(bl) != 3 {
t.Fatalf("%v: %d blocks", b, len(bl))
}
}
}
func FuzzSquarify(f *testing.F) {
f.Add(int64(1), 80, 24, 10)
f.Add(int64(2), 3, 2, 100)
f.Add(int64(3), 300, 90, 1000)
f.Fuzz(func(t *testing.T, seed int64, w, h, n int) {
w, h, n = w%400, h%150, n%3000
if w < 0 || h < 0 || n < 0 {
return
}
r := rand.New(rand.NewSource(seed))
items := make([]Item, n)
for i := range items {
items[i] = Item{int64(i), float64(r.Int63n(1 << uint(r.Intn(40)+1)))}
}
b := Rect{r.Intn(10), r.Intn(10), w, h}
bl := Squarify(items, b)
if !b.Empty() && len(bl) > 0 {
checkPartition(t, bl, b, true)
}
})
}
func TestVisible(t *testing.T) {
sizes := []float64{100, 50, 10, 1, 1, 1}
if k := Visible(sizes, 163, 1630, 20, 100); k != 3 {
t.Fatalf("k=%d", k)
}
if k := Visible(sizes, 163, 1630, 20, 2); k != 2 {
t.Fatalf("maxItems k=%d", k)
}
if k := Visible([]float64{10, 1}, 11, 1100, 200, 10); k != 2 {
t.Fatalf("single leftover k=%d", k)
}
}
// Layout:
//
// ┌───────┬───────┐
// │ A │ B │
// ├───┬───┴───────┤
// │ C │ D │
// └───┴───────────┘
func TestNeighbour(t *testing.T) {
A := Block{1, Rect{0, 0, 8, 4}}
B := Block{2, Rect{8, 0, 8, 4}}
C := Block{3, Rect{0, 4, 4, 4}}
D := Block{4, Rect{4, 4, 12, 4}}
bs := []Block{D, C, B, A} // order must not matter
cases := []struct {
from int64
d Direction
want int64
ok bool
}{
{1, Right, 2, true}, {2, Left, 1, true}, {1, Down, 3, true}, {2, Down, 4, true},
{3, Right, 4, true}, {4, Left, 3, true}, {3, Up, 1, true}, {4, Up, 2, true},
{1, Left, 0, false}, {1, Up, 0, false}, {4, Right, 0, false}, {4, Down, 0, false},
}
for _, c := range cases {
got, ok := Neighbour(bs, c.from, c.d)
if ok != c.ok || got != c.want {
t.Errorf("from %d dir %d: got %d,%v want %d,%v", c.from, c.d, got, ok, c.want, c.ok)
}
}
}
func TestNeighbourPrefersAdjacentOverArrayOrder(t *testing.T) {
// S is on the left; R is directly right; F is far right and lower but
// appears first in the slice.
S := Block{1, Rect{0, 0, 10, 5}}
R := Block{2, Rect{10, 0, 10, 5}}
F := Block{3, Rect{10, 20, 30, 5}}
if got, _ := Neighbour([]Block{F, S, R}, 1, Right); got != 2 {
t.Fatalf("got %d", got)
}
// Tall block on the left with three stacked blocks to the right: pick
// the one at the centre line.
T := Block{10, Rect{0, 0, 10, 9}}
r1 := Block{11, Rect{10, 0, 10, 3}}
r2 := Block{12, Rect{10, 3, 10, 3}}
r3 := Block{13, Rect{10, 6, 10, 3}}
if got, _ := Neighbour([]Block{r3, r1, r2, T}, 10, Right); got != 12 {
t.Fatalf("centre preference: got %d", got)
}
if got, _ := Neighbour([]Block{r3, r1, r2, T}, 13, Left); got != 10 {
t.Fatalf("back left: got %d", got)
}
}
func TestNeighbourRandomLayoutsAlwaysAdjacent(t *testing.T) {
r := rand.New(rand.NewSource(7))
for iter := 0; iter < 200; iter++ {
items := make([]Item, 2+r.Intn(30))
for i := range items {
items[i] = Item{int64(i), float64(1 + r.Intn(1000))}
}
bl := Squarify(items, Rect{0, 0, 120, 40})
for _, s := range bl {
for _, d := range []Direction{Left, Right, Up, Down} {
id, ok := Neighbour(bl, s.ID, d)
if !ok {
continue
}
var c Block
for _, b := range bl {
if b.ID == id {
c = b
}
}
// In a partition, if any block touches the edge in direction d
// with overlapping span, the chosen one must touch too.
var touch bool
switch d {
case Right:
touch = c.X == s.X+s.W
case Left:
touch = c.X+c.W == s.X
case Down:
touch = c.Y == s.Y+s.H
case Up:
touch = c.Y+c.H == s.Y
}
if !touch {
t.Fatalf("iter %d: from %+v dir %d chose non-adjacent %+v", iter, s, d, c)
}
}
}
}
}
func BenchmarkSquarify(b *testing.B) {
items := make([]Item, 300)
for i := range items {
items[i] = Item{int64(i), float64(1 + i*i)}
}
for b.Loop() {
Squarify(items, Rect{0, 0, 200, 60})
}
}
func BenchmarkNeighbour(b *testing.B) {
items := make([]Item, 300)
for i := range items {
items[i] = Item{int64(i), float64(1 + i*i)}
}
bl := Squarify(items, Rect{0, 0, 200, 60})
for b.Loop() {
Neighbour(bl, 150, Right)
}
}
internal/tui/actions.go
-276
View File
@@ -1,276 +0,0 @@
package tui
import (
"context"
"fmt"
"io/fs"
"time"
tea "charm.land/bubbletea/v2"
"github.com/andydixon/mapsize/internal/duplicate"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/platform"
"github.com/andydixon/mapsize/internal/textutil"
"github.com/andydixon/mapsize/internal/treemap"
)
// ---- Trash ---------------------------------------------------------------
type trashDoneMsg struct {
tree *inventory.Tree // the tree id belongs to; results for a replaced tree are dropped
id inventory.NodeID
err error
done string // "Moved to trash" or "Deleted"
}
// requestTrash asks for confirmation before moving the selection to the
// trash. On a headless system (no desktop, so nobody empties the trash and
// the space is never freed) it deletes permanently instead, after a
// confirmation that says so.
func (m *Model) requestTrash() tea.Cmd {
switch {
case m.opts.ReadOnly:
return m.warn("Read-only mode: modifying actions are disabled")
case m.snapshot:
return m.warn("Snapshot view: nothing to delete on the live filesystem")
case m.scanning:
return m.warn("Wait for the scan to finish before deleting items")
}
id := m.selectedNode()
if id == inventory.NoNode || id == m.tree.Root() {
return nil
}
n := m.tree.Node(id)
path := m.tree.Path(id)
want := platform.Expect{UID: n.UID, Mode: fs.FileMode(n.Mode)}
what := "file"
if n.IsDir() {
what = fmt.Sprintf("directory with %s files", textutil.Count(int64(n.Files)))
}
body := []string{textutil.Sanitize(path), "", fmt.Sprintf("%s · %s", what, textutil.Size(n.Total(m.sizeMode))), ""}
if platform.Headless() {
m.openModal(&confirmModal{
ttl: "Delete permanently? Are you sure?",
body: append(body, "This is a headless system (no X or Wayland display), so there is no trash:",
"it will be permanently deleted and cannot be recovered."),
yes: "delete permanently",
onYes: func(m *Model) tea.Cmd {
t := m.tree
return func() tea.Msg { return trashDoneMsg{t, id, platform.Delete(path, want), "Deleted"} }
},
})
return nil
}
m.openModal(&confirmModal{
ttl: "Move to trash?",
body: append(body, "It will be moved to the "+platform.TrashName+", not permanently deleted."),
yes: "move to " + platform.TrashName,
onYes: func(m *Model) tea.Cmd {
t := m.tree
return func() tea.Msg {
return trashDoneMsg{t, id, platform.Trash(path, want), "Moved to " + platform.TrashName}
}
},
})
return nil
}
func (m *Model) trashDone(msg trashDoneMsg) tea.Cmd {
if msg.err != nil {
return m.warn("Delete failed: " + textutil.Sanitize(msg.err.Error()))
}
if msg.tree != m.tree || m.scanning {
// A rescan replaced the inventory; the new scan reflects the change.
return m.info(msg.done)
}
// The scan has finished, so the UI goroutine is the tree's only writer.
t := m.tree
t.Lock()
n := t.Node(msg.id)
n.Flags |= inventory.FlagDeleted
d := inventory.Delta{Size: -n.TotSize, Alloc: -n.TotAlloc, Files: -int64(n.Files), Dirs: -int64(n.Dirs)}
if n.IsDir() {
d.Dirs--
for i, v := range t.CatSizes(msg.id) {
d.Cat[i] = -v
}
} else {
d.Files--
d.Cat[n.Cat] = -n.TotAlloc
}
t.Propagate(n.Parent, d)
n.TotSize, n.TotAlloc = 0, 0
name := n.Name
t.Unlock()
m.invalidate()
return m.info(msg.done + ": " + textutil.Sanitize(name))
}
// ---- Duplicates ----------------------------------------------------------
type dupState struct {
running bool
finder *duplicate.Finder
groups []duplicate.Group
err error
cancel context.CancelFunc
started time.Time
}
type (
dupProgressMsg struct{}
dupDoneMsg struct {
finder *duplicate.Finder // identifies the search; stale results are dropped
groups []duplicate.Group
err error
}
)
func (m *Model) startDuplicates() tea.Cmd {
if m.snapshot {
return m.warn("Duplicate detection needs the live filesystem")
}
if m.scanning {
return m.warn("Wait for the scan to finish first")
}
if m.dups != nil {
m.ensureDupView()
return nil
}
m.openModal(&confirmModal{
ttl: "Find duplicate files?",
body: []string{
"Files are grouped by size, then sampled, then fully hashed with SHA-256.",
"Only identical hashes are reported as duplicates.",
"", "This reads file contents (it may take a while and can update access times).",
},
yes: "start",
onYes: func(m *Model) tea.Cmd {
ctx, cancel := context.WithCancel(context.Background())
f := &duplicate.Finder{}
m.dups = &dupState{running: true, finder: f, cancel: cancel, started: time.Now()}
m.ensureDupView()
t := m.tree
return tea.Batch(m.startTicking(), func() tea.Msg {
g, err := f.Find(ctx, t, duplicate.Options{MinSize: 1, Workers: 4})
return dupDoneMsg{f, g, err}
})
},
})
return nil
}
func (m *Model) ensureDupView() {
for i, v := range m.views {
if _, ok := v.(*dupView); ok {
m.setView(i)
return
}
}
m.views = append(m.views, &dupView{})
m.setView(len(m.views) - 1)
}
func (m *Model) dupMsg(msg tea.Msg) tea.Cmd {
if d, ok := msg.(dupDoneMsg); ok && m.dups != nil && d.finder == m.dups.finder {
m.dups.running = false
m.dups.groups, m.dups.err = d.groups, d.err
m.dirty = true
var wasted int64
for _, g := range d.groups {
wasted += g.Wasted()
}
return m.info(fmt.Sprintf("Duplicates: %d verified groups, %s reclaimable", len(d.groups), textutil.Size(wasted)))
}
return nil
}
type dupView struct {
tb table
rows []dupRow
}
type dupRow struct {
group int
id inventory.NodeID // NoNode for a group header
}
func (*dupView) name() string { return "Dups" }
func (v *dupView) paint(m *Model, cv *Canvas, r treemap.Rect) {
t := m.theme
d := m.dups
if d == nil {
cv.TextCentered(r.X, r.Y+r.H/2, r.W, "Press D to search for duplicates", t.muted())
return
}
if d.running {
p := d.finder.Progress()
cv.Fill(r, " ", t.base())
y := r.Y + r.H/2 - 2
cv.TextCentered(r.X, y, r.W, "Finding duplicates — "+p.Stage+"…", Style{FG: t.Accent, BG: t.Bg, Attr: Bold})
cv.TextCentered(r.X, y+2, r.W, fmt.Sprintf("%s candidate files in %s size groups", textutil.Count(p.Candidates),
textutil.Count(p.CandidateGroups)), t.base())
if p.BytesToHash > 0 {
frac := float64(p.BytesHashed) / float64(p.BytesToHash)
bw := min(60, r.W-10)
m.paintBar(cv, r.X+(r.W-bw)/2, y+4, bw, frac, t.Accent, false)
cv.TextCentered(r.X, y+5, r.W, fmt.Sprintf("%s of %s hashed · %s skipped", textutil.Size(p.BytesHashed),
textutil.Size(p.BytesToHash), textutil.Count(p.Skipped)), t.muted())
}
return
}
v.rows = v.rows[:0]
var tr []tableRow
for gi, g := range d.groups {
v.rows = append(v.rows, dupRow{gi, inventory.NoNode})
tr = append(tr, tableRow{
cells: []string{textutil.Size(g.Wasted()), fmt.Sprintf("%d × %s", len(g.Files), textutil.Size(g.Size)), fmt.Sprintf("verified identical · sha256 %x…", g.Hash[:8])},
styles: []*Style{{FG: t.Warn, BG: t.Bg, Attr: Bold}, {FG: t.Fg, BG: t.Bg, Attr: Bold}, {FG: t.OK, BG: t.Bg}}, bar: -1,
})
for _, id := range g.Files {
v.rows = append(v.rows, dupRow{gi, id})
tr = append(tr, tableRow{cells: []string{"", "", " " + textutil.Sanitize(m.tree.Path(id))}, bar: -1})
}
}
title := fmt.Sprintf("Duplicates — %d groups (Enter details · Space go to)", len(d.groups))
if d.err != nil {
title += " — stopped: " + textutil.Sanitize(d.err.Error())
}
v.tb.paint(m, cv, r, []column{{"WASTED", 10, true, false}, {"COPIES", 16, false, false}, {"", 0, false, false}}, tr, title)
}
func (v *dupView) key(m *Model, k string) (bool, tea.Cmd) {
if v.tb.key(k, len(v.rows)) {
return true, nil
}
if v.tb.cursor < len(v.rows) {
r := v.rows[v.tb.cursor]
if r.id == inventory.NoNode && m.dups != nil && r.group < len(m.dups.groups) {
r.id = m.dups.groups[r.group].Files[0]
}
if r.id != inventory.NoNode {
switch k {
case "enter":
m.openModal(newInfoModal(m, r.id))
return true, nil
case "space":
m.jumpTo(r.id)
return true, nil
case "c", "o", "d":
m.sel = int64(r.id)
}
}
}
if k == "esc" && m.dups != nil && m.dups.running {
m.dups.cancel()
return true, nil
}
return false, nil
}
func (v *dupView) wheel(m *Model, up bool) tea.Cmd {
v.tb.scroll(up, len(v.rows))
return nil
}
internal/tui/bench_test.go
-114
View File
@@ -1,114 +0,0 @@
package tui
import (
"fmt"
"math/rand"
"testing"
"github.com/charmbracelet/colorprofile"
"github.com/andydixon/mapsize/internal/inventory"
)
// bigTree builds ~1M nodes in memory: 200 top-level dirs with nested
// subdirectories, plus one flat directory with 200k files.
func bigTree() *inventory.Tree {
r := rand.New(rand.NewSource(1))
t := inventory.New("/big", inventory.KindDir)
file := func(p inventory.NodeID, i int) {
id := t.Add(p, fmt.Sprintf("f%d.dat", i), inventory.KindFile)
n := t.Node(id)
n.Size = int64(r.ExpFloat64() * 1e6)
n.Alloc = n.Size
n.Cat = inventory.Category(r.Intn(int(inventory.NumCategories)))
}
flat := t.Add(0, "flat", inventory.KindDir)
for i := range 200_000 {
file(flat, i)
}
for d := range 200 {
top := t.Add(0, fmt.Sprintf("dir%d", d), inventory.KindDir)
for s := range 40 {
sub := t.Add(top, fmt.Sprintf("sub%d", s), inventory.KindDir)
for f := range 100 {
file(sub, f)
}
}
}
t.Recompute()
return t
}
var bigTreeCache *inventory.Tree
func benchModel(b *testing.B) *Model {
if bigTreeCache == nil {
bigTreeCache = bigTree()
}
m := newModel(Options{}, nil, bigTreeCache, func() {})
m.profile = colorprofile.TrueColor
send(m, size(200, 60))
return m
}
// Cold frame: sorting children, LOD selection, layout and paint.
func BenchmarkBigColdFrame(b *testing.B) {
m := benchModel(b)
b.ReportMetric(float64(m.tree.Len()), "nodes")
for b.Loop() {
m.invalidate()
m.View()
}
}
// Resize: child lists cached; layout and paint rerun for the new size.
func BenchmarkBigResize(b *testing.B) {
m := benchModel(b)
m.View()
i := 0
for b.Loop() {
send(m, size(150+i%100, 40+i%30))
m.View()
i++
}
}
// Zoomed into a 200k-entry directory (cold child sort each time).
func BenchmarkBigFlatDirCold(b *testing.B) {
m := benchModel(b)
m.tree.Children(0, func(id inventory.NodeID, n *inventory.Node) {
if n.Name == "flat" {
m.zoom = id
}
})
for b.Loop() {
m.invalidate()
m.View()
}
}
// Arrow-key navigation including the re-render it triggers.
func BenchmarkBigNavigate(b *testing.B) {
m := benchModel(b)
m.View()
keys := []string{"right", "down", "left", "up"}
i := 0
for b.Loop() {
send(m, key(keys[i%4]))
m.View()
i++
}
}
func BenchmarkBigFilter(b *testing.B) {
m := benchModel(b)
for _, q := range []string{"size > 1MB", "*.dat AND size > 500k", "path contains sub3"} {
b.Run(q, func(b *testing.B) {
m.search.ed.set(q)
m.queryChanged()
for b.Loop() {
m.applyFilter()()
}
})
}
}
internal/tui/canvas.go
-324
View File
@@ -1,324 +0,0 @@
package tui
import (
"strconv"
"strings"
"github.com/andydixon/mapsize/internal/textutil"
"github.com/andydixon/mapsize/internal/treemap"
)
// Color is 0 for the terminal default, otherwise 0x01RRGGBB.
type Color uint32
// RGB builds a colour.
func RGB(r, g, b uint8) Color { return Color(1<<24 | uint32(r)<<16 | uint32(g)<<8 | uint32(b)) }
// Hex parses "#rrggbb".
func Hex(s string) (Color, bool) {
s = strings.TrimPrefix(s, "#")
if len(s) != 6 {
return 0, false
}
v, err := strconv.ParseUint(s, 16, 32)
if err != nil {
return 0, false
}
return Color(1<<24 | uint32(v)), true
}
func (c Color) rgb() (r, g, b float64) {
return float64(c>>16&0xff) / 255, float64(c>>8&0xff) / 255, float64(c&0xff) / 255
}
// Mix blends c towards o by t (0..1). Default colours are returned as-is.
func (c Color) Mix(o Color, t float64) Color {
if c == 0 || o == 0 {
return c
}
r1, g1, b1 := c.rgb()
r2, g2, b2 := o.rgb()
f := func(a, b float64) uint8 { return uint8((a + (b-a)*t) * 255) }
return RGB(f(r1, r2), f(g1, g2), f(b1, b2))
}
// Luma returns perceived brightness 0..1.
func (c Color) Luma() float64 {
r, g, b := c.rgb()
return 0.299*r + 0.587*g + 0.114*b
}
// Attr is a text attribute bit set.
type Attr uint8
const (
Bold Attr = 1 << iota
Faint
Italic
Underline
Reverse
)
// Style is a cell style.
type Style struct {
FG, BG Color
Attr Attr
}
type cell struct {
s string // "" marks the continuation of a wide cluster
st Style
}
// Canvas is an in-memory cell grid that every view paints into. It is
// serialised once per frame.
type Canvas struct {
W, H int
c []cell
}
// NewCanvas creates a canvas filled with spaces in style st.
func NewCanvas(w, h int, st Style) *Canvas {
cv := &Canvas{W: max(w, 0), H: max(h, 0)}
cv.c = make([]cell, cv.W*cv.H)
for i := range cv.c {
cv.c[i] = cell{" ", st}
}
return cv
}
func (cv *Canvas) in(x, y int) bool { return x >= 0 && y >= 0 && x < cv.W && y < cv.H }
// put writes one cluster of width w at (x, y), repairing any wide cluster it
// overlaps so the grid never holds half a character.
func (cv *Canvas) put(x, y int, s string, w int, st Style) {
if !cv.in(x, y) || x+w > cv.W {
return
}
row := y * cv.W
for i := x; i < x+w; i++ {
c := &cv.c[row+i]
if c.s == "" && i > 0 && i == x { // overwriting the tail of a wide cluster
cv.c[row+i-1].s = " "
}
if c.s != "" && textutil.Width(c.s) > 1 && i+1 < cv.W { // overwriting its head
if n := &cv.c[row+i+1]; n.s == "" {
n.s = " "
}
}
}
cv.c[row+x] = cell{s, st}
for i := 1; i < w; i++ {
cv.c[row+x+i] = cell{"", st}
}
}
// Text draws s at (x, y), clipped to maxW cells and the canvas. Control
// characters are replaced as a second line of defence (callers sanitise
// untrusted strings first). It returns the number of cells used.
func (cv *Canvas) Text(x, y int, s string, maxW int, st Style) int {
if y < 0 || y >= cv.H {
return 0
}
limit := min(x+maxW, cv.W)
cx := x
textutil.Clusters(s, func(c string, w int) bool {
if r := c[0]; r < 0x20 || r == 0x7f || (len(c) > 1 && c[0] == 0xc2 && c[1] < 0xa0) {
c, w = "?", 1
}
if cx+w > limit {
// A wide cluster that does not fit leaves a blank cell.
for ; cx < limit; cx++ {
if cx >= 0 {
cv.put(cx, y, " ", 1, st)
}
}
return false
}
if cx >= 0 {
cv.put(cx, y, c, w, st)
}
cx += w
return true
})
return cx - x
}
// TextCentered draws s centred within [x, x+w).
func (cv *Canvas) TextCentered(x, y, w int, s string, st Style) {
s = textutil.Truncate(s, w)
cv.Text(x+(w-textutil.Width(s))/2, y, s, w, st)
}
// TextRight draws s right-aligned ending at x+w.
func (cv *Canvas) TextRight(x, y, w int, s string, st Style) {
s = textutil.Truncate(s, w)
cv.Text(x+w-textutil.Width(s), y, s, w, st)
}
// Fill fills r with the cluster ch.
func (cv *Canvas) Fill(r treemap.Rect, ch string, st Style) {
for y := max(r.Y, 0); y < min(r.Y+r.H, cv.H); y++ {
for x := max(r.X, 0); x < min(r.X+r.W, cv.W); x++ {
cv.put(x, y, ch, 1, st)
}
}
}
// Restyle applies fn to the style of every cell in r.
func (cv *Canvas) Restyle(r treemap.Rect, fn func(Style) Style) {
for y := max(r.Y, 0); y < min(r.Y+r.H, cv.H); y++ {
for x := max(r.X, 0); x < min(r.X+r.W, cv.W); x++ {
c := &cv.c[y*cv.W+x]
c.st = fn(c.st)
}
}
}
// Box glyph sets.
type BoxChars struct{ H, V, TL, TR, BL, BR string }
var (
BoxLight = BoxChars{"─", "│", "┌", "┐", "└", "┘"}
BoxRound = BoxChars{"─", "│", "╭", "╮", "╰", "╯"}
BoxDouble = BoxChars{"═", "║", "╔", "╗", "╚", "╝"}
BoxHeavy = BoxChars{"━", "┃", "┏", "┓", "┗", "┛"}
BoxASCII = BoxChars{"-", "|", "+", "+", "+", "+"}
BoxASCIISel = BoxChars{"=", "#", "#", "#", "#", "#"}
)
// Box draws a border around r (which must be at least 2×2).
func (cv *Canvas) Box(r treemap.Rect, b BoxChars, st Style) {
if r.W < 2 || r.H < 2 {
return
}
x1, y1 := r.X+r.W-1, r.Y+r.H-1
for x := r.X + 1; x < x1; x++ {
cv.put(x, r.Y, b.H, 1, st)
cv.put(x, y1, b.H, 1, st)
}
for y := r.Y + 1; y < y1; y++ {
cv.put(r.X, y, b.V, 1, st)
cv.put(x1, y, b.V, 1, st)
}
cv.put(r.X, r.Y, b.TL, 1, st)
cv.put(x1, r.Y, b.TR, 1, st)
cv.put(r.X, y1, b.BL, 1, st)
cv.put(x1, y1, b.BR, 1, st)
}
// HLine draws a horizontal rule.
func (cv *Canvas) HLine(x, y, w int, ch string, st Style) {
for i := x; i < x+w; i++ {
cv.put(i, y, ch, 1, st)
}
}
// Cell returns the text and style at (x, y) (for tests).
func (cv *Canvas) Cell(x, y int) (string, Style) {
if !cv.in(x, y) {
return "", Style{}
}
c := cv.c[y*cv.W+x]
return c.s, c.st
}
// Line returns the plain text of row y (for tests).
func (cv *Canvas) Line(y int) string {
var b strings.Builder
for x := 0; x < cv.W; x++ {
b.WriteString(cv.c[y*cv.W+x].s)
}
return b.String()
}
// colorDepth selects how colours are serialised.
type colorDepth uint8
const (
depthTrue colorDepth = iota
depth256
depth16
)
func sgr(b *strings.Builder, st Style, depth colorDepth) {
b.WriteString("\x1b[0")
if st.Attr&Bold != 0 {
b.WriteString(";1")
}
if st.Attr&Faint != 0 {
b.WriteString(";2")
}
if st.Attr&Italic != 0 {
b.WriteString(";3")
}
if st.Attr&Underline != 0 {
b.WriteString(";4")
}
if st.Attr&Reverse != 0 {
b.WriteString(";7")
}
col := func(c Color, base string) {
if c == 0 {
return
}
switch depth {
case depth256:
b.WriteString(";" + base + ";5;")
b.WriteString(strconv.Itoa(int(quant256(c))))
return
case depth16:
n := int(quant16(c))
code := 30 + n
if n >= 8 {
code = 90 + n - 8
}
if base == "48" {
code += 10
}
b.WriteString(";" + strconv.Itoa(code))
return
}
b.WriteString(";" + base + ";2;")
b.WriteString(strconv.Itoa(int(c >> 16 & 0xff)))
b.WriteByte(';')
b.WriteString(strconv.Itoa(int(c >> 8 & 0xff)))
b.WriteByte(';')
b.WriteString(strconv.Itoa(int(c & 0xff)))
}
col(st.FG, "38")
col(st.BG, "48")
b.WriteByte('m')
}
// String serialises the canvas as lines of text with SGR styling and
// 24-bit colours.
func (cv *Canvas) String() string { return cv.Render(depthTrue) }
// Render serialises the canvas. For 256 and 16 colours it quantizes with
// its own perceptual mapping (see color256.go); Bubble Tea strips colour
// entirely for colourless terminals.
func (cv *Canvas) Render(depth colorDepth) string {
var b strings.Builder
b.Grow(cv.W * cv.H * 4)
for y := 0; y < cv.H; y++ {
if y > 0 {
b.WriteByte('\n')
}
var cur Style
first := true
for x := 0; x < cv.W; x++ {
c := cv.c[y*cv.W+x]
if c.s == "" {
continue
}
if first || c.st != cur {
sgr(&b, c.st, depth)
cur, first = c.st, false
}
b.WriteString(c.s)
}
b.WriteString("\x1b[0m")
}
return b.String()
}
internal/tui/color256.go
-96
View File
@@ -1,96 +0,0 @@
package tui
import (
"math"
"sync"
)
// The stock 256-colour conversion maps dark, slightly tinted colours to
// saturated cube entries (olive, navy), which wrecks the subtle shading the
// treemap relies on. quant256 picks the perceptually nearest entry (redmean
// distance) among the cube and the grey ramp, never the palette-dependent
// first 16 entries.
var (
cubeLevels = [6]int{0, 95, 135, 175, 215, 255}
q256Cache sync.Map // Color -> uint8
)
func redmean(r1, g1, b1, r2, g2, b2 int) int {
rm := (r1 + r2) / 2
dr, dg, db := r1-r2, g1-g2, b1-b2
return ((512+rm)*dr*dr)>>8 + 4*dg*dg + ((767-rm)*db*db)>>8
}
func quant256(c Color) uint8 {
if v, ok := q256Cache.Load(c); ok {
return v.(uint8)
}
r, g, b := int(c>>16&0xff), int(c>>8&0xff), int(c&0xff)
best, bestD := 16, math.MaxInt
for i := 0; i < 216; i++ {
cr, cg, cb := cubeLevels[i/36], cubeLevels[i/6%6], cubeLevels[i%6]
if d := redmean(r, g, b, cr, cg, cb); d < bestD {
best, bestD = 16+i, d
}
}
for i := 0; i < 24; i++ {
v := 8 + 10*i
if d := redmean(r, g, b, v, v, v); d < bestD {
best, bestD = 232+i, d
}
}
q256Cache.Store(c, uint8(best))
return uint8(best)
}
var q16Cache sync.Map // Color -> uint8
// quant16 maps a colour to a basic colour. Dark shades become black and
// unsaturated colours grey/white, so tints and shadows stay quiet; saturated
// colours keep their hue (bright variant when light), so category colours
// survive. The stock conversion instead turns dark tints into bright
// yellow/cyan.
func quant16(c Color) uint8 {
if v, ok := q16Cache.Load(c); ok {
return v.(uint8)
}
rf, gf, bf := c.rgb()
mx, mn := max(rf, gf, bf), min(rf, gf, bf)
luma := c.Luma()
var n uint8
switch {
case luma < 0.2:
n = 0
case mx-mn < 0.12: // grey
switch {
case luma < 0.45:
n = 8
case luma < 0.8:
n = 7
default:
n = 15
}
default:
// Hue in sixths: red, yellow, green, cyan, blue, magenta.
var h float64
switch mx {
case rf:
h = math.Mod((gf-bf)/(mx-mn), 6)
case gf:
h = (bf-rf)/(mx-mn) + 2
default:
h = (rf-gf)/(mx-mn) + 4
}
if h < 0 {
h += 6
}
ansi := [6]uint8{1, 3, 2, 6, 4, 5}[int(math.Round(h))%6]
n = ansi
if luma > 0.5 {
n += 8
}
}
q16Cache.Store(c, n)
return n
}
internal/tui/data.go
-199
View File
@@ -1,199 +0,0 @@
package tui
import (
"sort"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/snapshot"
)
// kids is a cached, size-sorted child list (zero-size entries removed).
type kids struct {
ids []inventory.NodeID
sizes []int64
total int64
}
// sizeOf returns the size that drives layout: the filtered size when a
// filter is active, otherwise the aggregate in the current size mode.
// Callers hold the tree read lock.
func (m *Model) sizeOf(id inventory.NodeID) int64 {
if r := m.search.result; r != nil {
return r.Size(id)
}
n := m.tree.Node(id)
if n.Flags&inventory.FlagDeleted != 0 {
return 0
}
return n.Total(m.sizeMode)
}
// kidsOf returns the sorted children of id, cached until data changes.
func (m *Model) kidsOf(id inventory.NodeID) *kids {
if m.kidsVer != m.dataVer {
clear(m.kidsCache)
m.kidsVer = m.dataVer
}
if k, ok := m.kidsCache[id]; ok {
return k
}
k := &kids{}
m.tree.Children(id, func(c inventory.NodeID, _ *inventory.Node) {
if s := m.sizeOf(c); s > 0 {
k.ids = append(k.ids, c)
k.sizes = append(k.sizes, s)
k.total += s
}
})
sort.Sort(byKidSize{k})
m.kidsCache[id] = k
return k
}
type byKidSize struct{ *kids }
func (b byKidSize) Len() int { return len(b.ids) }
func (b byKidSize) Less(i, j int) bool {
if b.sizes[i] != b.sizes[j] {
return b.sizes[i] > b.sizes[j]
}
return b.ids[i] < b.ids[j]
}
func (b byKidSize) Swap(i, j int) {
b.ids[i], b.ids[j] = b.ids[j], b.ids[i]
b.sizes[i], b.sizes[j] = b.sizes[j], b.sizes[i]
}
// selectedNode returns the selected inventory node (NoNode for groups).
func (m *Model) selectedNode() inventory.NodeID {
if m.sel == noSel || isGroup(m.sel) || m.sel >= int64(m.tree.Len()) {
return inventory.NoNode
}
return inventory.NodeID(m.sel)
}
// ensureSelection keeps the selection meaningful after any change: if the
// selected node is no longer a direct, visible child of the zoom root it
// falls back to the visible ancestor, then the group containing it, then
// the largest child. Tree read lock held.
func (m *Model) ensureSelection() {
k := m.kidsOf(m.zoom)
if len(k.ids) == 0 {
m.sel = noSel
return
}
if !m.userSel {
// Until the user picks something, follow the largest block (it
// changes while a scan is running).
m.sel = int64(k.ids[0])
return
}
if isGroup(m.sel) {
if groupParent(m.sel) == m.zoom {
return
}
m.sel = noSel
}
id := m.selectedNode()
if id != inventory.NoNode {
// Walk up to the direct child of the zoom root.
for cur := id; cur != inventory.NoNode; cur = m.tree.Node(cur).Parent {
if m.tree.Node(cur).Parent == m.zoom {
if m.sizeOf(cur) > 0 {
m.sel = int64(cur)
return
}
break
}
}
}
m.sel = int64(k.ids[0])
}
// visibleSel maps the selection onto the treemap blocks: a node collapsed
// into the LOD group selects the group.
func (m *Model) visibleSel(blocks map[int64]bool) int64 {
if blocks[m.sel] {
return m.sel
}
if !isGroup(m.sel) && blocks[groupID(m.zoom)] {
return groupID(m.zoom)
}
return m.sel
}
// zoomTo makes dir the treemap root.
func (m *Model) zoomTo(dir inventory.NodeID) {
if dir == m.zoom {
return
}
prev := m.zoom
m.zoom = dir
// Coming back out, keep the directory we left selected.
if m.tree.IsAncestor(dir, prev) {
m.sel = int64(prev)
} else {
m.sel = noSel
}
m.userSel = m.sel != noSel
m.zoomed = true
m.dirty = true
}
func (m *Model) setView(i int) {
if i >= 0 && i < len(m.views) {
m.view = i
m.dirty = true
}
}
// colorOf returns the base colour for a block.
func (m *Model) colorOf(id int64) Color {
t := m.theme
if isGroup(id) {
return t.Group
}
nid := inventory.NodeID(id)
n := m.tree.Node(nid)
if d := m.opts.Diff; d != nil {
switch d.StatusOf(nid, m.sizeMode) {
case snapshot.Added:
return t.DiffNew
case snapshot.Grew:
old, _ := d.OldSize(nid, m.sizeMode)
return t.DiffSame.Mix(t.DiffGrow, ratio(n.Total(m.sizeMode)-old, old))
case snapshot.Shrank:
old, _ := d.OldSize(nid, m.sizeMode)
return t.DiffSame.Mix(t.DiffShrink, ratio(old-n.Total(m.sizeMode), old))
}
return t.DiffSame
}
if !n.IsDir() {
if t.Ext != nil {
if c, ok := t.Ext[m.tree.ExtName(n)]; ok {
return c
}
}
return t.Cat[n.Cat]
}
cs := m.tree.CatSizes(nid)
best, bestV := -1, int64(0)
for i, v := range cs {
if v > bestV {
best, bestV = i, v
}
}
if best < 0 {
return t.EmptyDir
}
return t.Cat[best]
}
// ratio maps a relative change to 0.35..1 for colour intensity.
func ratio(delta, base int64) float64 {
if base <= 0 {
return 1
}
r := float64(delta) / float64(base)
return min(1, 0.35+r*0.65)
}
internal/tui/infoview.go
-163
View File
@@ -1,163 +0,0 @@
package tui
import (
"fmt"
"runtime"
"strings"
tea "charm.land/bubbletea/v2"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/textutil"
"github.com/andydixon/mapsize/internal/treemap"
)
// infoView shows scan statistics and the error list.
type infoView struct{ tb table }
func (*infoView) name() string { return "Scan" }
func (v *infoView) focusErrors(m *Model) { v.tb.cursor = 0 }
func (v *infoView) paint(m *Model, cv *Canvas, r treemap.Rect) {
t := m.theme
cv.Fill(r, " ", t.base())
st := &m.tree.Stats
root := m.tree.Node(m.tree.Root())
lw := min(52, r.W/2)
if r.W < 100 {
lw = r.W - 2
}
y := r.Y + 1
x := r.X + 2
label := Style{FG: t.Muted, BG: t.Bg}
value := Style{FG: t.Fg, BG: t.Bg, Attr: Bold}
line := func(k, val string, vs Style) {
if y >= r.Y+r.H {
return
}
cv.Text(x, y, k, 20, label)
cv.Text(x+20, y, val, lw-20, vs)
y++
}
head := func(s string) {
if y >= r.Y+r.H-1 {
return
}
y++
cv.Text(x, y, s, lw, Style{FG: t.Accent, BG: t.Bg, Attr: Bold})
y++
}
state := "Complete"
stateSt := Style{FG: t.OK, BG: t.Bg, Attr: Bold}
switch {
case m.scanning:
state, stateSt.FG = "Scanning…", t.Accent
case st.Cancelled:
state, stateSt.FG = "Cancelled — totals incomplete", t.Warn
case st.Incomplete():
state, stateSt.FG = "Complete, with unreadable areas", t.Warn
}
cv.Text(x, y, "SCAN INFORMATION", lw, Style{FG: t.Accent, BG: t.Bg, Attr: Bold})
y++
line("Root", textutil.Sanitize(st.Root), value)
line("State", state, stateSt)
if st.FromSnapshot != "" {
line("Snapshot", textutil.Sanitize(st.FromSnapshot), value)
}
files := st.Files + st.Symlinks + st.Others
line("Items scanned", textutil.Count(files+st.Dirs), value)
line(" Files", textutil.Count(st.Files), value)
line(" Directories", textutil.Count(st.Dirs), value)
line(" Symlinks", textutil.Count(st.Symlinks), value)
line(" Special", textutil.Count(st.Others), value)
line("Logical size", fmt.Sprintf("%s (%s bytes)", textutil.Size(root.TotSize), textutil.Count(root.TotSize)), value)
line("Allocated size", fmt.Sprintf("%s (%s bytes)", textutil.Size(root.TotAlloc), textutil.Count(root.TotAlloc)), value)
el := st.Elapsed()
if st.FromSnapshot == "" {
line("Elapsed", textutil.Duration(el), value)
line("Rate", textutil.Count(int64(float64(files+st.Dirs)/max(el.Seconds(), 0.001)))+" items/s", value)
line("Workers", fmt.Sprintf("%d (%d CPUs)", st.Workers, runtime.NumCPU()), value)
}
head("ERRORS AND OMISSIONS")
anyIssue := false
for k, c := range st.ErrCounts {
if c > 0 {
line(inventory.ErrKind(k).String(), textutil.Count(c), Style{FG: t.Err, BG: t.Bg, Attr: Bold})
anyIssue = true
}
}
type row struct {
k string
v int64
}
for _, rr := range []row{{"Unscanned dirs", st.Unscanned}, {"Excluded", st.Excluded}, {"Other FS skipped", st.SkippedMounts},
{"Virtual FS skipped", st.VirtualSkipped}, {"Loops skipped", st.LoopsSkipped}, {"Broken symlinks", st.BrokenLinks},
{"Hard-link dups", st.HardlinkDups}} {
if rr.v > 0 {
line(rr.k, textutil.Count(rr.v), Style{FG: t.Warn, BG: t.Bg, Attr: Bold})
anyIssue = true
}
}
if !anyIssue {
line("None", "every entry was read", Style{FG: t.OK, BG: t.Bg})
}
head("OPTIONS")
excl := "none"
if len(st.Excludes) > 0 {
excl = textutil.Sanitize(strings.Join(st.Excludes, ", "))
}
line("Exclusions", excl, value)
line("One filesystem", fmt.Sprint(st.OneFileSystem), value)
line("Follow symlinks", textutil.Sanitize(st.Follow), value)
line("Size mode", m.sizeMode.String(), value)
units := "IEC (1024)"
if textutil.SI {
units = "SI (1000)"
}
line("Units", units, value)
if root.Flags&inventory.FlagAllocUnknown != 0 {
line("Note", "allocated size unavailable on this platform", Style{FG: t.Warn, BG: t.Bg})
}
// Error list on the right (or below in narrow terminals).
er := treemap.Rect{X: r.X + lw + 4, Y: r.Y, W: r.W - lw - 4, H: r.H}
if r.W < 100 {
er = treemap.Rect{X: r.X, Y: y + 1, W: r.W, H: r.Y + r.H - y - 1}
}
if er.W < 30 || er.H < 4 {
return
}
rows := make([]tableRow, len(st.Errors))
for i, e := range st.Errors {
p := m.tree.Path(e.Node)
if e.Name != "" {
p += "/" + e.Name
}
rows[i] = tableRow{cells: []string{e.Kind.String(), textutil.Sanitize(p), textutil.Sanitize(e.Msg)},
styles: []*Style{{FG: t.Err, BG: t.Bg}, nil, {FG: t.Muted, BG: t.Bg}}, bar: -1}
}
title := fmt.Sprintf("ERROR LIST (%s)", textutil.Count(st.TotalErrors()))
if int64(len(st.Errors)) < st.TotalErrors() {
title += fmt.Sprintf(" — first %s shown", textutil.Count(int64(len(st.Errors))))
}
v.tb.paint(m, cv, er, []column{{"KIND", 18, false, false}, {"PATH", 0, false, true}, {"DETAIL", 22, false, false}}, rows, title)
}
func (v *infoView) key(m *Model, k string) (bool, tea.Cmd) {
errs := m.tree.Stats.Errors
if v.tb.key(k, len(errs)) {
return true, nil
}
if v.tb.cursor < len(errs) && (k == "enter" || k == "space") {
m.jumpTo(errs[v.tb.cursor].Node)
return true, nil
}
return false, nil
}
func (v *infoView) wheel(m *Model, up bool) tea.Cmd {
v.tb.scroll(up, len(m.tree.Stats.Errors))
return nil
}
internal/tui/keys.go
-268
View File
@@ -1,268 +0,0 @@
package tui
import (
"fmt"
"os"
"path/filepath"
"time"
tea "charm.land/bubbletea/v2"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/platform"
"github.com/andydixon/mapsize/internal/textutil"
"github.com/andydixon/mapsize/internal/treemap"
)
// view is one of the Tab-cycled main views.
type view interface {
name() string
paint(m *Model, cv *Canvas, r treemap.Rect)
key(m *Model, k string) (bool, tea.Cmd)
wheel(m *Model, up bool) tea.Cmd
}
func keyName(k tea.KeyPressMsg) string {
switch s := k.String(); s {
case " ":
return "space"
default:
return s
}
}
// handleKey dispatches a key press. It holds the tree read lock for the
// whole handler; helpers it calls must not lock again (a recursive RLock
// can deadlock against a waiting writer).
func (m *Model) handleKey(msg tea.KeyPressMsg) tea.Cmd {
k := keyName(msg)
if k == "ctrl+c" {
return m.ctrlCPressed()
}
t := m.tree
t.RLock()
defer t.RUnlock()
m.ensureSelection()
if len(m.modals) > 0 {
return m.modalKey(msg)
}
if m.search.editing {
return m.searchKey(msg)
}
handled, cmd := m.views[m.view].key(m, k)
if handled {
return cmd
}
switch k {
case "q":
m.quit = true
case "?", "f1":
m.openModal(newHelpModal(m))
case "/":
m.search.begin()
case "esc":
if m.search.query != nil {
m.clearFilter()
} else if m.zoom != m.tree.Root() {
m.zoomOut()
}
case "tab":
m.setView((m.view + 1) % len(m.views))
case "shift+tab":
m.setView((m.view + len(m.views) - 1) % len(m.views))
case "1", "2", "3", "4", "5", "6", "7":
m.setView(int(k[0] - '1'))
case "enter":
return m.inspectSelection()
case "space", "right":
return m.zoomIntoSelection()
case "backspace", "left":
m.zoomOut()
case "home":
m.zoomTo(m.tree.Root())
case "a":
if m.sizeMode == inventory.SizeAllocated {
m.sizeMode = inventory.SizeLogical
} else {
m.sizeMode = inventory.SizeAllocated
}
m.invalidate()
var cmds []tea.Cmd
if m.search.query != nil {
cmds = append(cmds, m.applyFilter())
}
cmds = append(cmds, m.info("Sizing by "+m.sizeMode.String()+" size"))
return tea.Batch(cmds...)
case "r":
return m.rescan()
case "s":
m.openModal(newSaveModal(m))
case "e":
m.setViewByName("Scan")
if v, ok := m.views[m.view].(*infoView); ok {
v.focusErrors(m)
}
case "x":
m.setViewByName("Types")
case "g":
m.setViewByName("Top")
case "D":
return m.startDuplicates()
case "c":
return m.copySelected()
case "o":
return m.reveal()
case "d", "delete":
return m.requestTrash()
case "T":
names := ThemeNames()
cur := 0
for i, n := range names {
if n == m.theme.Name {
cur = i
}
}
ascii := m.theme.ASCII
m.theme = LoadTheme(names[(cur+1)%len(names)], m.opts.Settings)
m.theme.ASCII = ascii
return m.info("Theme: " + m.theme.Name)
}
return nil
}
func (m *Model) setViewByName(n string) {
for i, v := range m.views {
if v.name() == n {
m.setView(i)
}
}
}
func (m *Model) inspectSelection() tea.Cmd {
if isGroup(m.sel) {
if g, ok := m.groupInfo(m.sel); ok {
m.openModal(newGroupModal(m, g))
}
return nil
}
if id := m.selectedNode(); id != inventory.NoNode {
m.openModal(newInfoModal(m, id))
}
return nil
}
func (m *Model) zoomIntoSelection() tea.Cmd {
if isGroup(m.sel) {
// Groups cannot be zoomed: show their members in the list view.
if g, ok := m.groupInfo(m.sel); ok && len(g.ids) > 0 {
m.sel = int64(g.ids[0])
m.setViewByName("List")
}
return nil
}
id := m.selectedNode()
if id == inventory.NoNode {
return nil
}
if !m.tree.Node(id).IsDir() {
return m.info("Not a directory — Enter shows details")
}
if len(m.kidsOf(id).ids) == 0 {
return m.info("Directory is empty")
}
m.zoomTo(id)
return nil
}
func (m *Model) zoomOut() {
if p := m.tree.Node(m.zoom).Parent; p != inventory.NoNode {
m.zoomTo(p)
}
}
// jumpTo zooms to a node's parent directory and selects it.
func (m *Model) jumpTo(id inventory.NodeID) {
p := m.tree.Node(id).Parent
if p == inventory.NoNode {
m.zoomTo(id)
return
}
m.zoomTo(p)
m.sel = int64(id)
m.setViewByName("Map")
}
func (m *Model) selectedPath() (string, bool) {
id := m.selectedNode()
if id == inventory.NoNode {
return "", false
}
return m.tree.Path(id), true
}
func (m *Model) copySelected() tea.Cmd {
p, ok := m.selectedPath()
if !ok {
return nil
}
return m.copyPath(p)
}
// copyPath puts p on the clipboard (OSC 52). Paths that would not survive
// display unchanged are refused: a newline in a hostile filename runs
// commands when pasted into a shell, and bidi controls disguise them.
func (m *Model) copyPath(p string) tea.Cmd {
if textutil.Sanitize(p) != p {
return m.warn("Not copied: path contains control or bidi characters")
}
return tea.Batch(tea.SetClipboard(p), m.info("Copied path to clipboard (OSC 52)"))
}
func (m *Model) reveal() tea.Cmd {
if m.snapshot {
return m.warn("Snapshot — the files may not exist on this machine")
}
p, ok := m.selectedPath()
if !ok {
return nil
}
if platform.Headless() {
// No file manager: show the folder in the map instead.
id := m.selectedNode()
if m.tree.Node(id).IsDir() && len(m.kidsOf(id).ids) > 0 {
m.zoomTo(id)
m.setViewByName("Map")
} else {
m.jumpTo(id)
}
return m.info("Showing " + textutil.Sanitize(m.tree.Path(m.zoom)))
}
if err := platform.Reveal(p); err != nil {
return m.warn("Open failed: " + textutil.Sanitize(err.Error()))
}
return m.info("Opened in file manager")
}
func (m *Model) saveSnapshot(path string) tea.Cmd {
t := m.tree
return func() tea.Msg {
t.RLock()
err := saveFile(path, t)
t.RUnlock()
return saveDoneMsg{path, err}
}
}
type saveDoneMsg struct {
path string
err error
}
func defaultSnapshotName(root string) string {
base := filepath.Base(root)
if base == string(filepath.Separator) || base == "." || base == "" {
base = "root"
}
wd, _ := os.Getwd()
return filepath.Join(wd, fmt.Sprintf("%s-%s.msz", base, time.Now().Format("20060102-1504")))
}
internal/tui/live_test.go
-71
View File
@@ -1,71 +0,0 @@
package tui
import (
"context"
"fmt"
"os"
"path/filepath"
"testing"
"time"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/scan"
)
// TestLiveScanRendering renders, navigates, filters and resizes while the
// scanner mutates the inventory. Run with -race to verify the locking
// discipline.
func TestLiveScanRendering(t *testing.T) {
root := t.TempDir()
for i := range 30 {
d := filepath.Join(root, fmt.Sprintf("d%02d", i), "sub")
if err := os.MkdirAll(d, 0o755); err != nil {
t.Fatal(err)
}
for j := range 40 {
os.WriteFile(filepath.Join(d, fmt.Sprintf("f%d.log", j)), make([]byte, (i+1)*(j+1)*100), 0o644)
}
}
start := func(ctx context.Context) (*scan.Scanner, *inventory.Tree, error) {
s, err := scan.Start(ctx, root, scan.Options{Workers: 4, ChunkSize: 3})
if err != nil {
return nil, nil, err
}
return s, s.Tree, nil
}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
s, tr, err := start(ctx)
if err != nil {
t.Fatal(err)
}
m := newModel(Options{Start: start}, s, tr, cancel)
sizes := [][2]int{{120, 40}, {80, 24}, {200, 60}, {45, 12}}
deadline := time.After(20 * time.Second)
for i := 0; ; i++ {
send(m, size(sizes[i%len(sizes)][0], sizes[i%len(sizes)][1]), tickMsg(time.Now()))
send(m, key([]string{"right", "down", "left", "up"}[i%4]))
if i%5 == 0 {
send(m, key("tab"))
}
if i == 3 {
m.search.ed.set("ext = log")
m.queryChanged()
send(m, m.applyFilter()())
}
m.dirty = true
m.View()
select {
case <-s.Done():
send(m, scanDoneMsg{s})
m.View()
if m.scanning {
t.Fatal("scan should be finished")
}
return
case <-deadline:
t.Fatal("scan did not finish")
default:
}
}
}
internal/tui/mapview.go
-391
View File
@@ -1,391 +0,0 @@
package tui
import (
tea "charm.land/bubbletea/v2"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/textutil"
"github.com/andydixon/mapsize/internal/treemap"
)
// tmLayout is the treemap geometry for one (zoom, rect, data version). It is
// recomputed from scratch whenever any of those change — after a resize the
// layout algorithm runs again for the new rectangle rather than scaling the
// old one.
type tmLayout struct {
zoom inventory.NodeID
rect treemap.Rect
ver int
blocks []treemap.Block // top level: navigable
nested []nested // previews inside directories: drawn only
groups map[int64]*group
visible map[int64]bool
inner map[int64]bool // top-level blocks with nested previews
}
type nested struct {
id int64
r treemap.Rect
depth int
alt bool
inner bool // has nested children drawn inside it
}
type group struct {
parent inventory.NodeID
ids []inventory.NodeID
size int64
}
// LOD parameters.
const (
topMinArea = 12 // cells: room for a 4×3 framed block
topMaxItems = 400
nestMinArea = 3
nestMaxItems = 120
maxNestDepth = 8 // block-size thresholds stop recursion long before this
nestMinW = 4
nestMinH = 2
nestRecurseW = 14
nestRecurseH = 5
)
func (m *Model) layoutTreemap(r treemap.Rect) *tmLayout {
if l := m.tm; l != nil && l.zoom == m.zoom && l.rect == r && l.ver == m.dataVer {
return l
}
l := &tmLayout{zoom: m.zoom, rect: r, ver: m.dataVer, groups: map[int64]*group{}, visible: map[int64]bool{}, inner: map[int64]bool{}}
l.blocks = m.layoutLevel(l, m.zoom, r, topMinArea, topMaxItems)
for _, b := range l.blocks {
l.visible[b.ID] = true
if b.ID < 0 || b.W < 3 || b.H < 2 {
continue
}
if n := m.tree.Node(inventory.NodeID(b.ID)); n.IsDir() {
in := b.Inset(1)
if in.W >= nestMinW && in.H >= nestMinH {
before := len(l.nested)
m.layoutNested(l, m.chainEnd(inventory.NodeID(b.ID)), in, 2)
l.inner[b.ID] = len(l.nested) > before
}
}
}
m.tm = l
return l
}
// layoutLevel lays out the children of parent in r with level-of-detail
// grouping: children too small to be useful collapse into one group block.
func (m *Model) layoutLevel(l *tmLayout, parent inventory.NodeID, r treemap.Rect, minArea, maxItems int) []treemap.Block {
k := m.kidsOf(parent)
if len(k.ids) == 0 || r.Empty() {
return nil
}
sizes := make([]float64, len(k.sizes))
for i, s := range k.sizes {
sizes[i] = float64(s)
}
vis := treemap.Visible(sizes, float64(k.total), r.Area(), minArea, maxItems)
items := make([]treemap.Item, 0, vis+1)
for i := range vis {
items = append(items, treemap.Item{ID: int64(k.ids[i]), Size: sizes[i]})
}
if vis < len(k.ids) {
g := &group{parent: parent, ids: k.ids[vis:]}
for _, s := range k.sizes[vis:] {
g.size += s
}
gid := groupID(parent)
l.groups[gid] = g
items = append(items, treemap.Item{ID: gid, Size: float64(g.size)})
}
return treemap.Squarify(items, r)
}
func (m *Model) layoutNested(l *tmLayout, parent inventory.NodeID, r treemap.Rect, depth int) {
// Deeper levels need bigger blocks to stay legible rather than noisy.
blocks := m.layoutLevel(l, parent, r, nestMinArea*(depth-1), nestMaxItems)
for i, b := range blocks {
nb := nested{id: b.ID, r: b.Rect, depth: depth, alt: i%2 == 1}
idx := len(l.nested)
l.nested = append(l.nested, nb)
if depth < maxNestDepth && b.ID >= 0 && b.W >= nestRecurseW && b.H >= nestRecurseH &&
m.tree.Node(inventory.NodeID(b.ID)).IsDir() {
l.nested[idx].inner = true
m.layoutNested(l, m.chainEnd(inventory.NodeID(b.ID)), treemap.Rect{X: b.X, Y: b.Y + 1, W: b.W - 1, H: b.H - 2}, depth+1)
}
}
}
// chainEnd follows directories whose only non-empty child is a directory
// (mnt → raid → media), so nesting spends its space on the level that
// actually branches instead of on full-size wrappers.
func (m *Model) chainEnd(id inventory.NodeID) inventory.NodeID {
for {
k := m.kidsOf(id)
if len(k.ids) != 1 || !m.tree.Node(k.ids[0]).IsDir() {
return id
}
id = k.ids[0]
}
}
// groupInfo returns the LOD group for a group ID in the current layout.
func (m *Model) groupInfo(id int64) (*group, bool) {
if m.tm == nil {
return nil, false
}
g, ok := m.tm.groups[id]
return g, ok
}
type mapView struct{}
func (*mapView) name() string { return "Map" }
func (v *mapView) paint(m *Model, cv *Canvas, r treemap.Rect) {
t := m.theme
l := m.layoutTreemap(r)
if len(l.blocks) == 0 {
msg := "Empty directory"
switch {
case m.search.query != nil:
msg = "No matches here — Esc clears the filter"
case m.scanning:
msg = "Scanning…"
case !m.tree.Node(m.zoom).IsDir():
msg = textutil.Sanitize(m.tree.Node(m.zoom).Name) + " — " + textutil.Size(m.sizeOf(m.zoom))
}
cv.TextCentered(r.X, r.Y+r.H/2, r.W, msg, t.muted())
return
}
sel := m.visibleSel(l.visible)
total := m.sizeOf(m.zoom)
for _, b := range l.blocks {
m.paintTop(cv, b, b.ID == sel, b.ID == m.hover, total)
}
for _, nb := range l.nested {
m.paintNested(cv, nb)
}
for _, b := range l.blocks {
id := b.ID
m.hits = append(m.hits, hit{b.Rect, func(m *Model, double bool) tea.Cmd { return m.clickBlock(id, double) }})
}
}
// blockName returns a sanitised display name for a block.
func (m *Model) blockName(id int64) string {
if isGroup(id) {
if g, ok := m.groupInfo(id); ok {
return textutil.Count(int64(len(g.ids))) + " smaller items"
}
return "smaller items"
}
nid := inventory.NodeID(id)
n := m.tree.Node(nid)
name := textutil.Sanitize(n.Name)
if n.IsDir() && nid != m.tree.Root() {
name += "/"
for end := m.chainEnd(nid); nid != end; {
nid = m.kidsOf(nid).ids[0]
name += textutil.Sanitize(m.tree.Node(nid).Name) + "/"
}
}
return name
}
func (m *Model) blockSize(id int64) int64 {
if isGroup(id) {
if g, ok := m.groupInfo(id); ok {
return g.size
}
return 0
}
return m.sizeOf(inventory.NodeID(id))
}
// parentSize is the size of the folder a block sits in.
func (m *Model) parentSize(id int64) int64 {
if isGroup(id) {
if g, ok := m.groupInfo(id); ok {
return m.sizeOf(g.parent)
}
return 0
}
if p := m.tree.Node(inventory.NodeID(id)).Parent; p != inventory.NoNode {
return m.sizeOf(p)
}
return 0
}
func (m *Model) paintTop(cv *Canvas, b treemap.Block, sel, hov bool, total int64) {
t := m.theme
base := m.colorOf(b.ID)
fill := base.Mix(t.Bg, 0.62)
fillCh := " "
if isGroup(b.ID) {
fillCh = "░"
if t.ASCII {
fillCh = "."
}
}
name := m.blockName(b.ID)
size := m.blockSize(b.ID)
if b.W < 3 || b.H < 2 {
// Too small for a frame: a solid tile.
st := Style{FG: base, BG: base.Mix(t.Bg, 0.3)}
ch := fillCh
if sel {
st = Style{FG: t.textOn(t.Sel), BG: t.Sel, Attr: Bold | m.monoRev()}
ch = "◆"
if t.ASCII {
ch = "#"
}
} else if t.Mono {
ch = "▒"
}
cv.Fill(b.Rect, " ", st)
cv.Text(b.X+(b.W-1)/2, b.Y+(b.H-1)/2, ch, 1, st)
return
}
cv.Fill(b.Rect, fillCh, Style{FG: base.Mix(t.Bg, 0.35), BG: fill})
frame := Style{FG: base.Mix(t.Bg, 0.15), BG: fill}
box := t.box()
title := Style{FG: base.Mix(RGB(255, 255, 255), 0.35), BG: fill, Attr: Bold}
switch {
case sel:
box = t.selBox()
frame = Style{FG: t.Sel, BG: fill, Attr: Bold}
title = Style{FG: t.textOn(t.Sel), BG: t.Sel, Attr: Bold | m.monoRev()}
case hov:
frame.FG = base.Mix(RGB(255, 255, 255), 0.5)
}
if t.Mono && !sel {
title.Attr = Bold
}
cv.Box(b.Rect, box, frame)
cv.Text(b.X+1, b.Y, textutil.Truncate(" "+name+" ", b.W-2), b.W-2, title)
label := textutil.Size(size)
if pct := textutil.Percent(size, total); b.W-2 >= textutil.Width(label)+textutil.Width(pct)+5 {
label += " · " + pct
} else if b.W-2 < textutil.Width(label)+2 {
label = textutil.SizeCompact(size)
}
label = " " + label + " "
if b.H >= 3 && textutil.Width(label) <= b.W-2 {
cv.TextRight(b.X+1, b.Y+b.H-1, b.W-2, label, Style{FG: frame.FG, BG: fill, Attr: frame.Attr})
}
// Interior labels for blocks without nested previews.
in := b.Inset(1)
if in.Empty() || m.hasNested(b.ID) {
return
}
n := inventory.NoNode
if !isGroup(b.ID) {
n = inventory.NodeID(b.ID)
}
lines := []string{}
if n != inventory.NoNode {
nd := m.tree.Node(n)
if !nd.IsDir() {
lines = append(lines, kindLabel(m.tree, nd))
} else if nd.Files > 0 {
lines = append(lines, textutil.Count(int64(nd.Files))+" files")
}
}
if b.H < 4 || in.W < 6 {
return
}
y0 := in.Y + (in.H-len(lines))/2
for i, s := range lines {
cv.TextCentered(in.X, y0+i, in.W, s, Style{FG: base.Mix(t.Fg, 0.4), BG: fill})
}
}
func (m *Model) hasNested(id int64) bool { return m.tm != nil && m.tm.inner[id] }
func (m *Model) paintNested(cv *Canvas, nb nested) {
t := m.theme
base := m.colorOf(nb.id)
f := max(0.38-0.1*float64(nb.depth-2), 0.1)
if nb.alt {
f += 0.07
}
bg := base.Mix(t.Bg, f)
shadow := bg.Mix(RGB(0, 0, 0), 0.45)
st := Style{FG: t.textOn(bg), BG: bg}
ch := " "
if isGroup(nb.id) {
ch = "░"
if t.ASCII {
ch = "."
}
st.FG = bg.Mix(t.Fg, 0.25)
}
r := nb.r
cv.Fill(r, ch, st)
if t.Mono {
// No colour: draw a light outline so neighbours stay distinguishable.
if r.W >= 2 && r.H >= 2 {
cv.Box(r, BoxRound, Style{Attr: Faint})
}
} else {
if r.W >= 4 {
cv.Fill(treemap.Rect{X: r.X + r.W - 1, Y: r.Y, W: 1, H: r.H}, " ", Style{BG: shadow})
}
if r.H >= 3 {
cv.Fill(treemap.Rect{X: r.X, Y: r.Y + r.H - 1, W: r.W, H: 1}, " ", Style{BG: shadow})
}
}
lw := r.W - 1
if t.Mono {
lw = r.W - 2
}
if lw < 5 {
return
}
lx := r.X
if t.Mono {
lx++
}
ly := r.Y
if t.Mono && r.H >= 3 {
ly++
}
name := m.blockName(nb.id)
if textutil.Width(name) > lw && lw < 8 {
return // a stub like "te…" is noise, not information
}
cv.Text(lx, ly, textutil.Truncate(name, lw), lw, Style{FG: st.FG, BG: bg, Attr: Bold})
if !nb.inner && r.H >= 3 && lw >= 5 {
size := m.blockSize(nb.id)
label := textutil.SizeCompact(size)
if pct := textutil.Percent(size, m.parentSize(nb.id)); textutil.Width(label)+textutil.Width(pct)+3 <= lw {
label += " · " + pct
}
cv.Text(lx, ly+1, textutil.Truncate(label, lw), lw, Style{FG: bg.Mix(st.FG, 0.7), BG: bg})
}
}
func (v *mapView) key(m *Model, k string) (bool, tea.Cmd) {
dirs := map[string]treemap.Direction{"left": treemap.Left, "right": treemap.Right, "up": treemap.Up, "down": treemap.Down,
"h": treemap.Left, "l": treemap.Right, "k": treemap.Up, "j": treemap.Down}
if d, ok := dirs[k]; ok {
if l := m.tm; l != nil {
if id, ok := treemap.Neighbour(l.blocks, m.visibleSel(l.visible), d); ok {
m.sel = id
}
}
return true, nil
}
return false, nil
}
func (v *mapView) wheel(m *Model, up bool) tea.Cmd {
if up {
return m.zoomIntoSelection()
}
m.zoomOut()
return nil
}
internal/tui/modal.go
-221
View File
@@ -1,221 +0,0 @@
package tui
import (
"strings"
tea "charm.land/bubbletea/v2"
"github.com/andydixon/mapsize/internal/textutil"
"github.com/andydixon/mapsize/internal/treemap"
)
// span is a run of styled text; a line is a slice of spans.
type span struct {
s string
st *Style // nil = modal default
}
type line []span
func txt(s string) line { return line{{s, nil}} }
func styled(s string, st Style) line { return line{{s, &st}} }
func kv(k, v string, kst, vst Style) line { return line{{textutil.PadRight(k, 18), &kst}, {v, &vst}} }
// modal is an overlay. Modals receive all keys while open; Esc always
// closes them (handled by the frame before the modal sees it, unless the
// modal is editing text).
type modal interface {
title() string
content(m *Model, width int) []line // rebuilt every frame, so resizes reflow
footer(m *Model) string
key(m *Model, k string) (closeModal bool, cmd tea.Cmd)
width() int // preferred inner width
}
// modalFrame wraps a modal with scroll state.
type modalFrame struct {
modal
scroll int
viewH int // content rows visible in the last paint
contentN int
}
func (m *Model) openModal(md modal) {
m.modals = append(m.modals, &modalFrame{modal: md})
m.dirty = true
}
func (m *Model) closeModal() {
if len(m.modals) > 0 {
m.modals = m.modals[:len(m.modals)-1]
}
m.dirty = true
}
// textInputModal is implemented by modals that consume printable keys.
type textInputModal interface {
editing() bool
editor() *lineEditor
}
func (m *Model) modalKey(msg tea.KeyPressMsg) tea.Cmd {
k := keyName(msg)
f := m.modals[len(m.modals)-1].(*modalFrame)
if ti, ok := f.modal.(textInputModal); ok && ti.editing() && k != "esc" && k != "enter" {
ti.editor().key(msg)
return nil
}
if ti, ok := f.modal.(textInputModal); !ok || !ti.editing() {
page := max(f.viewH-1, 1)
switch k {
case "esc", "q":
m.closeModal()
return nil
case "up", "k":
f.scroll--
case "down", "j":
f.scroll++
case "pgup":
f.scroll -= page
case "pgdown", "space":
f.scroll += page
case "home":
f.scroll = 0
case "end":
f.scroll = f.contentN
default:
goto modalSpecific
}
f.clampScroll()
return nil
}
modalSpecific:
closeIt, cmd := f.key(m, k)
if closeIt {
// Close this modal specifically (it may have opened another).
for i, x := range m.modals {
if x == f {
m.modals = append(m.modals[:i], m.modals[i+1:]...)
break
}
}
m.dirty = true
}
return cmd
}
func (f *modalFrame) clampScroll() {
f.scroll = max(0, min(f.scroll, f.contentN-f.viewH))
}
// paintModal dims the base screen and draws the top modal centred,
// clamped to the terminal with a one-cell margin. Content that does not fit
// scrolls.
func (m *Model) paintModal(cv *Canvas) {
t := m.theme
// Dim everything underneath.
cv.Restyle(treemap.Rect{X: 0, Y: 0, W: cv.W, H: cv.H}, func(s Style) Style {
if t.Mono {
s.Attr |= Faint
return s
}
s.FG = s.FG.Mix(t.Bg, 0.6)
s.BG = s.BG.Mix(RGB(0, 0, 0), 0.45)
s.Attr &^= Bold
return s
})
f := m.modals[len(m.modals)-1].(*modalFrame)
maxW := cv.W - 2
w := min(f.width()+4, maxW)
w = max(w, min(30, maxW))
inner := w - 4
lines := f.content(m, inner)
foot := f.footer(m)
chrome := 4 // border top/bottom, title rule
if foot != "" {
chrome += 2
}
h := min(len(lines)+chrome, cv.H-2)
h = max(h, min(chrome+1, cv.H))
x0, y0 := (cv.W-w)/2, (cv.H-h)/2
r := treemap.Rect{X: x0, Y: y0, W: w, H: h}
bg := t.PanelBg
if bg == 0 {
bg = t.Bg
}
base := Style{FG: t.Fg, BG: bg}
// Drop shadow.
if !t.Mono {
cv.Restyle(treemap.Rect{X: x0 + 1, Y: y0 + h, W: w, H: 1}, func(s Style) Style { s.BG = RGB(0, 0, 0); return s })
cv.Restyle(treemap.Rect{X: x0 + w, Y: y0 + 1, W: 1, H: h}, func(s Style) Style { s.BG = RGB(0, 0, 0); return s })
}
cv.Fill(r, " ", base)
border := Style{FG: t.Accent, BG: bg, Attr: Bold}
box := BoxRound
if t.ASCII {
box = BoxASCII
}
cv.Box(r, box, border)
cv.Text(x0+2, y0+1, strings.ToUpper(f.title()), inner, Style{FG: t.Accent, BG: bg, Attr: Bold})
rule := "─"
if t.ASCII {
rule = "-"
}
cv.HLine(x0+1, y0+2, w-2, rule, Style{FG: t.Faint, BG: bg})
f.viewH = h - chrome
f.contentN = len(lines)
f.clampScroll()
for i := 0; i < f.viewH && f.scroll+i < len(lines); i++ {
x := x0 + 2
for _, sp := range lines[f.scroll+i] {
st := base
if sp.st != nil {
st = *sp.st
if st.BG == 0 {
st.BG = bg
}
}
x += cv.Text(x, y0+3+i, sp.s, x0+2+inner-x, st)
}
}
if len(lines) > f.viewH && f.viewH > 0 {
ind := textutil.Count(int64(f.scroll+1)) + "–" + textutil.Count(int64(min(f.scroll+f.viewH, len(lines)))) +
"/" + textutil.Count(int64(len(lines)))
if f.scroll > 0 {
ind = "▲ " + ind
}
if f.scroll+f.viewH < len(lines) {
ind += " ▼"
}
cv.TextRight(x0+1, y0+1, w-3, ind, Style{FG: t.Muted, BG: bg})
}
if foot != "" {
cv.HLine(x0+1, y0+h-3, w-2, rule, Style{FG: t.Faint, BG: bg})
cv.Text(x0+2, y0+h-2, foot, inner, Style{FG: t.Muted, BG: bg})
}
m.hits = append(m.hits[:0], hit{r, func(*Model, bool) tea.Cmd { return nil }})
}
// wrap breaks s into chunks of at most w cells (used for long paths).
func wrap(s string, w int) []string {
if w <= 0 {
return nil
}
var out []string
var cur strings.Builder
cw := 0
textutil.Clusters(s, func(c string, cwid int) bool {
if cw+cwid > w {
out = append(out, cur.String())
cur.Reset()
cw = 0
}
cur.WriteString(c)
cw += cwid
return true
})
if cur.Len() > 0 || len(out) == 0 {
out = append(out, cur.String())
}
return out
}
internal/tui/modals.go
-391
View File
@@ -1,391 +0,0 @@
package tui
import (
"fmt"
"io/fs"
"strings"
"time"
tea "charm.land/bubbletea/v2"
"github.com/andydixon/mapsize/internal/export"
"github.com/andydixon/mapsize/internal/filter"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/platform"
"github.com/andydixon/mapsize/internal/textutil"
)
// ---- Item information -------------------------------------------------------
type infoModal struct {
id inventory.NodeID
btime int64 // live birth time (0 if unknown)
live bool // metadata re-read from the filesystem
}
func newInfoModal(m *Model, id inventory.NodeID) modal {
md := &infoModal{id: id}
if !m.snapshot {
// Birth time is not kept in the inventory; read it live.
if meta, err := platform.Lstat(m.tree.Path(id)); err == nil {
md.btime, md.live = meta.BTime, true
}
}
return md
}
func (*infoModal) width() int { return 72 }
func (md *infoModal) title() string { return "Item information" }
func (md *infoModal) footer(m *Model) string {
s := "Esc close · c copy path · o " + revealVerb() + " · Space go to"
if !m.opts.ReadOnly && !m.snapshot {
s += " · d " + deleteVerb()
}
return s
}
func (md *infoModal) content(m *Model, w int) []line {
t := m.theme
tr := m.tree
n := tr.Node(md.id)
ks := Style{FG: t.Muted}
vs := Style{FG: t.Fg}
bold := Style{FG: t.Fg, Attr: Bold}
warn := Style{FG: t.Warn, Attr: Bold}
var out []line
add := func(k, v string) { out = append(out, kv(k, v, ks, vs)) }
addS := func(k, v string, st Style) { out = append(out, kv(k, v, ks, st)) }
blank := func() { out = append(out, txt("")) }
col := m.colorOf(int64(md.id))
title := Style{FG: col.Mix(RGB(255, 255, 255), 0.3), Attr: Bold}
for _, l := range wrap(textutil.Sanitize(n.Name), w) {
out = append(out, styled(l, title))
}
blank()
out = append(out, styled("Path", ks))
for _, l := range wrap(textutil.Sanitize(tr.Path(md.id)), w) {
out = append(out, styled(l, vs))
}
if textutil.Sanitize(n.Name) != n.Name {
out = append(out, styled("Name contains control or invalid characters (shown escaped)", warn))
}
blank()
if n.IsDir() {
addS("Total logical", fmt.Sprintf("%s (%s bytes)", textutil.Size(n.TotSize), textutil.Count(n.TotSize)), bold)
addS("Total allocated", fmt.Sprintf("%s (%s bytes)", textutil.Size(n.TotAlloc), textutil.Count(n.TotAlloc)), bold)
add("Files", textutil.Count(int64(n.Files)))
add("Directories", textutil.Count(int64(n.Dirs)))
if p := n.Parent; p != inventory.NoNode {
add("Of parent", textutil.Percent(n.Total(m.sizeMode), tr.Node(p).Total(m.sizeMode)))
}
add("Of scan", textutil.Percent(n.Total(m.sizeMode), tr.Node(0).Total(m.sizeMode)))
k := m.kidsOf(md.id)
if len(k.ids) > 0 {
add("Largest child", fmt.Sprintf("%s (%s)", textutil.Sanitize(tr.Node(k.ids[0]).Name), textutil.Size(k.sizes[0])))
}
if n.Errors > 0 {
addS("Errors beneath", textutil.Count(int64(n.Errors)), Style{FG: t.Err, Attr: Bold})
}
state := "complete"
switch {
case n.Flags&inventory.FlagIncomplete != 0:
state = "incomplete (not fully read)"
case n.Flags&inventory.FlagScanned == 0 && m.scanning:
state = "scanning…"
case n.Flags&(inventory.FlagSkippedFS|inventory.FlagVirtualFS|inventory.FlagLoop) != 0:
state = "not descended"
}
add("Scan state", state)
} else {
addS("Logical size", fmt.Sprintf("%s (%s bytes)", textutil.Size(n.Size), textutil.Count(n.Size)), bold)
alloc := fmt.Sprintf("%s (%s bytes)", textutil.Size(n.Alloc), textutil.Count(n.Alloc))
if n.Flags&inventory.FlagAllocUnknown != 0 {
alloc = "unknown on this platform"
}
addS("Allocated size", alloc, bold)
if p := n.Parent; p != inventory.NoNode {
add("Of parent", textutil.Percent(n.Own(m.sizeMode), tr.Node(p).Total(m.sizeMode)))
}
}
if r := m.search.result; r != nil {
add("Matching filter", textutil.Size(r.Size(md.id)))
}
if d := m.opts.Diff; d != nil {
if old, ok := d.OldSize(md.id, m.sizeMode); ok {
add("In old snapshot", textutil.Size(old))
addS("Change", textutil.SignedSize(n.Total(m.sizeMode)-old), bold)
} else {
addS("Change", "new since old snapshot", Style{FG: t.DiffNew, Attr: Bold})
}
}
blank()
add("Type", n.Kind.String())
if !n.IsDir() {
if e := tr.ExtName(n); e != "" {
add("Extension", textutil.Sanitize(e))
}
add("Category", n.Cat.String())
}
blank()
if n.MTime > 0 {
add("Modified", fmtTime(n.MTime))
}
if md.btime > 0 {
add("Created", fmtTime(md.btime))
}
blank()
add("Mode", fs.FileMode(n.Mode).String())
add("Owner", textutil.Sanitize(platform.UserName(n.UID)))
add("Group", textutil.Sanitize(platform.GroupName(n.GID)))
if !n.IsDir() {
add("Hard links", fmt.Sprint(n.Nlink))
sparse := "No"
if n.Flags&inventory.FlagSparse != 0 {
sparse = fmt.Sprintf("Yes (or compressed) — %s not allocated", textutil.Size(n.Size-n.Alloc))
}
add("Sparse", sparse)
}
if fl := export.FlagList(n.Flags &^ (inventory.FlagScanned | inventory.FlagSparse)); len(fl) > 0 {
addS("Flags", strings.Join(fl, ", "), warn)
}
if n.Flags&inventory.FlagHardlinkDup != 0 {
out = append(out, styled("Another path to this inode was counted; this one adds nothing to totals.", Style{FG: t.Muted}))
}
if m.snapshot {
out = append(out, txt(""), styled("From snapshot — metadata as recorded, not live.", Style{FG: t.Muted}))
}
return out
}
func fmtTime(ns int64) string { return time.Unix(0, ns).Format("2006-01-02 15:04:05") }
func (md *infoModal) key(m *Model, k string) (bool, tea.Cmd) {
switch k {
case "c":
return false, m.copyPath(m.tree.Path(md.id))
case "o":
m.sel = int64(md.id)
// Headless, o navigates the map, which the modal would cover.
return platform.Headless(), m.reveal()
case "enter":
return true, nil
case "g", "right":
m.jumpTo(md.id)
return true, nil
case "d", "delete":
m.sel = int64(md.id)
return true, m.requestTrash()
}
return false, nil
}
// ---- LOD group ------------------------------------------------------------------
type groupModal struct{ g *group }
func newGroupModal(_ *Model, g *group) modal { return &groupModal{g} }
func (*groupModal) width() int { return 70 }
func (*groupModal) title() string { return "Smaller items (grouped)" }
func (*groupModal) footer(*Model) string { return "Esc close · Space shows them in the list view" }
func (md *groupModal) content(m *Model, w int) []line {
t := m.theme
out := []line{
txt(fmt.Sprintf("%s items too small to draw individually at this size,", textutil.Count(int64(len(md.g.ids))))),
txt(fmt.Sprintf("totalling %s. Zoom in or enlarge the terminal to see them.", textutil.Size(md.g.size))),
txt(""),
}
for i, id := range md.g.ids {
if i == 200 {
out = append(out, styled(fmt.Sprintf("… and %s more", textutil.Count(int64(len(md.g.ids)-200))), Style{FG: t.Muted}))
break
}
n := m.tree.Node(id)
name := textutil.Sanitize(n.Name)
if n.IsDir() {
name += "/"
}
out = append(out, line{{textutil.PadLeft(textutil.Size(m.sizeOf(id)), 11) + " ", &Style{FG: t.Muted}},
{textutil.Truncate(name, w-13), &Style{FG: m.colorOf(int64(id)).Mix(t.Fg, 0.4)}}})
}
return out
}
func (md *groupModal) key(m *Model, k string) (bool, tea.Cmd) {
if k == "enter" {
m.sel = int64(md.g.ids[0])
m.setViewByName("List")
return true, nil
}
return false, nil
}
// ---- Help ---------------------------------------------------------------------
type helpModal struct{}
func newHelpModal(*Model) modal { return helpModal{} }
func (helpModal) width() int { return 78 }
func (helpModal) title() string { return "Keyboard reference" }
func (helpModal) footer(*Model) string { return "Esc close · ↑↓ scroll" }
func (helpModal) content(m *Model, _ int) []line {
t := m.theme
h := Style{FG: t.Accent, Attr: Bold}
k := Style{FG: t.Sel, Attr: Bold}
d := Style{FG: t.Fg}
var out []line
sec := func(s string) {
if len(out) > 0 {
out = append(out, txt(""))
}
out = append(out, styled(s, h))
}
row := func(keys, desc string) { out = append(out, line{{" " + textutil.PadRight(keys, 19), &k}, {desc, &d}}) }
sec("NAVIGATION")
row("↑ ↓ ← → hjkl", "move spatially through the treemap")
row("Enter", "inspect the selected item")
row("Space →", "zoom into the selected directory")
row("Backspace ←", "zoom out one level")
row("Home", "back to the scan root")
row("click / dbl-click", "select / zoom (mouse)")
row("wheel", "zoom in/out (map), scroll (lists)")
sec("SEARCH & FILTER")
row("/", "search or filter; the map shows only matches")
row("Esc", "clear the filter")
out = append(out, styled(" ubuntu *.iso size > 5GB ext IN (iso,qcow2) age > 365d", Style{FG: t.Muted}))
out = append(out, styled(" path contains cache AND NOT type = dir owner = andy flag = sparse", Style{FG: t.Muted}))
fields := strings.Fields(filter.Fields)
out = append(out, styled(" fields: "+strings.Join(fields[:7], " "), Style{FG: t.Muted}))
out = append(out, styled(" "+strings.Join(fields[7:], " "), Style{FG: t.Muted}))
out = append(out, styled(" sizes: KiB/MiB/GiB (1024), kB/MB/GB (1000); ages: s min h d w mo y", Style{FG: t.Muted}))
sec("VIEWS")
row("Tab Shift+Tab", "next / previous view")
row("1 – 7", "jump to a view")
row("x", "file types (extension statistics)")
row("g", "top lists: largest, oldest, sparse, hard links …")
row("e", "scan information and errors")
row("D", "find duplicate files (reads file contents)")
sec("APPLICATION")
row("a", "toggle allocated / logical (apparent) size")
row("c", "copy path to clipboard")
row("o", revealVerb())
if !m.opts.ReadOnly {
row("d", deleteVerb()+" (asks first)")
}
row("s", "save snapshot")
row("r", "rescan")
row("T", "cycle colour theme")
row("Ctrl+C", "cancel scan; press again to quit")
row("q", "quit")
out = append(out, txt(""))
units := "IEC (KiB = 1024 bytes)"
if textutil.SI {
units = "SI (kB = 1000 bytes)"
}
out = append(out, styled("Sizes: "+m.sizeMode.String()+" · units: "+units, Style{FG: t.Muted}))
return out
}
func (helpModal) key(*Model, string) (bool, tea.Cmd) { return false, nil }
// ---- Confirmation -------------------------------------------------------------
type confirmModal struct {
ttl string
body []string
yes string
onYes func(m *Model) tea.Cmd
}
func (c *confirmModal) width() int {
w := 40
for _, b := range c.body {
w = max(w, textutil.Width(b))
}
return min(w, 90)
}
func (c *confirmModal) title() string { return c.ttl }
func (c *confirmModal) footer(*Model) string { return "y " + c.yes + " · n / Esc cancel" }
func (c *confirmModal) content(m *Model, w int) []line {
var out []line
for _, b := range c.body {
for _, l := range wrap(b, w) {
out = append(out, txt(l))
}
}
return out
}
func (c *confirmModal) key(m *Model, k string) (bool, tea.Cmd) {
switch k {
case "y", "Y":
return true, c.onYes(m)
case "n", "N", "enter":
return true, nil
}
return false, nil
}
// ---- Save snapshot ---------------------------------------------------------------
type saveModal struct{ ed lineEditor }
func newSaveModal(m *Model) modal {
md := &saveModal{}
md.ed.set(defaultSnapshotName(m.tree.Stats.Root))
return md
}
func (*saveModal) editing() bool { return true }
func (md *saveModal) editor() *lineEditor { return &md.ed }
func (*saveModal) width() int { return 70 }
func (*saveModal) title() string { return "Save snapshot" }
func (*saveModal) footer(*Model) string { return "Enter save · Esc cancel" }
func (md *saveModal) content(m *Model, w int) []line {
t := m.theme
out := []line{txt("File:")}
s := textutil.Sanitize(md.ed.String())
for _, l := range wrap(s+"█", w) {
out = append(out, styled(l, Style{FG: t.Fg, Attr: Bold}))
}
if m.scanning {
out = append(out, txt(""), styled("The scan is still running: the snapshot will be marked incomplete.", Style{FG: t.Warn}))
}
return out
}
func (md *saveModal) key(m *Model, k string) (bool, tea.Cmd) {
switch k {
case "esc":
return true, nil
case "enter":
p := strings.TrimSpace(md.ed.String())
if p == "" {
return false, nil
}
return true, tea.Batch(m.saveSnapshot(p), m.info("Saving snapshot…"))
}
return false, nil
}
// revealVerb and deleteVerb describe o and d, which change meaning on a
// headless system.
func revealVerb() string {
if platform.Headless() {
return "show folder in map"
}
return "reveal in file manager"
}
func deleteVerb() string {
if platform.Headless() {
return "delete permanently"
}
return "move to " + platform.TrashName
}
internal/tui/model.go
-447
View File
@@ -1,447 +0,0 @@
// Package tui is the interactive terminal interface. All drawing goes to a
// Canvas; Bubble Tea only transports the finished frame.
//
// Threading: Update and View run on Bubble Tea's event goroutine. The scan
// controller mutates the inventory concurrently, so every read of tree data
// happens under tree.RLock (taken once per frame in View, and briefly in key
// handlers). The model itself is only touched on the event goroutine.
package tui
import (
"context"
"fmt"
"log/slog"
"os"
"path/filepath"
"strings"
"time"
tea "charm.land/bubbletea/v2"
"github.com/charmbracelet/colorprofile"
"github.com/andydixon/mapsize/internal/brand"
"github.com/andydixon/mapsize/internal/config"
"github.com/andydixon/mapsize/internal/filter"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/scan"
"github.com/andydixon/mapsize/internal/snapshot"
"github.com/andydixon/mapsize/internal/textutil"
"github.com/andydixon/mapsize/internal/treemap"
)
// Options configures the interactive session.
type Options struct {
ReadOnly bool
Theme string
SizeMode inventory.SizeMode
Mouse bool
Color string // auto, truecolor, 256, 16, none
Save string // save a snapshot here when the scan completes
Settings config.Settings
Diff *snapshot.Diff // compare mode
// Start begins a scan (or returns a loaded tree with a nil scanner).
Start func(ctx context.Context) (*scan.Scanner, *inventory.Tree, error)
}
// Messages.
type (
tickMsg time.Time
scanDoneMsg struct{ s *scan.Scanner }
fatalMsg struct {
v any
stack []byte
}
filterMsg struct {
gen int
res *filter.Result
err error
}
filterDebounceMsg struct{ gen int }
)
// Model is the Bubble Tea model.
type Model struct {
opts Options
theme *Theme
scanner *scan.Scanner
tree *inventory.Tree
cancel context.CancelFunc
scanning bool
snapshot bool // tree was loaded from a snapshot (not the live filesystem)
ctrlC bool // a Ctrl+C already cancelled the scan
w, h int
profile colorprofile.Profile
forced bool // profile forced by --color
sizeMode inventory.SizeMode
views []view
view int
zoom inventory.NodeID
sel int64 // selected block: NodeID, or groupID(parent) for an LOD group
// userSel is set once the user has chosen a selection; before that the
// selection follows the largest block.
userSel bool
zoomed bool // zoom changed during this message (zoomTo set userSel)
search searchState
modals []modal
toast string
toastStyle Style
toastUntil time.Time
// Frame and data caches.
dirty bool
frame string
dataVer int // bumped when sizes/filters change; invalidates caches
lastData time.Time // last time dataVer advanced during a scan
kidsCache map[inventory.NodeID]*kids
kidsVer int
tm *tmLayout
hits []hit // clickable regions from the last frame
hover int64
lastClick time.Time
lastClickI int64
ticking bool
pendingZoom string // path to re-zoom after a rescan
dups *dupState
fatal *fatalMsg
quit bool
}
// hit is a clickable region recorded while painting.
type hit struct {
r treemap.Rect
fn func(m *Model, double bool) tea.Cmd
}
func groupID(parent inventory.NodeID) int64 { return -1 - int64(parent) }
func isGroup(id int64) bool { return id < 0 }
func groupParent(id int64) inventory.NodeID { return inventory.NodeID(-1 - id) }
func newModel(opts Options, s *scan.Scanner, t *inventory.Tree, cancel context.CancelFunc) *Model {
m := &Model{
opts: opts, scanner: s, tree: t, cancel: cancel, scanning: s != nil, snapshot: s == nil,
sizeMode: opts.SizeMode, dirty: true, kidsCache: map[inventory.NodeID]*kids{}, hover: noSel,
lastClickI: noSel,
}
name := opts.Theme
if name == "" || name == "default" {
if _, ok := os.LookupEnv("NO_COLOR"); ok {
name = "mono"
}
}
m.theme = LoadTheme(name, opts.Settings)
if strings.EqualFold(os.Getenv("TERM"), "linux") || os.Getenv("MAPSIZE_ASCII") != "" {
m.theme.ASCII = true
}
m.views = []view{&mapView{}, &listView{}, &extView{}, &topView{}, &infoView{}}
if opts.Diff != nil {
m.views = append(m.views, &changesView{})
}
m.zoom = t.Root()
m.sel = noSel
return m
}
const noSel int64 = 1 << 62
// debugPanic lets developers verify terminal restoration on crashes:
// MAPSIZE_DEBUG_PANIC=ui panics on the first key press; =scanner panics in
// a background goroutine shortly after start.
var debugPanic = os.Getenv("MAPSIZE_DEBUG_PANIC")
// Init starts ticking and waits for scan completion.
func (m *Model) Init() tea.Cmd {
cmds := []tea.Cmd{tea.RequestWindowSize}
if m.scanning {
cmds = append(cmds, waitScan(m.scanner), m.startTicking())
}
return tea.Batch(cmds...)
}
func waitScan(s *scan.Scanner) tea.Cmd {
return func() tea.Msg { <-s.Done(); return scanDoneMsg{s} }
}
func (m *Model) refreshInterval() time.Duration {
hz := m.opts.Settings.RefreshHz
if hz <= 0 {
hz = 10
}
return time.Second / time.Duration(hz)
}
// startTicking returns a tick command unless one is already pending.
func (m *Model) startTicking() tea.Cmd {
if m.ticking {
return nil
}
m.ticking = true
return tea.Tick(m.refreshInterval(), func(t time.Time) tea.Msg { return tickMsg(t) })
}
func (m *Model) needsTicks() bool {
return m.scanning || time.Now().Before(m.toastUntil) || (m.dups != nil && m.dups.running)
}
// notify shows a transient status message.
func (m *Model) notify(s string, st Style) tea.Cmd {
m.toast, m.toastStyle, m.toastUntil = s, st, time.Now().Add(4*time.Second)
m.dirty = true
return m.startTicking()
}
func (m *Model) info(s string) tea.Cmd {
return m.notify(s, Style{FG: m.theme.Fg, BG: m.theme.StatusBg})
}
func (m *Model) warn(s string) tea.Cmd {
return m.notify(s, Style{FG: m.theme.Warn, BG: m.theme.StatusBg, Attr: Bold})
}
// invalidate marks size-dependent caches stale.
func (m *Model) invalidate() {
m.dataVer++
m.dirty = true
}
// Update handles one message.
func (m *Model) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
cmd := m.update(msg)
if m.quit {
return m, tea.Quit
}
return m, cmd
}
func (m *Model) update(msg tea.Msg) tea.Cmd {
switch msg := msg.(type) {
case tea.WindowSizeMsg:
// Only record the size; layout is recomputed lazily in View, so a
// storm of resize events costs at most one layout per frame.
if msg.Width != m.w || msg.Height != m.h {
m.w, m.h = msg.Width, msg.Height
m.dirty = true
}
case tickMsg:
m.ticking = false
if m.scanning {
m.dirty = true
if time.Since(m.lastData) >= 400*time.Millisecond {
m.invalidate()
m.lastData = time.Now()
}
m.resolvePendingZoom()
}
if m.toast != "" && time.Now().After(m.toastUntil) {
m.toast = ""
m.dirty = true
}
var cmds []tea.Cmd
if m.scanning && m.search.query != nil && time.Since(m.search.appliedAt) > time.Second {
cmds = append(cmds, m.applyFilter())
}
if m.dups != nil && m.dups.running {
m.dirty = true
}
if m.needsTicks() {
cmds = append(cmds, m.startTicking())
}
return tea.Batch(cmds...)
case scanDoneMsg:
if msg.s != m.scanner {
return nil // a superseded scan
}
return m.scanFinished()
case filterMsg:
return m.filterResult(msg)
case filterDebounceMsg:
if msg.gen == m.search.gen {
return m.applyFilter()
}
case dupProgressMsg, dupDoneMsg:
return m.dupMsg(msg)
case trashDoneMsg:
return m.trashDone(msg)
case saveDoneMsg:
if msg.err != nil {
return m.warn("Snapshot save failed: " + textutil.Sanitize(msg.err.Error()))
}
return m.info("Saved snapshot " + textutil.Sanitize(msg.path))
case fatalMsg:
m.fatal = &msg
m.quit = true
case tea.KeyPressMsg:
if debugPanic == "ui" {
panic("MAPSIZE_DEBUG_PANIC=ui: deliberate panic to test terminal restoration")
}
m.dirty = true
prev := m.sel
m.zoomed = false
cmd := m.handleKey(msg)
m.noteSelection(prev)
return cmd
case tea.MouseClickMsg:
prev := m.sel
m.zoomed = false
cmd := m.mouseClick(msg.Mouse())
m.noteSelection(prev)
return cmd
case tea.MouseWheelMsg:
return m.mouseWheel(msg.Mouse())
case tea.MouseMotionMsg:
m.mouseMotion(msg.Mouse())
case tea.ColorProfileMsg:
if !m.forced {
m.profile = msg.Profile
}
m.dirty = true
}
return nil
}
// noteSelection records that the user chose a selection explicitly.
func (m *Model) noteSelection(prev int64) {
if !m.zoomed && m.sel != prev && m.sel != noSel {
m.userSel = true
}
}
func (m *Model) scanFinished() tea.Cmd {
m.scanning = false
m.invalidate()
m.resolvePendingZoom()
t := m.tree
t.RLock()
st := t.Stats
r := t.Node(t.Root())
total := r.Total(m.sizeMode)
files := st.Files + st.Symlinks + st.Others
errs := st.TotalErrors()
el := st.Elapsed()
t.RUnlock()
slog.Info("scan finished", "files", files, "dirs", st.Dirs, "bytes", total, "errors", errs,
"cancelled", st.Cancelled, "elapsed", el)
var cmds []tea.Cmd
rate := float64(files+st.Dirs) / max(el.Seconds(), 0.001)
msg := fmt.Sprintf("✓ Scan complete %s · %s files · %s dirs · %s · %s/s",
textutil.Duration(el), textutil.Count(files), textutil.Count(st.Dirs), textutil.Size(total), textutil.Count(int64(rate)))
if st.Cancelled {
msg = "✗ Scan cancelled — totals are incomplete"
}
if errs > 0 {
msg += fmt.Sprintf(" · %s errors (e)", textutil.Count(errs))
}
if st.Cancelled || errs > 0 {
cmds = append(cmds, m.warn(msg))
} else {
cmds = append(cmds, m.notify(msg, Style{FG: m.theme.OK, BG: m.theme.StatusBg, Attr: Bold}))
}
if m.opts.Save != "" && !st.Cancelled {
cmds = append(cmds, m.saveSnapshot(m.opts.Save))
}
if m.search.query != nil {
cmds = append(cmds, m.applyFilter())
}
return tea.Batch(cmds...)
}
// ctrlCPressed: first press cancels a running scan, second quits.
func (m *Model) ctrlCPressed() tea.Cmd {
if m.scanning && !m.ctrlC {
m.ctrlC = true
m.cancel()
return m.warn("Cancelling scan… press Ctrl+C again to quit")
}
m.quit = true
return nil
}
func (m *Model) rescan() tea.Cmd {
if m.snapshot {
return m.warn("Snapshot loaded — nothing to rescan")
}
// Called from handleKey, which holds the old tree's read lock.
m.pendingZoom = m.tree.Path(m.zoom)
m.cancel()
ctx, cancel := context.WithCancel(context.Background())
s, t, err := m.opts.Start(ctx)
if err != nil {
cancel()
return m.warn("Rescan failed: " + textutil.Sanitize(err.Error()))
}
m.scanner, m.tree, m.cancel, m.scanning, m.ctrlC = s, t, cancel, s != nil, false
m.zoom, m.sel = t.Root(), noSel
m.kidsCache = map[inventory.NodeID]*kids{}
if m.dups != nil {
m.dups.cancel()
m.dups = nil
// Drop the duplicates view: its rows referred to the old tree.
for i, v := range m.views {
if _, ok := v.(*dupView); ok {
m.views = append(m.views[:i], m.views[i+1:]...)
m.view = 0
break
}
}
}
m.search.result = nil
m.search.gen++ // results computed against the old tree are discarded
m.invalidate()
return tea.Batch(waitScan(s), m.startTicking(), m.info("Rescanning…"), m.applyFilter())
}
// resolvePendingZoom re-zooms into the remembered directory once the new
// scan has discovered it.
func (m *Model) resolvePendingZoom() {
if m.pendingZoom == "" {
return
}
m.tree.RLock()
defer m.tree.RUnlock()
root := m.tree.Path(m.tree.Root())
rel, err := filepath.Rel(root, m.pendingZoom)
if err != nil || strings.HasPrefix(rel, "..") {
m.pendingZoom = ""
return
}
id := m.tree.Root()
if rel != "." {
for _, part := range strings.Split(rel, string(filepath.Separator)) {
next := inventory.NoNode
m.tree.Children(id, func(c inventory.NodeID, n *inventory.Node) {
if n.Name == part {
next = c
}
})
if next == inventory.NoNode {
return // not discovered yet
}
id = next
}
}
m.zoom = id
m.pendingZoom = ""
m.dirty = true
}
// View renders the frame. Expensive work happens only when something
// changed; otherwise the cached frame is returned.
func (m *Model) View() tea.View {
if m.dirty || m.frame == "" {
m.frame = m.render()
m.dirty = false
}
v := tea.NewView(m.frame)
v.AltScreen = true
if m.opts.Mouse {
v.MouseMode = tea.MouseModeAllMotion
}
v.WindowTitle = brand.Name
return v
}
internal/tui/mouse.go
-110
View File
@@ -1,110 +0,0 @@
package tui
import (
"time"
tea "charm.land/bubbletea/v2"
"github.com/andydixon/mapsize/internal/inventory"
)
const doubleClick = 400 * time.Millisecond
func (m *Model) mouseClick(ev tea.Mouse) tea.Cmd {
t := m.tree
t.RLock()
defer t.RUnlock()
m.dirty = true
if len(m.modals) > 0 {
// Clicking outside the modal closes it.
if len(m.hits) > 0 && !m.hits[0].r.Contains(ev.X, ev.Y) {
m.closeModal()
}
return nil
}
if ev.Button == tea.MouseRight {
for _, h := range m.hits {
if h.r.Contains(ev.X, ev.Y) {
h.fn(m, false)
return m.inspectSelection()
}
}
return nil
}
if ev.Button != tea.MouseLeft {
return nil
}
for i := len(m.hits) - 1; i >= 0; i-- {
h := m.hits[i]
if h.r.Contains(ev.X, ev.Y) {
key := int64(ev.Y)<<32 | int64(i)
double := time.Since(m.lastClick) < doubleClick && m.lastClickI == key
m.lastClick, m.lastClickI = time.Now(), key
if double {
m.lastClickI = noSel
}
return h.fn(m, double)
}
}
return nil
}
// clickBlock selects a treemap block; a double click zooms (or inspects).
func (m *Model) clickBlock(id int64, double bool) tea.Cmd {
m.sel = id
if !double {
return nil
}
if !isGroup(id) && !m.tree.Node(inventory.NodeID(id)).IsDir() {
return m.inspectSelection()
}
return m.zoomIntoSelection()
}
func (m *Model) mouseWheel(ev tea.Mouse) tea.Cmd {
t := m.tree
t.RLock()
defer t.RUnlock()
m.dirty = true
up := ev.Button == tea.MouseWheelUp
if ev.Button != tea.MouseWheelUp && ev.Button != tea.MouseWheelDown {
return nil
}
if len(m.modals) > 0 {
f := m.modals[len(m.modals)-1].(*modalFrame)
if up {
f.scroll -= 3
} else {
f.scroll += 3
}
f.clampScroll()
return nil
}
if up {
// Zoom towards the block under the pointer.
if l := m.tm; l != nil && m.view == 0 {
for _, b := range l.blocks {
if b.Contains(ev.X, ev.Y) {
m.sel = b.ID
}
}
}
}
return m.views[m.view].wheel(m, up)
}
func (m *Model) mouseMotion(ev tea.Mouse) {
h := noSel
if l := m.tm; l != nil && m.view == 0 && len(m.modals) == 0 {
for _, b := range l.blocks {
if b.Contains(ev.X, ev.Y) {
h = b.ID
break
}
}
}
if h != m.hover {
m.hover = h
m.dirty = true
}
}
internal/tui/nest_test.go
-89
View File
@@ -1,89 +0,0 @@
package tui
import (
"context"
"fmt"
"strings"
"testing"
"github.com/charmbracelet/colorprofile"
"github.com/charmbracelet/x/ansi"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/scan"
)
func rootTree() *inventory.Tree {
t := inventory.New("/", inventory.KindDir)
add := func(p inventory.NodeID, name string, size int64) {
id := t.Add(p, name, inventory.KindFile)
n := t.Node(id)
n.Size, n.Alloc = size, size
n.Cat = inventory.CategoryFor(inventory.Ext(name), false, false)
}
dir := func(p inventory.NodeID, name string) inventory.NodeID {
id := t.Add(p, name, inventory.KindDir)
t.Node(id).Flags |= inventory.FlagScanned
return id
}
mnt := dir(0, "mnt")
raid := dir(mnt, "raid")
media := dir(raid, "media")
for _, show := range []string{"X-Files", "Simpsons", "Frasier"} {
s := dir(media, show)
for i := 1; i <= 5; i++ {
se := dir(s, fmt.Sprintf("S%02d", i))
for e := 1; e <= 20; e++ {
add(se, fmt.Sprintf("E%02d.mkv", e), int64(e)<<28)
}
}
}
usr := dir(0, "usr")
lib := dir(usr, "lib")
for i := range 50 {
add(lib, fmt.Sprintf("lib%d.so", i), int64(i+1)<<22)
}
v := dir(0, "var")
add(dir(v, "log"), "syslog", 2<<30)
t.Recompute()
t.Stats.Complete = true
return t
}
// A single-child chain (mnt → raid → media) must not use up the nesting
// budget: the map at / should reach the seasons and their episodes.
func TestNestingReachesDeepMedia(t *testing.T) {
tr := rootTree()
m := newModel(Options{Start: func(context.Context) (*scan.Scanner, *inventory.Tree, error) { return nil, tr, nil }}, nil, tr, func() {})
m.profile = colorprofile.TrueColor
send(m, size(160, 45))
if out := ansi.Strip(m.View().Content); !strings.Contains(out, "mnt/raid/media/") || !strings.Contains(out, "S01/") {
t.Fatalf("chain or seasons missing:\n%s", out)
}
files := 0
for _, nb := range m.tm.nested {
if nb.id >= 0 && !tr.Node(inventory.NodeID(nb.id)).IsDir() {
files++
}
}
if files < 50 {
t.Fatalf("only %d episode blocks drawn", files)
}
}
// Headless, o shows the selected folder in the map instead of launching a
// file manager.
func TestRevealHeadlessZoomsMap(t *testing.T) {
t.Setenv("DISPLAY", "")
t.Setenv("WAYLAND_DISPLAY", "")
tr := rootTree()
m := newModel(Options{Start: func(context.Context) (*scan.Scanner, *inventory.Tree, error) { return nil, tr, nil }}, nil, tr, func() {})
send(m, size(160, 45))
mnt := m.kidsOf(m.zoom).ids[0]
m.snapshot = false // reveal is refused for snapshots
m.sel, m.userSel = int64(mnt), true
send(m, key("o"))
if m.zoom != mnt {
t.Fatalf("zoom = %q, want mnt", tr.Path(m.zoom))
}
}
internal/tui/render.go
-422
View File
@@ -1,422 +0,0 @@
package tui
import (
"fmt"
"strings"
"time"
tea "charm.land/bubbletea/v2"
"github.com/charmbracelet/colorprofile"
"github.com/andydixon/mapsize/internal/brand"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/textutil"
"github.com/andydixon/mapsize/internal/treemap"
)
// Minimum usable terminal size.
const minW, minH = 40, 10
type sizeClass int
const (
tooSmall sizeClass = iota
small
medium
large
)
// geometry is the panel layout for one terminal size. It is recomputed from
// scratch for every frame, so resizing can never leave stale geometry.
type geometry struct {
class sizeClass
tabs, crumbs treemap.Rect // tabs is empty in the small layout
main, side, info, stat treemap.Rect
}
func computeGeometry(w, h int) geometry {
g := geometry{}
switch {
case w < minW || h < minH:
g.class = tooSmall
return g
case w >= 150 && h >= 36:
g.class = large
case w >= 90 && h >= 22:
g.class = medium
default:
g.class = small
}
y := 0
if g.class >= medium {
g.tabs = treemap.Rect{X: 0, Y: 0, W: w, H: 1}
y = 1
}
g.crumbs = treemap.Rect{X: 0, Y: y, W: w, H: 1}
y++
bottom := h - 1
g.stat = treemap.Rect{X: 0, Y: h - 1, W: w, H: 1}
if g.class >= medium {
g.info = treemap.Rect{X: 0, Y: h - 2, W: w, H: 1}
bottom = h - 2
}
g.main = treemap.Rect{X: 0, Y: y, W: w, H: bottom - y}
if g.class == large {
sw := min(40, w/5)
g.side = treemap.Rect{X: 0, Y: y, W: sw, H: g.main.H}
g.main.X, g.main.W = sw, w-sw
}
return g
}
func (m *Model) render() string {
t := m.theme
cv := NewCanvas(m.w, m.h, t.base())
m.hits = m.hits[:0]
g := computeGeometry(m.w, m.h)
if g.class == tooSmall {
m.renderTooSmall(cv)
return cv.Render(m.depth())
}
m.tree.RLock()
defer m.tree.RUnlock()
m.ensureSelection()
if g.tabs.W > 0 {
m.paintTabs(cv, g.tabs)
}
m.paintCrumbs(cv, g.crumbs)
if !g.side.Empty() {
m.paintSide(cv, g.side)
}
m.views[m.view].paint(m, cv, g.main)
if !g.info.Empty() {
m.paintInfoLine(cv, g.info)
}
m.paintStatus(cv, g.stat, g.class)
if g.class == small && m.search.editing {
m.paintSearch(cv, g.stat)
}
if len(m.modals) > 0 {
m.paintModal(cv)
}
return cv.Render(m.depth())
}
func (m *Model) depth() colorDepth {
switch m.profile {
case colorprofile.ANSI256:
return depth256
case colorprofile.ANSI:
return depth16
}
return depthTrue
}
func (m *Model) renderTooSmall(cv *Canvas) {
st := m.theme.muted()
lines := []string{"Terminal too small", fmt.Sprintf("%d×%d — need %d×%d", m.w, m.h, minW, minH), "q quits"}
y0 := max(0, (m.h-len(lines))/2)
for i, l := range lines {
cv.TextCentered(0, y0+i, m.w, l, st)
}
}
// paintTabs draws the view tabs and mode chips.
func (m *Model) paintTabs(cv *Canvas, r treemap.Rect) {
t := m.theme
cv.Fill(r, " ", t.header())
badge := " ◧ " + brand.Name + " "
if t.ASCII {
badge = " " + brand.Name + " "
}
x := cv.Text(r.X, r.Y, badge, r.W, Style{FG: t.textOn(t.Accent), BG: t.Accent, Attr: Bold | m.monoRev()})
x++
for i, v := range m.views {
label := " " + v.name() + " "
st := Style{FG: t.Muted, BG: t.HeaderBg}
if i == m.view {
st = Style{FG: t.Fg, BG: t.Bg, Attr: Bold | Underline}
}
if x+textutil.Width(label) >= r.W-30 {
break
}
w := cv.Text(x, r.Y, label, r.W-x, st)
idx := i
m.hits = append(m.hits, hit{treemap.Rect{X: x, Y: r.Y, W: w, H: 1}, func(m *Model, _ bool) tea.Cmd { m.setView(idx); return nil }})
x += w
}
// Right-aligned chips.
var chips []struct {
s string
bg Color
}
add := func(s string, bg Color) {
chips = append(chips, struct {
s string
bg Color
}{s, bg})
}
if m.opts.ReadOnly {
add("READ ONLY", t.Err)
}
if m.snapshot {
add("SNAPSHOT", t.Accent)
}
if m.opts.Diff != nil {
add("COMPARE", t.DiffNew)
}
if m.search.query != nil {
add("FILTER", t.Warn)
}
if len(m.tree.Stats.Excludes) > 0 {
add(fmt.Sprintf("EXCL %d", len(m.tree.Stats.Excludes)), t.Faint)
}
add(strings.ToUpper(m.sizeMode.String()), t.Faint)
rx := r.X + r.W
for i := len(chips) - 1; i >= 0; i-- {
s := " " + chips[i].s + " "
w := textutil.Width(s)
if rx-w-1 < x {
break
}
rx -= w
cv.Text(rx, r.Y, s, w, Style{FG: t.textOn(chips[i].bg), BG: chips[i].bg, Attr: Bold | m.monoRev()})
rx--
}
}
func (m *Model) monoRev() Attr {
if m.theme.Mono {
return Reverse
}
return 0
}
// paintCrumbs draws "/ › home › andy" breadcrumbs, truncated from the left,
// with the zoom root's size on the right.
func (m *Model) paintCrumbs(cv *Canvas, r treemap.Rect) {
t := m.theme
cv.Fill(r, " ", t.header())
tr := m.tree
z := tr.Node(m.zoom)
size := m.sizeOf(m.zoom)
right := " " + textutil.Size(size)
if root := m.sizeOf(tr.Root()); m.zoom != tr.Root() && root > 0 {
right += " " + textutil.Percent(size, root) + " of scan"
}
if m.search.result != nil {
right += fmt.Sprintf(" (%s matches)", textutil.Count(m.search.result.Count))
}
if z.IsDir() {
right += fmt.Sprintf(" %s files", textutil.Count(int64(z.Files)))
}
right += " "
rw := textutil.Width(right)
if rw > r.W/2 {
right = " " + textutil.SizeCompact(size) + " "
rw = textutil.Width(right)
}
cv.TextRight(r.X, r.Y, r.W, right, Style{FG: t.HeaderFg, BG: t.HeaderBg, Attr: Bold})
sep := " › "
if t.ASCII {
sep = " > "
}
chain := tr.Ancestors(m.zoom)
type seg struct {
id inventory.NodeID
s string
}
segs := make([]seg, len(chain))
for i, id := range chain {
segs[i] = seg{id, textutil.Sanitize(tr.Node(id).Name)}
}
avail := r.W - rw - 2
// Drop leading segments until the rest fits; keep at least the last.
total := func(from int) int {
w := 0
for i := from; i < len(segs); i++ {
w += textutil.Width(segs[i].s)
if i > from {
w += textutil.Width(sep)
}
}
if from > 0 {
w += textutil.Width("…" + sep)
}
return w
}
from := 0
for from < len(segs)-1 && total(from) > avail {
from++
}
x := r.X + 1
if from > 0 {
x += cv.Text(x, r.Y, "…"+sep, avail, Style{FG: t.Muted, BG: t.HeaderBg})
}
for i := from; i < len(segs); i++ {
if i > from {
x += cv.Text(x, r.Y, sep, r.X+1+avail-x, Style{FG: t.Faint, BG: t.HeaderBg})
}
st := Style{FG: t.Muted, BG: t.HeaderBg}
if i == len(segs)-1 {
st = Style{FG: t.Fg, BG: t.HeaderBg, Attr: Bold}
}
s := segs[i].s
if i == len(segs)-1 {
s = textutil.TruncateLeft(s, max(1, r.X+1+avail-x))
}
w := cv.Text(x, r.Y, s, r.X+1+avail-x, st)
id := segs[i].id
m.hits = append(m.hits, hit{treemap.Rect{X: x, Y: r.Y, W: w, H: 1}, func(m *Model, _ bool) tea.Cmd { m.zoomTo(id); return nil }})
x += w
}
}
// paintInfoLine describes the selection (or shows the search editor).
func (m *Model) paintInfoLine(cv *Canvas, r treemap.Rect) {
t := m.theme
cv.Fill(r, " ", t.panel())
if m.search.editing {
m.paintSearch(cv, r)
return
}
if m.search.query != nil {
m.paintFilterSummary(cv, r)
return
}
cv.Text(r.X+1, r.Y, m.describeSelection(), r.W-2, t.panel())
}
func (m *Model) describeSelection() string {
id := m.selectedNode()
tr := m.tree
if isGroup(m.sel) {
if g, ok := m.groupInfo(m.sel); ok {
return fmt.Sprintf("▸ %s smaller items · %s · %s of %s · Enter lists them",
textutil.Count(int64(len(g.ids))), textutil.Size(g.size), textutil.Percent(g.size, m.sizeOf(m.zoom)),
textutil.Sanitize(tr.Node(m.zoom).Name))
}
}
if id == inventory.NoNode {
return "Nothing selected"
}
n := tr.Node(id)
parts := []string{"▸ " + textutil.Sanitize(n.Name)}
parts = append(parts, textutil.Size(m.sizeOf(id)))
if p := n.Parent; p != inventory.NoNode {
parts = append(parts, textutil.Percent(m.sizeOf(id), m.sizeOf(p))+" of parent")
}
if n.IsDir() {
parts = append(parts, textutil.Count(int64(n.Files))+" files", textutil.Count(int64(n.Dirs))+" dirs")
if n.Flags&inventory.FlagScanned == 0 && m.scanning {
parts = append(parts, "scanning…")
}
} else {
parts = append(parts, kindLabel(tr, n))
}
if n.MTime > 0 {
parts = append(parts, "modified "+time.Unix(0, n.MTime).Format("2006-01-02"))
}
if n.Errors > 0 {
parts = append(parts, fmt.Sprintf("%d errors", n.Errors))
}
if m.opts.Diff != nil {
parts = append(parts, textutil.SignedSize(m.opts.Diff.Delta(id, m.sizeMode))+" vs old snapshot")
}
return strings.Join(parts, " · ")
}
func kindLabel(tr *inventory.Tree, n *inventory.Node) string {
switch n.Kind {
case inventory.KindFile:
return n.Cat.String()
case inventory.KindSymlink:
return "symlink"
}
return n.Kind.String()
}
var spinner = []string{"⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"}
// paintStatus draws the live status bar.
func (m *Model) paintStatus(cv *Canvas, r treemap.Rect, class sizeClass) {
t := m.theme
cv.Fill(r, " ", t.status())
help := " ? help "
if class == small {
help = " ? "
}
hw := cv.Text(r.X+r.W-textutil.Width(help), r.Y, help, r.W, Style{FG: t.Muted, BG: t.StatusBg})
avail := r.W - hw - 1
if m.toast != "" {
cv.Text(r.X+1, r.Y, m.toast, avail-1, m.toastStyle)
return
}
st := &m.tree.Stats
root := m.tree.Node(m.tree.Root())
files := st.Files + st.Symlinks + st.Others
var parts []struct {
s string
fg Color
}
add := func(s string, fg Color) {
parts = append(parts, struct {
s string
fg Color
}{s, fg})
}
switch {
case m.scanning:
frame := spinner[int(time.Now().UnixMilli()/100)%len(spinner)]
if t.ASCII {
frame = "*"
}
add(frame+" scanning", t.Accent)
case st.Cancelled:
add("✗ cancelled", t.Warn)
case m.snapshot:
add("◉ snapshot", t.Accent)
default:
add("✓ complete", t.OK)
}
add(textutil.Count(files)+" files", t.StatusFg)
add(textutil.Size(root.Total(m.sizeMode)), t.StatusFg)
if m.scanning {
el := st.Elapsed()
p := m.scanner.Progress()
add(textutil.Count(int64(float64(p.Entries)/max(el.Seconds(), 0.001)))+"/s", t.StatusFg)
if class != small {
add(fmt.Sprintf("%d workers", p.Workers), t.Muted)
add(fmt.Sprintf("queue %s", textutil.Count(p.Pending)), t.Muted)
}
add(textutil.Duration(el), t.Muted)
} else if !m.snapshot && class != small {
add(textutil.Duration(st.Elapsed()), t.Muted)
}
if e := st.TotalErrors(); e > 0 {
add(fmt.Sprintf("%s errors", textutil.Count(e)), t.Err)
}
if st.Excluded > 0 {
add(fmt.Sprintf("%s excluded", textutil.Count(st.Excluded)), t.Warn)
}
if st.Incomplete() && !m.scanning && class != small {
add("totals incomplete", t.Warn)
}
x := r.X + 1
for i, p := range parts {
s := p.s
if i > 0 {
s = " " + s
}
if x+textutil.Width(s) > r.X+avail {
break
}
st := Style{FG: p.fg, BG: t.StatusBg}
if i == 0 {
st.Attr = Bold
}
x += cv.Text(x, r.Y, s, avail-x, st)
}
}
internal/tui/run.go
-72
View File
@@ -1,72 +0,0 @@
package tui
import (
"context"
"fmt"
"os"
"runtime/debug"
"time"
tea "charm.land/bubbletea/v2"
"github.com/charmbracelet/colorprofile"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/scan"
"github.com/andydixon/mapsize/internal/snapshot"
"github.com/andydixon/mapsize/internal/textutil"
)
var profiles = map[string]colorprofile.Profile{
"truecolor": colorprofile.TrueColor, "24bit": colorprofile.TrueColor,
"256": colorprofile.ANSI256, "16": colorprofile.ANSI, "none": colorprofile.ASCII,
}
func saveFile(path string, t *inventory.Tree) error { return snapshot.SaveFile(path, t) }
// Run starts the interactive UI. The terminal is restored on every exit
// path: normal quit, Ctrl+C, errors and panics (Bubble Tea recovers panics
// in Update/View; scanner goroutine panics are routed here via
// scan.OnPanic and reported after the terminal is restored).
func Run(opts Options) error {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
s, t, err := opts.Start(ctx)
if err != nil {
return err
}
m := newModel(opts, s, t, cancel)
var popts []tea.ProgramOption
if p, ok := profiles[opts.Color]; ok {
m.profile, m.forced = p, true
popts = append(popts, tea.WithColorProfile(p))
} else if opts.Color != "" && opts.Color != "auto" {
return fmt.Errorf("invalid --color %q (auto, truecolor, 256, 16, none)", opts.Color)
}
p := tea.NewProgram(m, popts...)
prev := scan.OnPanic
scan.OnPanic = func(v any, stack []byte) {
p.Send(fatalMsg{v, stack})
select {} // park the broken goroutine; the program is quitting
}
defer func() { scan.OnPanic = prev }()
if debugPanic == "scanner" {
go func() {
time.Sleep(500 * time.Millisecond)
defer func() {
if v := recover(); v != nil {
scan.OnPanic(v, debug.Stack())
}
}()
panic("MAPSIZE_DEBUG_PANIC=scanner: deliberate panic to test terminal restoration")
}()
}
_, err = p.Run()
cancel()
if m.fatal != nil {
fmt.Fprintf(os.Stderr, "mapsize: internal error in scanner: %s\n\n%s\n",
textutil.Sanitize(fmt.Sprint(m.fatal.v)), m.fatal.stack)
return fmt.Errorf("scanner crashed")
}
return err
}
internal/tui/search.go
-260
View File
@@ -1,260 +0,0 @@
package tui
import (
"context"
"fmt"
"time"
tea "charm.land/bubbletea/v2"
"github.com/andydixon/mapsize/internal/filter"
"github.com/andydixon/mapsize/internal/textutil"
"github.com/andydixon/mapsize/internal/treemap"
)
// lineEditor is a minimal single-line text editor.
type lineEditor struct {
buf []rune
cur int
}
func (e *lineEditor) String() string { return string(e.buf) }
func (e *lineEditor) set(s string) { e.buf, e.cur = []rune(s), len([]rune(s)) }
// key applies an editing key; it reports whether the text changed.
func (e *lineEditor) key(msg tea.KeyPressMsg) (changed, handled bool) {
switch msg.Keystroke() {
case "left", "ctrl+b":
e.cur = max(0, e.cur-1)
case "right", "ctrl+f":
e.cur = min(len(e.buf), e.cur+1)
case "home", "ctrl+a":
e.cur = 0
case "end", "ctrl+e":
e.cur = len(e.buf)
case "backspace", "ctrl+h":
if e.cur > 0 {
e.buf = append(e.buf[:e.cur-1], e.buf[e.cur:]...)
e.cur--
return true, true
}
case "delete", "ctrl+d":
if e.cur < len(e.buf) {
e.buf = append(e.buf[:e.cur], e.buf[e.cur+1:]...)
return true, true
}
case "ctrl+u":
e.buf, e.cur = e.buf[e.cur:], 0
return true, true
case "ctrl+k":
e.buf = e.buf[:e.cur]
return true, true
case "ctrl+w", "alt+backspace":
i := e.cur
for i > 0 && e.buf[i-1] == ' ' {
i--
}
for i > 0 && e.buf[i-1] != ' ' {
i--
}
e.buf = append(e.buf[:i], e.buf[e.cur:]...)
e.cur = i
return true, true
default:
txt := msg.Text
if txt == "" || msg.Mod&(tea.ModCtrl|tea.ModAlt) != 0 {
return false, false
}
var rs []rune
for _, r := range txt {
if r >= 0x20 && r != 0x7f {
rs = append(rs, r)
}
}
if len(rs) == 0 || len(e.buf)+len(rs) > 1024 {
return false, true
}
e.buf = append(e.buf[:e.cur], append(rs, e.buf[e.cur:]...)...)
e.cur += len(rs)
return true, true
}
return false, true
}
type searchState struct {
editing bool
ed lineEditor
query *filter.Query
result *filter.Result
err string
gen int
busy bool
appliedAt time.Time
cancel context.CancelFunc
}
func (s *searchState) begin() {
s.editing = true
if s.query != nil {
s.ed.set(s.query.String())
}
}
func (m *Model) searchKey(msg tea.KeyPressMsg) tea.Cmd {
s := &m.search
switch msg.Keystroke() {
case "esc":
s.editing = false
m.clearFilter()
return nil
case "enter":
s.editing = false
if s.err != "" {
return m.warn("Invalid query: " + s.err)
}
return nil
}
changed, _ := s.ed.key(msg)
if changed {
return m.queryChanged()
}
return nil
}
// queryChanged parses the editor contents and schedules a debounced apply.
func (m *Model) queryChanged() tea.Cmd {
s := &m.search
s.gen++
text := s.ed.String()
if text == "" {
s.err = ""
m.dropFilter()
return nil
}
q, err := filter.Parse(text, time.Now())
if err != nil {
s.err = err.Error()
return nil
}
s.err = ""
s.query = q
gen := s.gen
return tea.Tick(120*time.Millisecond, func(time.Time) tea.Msg { return filterDebounceMsg{gen} })
}
// setFilter replaces the filter with q and applies it immediately.
func (m *Model) setFilter(q string) tea.Cmd {
m.search.ed.set(q)
m.search.editing = false
m.queryChanged()
if m.search.err != "" {
return m.warn("Invalid query: " + m.search.err)
}
return m.applyFilter()
}
// applyFilter evaluates the current query in the background. The tree is
// read-locked by the worker; stale results are discarded by generation.
func (m *Model) applyFilter() tea.Cmd {
s := &m.search
if s.query == nil {
return nil
}
if s.cancel != nil {
s.cancel()
}
ctx, cancel := context.WithCancel(context.Background())
s.cancel = cancel
s.busy = true
s.appliedAt = time.Now()
t, q, mode, gen := m.tree, s.query, m.sizeMode, s.gen
return func() tea.Msg {
t.RLock()
res, err := filter.Apply(ctx, t, q, mode)
t.RUnlock()
return filterMsg{gen: gen, res: res, err: err}
}
}
func (m *Model) filterResult(msg filterMsg) tea.Cmd {
s := &m.search
if msg.gen != s.gen || s.query == nil {
return nil
}
s.busy = false
if msg.err != nil {
return nil
}
s.result = msg.res
m.invalidate()
return nil
}
// dropFilter removes the active filter but keeps the editor open.
func (m *Model) dropFilter() {
s := &m.search
if s.cancel != nil {
s.cancel()
}
s.query, s.result, s.busy = nil, nil, false
m.invalidate()
}
func (m *Model) clearFilter() {
m.dropFilter()
m.search.ed.set("")
m.search.err = ""
m.search.gen++
}
func (m *Model) paintSearch(cv *Canvas, r treemap.Rect) {
t := m.theme
s := &m.search
st := Style{FG: t.Fg, BG: t.PanelBg}
cv.Fill(r, " ", st)
x := r.X + 1
x += cv.Text(x, r.Y, "/ ", 2, Style{FG: t.Accent, BG: t.PanelBg, Attr: Bold})
status, statusSt := "", Style{FG: t.Muted, BG: t.PanelBg}
switch {
case s.err != "":
status, statusSt.FG = "✗ "+s.err, t.Err
case s.busy:
status = "filtering…"
case s.result != nil:
status = fmt.Sprintf("%s matches · %s", textutil.Count(s.result.Count), textutil.Size(s.result.Total))
case len(s.ed.buf) == 0:
status = "name, *.iso, size > 1GB, ext IN (iso,vmdk), age > 365d … Enter keep · Esc clear"
}
sw := min(textutil.Width(status), r.W/2)
avail := r.W - (x - r.X) - sw - 3
// Scroll the text so the cursor stays visible.
text := s.ed.buf
start := 0
if s.ed.cur > avail-1 {
start = s.ed.cur - avail + 1
}
vis := string(text[start:min(len(text), start+avail)])
cv.Text(x, r.Y, textutil.Sanitize(vis), avail, st)
cx := x + textutil.Width(textutil.Sanitize(string(text[start:s.ed.cur])))
under := " "
if s.ed.cur < len(text) {
under = textutil.Sanitize(string(text[s.ed.cur]))
}
cv.Text(cx, r.Y, under, 2, Style{FG: t.PanelBg, BG: t.Fg, Attr: m.monoRev()})
cv.TextRight(r.X, r.Y, r.W-1, textutil.Truncate(status, sw), statusSt)
}
func (m *Model) paintFilterSummary(cv *Canvas, r treemap.Rect) {
t := m.theme
s := &m.search
x := r.X + 1
x += cv.Text(x, r.Y, " FILTER ", 8, Style{FG: t.textOn(t.Warn), BG: t.Warn, Attr: Bold | m.monoRev()})
x += cv.Text(x+1, r.Y, textutil.Sanitize(s.query.String()), r.W/2, Style{FG: t.Fg, BG: t.PanelBg, Attr: Bold}) + 1
info := "filtering…"
if s.result != nil && !s.busy {
info = fmt.Sprintf("%s matches · %s of %s · / edit · Esc clear", textutil.Count(s.result.Count),
textutil.Size(s.result.Total), textutil.Size(m.tree.Node(0).Total(m.sizeMode)))
}
cv.Text(x+2, r.Y, info, r.X+r.W-x-3, Style{FG: t.Muted, BG: t.PanelBg})
}
internal/tui/side.go
-118
View File
@@ -1,118 +0,0 @@
package tui
import (
"fmt"
"sort"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/textutil"
"github.com/andydixon/mapsize/internal/treemap"
)
// paintSide draws the large-layout side panel: directory summary, category
// breakdown and the selected item.
func (m *Model) paintSide(cv *Canvas, r treemap.Rect) {
t := m.theme
ps := t.panel()
cv.Fill(r, " ", ps)
cv.Fill(treemap.Rect{X: r.X + r.W - 1, Y: r.Y, W: 1, H: r.H}, "│", Style{FG: t.Faint, BG: t.PanelBg})
w := r.W - 3
x := r.X + 1
y := r.Y + 1
end := r.Y + r.H
head := func(s string) {
if y < end {
cv.Text(x, y, s, w, Style{FG: t.Accent, BG: t.PanelBg, Attr: Bold})
y++
}
}
kvl := func(k, v string, vs Style) {
if y < end {
cv.Text(x, y, k, w, Style{FG: t.Muted, BG: t.PanelBg})
cv.TextRight(x, y, w, v, vs)
y++
}
}
val := Style{FG: t.Fg, BG: t.PanelBg, Attr: Bold}
tr := m.tree
z := tr.Node(m.zoom)
head("DIRECTORY")
for _, l := range wrap(textutil.Sanitize(z.Name), w) {
if y < end {
cv.Text(x, y, l, w, Style{FG: t.Fg, BG: t.PanelBg, Attr: Bold})
y++
}
}
y++
kvl("Size", textutil.Size(m.sizeOf(m.zoom)), val)
kvl("Logical", textutil.Size(z.TotSize), Style{FG: t.Fg, BG: t.PanelBg})
kvl("Allocated", textutil.Size(z.TotAlloc), Style{FG: t.Fg, BG: t.PanelBg})
kvl("Files", textutil.Count(int64(z.Files)), Style{FG: t.Fg, BG: t.PanelBg})
kvl("Directories", textutil.Count(int64(z.Dirs)), Style{FG: t.Fg, BG: t.PanelBg})
if z.Errors > 0 {
kvl("Errors", textutil.Count(int64(z.Errors)), Style{FG: t.Err, BG: t.PanelBg, Attr: Bold})
}
y++
// Category breakdown with proportional bars.
if cs := tr.CatSizes(m.zoom); cs != nil {
head("CATEGORIES")
type cat struct {
c inventory.Category
v int64
}
var cats []cat
var total int64
for i, v := range cs {
if v > 0 {
cats = append(cats, cat{inventory.Category(i), v})
total += v
}
}
sort.Slice(cats, func(i, j int) bool { return cats[i].v > cats[j].v })
for _, c := range cats {
if y >= end-8 {
break
}
col := t.Cat[c.c]
sw := "██ "
if t.Mono || t.ASCII {
sw = " "
}
cv.Text(x, y, sw, 3, Style{FG: col, BG: t.PanelBg})
cv.Text(x+3, y, c.c.String(), w-3, Style{FG: t.Fg, BG: t.PanelBg})
cv.TextRight(x, y, w, textutil.Percent(c.v, total), Style{FG: t.Muted, BG: t.PanelBg})
y++
m.paintBar(cv, x+3, y, w-3, float64(c.v)/float64(cats[0].v), col, false)
y++
}
y++
}
// Selected item.
if y < end-4 {
head("SELECTED")
desc := []string{}
if isGroup(m.sel) {
if g, ok := m.groupInfo(m.sel); ok {
desc = append(desc, fmt.Sprintf("%s smaller items", textutil.Count(int64(len(g.ids)))), textutil.Size(g.size))
}
} else if id := m.selectedNode(); id != inventory.NoNode {
n := tr.Node(id)
desc = append(desc, wrap(textutil.Sanitize(n.Name), w)...)
desc = append(desc, textutil.Size(m.sizeOf(id))+" · "+textutil.Percent(m.sizeOf(id), m.sizeOf(m.zoom)))
if n.IsDir() {
desc = append(desc, textutil.Count(int64(n.Files))+" files")
} else {
desc = append(desc, kindLabel(tr, n))
}
}
for _, d := range desc {
if y < end {
cv.Text(x, y, d, w, Style{FG: t.Fg, BG: t.PanelBg})
y++
}
}
}
}
internal/tui/table.go
-222
View File
@@ -1,222 +0,0 @@
package tui
import (
tea "charm.land/bubbletea/v2"
"github.com/andydixon/mapsize/internal/textutil"
"github.com/andydixon/mapsize/internal/treemap"
)
// column describes one table column.
type column struct {
title string
width int // 0 = flexible (takes the remaining space)
right bool
clipLeft bool // truncate from the left (paths: the tail matters)
}
// tableRow is one rendered row: cell strings plus optional per-cell styles.
type tableRow struct {
cells []string
styles []*Style // nil = default
bar float64 // 0..1 for an optional bar column (-1 = none)
barCol Color
}
// table is a scrollable list with a cursor, shared by the list-style views.
type table struct {
cursor, offset int
height int // rows visible in the last paint
}
// scroll moves the cursor three rows for a mouse wheel step.
func (tb *table) scroll(up bool, n int) {
if up {
tb.move(-3, n)
} else {
tb.move(3, n)
}
}
func (tb *table) move(d, n int) {
if n == 0 {
tb.cursor = 0
return
}
tb.cursor = min(max(tb.cursor+d, 0), n-1)
}
// key handles navigation keys common to all tables.
func (tb *table) key(k string, n int) bool {
page := max(tb.height-1, 1)
switch k {
case "up", "k":
tb.move(-1, n)
case "down", "j":
tb.move(1, n)
case "pgup", "ctrl+b":
tb.move(-page, n)
case "pgdown", "ctrl+f":
tb.move(page, n)
case "end", "G":
tb.move(n, n)
case "g":
tb.cursor = 0
default:
return false
}
return true
}
func (tb *table) paint(m *Model, cv *Canvas, r treemap.Rect, cols []column, rows []tableRow, title string) {
t := m.theme
cv.Fill(r, " ", t.base())
if r.H < 3 {
return
}
y := r.Y
if title != "" {
cv.Text(r.X+1, y, title, r.W-2, Style{FG: t.Accent, BG: t.Bg, Attr: Bold})
y++
}
// Resolve flexible width.
fixed := 0
for _, c := range cols {
fixed += c.width + 1
}
flex := max(r.W-2-fixed, 8)
x := r.X + 1
hdr := Style{FG: t.Muted, BG: t.Bg, Attr: Bold}
for _, c := range cols {
w := c.width
if w == 0 {
w = flex
}
s := c.title
if c.right {
s = textutil.PadLeft(s, w)
}
cv.Text(x, y, s, w, hdr)
x += w + 1
}
y++
cv.HLine(r.X+1, y, r.W-2, "─", Style{FG: t.Faint, BG: t.Bg})
y++
tb.height = r.Y + r.H - y
if tb.cursor >= len(rows) {
tb.cursor = max(len(rows)-1, 0)
}
if tb.cursor < tb.offset {
tb.offset = tb.cursor
}
if tb.cursor >= tb.offset+tb.height {
tb.offset = tb.cursor - tb.height + 1
}
tb.offset = max(0, min(tb.offset, len(rows)-tb.height))
for i := tb.offset; i < len(rows) && y < r.Y+r.H; i++ {
row := rows[i]
sel := i == tb.cursor
base := t.base()
if sel {
base = Style{FG: t.textOn(t.Sel), BG: t.Sel, Attr: Bold | m.monoRev()}
cv.Fill(treemap.Rect{X: r.X, Y: y, W: r.W, H: 1}, " ", base)
}
x := r.X + 1
for ci, c := range cols {
w := c.width
if w == 0 {
w = flex
}
s := ""
if ci < len(row.cells) {
s = row.cells[ci]
}
st := base
if !sel && ci < len(row.styles) && row.styles[ci] != nil {
st = *row.styles[ci]
}
if c.title == "" && row.bar >= 0 {
m.paintBar(cv, x, y, w, row.bar, row.barCol, sel)
} else if c.right {
cv.Text(x, y, textutil.PadLeft(s, w), w, st)
} else if c.clipLeft {
cv.Text(x, y, textutil.TruncateLeft(s, w), w, st)
} else {
cv.Text(x, y, textutil.Truncate(s, w), w, st)
}
x += w + 1
}
idx := i
m.hits = append(m.hits, hit{treemap.Rect{X: r.X, Y: y, W: r.W, H: 1}, func(m *Model, double bool) tea.Cmd {
tb.cursor = idx
if double {
return m.handleKeyName("enter")
}
return nil
}})
y++
}
if len(rows) == 0 {
cv.TextCentered(r.X, r.Y+r.H/2, r.W, "Nothing to show", t.muted())
}
if len(rows) > tb.height && tb.height > 0 {
pos := textutil.Count(int64(tb.cursor+1)) + "/" + textutil.Count(int64(len(rows)))
cv.TextRight(r.X, r.Y, r.W-1, pos, t.muted())
}
}
// paintBar draws a proportional bar with eighth-block precision.
func (m *Model) paintBar(cv *Canvas, x, y, w int, frac float64, col Color, sel bool) {
t := m.theme
if col == 0 {
col = t.Accent
}
bg := t.Bg.Mix(t.Fg, 0.08)
if sel {
bg = t.Sel.Mix(RGB(0, 0, 0), 0.25)
}
eighths := int(frac*float64(w*8) + 0.5)
parts := []string{"", "▏", "▎", "▍", "▌", "▋", "▊", "▉"}
for i := 0; i < w; i++ {
ch := " "
switch {
case eighths >= 8:
ch = "█"
case eighths > 0:
ch = parts[eighths]
}
if t.ASCII {
ch = " "
if eighths >= 4 {
ch = "#"
}
}
eighths -= 8
st := Style{FG: col, BG: bg}
if t.Mono {
st = Style{}
}
cv.Text(x+i, y, ch, 1, st)
}
}
// handleKeyName lets mouse actions reuse key handling (lock already held).
func (m *Model) handleKeyName(k string) tea.Cmd {
if handled, cmd := m.views[m.view].key(m, k); handled {
return cmd
}
switch k {
case "enter":
return m.inspectSelection()
case "space":
return m.zoomIntoSelection()
}
return nil
}
func plural(n int, s string) string {
if n == 1 {
return "1 " + s
}
return textutil.Count(int64(n)) + " " + s + "s"
}
internal/tui/theme.go
-165
View File
@@ -1,165 +0,0 @@
package tui
import (
"sort"
"strings"
"github.com/andydixon/mapsize/internal/config"
"github.com/andydixon/mapsize/internal/inventory"
)
// Theme holds every colour the UI uses. Nothing else in the TUI hard-codes
// colours.
type Theme struct {
Name string
Mono bool // no colour: rely on attributes and glyphs
ASCII bool // no box-drawing glyphs
Bg, Fg, Muted, Faint Color
Accent, Sel Color
HeaderBg, HeaderFg Color
StatusBg, StatusFg Color
PanelBg Color
Warn, Err, OK Color
Group Color
EmptyDir Color
Cat [inventory.NumCategories]Color
DiffGrow, DiffShrink Color
DiffNew, DiffSame Color
Ext map[string]Color
}
var themes = map[string]func() *Theme{
"default": defaultTheme,
"dark": darkTheme,
"high-contrast": highContrastTheme,
"mono": monoTheme,
}
// ThemeNames lists built-in themes.
func ThemeNames() []string {
var out []string
for k := range themes {
out = append(out, k)
}
sort.Strings(out)
return out
}
func defaultTheme() *Theme {
return &Theme{
Name: "default",
Bg: RGB(0x16, 0x18, 0x1d), Fg: RGB(0xdc, 0xdf, 0xe4), Muted: RGB(0x8a, 0x91, 0x9c), Faint: RGB(0x4a, 0x50, 0x5a),
Accent: RGB(0x61, 0xaf, 0xef), Sel: RGB(0xff, 0xd8, 0x66),
HeaderBg: RGB(0x21, 0x25, 0x2b), HeaderFg: RGB(0xdc, 0xdf, 0xe4),
StatusBg: RGB(0x2c, 0x31, 0x3a), StatusFg: RGB(0xc8, 0xcc, 0xd4),
PanelBg: RGB(0x1c, 0x1f, 0x25),
Warn: RGB(0xe5, 0xc0, 0x7b), Err: RGB(0xe0, 0x6c, 0x75), OK: RGB(0x98, 0xc3, 0x79),
Group: RGB(0x5c, 0x63, 0x70), EmptyDir: RGB(0x4b, 0x52, 0x63),
Cat: [inventory.NumCategories]Color{
inventory.CatOther: RGB(0x7f, 0x8c, 0x9d),
inventory.CatImage: RGB(0xe5, 0xc0, 0x7b),
inventory.CatVideo: RGB(0xe0, 0x6c, 0x75),
inventory.CatAudio: RGB(0xd1, 0x9a, 0x66),
inventory.CatArchive: RGB(0xc6, 0x78, 0xdd),
inventory.CatDiskImage: RGB(0x8e, 0x7c, 0xf0),
inventory.CatDatabase: RGB(0x56, 0xb6, 0xc2),
inventory.CatCode: RGB(0x98, 0xc3, 0x79),
inventory.CatDocument: RGB(0x61, 0xaf, 0xef),
inventory.CatExecutable: RGB(0xbe, 0x50, 0x46),
inventory.CatLog: RGB(0xa3, 0xa8, 0x6a),
inventory.CatCache: RGB(0x6b, 0x77, 0x88),
},
DiffGrow: RGB(0xf0, 0x5a, 0x5a), DiffShrink: RGB(0x5a, 0xd0, 0x7a),
DiffNew: RGB(0x4f, 0xa8, 0xff), DiffSame: RGB(0x50, 0x56, 0x60),
}
}
func darkTheme() *Theme {
t := defaultTheme()
t.Name = "dark"
t.Bg, t.PanelBg = RGB(0, 0, 0), RGB(0x0c, 0x0c, 0x0e)
t.HeaderBg, t.StatusBg = RGB(0x12, 0x12, 0x16), RGB(0x1a, 0x1a, 0x20)
for i, c := range t.Cat {
t.Cat[i] = c.Mix(RGB(0, 0, 0), 0.2)
}
return t
}
func highContrastTheme() *Theme {
t := defaultTheme()
t.Name = "high-contrast"
t.Bg, t.Fg, t.Muted = RGB(0, 0, 0), RGB(0xff, 0xff, 0xff), RGB(0xd0, 0xd0, 0xd0)
t.HeaderBg, t.StatusBg, t.PanelBg = RGB(0, 0, 0), RGB(0x20, 0x20, 0x20), RGB(0, 0, 0)
t.Sel, t.Accent = RGB(0xff, 0xff, 0x00), RGB(0x00, 0xff, 0xff)
t.Cat = [inventory.NumCategories]Color{
RGB(0xb0, 0xb0, 0xb0), RGB(0xff, 0xd7, 0x00), RGB(0xff, 0x30, 0x30), RGB(0xff, 0x8c, 0x00),
RGB(0xff, 0x00, 0xff), RGB(0x9b, 0x7b, 0xff), RGB(0x00, 0xe5, 0xff), RGB(0x00, 0xff, 0x00),
RGB(0x40, 0x9c, 0xff), RGB(0xff, 0x55, 0x55), RGB(0xc8, 0xff, 0x00), RGB(0x90, 0x90, 0xa0),
}
return t
}
func monoTheme() *Theme {
return &Theme{Name: "mono", Mono: true}
}
// LoadTheme returns a built-in theme with user overrides applied.
func LoadTheme(name string, s config.Settings) *Theme {
f, ok := themes[strings.ToLower(name)]
if !ok {
f = defaultTheme
}
t := f()
if t.Mono {
return t
}
for k, v := range s.CategoryColors {
if c, ok := inventory.ParseCategory(k); ok {
if col, ok := Hex(v); ok {
t.Cat[c] = col
}
}
}
for k, v := range s.ExtensionColors {
if col, ok := Hex(v); ok {
if t.Ext == nil {
t.Ext = map[string]Color{}
}
t.Ext[strings.ToLower(strings.TrimPrefix(k, "."))] = col
}
}
return t
}
// Styles derived from the theme.
func (t *Theme) base() Style { return Style{FG: t.Fg, BG: t.Bg} }
func (t *Theme) muted() Style { return Style{FG: t.Muted, BG: t.Bg} }
func (t *Theme) header() Style { return Style{FG: t.HeaderFg, BG: t.HeaderBg} }
func (t *Theme) status() Style { return Style{FG: t.StatusFg, BG: t.StatusBg} }
func (t *Theme) panel() Style { return Style{FG: t.Fg, BG: t.PanelBg} }
func (t *Theme) box() BoxChars {
if t.ASCII {
return BoxASCII
}
return BoxLight
}
func (t *Theme) selBox() BoxChars {
if t.ASCII {
return BoxASCIISel
}
return BoxDouble
}
// textOn picks a readable text colour for a background.
func (t *Theme) textOn(bg Color) Color {
if bg == 0 {
return t.Fg
}
if bg.Luma() > 0.55 {
return RGB(0x10, 0x12, 0x16)
}
return RGB(0xf2, 0xf4, 0xf7)
}
internal/tui/tui_test.go
-499
View File
@@ -1,499 +0,0 @@
package tui
import (
"context"
"fmt"
"math/rand"
"strings"
"testing"
tea "charm.land/bubbletea/v2"
"github.com/charmbracelet/colorprofile"
"github.com/charmbracelet/x/ansi"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/scan"
"github.com/andydixon/mapsize/internal/treemap"
)
const hostile = "evil\x1b]0;pwned\x07\x1b[2J.iso"
func testTree() *inventory.Tree {
t := inventory.New("/srv", inventory.KindDir)
add := func(p inventory.NodeID, name string, size int64) inventory.NodeID {
id := t.Add(p, name, inventory.KindFile)
n := t.Node(id)
n.Size, n.Alloc = size, size
n.Cat = inventory.CategoryFor(inventory.Ext(name), false, false)
n.MTime = 1_700_000_000_000_000_000
return id
}
dir := func(p inventory.NodeID, name string) inventory.NodeID {
id := t.Add(p, name, inventory.KindDir)
t.Node(id).Flags |= inventory.FlagScanned
return id
}
backups := dir(0, "backups")
for i := range 40 {
add(backups, fmt.Sprintf("dump-%02d.tar.gz", i), int64(1+i)<<30)
}
vms := dir(0, "vms")
add(vms, "ubuntu.qcow2", 86<<30)
add(vms, "win.vmdk", 40<<30)
media := dir(0, "media")
photos := dir(media, "photos 日本語")
for i := range 300 {
add(photos, fmt.Sprintf("img%03d.jpg", i), int64(i+1)<<20)
}
add(media, hostile, 9<<30)
add(media, "👨‍👩‍👧 family.mp4", 12<<30)
tiny := dir(0, "tiny")
for i := range 2000 {
add(tiny, fmt.Sprintf("f%d", i), 100)
}
add(0, "README", 10)
t.Recompute()
t.Stats.Complete = true
return t
}
func newTestModel(t *testing.T) *Model {
t.Helper()
tr := testTree()
m := newModel(Options{Start: func(context.Context) (*scan.Scanner, *inventory.Tree, error) { return nil, tr, nil }},
nil, tr, func() {})
m.profile = colorprofile.TrueColor
return m
}
func send(m *Model, msgs ...tea.Msg) {
for _, msg := range msgs {
m.Update(msg)
}
}
func key(s string) tea.KeyPressMsg {
switch s {
case "up":
return tea.KeyPressMsg{Code: tea.KeyUp}
case "down":
return tea.KeyPressMsg{Code: tea.KeyDown}
case "left":
return tea.KeyPressMsg{Code: tea.KeyLeft}
case "right":
return tea.KeyPressMsg{Code: tea.KeyRight}
case "enter":
return tea.KeyPressMsg{Code: tea.KeyEnter}
case "esc":
return tea.KeyPressMsg{Code: tea.KeyEscape}
case "space":
return tea.KeyPressMsg{Code: tea.KeySpace, Text: " "}
case "backspace":
return tea.KeyPressMsg{Code: tea.KeyBackspace}
case "home":
return tea.KeyPressMsg{Code: tea.KeyHome}
case "tab":
return tea.KeyPressMsg{Code: tea.KeyTab}
}
r := []rune(s)[0]
return tea.KeyPressMsg{Code: r, Text: s}
}
func size(w, h int) tea.WindowSizeMsg { return tea.WindowSizeMsg{Width: w, Height: h} }
// checkFrame verifies the rendered frame exactly fills w×h cells and that no
// filename control sequence leaked through.
func checkFrame(t *testing.T, m *Model, w, h int) string {
t.Helper()
content := m.View().Content
if strings.Contains(content, "\x1b]") || strings.Contains(content, "\x07") || strings.Contains(content, "\x1b[2J") {
t.Fatalf("terminal control sequence from a filename leaked into the frame")
}
lines := strings.Split(content, "\n")
if len(lines) != h {
t.Fatalf("%dx%d: frame has %d lines", w, h, len(lines))
}
for i, l := range lines {
if got := ansi.StringWidth(l); got != w {
t.Fatalf("%dx%d: line %d has width %d: %q", w, h, i, got, ansi.Strip(l))
}
}
return ansi.Strip(content)
}
func TestResizeTransitionsPreserveState(t *testing.T) {
m := newTestModel(t)
send(m, size(120, 40))
checkFrame(t, m, 120, 40)
// Zoom into media and select something that is not the default.
m.tree.RLock()
media := m.kidsOf(0)
m.tree.RUnlock()
var mediaID inventory.NodeID
for _, id := range media.ids {
if m.tree.Node(id).Name == "media" {
mediaID = id
}
}
m.sel, m.userSel = int64(mediaID), true
send(m, key("space"))
if m.zoom != mediaID {
t.Fatalf("zoom = %d want media %d", m.zoom, mediaID)
}
checkFrame(t, m, 120, 40)
send(m, key("right"))
sel := m.sel
for _, sz := range [][2]int{{80, 24}, {120, 40}, {300, 80}, {70, 20}, {180, 50}, {40, 10}, {39, 9}, {200, 60}} {
send(m, size(sz[0], sz[1]))
checkFrame(t, m, sz[0], sz[1])
if m.zoom != mediaID {
t.Fatalf("%v: zoom lost", sz)
}
if m.sel != sel && !isGroup(m.sel) {
t.Fatalf("%v: selection changed from %d to %d", sz, sel, m.sel)
}
}
// The layout must be recomputed for the new geometry, not stretched.
if m.tm == nil || m.tm.rect.W > 200 || m.tm.rect.H > 60 {
t.Fatalf("layout not recomputed for final size: %+v", m.tm.rect)
}
for _, b := range m.tm.blocks {
r := m.tm.rect
if b.X < r.X || b.Y < r.Y || b.X+b.W > r.X+r.W || b.Y+b.H > r.Y+r.H {
t.Fatalf("block %+v outside treemap %+v", b, r)
}
}
}
func TestResizeStorm(t *testing.T) {
m := newTestModel(t)
r := rand.New(rand.NewSource(1))
send(m, size(120, 40))
m.View()
send(m, key("right"), key("down"))
sel := m.sel
for i := 0; i < 300; i++ {
send(m, size(20+r.Intn(300), 5+r.Intn(90)))
if i%7 == 0 {
m.View() // renders interleave with resizes like the real event loop
}
}
send(m, size(150, 45))
checkFrame(t, m, 150, 45)
if m.sel != sel {
t.Fatalf("selection lost in resize storm")
}
}
func TestModalSurvivesResize(t *testing.T) {
m := newTestModel(t)
send(m, size(120, 40), key("enter"))
if len(m.modals) != 1 {
t.Fatal("Enter should open the info modal")
}
for _, sz := range [][2]int{{120, 40}, {60, 14}, {41, 10}, {300, 80}, {45, 12}} {
send(m, size(sz[0], sz[1]))
out := checkFrame(t, m, sz[0], sz[1])
if !strings.Contains(out, "ITEM INFORMATION") {
t.Fatalf("%v: modal title missing:\n%s", sz, out)
}
f := m.modals[0].(*modalFrame)
if f.viewH < 1 || f.viewH > sz[1] {
t.Fatalf("%v: modal viewport %d", sz, f.viewH)
}
}
// Scrolling in a tiny terminal must stay in range.
send(m, size(45, 12))
for range 100 {
send(m, key("down"))
}
checkFrame(t, m, 45, 12)
f := m.modals[0].(*modalFrame)
if f.scroll < 0 || f.scroll > f.contentN {
t.Fatalf("scroll out of range: %d of %d", f.scroll, f.contentN)
}
send(m, key("esc"))
if len(m.modals) != 0 {
t.Fatal("Esc should close the modal")
}
}
func TestSpatialNavigationUsesGeometry(t *testing.T) {
m := newTestModel(t)
send(m, size(160, 45))
m.View()
blocks := m.tm.blocks
for _, dir := range []string{"right", "down", "left", "up"} {
before := m.sel
send(m, key(dir))
m.View()
if m.sel == before {
continue // edge of the map
}
var a, b = findBlock(blocks, before), findBlock(blocks, m.sel)
switch dir {
case "right":
if b.X < a.X+a.W {
t.Fatalf("right moved to a block that is not to the right: %+v -> %+v", a, b)
}
case "left":
if b.X+b.W > a.X {
t.Fatalf("left moved wrongly: %+v -> %+v", a, b)
}
case "down":
if b.Y < a.Y+a.H {
t.Fatalf("down moved wrongly: %+v -> %+v", a, b)
}
case "up":
if b.Y+b.H > a.Y {
t.Fatalf("up moved wrongly: %+v -> %+v", a, b)
}
}
}
}
func findBlock(bs []treemap.Block, id int64) treemap.Block {
for _, b := range bs {
if b.ID == id {
return b
}
}
return treemap.Block{}
}
func TestZoomBreadcrumbsAndHome(t *testing.T) {
m := newTestModel(t)
send(m, size(100, 30))
m.tree.RLock()
var media, photos inventory.NodeID
m.tree.Walk(0, func(id inventory.NodeID, n *inventory.Node) bool {
switch n.Name {
case "media":
media = id
case "photos 日本語":
photos = id
}
return true
})
m.tree.RUnlock()
m.sel, m.userSel = int64(media), true
send(m, key("space"))
m.sel = int64(photos)
send(m, key("space"))
if m.zoom != photos {
t.Fatal("did not zoom into photos")
}
out := checkFrame(t, m, 100, 30)
if !strings.Contains(out, "media › photos 日本語") {
t.Fatalf("breadcrumbs missing:\n%s", out)
}
// Narrow terminal: breadcrumbs truncate from the left, keeping the tail.
send(m, size(42, 12))
out = checkFrame(t, m, 42, 12)
if !strings.Contains(out, "photos") {
t.Fatalf("breadcrumb tail missing:\n%s", out)
}
send(m, key("backspace"))
if m.zoom != media || m.sel != int64(photos) {
t.Fatalf("backspace: zoom %d sel %d", m.zoom, m.sel)
}
send(m, key("home"))
if m.zoom != 0 {
t.Fatal("home did not return to root")
}
}
func TestFilterTransformsMap(t *testing.T) {
m := newTestModel(t)
send(m, size(120, 40), key("/"))
for _, r := range "ext = gz" {
send(m, tea.KeyPressMsg{Code: r, Text: string(r)})
}
if m.search.err != "" || m.search.query == nil {
t.Fatalf("query not parsed: %q", m.search.err)
}
msg := m.applyFilter()()
send(m, msg, key("enter"))
if m.search.result == nil {
t.Fatal("no filter result")
}
out := checkFrame(t, m, 120, 40)
if strings.Contains(out, "vms/") || !strings.Contains(out, "backups/") {
t.Fatalf("filter not applied to the map:\n%s", out)
}
if !strings.Contains(out, "FILTER") {
t.Fatal("filter indicator missing")
}
send(m, key("esc"))
if m.search.query != nil || m.search.result != nil {
t.Fatal("Esc should clear the filter")
}
out = checkFrame(t, m, 120, 40)
if !strings.Contains(out, "vms/") {
t.Fatalf("map not restored after clearing filter:\n%s", out)
}
}
func TestHostileNameSanitisedInModal(t *testing.T) {
m := newTestModel(t)
send(m, size(120, 40))
var id inventory.NodeID
m.tree.Walk(0, func(i inventory.NodeID, n *inventory.Node) bool {
if n.Name == hostile {
id = i
}
return true
})
m.openModal(newInfoModal(m, id))
out := checkFrame(t, m, 120, 40)
if !strings.Contains(out, `\x1b]0;pwned\x07`) {
t.Fatalf("escaped name not shown:\n%s", out)
}
}
func TestViewsRenderAtManySizes(t *testing.T) {
m := newTestModel(t)
for range m.views {
for _, sz := range [][2]int{{40, 10}, {80, 24}, {150, 45}, {91, 22}} {
send(m, size(sz[0], sz[1]))
checkFrame(t, m, sz[0], sz[1])
send(m, key("down"), key("down"))
checkFrame(t, m, sz[0], sz[1])
}
send(m, key("tab"))
}
send(m, key("?"))
send(m, size(60, 15))
checkFrame(t, m, 60, 15)
}
func TestLevelOfDetailGroupsTinyFiles(t *testing.T) {
m := newTestModel(t)
send(m, size(100, 30))
m.tree.RLock()
var tiny inventory.NodeID
m.tree.Children(0, func(id inventory.NodeID, n *inventory.Node) {
if n.Name == "tiny" {
tiny = id
}
})
m.tree.RUnlock()
m.zoomTo(tiny)
m.View()
if len(m.tm.blocks) > 100*30/topMinArea+1 {
t.Fatalf("%d blocks for 2000 equal files: LOD not applied", len(m.tm.blocks))
}
if _, ok := m.tm.groups[groupID(tiny)]; !ok {
t.Fatal("expected a group block for small items")
}
}
func TestCtrlCQuitsWhenIdle(t *testing.T) {
m := newTestModel(t)
send(m, size(80, 24))
_, cmd := m.Update(tea.KeyPressMsg{Code: 'c', Mod: tea.ModCtrl})
if cmd == nil || !m.quit {
t.Fatal("Ctrl+C should quit when no scan runs")
}
}
func BenchmarkRenderFrame(b *testing.B) {
tr := testTree()
m := newModel(Options{}, nil, tr, func() {})
m.profile = colorprofile.TrueColor
send(m, size(200, 60))
for b.Loop() {
m.dirty = true
m.View()
}
}
func BenchmarkResizeRelayout(b *testing.B) {
tr := testTree()
m := newModel(Options{}, nil, tr, func() {})
i := 0
for b.Loop() {
send(m, size(100+i%100, 30+i%30))
m.View()
i++
}
}
func TestMouse(t *testing.T) {
m := newTestModel(t)
send(m, size(120, 40))
m.View()
// Click the centre of a block other than the selected one.
var target treemap.Block
for _, b := range m.tm.blocks {
if b.ID != m.sel && b.ID >= 0 && m.tree.Node(inventory.NodeID(b.ID)).IsDir() && b.W > 4 {
target = b
break
}
}
click := tea.MouseClickMsg{X: target.X + target.W/2, Y: target.Y + target.H/2, Button: tea.MouseLeft}
send(m, click)
if m.sel != target.ID {
t.Fatalf("click selected %d, want %d", m.sel, target.ID)
}
m.View()
send(m, click) // second click within the double-click window
if m.zoom != inventory.NodeID(target.ID) {
t.Fatalf("double click should zoom into %d (zoom=%d)", target.ID, m.zoom)
}
m.View()
send(m, tea.MouseWheelMsg{X: 50, Y: 20, Button: tea.MouseWheelDown})
if m.zoom != 0 {
t.Fatal("wheel down should zoom out")
}
// A modal swallows clicks; clicking outside closes it.
send(m, key("?"))
m.View()
send(m, tea.MouseClickMsg{X: 0, Y: 39, Button: tea.MouseLeft})
if len(m.modals) != 0 {
t.Fatal("click outside modal should close it")
}
}
func TestQuantizers(t *testing.T) {
if quant16(RGB(10, 12, 8)) != 0 {
t.Fatal("near-black should map to black")
}
if n := quant16(RGB(0xe0, 0x6c, 0x75)); n != 1 && n != 9 {
t.Fatalf("red-ish mapped to %d", n)
}
if n := quant256(RGB(40, 48, 30)); n < 16 {
t.Fatalf("256 quantizer used palette-dependent index %d", n)
}
// A dark, slightly tinted shade must land on the grey ramp or a dark
// cube entry, not a saturated one like olive (58) or navy (17).
if n := quant256(RGB(40, 48, 30)); n == 58 || n == 17 || n == 22 {
t.Fatalf("dark tint mapped to saturated %d", n)
}
}
func TestColourDepthsRenderSameGeometry(t *testing.T) {
for _, p := range []colorprofile.Profile{colorprofile.TrueColor, colorprofile.ANSI256, colorprofile.ANSI, colorprofile.ASCII} {
m := newTestModel(t)
m.profile = p
send(m, size(100, 30))
checkFrame(t, m, 100, 30)
}
}
// A filename with a newline would run commands when pasted into a shell.
func TestCopyRefusesControlCharacters(t *testing.T) {
m := newTestModel(t)
m.copyPath("/srv/x\ncurl evil | sh\n")
if !strings.HasPrefix(m.toast, "Not copied") {
t.Fatalf("hostile path copied: %q", m.toast)
}
m.copyPath("/srv/media/" + hostile)
if !strings.HasPrefix(m.toast, "Not copied") {
t.Fatalf("escape sequence copied: %q", m.toast)
}
m.copyPath("/srv/photos 日本語/img001.jpg")
if !strings.HasPrefix(m.toast, "Copied") {
t.Fatalf("plain path refused: %q", m.toast)
}
}
internal/tui/views.go
-457
View File
@@ -1,457 +0,0 @@
package tui
import (
"fmt"
"sort"
"time"
tea "charm.land/bubbletea/v2"
"github.com/andydixon/mapsize/internal/inventory"
"github.com/andydixon/mapsize/internal/snapshot"
"github.com/andydixon/mapsize/internal/textutil"
"github.com/andydixon/mapsize/internal/treemap"
)
// ---- Directory list -------------------------------------------------------
type listView struct {
tb table
lastSel int64
}
func (*listView) name() string { return "List" }
// sync reconciles the table cursor with the shared selection: if the
// selection changed elsewhere (treemap, jump) the cursor follows it;
// otherwise the cursor (moved by keys or clicks) drives the selection.
func (v *listView) sync(m *Model) {
k := m.kidsOf(m.zoom)
if m.sel != v.lastSel {
for i, id := range k.ids {
if int64(id) == m.sel {
v.tb.cursor = i
}
}
} else if v.tb.cursor < len(k.ids) {
m.sel = int64(k.ids[v.tb.cursor])
}
v.lastSel = m.sel
}
func (v *listView) paint(m *Model, cv *Canvas, r treemap.Rect) {
t := m.theme
k := m.kidsOf(m.zoom)
v.sync(m)
total := m.sizeOf(m.zoom)
rows := make([]tableRow, len(k.ids))
for i, id := range k.ids {
n := m.tree.Node(id)
name := textutil.Sanitize(n.Name)
files := ""
if n.IsDir() {
name += "/"
files = textutil.Count(int64(n.Files))
}
mod := ""
if n.MTime > 0 {
mod = time.Unix(0, n.MTime).Format("2006-01-02")
}
col := m.colorOf(int64(id))
nameSt := &Style{FG: col.Mix(t.Fg, 0.45), BG: t.Bg}
if n.IsDir() {
nameSt.Attr = Bold
}
frac := 0.0
if k.sizes[0] > 0 {
frac = float64(k.sizes[i]) / float64(k.sizes[0])
}
rows[i] = tableRow{
cells: []string{textutil.Size(k.sizes[i]), textutil.Percent(k.sizes[i], total), "", files, mod, name},
styles: []*Style{nil, {FG: t.Muted, BG: t.Bg}, nil, {FG: t.Muted, BG: t.Bg}, {FG: t.Muted, BG: t.Bg}, nameSt},
bar: frac, barCol: col,
}
if m.opts.Diff != nil {
rows[i].cells[4] = textutil.SignedSize(m.opts.Diff.Delta(id, m.sizeMode))
}
}
cols := []column{{"SIZE", 10, true, false}, {"%", 5, true, false}, {"", 12, false, false}, {"FILES", 10, true, false}, {"MODIFIED", 10, false, false}, {"NAME", 0, false, false}}
if m.opts.Diff != nil {
cols[4] = column{"CHANGE", 12, true, false}
}
if r.W < 70 {
cols = []column{{"SIZE", 10, true, false}, {"%", 5, true, false}, {"NAME", 0, false, false}}
for i := range rows {
rows[i].cells = []string{rows[i].cells[0], rows[i].cells[1], rows[i].cells[5]}
rows[i].styles = []*Style{nil, rows[i].styles[1], rows[i].styles[5]}
}
}
v.tb.paint(m, cv, r, cols, rows, "")
v.sync(m)
}
func (v *listView) key(m *Model, k string) (bool, tea.Cmd) {
v.sync(m)
defer v.sync(m)
return v.tb.key(k, len(m.kidsOf(m.zoom).ids)), nil
}
func (v *listView) wheel(m *Model, up bool) tea.Cmd {
v.sync(m)
v.tb.scroll(up, len(m.kidsOf(m.zoom).ids))
v.sync(m)
return nil
}
// ---- Extension statistics ------------------------------------------------
type extView struct {
tb table
cache []inventory.ExtStat
cacheKey [2]int
}
func (*extView) name() string { return "Types" }
func (v *extView) stats(m *Model) []inventory.ExtStat {
k := [2]int{int(m.zoom), m.dataVer*2 + int(m.sizeMode)}
if v.cache == nil || v.cacheKey != k {
v.cache = m.tree.ExtStats(m.zoom)
sort.Slice(v.cache, func(i, j int) bool {
a, b := v.cache[i], v.cache[j]
as, bs := a.Alloc, b.Alloc
if m.sizeMode == inventory.SizeLogical {
as, bs = a.Size, b.Size
}
if as != bs {
return as > bs
}
return a.Ext < b.Ext
})
v.cacheKey = k
}
return v.cache
}
func (v *extView) paint(m *Model, cv *Canvas, r treemap.Rect) {
t := m.theme
st := v.stats(m)
var total int64
for _, s := range st {
total += pick(m, s)
}
rows := make([]tableRow, len(st))
for i, s := range st {
ext := "." + s.Ext
if s.Ext == "" {
ext = "(none)"
}
col := t.Cat[s.Cat]
if c, ok := t.Ext[s.Ext]; ok {
col = c
}
frac := 0.0
if v0 := pick(m, st[0]); v0 > 0 {
frac = float64(pick(m, s)) / float64(v0)
}
rows[i] = tableRow{
cells: []string{textutil.Sanitize(ext), s.Cat.String(), textutil.Count(s.Count), textutil.Size(pick(m, s)), textutil.Percent(pick(m, s), total), ""},
styles: []*Style{{FG: col, BG: t.Bg, Attr: Bold}, {FG: t.Muted, BG: t.Bg}, nil, nil, {FG: t.Muted, BG: t.Bg}, nil},
bar: frac, barCol: col,
}
}
cols := []column{{"EXTENSION", 14, false, false}, {"CATEGORY", 12, false, false}, {"FILES", 11, true, false}, {"SIZE", 10, true, false}, {"%", 5, true, false}, {"", 0, false, false}}
v.tb.paint(m, cv, r, cols, rows, fmt.Sprintf("File types under %s — Enter filters the map by extension", textutil.Sanitize(m.tree.Node(m.zoom).Name)))
}
func pick(m *Model, s inventory.ExtStat) int64 {
if m.sizeMode == inventory.SizeLogical {
return s.Size
}
return s.Alloc
}
func (v *extView) key(m *Model, k string) (bool, tea.Cmd) {
st := v.stats(m)
if v.tb.key(k, len(st)) {
return true, nil
}
if (k == "enter" || k == "space") && v.tb.cursor < len(st) {
e := st[v.tb.cursor].Ext
q := "ext = \"" + e + "\""
if e == "" {
q = "type = file AND ext = \"\""
}
m.setViewByName("Map")
return true, m.setFilter(q)
}
return false, nil
}
func (v *extView) wheel(m *Model, up bool) tea.Cmd {
v.tb.scroll(up, len(v.stats(m)))
return nil
}
// ---- Top-N investigation views --------------------------------------------
type topMode int
const (
topFiles topMode = iota
topDirs
topOld
topNew
topCount
topSparse
topHardlinks
numTopModes
)
var topNames = [...]string{"Largest files", "Largest directories (own files)", "Oldest large files",
"Newest large files", "Most files", "Sparse files", "Hard-linked files"}
type topView struct {
tb table
mode topMode
ids []inventory.NodeID
vals []int64
cacheKey [3]int
limit int
}
func (*topView) name() string { return "Top" }
const topLimit = 500
func (v *topView) compute(m *Model) {
k := [3]int{int(m.zoom), m.dataVer*2 + int(m.sizeMode), int(v.mode)}
if v.ids != nil && v.cacheKey == k {
return
}
v.cacheKey = k
tr := m.tree
isFile := func(n *inventory.Node) bool {
return n.Kind == inventory.KindFile && n.Flags&(inventory.FlagHardlinkDup|inventory.FlagDeleted) == 0
}
matches := func(id inventory.NodeID) bool {
return m.search.result == nil || m.search.result.Size(id) > 0
}
tk := inventory.NewTopK(topLimit)
val := map[inventory.NodeID]int64{}
const largeFile = 1 << 20
switch v.mode {
case topDirs:
own := map[inventory.NodeID]int64{}
tr.Walk(m.zoom, func(id inventory.NodeID, n *inventory.Node) bool {
if isFile(n) && matches(id) {
own[n.Parent] += n.Own(m.sizeMode)
}
return true
})
for id, s := range own {
tk.Offer(id, s)
val[id] = s
}
case topCount:
tr.Walk(m.zoom, func(id inventory.NodeID, n *inventory.Node) bool {
if n.IsDir() && matches(id) {
c := int64(tr.ChildCount(id))
tk.Offer(id, c)
val[id] = c
}
return true
})
default:
tr.Walk(m.zoom, func(id inventory.NodeID, n *inventory.Node) bool {
if !isFile(n) || !matches(id) {
return true
}
sz := n.Own(m.sizeMode)
switch v.mode {
case topFiles:
tk.Offer(id, sz)
val[id] = sz
case topOld, topNew:
if sz >= largeFile && n.MTime > 0 {
key := n.MTime
if v.mode == topOld {
key = -key
}
tk.Offer(id, key)
val[id] = sz
}
case topSparse:
if n.Flags&inventory.FlagSparse != 0 {
tk.Offer(id, n.Size-n.Alloc)
val[id] = n.Size - n.Alloc
}
case topHardlinks:
if n.Flags&inventory.FlagHardlinked != 0 {
tk.Offer(id, n.Size)
val[id] = n.Size
}
}
return true
})
}
v.ids = tk.Sorted()
v.vals = v.vals[:0]
for _, id := range v.ids {
v.vals = append(v.vals, val[id])
}
}
func (v *topView) paint(m *Model, cv *Canvas, r treemap.Rect) {
t := m.theme
v.compute(m)
rows := make([]tableRow, len(v.ids))
var first int64
if len(v.vals) > 0 {
first = v.vals[0]
}
for i, id := range v.ids {
n := m.tree.Node(id)
val := textutil.Size(v.vals[i])
if v.mode == topCount {
val = textutil.Count(v.vals[i])
}
mod := ""
if n.MTime > 0 {
mod = time.Unix(0, n.MTime).Format("2006-01-02")
}
frac := 0.0
if first > 0 && v.mode != topOld && v.mode != topNew {
frac = float64(v.vals[i]) / float64(first)
}
col := m.colorOf(int64(id))
rows[i] = tableRow{
cells: []string{val, mod, "", textutil.Sanitize(m.tree.Path(id))},
styles: []*Style{nil, {FG: t.Muted, BG: t.Bg}, nil, {FG: col.Mix(t.Fg, 0.45), BG: t.Bg}},
bar: frac, barCol: col,
}
}
title := fmt.Sprintf("◂ %s ▸ (←/→ change list · Enter details · Space jump to it)", topNames[v.mode])
if m.search.result != nil {
title += " · filtered"
}
valTitle := "SIZE"
switch v.mode {
case topCount:
valTitle = "ENTRIES"
case topSparse:
valTitle = "SAVED"
}
cols := []column{{valTitle, 10, true, false}, {"MODIFIED", 10, false, false}, {"", 10, false, false}, {"PATH", 0, false, true}}
v.tb.paint(m, cv, r, cols, rows, title)
}
func (v *topView) key(m *Model, k string) (bool, tea.Cmd) {
v.compute(m)
switch k {
case "left", "h":
v.mode = (v.mode + numTopModes - 1) % numTopModes
v.tb.cursor = 0
return true, nil
case "right", "l":
v.mode = (v.mode + 1) % numTopModes
v.tb.cursor = 0
return true, nil
}
if v.tb.key(k, len(v.ids)) {
return true, nil
}
if v.tb.cursor < len(v.ids) {
id := v.ids[v.tb.cursor]
switch k {
case "enter":
m.openModal(newInfoModal(m, id))
return true, nil
case "space":
m.jumpTo(id)
return true, nil
case "c", "o", "d":
m.sel = int64(id)
}
}
return false, nil
}
func (v *topView) wheel(m *Model, up bool) tea.Cmd {
v.tb.scroll(up, len(v.ids))
return nil
}
// ---- Snapshot comparison -----------------------------------------------------
type changesView struct {
tb table
rows []snapshot.Change
cacheKey int
}
func (*changesView) name() string { return "Changes" }
func (v *changesView) paint(m *Model, cv *Canvas, r treemap.Rect) {
t := m.theme
d := m.opts.Diff
if v.rows == nil || v.cacheKey != int(m.sizeMode) {
v.rows = d.Changes(1000, m.sizeMode)
v.cacheKey = int(m.sizeMode)
}
var maxAbs int64 = 1
for _, c := range v.rows {
maxAbs = max(maxAbs, abs64(c.Delta))
}
rows := make([]tableRow, len(v.rows))
for i, c := range v.rows {
col := map[snapshot.Status]Color{snapshot.Added: t.DiffNew, snapshot.Removed: t.Muted,
snapshot.Grew: t.DiffGrow, snapshot.Shrank: t.DiffShrink}[c.Status]
rows[i] = tableRow{
cells: []string{textutil.SignedSize(c.Delta), c.Status.String(), textutil.Size(c.Old), textutil.Size(c.New), "", textutil.Sanitize(c.Path)},
styles: []*Style{{FG: col, BG: t.Bg, Attr: Bold}, {FG: col, BG: t.Bg}, {FG: t.Muted, BG: t.Bg}, nil, nil, nil},
bar: float64(abs64(c.Delta)) / float64(maxAbs), barCol: col,
}
}
cols := []column{{"CHANGE", 13, true, false}, {"STATUS", 9, false, false}, {"OLD", 10, true, false}, {"NEW", 10, true, false},
{"", 10, false, false}, {"PATH", 0, false, true}}
ro, rn := d.Old.Node(0).Total(m.sizeMode), d.New.Node(0).Total(m.sizeMode)
title := fmt.Sprintf("Changes %s → %s (%s)", textutil.Size(ro), textutil.Size(rn), textutil.SignedSize(rn-ro))
v.tb.paint(m, cv, r, cols, rows, title)
}
func abs64(v int64) int64 {
if v < 0 {
return -v
}
return v
}
func (v *changesView) key(m *Model, k string) (bool, tea.Cmd) {
if v.tb.key(k, len(v.rows)) {
return true, nil
}
if v.tb.cursor < len(v.rows) {
c := v.rows[v.tb.cursor]
if c.Node == inventory.NoNode {
if k == "enter" || k == "space" {
return true, m.info("Removed since the old snapshot: " + textutil.Sanitize(c.Path))
}
return false, nil
}
switch k {
case "enter":
m.openModal(newInfoModal(m, c.Node))
return true, nil
case "space":
m.jumpTo(c.Node)
return true, nil
}
}
return false, nil
}
func (v *changesView) wheel(m *Model, up bool) tea.Cmd {
v.tb.scroll(up, len(v.rows))
return nil
}
nfpm.yaml
-14
View File
@@ -1,14 +0,0 @@
name: mapsize
arch: ${ARCH}
version: ${VERSION}
maintainer: Andy Dixon <andy@dixon.cx>
description: Interactive full-terminal disk usage analyser with a live treemap.
homepage: https://github.com/andydixon/mapsize
license: GPL-3.0-or-later
contents:
- src: bin/release/pkg/mapsize
dst: /usr/bin/mapsize
- src: docs/mapsize.1
dst: /usr/share/man/man1/mapsize.1
file_info:
mode: 0644
packaging/homebrew/mapsize.rb.in
+3 -4
View File
@@ -1,6 +1,6 @@
# Homebrew formula template for mapsize. `make brew-formula` fills in the
# version and the checksum of the GitHub source tarball and writes the
# result to bin/homebrew/mapsize.rb; copy that into your tap
# result to dist/homebrew/mapsize.rb; copy that into your tap
# (github.com/andydixon/homebrew-tap, Formula/mapsize.rb).
class Mapsize < Formula
desc "Interactive terminal disk usage analyser with a navigable treemap"
@@ -10,11 +10,10 @@ class Mapsize < Formula
license "GPL-3.0-or-later"
head "https://github.com/andydixon/mapsize.git", branch: "master"
depends_on "go" => :build
depends_on "rust" => :build
def install
ldflags = "-X github.com/andydixon/mapsize/internal/brand.Version=#{version}"
system "go", "build", *std_go_args(ldflags:), "./cmd/mapsize"
system "cargo", "install", *std_cargo_args
man1.install "docs/mapsize.1"
end
packaging/repo.sh
+24 -12
View File
@@ -10,10 +10,11 @@
# Every index is rebuilt over all the packages in the repository, so other
# projects publishing there with their copy of this script are kept.
#
# Needs: go, docker, gpg, apt-ftparchive (apt-utils), flock.
# Needs: cargo with cargo-zigbuild and zig (static musl cross-builds), curl
# (to fetch the pinned nfpm), docker, gpg, apt-ftparchive (apt-utils), flock.
# REPO_DIR where the repository lives (default below)
# REPO_GNUPGHOME GnuPG home holding the signing key (default below)
# ARCHS Go architectures to package (default below)
# ARCHS package architectures (default below)
#
# The same script, with its own project block, is in ../vault/packaging.
set -euo pipefail
@@ -27,19 +28,30 @@ HOMEPAGE=https://github.com/andydixon/mapsize
LICENSE=GPL-3.0-or-later
MANPAGES=docs/*.1
DOCS="README.md CHANGELOG.md LICENSE"
# build OUT: build the binary to OUT, in the source tree, with GOOS, GOARCH
# and GOARM set and VERSION the release.
# Package architecture -> Rust target (static musl binaries).
triple() {
case $1 in
amd64) echo x86_64-unknown-linux-musl ;;
arm64) echo aarch64-unknown-linux-musl ;;
arm) echo armv7-unknown-linux-musleabihf ;;
386) echo i686-unknown-linux-musl ;;
riscv64) echo riscv64gc-unknown-linux-musl ;;
*) return 1 ;;
esac
}
# build ARCH OUT: build the binary for ARCH to OUT, in the source tree.
build() {
CGO_ENABLED=0 go build -trimpath \
-ldflags "-s -w -X github.com/andydixon/mapsize/internal/brand.Version=$VERSION" \
-o "$1" ./cmd/mapsize
local t
t=$(triple "$1") || die "no Rust target for $1"
RUSTFLAGS="--remap-path-prefix=$PWD=$NAME" cargo zigbuild --quiet --release --locked --target "$t"
install -D -m 0755 "target/$t/release/$NAME" "$2"
}
# ----------------------------------------------------------------------------
REPO_DIR=${REPO_DIR:-/home/andy/domains/repo.dixon.cx/public_html}
REPO_GNUPGHOME=${REPO_GNUPGHOME:-$HOME/.config/repo.dixon.cx/gnupg}
ARCHS=${ARCHS:-"amd64 arm64 arm 386 riscv64"}
NFPM="go run github.com/goreleaser/nfpm/v2/cmd/nfpm@v2.41.1"
NFPM="$(cd "$(dirname "$0")/.." && pwd)/scripts/nfpm.sh"
SUITE=stable
COMPONENT=main
ARCH_REPO=dixon # the pacman repository name: [dixon]
@@ -47,7 +59,7 @@ ARCH_REPO=dixon # the pacman repository name: [dixon]
say() { echo "repo: $*" >&2; }
die() { say "$*"; exit 1; }
for tool in go docker gpg apt-ftparchive flock; do
for tool in cargo cargo-zigbuild zig curl docker gpg apt-ftparchive flock; do
command -v $tool >/dev/null || die "needs $tool"
done
[[ -d $REPO_DIR ]] || die "$REPO_DIR does not exist"
@@ -70,9 +82,9 @@ gpg --batch --armor --export-secret-keys "$KEY" >"$WORK/signing.asc"
say "building $PACKAGE $VERSION for $ARCHS"
mkdir -p "$WORK/out"
for arch in $ARCHS; do
goarm= nfarch=$arch
[[ $arch == arm ]] && goarm=7 nfarch=arm7
(cd "$WORK/src" && GOOS=linux GOARCH=$arch GOARM=$goarm build "$WORK/bin-$arch/$NAME")
nfarch=$arch
[[ $arch == arm ]] && nfarch=arm7
(cd "$WORK/src" && build "$arch" "$WORK/bin-$arch/$NAME")
{
cat <<-EOF
name: $PACKAGE
rust-toolchain.toml
+4
View File
@@ -0,0 +1,4 @@
[toolchain]
channel = "1.99.0"
components = ["rustfmt", "clippy"]
profile = "minimal"
rust/.gitignore
-1
View File
@@ -1 +0,0 @@
/target
rust/README.md
-95
View File
@@ -1,95 +0,0 @@
# mapsize — Rust port
A port of the Go implementation (repo root) to Rust, with the same CLI, TUI,
keys, themes and snapshot format. `.msz` snapshots are byte-compatible in
both directions. The code is Linux/Unix only; the Windows code paths were
not ported.
```
cd rust
cargo build --release # target/release/mapsize
cargo test --release # 69 tests (ported from the Go suites, fuzz tests excepted)
```
Module layout mirrors `internal/`: `inventory`, `scan`, `platform`, `treemap`,
`filter`, `snapshot` (+ `compare`), `duplicate`, `export`, `config`,
`textutil`, `tui`, `app`. The tree is shared as `Arc<RwLock<Tree>>` with the
same single-writer invariants as ARCHITECTURE.md; `cancel::Cancel` replaces
`context.Context`; crossterm replaces Bubble Tea as the event loop (the
canvas renderer is ported as-is, plus line-diffed frame output).
## Known differences from Go
* `--log`/`--log-level` are accepted but there is no debug log yet.
* A scanner thread panic restores the terminal and aborts the process instead
of returning an "internal error" message.
* The colour profile is detected from `NO_COLOR`/`TERM`/`COLORTERM` only (no
terminal query); `--color` still overrides.
* JSON export does not HTML-escape `<>&`; `started` is written in UTC.
* The filter lexer no longer hangs on queries containing bytes Go treats as
white space but never skips (e.g. `voilà`, whose UTF-8 contains 0xA0).
* Huge sizes saturate rather than wrap in filters and snapshot comparison.
## Benchmarks
Machine: Intel Core i7-7700 (4C/8T), 32 GiB, ext4 on md RAID over 7200 rpm
HDDs. Go 1.27.1, Rust 1.91.1 (`lto`, `codegen-units = 1`). The host was
shared and heavily loaded during the runs (load average 30–50 on 8 threads),
so absolute times are inflated 2–4× compared with PERFORMANCE.md. To keep the
comparison fair, Go and Rust runs were interleaved and the tables report
best-of (micro) or median and best (end-to-end). Re-run on an idle machine
before quoting absolute numbers:
```
go test -c -o /tmp/tui.test ./internal/tui && go test -c -o /tmp/treemap.test ./internal/treemap
(cd rust && cargo build --release --example bench)
rust/bench/micro.py /tmp/tui.test /tmp/treemap.test rust/target/release/examples/bench 5
rust/bench/compare.py bin/mapsize rust/target/release/mapsize ~/go/pkg/mod 7
```
### In-process (Go `go test -bench` suite vs `examples/bench.rs`)
The tree is 1,008,201 synthetic nodes, rendered at 200×60.
| benchmark | Go | Rust | speed-up |
|---|---:|---:|---:|
| treemap/Squarify(300) | 299.3 µs | 22.2 µs | 13.48× |
| treemap/Neighbour(300) | 5.0 µs | 2.8 µs | 1.76× |
| tui/BigColdFrame | 86.88 ms | 29.32 ms | 2.96× |
| tui/BigResize | 7.15 ms | 4.76 ms | 1.50× |
| tui/BigFlatDirCold | 86.33 ms | 22.90 ms | 3.77× |
| tui/BigNavigate | 4.85 ms | 4.07 ms | 1.19× |
| tui/BigFilter/size > 1MB | 78.62 ms | 59.51 ms | 1.32× |
| tui/BigFilter/*.dat AND size > 500k | 302.03 ms | 154.90 ms | 1.95× |
| tui/BigFilter/path contains sub3 | 337.79 ms | 136.85 ms | 2.47× |
### End to end (`~/go/pkg/mod`: 388k files + 65k dirs, warm cache)
| scenario | Go median | Rust median | Go best | Rust best | Go RSS | Rust RSS | speed-up |
|---|---:|---:|---:|---:|---:|---:|---:|
| scan, 1 workers | 4.68 s | 5.51 s | 4.34 s | 4.86 s | 114 MB | 75 MB | 0.85× |
| scan, 8 workers | 2.03 s | 2.03 s | 1.61 s | 1.42 s | 113 MB | 78 MB | 1.00× |
| scan, 32 workers | 3.09 s | 2.59 s | 2.85 s | 2.07 s | 116 MB | 89 MB | 1.19× |
| scan + save snapshot | 4.69 s | 3.60 s | 4.02 s | 3.29 s | 117 MB | 80 MB | 1.30× |
| load own snapshot | 1.20 s | 0.71 s | 0.71 s | 0.48 s | 85 MB | 78 MB | 1.69× |
| load Go snapshot | 1.03 s | 1.14 s | 0.84 s | 0.48 s | 85 MB | 78 MB | 0.90× |
| scan + JSON export | 6.10 s | 4.74 s | 5.44 s | 4.16 s | 152 MB | 79 MB | 1.29× |
| scan + top 50 largest files | 4.49 s | 4.31 s | 3.62 s | 3.65 s | 117 MB | 79 MB | 1.04× |
Reading the numbers:
* **Scanning** is syscall-bound (`statx`/`getdents` were about 60% of Go's
profile), so wall time is roughly at parity. Rust edges ahead at high
worker counts. The single-worker case is about 15% slower, but that run was
the noisiest of the set.
* **Memory**: peak RSS is 30–48% lower (78 MB vs 113 MB while scanning
458k nodes). The JSON export streams without Go's per-node allocations
(79 MB vs 152 MB).
* **Snapshots**: loading is about 1.7× faster on the median. The two binaries
load each other's files.
* **UI**: cold frames are 3–3.8× faster, and squarified layout is about 13×
faster. Navigation and resize gain less (1.2–1.5×) because they mostly
reuse cached layout.
* **Filters**: 1.3–2.5× faster. This required caching lower-cased directory
paths in a `Vec` with reused buffers, and a `*.ext` fast path in the
`filepath.Match` port.

Some files were not shown because too many files have changed in this diff Show More