Initial Go project structure, docs, build and CI
This commit is contained in:
Andy Dixon
@@ -0,0 +1,30 @@
|
||||
name: ci
|
||||
on:
|
||||
push:
|
||||
pull_request:
|
||||
jobs:
|
||||
test:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
os: [ubuntu-latest, macos-latest, windows-latest]
|
||||
runs-on: ${{ matrix.os }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version-file: go.mod
|
||||
- name: format
|
||||
if: runner.os == 'Linux'
|
||||
run: test -z "$(gofmt -l .)"
|
||||
- run: go vet ./...
|
||||
- run: go test ./...
|
||||
- name: race
|
||||
if: runner.os != 'Windows'
|
||||
run: go test -race ./...
|
||||
- name: cross-build
|
||||
if: runner.os == 'Linux'
|
||||
run: |
|
||||
for t in linux/arm64 darwin/arm64 windows/amd64 freebsd/amd64; do
|
||||
GOOS=${t%/*} GOARCH=${t#*/} CGO_ENABLED=0 go build -o /dev/null ./cmd/mapsize
|
||||
done
|
||||
@@ -0,0 +1,5 @@
|
||||
bin/
|
||||
*.msz
|
||||
*.prof
|
||||
*.test
|
||||
/mapsize
|
||||
+279
@@ -0,0 +1,279 @@
|
||||
# mapsize architecture
|
||||
|
||||
This document describes how mapsize is put together and the invariants each
|
||||
part relies on. It is kept in sync with the code; if they disagree, the code
|
||||
is wrong or this file is stale — fix one of them.
|
||||
|
||||
```
|
||||
cmd/mapsize CLI entry point, flag parsing, mode selection
|
||||
cmd/treegen synthetic test-tree generator
|
||||
internal/brand product name / file extensions (rename in one place)
|
||||
internal/inventory flat node store, aggregation, indexes, categories
|
||||
internal/scan bounded concurrent scanner (controller + workers)
|
||||
internal/platform OS-specific metadata (build-tagged files)
|
||||
internal/treemap squarified layout, LOD, spatial navigation (pure geometry)
|
||||
internal/filter query lexer / parser / evaluator
|
||||
internal/snapshot versioned binary snapshot format + comparison
|
||||
internal/duplicate staged duplicate detection
|
||||
internal/export JSON / CSV / text reports
|
||||
internal/config settings, themes, JSON config file
|
||||
internal/tui Bubble Tea model, canvas renderer, views, modals
|
||||
internal/textutil display sanitising, width-aware truncation, units
|
||||
```
|
||||
|
||||
Dependency direction is strictly downward: `tui` → (`treemap`, `filter`,
|
||||
`inventory`, `scan`, …); `scan` → (`inventory`, `platform`); `treemap` depends
|
||||
on nothing but the standard library. The TUI framework (Bubble Tea v2) is only
|
||||
imported by `internal/tui` and `cmd/mapsize`.
|
||||
|
||||
## Inventory representation
|
||||
|
||||
`inventory.Tree` is an append-only arena of `Node` records addressed by
|
||||
`NodeID` (`uint32`). Nodes are stored in fixed-size chunks
|
||||
(`[][]Node`, 65 536 nodes per chunk), so growing the arena never copies or
|
||||
moves existing nodes and peak memory never doubles during growth.
|
||||
|
||||
A node holds: parent ID, first-child / next-sibling IDs (an intrusive child
|
||||
list — no per-directory slices), name (the single path component, not the full
|
||||
path), kind, flags, own logical size, own allocated size, mtime, mode,
|
||||
uid/gid, link count, and aggregates (total logical, total allocated, file
|
||||
count, directory count, error count). Directory-only data — per-category byte
|
||||
totals used for colouring — lives in a side table indexed by a directory index,
|
||||
so files do not pay for it.
|
||||
|
||||
Full paths are never stored; `Tree.Path(id)` walks parents. Extensions are
|
||||
interned into a small table and stored as a `uint16`. All sizes are `int64`.
|
||||
|
||||
Node IDs are stable for the lifetime of a `Tree`: nodes are never removed or
|
||||
reordered. A rescan produces a new `Tree`. This is what makes selection
|
||||
persistence trivial (see below).
|
||||
|
||||
## Scanner worker architecture
|
||||
|
||||
```
|
||||
+------------------ controller goroutine ------------------+
|
||||
| owns: Tree (write lock), pending FIFO of dir NodeIDs, |
|
||||
| hard-link table, visited-dir table |
|
||||
+----------------------------------------------------------+
|
||||
| jobs (bounded chan, cap = workers*2) ^ results (bounded chan)
|
||||
v |
|
||||
worker 1 .. worker N (fixed pool, N from --workers / mode)
|
||||
```
|
||||
|
||||
* Workers do I/O only: open a directory, read entries in chunks of up to 1024,
|
||||
`lstat` each entry (relative to the directory fd where the platform allows),
|
||||
and send a `result` chunk back. Workers never touch the `Tree`.
|
||||
* The controller is the only writer of the `Tree`. It applies result chunks,
|
||||
creates child nodes, assigns IDs, propagates aggregate deltas, and decides
|
||||
which subdirectories become new jobs.
|
||||
* Worker count: `conservative` = 2, `balanced` = min(8, 2×CPU),
|
||||
`aggressive` = min(64, 8×CPU), or an explicit number. More is not always
|
||||
faster — on rotational media 2–4 is often best.
|
||||
|
||||
## Bounded directory scheduling (no deadlock under backpressure)
|
||||
|
||||
The classic deadlock is: workers block sending new work into a full queue
|
||||
while the only consumer of that queue is also blocked. mapsize avoids it by
|
||||
construction:
|
||||
|
||||
* Only the controller produces jobs. Workers only produce *results*.
|
||||
* The controller's main loop is a single `select` that simultaneously offers
|
||||
the next pending job to the bounded `jobs` channel *and* receives from the
|
||||
bounded `results` channel. It never performs a blocking send without also
|
||||
being ready to receive. Therefore a worker blocked on `results <-` is always
|
||||
eventually served, and the controller is never blocked on `jobs <-` alone.
|
||||
* Discovered-but-unscheduled directories wait in the controller's pending FIFO
|
||||
as 4-byte NodeIDs (paths are rebuilt at dispatch). Its size is bounded by the
|
||||
number of directory nodes already in the inventory, so it adds O(1) bytes per
|
||||
directory on top of memory we already spend.
|
||||
* FIFO order means the scan is roughly breadth-first: every top-level directory
|
||||
starts growing early, which is what makes the live treemap informative.
|
||||
|
||||
Huge directories are streamed: a worker sends a result per 1024 entries, so a
|
||||
directory with 5 M entries neither balloons worker memory nor stalls the UI.
|
||||
|
||||
Completion: the controller tracks `inflight` (dispatched directories whose
|
||||
final chunk has not arrived). The scan is finished when `pending` is empty and
|
||||
`inflight == 0`. Cancellation (`context.Context`) stops dispatch immediately;
|
||||
workers check the context between chunks and between entries, and the
|
||||
controller drains outstanding results so no goroutine leaks.
|
||||
|
||||
Stress tests (`internal/scan/scan_test.go`) run with 1 worker and job/result
|
||||
channel capacity 1 over trees with thousands of directories to prove progress
|
||||
under maximum backpressure.
|
||||
|
||||
## Concurrent aggregation strategy
|
||||
|
||||
Because the controller is the single writer, aggregation needs no atomics.
|
||||
For each result chunk it sums the chunk's file sizes, allocated sizes, counts
|
||||
and per-category bytes **once**, then walks the ancestor chain adding that one
|
||||
delta. Cost is O(depth) per 1024-entry chunk rather than O(depth) per file —
|
||||
for a 1 M-file tree at depth 20 that is ~20 K ancestor updates instead of
|
||||
~20 M. There is no reconciliation pass: totals are exact at every moment for
|
||||
the data seen so far.
|
||||
|
||||
The UI reads the tree under `Tree.RLock`; the controller applies batches of
|
||||
results under `Tree.Lock`, holding it for at most a few milliseconds (it
|
||||
releases the lock between batches). This is the only lock in the data path.
|
||||
|
||||
**Concurrency invariants**
|
||||
|
||||
1. Only the scan controller (or a snapshot loader before publication) mutates
|
||||
a `Tree`.
|
||||
2. Every read of `Tree` node data from another goroutine holds `RLock`.
|
||||
3. Nodes are never deleted; a `NodeID` once observed stays valid.
|
||||
4. Workers share nothing mutable except the bounded channels and atomic
|
||||
progress counters.
|
||||
|
||||
## Hard links and sizes
|
||||
|
||||
* *Logical size* = `st_size`. *Allocated size* = `st_blocks × 512` on Unix.
|
||||
On Windows allocated size is reported as unknown (equal to logical, flagged)
|
||||
because getting it costs an extra handle-open per file.
|
||||
* For regular files with link count > 1, the controller keys `(dev, inode)`.
|
||||
The first path seen contributes to directory aggregates; later paths are
|
||||
flagged `HardlinkDup` and contribute **zero** to aggregates (their own size
|
||||
is still shown). Which path is "first" depends on scan order and is not
|
||||
deterministic. This matches `du`'s default semantics.
|
||||
* Windows does not expose link counts from directory enumeration, so hard
|
||||
links are not deduplicated there. This is documented, not hidden.
|
||||
|
||||
## Symlinks, mount points, exclusions
|
||||
|
||||
* `--follow-symlinks=none` (default): symlinks are leaf nodes counted at their
|
||||
own (tiny) size. `same-filesystem` / `all` follow directory symlinks; every
|
||||
descended directory's `(dev, inode)` is recorded and a directory already
|
||||
visited is never descended twice — loops terminate and nothing is counted
|
||||
twice.
|
||||
* A directory whose device differs from its parent is flagged `MountPoint`.
|
||||
With `--one-file-system` it is not descended. On Linux, virtual filesystems
|
||||
(proc, sysfs, cgroup, debugfs, …) are never descended — `/proc/kcore` would
|
||||
otherwise dominate any scan of `/` — and are reported in scan info.
|
||||
* `--exclude PATTERN`: patterns containing a path separator match full paths;
|
||||
others match the entry name with glob semantics. Excluded entries are
|
||||
counted and the UI shows that exclusions are active.
|
||||
|
||||
## TUI event flow
|
||||
|
||||
```
|
||||
scan controller ──(applies under Lock)──> Tree
|
||||
│ atomic progress counters
|
||||
▼
|
||||
tea.Tick (10 Hz while scanning) ──> tickMsg ──> Update marks frame dirty
|
||||
key/mouse/resize ──> Update ──> state change ──> frame dirty
|
||||
View(): if dirty → RLock tree → layout (if layout dirty) → paint canvas → string
|
||||
```
|
||||
|
||||
There is never one message per file: the scanner does not talk to the UI at
|
||||
all except for a single `scanDoneMsg`. The UI polls at a fixed rate.
|
||||
|
||||
`View()` is called by Bubble Tea after *every* `Update`. The model therefore
|
||||
caches the rendered frame and only repaints when something visible changed.
|
||||
|
||||
## Rendering: the canvas
|
||||
|
||||
All drawing goes to an in-memory `Canvas` of cells (grapheme, fg, bg, attrs).
|
||||
Views and modals paint into the canvas; the canvas is serialised to a string
|
||||
with minimal SGR changes. Bubble Tea's renderer diffs frames and downsamples
|
||||
true-colour to the detected profile (256 / 16 / none), so the drawing code
|
||||
uses RGB throughout. Selection never relies on colour alone: the selected
|
||||
rectangle uses a double-line border, bold reversed title and a status-line
|
||||
description.
|
||||
|
||||
## Treemap algorithm
|
||||
|
||||
`treemap.Squarify` implements Bruls, Huizing & van Wijk's squarified layout
|
||||
on float rectangles: items sorted by size descending are added to the current
|
||||
row while the worst aspect ratio improves; then the row is laid out along the
|
||||
shorter side. Float rectangles are snapped to the cell grid by rounding the
|
||||
*edges* (not the widths), so neighbouring rectangles share edges exactly — no
|
||||
gaps, no overlaps. Layout is deterministic (stable sort by size, then ID).
|
||||
|
||||
Nested levels are laid out recursively inside each block's interior (inside
|
||||
its border) while the interior is large enough to be useful.
|
||||
|
||||
## Level-of-detail model
|
||||
|
||||
A 160×50 terminal has 8 000 cells; the tree may have millions of entries.
|
||||
For each directory being laid out, children are reduced to the top K by size
|
||||
using a bounded heap (O(n log K)). Children whose proportional area would be
|
||||
smaller than a minimum useful block (a few cells), or beyond K, are merged
|
||||
into one synthetic *group* block ("312 smaller items, 8.4 GiB"). Zooming into
|
||||
a directory gives its children the whole screen, so they become individually
|
||||
visible. Groups are selectable and inspectable.
|
||||
|
||||
## Spatial navigation algorithm
|
||||
|
||||
`treemap.Neighbour(rects, selected, dir)` is pure geometry:
|
||||
|
||||
1. Candidates must lie in the requested half-plane: for RIGHT, `c.X ≥
|
||||
sel.X + sel.W − tolerance` (with the candidate's centre to the right of the
|
||||
selected centre).
|
||||
2. Each candidate is scored by the primary-axis gap (edge to edge) plus a
|
||||
penalty for the orthogonal offset. Candidates overlapping the selected
|
||||
rectangle's orthogonal span get the offset term set to zero and are
|
||||
preferred over any non-overlapping candidate.
|
||||
3. Among overlapping candidates, the one whose overlap is largest relative to
|
||||
the source, then nearest to the source's centre line, wins.
|
||||
4. Ties are broken by top-left position, then NodeID — fully deterministic.
|
||||
|
||||
## Terminal resize architecture
|
||||
|
||||
`tea.WindowSizeMsg` only records the new size and marks layout dirty. The next
|
||||
`View()` recomputes panel geometry for the new size class (large / medium /
|
||||
small / too-small), reruns the treemap layout for the new treemap rectangle
|
||||
(never stretches old coordinates), and repaints. Because work happens only in
|
||||
`View()` and the frame cache absorbs repeated calls, a storm of resize events
|
||||
costs at most one layout per rendered frame. Modal geometry is derived from
|
||||
the current size every frame, clamped to the screen, and becomes scrollable
|
||||
when content does not fit.
|
||||
|
||||
## Selection persistence
|
||||
|
||||
Selection is a `NodeID` (plus, for the synthetic LOD group, the group's parent
|
||||
ID and a group marker). After any relayout the renderer looks the ID up in the
|
||||
new rectangle list. If it is absent (filtered out, collapsed into a group,
|
||||
zoom changed) selection falls back to: the group containing it, then its
|
||||
nearest visible ancestor, then the largest block.
|
||||
|
||||
## Modal architecture
|
||||
|
||||
A modal is a small interface: `Title()`, `Lines(width)`, `Footer()`, and
|
||||
`HandleKey`. A generic frame renders any modal centred, clamped to the screen
|
||||
(min 1-cell margin), with a scrollable viewport when content exceeds height.
|
||||
The base screen is dimmed before the modal is painted. While a modal is open
|
||||
it receives all keys; `Esc` always closes it.
|
||||
|
||||
## OS abstraction
|
||||
|
||||
`internal/platform` exposes `ReadDir(path, func(chunk []Entry) error)` and
|
||||
`Meta` structs. Implementations:
|
||||
|
||||
* `readdir_linux.go` — `getdents` via `os.File.ReadDir` + `statx` relative to
|
||||
the directory fd (one syscall per entry, gives birth time).
|
||||
* `readdir_unix.go` — darwin/BSD: `fstatat` relative to the directory fd.
|
||||
* `readdir_windows.go` — `os.File.ReadDir` + `Lstat`; reparse points
|
||||
(junctions, symlinks) are never descended by default.
|
||||
* `fs_*.go` — virtual-filesystem detection, owner name lookup.
|
||||
|
||||
No other package switches on `runtime.GOOS`.
|
||||
|
||||
## Security handling for hostile filenames
|
||||
|
||||
Filenames and snapshot contents are untrusted.
|
||||
|
||||
* Every string that reaches the terminal passes through
|
||||
`textutil.Sanitize`, which replaces C0/C1 control characters (including
|
||||
ESC, CSI, BEL, DEL) with visible escapes (`\x1b`), replaces invalid UTF-8
|
||||
with U+FFFD, and neutralises bidi override characters. The canvas also
|
||||
refuses control characters as a second line of defence. Internal names keep
|
||||
the raw bytes.
|
||||
* Text is clipped by display width (grapheme clusters, East-Asian wide
|
||||
characters) so labels never overflow their rectangle.
|
||||
* Snapshot decoding validates magic, version, every length, every parent
|
||||
reference (a parent must precede its child, so cycles and recursion are
|
||||
impossible), caps decompressed size and node count, and verifies a SHA-256
|
||||
trailer. Aggregates are recomputed rather than trusted.
|
||||
* No shell is ever invoked. "Open/reveal" and trash use direct `exec` with
|
||||
argument vectors or native APIs; paths are passed as single arguments.
|
||||
* `--read-only` disables every mutating action in the UI.
|
||||
@@ -0,0 +1,8 @@
|
||||
# Changelog
|
||||
|
||||
## Unreleased
|
||||
- Bounded concurrent scanner with incremental aggregation, cancellation,
|
||||
hard-link and symlink-loop handling, error accounting.
|
||||
- Non-interactive reports: `--top`, `--largest-files`, `--json`, `--csv`,
|
||||
`--duplicates`, `--no-ui`.
|
||||
- Versioned, checksummed snapshot format; `--compare`.
|
||||
@@ -0,0 +1,41 @@
|
||||
# Development
|
||||
|
||||
Requires Go (see `go.mod`). No CGO, no external tools.
|
||||
|
||||
```sh
|
||||
make build # bin/mapsize, bin/treegen
|
||||
make test # go test ./...
|
||||
make race # go test -race ./...
|
||||
make vet fmt-check
|
||||
make benchmark # go test -bench
|
||||
make fuzz # short fuzz runs of snapshot / treemap / filter
|
||||
make release # cross-compiled binaries in bin/release
|
||||
```
|
||||
|
||||
## Test data
|
||||
|
||||
`treegen` creates synthetic trees:
|
||||
|
||||
```sh
|
||||
bin/treegen --out /tmp/t --directories 50000 --files 1000000 --max-depth 20 --sparse --unicode
|
||||
bin/mapsize /tmp/t
|
||||
```
|
||||
|
||||
## Profiling
|
||||
|
||||
```sh
|
||||
bin/mapsize --no-ui --cpuprofile cpu.prof --memprofile mem.prof /path
|
||||
go tool pprof -http=: cpu.prof
|
||||
```
|
||||
|
||||
`--log FILE --log-level debug` writes a log without disturbing the TUI.
|
||||
Paths are logged only at `debug`/`trace` level.
|
||||
|
||||
## Layout
|
||||
|
||||
See ARCHITECTURE.md. Rules of thumb:
|
||||
|
||||
* `internal/treemap` and `internal/filter` must not import UI packages.
|
||||
* Only `internal/platform` may contain OS-specific code (build tags).
|
||||
* All strings derived from filenames go through `textutil.Sanitize` before
|
||||
reaching a terminal.
|
||||
@@ -0,0 +1,44 @@
|
||||
BIN := bin
|
||||
PKG := ./...
|
||||
VERSION ?= $(shell git describe --tags --always --dirty 2>/dev/null || echo dev)
|
||||
LDFLAGS := -s -w
|
||||
|
||||
.PHONY: build test race vet fmt fmt-check benchmark fuzz release clean treegen
|
||||
|
||||
build:
|
||||
go build -trimpath -ldflags '$(LDFLAGS)' -o $(BIN)/mapsize ./cmd/mapsize
|
||||
go build -trimpath -ldflags '$(LDFLAGS)' -o $(BIN)/treegen ./cmd/treegen
|
||||
|
||||
test:
|
||||
go test $(PKG)
|
||||
|
||||
race:
|
||||
go test -race $(PKG)
|
||||
|
||||
vet:
|
||||
go vet $(PKG)
|
||||
|
||||
fmt:
|
||||
gofmt -w .
|
||||
|
||||
fmt-check:
|
||||
@test -z "$$(gofmt -l .)" || (gofmt -l . && echo 'run make fmt' && exit 1)
|
||||
|
||||
benchmark:
|
||||
go test -run=NONE -bench=. -benchmem $(PKG)
|
||||
|
||||
fuzz:
|
||||
go test -run=NONE -fuzz=FuzzLoad -fuzztime=30s ./internal/snapshot
|
||||
go test -run=NONE -fuzz=FuzzSquarify -fuzztime=30s ./internal/treemap
|
||||
go test -run=NONE -fuzz=FuzzParse -fuzztime=30s ./internal/filter
|
||||
|
||||
release:
|
||||
@for t in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64 windows/amd64 windows/arm64; do \
|
||||
os=$${t%/*}; arch=$${t#*/}; ext=; [ $$os = windows ] && ext=.exe; \
|
||||
echo "$$os/$$arch"; \
|
||||
CGO_ENABLED=0 GOOS=$$os GOARCH=$$arch go build -trimpath -ldflags '$(LDFLAGS)' \
|
||||
-o $(BIN)/release/mapsize-$(VERSION)-$$os-$$arch$$ext ./cmd/mapsize || exit 1; \
|
||||
done
|
||||
|
||||
clean:
|
||||
rm -rf $(BIN)
|
||||
@@ -0,0 +1,86 @@
|
||||
# mapsize
|
||||
|
||||
An interactive, full-terminal disk usage analyser built around a live,
|
||||
navigable treemap. Think WinDirStat + ncdu + WizTree, in one Go binary that
|
||||
runs over SSH.
|
||||
|
||||
> `mapsize` is a working name; branding lives in `internal/brand`.
|
||||
|
||||
## Status
|
||||
|
||||
Under active development — see [ROADMAP.md](ROADMAP.md) for what works today.
|
||||
|
||||
## Install
|
||||
|
||||
```sh
|
||||
go install github.com/andydixon/mapsize/cmd/mapsize@latest
|
||||
# or
|
||||
make build # → bin/mapsize
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```sh
|
||||
mapsize . # scan and explore interactively
|
||||
mapsize --read-only /mnt/evidence
|
||||
mapsize /srv --save srv.msz # explore, and save a snapshot when done
|
||||
mapsize srv.msz # reopen a snapshot (no rescan)
|
||||
mapsize --compare old.msz new.msz
|
||||
|
||||
# non-interactive
|
||||
mapsize /srv --top 50
|
||||
mapsize /srv --largest-files 20
|
||||
mapsize /srv --json --depth 3
|
||||
mapsize /srv --csv > inventory.csv
|
||||
mapsize /srv --no-ui --save snapshot.msz
|
||||
mapsize /srv --duplicates --no-ui
|
||||
```
|
||||
|
||||
### Sizes and units
|
||||
|
||||
* By default sizes are **allocated** (disk usage, like `du`); `--apparent`
|
||||
switches to logical size. Both are shown in the info modal.
|
||||
* Units are **IEC** (KiB, MiB, GiB — powers of 1024) everywhere; `--si`
|
||||
switches everything to SI (kB, MB, GB — powers of 1000). Never mixed.
|
||||
* Hard links are counted once (first path seen); see ARCHITECTURE.md.
|
||||
|
||||
### Filesystem boundaries and exclusions
|
||||
|
||||
* Default: other filesystems *are* descended (like `du`), except Linux
|
||||
virtual filesystems (proc, sysfs, cgroup, …) which are never scanned.
|
||||
Mount points are marked.
|
||||
* `--one-file-system` / `-x`: stay on the root's filesystem.
|
||||
* `--follow-symlinks=none|same-filesystem|all` (default `none`). Loops are
|
||||
detected and nothing is counted twice.
|
||||
* `--exclude PATTERN` (repeatable): glob on entry name, or a path prefix if
|
||||
the pattern contains a separator. Exclusions are always shown in the UI.
|
||||
|
||||
### Configuration
|
||||
|
||||
Optional JSON file at `$XDG_CONFIG_HOME/mapsize/config.json`
|
||||
(macOS: `~/Library/Application Support/mapsize/config.json`, Windows:
|
||||
`%AppData%\mapsize\config.json`):
|
||||
|
||||
```json
|
||||
{
|
||||
"workers": "balanced",
|
||||
"theme": "default",
|
||||
"units": "iec",
|
||||
"excludes": ["/proc", "*.tmp"],
|
||||
"category_colors": {"Video": "#e06c75"},
|
||||
"extension_colors": {"qcow2": "#c678dd"}
|
||||
}
|
||||
```
|
||||
|
||||
### Read-only mode
|
||||
|
||||
`--read-only` disables every modifying action and shows `READ ONLY` in the
|
||||
status bar. It does not make mapsize a forensic acquisition tool: scanning
|
||||
uses ordinary filesystem APIs, and duplicate detection reads file content
|
||||
(which may update access times).
|
||||
|
||||
## Documentation
|
||||
|
||||
* [ARCHITECTURE.md](ARCHITECTURE.md) — design and invariants
|
||||
* [DEVELOPMENT.md](DEVELOPMENT.md) — building, testing, profiling
|
||||
* [PERFORMANCE.md](PERFORMANCE.md) — measured results
|
||||
+47
@@ -0,0 +1,47 @@
|
||||
# Roadmap
|
||||
|
||||
Status markers: `[x]` done, `[~]` partial, `[ ]` not started.
|
||||
|
||||
## 0.1 — inventory engine
|
||||
- [ ] Go project skeleton, Makefile, CI
|
||||
- [ ] portable filesystem scanner, bounded worker pool, no-deadlock scheduler
|
||||
- [ ] inventory node model, incremental aggregation
|
||||
- [ ] cancellation, progress counters, error accounting
|
||||
- [ ] CLI: `--top`, `--json`, `--csv`, `--no-ui`
|
||||
|
||||
## 0.2 — TUI shell
|
||||
- [ ] full-screen Bubble Tea v2 app, canvas renderer, frame cache
|
||||
- [ ] responsive layout (large / medium / small / too-small)
|
||||
- [ ] resize handling, status bar, progressive display, directory list
|
||||
|
||||
## 0.3 — treemap
|
||||
- [ ] squarified layout, edge-snapped cell rectangles
|
||||
- [ ] renderer with borders, nested preview, adaptive labels
|
||||
- [ ] level-of-detail grouping, category colouring
|
||||
|
||||
## 0.4 — interaction
|
||||
- [ ] spatial arrow navigation, stable NodeID selection
|
||||
- [ ] info modal (scrollable, resize-aware), zoom, breadcrumbs, mouse
|
||||
|
||||
## 0.5 — investigation
|
||||
- [ ] search & filter language (AND/OR/NOT, parentheses, units, ages)
|
||||
- [ ] extension statistics, largest files / dirs, error view
|
||||
|
||||
## 0.6 — snapshots
|
||||
- [ ] versioned binary snapshot format, save/load, comparison, growth treemap
|
||||
|
||||
## 0.7 — duplicates
|
||||
- [ ] staged duplicate detection (size → sampled hash → SHA-256), view
|
||||
|
||||
## 0.8 — operations
|
||||
- [ ] read-only mode, JSON/CSV export
|
||||
- [ ] move-to-trash (freedesktop, macOS ~/.Trash, Windows recycle bin)
|
||||
|
||||
## 0.9 — hardening (ongoing)
|
||||
- [ ] profiling & benchmarks (see PERFORMANCE.md)
|
||||
- [ ] memory reduction (name arena) if profiles justify it
|
||||
- [ ] Windows allocated-size via GetCompressedFileSizeW (opt-in)
|
||||
- [ ] configurable keybindings
|
||||
|
||||
## 1.0
|
||||
- [ ] stable snapshot format v1 guarantee, packaged releases
|
||||
@@ -0,0 +1,8 @@
|
||||
module github.com/andydixon/mapsize
|
||||
|
||||
go 1.26.0
|
||||
|
||||
require (
|
||||
github.com/rivo/uniseg v0.4.7
|
||||
golang.org/x/sys v0.48.0
|
||||
)
|
||||
@@ -0,0 +1,4 @@
|
||||
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
|
||||
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
|
||||
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
|
||||
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
|
||||
@@ -0,0 +1,9 @@
|
||||
// Package brand isolates the product name so it can be changed in one place.
|
||||
package brand
|
||||
|
||||
const (
|
||||
Name = "mapsize"
|
||||
SnapshotExt = ".msz"
|
||||
SnapshotMagic = "MAPSIZE\x00"
|
||||
Version = "0.9.0-dev"
|
||||
)
|
||||
@@ -0,0 +1,72 @@
|
||||
// Package config loads user settings from a JSON file in the OS-specific
|
||||
// user configuration directory. It knows nothing about rendering.
|
||||
package config
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
|
||||
"github.com/andydixon/mapsize/internal/brand"
|
||||
)
|
||||
|
||||
// Settings are user preferences. Zero values are never used directly; Load
|
||||
// starts from Defaults and overlays the file.
|
||||
type Settings struct {
|
||||
Workers string `json:"workers"` // conservative|balanced|aggressive|N
|
||||
FollowSymlinks string `json:"follow_symlinks"` // none|same-filesystem|all
|
||||
OneFileSystem bool `json:"one_file_system"`
|
||||
Units string `json:"units"` // iec|si
|
||||
SizeMode string `json:"size_mode"` // allocated|logical
|
||||
Theme string `json:"theme"`
|
||||
Mouse bool `json:"mouse"`
|
||||
Excludes []string `json:"excludes"`
|
||||
// ExtensionColors maps an extension (without dot) to "#rrggbb".
|
||||
ExtensionColors map[string]string `json:"extension_colors"`
|
||||
// CategoryColors maps a category name to "#rrggbb".
|
||||
CategoryColors map[string]string `json:"category_colors"`
|
||||
// RefreshHz is the UI refresh rate while scanning.
|
||||
RefreshHz int `json:"refresh_hz"`
|
||||
}
|
||||
|
||||
// Defaults returns the built-in settings.
|
||||
func Defaults() Settings {
|
||||
return Settings{
|
||||
Workers: "balanced", FollowSymlinks: "none", Units: "iec", SizeMode: "allocated",
|
||||
Theme: "default", Mouse: true, RefreshHz: 10,
|
||||
}
|
||||
}
|
||||
|
||||
// Path returns the config file location.
|
||||
func Path() (string, error) {
|
||||
dir, err := os.UserConfigDir()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return filepath.Join(dir, brand.Name, "config.json"), nil
|
||||
}
|
||||
|
||||
// Load reads the config file if present. A missing file is not an error.
|
||||
func Load() (Settings, error) {
|
||||
s := Defaults()
|
||||
p, err := Path()
|
||||
if err != nil {
|
||||
return s, nil
|
||||
}
|
||||
b, err := os.ReadFile(p)
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
return s, nil
|
||||
}
|
||||
if err != nil {
|
||||
return s, err
|
||||
}
|
||||
if err := json.Unmarshal(b, &s); err != nil {
|
||||
return Defaults(), err
|
||||
}
|
||||
if s.RefreshHz < 1 || s.RefreshHz > 60 {
|
||||
s.RefreshHz = 10
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
@@ -0,0 +1,125 @@
|
||||
// Package textutil holds display helpers: sanitising untrusted strings for
|
||||
// the terminal, width-aware truncation, and human-readable units.
|
||||
package textutil
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/rivo/uniseg"
|
||||
)
|
||||
|
||||
// Sanitize makes an untrusted string (a filename, snapshot data) safe to
|
||||
// write to a terminal. Control characters (C0, DEL, C1) and bidi overrides are
|
||||
// replaced by visible escapes; invalid UTF-8 becomes U+FFFD. The original
|
||||
// string should be kept for filesystem operations.
|
||||
func Sanitize(s string) string {
|
||||
clean := true
|
||||
for i := 0; i < len(s); i++ {
|
||||
if c := s[i]; c < 0x20 || c >= 0x7f {
|
||||
clean = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if clean {
|
||||
return s
|
||||
}
|
||||
var b strings.Builder
|
||||
b.Grow(len(s) + 8)
|
||||
for i := 0; i < len(s); {
|
||||
r, size := utf8.DecodeRuneInString(s[i:])
|
||||
if r == utf8.RuneError && size <= 1 {
|
||||
b.WriteRune(utf8.RuneError)
|
||||
i++
|
||||
continue
|
||||
}
|
||||
i += size
|
||||
switch {
|
||||
case r < 0x20 || r == 0x7f || (r >= 0x80 && r <= 0x9f):
|
||||
fmt.Fprintf(&b, `\x%02x`, r)
|
||||
case isBidiControl(r):
|
||||
fmt.Fprintf(&b, `\u%04x`, r)
|
||||
default:
|
||||
b.WriteRune(r)
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func isBidiControl(r rune) bool {
|
||||
return (r >= 0x202a && r <= 0x202e) || (r >= 0x2066 && r <= 0x2069) ||
|
||||
r == 0x200e || r == 0x200f || r == 0x061c
|
||||
}
|
||||
|
||||
// Width returns the display width of s in terminal cells.
|
||||
func Width(s string) int { return uniseg.StringWidth(s) }
|
||||
|
||||
// Truncate clips s to at most w cells, appending "…" if anything was cut.
|
||||
func Truncate(s string, w int) string {
|
||||
if w <= 0 {
|
||||
return ""
|
||||
}
|
||||
if Width(s) <= w {
|
||||
return s
|
||||
}
|
||||
return clip(s, w-1) + "…"
|
||||
}
|
||||
|
||||
// TruncateLeft clips from the left, prefixing "…" (used for paths and
|
||||
// breadcrumbs where the tail is the informative part).
|
||||
func TruncateLeft(s string, w int) string {
|
||||
if w <= 0 {
|
||||
return ""
|
||||
}
|
||||
total := Width(s)
|
||||
if total <= w {
|
||||
return s
|
||||
}
|
||||
skip := total - (w - 1)
|
||||
state := -1
|
||||
rest := s
|
||||
for rest != "" && skip > 0 {
|
||||
var cw int
|
||||
_, rest, cw, state = uniseg.FirstGraphemeClusterInString(rest, state)
|
||||
skip -= cw
|
||||
}
|
||||
// A wide cluster may have overshot by one cell; pad to keep width exact.
|
||||
return "…" + strings.Repeat(" ", -skip) + rest
|
||||
}
|
||||
|
||||
// clip returns the longest prefix of s that fits in w cells.
|
||||
func clip(s string, w int) string {
|
||||
state := -1
|
||||
used, end := 0, 0
|
||||
rest := s
|
||||
for rest != "" {
|
||||
var cluster string
|
||||
var cw int
|
||||
cluster, rest, cw, state = uniseg.FirstGraphemeClusterInString(rest, state)
|
||||
if used+cw > w {
|
||||
break
|
||||
}
|
||||
used += cw
|
||||
end += len(cluster)
|
||||
}
|
||||
return s[:end]
|
||||
}
|
||||
|
||||
// PadRight pads s with spaces to exactly w cells (truncating if longer).
|
||||
func PadRight(s string, w int) string {
|
||||
s = Truncate(s, w)
|
||||
if d := w - Width(s); d > 0 {
|
||||
s += strings.Repeat(" ", d)
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// PadLeft right-aligns s in w cells.
|
||||
func PadLeft(s string, w int) string {
|
||||
s = Truncate(s, w)
|
||||
if d := w - Width(s); d > 0 {
|
||||
s = strings.Repeat(" ", d) + s
|
||||
}
|
||||
return s
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
package textutil
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestSanitize(t *testing.T) {
|
||||
cases := map[string]string{
|
||||
"plain.txt": "plain.txt",
|
||||
"evil\x1b[2Jname": `evil\x1b[2Jname`,
|
||||
"bell\a": `bell\x07`,
|
||||
"c1\u009bx": `c1\x9bx`,
|
||||
"bad\xffutf8": "bad�utf8",
|
||||
"rtl\u202egnp.exe": `rtl\u202egnp.exe`,
|
||||
"日本語": "日本語",
|
||||
"tab\tnew\nline\rdel\x7f": `tab\x09new\x0aline\x0ddel\x7f`,
|
||||
}
|
||||
for in, want := range cases {
|
||||
if got := Sanitize(in); got != want {
|
||||
t.Errorf("Sanitize(%q) = %q, want %q", in, got, want)
|
||||
}
|
||||
if strings.ContainsAny(Sanitize(in), "\x1b\a\u009b") {
|
||||
t.Errorf("control byte survived in %q", in)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTruncate(t *testing.T) {
|
||||
if got := Truncate("hello world", 5); got != "hell…" {
|
||||
t.Errorf("got %q", got)
|
||||
}
|
||||
if got := Truncate("日本語テキスト", 5); Width(got) > 5 {
|
||||
t.Errorf("wide truncation overflow: %q width %d", got, Width(got))
|
||||
}
|
||||
if got := Truncate("abc", 3); got != "abc" {
|
||||
t.Errorf("got %q", got)
|
||||
}
|
||||
if got := Truncate("abc", 0); got != "" {
|
||||
t.Errorf("got %q", got)
|
||||
}
|
||||
if got := TruncateLeft("/a/very/long/path", 8); Width(got) != 8 || !strings.HasSuffix(got, "path") {
|
||||
t.Errorf("TruncateLeft got %q", got)
|
||||
}
|
||||
if got := TruncateLeft("ab日本語", 4); Width(got) != 4 {
|
||||
t.Errorf("TruncateLeft wide got %q (%d)", got, Width(got))
|
||||
}
|
||||
}
|
||||
|
||||
func TestSize(t *testing.T) {
|
||||
cases := map[int64]string{0: "0 B", 1023: "1023 B", 1024: "1.00 KiB", 1536: "1.50 KiB",
|
||||
181 << 30: "181 GiB", 1 << 62: "4.00 EiB"}
|
||||
for in, want := range cases {
|
||||
if got := Size(in); got != want {
|
||||
t.Errorf("Size(%d)=%q want %q", in, got, want)
|
||||
}
|
||||
}
|
||||
if got := Count(2184392); got != "2,184,392" {
|
||||
t.Errorf("Count got %q", got)
|
||||
}
|
||||
if got := SignedSize(-2048); got != "-2.00 KiB" {
|
||||
t.Errorf("SignedSize got %q", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,125 @@
|
||||
package textutil
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strconv"
|
||||
"time"
|
||||
)
|
||||
|
||||
// SI selects decimal units (kB, MB, …). The default is IEC (KiB, MiB, …).
|
||||
// It is set once at start-up, before any goroutines format sizes.
|
||||
var SI bool
|
||||
|
||||
var (
|
||||
iecUnits = []string{"B", "KiB", "MiB", "GiB", "TiB", "PiB", "EiB"}
|
||||
siUnits = []string{"B", "kB", "MB", "GB", "TB", "PB", "EB"}
|
||||
)
|
||||
|
||||
func scaled(n int64) (float64, string) {
|
||||
base, units := 1024.0, iecUnits
|
||||
if SI {
|
||||
base, units = 1000.0, siUnits
|
||||
}
|
||||
neg := n < 0
|
||||
v := float64(n)
|
||||
if neg {
|
||||
v = -v
|
||||
}
|
||||
i := 0
|
||||
for v >= base && i < len(units)-1 {
|
||||
v /= base
|
||||
i++
|
||||
}
|
||||
if neg {
|
||||
v = -v
|
||||
}
|
||||
return v, units[i]
|
||||
}
|
||||
|
||||
func num(v float64, unit string) string {
|
||||
a := v
|
||||
if a < 0 {
|
||||
a = -a
|
||||
}
|
||||
switch {
|
||||
case unit == "B":
|
||||
return strconv.FormatFloat(v, 'f', 0, 64)
|
||||
case a < 10:
|
||||
return strconv.FormatFloat(v, 'f', 2, 64)
|
||||
case a < 100:
|
||||
return strconv.FormatFloat(v, 'f', 1, 64)
|
||||
default:
|
||||
return strconv.FormatFloat(v, 'f', 0, 64)
|
||||
}
|
||||
}
|
||||
|
||||
// Size formats a byte count, e.g. "86.4 GiB".
|
||||
func Size(n int64) string {
|
||||
v, u := scaled(n)
|
||||
return num(v, u) + " " + u
|
||||
}
|
||||
|
||||
// SizeCompact formats a byte count in at most ~7 cells, e.g. "86GiB".
|
||||
func SizeCompact(n int64) string {
|
||||
v, u := scaled(n)
|
||||
a := v
|
||||
if a < 0 {
|
||||
a = -a
|
||||
}
|
||||
if u != "B" && a < 10 {
|
||||
return strconv.FormatFloat(v, 'f', 1, 64) + u
|
||||
}
|
||||
return strconv.FormatFloat(v, 'f', 0, 64) + u
|
||||
}
|
||||
|
||||
// SignedSize formats a delta with an explicit sign.
|
||||
func SignedSize(n int64) string {
|
||||
if n > 0 {
|
||||
return "+" + Size(n)
|
||||
}
|
||||
if n < 0 {
|
||||
return "-" + Size(-n)
|
||||
}
|
||||
return "±0 B"
|
||||
}
|
||||
|
||||
// Count formats an integer with thousands separators.
|
||||
func Count(n int64) string {
|
||||
s := strconv.FormatInt(n, 10)
|
||||
neg := false
|
||||
if n < 0 {
|
||||
neg, s = true, s[1:]
|
||||
}
|
||||
out := make([]byte, 0, len(s)+len(s)/3)
|
||||
for i := range len(s) {
|
||||
if i > 0 && (len(s)-i)%3 == 0 {
|
||||
out = append(out, ',')
|
||||
}
|
||||
out = append(out, s[i])
|
||||
}
|
||||
if neg {
|
||||
return "-" + string(out)
|
||||
}
|
||||
return string(out)
|
||||
}
|
||||
|
||||
// Duration formats an elapsed time as mm:ss or h:mm:ss.
|
||||
func Duration(d time.Duration) string {
|
||||
s := int64(d.Seconds())
|
||||
if s >= 3600 {
|
||||
return fmt.Sprintf("%d:%02d:%02d", s/3600, s/60%60, s%60)
|
||||
}
|
||||
return fmt.Sprintf("%02d:%02d", s/60, s%60)
|
||||
}
|
||||
|
||||
// Percent formats part/whole as a percentage.
|
||||
func Percent(part, whole int64) string {
|
||||
if whole <= 0 {
|
||||
return "–"
|
||||
}
|
||||
p := float64(part) * 100 / float64(whole)
|
||||
if p < 10 {
|
||||
return strconv.FormatFloat(p, 'f', 1, 64) + "%"
|
||||
}
|
||||
return strconv.FormatFloat(p, 'f', 0, 64) + "%"
|
||||
}
|
||||
Reference in New Issue
Block a user