Docs for the Rust version; benchmarks; build.sh fixes
README, CLAUDE.md, man pages, the user guide and its screenshot script now describe the Rust build and the intentional behaviour changes, which docs/MIGRATION.md lists. Go and Rust micro-benchmarks for the same operations, and scripts/bench.sh for end-to-end comparison. build.sh exports its tree root to the per-target jobs and keeps logs out of the output directory; riscv64 musl builds use getrandom's raw syscall backend, since zig's musl getrandom never reaches the kernel there. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,95 @@
|
||||
# Migrating from the Go implementation
|
||||
|
||||
vault 0.6 is a rewrite in Rust of the Go vault (0.5 and earlier). It is an
|
||||
implementation migration only: key files, vaults, the lock and the unlock
|
||||
agent are the same formats, byte for byte where the format is
|
||||
deterministic (see `COMPATIBILITY.md`). There is nothing to convert. Go and
|
||||
Rust builds can be used on the same vault, even at the same time: they
|
||||
honour each other's locks, and either can use an agent the other started.
|
||||
|
||||
To go back, install 0.5: everything 0.6 writes, 0.5 reads.
|
||||
|
||||
## Intentional behaviour changes
|
||||
|
||||
Safety and correctness:
|
||||
|
||||
- **`get` never replaces anything.** Output is published with an atomic
|
||||
no-replace rename or link, so a file, a symlink (even a dangling one) or a
|
||||
file created while `get` runs is left alone. Go checked first and then
|
||||
renamed, which a concurrent creator could race.
|
||||
- **`get -r` does not follow symlinks inside the output directory.** Each
|
||||
directory below the chosen one is opened without following links; a link
|
||||
or non-directory in the way stops the run. (Windows: checked just before
|
||||
use, see `SECURITY.md`.)
|
||||
- **Extracted files are synced** to disk before they appear under their
|
||||
name, and their directory afterwards.
|
||||
- **A source file that changes while being added is not stored.** If it
|
||||
grows, shrinks, is modified or replaced during `add`/`migrate`, the
|
||||
command fails and nothing is stored. Go stored whatever it read (for a
|
||||
file that grew, a truncated copy).
|
||||
- **`migrate` deletes an original only if it is still the file stored**
|
||||
(same inode, size and time), after the record is durable.
|
||||
- **Deletes are durable in order.** Removing a file syncs the removal of its
|
||||
record before deleting its objects, so a crash cannot leave a record
|
||||
pointing at deleted objects. A failed sync after a record is already in
|
||||
place no longer deletes its objects.
|
||||
- **Lock errors are reported.** Go let a reader continue without the lock on
|
||||
any error; Rust does so only on a read-only filesystem.
|
||||
- **Stricter records.** A record must have every field, with exactly the
|
||||
field names Go writes, and nothing after the JSON. Go also accepted
|
||||
differently-cased names, missing fields (as zero) and trailing bytes;
|
||||
Go never writes such records, so no existing vault is affected.
|
||||
- **Key files over 64 KiB** are rejected without reading them whole.
|
||||
|
||||
Agent (`unlock`):
|
||||
|
||||
- The agent **refuses the key if it cannot protect it**: locking its memory,
|
||||
disabling core dumps and (Linux) becoming undumpable must all succeed. Go
|
||||
ignored failures there.
|
||||
- Its environment is stripped of `VAULT_PASSPHRASE` and
|
||||
`VAULT_DECOY_PASSPHRASE`; it runs in `/` with standard error closed.
|
||||
- Expiry no longer waits for the request in progress beyond its deadline,
|
||||
and a key is never handed out after expiry.
|
||||
- `unlock` waits at most ten seconds for the agent to start and kills it on
|
||||
failure; the agent removes its socket only if it is still its own.
|
||||
- **Windows: `unlock` is not available** (Go's agent there had no memory
|
||||
protection). Every command asks for the passphrase.
|
||||
|
||||
Network (`add URL`, `mirror`):
|
||||
|
||||
- Explicit limits, all adjustable: connect 30 s, response headers 60 s,
|
||||
whole request 2 h (`add`) or 5 min (`mirror`, per request), download size
|
||||
16 GiB (`add`) or 4 GiB in total (`mirror`), 10 redirects, 10 000 mirror
|
||||
pages, 16 MiB per HTML or CSS page. Bytes are counted after decompression.
|
||||
New flags `--max-bytes`, `--timeout`, `--max-pages`. Go had no limits.
|
||||
- Redirects go to http or https only; a `mirror` redirect to another host is
|
||||
skipped (Go followed it).
|
||||
- `mirror` resolves links against the final address after redirects and the
|
||||
page's `<base href>`; it parses HTML instead of matching a pattern, so it
|
||||
rewrites only `href`, `src`, `action` and `poster` on real tags, plus
|
||||
`url(...)` in CSS, `<style>` and `style=`, keeping every other byte and
|
||||
the original quoting. Text in scripts and comments is no longer touched.
|
||||
- `mirror` never lets two different URLs in one crawl overwrite each other's
|
||||
file; the second is skipped. It exits with status **3** when it stored
|
||||
some resources but skipped others (Go always exited 0).
|
||||
- Proxies follow `HTTPS_PROXY`/`HTTP_PROXY`/`NO_PROXY` as in Go; local
|
||||
addresses are never proxied.
|
||||
|
||||
Command line:
|
||||
|
||||
- New option `--workers N` (1–256) on every command.
|
||||
- `--key`/`--vault` no longer need a home directory when given explicitly.
|
||||
- Command-line arguments must be valid UTF-8 (Go accepted any bytes, but
|
||||
vault names had to be UTF-8 already).
|
||||
- `ls`/`list` piped into a reader that stops early end quietly with
|
||||
status 141, as Go's did by `SIGPIPE`.
|
||||
|
||||
Platforms: see `PLATFORMS.md`. Some targets Go built have no stable Rust
|
||||
equivalent; they are listed there and need a decision before release.
|
||||
|
||||
## Unchanged on purpose
|
||||
|
||||
Formats, limits (1024-byte names, 1 TiB files, 4 MiB objects, 4096-byte
|
||||
records, 16 MiB headers, 1 MiB stanzas), Argon2id parameters, the hybrid
|
||||
suite, filler, decoy behaviour, messages and exit statuses (apart from the
|
||||
above), the shape of `ls` and `list` output, and the agent protocol.
|
||||
+6
-3
@@ -7,7 +7,9 @@ sandbox path and file owner are shown as /home/sam and "sam sam", key lines
|
||||
are highlighted in amber for the reader, and "moment" frames show a prompt
|
||||
with a cursor, exactly as vault prints it before you type.
|
||||
|
||||
Needs: go, python3, and Chrome or Chromium (set CHROME=/path if not found).
|
||||
Needs: cargo (with the pinned toolchain), cargo-zigbuild and zig for the
|
||||
build screenshot, python3, and Chrome or Chromium (set CHROME=/path if not
|
||||
found).
|
||||
Output: docs/guide/img/*.png
|
||||
"""
|
||||
|
||||
@@ -75,7 +77,8 @@ class Sandbox:
|
||||
self.bin = os.path.join(self.root, "bin")
|
||||
os.makedirs(self.home)
|
||||
os.makedirs(self.bin)
|
||||
subprocess.run(["go", "build", "-o", os.path.join(self.bin, "vault"), "."], cwd=REPO, check=True)
|
||||
subprocess.run(["cargo", "build", "--release", "--locked", "--quiet"], cwd=REPO, check=True)
|
||||
shutil.copy2(os.path.join(REPO, "target", "release", "vault"), os.path.join(self.bin, "vault"))
|
||||
|
||||
def env(self, extra=None):
|
||||
e = {
|
||||
@@ -406,7 +409,7 @@ def main():
|
||||
|
||||
# building, into a temporary OUT so an existing dist/ is left alone
|
||||
rel = tempfile.mkdtemp(prefix="vault-release-")
|
||||
targets = "linux/amd64 linux/arm darwin/arm64 windows/amd64"
|
||||
targets = "linux/amd64 linux/arm64 linux/armv7 windows/amd64"
|
||||
t = Term(box, title="sam@laptop: ~/src/vault")
|
||||
out = subprocess.run(["./build.sh"], cwd=REPO, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
|
||||
text=True, env=dict(os.environ, TARGETS=targets, OUT=rel)).stdout
|
||||
|
||||
+36
-17
@@ -222,14 +222,17 @@ Linux on ARM64 & Pi 4/5 (64-bit OS), Graviton, Ampere & vault-linux-arm64 \\
|
||||
Raspberry Pi, 32-bit & Pi 2, 3, 4 on a 32-bit OS & vault-linux-armv7 \\
|
||||
Raspberry Pi Zero / 1 & the tiny older ones & vault-linux-armv6 \\
|
||||
Android & in Termux & vault-android-arm64 \\
|
||||
FreeBSD, OpenBSD, NetBSD & amd64, arm64, 386, arm\ldots & vault-freebsd-amd64 etc. \\
|
||||
Something exotic & RISC-V, POWER, s390x, MIPS, LoongArch & see \cmd{dist/} \\
|
||||
FreeBSD, NetBSD & amd64 (FreeBSD also 386) & vault-freebsd-amd64 etc. \\
|
||||
Something exotic & RISC-V, POWER, s390x, LoongArch, illumos, Solaris & see \cmd{dist/} \\
|
||||
\end{vtable}
|
||||
|
||||
\begin{squirrel}
|
||||
The full release covers 43 platform builds, including AIX, Solaris, illumos,
|
||||
Plan~9 and even WebAssembly. Will you ever need \file{vault-plan9-arm}?
|
||||
Almost certainly not. Is it pleasing that it exists? Extremely.
|
||||
The release covers some twenty platforms, from Windows on ARM to IBM
|
||||
mainframes, and even WebAssembly. A few that the older Go builds reached,
|
||||
such as Plan~9, OpenBSD and MIPS, have no stable Rust toolchain yet;
|
||||
\file{docs/PLATFORMS.md} keeps the list. Will you ever need
|
||||
\file{vault-linux-s390x}? Almost certainly not. Is it pleasing that it
|
||||
exists? Extremely.
|
||||
\end{squirrel}
|
||||
|
||||
\begin{headsup}
|
||||
@@ -299,24 +302,29 @@ Run \cmd{vault help}. You should see the full command list:
|
||||
|
||||
\section{Building it yourself}
|
||||
|
||||
Prefer to build from source? You need Go 1.26.6 or newer. Then:
|
||||
Prefer to build from source? You need Rust, through \cmd{rustup}, which
|
||||
fetches the exact compiler version the project pins. Then:
|
||||
|
||||
\begin{shell}
|
||||
$ go build . # just your platform, produces ./vault
|
||||
$ ./build.sh # every platform, into ./dist, with SHA256SUMS
|
||||
$ cargo build --release --locked # just your platform: target/release/vault
|
||||
$ ./build.sh # every platform it can, into ./dist, with SHA256SUMS
|
||||
\end{shell}
|
||||
|
||||
\cmd{build.sh} cross-compiles every target Go supports without C libraries.
|
||||
\cmd{build.sh} cross-compiles Linux and Windows with \cmd{cargo-zigbuild}
|
||||
and \cmd{zig}, macOS on a Mac, and the BSDs, illumos and Solaris with
|
||||
\cmd{cross} and Docker; it says which targets your machine can't build.
|
||||
32-bit ARM Linux gets two builds, armv6 and armv7. Want just a few? Set
|
||||
\env{TARGETS}. Want them somewhere else? Set \env{OUT}.
|
||||
|
||||
\shot{build}{A partial build, followed by checking every binary against SHA256SUMS}
|
||||
|
||||
\begin{whyso}
|
||||
The binaries are \emph{static} and \emph{reproducible}: no C code, no system
|
||||
libraries, file paths stripped, build IDs zeroed. Two people building the same
|
||||
commit get byte-identical files, so you can check a release against your own
|
||||
build.
|
||||
The Linux binaries are \emph{static}: they need no system libraries at all.
|
||||
Every build uses the pinned compiler and the locked versions of every
|
||||
dependency, with build paths stripped, so the result doesn't depend on whose
|
||||
machine or which folder built it. Built with the same compiler for the same
|
||||
target, the same commit should give the same file, so you can check a
|
||||
release against your own build.
|
||||
\end{whyso}
|
||||
|
||||
\begin{pocket}
|
||||
@@ -324,7 +332,7 @@ build.
|
||||
\item One file per platform. Rename it \cmd{vault} and put it on your \env{PATH}.
|
||||
\item Always check the SHA-256 against \file{SHA256SUMS}.
|
||||
\item macOS: clear the quarantine flag after checking.
|
||||
\item \cmd{./build.sh} builds all 43 platform binaries from source.
|
||||
\item \cmd{./build.sh} builds every platform your machine can, from source.
|
||||
\end{pocketlist}
|
||||
\end{pocket}
|
||||
|
||||
@@ -1071,8 +1079,9 @@ who's slow at typing never blocks you.
|
||||
\begin{whyso}
|
||||
\textbf{What if vault crashes while holding the lock?} On Linux, macOS, the
|
||||
BSDs and Windows, the operating system drops the lock the moment the process
|
||||
dies. Nothing to clean up. On Plan~9 and WebAssembly, which have no file
|
||||
locks, vault uses a \file{.lock} file instead. If one's ever left behind,
|
||||
dies. Nothing to clean up. Older Go builds of vault use the very same locks,
|
||||
so mixing versions is safe. On WebAssembly, which has no file locks, vault
|
||||
uses a \file{.lock} file instead. If one's ever left behind,
|
||||
vault tells you after a minute exactly which file to delete.
|
||||
\end{whyso}
|
||||
|
||||
@@ -1326,7 +1335,17 @@ could not read passphrase (set VAULT\_PASSPHRASE\ldots) & No terminal to ask on,
|
||||
agent did not start: \ldots & \cmd{unlock} couldn't start its helper. The reason follows; commands still work, asking for the passphrase. \\
|
||||
nothing was unlocked & \cmd{vault lock} found nothing to lock. Not an error. \\
|
||||
waiting for another vault process to finish & Not an error. Another command is writing; this one continues when it's done. \\
|
||||
vault still locked after a minute\ldots & Plan~9 and WebAssembly only. If nothing else is running, delete the file it names. \\
|
||||
vault still locked after a minute\ldots & WebAssembly only. If nothing else is running, delete the file it names. \\
|
||||
cannot lock the vault: \ldots & The lock file couldn't be opened or locked; the reason follows. Check the vault folder's permissions. \\
|
||||
\ldots\ changed while being added & The file was being written while vault read it. Nothing was stored; try again when it's finished. \\
|
||||
\ldots\ changed after it was stored; not deleting it & \cmd{migrate} stored the file, but it changed before it could be deleted. The original is kept; the vault has the version it read. \\
|
||||
\ldots\ already exists & \cmd{get} never overwrites, not even a symlink. Pick another name with \cmd{-o}. \\
|
||||
\ldots\ refusing to extract through it & \cmd{get -r} found a symlink (or a file) where a folder should be inside the output folder, and stopped rather than follow it. \\
|
||||
cannot lock agent memory \ldots, cannot make the agent undumpable \ldots & \cmd{unlock}'s helper couldn't protect the key in memory, so it refused it. Commands still work, asking for the passphrase. \\
|
||||
vault unlock is not supported on this platform\ldots & Windows: every command asks for the passphrase. \\
|
||||
download exceeds \ldots\ bytes & A download passed its size limit. Raise it with \cmd{--max-bytes} if you meant it. \\
|
||||
redirected to another host: \ldots & \cmd{mirror} only follows redirects within the site it is copying. \\
|
||||
mirror incomplete: stored \ldots, skipped \ldots & Some pages failed; the reasons were printed as they happened. Exit status 3. \\
|
||||
skipping unreadable record \ldots & A catalogue entry is damaged. Everything else still works; restore \file{records/} from backup to recover that file. \\
|
||||
object does not match its catalogue record & The encrypted contents were swapped or damaged. Nothing was written. Restore \file{objects/} from backup. \\
|
||||
object size does not match its catalogue record & Same as above: damage or tampering, caught before any output appeared. \\
|
||||
|
||||
+21
-2
@@ -1,4 +1,4 @@
|
||||
.TH VAULT-ADD 1 "September 2026" "vault" "User Commands"
|
||||
.TH VAULT-ADD 1 "October 2026" "vault" "User Commands"
|
||||
.SH NAME
|
||||
vault-add \- encrypt files, directories or URLs into the vault
|
||||
.SH SYNOPSIS
|
||||
@@ -7,6 +7,10 @@ vault-add \- encrypt files, directories or URLs into the vault
|
||||
.IR PATH ]
|
||||
.RB [ \-\-vault
|
||||
.IR DIR ]
|
||||
.RB [ \-\-max\-bytes
|
||||
.IR SIZE ]
|
||||
.RB [ \-\-timeout
|
||||
.IR DURATION ]
|
||||
.IR FILE | DIR | URL
|
||||
.RI [ FOLDER ]
|
||||
.SH DESCRIPTION
|
||||
@@ -17,7 +21,14 @@ and the key are skipped. A stored name that already exists is replaced.
|
||||
.PP
|
||||
An \fBhttp://\fR or \fBhttps://\fR \fIURL\fR is downloaded with a Chrome user
|
||||
agent and stored under the last element of its final (post-redirect) path, or
|
||||
the host name if there is none. Only a 200 response is accepted.
|
||||
the host name if there is none. Only a 200 response is accepted. Certificates
|
||||
are always verified; at most 10 redirects are followed, to http or https
|
||||
only. The download is staged in a private temporary file (it must be
|
||||
complete before it can be stored), which is removed afterwards whatever
|
||||
happens; it is plaintext on disk meanwhile.
|
||||
.PP
|
||||
A file that changes while it is being added (it grows, shrinks, or is
|
||||
replaced) is not stored, and the command fails.
|
||||
.PP
|
||||
\fIFOLDER\fR places the result inside a folder in the vault. File contents
|
||||
are cut into 4 MiB chunks, the last bucket-padded.
|
||||
@@ -28,6 +39,14 @@ Key file. Default: $VAULT_KEY, else ~/.vault/key.
|
||||
.TP
|
||||
.BI \-\-vault " DIR"
|
||||
Vault directory. Default: $VAULT_DIR, else ~/.vault/store.
|
||||
.TP
|
||||
.BI \-\-max\-bytes " SIZE"
|
||||
Largest download accepted, counted after decompression, such as 500M or 50G.
|
||||
Default 16G; at most 1T.
|
||||
.TP
|
||||
.BI \-\-timeout " DURATION"
|
||||
Deadline for the whole download, such as 30m. Default 2h. Connecting must
|
||||
take under 30 seconds and the response headers must arrive within a minute.
|
||||
.SH ENVIRONMENT
|
||||
.TP
|
||||
.B VAULT_PASSPHRASE
|
||||
|
||||
+9
-4
@@ -1,4 +1,4 @@
|
||||
.TH VAULT-GET 1 "September 2026" "vault" "User Commands"
|
||||
.TH VAULT-GET 1 "October 2026" "vault" "User Commands"
|
||||
.SH NAME
|
||||
vault-get \- decrypt a stored file
|
||||
.SH SYNOPSIS
|
||||
@@ -13,13 +13,18 @@ vault-get \- decrypt a stored file
|
||||
.I NAME
|
||||
.SH DESCRIPTION
|
||||
Decrypts \fINAME\fR. Nothing appears at the output path until every object has
|
||||
authenticated. The stored mode and modification time are restored. It refuses
|
||||
to overwrite an existing file. Takes a shared lock.
|
||||
authenticated, padding and end of file included; until then the output is a
|
||||
private temporary file in the same directory, removed on failure. The stored
|
||||
mode and modification time are restored. It never replaces anything already
|
||||
at the output path, including a symbolic link (even a dangling one) or a
|
||||
file created while it runs. Takes a shared lock.
|
||||
.PP
|
||||
With \fB\-r\fR, \fINAME\fR is a folder, and every file under it is
|
||||
decrypted into the output directory, keeping the folder structure. Each file
|
||||
is checked as above; a clash stops the run, leaving the files already
|
||||
written.
|
||||
written. Directories are created as needed, mode 0700; a symbolic link (or
|
||||
anything but a directory) where a directory belongs inside the output
|
||||
directory stops the run rather than being followed.
|
||||
.SH OPTIONS
|
||||
.TP
|
||||
.BI \-o " OUTPUT"
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
.TH VAULT-MIGRATE 1 "September 2026" "vault" "User Commands"
|
||||
.TH VAULT-MIGRATE 1 "October 2026" "vault" "User Commands"
|
||||
.SH NAME
|
||||
vault-migrate \- move files into the vault, deleting the originals
|
||||
.SH SYNOPSIS
|
||||
@@ -13,7 +13,10 @@ vault-migrate \- move files into the vault, deleting the originals
|
||||
.SH DESCRIPTION
|
||||
Works like
|
||||
.BR vault\-add (1)
|
||||
but deletes each original once it is stored, then any directories left empty.
|
||||
but deletes each original once it is stored (and its record is on disk),
|
||||
then any directories left empty. An original is deleted only if it is still
|
||||
the same file, unchanged, that was stored; otherwise it is kept and the
|
||||
command fails.
|
||||
Directories still holding skipped files stay. A URL cannot be migrated.
|
||||
.PP
|
||||
Before storing anything it lists the vault names that would be overwritten and
|
||||
|
||||
+60
-17
@@ -1,4 +1,4 @@
|
||||
.TH VAULT-MIRROR 1 "September 2026" "vault" "User Commands"
|
||||
.TH VAULT-MIRROR 1 "October 2026" "vault" "User Commands"
|
||||
.SH NAME
|
||||
vault-mirror \- mirror a website into an encrypted vault
|
||||
.SH SYNOPSIS
|
||||
@@ -7,6 +7,12 @@ vault-mirror \- mirror a website into an encrypted vault
|
||||
.IR PATH ]
|
||||
.RB [ \-\-vault
|
||||
.IR DIR ]
|
||||
.RB [ \-\-max\-pages
|
||||
.IR N ]
|
||||
.RB [ \-\-max\-bytes
|
||||
.IR SIZE ]
|
||||
.RB [ \-\-timeout
|
||||
.IR DURATION ]
|
||||
.I URL
|
||||
.RI [ FOLDER ]
|
||||
.SH DESCRIPTION
|
||||
@@ -20,21 +26,30 @@ and stores every page and asset it finds in the vault, beneath
|
||||
if no folder is given).
|
||||
.PP
|
||||
The crawl is single threaded: one request at a time, in breadth-first order.
|
||||
It follows only links on the same host as
|
||||
It follows only links on the same host (and port) as
|
||||
.IR URL ,
|
||||
and requests are sent with a Chrome user agent string.
|
||||
and only redirects to that host; a redirect elsewhere skips the page.
|
||||
Requests are sent with a Chrome user agent string, and certificates are
|
||||
always verified.
|
||||
.PP
|
||||
Links in HTML
|
||||
Links in HTML start tags
|
||||
.RB ( href ,
|
||||
.BR src ,
|
||||
.BR action
|
||||
and
|
||||
.B poster
|
||||
attributes) and in CSS
|
||||
.RB ( url(...) )
|
||||
that point into the mirror are rewritten to relative paths, so the copy
|
||||
browses correctly once fetched out of the vault. Links to other hosts, and
|
||||
fragments, are left as they were.
|
||||
attributes) and CSS
|
||||
.RB ( url(...)
|
||||
in style sheets,
|
||||
.B <style>
|
||||
elements and
|
||||
.B style
|
||||
attributes) that point into the mirror are rewritten to relative paths, so
|
||||
the copy browses correctly once fetched out of the vault. Relative links are
|
||||
resolved against the page's final address and its
|
||||
.B <base href>
|
||||
if it has one. Links to other hosts, script and comment text, and every
|
||||
other byte of the page are left as they were. Fragments are kept.
|
||||
.PP
|
||||
Pages are named after their URL path. A path that is empty or ends in
|
||||
.B /
|
||||
@@ -42,19 +57,39 @@ becomes
|
||||
.BR index.html ;
|
||||
a query string is appended as
|
||||
.BI _ query
|
||||
(URL-escaped). Anything already stored under the same name is replaced.
|
||||
(URL-escaped). A name already stored by an earlier run is replaced, but if
|
||||
two different URLs in one crawl would get the same name, the second is
|
||||
skipped rather than overwriting the first.
|
||||
.PP
|
||||
A page that fails to download, or answers with anything but
|
||||
A page that fails to download, answers with anything but
|
||||
.BR "200 OK" ,
|
||||
is reported on standard error and skipped; the crawl carries on.
|
||||
The vault is locked exclusively for the whole run.
|
||||
or exceeds a limit is reported on standard error as
|
||||
.BI "skipping " URL ": " reason
|
||||
and skipped; the crawl carries on. HTML and CSS are rewritten in memory;
|
||||
other files are staged in a private temporary file (plaintext on disk
|
||||
meanwhile) that is always removed. The vault is locked exclusively for the
|
||||
whole run.
|
||||
.SH OPTIONS
|
||||
.TP
|
||||
.BI \-\-max\-pages " N"
|
||||
Pages and assets to fetch at most. Default 10000.
|
||||
.TP
|
||||
.BI \-\-max\-bytes " SIZE"
|
||||
Total bytes to download at most, counted after decompression. Default 4G.
|
||||
.TP
|
||||
.BI \-\-timeout " DURATION"
|
||||
Deadline for each request. Default 5m.
|
||||
.TP
|
||||
.BI \-\-key " PATH"
|
||||
Key file. Default: $VAULT_KEY, else ~/.vault/key.
|
||||
.TP
|
||||
.BI \-\-vault " DIR"
|
||||
Vault directory. Default: $VAULT_DIR, else ~/.vault/store.
|
||||
.PP
|
||||
A single HTML or CSS file larger than 16 MiB is skipped.
|
||||
.SH EXIT STATUS
|
||||
0 when everything found was stored, 3 when some of it was skipped, 1 on
|
||||
an error that stops the crawl.
|
||||
.SH ENVIRONMENT
|
||||
.TP
|
||||
.B VAULT_PASSPHRASE
|
||||
@@ -75,11 +110,19 @@ vault mirror https://example.com/ archive/2026
|
||||
vault get archive/2026/example.com/index.html
|
||||
.RE
|
||||
.SH LIMITATIONS
|
||||
Links are found by pattern matching, not by parsing HTML: URLs assembled by
|
||||
scripts, and
|
||||
URLs assembled by scripts, and
|
||||
.B srcset
|
||||
lists, are neither followed nor rewritten. There is no depth limit, rate
|
||||
limit or robots.txt handling; only mirror sites you may copy.
|
||||
lists, are neither followed nor rewritten. In attribute values only numeric
|
||||
character references and
|
||||
.BR & ,
|
||||
.BR < ,
|
||||
.BR > ,
|
||||
.BR " ,
|
||||
.B '
|
||||
and
|
||||
.B
|
||||
are decoded. There is no rate limit or robots.txt handling; only mirror
|
||||
sites you may copy.
|
||||
.SH SEE ALSO
|
||||
.BR vault (1)
|
||||
.SH AUTHOR
|
||||
|
||||
@@ -32,6 +32,14 @@ memory is locked against swapping and excluded from core dumps; on Linux the
|
||||
helper is also not debuggable. It answers only on a Unix socket inside
|
||||
\fI$XDG_RUNTIME_DIR/vault-UID\fR (or the temporary directory), a directory
|
||||
that must belong to this user with mode 0700. Rebooting ends it too.
|
||||
If the helper cannot lock its memory, disable core dumps or (on Linux) make
|
||||
itself undebuggable, it refuses the key and
|
||||
.B unlock
|
||||
fails, saying why. The helper is the same program, started with the
|
||||
passphrase variables removed from its environment. Helpers started by
|
||||
earlier (Go) versions of vault are compatible and can be used and locked.
|
||||
.PP
|
||||
Not available on Windows, where every command asks for the passphrase.
|
||||
.PP
|
||||
While the key is unlocked, any program running as this user can use it, and
|
||||
it is whichever key the passphrase opened, real or decoy. Lock before
|
||||
|
||||
+23
-2
@@ -1,4 +1,4 @@
|
||||
.TH VAULT 1 "September 2026" "vault" "User Commands"
|
||||
.TH VAULT 1 "October 2026" "vault" "User Commands"
|
||||
.SH NAME
|
||||
vault \- post-quantum encrypted file vault
|
||||
.SH SYNOPSIS
|
||||
@@ -77,6 +77,11 @@ Key file. Default: $VAULT_KEY, else ~/.vault/key.
|
||||
.TP
|
||||
.BI \-\-vault " DIR"
|
||||
Vault directory. Default: $VAULT_DIR, else ~/.vault/store.
|
||||
.TP
|
||||
.BI \-\-workers " N"
|
||||
Threads for encryption, decryption and file I/O, 1 to 256. Default: the
|
||||
number of CPUs, at most 8. Up to eight 4 MiB objects are in flight at once
|
||||
whatever the count.
|
||||
.SH ENVIRONMENT
|
||||
.TP
|
||||
.B VAULT_KEY
|
||||
@@ -92,6 +97,19 @@ Passphrase, instead of prompting on the terminal or using a key remembered by
|
||||
.B VAULT_DECOY_PASSPHRASE
|
||||
Decoy passphrase used by
|
||||
.BR "vault genkey" .
|
||||
.TP
|
||||
.BR HTTPS_PROXY ", " HTTP_PROXY ", " NO_PROXY
|
||||
Proxies for
|
||||
.B add
|
||||
.I URL
|
||||
and
|
||||
.BR mirror ,
|
||||
as other programs use them; local addresses are never proxied.
|
||||
.TP
|
||||
.BR XDG_RUNTIME_DIR ", " TMPDIR
|
||||
Where
|
||||
.BR vault\-unlock (1)
|
||||
keeps its socket, and where downloads are staged.
|
||||
.SH LOCKING
|
||||
Commands that change the vault hold an exclusive lock on
|
||||
.IR DIR/.lock ;
|
||||
@@ -106,7 +124,10 @@ and
|
||||
fills each vault with unreadable filler, so hidden files look like filler to
|
||||
anyone holding only the decoy passphrase.
|
||||
.SH EXIT STATUS
|
||||
0 on success; 1 on error (diagnostics on standard error); 2 when run with no command.
|
||||
0 on success; 1 on error (diagnostics on standard error); 2 when run with no
|
||||
command; 3 when
|
||||
.BR vault\-mirror (1)
|
||||
stored some resources but had to skip others.
|
||||
.SH EXAMPLES
|
||||
.nf
|
||||
vault genkey
|
||||
|
||||
Reference in New Issue
Block a user