Chunk objects, shard the store, and make filler configurable

File bodies are now cut into 4 MiB objects, the last padded to its
power-of-two bucket, so every object is 64 KiB to 4 MiB whatever the file
size. Object IDs are HMAC'd from a secret per-record seed, so nothing on
disk links a file's pieces, and each object still leads with its own ID,
binding it to its record and position. Chunks are encrypted and synced by
a small worker pool so their fsyncs overlap.

records/ and objects/ are sharded into 256 subdirectories by ID prefix,
all created at init, with every shard's mtime pinned to the epoch.

vault init --filler SIZE writes at least SIZE of filler. Filler entries
are now real entries for a throwaway key, of log-uniform random sizes, so
they carry multi-chunk files like a real collection.

The marker becomes VAULT 2; earlier vaults are not supported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CLAUDE.md

@@ -12,7 +12,9 @@ root that builds a single `vault` binary. Module `github.com/andydixon/vault`.
   XChaCha20-Poly1305 sealing a suite ID and 32-byte master seed). The second
   slot is either a decoy identity or random bytes; slot order is random.
 - `store.go` - on-disk vault: `records/` holds fixed-size encrypted catalogue
-  records, `objects/` holds padded encrypted file bodies. Records a key cannot
+  records, `objects/` holds file bodies cut into 4 MiB chunks (last one
+  bucket-padded) whose IDs are HMAC'd from a per-record seed. Both are
+  sharded into 256 `xx/` subdirectories by ID prefix. Records a key cannot
   open are skipped, not treated as errors.
 - `lock*.go` - vault lock on `DIR/.lock`: fcntl on unix, LockFileEx on
   Windows, an exclusive lock file elsewhere. Writers lock exclusively,
@@ -46,10 +48,12 @@ export VAULT_PASSPHRASE=test VAULT_KEY=$PWD/k VAULT_DIR=$PWD/s
 
 - Never write a seed or private key unencrypted.
 - Keep suite IDs, stanza tags, HKDF info strings and `fileKeySize` (32) stable.
-- Every ciphertext's mtime is pinned to the epoch; records stay one fixed
-  size; objects stay bucket-padded. These hide metadata from the storage host.
-- An object's plaintext starts with its own ID, binding it to its record.
-- `get` publishes output only after the entire object has authenticated.
+- Every ciphertext's and shard directory's mtime is pinned to the epoch;
+  records stay one fixed size; objects stay at most 4 MiB and bucket-padded.
+  These hide metadata from the storage host.
+- An object's plaintext starts with its own ID, binding it to its record
+  and its position in the file.
+- `get` publishes output only after every object has authenticated.
 - Decoy deniability: both key slots stay the same size and are both always
   tried; `init` always writes filler; foreign records are skipped silently.
 - Every command that changes the vault holds the exclusive lock.

README.md

@@ -5,8 +5,11 @@ A post-quantum encrypted file vault in a single binary.
 Files are encrypted with a hybrid FrodoKEM-640 + ML-KEM-768 key encapsulation
 and ChaCha20-Poly1305. The vault directory holds only randomly named
 ciphertexts. Names, sizes, modes and timestamps live inside encrypted
-catalogue records of one fixed size. File bodies are padded to power-of-two
-buckets.
+catalogue records of one fixed size. File bodies are cut into 4 MiB objects,
+the last padded to a power-of-two bucket, so every object is 64 KiB to 4 MiB
+whatever the file's size. Object names derive from a secret seed in the
+record, so nothing on disk links a file's pieces. Records and objects are
+spread over 256 subfolders (`records/3f/3fa9…`) to keep directories small.
 
 ## Build
 
@@ -32,6 +35,7 @@ A full illustrated guide lives in `docs/guide`. Build the PDF with
 ```sh
 vault genkey                     # create ~/.vault/key, protected by a passphrase
 vault init                       # create ~/.vault/store
+vault init --filler 2G           # the same, with at least 2 GiB of filler
 vault add report.pdf             # stored as report.pdf
 vault add report.pdf work/2026   # stored as work/2026/report.pdf
 vault add ./photos               # every regular file under photos/
@@ -84,10 +88,22 @@ Two measures keep the real files deniable:
 A decoy can only be set when the key is created. Someone who watches the
 vault over time can still see it grow when real files are added.
 
+Default filler is only a few tens of megabytes, so someone holding the decoy
+passphrase can see when far more data than that is unopenable. To hide more,
+create the vault with `vault init --filler SIZE` (`500M`, `2G`, `1T`), at
+least as large as what you will hide. Filler entries are written by the
+same code as real files, with a spread of small and multi-chunk sizes, so
+they look like a real collection. Filler can only be added at `init`;
+added later it would stand out as a burst of new entries.
+
 ## Limits
 
-- The storage host can see how many files exist, their size buckets and when
-  the vault changes.
+- The storage host can see how many records and objects exist, so roughly
+  how much is stored, and when the vault changes. Chunking hides a file's
+  size in any single snapshot, but objects written together can be grouped
+  by anyone watching the folder or a sync service's upload log.
+- Padding still costs space: small files round up to a 64 KiB-or-larger
+  bucket and a big file's last piece to its bucket, at most 4 MiB per file.
 - The key file is the only way in. Losing it or its passphrase loses the data.
 
 ## Licence

docs/guide/shots.py

@@ -232,7 +232,7 @@ def main():
     t = Term(box)
     t.cmd("vault init")
     t.cmd("ls -A ~/.vault/store/")
-    t.cmd("ls ~/.vault/store/records | wc -l")
+    t.cmd("find ~/.vault/store/records -type f | wc -l")
     shoot("init", t)
 
     t = Term(box)
@@ -261,7 +261,7 @@ def main():
     shoot("mv-rm", t)
 
     t = Term(box, title="sam@laptop: ~/.vault/store")
-    t.cmd("cd .vault/store && ls -l records | head -5", cwd="~/.vault/store", shown="ls -l records | head -5")
+    t.cmd("cd .vault/store && ls -l records/*/* | head -5", cwd="~/.vault/store", shown="ls -l records/*/* | head -5")
     shoot("on-disk", t)
 
     # decoy: fill the decoy side, then show both views side by side
@@ -275,7 +275,7 @@ def main():
 
     t = Term(box)
     t.cmd("vault get ideas/notes.md -o /tmp/x.md", D, highlight=("not in the vault",))
-    t.cmd("ls ~/.vault/store/records | wc -l")
+    t.cmd("find ~/.vault/store/records -type f | wc -l")
     shoot("decoy-hidden", t)
 
     # errors

docs/guide/vault-guide.tex

@@ -93,9 +93,11 @@ decrypts it. That's the core loop. Put in, take out.
 
 The interesting part is what the folder looks like from the outside. Nothing.
 Well, not \emph{nothing}. A pile of randomly named files, all dated 1 January
-1970, in a handful of suspiciously round sizes. No names. No real sizes. No
-dates. No folders. Someone who steals the folder learns roughly how much stuff
-you have, and that's about it.
+1970, all in a handful of suspiciously round sizes, none bigger than 4\,MiB,
+scattered across folders named \file{00} to \file{ff}. No names. No real
+sizes. No dates. None of \emph{your} folders. Big files are cut into pieces
+that look like everything else. Someone who steals the folder learns roughly
+how much stuff you have, and that's about it.
 
 \begin{center}
 \begin{tikzpicture}[
@@ -436,6 +438,12 @@ protect yet.
 
 \shot{init}{A fresh vault. Two sub-folders, a marker, and\ldots\ records already?!}
 
+Inside \file{records/} and \file{objects/} are 256 folders each, named
+\file{00} to \file{ff}. Every encrypted file lives in the one matching the
+first two characters of its name. That just keeps any one folder from
+getting huge, which some file systems and sync services dislike. All 256
+exist from the start, so the layout itself gives nothing away.
+
 \section{Wait, why is there already stuff in it?}
 
 Good catch. A brand new vault holds somewhere between 8 and 32 entries of
@@ -448,10 +456,33 @@ with your decoy passphrase sees unopenable entries, that's completely
 unremarkable. It's just filler. (Some of it might not be. They can't tell.)
 
 \begin{headsup}
-The filler takes a few tens of megabytes. That's the price of plausible
+By default the filler takes a few tens of megabytes. That's the price of plausible
 deniability, and it's paid once.
 \end{headsup}
 
+\section{More filler, for bigger secrets}
+
+A few tens of megabytes of filler can only cover for a few files. If you
+plan to hide more than that behind a decoy, ask for more when you create the
+vault:
+
+\begin{shell}
+$ vault init --filler 2G
+\end{shell}
+
+vault keeps writing filler until there's at least that much. Sizes take
+\cmd{K}, \cmd{M}, \cmd{G} or \cmd{T}. The filler entries are built exactly
+like real ones, from a mix of small and large random ``files'', so a big
+file of yours sits among big files of filler. It costs the disk space and a
+few seconds per gigabyte, once. It can only be chosen at \cmd{init}: added
+later, the filler would show up as a burst of new entries, which is exactly
+what it's meant to hide.
+
+\begin{quickwin}
+Rule of thumb: set \cmd{--filler} to at least as much as you ever expect to
+hide from the decoy passphrase. Without a decoy, you don't need it.
+\end{quickwin}
+
 \section{More than one vault}
 
 Want a separate vault for work stuff? Point \cmd{--vault} anywhere:
@@ -473,6 +504,7 @@ type \cmd{--vault} again.
 \begin{pocketlist}
   \item \cmd{vault init} makes \file{\tld/.vault/store}. \cmd{--vault DIR} to put it elsewhere.
   \item Filler entries are normal. Nobody can open them. That's the point.
+  \item Hiding a lot behind a decoy? \cmd{vault init --filler 2G} (or however much you'll hide).
   \item One key, as many vaults as you like.
 \end{pocketlist}
 \end{pocket}
@@ -677,7 +709,8 @@ The decoy is chosen when the key is made, and only then:
 \begin{steps}
   \item Run \cmd{vault genkey}. Type your \emph{real} passphrase, twice.
   \item At \hl{Enter decoy passphrase (empty for none)}, type the decoy. Twice.
-  \item Run \cmd{vault init} as normal.
+  \item Run \cmd{vault init}, adding \cmd{--filler} with at least as much as
+        you plan to hide, say \cmd{--filler 5G}.
   \item Add some decoy files, typing the \emph{decoy} passphrase when asked.
   \item Add your real files, typing the \emph{real} passphrase.
 \end{steps}
@@ -704,8 +737,9 @@ there's nothing to safely ``upgrade''.
   \item \textbf{The vault can't tell.} Real and decoy entries are sealed to
         different keys, but they're the same size, the same shape, and all
         dated 1970. Nothing on disk says which key an entry belongs to.
-  \item \textbf{The filler covers for you.} Every vault starts with 8 to 32
-        entries nobody can open. So ``some entries I can't open'' is true of
+  \item \textbf{The filler covers for you.} Every vault starts with at
+        least 8 to 32 entries nobody can open, and as much more as you ask
+        for with \cmd{--filler}. So ``some entries I can't open'' is true of
         \emph{every} vault, decoy or not.
   \item \textbf{The errors are identical.} A real file asked for by name with
         the decoy passphrase gives exactly the same ``not in the vault'' as
@@ -734,10 +768,14 @@ one with two files called \file{test.txt}, is a neon sign.
 \section{The honest limits}
 
 \begin{headsup}
-\textbf{Numbers can give you away.} Every file is one entry (one file in
-\file{records/}), and filler adds at most 32. If the decoy shows 3 files but
-\file{records/} holds 400, the filler can't explain the difference. Keep the decoy collection
-large enough that the leftovers look like filler.
+\textbf{Numbers can give you away.} Someone holding the decoy passphrase
+can count the records and pieces they can't open. Without \cmd{--filler}
+there are only a few tens of megabytes of filler, so if \file{records/} holds
+400 entries, or \file{objects/} holds a 300\,MB video's 72 pieces, the
+filler can't explain the difference. They can't see \emph{what} the
+leftovers are, but they can see there's more than filler. Create the vault
+with \cmd{--filler} at least as large as what you'll hide, and keep the
+decoy collection believable next to it.
 \end{headsup}
 
 \begin{headsup}
@@ -950,8 +988,9 @@ point \cmd{--vault} at the backup copy and run \cmd{vault list} now and then.
 \begin{tldr}
 Each file gets its own random 256-bit key, wrapped with a hybrid of FrodoKEM
 and ML-KEM, and is encrypted in authenticated 64\,KiB chunks. Catalogue
-entries are padded to one fixed size, file contents to power-of-two buckets,
-and every timestamp on disk is set to 1970. An attacker learns roughly how
+entries are padded to one fixed size, file contents are cut into 4\,MiB
+pieces with the last padded to a power-of-two bucket, and every timestamp on
+disk is set to 1970. An attacker learns roughly how
 much you store. They can delete or roll back entries, but not read or forge them.
 \end{tldr}
 
@@ -983,26 +1022,60 @@ either one alone isn't enough.
   note/.style={font=\footnotesize\color{vslate},anchor=west},
   tree/.style={draw=vslate!45,line width=0.8pt}]
   \node[dir] (root) at (0,0) {\tld/.vault/store/};
-  \node[f] (m) at (0.6,-0.7) {.vault};      \node[note] at (4.2,-0.7) {marker: ``VAULT 1''};
+  \node[f] (m) at (0.6,-0.7) {.vault};      \node[note] at (4.2,-0.7) {marker: ``VAULT 2''};
   \node[f] (l) at (0.6,-1.3) {.lock};       \node[note] at (4.2,-1.3) {lock file, always empty};
-  \node[dir] (r) at (0.6,-1.9) {records/};  \node[note] at (4.2,-1.9) {one encrypted catalogue entry per file};
+  \node[dir] (r) at (0.6,-1.9) {records/0f/};  \node[note] at (4.2,-1.9) {one encrypted catalogue entry per file};
   \node[f] (r1) at (1.2,-2.5) {0f1170a6\ldots}; \node[note] at (4.2,-2.5) {every one exactly 18,704 bytes};
-  \node[dir] (o) at (0.6,-3.1) {objects/};  \node[note] at (4.2,-3.1) {one encrypted file body per file};
-  \node[f] (o1) at (1.2,-3.7) {3e78275e\ldots}; \node[note] at (4.2,-3.7) {64\,KiB, 128\,KiB, 256\,KiB\ldots\ never in between};
+  \node[dir] (o) at (0.6,-3.1) {objects/3e/};  \node[note] at (4.2,-3.1) {file contents, in 4\,MiB pieces};
+  \node[f] (o1) at (1.2,-3.7) {3e78275e\ldots}; \node[note] at (4.2,-3.7) {64\,KiB, 128\,KiB\ldots\ 4\,MiB, never in between};
   \draw[tree] (0.3,-0.25) |- (m.west); \draw[tree] (0.3,-0.7) |- (l.west); \draw[tree] (0.3,-1.3) |- (r.west);
   \draw[tree] (0.3,-1.9) |- (o.west);
   \draw[tree] (0.9,-2.15) |- (r1.west); \draw[tree] (0.9,-3.35) |- (o1.west);
 \end{tikzpicture}
 \end{center}
 
-A \hl{record} holds one file's name, size, permissions, dates and the ID of
-its object. It's padded to 4096 bytes before encryption, so every record is
-the same size whatever the name. An \hl{object} holds the file's contents,
-prefixed with its own ID and padded with random bytes up to the next power
-of two, at least 64\,KiB.
+A \hl{record} holds one file's name, size, permissions, dates and a secret
+random \hl{seed}. It's padded to 4096 bytes before encryption, so every
+record is the same size whatever the name. The file's contents are cut into
+\hl{objects} of just under 4\,MiB each. Every object starts with its own ID,
+the last one is padded with random bytes up to the next power of two (at
+least 64\,KiB), and each is encrypted separately. The objects' names are
+worked out from the seed, so only someone who can open the record can tell
+which objects belong together.
 
 \shot{on-disk}{Random names, identical sizes, and all ``modified'' at the dawn of Unix time}
 
+\section{Why big files are cut up}
+
+With one object per file, a 300\,MB video would sit in the folder as a
+single 512\,MiB blob. Everyone could see there's one large file, roughly how
+large, and, since filler is never that big, that it can't be filler. Cut
+into pieces, the same video is 72 objects of exactly the same size as the
+largest filler and as every other big file's pieces. Nothing on disk says
+which pieces belong together, or where one file ends and the next begins.
+
+It's not magic, and some padding and fluff is still doing the work:
+
+\begin{itemize}
+  \item \textbf{Small files are still bucketed.} A file under 4\,MiB is one
+        object padded to 64\,KiB, 128\,KiB, 256\,KiB and so on, exactly as
+        before. The last piece of a big file is padded the same way, so no
+        file wastes more than 4\,MiB, where a single padded blob could
+        waste nearly half its size.
+  \item \textbf{Filler is still the cover story.} The random entries
+        \cmd{init} writes are the reason ``objects I can't open'' is
+        unremarkable. By default there are only a few tens of megabytes of
+        them, enough to hide a few files, not gigabytes. \cmd{vault init
+        --filler SIZE} writes more, shaped like a real collection of
+        files, at the cost of that much disk.
+  \item \textbf{Totals still show.} The number of objects times 4\,MiB is
+        a fair estimate of how much you store.
+  \item \textbf{Timing still shows.} Someone who watches the folder change,
+        or reads a sync service's upload log, sees 72 objects appear together
+        and can guess they're one file. The pieces hide a file's size from a
+        snapshot, not from someone watching you add it.
+\end{itemize}
+
 \section{What an attacker learns}
 
 \begin{vtable}{Y>{\centering\arraybackslash}p{22mm}}
@@ -1014,7 +1087,8 @@ Learn when a file was created or changed & \textcolor{vgreendeep}{\faIcon{times}
 Tell real entries from decoy or filler & \textcolor{vgreendeep}{\faIcon{times}\ no} \\
 Alter a file without it being detected & \textcolor{vgreendeep}{\faIcon{times}\ no} \\
 Swap one file's contents for another's & \textcolor{vgreendeep}{\faIcon{times}\ no} \\
-Count entries, and see rough size buckets & \textcolor{vamberdeep}{\faIcon{check}\ yes} \\
+Tell which pieces make up one file, from a single copy & \textcolor{vgreendeep}{\faIcon{times}\ no} \\
+Count files, and see roughly how much you store & \textcolor{vamberdeep}{\faIcon{check}\ yes} \\
 See that the folder changed between two copies & \textcolor{vamberdeep}{\faIcon{check}\ yes} \\
 Delete entries & \textcolor{vcoraldeep}{\faIcon{check}\ yes} \\
 Put back an older copy of an entry (roll back) & \textcolor{vcoraldeep}{\faIcon{check}\ yes} \\
@@ -1029,15 +1103,17 @@ If the storage isn't under your control, keep an independent backup.
 \section{What happens during \texttt{add}}
 
 \begin{steps}
-  \item Read your file and encrypt its contents, plus random padding, to a
-        hidden temporary file in \file{objects/}.
-  \item Give it its random name. It's now safely stored, but nothing refers to it.
-  \item Encrypt a record pointing at it, the same way, into \file{records/}.
+  \item Pick a random seed. Read your file 4\,MiB at a time and encrypt each
+        piece, the last with random padding, to a hidden temporary file in
+        its \file{objects/} folder. Several pieces are written at once.
+  \item Give each piece the name the seed dictates. They're now safely
+        stored, but nothing refers to them.
+  \item Encrypt a record holding the seed, the same way, into \file{records/}.
   \item Only then, delete any older version of the same name.
 \end{steps}
 
-Interrupt it anywhere and the worst case is an orphaned encrypted object that
-nothing points to. You never end up with a record pointing at nothing, or
+Interrupt it anywhere and the worst case is some orphaned encrypted objects
+that nothing points to. You never end up with a record pointing at nothing, or
 without either version.
 
 \begin{squirrel}
@@ -1051,7 +1127,8 @@ is how bugs get in.
 \begin{pocket}
 \begin{pocketlist}
   \item Hybrid post-quantum key wrapping, per-file keys, authenticated streaming encryption.
-  \item Fixed-size records, power-of-two objects, 1970 timestamps: metadata stays private.
+  \item Fixed-size records, 4\,MiB pieces, power-of-two tails, 1970 timestamps: metadata stays private.
+  \item Pieces hide a file's size in any one copy of the folder, not from someone watching it change.
   \item Tampering is caught. Deletion and rollback are not, so keep backups.
 \end{pocketlist}
 \end{pocket}
@@ -1137,7 +1214,7 @@ passphrase. Nothing in this chapter helps with that. Back up your key.
 \begin{tcbraster}[raster columns=2,raster equal height=rows,raster column skip=8pt,raster row skip=8pt,
   enhanced,colback=vmist,colframe=vline,boxrule=0.4pt,arc=4pt,left=9pt,right=9pt,top=7pt,bottom=7pt]
 \begin{tcolorbox}{\ttfamily\color{vtealdeep}\bfseries vault genkey}\\[2pt]\small Make a key. Asks for a passphrase and an optional decoy.\end{tcolorbox}
-\begin{tcolorbox}{\ttfamily\color{vtealdeep}\bfseries vault init}\\[2pt]\small Make an empty vault, with filler.\end{tcolorbox}
+\begin{tcolorbox}{\ttfamily\color{vtealdeep}\bfseries vault init}\\[2pt]\small Make an empty vault, with filler. \texttt{--filler 2G} for more.\end{tcolorbox}
 \begin{tcolorbox}{\ttfamily\color{vtealdeep}\bfseries vault add FILE|DIR [FOLDER]}\\[2pt]\small Encrypt in, keeping the name. Replaces an existing name.\end{tcolorbox}
 \begin{tcolorbox}{\ttfamily\color{vtealdeep}\bfseries vault list [FOLDER]}\\[2pt]\small Size, modified time and name, sorted.\end{tcolorbox}
 \begin{tcolorbox}{\ttfamily\color{vtealdeep}\bfseries vault get NAME [-o OUT]}\\[2pt]\small Decrypt out. Never overwrites. Verifies first.\end{tcolorbox}
@@ -1154,6 +1231,7 @@ passphrase. Nothing in this chapter helps with that. Back up your key.
 \textrm{(prompt)} & VAULT\_PASSPHRASE & asked on the terminal \\
 \textrm{(genkey prompt)} & VAULT\_DECOY\_PASSPHRASE & asked; empty means none \\
 -o OUT & \textrm{---} & the file's own name, here \\
+--filler SIZE & \textrm{---} & a few tens of MB, for \cmd{init} \\
 \end{vtable}
 
 \section*{Numbers worth knowing}
@@ -1162,7 +1240,9 @@ passphrase. Nothing in this chapter helps with that. Back up your key.
 Largest file & 1\,TiB \\
 Longest stored name & 1024 bytes \\
 Smallest object on disk & 64\,KiB \\
-Filler entries in a new vault & 8 to 32 \\
+Largest object on disk & 4\,MiB \\
+Folders under \file{records/} and \file{objects/} & 256 each \\
+Filler entries in a new vault & 8 to 32, or more with \cmd{--filler} \\
 Key file size & 319 bytes \\
 Exit codes & 0 ok, 1 error, 2 no command \\
 \end{vtable}
@@ -1177,13 +1257,13 @@ Exit codes & 0 ok, 1 error, 2 no command \\
   \item[Argon2id] A deliberately slow, memory-hard function that turns your passphrase into a key, making guessing expensive.
   \item[Catalogue] The encrypted list of what's in your vault, stored as one record per file.
   \item[Decoy passphrase] A second passphrase that opens a separate, harmless set of files.
-  \item[Entry] One record plus its object. Real, decoy and filler entries all look identical on disk.
+  \item[Entry] One record plus its objects. Real, decoy and filler entries all look identical on disk.
   \item[Filler] Entries sealed to a key that was immediately destroyed. Nobody can open them. They make the decoy plausible.
   \item[FrodoKEM, ML-KEM] Post-quantum key-encapsulation algorithms. vault uses both at once.
   \item[Key file] The small file holding your identity, encrypted under your passphrase (and decoy).
-  \item[Object] The encrypted contents of one file, padded to a power-of-two size.
+  \item[Object] One encrypted piece of a file, 4\,MiB, or padded to a smaller power of two if it's the last piece.
   \item[Post-quantum] Designed to stay secure against attackers with large quantum computers.
-  \item[Record] The encrypted catalogue entry for one file: its name, size, dates and object ID.
+  \item[Record] The encrypted catalogue entry for one file: its name, size, dates and the seed its objects' names come from.
   \item[Vault] The folder where your encrypted files live.
 \end{description}
 

main.go

@@ -13,6 +13,7 @@ import (
 	"path"
 	"path/filepath"
 	"sort"
+	"strconv"
 	"strings"
 
 	"github.com/andydixon/vault/internal/term"
@@ -20,7 +21,8 @@ import (
 
 const usage = `Usage:
     vault genkey                    create a new passphrase-protected key
-    vault init                      create a new, empty vault
+    vault init [--filler SIZE]      create a new, empty vault, with at least
+                                    SIZE of filler (e.g. 500M, 2G)
     vault add FILE|DIR [FOLDER]     encrypt a file or directory into the vault,
                                     optionally under a folder inside the vault
     vault list [PREFIX]             list stored files
@@ -62,6 +64,7 @@ func main() {
 	keyPath := fset.String("key", defaultPath("VAULT_KEY", "key"), "")
 	vaultDir := fset.String("vault", defaultPath("VAULT_DIR", "store"), "")
 	output := fset.String("o", "", "")
+	filler := fset.String("filler", "0", "")
 	pos, err := parseInterleaved(fset, args)
 	if err != nil {
 		fail("%v\n\n%s", err, usage)
@@ -87,7 +90,9 @@ func main() {
 		fmt.Fprintf(os.Stderr, "key written to %s\n", *keyPath)
 	case "init":
 		want(0, 0)
-		check(initStore(*vaultDir))
+		size, err := parseSize(*filler)
+		check(err)
+		check(initStore(*vaultDir, size))
 		fmt.Fprintf(os.Stderr, "vault created at %s\n", *vaultDir)
 	case "add":
 		want(1, 2)
@@ -144,6 +149,22 @@ func parseInterleaved(fset *flag.FlagSet, args []string) ([]string, error) {
 	}
 }
 
+// parseSize reads a byte count with an optional K, M, G or T suffix, in
+// powers of 1024.
+func parseSize(s string) (int64, error) {
+	num, shift := strings.TrimSuffix(strings.ToUpper(s), "B"), 0
+	for i, unit := range "KMGT" {
+		if strings.HasSuffix(num, string(unit)) {
+			num, shift = strings.TrimSuffix(num, string(unit)), 10*(i+1)
+		}
+	}
+	n, err := strconv.ParseInt(num, 10, 64)
+	if err != nil || n < 0 || n > fileLimit>>shift {
+		return 0, fmt.Errorf("invalid size %q", s)
+	}
+	return n << shift, nil
+}
+
 func defaultPath(env, name string) string {
 	if v := os.Getenv(env); v != "" {
 		return v

store.go

@@ -5,18 +5,23 @@ package main
 
 import (
 	"bytes"
+	"crypto/hmac"
 	"crypto/rand"
+	"crypto/sha256"
 	"encoding/binary"
 	"encoding/hex"
 	"encoding/json"
 	"errors"
 	"fmt"
 	"io"
+	"math"
+	mrand "math/rand/v2"
 	"os"
 	"path"
 	"path/filepath"
 	"sort"
 	"strings"
+	"sync"
 	"time"
 	"unicode"
 	"unicode/utf8"
@@ -26,31 +31,40 @@ import (
 
 // On disk a vault is:
 //
-//	DIR/.vault            marker
-//	DIR/records/<id>      one encrypted catalogue record per file
-//	DIR/objects/<id>      one encrypted file body per record
+//	DIR/.vault              marker
+//	DIR/records/<xx>/<id>   one encrypted catalogue record per file
+//	DIR/objects/<xx>/<id>   the encrypted body of a file, in one or more chunks
 //
-// IDs are random. Records are padded to one fixed size and objects to
-// power-of-two buckets, so the storage reveals neither names, exact sizes nor
-// timestamps. Every ciphertext's mtime is pinned to the epoch.
+// <xx> is the first two hex digits of <id>, spreading entries over 256
+// shards. IDs look random: record IDs are, and a file's object IDs derive
+// from a secret seed in its record, so nothing on disk links a file's
+// chunks. Records are padded to one fixed size. A body is cut into
+// chunkSize objects and the last is padded to a power-of-two bucket, so
+// every object is 64 KiB to 4 MiB whatever the file size. Every ciphertext's
+// and shard's mtime is pinned to the epoch.
 
 const (
-	markerName    = ".vault"
-	markerContent = "VAULT 1\n"
-	recordsDir    = "records"
-	objectsDir    = "objects"
-	recordSize    = 4096
-	nameLimit     = 1024
-	fileLimit     = int64(1) << 40
-	minPadding    = int64(64 << 10)
-	idSize        = 16
-	fillerMin     = 8
-	fillerMax     = 32
+	markerName     = ".vault"
+	markerContent  = "VAULT 2\n"
+	recordsDir     = "records"
+	objectsDir     = "objects"
+	recordSize     = 4096
+	nameLimit      = 1024
+	fileLimit      = int64(1) << 40
+	minPadding     = int64(64 << 10)
+	chunkSize      = minPadding << 6 // 4 MiB, the largest bucket
+	chunkData      = chunkSize - idSize
+	idSize         = 16
+	objectSeedSize = 32
+	fillerMin      = 8
+	fillerMax      = 32
 )
 
+var epoch = time.Unix(0, 0)
+
 type record struct {
 	Name     string    `json:"name"`
-	Object   string    `json:"object"`
+	Seed     string    `json:"seed"` // derives the file's object IDs
 	Size     int64     `json:"size"`
 	Mode     uint32    `json:"mode"`
 	Modified time.Time `json:"modified"`
@@ -65,57 +79,68 @@ type store struct {
 	noSync bool // filler needs no per-file durability
 }
 
-func initStore(root string) error {
+// initStore creates a vault at root holding at least filler bytes of filler.
+func initStore(root string, filler int64) error {
 	if _, err := os.Stat(filepath.Join(root, markerName)); err == nil {
 		return fmt.Errorf("%s is already a vault", root)
 	}
-	for _, dir := range []string{root, filepath.Join(root, recordsDir), filepath.Join(root, objectsDir)} {
-		if err := os.MkdirAll(dir, 0o700); err != nil {
-			return err
+	// Every shard exists from the start, so the tree's shape says nothing.
+	for _, dir := range []string{recordsDir, objectsDir} {
+		for i := range 256 {
+			if err := os.MkdirAll(filepath.Join(root, dir, fmt.Sprintf("%02x", i)), 0o700); err != nil {
+				return err
+			}
 		}
 	}
-	if err := writeFiller(root); err != nil {
+	if err := writeFiller(root, filler); err != nil {
 		return err
 	}
+	for _, dir := range []string{recordsDir, objectsDir} {
+		shards, _ := filepath.Glob(filepath.Join(root, dir, "??"))
+		for _, p := range append(shards, filepath.Join(root, dir)) {
+			if err := os.Chtimes(p, epoch, epoch); err != nil {
+				return err
+			}
+		}
+	}
 	return os.WriteFile(filepath.Join(root, markerName), []byte(markerContent), 0o600)
 }
 
-// writeFiller stores a random number of record/object pairs sealed to a key
-// that is thrown away. Every vault therefore holds entries nobody can open,
-// so a file hidden from a decoy passphrase is indistinguishable from filler.
-func writeFiller(root string) error {
+// writeFiller stores random entries sealed to a key that is thrown away.
+// Every vault therefore holds entries nobody can open, so a file hidden from
+// a decoy passphrase is indistinguishable from filler. Filler entries are
+// real entries, written by the same code as add, of log-uniformly random
+// sizes: fillerMin to fillerMax of them, or more until they total target
+// bytes.
+func writeFiller(root string, target int64) error {
 	throwaway, err := pqcrypt.GeneratePQIdentity()
 	if err != nil {
 		return err
 	}
 	defer throwaway.Close()
 	s := &store{root: root, id: throwaway, noSync: true}
-	var b [2]byte
-	if _, err := rand.Read(b[:1]); err != nil {
-		return err
-	}
-	for range fillerMin + int(b[0])%(fillerMax-fillerMin+1) {
-		if _, err := rand.Read(b[:]); err != nil {
+	// Bigger targets allow bigger files, so the filler holds multi-chunk
+	// files in about the proportion a real collection would.
+	upper := float64(min(max(target/8, chunkSize), 1<<30))
+	count := fillerMin + mrand.IntN(fillerMax-fillerMin+1)
+	for i, total := 0, int64(0); i < count || total < target; i++ {
+		size := int64(1024 * math.Exp(mrand.Float64()*math.Log(upper/1024)))
+		seed, err := randomHex(objectSeedSize)
+		if err != nil {
 			return err
 		}
-		// Object buckets from 64 KiB to 4 MiB, weighted towards small ones.
-		objSize := minPadding << (min(b[0]%8, b[1]%8) % 7)
-		for _, part := range []struct {
-			dir  string
-			size int64
-		}{{recordsDir, recordSize}, {objectsDir, objSize}} {
-			id, err := randomID()
-			if err != nil {
-				return err
-			}
-			err = s.writeCiphertext(part.dir, id, func(w io.Writer) error {
-				_, err := io.CopyN(w, rand.Reader, part.size)
-				return err
-			})
-			if err != nil {
-				return err
-			}
+		if err := s.writeObjects(rand.Reader, seed, size); err != nil {
+			return err
 		}
+		name, err := randomHex(8)
+		if err != nil {
+			return err
+		}
+		now := time.Now().UTC()
+		if err := s.writeRecord(&record{Name: name, Seed: seed, Size: size, Mode: 0o600, Modified: now, Added: now}); err != nil {
+			return err
+		}
+		total += size
 	}
 	return nil
 }
@@ -128,17 +153,47 @@ func openStore(root string, id *pqcrypt.PQIdentity) (*store, error) {
 	return &store{root: root, id: id}, nil
 }
 
-func randomID() (string, error) {
-	b := make([]byte, idSize)
+func randomHex(n int) (string, error) {
+	b := make([]byte, n)
 	if _, err := rand.Read(b); err != nil {
 		return "", err
 	}
 	return hex.EncodeToString(b), nil
 }
 
-func validID(s string) bool {
+func validHex(s string, n int) bool {
 	b, err := hex.DecodeString(s)
-	return err == nil && len(b) == idSize && hex.EncodeToString(b) == s
+	return err == nil && len(b) == n && hex.EncodeToString(b) == s
+}
+
+func validID(s string) bool { return validHex(s, idSize) }
+
+// objectID derives the ID of a file's i-th object from its record's seed.
+// Without the seed the IDs are unlinkable, so a file's chunks can't be told
+// apart from any other object.
+func objectID(seed string, i int64) string {
+	key, _ := hex.DecodeString(seed)
+	m := hmac.New(sha256.New, key)
+	binary.Write(m, binary.BigEndian, i)
+	return hex.EncodeToString(m.Sum(nil)[:idSize])
+}
+
+// objectCount is how many objects hold a file of size bytes.
+func objectCount(size int64) int64 {
+	return max(1, (size+chunkData-1)/chunkData)
+}
+
+func (s *store) path(dir, id string) string {
+	return filepath.Join(s.root, dir, id[:2], id)
+}
+
+// removeFile deletes dir/id and re-pins its shard's mtime.
+func (s *store) removeFile(dir, id string) error {
+	p := s.path(dir, id)
+	if err := os.Remove(p); err != nil {
+		return err
+	}
+	return os.Chtimes(filepath.Dir(p), epoch, epoch)
 }
 
 func validName(name string) bool {
@@ -166,7 +221,8 @@ func paddedSize(size int64) int64 {
 // writeCiphertext encrypts to dir/id via a temporary file, so a partial
 // ciphertext never appears under a real ID.
 func (s *store) writeCiphertext(dir, id string, write func(io.Writer) error) error {
-	dir = filepath.Join(s.root, dir)
+	dst := s.path(dir, id)
+	dir = filepath.Dir(dst)
 	tmp, err := os.CreateTemp(dir, ".pending-")
 	if err != nil {
 		return err
@@ -191,11 +247,13 @@ func (s *store) writeCiphertext(dir, id string, write func(io.Writer) error) err
 	if err := tmp.Close(); err != nil {
 		return err
 	}
-	epoch := time.Unix(0, 0)
 	if err := os.Chtimes(tmp.Name(), epoch, epoch); err != nil {
 		return err
 	}
-	if err := os.Rename(tmp.Name(), filepath.Join(dir, id)); err != nil || s.noSync {
+	if err := os.Rename(tmp.Name(), dst); err != nil {
+		return err
+	}
+	if err := os.Chtimes(dir, epoch, epoch); err != nil || s.noSync {
 		return err
 	}
 	return syncDir(dir)
@@ -224,7 +282,7 @@ func (s *store) writeRecord(r *record) error {
 	if _, err := rand.Read(plain[4+len(data):]); err != nil {
 		return err
 	}
-	if r.id, err = randomID(); err != nil {
+	if r.id, err = randomHex(idSize); err != nil {
 		return err
 	}
 	return s.writeCiphertext(recordsDir, r.id, func(w io.Writer) error {
@@ -236,25 +294,34 @@ func (s *store) writeRecord(r *record) error {
 // catalogue decrypts every record this identity can open. Records sealed to
 // a different key are skipped silently: they are simply not ours.
 func (s *store) catalogue() (map[string][]*record, error) {
-	entries, err := os.ReadDir(filepath.Join(s.root, recordsDir))
+	shards, err := os.ReadDir(filepath.Join(s.root, recordsDir))
 	if err != nil {
 		return nil, err
 	}
 	byName := map[string][]*record{}
-	for _, e := range entries {
-		if !validID(e.Name()) {
-			continue
-		}
-		r, err := s.readRecord(e.Name())
-		var noMatch *pqcrypt.NoIdentityMatchError
-		if errors.As(err, &noMatch) {
+	for _, shard := range shards {
+		if !shard.IsDir() {
 			continue
 		}
+		entries, err := os.ReadDir(filepath.Join(s.root, recordsDir, shard.Name()))
 		if err != nil {
-			fmt.Fprintf(os.Stderr, "vault: skipping unreadable record %s: %v\n", e.Name(), err)
-			continue
+			return nil, err
+		}
+		for _, e := range entries {
+			if !validID(e.Name()) || e.Name()[:2] != shard.Name() {
+				continue
+			}
+			r, err := s.readRecord(e.Name())
+			var noMatch *pqcrypt.NoIdentityMatchError
+			if errors.As(err, &noMatch) {
+				continue
+			}
+			if err != nil {
+				fmt.Fprintf(os.Stderr, "vault: skipping unreadable record %s: %v\n", e.Name(), err)
+				continue
+			}
+			byName[r.Name] = append(byName[r.Name], r)
 		}
-		byName[r.Name] = append(byName[r.Name], r)
 	}
 	// Newest first, so [0] is current. Duplicates only exist after an
 	// interrupted replace; the next write of that name cleans them up.
@@ -265,7 +332,7 @@ func (s *store) catalogue() (map[string][]*record, error) {
 }
 
 func (s *store) readRecord(id string) (*record, error) {
-	f, err := os.Open(filepath.Join(s.root, recordsDir, id))
+	f, err := os.Open(s.path(recordsDir, id))
 	if err != nil {
 		return nil, err
 	}
@@ -291,7 +358,7 @@ func (s *store) readRecord(id string) (*record, error) {
 	if err := dec.Decode(r); err != nil {
 		return nil, err
 	}
-	if !validName(r.Name) || !validID(r.Object) || r.Size < 0 || r.Size > fileLimit {
+	if !validName(r.Name) || !validHex(r.Seed, objectSeedSize) || r.Size < 0 || r.Size > fileLimit {
 		return nil, errors.New("malformed record")
 	}
 	return r, nil
@@ -315,36 +382,23 @@ func (s *store) add(cat map[string][]*record, src, name string) (bool, error) {
 	if !info.Mode().IsRegular() || info.Size() > fileLimit {
 		return false, fmt.Errorf("%s: not a regular file of at most 1 TiB", src)
 	}
-	objID, err := randomID()
+	seed, err := randomHex(objectSeedSize)
 	if err != nil {
 		return false, err
 	}
-	err = s.writeCiphertext(objectsDir, objID, func(w io.Writer) error {
-		// The object ID leads the plaintext so a record can only ever
-		// open the object it names, not one swapped in under its ID.
-		raw, _ := hex.DecodeString(objID)
-		if _, err := w.Write(raw); err != nil {
-			return err
+	if err := s.writeObjects(f, seed, info.Size()); err != nil {
+		s.dropObjects(seed, info.Size())
+		if errors.Is(err, io.ErrUnexpectedEOF) || errors.Is(err, io.EOF) {
+			err = fmt.Errorf("%s changed while being added", src)
 		}
-		n, err := io.Copy(w, io.LimitReader(f, info.Size()))
-		if err != nil {
-			return err
-		}
-		if n != info.Size() {
-			return fmt.Errorf("%s changed while being added", src)
-		}
-		_, err = io.CopyN(w, rand.Reader, paddedSize(idSize+n)-idSize-n)
-		return err
-	})
-	if err != nil {
 		return false, err
 	}
 	r := &record{
-		Name: name, Object: objID, Size: info.Size(), Mode: uint32(info.Mode().Perm()),
+		Name: name, Seed: seed, Size: info.Size(), Mode: uint32(info.Mode().Perm()),
 		Modified: info.ModTime().UTC(), Added: time.Now().UTC(),
 	}
 	if err := s.writeRecord(r); err != nil {
-		os.Remove(filepath.Join(s.root, objectsDir, objID))
+		s.dropObjects(seed, info.Size())
 		return false, err
 	}
 	replaced := len(cat[name]) > 0
@@ -355,6 +409,67 @@ func (s *store) add(cat map[string][]*record, src, name string) (bool, error) {
 	return replaced, nil
 }
 
+// writeObjects reads size bytes from f into the objects derived from seed.
+// Chunks are read in order but encrypted, written and synced by a few
+// workers at once, so their fsyncs overlap instead of queueing.
+func (s *store) writeObjects(f io.Reader, seed string, size int64) error {
+	var (
+		wg       sync.WaitGroup
+		mu       sync.Mutex
+		firstErr error
+	)
+	fail := func(err error) {
+		mu.Lock()
+		defer mu.Unlock()
+		if firstErr == nil {
+			firstErr = err
+		}
+	}
+	failed := func() bool {
+		mu.Lock()
+		defer mu.Unlock()
+		return firstErr != nil
+	}
+	slots := make(chan struct{}, 8) // at most 8 chunks, 32 MiB, in memory
+	remaining := size
+	for i := range objectCount(size) {
+		if failed() {
+			break
+		}
+		// The object ID leads the plaintext so a record can only ever
+		// open the objects it derives, each in its place, not one
+		// swapped in under its ID.
+		id := objectID(seed, i)
+		part := min(remaining, chunkData)
+		remaining -= part
+		slots <- struct{}{}
+		buf := make([]byte, idSize+part)
+		hex.Decode(buf, []byte(id))
+		if _, err := io.ReadFull(f, buf[idSize:]); err != nil {
+			<-slots
+			fail(err)
+			break
+		}
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			defer func() { <-slots }()
+			err := s.writeCiphertext(objectsDir, id, func(w io.Writer) error {
+				if _, err := w.Write(buf); err != nil {
+					return err
+				}
+				_, err := io.CopyN(w, rand.Reader, paddedSize(int64(len(buf)))-int64(len(buf)))
+				return err
+			})
+			if err != nil {
+				fail(err)
+			}
+		}()
+	}
+	wg.Wait()
+	return firstErr
+}
+
 // remove deletes every record for name and, when dropObjects is set, any of
 // their objects no other record still uses. Records go first, so an
 // interruption only ever leaves an orphaned object.
@@ -363,16 +478,16 @@ func (s *store) remove(cat map[string][]*record, name string, dropObjects bool)
 	for n, rs := range cat {
 		for _, r := range rs {
 			if n != name {
-				inUse[r.Object] = true
+				inUse[r.Seed] = true
 			}
 		}
 	}
 	for _, r := range cat[name] {
-		if err := os.Remove(filepath.Join(s.root, recordsDir, r.id)); err != nil {
+		if err := s.removeFile(recordsDir, r.id); err != nil {
 			return err
 		}
-		if dropObjects && !inUse[r.Object] {
-			if err := os.Remove(filepath.Join(s.root, objectsDir, r.Object)); err != nil && !os.IsNotExist(err) {
+		if dropObjects && !inUse[r.Seed] {
+			if err := s.dropObjects(r.Seed, r.Size); err != nil {
 				return err
 			}
 		}
@@ -380,6 +495,17 @@ func (s *store) remove(cat map[string][]*record, name string, dropObjects bool)
 	return nil
 }
 
+// dropObjects deletes the objects holding a file of size bytes, skipping any
+// already gone.
+func (s *store) dropObjects(seed string, size int64) error {
+	for i := range objectCount(size) {
+		if err := s.removeFile(objectsDir, objectID(seed, i)); err != nil && !os.IsNotExist(err) {
+			return err
+		}
+	}
+	return nil
+}
+
 func (s *store) lookup(name string) (map[string][]*record, *record, error) {
 	cat, err := s.catalogue()
 	if err != nil {
@@ -419,7 +545,7 @@ func (s *store) rename(oldName, newName string) error {
 	return s.remove(cat, oldName, false)
 }
 
-// get decrypts name to out. Nothing appears at out until the entire object,
+// get decrypts name to out. Nothing appears at out until every object,
 // padding included, has authenticated.
 func (s *store) get(name, out string) error {
 	_, r, err := s.lookup(name)
@@ -429,34 +555,19 @@ func (s *store) get(name, out string) error {
 	if _, err := os.Lstat(out); err == nil {
 		return fmt.Errorf("%s already exists", out)
 	}
-	f, err := os.Open(filepath.Join(s.root, objectsDir, r.Object))
-	if err != nil {
-		return err
-	}
-	defer f.Close()
-	pr, err := pqcrypt.Decrypt(f, s.id)
-	if err != nil {
-		return err
-	}
-	head := make([]byte, idSize)
-	if _, err := io.ReadFull(pr, head); err != nil || hex.EncodeToString(head) != r.Object {
-		return errors.New("object does not match its catalogue record")
-	}
 	tmp, err := os.CreateTemp(filepath.Dir(out), ".vault-get-")
 	if err != nil {
 		return err
 	}
 	defer os.Remove(tmp.Name())
 	defer tmp.Close()
-	if _, err := io.CopyN(tmp, pr, r.Size); err != nil {
-		return err
-	}
-	pad, err := io.Copy(io.Discard, pr)
-	if err != nil {
-		return err
-	}
-	if idSize+r.Size+pad != paddedSize(idSize+r.Size) {
-		return errors.New("object size does not match its catalogue record")
+	remaining := r.Size
+	for i := range objectCount(r.Size) {
+		part := min(remaining, chunkData)
+		if err := s.readObject(tmp, objectID(r.Seed, i), part); err != nil {
+			return err
+		}
+		remaining -= part
 	}
 	if err := tmp.Chmod(os.FileMode(r.Mode) & 0o777); err != nil {
 		return err
@@ -472,3 +583,32 @@ func (s *store) get(name, out string) error {
 	}
 	return os.Rename(tmp.Name(), out)
 }
+
+// readObject decrypts object id to w, checking it leads with its own ID and
+// holds part bytes padded to their bucket.
+func (s *store) readObject(w io.Writer, id string, part int64) error {
+	f, err := os.Open(s.path(objectsDir, id))
+	if err != nil {
+		return err
+	}
+	defer f.Close()
+	pr, err := pqcrypt.Decrypt(f, s.id)
+	if err != nil {
+		return err
+	}
+	head := make([]byte, idSize)
+	if _, err := io.ReadFull(pr, head); err != nil || hex.EncodeToString(head) != id {
+		return errors.New("object does not match its catalogue record")
+	}
+	if _, err := io.CopyN(w, pr, part); err != nil {
+		return err
+	}
+	pad, err := io.Copy(io.Discard, pr)
+	if err != nil {
+		return err
+	}
+	if idSize+part+pad != paddedSize(idSize+part) {
+		return errors.New("object size does not match its catalogue record")
+	}
+	return nil
+}

vault_test.go

@@ -6,7 +6,9 @@ package main
 import (
 	"bufio"
 	"bytes"
+	"crypto/rand"
 	"io"
+	"io/fs"
 	"os"
 	"os/exec"
 	"path/filepath"
@@ -46,7 +48,7 @@ func TestStoreLifecycle(t *testing.T) {
 	if err := writeKeyFile(keyPath, "pass", ""); err != nil {
 		t.Fatal(err)
 	}
-	if err := initStore(filepath.Join(dir, "store")); err != nil {
+	if err := initStore(filepath.Join(dir, "store"), 0); err != nil {
 		t.Fatal(err)
 	}
 	s := newTestStore(t, keyPath, "pass")
@@ -84,9 +86,8 @@ func TestStoreLifecycle(t *testing.T) {
 	// Swapping one object for another must be detected.
 	cat, _ = s.catalogue()
 	d, c := cat["d"][0], cat["c"][0]
-	objects := filepath.Join(s.root, objectsDir)
-	data, _ := os.ReadFile(filepath.Join(objects, c.Object))
-	os.WriteFile(filepath.Join(objects, d.Object), data, 0o600)
+	data, _ := os.ReadFile(s.path(objectsDir, objectID(c.Seed, 0)))
+	os.WriteFile(s.path(objectsDir, objectID(d.Seed, 0)), data, 0o600)
 	if err := s.get("d", filepath.Join(dir, "swapped")); err == nil {
 		t.Fatal("swapped object was accepted")
 	}
@@ -105,7 +106,7 @@ func TestDecoy(t *testing.T) {
 	if err := writeKeyFile(keyPath, "real", "decoy"); err != nil {
 		t.Fatal(err)
 	}
-	if err := initStore(filepath.Join(dir, "store")); err != nil {
+	if err := initStore(filepath.Join(dir, "store"), 0); err != nil {
 		t.Fatal(err)
 	}
 	src := filepath.Join(dir, "in")
@@ -128,9 +129,96 @@ func TestDecoy(t *testing.T) {
 	if _, err := unlockKeyFile(keyPath, "neither"); err == nil {
 		t.Fatal("wrong passphrase unlocked the key")
 	}
-	entries, _ := os.ReadDir(filepath.Join(dir, "store", recordsDir))
-	if len(entries) < fillerMin+2 {
-		t.Fatalf("only %d records; filler missing", len(entries))
+	if n := len(ciphertexts(t, filepath.Join(dir, "store", recordsDir))); n < fillerMin+2 {
+		t.Fatalf("only %d records; filler missing", n)
+	}
+}
+
+// ciphertexts maps every file under dir to its size.
+func ciphertexts(t *testing.T, dir string) map[string]int64 {
+	t.Helper()
+	sizes := map[string]int64{}
+	err := filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error {
+		if err != nil || d.IsDir() {
+			return err
+		}
+		info, err := d.Info()
+		sizes[p] = info.Size()
+		return err
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+	return sizes
+}
+
+// TestChunks checks a file bigger than one chunk becomes unlinkable 4 MiB
+// objects plus a bucket-sized tail, all in their shards, and that reordering
+// the chunks is caught.
+func TestChunks(t *testing.T) {
+	dir := t.TempDir()
+	keyPath := filepath.Join(dir, "key")
+	if err := writeKeyFile(keyPath, "pass", ""); err != nil {
+		t.Fatal(err)
+	}
+	if err := initStore(filepath.Join(dir, "store"), 0); err != nil {
+		t.Fatal(err)
+	}
+	s := newTestStore(t, keyPath, "pass")
+	objects := filepath.Join(s.root, objectsDir)
+	before := ciphertexts(t, objects)
+
+	src := filepath.Join(dir, "in")
+	want := make([]byte, 2*chunkData+100)
+	rand.Read(want)
+	os.WriteFile(src, want, 0o600)
+	cat, _ := s.catalogue()
+	if _, err := s.add(cat, src, "big"); err != nil {
+		t.Fatal(err)
+	}
+	cat, _ = s.catalogue()
+	r := cat["big"][0]
+	var sizes []int64
+	for i := range objectCount(r.Size) {
+		p := s.path(objectsDir, objectID(r.Seed, i))
+		if filepath.Base(filepath.Dir(p)) != filepath.Base(p)[:2] {
+			t.Fatalf("%s is in the wrong shard", p)
+		}
+		sizes = append(sizes, ciphertexts(t, p)[p])
+	}
+	if len(sizes) != 3 || sizes[0] != sizes[1] || sizes[2] >= sizes[0] {
+		t.Fatalf("object sizes = %v, want two full chunks and a smaller tail", sizes)
+	}
+	if n := len(ciphertexts(t, objects)) - len(before); n != 3 {
+		t.Fatalf("add wrote %d objects, want 3", n)
+	}
+	for p, size := range ciphertexts(t, objects) {
+		if size > sizes[0] {
+			t.Fatalf("%s is %d bytes, larger than a chunk", p, size)
+		}
+	}
+
+	out := filepath.Join(dir, "out")
+	if err := s.get("big", out); err != nil {
+		t.Fatal(err)
+	}
+	if got, _ := os.ReadFile(out); !bytes.Equal(got, want) {
+		t.Fatal("round trip mismatch")
+	}
+
+	a, b := s.path(objectsDir, objectID(r.Seed, 0)), s.path(objectsDir, objectID(r.Seed, 1))
+	os.Rename(a, a+".x")
+	os.Rename(b, a)
+	os.Rename(a+".x", b)
+	if err := s.get("big", filepath.Join(dir, "swapped")); err == nil {
+		t.Fatal("reordered chunks were accepted")
+	}
+
+	if err := s.delete("big"); err != nil {
+		t.Fatal(err)
+	}
+	if n := len(ciphertexts(t, objects)); n != len(before) {
+		t.Fatalf("%d objects after rm, want %d", n, len(before))
 	}
 }
 
@@ -175,3 +263,35 @@ func TestLockExcludesOtherProcess(t *testing.T) {
 		}
 	}
 }
+
+func TestParseSize(t *testing.T) {
+	for in, want := range map[string]int64{"0": 0, "512": 512, "2k": 2 << 10, "500M": 500 << 20, "2G": 2 << 30, "1TB": 1 << 40} {
+		if got, err := parseSize(in); err != nil || got != want {
+			t.Errorf("parseSize(%q) = %d, %v; want %d", in, got, err, want)
+		}
+	}
+	for _, in := range []string{"", "G", "-1", "2X", "1.5G", "2T"} {
+		if _, err := parseSize(in); err == nil {
+			t.Errorf("parseSize(%q) accepted", in)
+		}
+	}
+}
+
+// TestFiller checks a filler target is met with entries shaped like real
+// files: fixed-size records and objects no bigger than a chunk.
+func TestFiller(t *testing.T) {
+	root := filepath.Join(t.TempDir(), "store")
+	if err := initStore(root, 24<<20); err != nil {
+		t.Fatal(err)
+	}
+	if n := len(ciphertexts(t, filepath.Join(root, recordsDir))); n < fillerMin {
+		t.Fatalf("%d filler records, want at least %d", n, fillerMin)
+	}
+	var total, largest int64
+	for _, size := range ciphertexts(t, filepath.Join(root, objectsDir)) {
+		total, largest = total+size, max(largest, size)
+	}
+	if total < 24<<20 || largest > chunkSize+chunkSize/64 {
+		t.Fatalf("filler objects total %d, largest %d", total, largest)
+	}
+}