Chunk objects, shard the store, and make filler configurable
File bodies are now cut into 4 MiB objects, the last padded to its power-of-two bucket, so every object is 64 KiB to 4 MiB whatever the file size. Object IDs are HMAC'd from a secret per-record seed, so nothing on disk links a file's pieces, and each object still leads with its own ID, binding it to its record and position. Chunks are encrypted and synced by a small worker pool so their fsyncs overlap. records/ and objects/ are sharded into 256 subdirectories by ID prefix, all created at init, with every shard's mtime pinned to the epoch. vault init --filler SIZE writes at least SIZE of filler. Filler entries are now real entries for a throwaway key, of log-uniform random sizes, so they carry multi-chunk files like a real collection. The marker becomes VAULT 2; earlier vaults are not supported. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Andy Dixon
Claude Opus 5.5
|
Before Width: | Height: | Size: 165 KiB After Width: | Height: | Size: 165 KiB |
|
Before Width: | Height: | Size: 73 KiB After Width: | Height: | Size: 76 KiB |
|
Before Width: | Height: | Size: 134 KiB After Width: | Height: | Size: 133 KiB |
|
Before Width: | Height: | Size: 75 KiB After Width: | Height: | Size: 75 KiB |
|
Before Width: | Height: | Size: 256 KiB After Width: | Height: | Size: 266 KiB |
|
Before Width: | Height: | Size: 87 KiB After Width: | Height: | Size: 89 KiB |
|
Before Width: | Height: | Size: 129 KiB After Width: | Height: | Size: 128 KiB |
|
Before Width: | Height: | Size: 133 KiB After Width: | Height: | Size: 133 KiB |
|
Before Width: | Height: | Size: 116 KiB After Width: | Height: | Size: 140 KiB |
|
Before Width: | Height: | Size: 88 KiB After Width: | Height: | Size: 88 KiB |