Add a mirror command that crawls a site into the vault

Single-threaded, same host only, Chrome user agent. Links in HTML and
CSS are rewritten to relative paths. Includes a man page and a guide note.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Andy Dixon
2026-09-29 15:02:12 +01:00
co-authored by Claude Sonnet 5.5
parent e40a803367
commit 10f2a88493
5 changed files with 315 additions and 0 deletions
+6
View File
@@ -561,6 +561,12 @@ downloads it (as Chrome, so fussy sites play along) and stores it under the
last part of the address, such as \file{report.pdf}. The download sits in a
private temporary file only until it's encrypted.
\subsection{Copying a website: \texttt{mirror}}
\cmd{mirror} takes an \file{http://} or \file{https://} address and copies
the whole site, one request at a time, into a folder named after its host.
Links inside pages and stylesheets are rewritten to relative paths, so the
copy still browses once you get it out. Only the same host is followed.
\subsection{Moving in: \texttt{migrate}}
\cmd{migrate} works exactly like \cmd{add}, but deletes each original once
it's safely stored, then tidies away any folders left empty. It won't take a