Network import: bounded add URL and mirror

An explicit ureq client with platform TLS verification, connect/header/
request deadlines, manual bounded redirects, decoded-byte budgets and
Go-style proxy selection. A Go-compatible URL module checked against
oracle vectors, and a byte-preserving HTML tokenizer for link rewriting
(href/src/action/poster, <style> and style= url(), <base href>). Mirror
keeps to one host across redirects, bounds pages and bytes, refuses
local-name collisions and reports partial results.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Andy Dixon
2026-10-08 10:34:28 +01:00
co-authored by Claude
parent 24ebede9bb
commit aedc605a82
7 changed files with 5326 additions and 1 deletions
+1
View File
@@ -10,4 +10,5 @@ cd "$(dirname "$0")/.."
out=$PWD/tests/vectors
mkdir -p "$out"
(cd oracle/go && VAULT_ORACLE_OUT=$out go test -count=1 -run '^TestOracleVectors$' ./ ./internal/pqcrypt ./internal/stream)
(cd oracle/go && VAULT_ORACLE_OUT=$out go test -count=1 -run '^TestOracleNetVectors$' ./)
echo "vectors written to $out"