andy/CodeNavigator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Write full documentation with screenshots

Browse Source 
README covers what CodeNavigator is, screenshots of each view, quick
start, configuration, keyboard shortcuts and links to docs/: a city
guide (what every visual means, getting around), architecture (indexer,
sessions, GitHub, rendering passes, routing), the HTTP API, GitHub App
setup including the alert permissions, deployment (Docker, systemd,
nginx, limits, security notes) and development (tests, layout,
conventions, design notes).
This commit is contained in:
Andy Dixon
2026-08-09 10:15:59 +01:00
parent 3b4f1505d1
commit d7318acede
13 changed files with 635 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+67 -62
View File
@@ -1,74 +1,79 @@
# CodeNavigator
A single binary that serves the WebGL frontend and the
indexing API (jobs, SSE progress, snapshots, search, source) on one origin.
No dependencies beyond the Go standard library.
**See your code as a city.** CodeNavigator indexes a GitHub repository (or, if you allow it, a local
folder) and turns it into a place you can explore: a zoomable map, a 3D landscape, and a walkable
night-time city where every file is a building and every folder is a street.
![CodeNavigator city view](docs/images/city.png)
It's a single Go binary with no dependencies beyond the standard library. It serves both the
indexing API and the browser frontend (plain ES modules and WebGL2, no build step).
## What you can do
| | |
|---|---|
| ![2D map](docs/images/map-2d.png) | **2D map**: a squarified treemap of every file, sized by lines or references, with source code rendered right on the tiles as you zoom in. |
| ![3D landscape](docs/images/landscape-3d.png) | **3D landscape**: the same map extruded, orbit, pan and dolly around it. |
| ![Routes in the city](docs/images/city-routes.png) | **City**: folders become districts and blocks, files become buildings (footprint from lines, height from complexity, lit windows from definitions). Select a file to see its imports as cyan routes and its dependents as pulsing magenta routes along the streets. |
| ![Walking the city](docs/images/city-walk.png) | **Walk and fly** at street level with signs on every wall, folder street signs on the corners, source code on the facade in front of you, wandering residents and a city wall covered in posters. |
Also:
- **Search** every file and symbol, with results raised as light beacons in the city.
- **Private GitHub repos** via a GitHub App sign-in, scoped to the visitor's own access.
- **Security alerts**: with the right GitHub permissions, files with open code scanning or
Dependabot alerts smoke, burn (critical/high) and get wrapped in hazard tape.
- **Multi-user**: every browser gets its own private sessions, snapshots and sign-in.
- **Guided tour** of the biggest districts, and **shareable links** to an exact spot.
- **Touch controls**: joystick, look and pinch.
## Quick start
```bash
go run . # http://127.0.0.1:4177
go test -race ./...
node --test webtest/*.test.mjs # frontend maths: camera, city layout, collision, signs, links
go test -run x -bench Index # indexes $CODENAV_BENCH_DIR (default: Go's stdlib source)
./build.sh [tag] # Docker image, tests run during the build
go run . # http://127.0.0.1:4177
```
Open the page, click **Open a codebase**, and paste a public GitHub URL such as
`https://github.com/prometheus/prometheus`. `git` must be on your `PATH` for cloning.
With Docker:
```bash
./build.sh # builds codenavigator:latest, running the tests inside
docker run --rm -p 4177:4177 codenavigator:latest
```
Configuration:
## Configuration
- `PORT` (default `4177`, or `--port`), `CORS_ORIGIN` (default `*`)
- `GITHUB_CLIENT_ID`, `GITHUB_CLIENT_SECRET`: a GitHub App's credentials; enables sign-in for private repositories
- `GITHUB_APP_SLUG`: the app's URL name, for the "grant repository access" link
| Variable | Default | Purpose |
|---|---|---|
| `PORT` | `4177` | Listening port (`--port` overrides) |
| `CORS_ORIGIN` | `*` | `Access-Control-Allow-Origin` for the API |
| `SHOW_LOCAL` | unset | `true` enables uploading local folders; otherwise only GitHub repositories |
| `GITHUB_CLIENT_ID` | unset | GitHub App client ID; enables sign-in for private repos and security alerts |
| `GITHUB_CLIENT_SECRET` | unset | GitHub App client secret |
| `GITHUB_APP_SLUG` | unset | The app's URL name, for the "grant repository access" link |
`git` must be on `PATH` to load GitHub repositories (the image includes it).
## Keyboard
## Views
| Key | Where | Action |
|---|---|---|
| `Ctrl`+`O` | anywhere | Open a codebase |
| `/` | anywhere | Search |
| `F` | map views | Fit everything in view |
| `[` | anywhere | Hide or show the sidebar |
| `1` `2` `3` | city | Helicopter, walk, fly |
| `W` `A` `S` `D`, `Shift` | walk/fly | Move, run |
| `Space` / `C` | fly | Up / down |
| `E` | walk/fly | Inspect the building in the crosshair |
| `T` | city | Guided tour |
- **2D map**: squarified treemap of files, sized by lines or references.
- **3D landscape**: the same map extruded, orbit with drag, pan with Shift+drag, scroll to dolly.
- **City**: folders become districts and blocks separated by roads, files become buildings
(footprint from lines, height from complexity, lit windows from definitions).
- `1` Helicopter, `2` Walk, `3` Fly. In walk/fly: click for mouse-look, WASD, Shift to run,
Space/C up and down when flying, `E` inspects the building in the crosshair.
- Double-click a street to drop in; the minimap shows your heading and teleports on click.
- Selecting a file draws import trails (cyan known, amber inferred); search hits raise beacons.
- `T` or **Tour** flies over the largest districts. **Copy link** shares the repository, view,
camera and selected file as a URL fragment (GitHub repositories only).
- Touch: left thumb joystick, right thumb look, pinch to zoom.
- Selecting a file also lays glowing routes along the streets: cyan/amber chevrons out to each
file it imports, and pulsing magenta dashes in from each file that imports it, in separate lanes.
- Residents wander the streets (toggle in Settings). A wall with posters encloses the city.
- Signed in with GitHub, files with open code scanning or Dependabot alerts smoke (and burn
for critical/high) and are wrapped in hazard tape. The GitHub App needs read access to
code scanning alerts and Dependabot alerts.
## Documentation
Frontend modules (plain ES modules, no build step): `camera.js` (matrices and orbit pose),
`city.js` (layout, collision, picking, sign placement, tour and links), `labels.js` (text atlas),
`renderer.js` (WebGL2 passes) and `app.js` (UI and input).
## Multiple users
Every browser gets an HttpOnly session cookie. Jobs, snapshots and GitHub sign-ins belong to
that session and are invisible (404) to others. Each session keeps its latest snapshot; snapshots
idle for an hour are deleted, at most 8 are held (least recently used evicted), and two indexes
run at once while the rest queue. State is in memory, so a restart signs everyone out.
## Private GitHub repositories
Uses a GitHub App's user authorisation. When an anonymous clone fails, the frontend offers
**Sign in with GitHub**; the callback (`/api/github/callback`) stores an 8-hour user token in
memory for the session. Clones send it as an HTTP header via environment git config, never in
the URL or arguments, and host git config and credential helpers are ignored. A signed-in user
can only open repositories where the app is installed and they have access, otherwise the
frontend links to the app's installation page.
GitHub App settings: callback URL `https://<host>/api/github/callback`, setup URL `https://<host>/`
with "Redirect on update", token expiry on, no webhook, repository permission Contents: read-only.
The browser frontend in `web/` is plain JavaScript (it has to run in the browser) and
is embedded into the binary; `web/config.js` points it at the same origin.
Notes:
- `GET /` serves the frontend instead of a JSON service description.
- Jobs and snapshots are private to the browser session that created them (see above).
- Search queries are percent-decoded properly.
- `.gitignore` support covers standard globs but not global excludes or `.git/info/exclude`.
- [City guide](docs/CITY.md): what everything in the city means and how to get around
- [Architecture](docs/ARCHITECTURE.md): indexer, server, sessions, rendering pipeline
- [HTTP API](docs/API.md): every endpoint, with request and response shapes
- [GitHub App setup](docs/GITHUB_APP.md): private repos and security alerts
- [Deployment](docs/DEPLOYMENT.md): Docker, systemd, nginx, operating limits
- [Development](docs/DEVELOPMENT.md): running, testing, benchmarking, project layout
docs/API.md
+156
View File
@@ -0,0 +1,156 @@
# HTTP API
All endpoints are under `/api/` and return JSON unless noted. Everything else is the static frontend.
Every response carries `Cache-Control: no-store`, `X-Content-Type-Options: nosniff` and CORS
headers (`Access-Control-Allow-Origin` from `CORS_ORIGIN`). `OPTIONS` on any path returns `204`.
Errors look like `{"error": "message"}`.
## Sessions and ownership
The first `POST /api/jobs` (or GitHub sign-in) sets an `HttpOnly`, `SameSite=Lax` cookie called
`codenav_session` (`Secure` when served over HTTPS or behind a proxy sending
`X-Forwarded-Proto: https`). Jobs, snapshots and GitHub sign-ins belong to that session. Any other
session gets `404` for them, exactly as if they didn't exist. IDs are random (`crypto/rand`).
## Health
### `GET /api/health`
```json
{ "ok": true, "version": "0.1.0", "localFolders": false }
```
`localFolders` is `true` only when the server runs with `SHOW_LOCAL=true`.
## Jobs (indexing)
### `POST /api/jobs`
```json
{ "kind": "github", "url": "https://github.com/owner/repo", "name": "repo" }
{ "kind": "local", "name": "my-folder" }
```
| Status | Meaning |
|---|---|
| `201` | `{"jobId": "job-…"}`; GitHub jobs start cloning immediately |
| `400` | Invalid body, unknown kind, missing or invalid GitHub URL (`https://github.com/owner/repo` only) |
| `403` | `kind: local` while local folders are disabled |
GitHub clones are shallow (`--depth=1`), time out after 10 minutes, ignore the host's git config and
credential helpers, and authenticate with the visitor's GitHub token when they're signed in.
### `POST /api/jobs/{id}/files?path=relative/path`
Local jobs only. The request body is the raw file content (at most 64 MB). `path` must be relative
with no `..` segments. Returns `{"ok": true}`, `400` for unsafe paths, `404` for unknown or non-local jobs.
### `POST /api/jobs/{id}/commit`
Local jobs only. Starts indexing the uploaded files. Returns `202 {"ok": true}`.
### `GET /api/jobs/{id}`
```json
{
"id": "job-…", "sourceKind": "github", "name": "prometheus",
"phase": "parsing", "completed": 420, "total": 1676, "message": "Parsing 421 of 1676",
"snapshotId": null, "error": null, "warnings": [],
"authRequired": "signin"
}
```
`phase` is `receiving`, `cloning`, `queued`, `scanning`, `parsing`, `linking`, `ready` or `error`.
`authRequired` only appears on failed clones that signing in (`signin`) or granting the GitHub App
access (`install`) could fix.
### `GET /api/jobs/{id}/events`
Server-sent events. Each change to the job is sent as:
```
event: progress
data: {…same shape as GET /api/jobs/{id}…}
```
A `: keep-alive` comment is sent every 15 seconds. The stream ends when the job has a `snapshotId`
or an `error`, or when the client disconnects. At most two jobs index at once; others report
`queued`.
## Snapshots
### `GET /api/snapshots/{id}`
```json
{ "id": "snapshot-…", "name": "prometheus", "source": "github", "fileCount": 1676,
"totalLines": 568224, "definitions": 13717, "references": 790, "edges": 790, "warnings": [] }
```
### `GET /api/snapshots/{id}/scene`
Everything the frontend needs to lay out the map and city:
```json
{
"id": "snapshot-…", "name": "prometheus", "source": "github",
"files": [{ "id": 1, "path": "cmd/prometheus/main.go", "name": "main.go", "directory": "cmd/prometheus",
"extension": "go", "language": "Go", "layer": "application", "lines": 2296,
"bytes": 91234, "complexity": 90, "preview": "// first 60 lines…", "symbolCount": 55 }],
"edges": [{ "from": 1, "to": 2, "kind": "import", "confidence": "known" }],
"totalLines": 568224, "definitions": 13717, "references": 790
}
```
After a scene is delivered the server drops its copy of the previews. `layer` is one of
`application`, `library`, `test`, `generated`, `vendor`, `platform`, `docs`. `confidence` is `known`
(resolved path) or `inferred` (matched by module name).
### `GET /api/snapshots/{id}/search?q=text`
Case-insensitive substring match on file paths and symbol names (at least 2 characters, at most 180 hits):
```json
[{ "entityId": 1, "path": "cmd/prometheus/main.go", "name": "main", "line": 120, "kind": "definition", "preview": "func main() {" }]
```
### `GET /api/snapshots/{id}/entities/{fileId}`
The full file record, including `symbols: [{name, kind, line, signature}]`.
### `GET /api/snapshots/{id}/entities/{fileId}/source?start=0&limit=600`
```json
{ "start": 0, "lines": ["package main", "…"], "totalLines": 2296 }
```
`limit` is clamped to 1–4096. Once every line of a file has been delivered the server deletes it
from disk, and the workspace is removed once all files are delivered.
### `GET /api/snapshots/{id}/alerts`
Open security alerts for a GitHub snapshot, fetched with the viewer's own token:
```json
{
"available": true,
"codeScanning": { "status": "ok", "alerts": [
{ "path": "web/app.js", "title": "Cross-site scripting", "severity": "critical", "url": "https://github.com/…", "line": 7 } ] },
"dependabot": { "status": "forbidden", "alerts": [] }
}
```
When alerts can't be fetched at all: `{"available": false, "reason": "not-github" | "signin"}`.
Per-set `status` is `ok`, `forbidden` (permission not granted or feature disabled) or
`unavailable`. Severities are normalised to `critical`, `high`, `medium` or `low`. Dependabot alerts
point at the manifest that declares the vulnerable package.
## GitHub sign-in
| Endpoint | Purpose |
|---|---|
| `GET /api/github/status` | `{"configured", "connected", "login", "installUrl"}` for the current session |
| `GET /api/github/login` | Redirects to GitHub with a single-use `state` bound to the session (10 minutes) |
| `GET /api/github/callback` | Exchanges the code, stores the user token in memory, redirects to `/?github=connected` (or `denied`/`error`) |
| `POST /api/github/logout` | Forgets the session's token |
Login, callback and logout return `404` when no GitHub App is configured.
docs/ARCHITECTURE.md
+92
View File
@@ -0,0 +1,92 @@
# Architecture
```
browser ──HTTP/SSE──▶ Go server (main.go, github.go) ──▶ indexer (indexer.go) ──▶ temp workspace
│ │
│ └── GitHub (OAuth token exchange, alerts API, git clone)
└── WebGL2 frontend (web/): app.js · renderer.js · city.js · camera.js · labels.js
```
## Server
One Go binary using only the standard library. The frontend is embedded with `go:embed` and served
from the same origin as the API.
### Indexing (`indexer.go`)
1. **Walk** the checkout in lexical order. Skip `.git`, `node_modules`, `target`, `dist`, `build`
and `.next`, honour `.ignore` files everywhere and `.gitignore` inside git checkouts. Only regular
files count (no symlinks).
2. **Parse** files in parallel (one worker per CPU). Binary files (a NUL byte in the first 4 KB) and
files over 8 MB are skipped. For each file it records lines, bytes, a complexity score (branching
tokens), the first 60 lines as a preview, definitions (class/struct/function patterns) and import
targets (from/import/require/#include/use/using patterns). Each regex has a cheap literal
prefilter, which made indexing about 3× faster than regex alone.
3. **Link** imports to files: relative paths first (`known`), then module-style names resolved by
path suffix or file stem (`inferred`). Ambiguous names are never guessed.
4. Classify each file's **language** (by extension) and **layer** (test, vendor, generated, platform,
library, docs or application) by path.
### Sessions, jobs and snapshots (`main.go`)
- **Sessions:** a random cookie identifies each browser, and jobs, snapshots and GitHub tokens are
owned by it.
- **Jobs:** a job is a GitHub clone or a local upload, reporting progress over SSE.
- **Snapshots:** each session keeps only its latest snapshot. Snapshots idle for an hour are expired,
at most 8 are held (least recently used evicted), and at most 2 indexes run at once.
- **Delivery:** once the scene is sent, previews are released, and each source file is deleted after
every line of it has been fetched.
- **Workspace cleanup:** only paths inside the workspace directory are ever deleted.
### GitHub (`github.go`)
- **Sign-in:** the GitHub App user-authorisation flow. A single-use `state` is bound to the
session, and the resulting user token is held in memory only (8-hour expiry).
- **Clones** pass the token as an HTTP header through `GIT_CONFIG_*` environment variables, never in
arguments or URLs, with host git config and credential helpers disabled.
- **Security alerts** are fetched per request with the viewer's token, following pagination only on
the API host.
## Frontend (`web/`)
Plain ES modules, no framework, no build step.
| Module | Responsibility |
|---|---|
| `app.js` | UI, input, data loading, 2D overlay canvas, city orchestration (signs, routes, residents, tour, links, alerts) |
| `renderer.js` | All WebGL2 drawing: instanced boxes, sign quads, route ribbons, particles, sprites |
| `camera.js` | Matrix maths: perspective, lookAt, inverse, orbit pose |
| `city.js` | Pure city logic, unit tested: layout, collision, navigation grid and routing, sign placement, tape, walls, posters, residents, place links |
| `labels.js` | Text atlas for signs (shelf-packed canvas) and the poster atlas |
| `api.mjs` | `fetch` wrapper and SSE-over-fetch parser |
### Views
- **2D:** squarified treemap in a 1000×680 world, orthographic. Source text is painted into an
overview canvas when zoomed out, and into per-tile textures (LRU, 192 MB budget) when zoomed in.
- **3D:** the same instances extruded, rendered through a perspective orbit camera. Roofs sample the
overview canvas as a GPU texture, and only nearby roofs get sharp per-file textures.
- **City:** `layoutCity` lays the directory tree out in metres. Folders shrink by half a road on
each side (18 m avenues between districts, narrower streets deeper down). Files become buildings,
with footprint ≈ 1.1 m² per line and height 6 m + 10 m × log₂(1 + complexity).
### City rendering passes (in order)
1. **Floor:** ground and folder pavements in nesting order, depth test off, top faces only (so
nested pavements a few centimetres apart can't z-fight).
2. **Buildings and wall:** instanced boxes. The city shader adds procedural windows (lit share from
definition count), neon edges, a concrete wall and exponential fog.
3. **Effects:** additive and never occluding. Search beacons, fire and smoke particles (stateless,
animated in the vertex shader), residents (procedural stick-figure sprites), street routes
(mitred ribbons with chevrons or pulsing dashes, kept in right-hand lanes) and import arcs.
4. **Signs:** textured quads from the label atlas, depth-tested but not depth-writing. A shader test
discards any quad face that would read mirrored, which is how two-faced blade signs work.
### Routing
- **Grid:** a 3 m occupancy grid blocks buildings (grown by 0.5 m) and the city wall, and makes
pavements slightly dearer than roads.
- **Search:** one Dijkstra run from the selected building's doors reaches every import and
dependent (paths cost the same both ways). It uses float64 distances and a settled set.
- **Shaping:** routes are straightened by line-of-sight, extended into both buildings, and boxed-in
targets fall back to the nearest reachable street.
docs/CITY.md
+66
View File
@@ -0,0 +1,66 @@
# City guide
Switch to **City** in the toolbar. The city is laid out from the repository's folder tree at human
scale, so a large repository is a large city.
## Reading the city
| You see | It means |
|---|---|
| A **district** (big block, name floating above its tallest tower) | A top-level folder |
| A **block** inside it, separated by streets | A nested folder; deeper folders have narrower streets |
| A **building** | A file |
| Building **footprint** | Lines of code |
| Building **height** | Complexity (branching) |
| **Lit windows** | Definitions: more functions and classes, more lights on |
| Building **colour** | Semantic layer (application, tests, vendor, …), switchable palette |
| Name on the **roof** and **wall** | The file name |
| **Green street sign** sticking out at a corner | The folder that block belongs to; each corner of an intersection names its own folder |
| **Smoke** | The file has open medium/low security alerts |
| **Fire and smoke** | The file has an open critical/high security alert |
| **Yellow tape** rising across the walls | Dependabot alert (on the manifest file) |
| **Red tape** falling across the walls | Code scanning alert |
| **Light beam** into the sky | A search result |
| **Cyan chevrons** on the road | Route to a file the selected file imports (**amber** if the import was inferred) |
| **Pulsing magenta dashes** on the road | Route from a file that imports the selected file |
| **Arcs** in the sky | The same import relationships, as the crow flies |
| **Green figures** | Residents. They don't mean anything; they just live there |
| **Wall with posters** | The city limits, with a few facts about the codebase pasted on |
## Getting around
**Helicopter (`1`)**: drag to orbit, Shift+drag (or right-drag) to pan, scroll to zoom.
Double-click a building to fly to it, or double-click a street to drop down and walk.
**Walk (`2`)**: click the map to capture the mouse and look around, `WASD` to move, `Shift` to run,
`Esc` to release the mouse. The crosshair names the building ahead. Press `E` (or click) to open
it in the sidebar. Walk right up to a building and its source code appears on the wall. You can't
walk through buildings or out past the city wall.
**Fly (`3`)**: like walking, but `Space` and `C` climb and descend, and moving follows where you
look.
**Touch**: left thumb is a joystick, right thumb looks around, pinch zooms (or climbs when flying).
**Minimap**: click it to jump there. It shows your heading, the city wall, security alerts and
the selected file's routes.
## Following dependencies
Select a file (click it, search for it, or pick it from the sidebar). Routes appear on the streets
to everything it imports and from everything that imports it. With a file selected, switching to
**Walk** puts you at its door facing along the first route: follow the arrows.
## Tour and links
- **Tour (`T`)**: flies over the six largest districts with captions, and stops as soon as you
touch anything.
- **Copy link**: copies a URL with the repository, view, camera position and selected file.
Opening it loads the repository and puts you in the same spot. Links work for GitHub
repositories only.
## Settings
The gear opens settings: landscape palette, source text on tiles, **city residents** (they keep
the city animating, so switch them off on slow machines) and a frame-rate monitor. The sidebar
folds away with the tab on its left edge, or `[`.
docs/DEPLOYMENT.md
+114
View File
@@ -0,0 +1,114 @@
# Deployment
## Docker
```bash
./build.sh [tag] # default codenavigator:latest; runs go test inside the build
docker run -d --name codenavigator -p 127.0.0.1:4177:4177 \
--env-file /etc/codenavigator/app.env \
--memory 2g --cpus 4 --pids-limit 512 \
codenavigator:latest
```
The image is a static binary on Alpine with `git` and CA certificates, running as a non-root user,
with a `HEALTHCHECK` on `/api/health`. Workspaces live in the container's `/tmp` and disappear with
it.
`app.env` (keep it `chmod 600`):
```bash
GITHUB_CLIENT_ID=
GITHUB_CLIENT_SECRET=
GITHUB_APP_SLUG=
# SHOW_LOCAL=true # only if visitors should be able to upload local folders
```
## systemd
`/etc/systemd/system/codenavigator.service`:
```ini
[Unit]
Description=CodeNavigator (docker container)
After=docker.service network-online.target
Requires=docker.service
Wants=network-online.target
[Service]
Restart=always
RestartSec=5
TimeoutStartSec=0
ExecStartPre=-/usr/bin/docker rm -f codenavigator
ExecStart=/usr/bin/docker run --rm --name codenavigator \
--env-file /etc/codenavigator/app.env \
-p 127.0.0.1:4177:4177 \
--memory 2g --cpus 4 --pids-limit 512 \
codenavigator:latest
ExecStop=/usr/bin/docker stop -t 30 codenavigator
[Install]
WantedBy=multi-user.target
```
```bash
sudo systemctl daemon-reload && sudo systemctl enable --now codenavigator
```
Redeploy: `./build.sh && sudo systemctl restart codenavigator`.
## nginx
```nginx
server {
listen 80;
server_name codenav.example.com;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl http2;
server_name codenav.example.com;
ssl_certificate /etc/letsencrypt/live/codenav.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/codenav.example.com/privkey.pem;
# Local folder uploads send one file per request; the frontend skips files over 8 MB.
client_max_body_size 10m;
location / {
proxy_pass http://127.0.0.1:4177;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Indexing progress is a server-sent event stream: don't buffer it.
proxy_buffering off;
proxy_read_timeout 300s;
}
}
```
`X-Forwarded-Proto: https` makes the session cookie `Secure`. Certificates: `certbot certonly --nginx -d codenav.example.com`.
## Operating limits
| Limit | Value |
|---|---|
| Concurrent indexing | 2 jobs; others queue |
| Snapshots held | 8 in total, 1 per session, least recently used evicted |
| Idle expiry | 1 hour for snapshots, finished jobs and abandoned uploads |
| Clone timeout | 10 minutes |
| File size | files over 8 MB are skipped; uploads capped at 64 MB |
| GitHub tokens | in memory, 8 hours |
All state is in memory. A restart clears snapshots and signs everyone out.
## Security notes
- **Local folders** are off unless `SHOW_LOCAL=true`. With them on, anyone who can reach the site
can upload files to the server's temporary workspace.
- **Anyone who can reach the site can clone public repositories** through it. Put it behind
authentication if that matters to you.
- **Visitors' GitHub tokens** never reach the browser, git arguments or logs. Host git config and
credential helpers are ignored, so the server's own credentials can't be used by a visitor.
docs/DEVELOPMENT.md
+75
View File
@@ -0,0 +1,75 @@
# Development
Requirements: Go 1.26+, git, and Node 20+ (only to run the frontend tests).
```bash
go run . # http://127.0.0.1:4177, frontend served from web/ via go:embed
SHOW_LOCAL=true go run . # also allow local folder uploads
```
The frontend is embedded at build time, so **restart `go run` after changing files in `web/`**.
## Tests
```bash
go test -race ./... # server, indexer, GitHub flows (fake GitHub)
node --test webtest/*.test.mjs # frontend maths: camera, city, routing, signs, posters
go test -run x -bench Index # indexing benchmark on $CODENAV_BENCH_DIR (default: Go's stdlib)
```
What the suites cover:
- **Go:**
- Indexing: ignore rules, parsing and linking.
- Jobs: the local job lifecycle end to end (upload → SSE → scene → source release), session
isolation, expiry and eviction, the concurrency queue, SSE keep-alives, and the `SHOW_LOCAL` gate.
- GitHub: the sign-in flow including forged and reused state, clone token handling and timeouts,
security alert pagination and permissions.
- **Node:**
- Camera: parity with the original landscape projection.
- City: layout invariants (no overlaps, avenue widths), collision and movement, routing (detours,
enclosed buildings, a large-city time budget), ribbon lanes and mitred corners.
- Signs and walls: sign readability and placement, hazard tape, walls and posters, text fitting.
- Other: residents staying walkable, and place links rejecting untrusted input.
## Project layout
```
main.go HTTP server, sessions, jobs, snapshots, expiry
github.go GitHub App sign-in, clone command, security alerts
indexer.go Walking, parsing, linking, classification
*_test.go Go tests
web/
index.html Markup
styles.css All styling (see the design notes below)
app.js UI, input, overlays, city orchestration
renderer.js WebGL2 passes and shaders
city.js Pure city logic (layout, routing, signs, residents…)
camera.js Matrix maths
labels.js Sign atlas, poster atlas, text fitting
api.mjs fetch + SSE helpers
fonts/ Self-hosted Archivo Black, Atkinson Hyperlegible, Space Mono
webtest/ node:test suites for web/ modules
docs/ This documentation and screenshots
Dockerfile, build.sh
```
## Conventions
- **No dependencies:** the Go side uses only the standard library, and the frontend is plain ES
modules with no bundler.
- **Keep logic testable:** anything that can be pure maths or data goes in `city.js`, `camera.js`
or `labels.js`, with a `node:test` test. `app.js` wires it to the DOM and `renderer.js` draws it.
- **Commits** are small and describe why, not just what.
## Design notes
The interface is deliberately hand-made rather than polished:
- **Colour:** flat warm ink surfaces, hard 2px black edges, offset shadows instead of blur, and a
clashing lime/pink/sky/tangerine palette.
- **Type:** chunky Archivo Black headings, Atkinson Hyperlegible body text and Space Mono labels.
- **Personality:** small tilts, stickers and wiggles give it some energy. Everything animated
respects `prefers-reduced-motion`.
- **Tokens:** colours and fonts are CSS variables at the top of `styles.css`, and the city's
palette lives in the shaders in `renderer.js`.
docs/GITHUB_APP.md
+65
View File
@@ -0,0 +1,65 @@
# GitHub App setup
Public repositories work without any setup. A GitHub App adds two things:
- **Private repositories**: visitors sign in and can open repositories the app is installed on and
they have access to.
- **Security alerts**: files with open code scanning or Dependabot alerts burn in the city.
CodeNavigator uses the app's *user authorisation* only: no private key, no webhooks, and tokens act
with the visitor's own permissions.
## 1. Register the app
Go to **https://github.com/settings/apps/new** (or an organisation's Developer settings) and fill in:
| Field | Value |
|---|---|
| GitHub App name | anything; it becomes the slug in `github.com/apps/<slug>` |
| Homepage URL | `https://your-host` |
| Callback URL | `https://your-host/api/github/callback` |
| Expire user authorization tokens | ✅ |
| Request user authorization (OAuth) during installation | ☐ off |
| Enable Device Flow | ☐ off |
| Setup URL | `https://your-host/`, with **Redirect on update** ✅ |
| Webhook → Active | ☐ off |
| Where can this GitHub App be installed? | **Any account** to let others use it |
**Repository permissions** (everything else "No access"):
| Permission | Access | Needed for |
|---|---|---|
| Contents | Read-only | Cloning private repositories |
| Metadata | Read-only | Added automatically |
| Code scanning alerts | Read-only | Red tape and fire |
| Dependabot alerts | Read-only | Yellow tape and fire |
Create the app, copy the **Client ID**, and click **Generate a new client secret**.
## 2. Configure the server
```bash
GITHUB_CLIENT_ID=Iv23li...
GITHUB_CLIENT_SECRET=...
GITHUB_APP_SLUG=your-app-slug
```
Restart the server. `GET /api/github/status` should now report `"configured": true`.
## 3. Install it
Visit `https://github.com/apps/<slug>`, click **Install**, and choose the repositories it may read.
Each user (or organisation admin) installs it for their own repositories. When a signed-in visitor
opens a repository the app can't see, CodeNavigator offers a **Grant repository access** button
linking to that page.
## How sign-in behaves
- A private repo that fails to clone shows **Sign in with GitHub**. After approving on GitHub,
visitors land back on the site with the Open codebase dialog already showing the result and the
repository filled in.
- Tokens live only in server memory for 8 hours and are never sent to the browser. Restarting the
server signs everyone out.
- **Adding permissions later** (e.g. alerts): existing installations must accept the new permissions
on GitHub, and visitors need to sign out and back in to get a token with them. Until then the
sidebar reports which alert kinds it has no access to.
docs/images/city-routes.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 426 KiB

docs/images/city-walk.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 393 KiB

docs/images/city.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 757 KiB

docs/images/landscape-3d.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 530 KiB

docs/images/map-2d.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 334 KiB

docs/images/welcome.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 112 KiB