Files
website/README.md
wtclaude 9b16f39a52
Some checks failed
PR Checks / bot-install (pull_request) Successful in 23s
PR Checks / client-build (pull_request) Successful in 30s
PR Checks / server-tests (pull_request) Failing after 4m23s
feat(modules): the declarative Docker path (phase 4, slice 3)
MODULES declares the module set a deployment runs, one entry per module as
`<id>@<version>=<install manifest URL>`, and the container arrives at it by
itself (MODULE_SYSTEM.md §2.7.2 decision 4). A module already unpacked at the
declared version is a no-op that makes NO network call, so a restart with the
network down comes up unchanged; anything else goes through install.js — same
allowlist, same sha256, same inspect-then-extract — and install() now takes an
`expect: {id, version}` so a URL resolving to another module or version is
refused while it is still only a manifest.

Resolution runs inside start(), between the seed and the require of app.js: the
seed is where the host allowlist setting comes from, and the require is what
scans the volume. That buys it the database, so a compose-installed module gets
the same provenance columns an admin install writes.

A failure is logged and carried, never fatal — an unreachable release host must
not take the site down. The declaration owns what is on the volume; the row owns
whether a module runs, so uninstalling a declared module returns its files at
the next start and leaves it disabled. The admin list gains that as a fourth
source (declared / declaredVersion / declaredError), because a declared module
that failed to resolve has no row, no directory and nothing mounted.

Deferring the app require moved core's schema ahead of the volume scan, and the
module schema-fragment replay was wired to core's schema — so every installed
module silently got no tables. Invisible to the suite (each one stubs the loader
or the pool) and to a smoke on a database that already had the tables; found by
booting against an empty one. ensureSchema() now takes `replayModules: false`
for the one caller that scans later, server.js replays them itself after the
require, and a bootOrder test pins the five steps in the only order they work in.

741 server tests (+18), 187 client (+5); manifest unchanged at 166 public + 2
internal, OpenAPI byte-identical.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-12 07:57:08 -05:00

758 lines
42 KiB
Markdown

# Runic Gateway Website
[![Bugs](https://sonar.whitlocktech.com/api/project_badges/measure?project=runic-gateway-website&metric=bugs&token=sqb_d3593f26ac5663cd3e666039b7038f3248e8df50)](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
[![Code Smells](https://sonar.whitlocktech.com/api/project_badges/measure?project=runic-gateway-website&metric=code_smells&token=sqb_d3593f26ac5663cd3e666039b7038f3248e8df50)](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
[![Duplicated Lines (%)](https://sonar.whitlocktech.com/api/project_badges/measure?project=runic-gateway-website&metric=duplicated_lines_density&token=sqb_d3593f26ac5663cd3e666039b7038f3248e8df50)](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
[![Lines of Code](https://sonar.whitlocktech.com/api/project_badges/measure?project=runic-gateway-website&metric=ncloc&token=sqb_d3593f26ac5663cd3e666039b7038f3248e8df50)](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
[![Security Hotspots](https://sonar.whitlocktech.com/api/project_badges/measure?project=runic-gateway-website&metric=security_hotspots&token=sqb_d3593f26ac5663cd3e666039b7038f3248e8df50)](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
[![Security Rating](https://sonar.whitlocktech.com/api/project_badges/measure?project=runic-gateway-website&metric=security_rating&token=sqb_d3593f26ac5663cd3e666039b7038f3248e8df50)](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
[![Vulnerabilities](https://sonar.whitlocktech.com/api/project_badges/measure?project=runic-gateway-website&metric=vulnerabilities&token=sqb_d3593f26ac5663cd3e666039b7038f3248e8df50)](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
Public site, wiki, and protected admin panel for a game community — a full-stack app
in one repo. Everything specific to a *particular* game lives in an installable
module, not here. Branding is instance-configurable via `BRAND_*` (see
[Branding](#branding)); **UOMysticmoon**, an Ultima Online shard, is the first
instance, and its game half is
[RunicGateway/Module-uo](https://gitea.whitlocktech.com/RunicGateway/Module-uo).
A full-stack app in one repo:
- **Backend** — Node.js + Express REST API (layered `router → controller → model → db`), MariaDB, a provider-agnostic session layer (JWT cookie for web, bearer tokens for mobile, pluggable SSO).
- **Frontend** — React + Vite single-page app (public site, wiki, and the admin panel), dark "gothic" theme (Cinzel + Georgia).
- **Deploy** — Docker Compose (app + MariaDB) behind a reverse proxy (Pangolin, Nginx, Caddy, Traefik, …). Express serves the built SPA in production.
- **Modules** — the game-specific half of a site is a module dropped onto a volume: it adds routes, database tables, nav entries and whole SPA pages without this repo knowing anything about the game. See [Modules](#modules).
The design reference is [BACKEND_DESIGN.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/BACKEND_DESIGN.md) (API contract, schema, security), in the [**RunicGateway/docs**](https://gitea.whitlocktech.com/RunicGateway/docs) repo — where all project documentation now lives.
---
## Contents
- [Architecture](#architecture)
- [Tech stack](#tech-stack)
- [Project structure](#project-structure)
- [Prerequisites](#prerequisites)
- [Setup & run](#setup--run)
- [Option A — Docker Compose (full stack)](#option-a--docker-compose-full-stack)
- [Option B — Local development (hot reload)](#option-b--local-development-hot-reload)
- [Option C — Production build without Docker](#option-c--production-build-without-docker)
- [First admin & site mode](#first-admin--site-mode)
- [Pages & routes](#pages--routes)
- [API endpoints](#api-endpoints)
- [API documentation (Swagger)](#api-documentation-swagger)
- [Modules](#modules)
- [Environment variables](#environment-variables)
- [Security](#security)
- [Logging](#logging)
- [Deployment behind a reverse proxy](#deployment-behind-a-reverse-proxy)
---
## Architecture
How the pieces fit together — the React SPA and native app talk to one Express backend
(`router → controller → model → db`), which persists to MariaDB. Anything that knows
what game this site is about lives in an installed module, on the right of the diagram.
```mermaid
flowchart TB
%% ---------- Clients ----------
subgraph clients["Clients"]
browser["Browser<br/>React + Vite SPA<br/>(public · wiki · admin)"]
mobile["Native mobile app<br/>(bearer tokens)"]
end
idp["SSO providers<br/>Google · Discord · custom OIDC"]
discord["Discord"]
%% ---------- Website (one repo) ----------
subgraph website["website/ &nbsp;— Node app (one repo)"]
direction TB
subgraph backend["server/ — Express backend"]
direction TB
mw["Middleware<br/>helmet · siteMode · noindex<br/>rateLimit · loginProtection · botScore · validate"]
router["Router /api/v1<br/>auth (web · mobile · sso) · public · admin · player"]
ctrl["Controllers"]
auth["Session layer (auth/)<br/>sessionService · JWT/cookie · bearer · SSO+PKCE"]
model["Models (.model + .db)<br/>raw parameterized SQL — no ORM"]
sse["SSE fan-out<br/>public stream (allowlist) · admin stream (sensitive)"]
loader["modules/loader.js<br/>scans the volume · mounts · registries · lifecycle"]
secret["secretBox.js<br/>AES-256-GCM secrets at rest"]
end
bot["bot/<br/>Discord bot"]
end
db[("MariaDB<br/>users · posts · wiki · settings · activity<br/>mobileSessions · authProviders · userIdentities<br/>installed_modules · &lt;module&gt;_*")]
%% ---------- Module side ----------
subgraph modside["modules/&lt;id&gt;/ &nbsp;— installed, not built (e.g. Module-uo)"]
direction TB
modsrv["server/ — routers, models, schema fragment<br/>reaches core only through ctx"]
modcli["client/dist/entry.js — prebuilt ESM chunk<br/>React shared via window.__rg"]
end
game["The game<br/>whatever the module talks to<br/>(for Module-uo: a ServUO shard,<br/>via the uo-link sidecar)"]
%% ---------- Edges ----------
browser <-->|"same-origin JSON + SSE (cookie)"| mw
mobile -->|"REST (bearer access/refresh)"| mw
browser -.->|"OAuth redirect + PKCE"| idp
auth -.->|"token exchange"| idp
mw --> router --> ctrl
ctrl --> auth
ctrl --> model
ctrl --> sse
auth --> model
model <--> db
auth -. reads/writes secrets .-> secret
sse -->|"live events"| browser
bot -->|"messages"| discord
bot <--> db
loader -->|"mounts under /api/v1/&lt;tier&gt;/&lt;prefix&gt;"| router
loader -->|"require() + register(ctx, api)"| modsrv
modsrv -->|"ctx.db · ctx.push · ctx.activity …"| model
modsrv <--> game
browser -->|"&lt;script type=module&gt; injected by htmlShell"| modcli
%% ---------- Styling ----------
classDef ext fill:#2d2233,stroke:#7a5c94,color:#e8dff0;
classDef store fill:#1f2d2a,stroke:#4c8c7d,color:#dff0ea;
classDef mod fill:#2d2620,stroke:#94764c,color:#f0e6d8;
class idp,discord,game ext;
class db store;
class modsrv,modcli mod;
```
- **One backend, layered.** Every request flows `middleware → router → controller → model → db`.
Web browsers authenticate with an httpOnly JWT cookie; the native app uses short-lived bearer
access tokens plus rotated refresh tokens; SSO (Google/Discord/OIDC) is link-only and PKCE-guarded.
All three surfaces produce the *same* session via the session layer.
- **Core knows nothing about any game.** Routes, tables, nav entries, SPA pages and push streams for
a specific game arrive from a module the operator installed. Core provides the seams; the module
fills them. See [Modules](#modules).
- **A module that fails must never take the site down.** The loader catches failures across a
module's whole lifecycle and marks that one module `startup_failed`; the site comes up with its
routes and nav absent, and the admin panel says why.
- **Sensitive events stay private.** Events fan out to browsers over two SSE channels — a public
allowlist stream and an admin-only stream that adds staff audit / cheat / login events. Which
event kinds are public is decided by the module that publishes them, and core enforces the split.
---
## Tech stack
| Layer | Tech |
|---|---|
| Backend | Node.js 20+, Express 4, `mariadb` driver (parameterized SQL, no ORM) |
| Auth | Session service over JWT: httpOnly cookie (web) + bearer access/refresh tokens (mobile), bcrypt hashing, optional TOTP 2FA (`speakeasy` + `qrcode`), pluggable OAuth2/OIDC SSO (built-in Google & Discord + generic) |
| Database | MariaDB 11 (own container) |
| Frontend | React 18, Vite 5, React Router 6 |
| Email | Nodemailer via Gmail OAuth2 (configured in admin), with a `mailto:` fallback |
| API docs | OpenAPI 3.0 via `swagger-autogen`, served with `swagger-ui-express` at `/api/docs` |
| Deploy | Docker Compose, any reverse proxy (Pangolin, Nginx, Caddy, Traefik, …) |
---
## Project structure
```
website/
├─ server/ Express API
│ ├─ src/
│ │ ├─ server.js bootstrap: ensure schema → seed → listen (0.0.0.0)
│ │ ├─ app.js middleware + static SPA + routes
│ │ ├─ auth/ session layer: session.service · token (JWT/cookies) · session.middleware · ssoState (PKCE/CSRF) · providers/ (base · oauth2 · google · discord · genericOidc · registry)
│ │ ├─ router/v1/ auth (web · mobile · sso) / public / admin / player route groups
│ │ ├─ model/ users · posts · wiki · settings · activity · mobileSessions · authProviders · userIdentities · modules (.model + .db)
│ │ ├─ modules/ loader (scan · validate · mount) · registries (the seams) · lifecycle (boot/shutdown + reconcile)
│ │ ├─ middleware/ siteMode · noindex · rateLimit · loginProtection · botScore · validate
│ │ └─ utils/ auth (compat facade) · totp (2FA) · secretBox (AES-GCM secrets) · db (pool) · mailer · logger · htmlShell
│ ├─ db/ schema.sql + seed.js
│ ├─ swagger/ swagger.js (generator config) · swagger-output.json (generated, core only) · docsSpec.js (merges module fragments at request time)
│ └─ .env.example
├─ client/ React + Vite SPA
│ ├─ src/
│ │ ├─ routes/public/ Portal, Website, News, Screenshots, FiveOnFriday, Newsletter(+Issue), Status, About, Maintenance
│ │ ├─ routes/wiki/ Wiki landing + WikiArticle
│ │ ├─ routes/admin/ AdminLogin (password + TOTP + SSO buttons), AdminLayout, views/ (Dashboard, Posts, Wiki, Settings, Activity, Bot Activity, Authentication, Users, Account) + editors
│ │ ├─ components/ SiteHeader, SiteFooter, layout, guards, Modal, ProviderIcon (inline SSO SVGs), …
│ │ ├─ contexts/ AuthContext, SiteContext
│ │ ├─ modules/ the client registry: routes · nav · slots · feature gates · window.__rg
│ │ ├─ api/client.js fetch wrapper (sends cookies)
│ │ └─ styles/theme.css design tokens
│ └─ public/assets/img/ hero image
├─ modules/ installed modules, one directory each — a Docker bind mount; empty here
├─ Dockerfile builds client → serves via Express
├─ docker-compose.yml app + MariaDB
├─ .env.example root env (used by Compose)
└─ package.json workspace scripts
```
---
## Prerequisites
- **Node.js 20+** and npm (Node 22/24 are fine).
- **Docker Desktop** (for MariaDB, and for the full Compose deploy).
---
## Setup & run
### Option A — Docker Compose (full stack)
`docker-compose.yml` is **production-shaped**: it *pulls* the prebuilt `app` and `bot` images from
the Gitea container registry (published by `.gitea/workflows/build-images.yml` on every merge to
`main`) — it never builds. Each image already bundles the server deps and the built React client,
which Express serves. MariaDB runs in its own container; tables + defaults + the first admin are
created automatically on first boot.
```bash
cp .env.example .env
# Edit .env and set at least:
# DB_PASSWORD, DB_ROOT_PASSWORD (any strong values)
# JWT_SECRET (a long random string)
# ADMIN_USERNAME, ADMIN_PASSWORD (your first admin login)
docker compose pull && docker compose up -d # IMAGE_TAG defaults to `latest`
# pin a specific build (reproducible deploy / rollback):
IMAGE_TAG=sha-042a151 docker compose pull && docker compose up -d
```
- App: **http://localhost:3000** (binds `0.0.0.0`)
- Health check: `GET http://localhost:3000/api/health``{ "status": "ok" }`
- Logs: `docker compose logs -f app` (and `./logs/app.log` on the host)
- Stop: `docker compose down` (add `-v` to also wipe the database + uploads volumes)
- Modules: installed into `./modules` on the host (bind-mounted to `/app/modules`), never baked into
the image — an operator adds one to a pull-only deployment without building anything. Adding or
removing one takes a `docker compose restart app`; the scan is synchronous at startup. See
[`modules/README.md`](modules/README.md).
**Build the images locally instead of pulling** (offline, or to test an unmerged change) — overlay
the dev file, which adds `build:` back:
```bash
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d --build
```
Keeping `build:` out of the base file means a production host can only ever pull — it can never
accidentally build.
### Option B — Local development (hot reload)
Run the API and the Vite dev server separately. The Vite server proxies `/api` and `/uploads`
to the backend, so the SPA stays same-origin (cookies work).
**1. Start a MariaDB the backend can reach** (published on `localhost:3306`):
```bash
docker run -d --name rg-db -p 3306:3306 -e MARIADB_DATABASE=runic_gateway -e MARIADB_USER=runic -e MARIADB_PASSWORD=devpass -e MARIADB_ROOT_PASSWORD=rootpass mariadb:11
```
**2. Configure + start the backend** (terminal 1):
```bash
cp server/.env.example server/.env
# Set DB_HOST=127.0.0.1, DB_PORT=3306, DB_USER=runic, DB_PASSWORD=devpass,
# JWT_SECRET=<anything>, ADMIN_USERNAME=admin, ADMIN_PASSWORD=<your password>
npm run install-server
npm run server # nodemon → http://localhost:3000
```
**3. Start the frontend** (terminal 2):
```bash
npm run install-client
npm run client # Vite → http://localhost:5173
```
Develop at **http://localhost:5173** (hot reload). On Windows, the Vite proxy targets
`127.0.0.1:3000` to avoid the IPv6-`localhost` pitfall.
> Tip: `npm run install-all` installs both server and client deps in one go.
### Option C — Production build without Docker
Build the SPA and let Express serve it on a single port (still needs a MariaDB + `server/.env`):
```bash
npm run install-all
npm run build # → client/dist
npm start # node server → serves API + SPA at http://localhost:3000
```
---
## First admin & site mode
- On first boot, if the `users` table is empty and `ADMIN_USERNAME` / `ADMIN_PASSWORD` are set,
the first admin is created automatically. You can also run `npm run seed`. After it exists you
may blank those env vars.
- The site **starts in `maintenance` mode**: public visitors see the polished "coming soon" page;
the admin login and panel are always reachable.
- Sign in at **`/admin/login`**, then flip **Maintenance → Live** from the Dashboard. A logged-in
admin can preview the live site even while it's in maintenance.
---
## Pages & routes
**Public** (gated by site mode):
| Route | Page |
|---|---|
| `/` | Portal landing (hero + destinations) |
| `/site` | Website index (section cards) |
| `/site/news` | News feed |
| `/site/screenshots` | Screenshot gallery |
| `/site/five-on-friday` | Five on Friday |
| `/site/newsletter` · `/site/newsletter/:id` | Newsletter list + issue |
| `/site/about` · `/site/status` | About · Site status |
| `/wiki` · `/wiki/:slug` | Wiki landing + article (auto table-of-contents) |
**Admin** (cookie auth, `noindex`):
| Route | View |
|---|---|
| `/admin/login` | Sign in |
| `/admin` | Dashboard (mode toggle, stats, recent activity) |
| `/admin/posts` | Posts CRUD + publish + image upload |
| `/admin/wiki` | Wiki pages CRUD |
| `/admin/settings` | Site settings |
| `/admin/activity` | Activity log |
| `/admin/bot-activity` | Bot activity — banned IPs + recent scoring events, emergency unban (admin only) |
| `/admin/auth-providers` | Authentication — enable/configure SSO providers: built-in Google & Discord + custom OIDC/OAuth2 (admin only) |
| `/admin/users` | User management |
| `/admin/account` | Account security (self-service TOTP two-factor + linked SSO accounts) |
**Player** (any signed-in account, `noindex`): `/player` and its self-service views. Staff are a
superset of players and reach these too.
An installed module adds its own pages under `/<id>/*`, `/admin/<id>/*` and `/player/<id>/*` — for
Module-uo that is `/uo/shard`, `/admin/uo/link`, `/player/uo/characters` and the rest. Core does not
know their names; they arrive with the module and are interleaved into the nav.
---
## API endpoints
| Group | Base | Auth |
|---|---|---|
| Auth (web) | `/api/v1/auth` (`login`, `login/totp`, `logout`, `me`) | cookie |
| Auth (mobile) | `/api/v1/auth/mobile` (`login`, `refresh`, `logout`) | bearer (access + refresh tokens) |
| SSO | `/api/v1/auth` (`providers` — public discovery; `sso/:provider/start`, `sso/:provider/link`, `sso/:provider/callback`) | redirect flow |
| Public | `/api/v1/public` (`settings`, `status`, `posts/:category`, `posts/:category/:idOrSlug`, `wiki`, `wiki/:slug`, `contact`) | none |
| Admin | `/api/v1/admin` (`dashboard`, `site-mode`, `posts`, `posts/upload`, `wiki`, `settings`, `activity`, `bot-activity`, `bot-activity/unban`, `auth/providers` (CRUD), `users`, `account`, `account/totp/*`, `account/identities`) | cookie (admin) |
| Player | `/api/v1/player` (`me`, credentials, 2FA, identities, appeals) | cookie/bearer (any signed-in account) |
| Modules | `/api/v1/public/modules` — id, name, version and capabilities of the modules currently serving | none |
**Module routes are not in this table**, because they are not core's. An installed module mounts
under `/api/v1/public/<prefix>`, `/api/v1/admin/<prefix>` and `/api/v1/player/<prefix>`; which
prefixes exist depends on what is installed. Module-uo, for instance, serves 72 routes under
`/shard`, `/atlas` and `/uo-link` — see its own
[`routes.manifest.json`](https://gitea.whitlocktech.com/RunicGateway/Module-uo/src/branch/main/routes.manifest.json).
On a running instance, `/api/docs` lists everything, core and modules together.
Post categories (URL form): `news`, `five-on-friday`, `newsletter`, `screenshots`.
`authMethod` on a session ∈ `local · totp · mobile · google · discord · oidc`.
See [BACKEND_DESIGN.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/BACKEND_DESIGN.md) §4 for the full contract, or the interactive Swagger
docs below for a per-endpoint reference (parameters, request bodies, response codes).
---
## API documentation (Swagger)
The full API is documented as an **OpenAPI 3.0** spec and served with **Swagger UI**:
| URL | What |
|---|---|
| `http://localhost:3000/api/docs` | Interactive Swagger UI (try-it-out, auth) |
| `http://localhost:3000/api/docs.json` | Raw OpenAPI 3.0 spec (JSON) |
Every endpoint is tagged and grouped (Auth, Auth · Mobile, Auth · SSO, Public, and the Admin
groups) with its summary, parameters, request body, security requirement, and the response codes it
actually returns (`400` validation, `401`/`403` auth, `404`, `409` conflicts, `429` rate limits, …).
**Authentication in the UI** — click **Authorize** and provide either:
- `cookieAuth` — the session cookie (name `rg_token`, configurable via `COOKIE_NAME`; set automatically in the browser after
`POST /api/v1/auth/login`), or
- `bearerAuth` — a mobile access token from `POST /api/v1/auth/mobile/login` (sent as
`Authorization: Bearer <token>`).
**Regenerating the spec** — the spec is generated from `#swagger.*` annotations next to each route
(`server/src/router/**`) plus the shared definitions in `server/swagger/swagger.js`
([swagger-autogen](https://github.com/davibaltar/swagger-autogen)). The output
`server/swagger/swagger-output.json` is committed so the docs work with no build step. After adding
or changing a route, regenerate it:
```bash
cd server
npm run swagger # → server/swagger/swagger-output.json
```
If the generated spec is missing, the server logs a warning and simply disables `/api/docs` (it does
not crash).
**The committed spec is core only, and the served one is not.** swagger-autogen is *static
analysis* — it parses `src/app.js` as text and follows the literal `app.use(…)` chain — so it can
see neither an installed module (which arrives on a volume long after the image was built, and
mounts through a call no parser can follow) nor an extension slot (whose router is created empty and
filled later). Both are handled by merging a **fragment**:
- **Extension slots** contribute at generation time, from `server/swagger/slotSpecs.js`, so they are
in the committed file.
- **Modules** contribute at request time, from the `swagger-fragment.json` each one ships, merged by
`server/swagger/docsSpec.js`. So `/api/docs.json` on a running instance describes more than
`npm run swagger` produces here, and `swagger-output.json` stays reproducible on any machine
regardless of what is installed.
**Core wins every key collision** — a module cannot redefine a core path, tag or schema by shipping
one with the same name; the collision is logged and the module's version dropped.
One thing worth knowing if you edit an annotation: swagger-autogen **reports a broken one and then
succeeds anyway**, dropping it. `npm run swagger` now captures those diagnostics and fails, which is
how two annotations that had been silently documenting an empty request body were found. If it
rejects yours, the usual causes are an object literal a brace short, or a `"` or backtick inside a
single-quoted description (it re-quotes both to `'` before evaluating).
### The route manifest (frozen URL surface)
`server/routes.manifest.json` is a generated, sorted `{ method, path }` list of every route the two
Express listeners actually expose. It is **not** documentation — it is the machine-checkable freeze of
the URL surface, so that carving the router files up by business capability
(`docs/website/API_V2_PLAN.md`) can be proved to move no URL instead of merely claiming it.
```bash
cd server
npm run routes:manifest # → routes.manifest.json + routes.guards.json
npm run routes:manifest -- --check # exit 1 if either file is stale (what CI runs)
```
The generator walks the live Express stack (runtime introspection, not source parsing — a route's path
sits on the line *after* `router.get(`, which defeats greps) and keeps only
`/api/**` and `/.well-known/**` plus the internal listener. The SPA catch-all, `/uploads`, `/brand`
and installed modules' `/modules/<id>` chunks are filesystem-conditional static mounts, not API
contract, so they are excluded and the output depends neither on whether the client has been built
nor on which modules are mounted.
Two generated files, two very different meanings:
| File | Meaning of a diff |
|---|---|
| `routes.manifest.json` | **Contract change.** A URL moved. Justify it in the PR description; never let one ride along in a "mechanical" refactor. |
| `routes.guards.json` | **Review aid.** Per route: handler count + the *named* middleware on its mount chain. Names are a hint only — `requireRole(...)` returns an anonymous arrow and cannot be seen — but a vanished `requireAuth` is unambiguous. |
Unlike the Swagger spec, the manifest is annotation-free: `swagger-output.json` documents intent (only
annotated routes appear), the manifest records reality.
---
## Modules
**Everything specific to a game is a module.** Core has no idea what an "account", a "character" or
a "shard" is; it provides seams, and a module fills them. That is what makes one image able to run a
site for any game rather than for Ultima Online in particular.
The design of record is
[MODULE_SYSTEM.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_SYSTEM.md);
the normative contract — the one to read before writing a module — is
[MODULE_API.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_API.md).
The worked example is [RunicGateway/Module-uo](https://gitea.whitlocktech.com/RunicGateway/Module-uo),
which is where everything this README used to describe under *Shard integration (uo-link)* now
lives: the sidecar client, the ingest dispatcher, account linking, the town crier, the spawn atlas,
and every page that renders them.
### An operator never builds anything
That constraint shapes the whole design. Installing a module is the WordPress-plugin experience — an
admin-panel action, or a directory dropped onto the `modules/` volume — because production runs a
prebuilt, pull-only image with no toolchain in it. So a module ships **assembled**: its client half
is a prebuilt ESM chunk that resolves React from a `window.__rg` global core owns (an import map
would have to be inline, and the CSP is `script-src 'self'`), and its one runtime dependency travels
inside the tarball.
```
modules/
└─ uo/ one directory per module; the id is the directory name
├─ module.json id, version, coreApi range, mounts, extensions, capabilities
├─ swagger-fragment.json merged into /api/docs.json while the module is running
├─ server/ routers, models, and an idempotent schema.sql fragment
└─ client/dist/entry.js the prebuilt chunk, injected by utils/htmlShell.js
```
`modules/` is a bind mount in `docker-compose.yml`, so placing a directory there by hand is a
supported install. The directory is tracked in git (via its README) on purpose: Docker recreates a
*missing* bind-mount source as `root:root`, and the container is uid 1000.
### Three ways in, and none of them is a build
| | How | Where it fits |
|---|---|---|
| **Admin panel** | Admin → Modules, paste the URL of a release's install manifest | The click path. Installs, upgrades, disables, uninstalls and purges, with a restart button — no shell on the box |
| **`MODULES`** | Declare the set in the environment; the container resolves it at every start | The compose-managed host. The running set is a line in a file you version-control, not the residue of past clicks |
| **By hand** | `tar -xf module-uo-0.3.0.tar.gz -C ./modules && mv modules/module-uo-0.3.0 modules/uo`, then restart | Development, and any host where the other two do not fit |
`MODULES` takes one entry per module, whitespace- or comma-separated:
```
MODULES=uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json
```
The id and the version are written out rather than discovered inside the manifest so that **the
no-op case needs no network**: a module already unpacked at the declared version is answered by
reading its own `module.json`, so a restart with the internet down brings the site up exactly as it
was. Only a missing or different version is fetched, and it goes through the same
verify-and-unpack path — allowlisted `https` host, sha256 from the manifest, whole-archive
inspection before anything is written — that the admin panel uses. A version that cannot be
resolved is logged and shown on the admin screen; **it never stops the site from starting**.
The declaration owns what is *on the volume*, never what runs. A module disabled from the admin
panel gets its files back at the next start and stays disabled, because the row and the variable are
answering different questions.
### What a module gets, and what it may not do
At boot, `app.js` scans the volume synchronously, validates each `module.json`, and calls the
module's `register(ctx, api)`:
- **`ctx` is everything core hands over** — the database, the logger, settings, the session reader,
push, the secret box, the middleware, the rate-limit factory, the activity log, and **express
itself**. A module lives outside `server/`, so Node's resolver never reaches core's
`node_modules`; anything it must share has to be handed to it, or there would be two Expresses and
two Reacts in one process.
- **`api` is everything it may register** — routes (one prefix per tier), an extension slot fill,
notification streams, a news-announce leg, a post hook, and `onBoot`/`onShutdown`.
- **It may not reach into core's tree**, mount outside its declared prefixes, or create tables
outside its `<id>_` prefix. Each of those is checked, in the module's CI and again by the loader.
Two things are guaranteed regardless of what a module does. **A failure never takes the site down**:
the loader catches everything from `require` to `onBoot`, marks that module `startup_failed`, and
the site comes up with its routes and nav absent and the reason on the admin screen. And **no URL of
core's may move** — a module that displaced one is caught by the frozen route manifest, which is
generated from a real core with the module loaded.
### What is running right now
```
GET /api/v1/public/modules
{ "modules": [ { "id": "uo", "name": "Ultima Online", "version": "0.3.0",
"capabilities": ["shard", "atlas", "market", …] } ] }
```
Anonymous, database-free, never site-mode gated, and **`started` modules only** — a module that is
disabled or failed is absent, exactly as its routes and its nav already are. Clients feature-detect
against it; they do not use it to decide what to load (the HTML shell injects each chunk's tag).
---
## Environment variables
Copy `.env.example` (Compose) or `server/.env.example` (local) and fill in. **`.env` is git-ignored.**
| Var | Default | Notes |
|---|---|---|
| `NODE_ENV` | `production` | |
| `PORT` | `3000` | server listens on `0.0.0.0:PORT` |
| `UPLOAD_DIR` | `<server>/uploads` | where post images are written (`/app/uploads`, volume-mounted, in Compose) |
| `MODULES_DIR` | `<repo>/modules` | where installed modules are scanned from (`/app/modules`, bind-mounted, in Compose) |
| `MODULES` | — | the module set this deployment runs, resolved at every start: `<id>@<version>=<install manifest URL>`, whitespace/comma separated. Already at the declared version = no network. A failure is logged and shown in Admin → Modules, never fatal. See [Modules](#modules) |
| `MODULE_SOURCE_HOSTS` | `gitea.whitlocktech.com` | **bootstrap only** — seeds the `module_source_hosts` setting on first boot; after that the setting is authoritative and is edited in Admin → Modules |
| `DB_HOST` / `DB_PORT` | `db` / `3306` | `db` in Compose; `127.0.0.1` for local dev |
| `DB_NAME` / `DB_USER` / `DB_PASSWORD` | `runic_gateway` / `runic` / — | app database credentials |
| `DB_ROOT_PASSWORD` | — | MariaDB root (Compose only) |
| `JWT_SECRET` | — | **required** — long random string; signs session, mobile, and SSO-flow tokens |
| `JWT_EXPIRES_IN` | `1d` | web session token + cookie lifetime |
| `COOKIE_SECURE` | `auto` | `auto` = Secure only over HTTPS (works on LAN HTTP + proxy HTTPS) |
| `COOKIE_NAME` | `rg_token` | changing it on a live instance invalidates existing sessions |
| `BRAND_*` | Runic Gateway | instance branding (name, tagline, colors, logo/hero/favicon) — see [Branding](#branding) |
| `SECRET_ENC_KEY` | — | **required in prod** — key for AES-256-GCM encryption of stored OAuth client secrets. Dev falls back to a key derived from `JWT_SECRET` (with a warning) |
| `APP_BASE_URL` | — | public base URL, used to build the SSO OAuth `redirect_uri` (`${APP_BASE_URL}/api/v1/auth/sso/:provider/callback`). Set in prod to match what you register with Google/Discord; if unset it is derived from the request (fine for local dev) |
| `MOBILE_ACCESS_TTL` | `15m` | mobile bearer **access** token lifetime (short-lived) |
| `MOBILE_REFRESH_TTL_DAYS` | `30` | mobile **refresh** token lifetime (long-lived, rotated on use) |
| `TRUST_PROXY` | `1` | reverse-proxy trust for correct `req.ip` / `req.secure` (rate limiting, backoff, bot-ban). Pin to the proxy hop's LAN IP in prod. A blanket `true` is rejected (coerced to `1`) to block `X-Forwarded-For` spoofing |
| `DEBUG_TRUST_PROXY` | `0` | `1` logs raw peer address + `X-Forwarded-For` + resolved `req.ip` per request (to verify/refresh the proxy IP). Noisy — leave off |
| `TOTP_ISSUER` | `BRAND_NAME` | label shown in authenticator apps for optional per-user 2FA |
| `TOTP_CHALLENGE_TTL` | `5m` | lifetime of the short-lived post-password "awaiting code" step |
| `ADMIN_USERNAME` / `ADMIN_PASSWORD` | — | first-admin bootstrap (first boot only) |
| _Email_ | — | configured in Admin → Settings → Email (Gmail OAuth2), not via env; recipient = `contact_email` setting |
| `CLIENT_ORIGIN` | `http://localhost:5173` | enables CORS in dev only |
| `LOG_LEVEL` / `FILE_LOG_LEVEL` | `info` / `debug` | console / file verbosity |
| `LOG_TO_FILE` / `LOG_DIR` / `LOG_FILE` | `true` / `<server>/logs` / `app.log` | log file (bind-mounted to `./logs` in Docker) |
| `ANNOUNCE_POLL_MS` | `15000` | how often the news-announcement dispatcher sweeps `announce_jobs` for legs that are due or retrying. Which legs exist is up to what has registered one — Discord is core's; a module may add its own |
---
## Branding
Instance identity is data, not code — set via `BRAND_*` env vars, so one prebuilt
image can run as any community. With none set, everything renders as **Runic Gateway**.
| Var | What |
|---|---|
| `BRAND_NAME` / `BRAND_SHORT_NAME` | display name (full / short-in-prose) |
| `BRAND_TAGLINE` / `BRAND_DESCRIPTION` | tagline + meta/OG description |
| `BRAND_CONTACT_EMAIL` / `BRAND_URL` | contact + canonical URL (for OG/absolute links) |
| `BRAND_ACCENT_COLOR` | theme `--accent` (web) + Discord embed color |
| `BRAND_LOGO` / `BRAND_HERO` / `BRAND_FAVICON` | image paths under the `/brand` mount, or absolute URLs |
**How it flows:** text/colors reach the SPA at runtime through the public settings
API (`SiteContext`), so no rebuild is needed; the server templates `index.html`
`<title>`/meta/OG/favicon at boot; emails, TOTP issuer, and the Discord bot read
`BRAND_*` directly. The admin-editable **site title** and **contact email**
settings override `BRAND_NAME` / `BRAND_CONTACT_EMAIL` when set. Image assets are
delivered from the `./brand` bind-mount (see `brand/README.md`).
**UOMysticmoon** is the first instance — [`.env.uomysticmoon.example`](.env.uomysticmoon.example)
holds the exact `BRAND_*` + infra (`DB_NAME`/`DB_USER`/`COOKIE_NAME`) pinning to
run this repo as UOMysticmoon.
---
## Security
**Session & authorization**
- All auth flows go through one **session service** (`server/src/auth/`): controllers call
`sessionService.createSession(user, authMethod)` and middleware calls `validateSession()`, so web
cookies, mobile bearer tokens, and SSO all produce the *same* authenticated session model.
`utils/auth.js` remains a thin backward-compat facade.
- JWT in an httpOnly, `SameSite=Lax` cookie (`Secure` auto-detected), bcrypt password hashing.
- Admin routes are **re-validated against the database on every request**, so a demoted or deleted
user loses access immediately instead of keeping their old role until the token expires.
- **Role-based authorization** — admin-only endpoints (users, site mode, settings, auth providers)
are gated by a `requireRole` check, so a lower-privilege editor can't reach them.
**Mobile bearer auth**
- Native clients use `/api/v1/auth/mobile/*`: a short-lived **access token** (bearer JWT, validated
by the same middleware as the cookie) plus a long-lived, **server-stored, revocable refresh
token** that is **rotated on every refresh** (a replayed refresh token is single-use). Refresh
tokens are stored **hashed** (never in the clear); logout revokes one or all. Mobile login reuses
the same bot-scoring + backoff defenses as web, with single-request TOTP.
**Single sign-on (OAuth2 / OIDC)**
- Pluggable providers — built-in **Google** and **Discord** (endpoints fixed in code; admins supply
only client id/secret) plus fully-configurable **custom OIDC/OAuth2** providers, managed from the
**Authentication** admin panel. Only `enabled` + fully-configured providers are shown to users.
- **Link-only** by policy: an SSO login succeeds *only* if the external identity is already linked to
an existing account (linked by the user from **Account**). External identities are **never
auto-provisioned** — no one gains access without an account you created.
- The redirect flow is CSRF-protected with a signed, httpOnly, short-lived transaction cookie plus
**PKCE**; OAuth client secrets are **encrypted at rest** (AES-256-GCM) and never returned to any
client. SSO logins go through the same `sessionService`, so login/activity logging, RBAC, and bot
protection are identical to a local login.
**Login hardening**
- **Optional per-user TOTP two-factor** (opt-in, self-service on `/admin/account`). When enabled,
the password step issues only a short-lived, non-session `stage:'totp'` challenge; a session
cookie is granted only after the second factor verifies.
- **Login throttling** — `express-slow-down` + a hard rate cap + a separate per-IP exponential
backoff, with generic error messages that don't reveal whether the username exists.
- **Honeypot** field on the login form; submissions that fill it are treated as bots.
- **Bot-scoring + automatic IP ban** — weighted scoring of CMS-scanner paths and junk 404s (with a
periodic sweep of stale entries) bans hostile scanners; failed logins and honeypot hits feed the
score. Admins get visibility into this on the **Bot Activity** panel: currently banned IPs and a
recent-events feed (in-memory, most-recent-first), plus a logged emergency **unban** for false
positives — read + unban only, not a scoring-config surface.
**Uploads & input**
- Uploaded file extensions are derived from the **validated mimetype**, not the client-supplied
filename (prevents a disguised-extension upload).
- `express-validator` on all writes; usernames are validated **and** uniqueness-checked on update.
**Platform**
- `helmet`, admin routes `noindex` + `robots.txt` disallow, `trust proxy` for correct client IPs
behind a reverse proxy (see `TRUST_PROXY`), first admin seeded from env (no hardcoded credentials),
`.env` git-ignored. Passwords and request bodies are never logged. Email sends through Gmail
OAuth2 configured in the admin (refresh token stored AES-GCM-encrypted, never in env); the
contact form falls back to a `mailto:` link when unconfigured.
---
## Logging
Every log line goes to **both the console and a log file**, timestamped and leveled
(`error` / `warn` / `info` / `debug`):
```
2026-06-26T18:55:01.123Z INFO [server] listening on http://0.0.0.0:3000 ...
2026-06-26T18:55:09.880Z INFO [http] 192.168.1.40 admin POST /api/v1/auth/login 200 12 ms - 48 bytes
2026-06-26T18:55:14.402Z WARN [auth] login failed {"username":"root","ip":"192.168.1.40"}
2026-06-26T18:55:20.110Z ERROR [error] GET /api/v1/public/wiki -> 500 ... {"stack":"..."}
```
Captured: startup config banner, schema/seed steps, **HTTP access logs** (real client IP via
`trust proxy`, the authenticated admin, method/URL/status/time/size), login success/failure,
rate-limit hits, site-mode changes, all errors with stack traces, and graceful shutdown. Console
verbosity is `LOG_LEVEL`; the file keeps the fuller `FILE_LOG_LEVEL` record. In Docker the file is
bind-mounted to `./logs/app.log` and `docker compose logs -f app` shows the console stream.
---
## Deployment behind a reverse proxy
`docker compose up -d --build` exposes the `app` container on `0.0.0.0:3000` (no `127.0.0.1`
binding) so a reverse proxy — Pangolin, Nginx, Caddy, Traefik, etc. — can reach it. Point the
proxy at `app:3000` (or the host's `:3000` if the proxy runs outside Compose) and terminate TLS
there. Because `COOKIE_SECURE` defaults to `auto`, the admin login works both directly via the
LAN IP over HTTP **and** through the proxy over HTTPS — no config change needed. MariaDB stays on
the private Compose network (no published port by default); data persists in the `dbdata` volume,
uploads in `uploads`.
Set `TRUST_PROXY` so Express reads the real client IP from the proxy's `X-Forwarded-For` header
(see [Environment variables](#environment-variables)) — required for rate limiting, bot scoring,
and correct logging. Forward the standard `X-Forwarded-For` and `X-Forwarded-Proto` headers from
your proxy.
Minimal proxy examples:
```nginx
# Nginx
location / {
proxy_pass http://app:3000;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
```
```caddy
# Caddy — Caddyfile (automatic HTTPS; forwards X-Forwarded-* by default)
your.domain {
reverse_proxy app:3000
}
```
**Pangolin:** create a resource targeting `app:3000`; it forwards the required headers and
terminates HTTPS out of the box, so no extra configuration is needed.
---
## License
Runic Gateway is free software, licensed under the **GNU General Public License
v3.0 or later** — see [LICENSE.md](LICENSE.md).
Copyright (C) 2026 Runic Gateway
This program is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation, either version 3 of the License, or (at your option) any later
version. It is distributed WITHOUT ANY WARRANTY; without even the implied
warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details.
Contributions are welcome — please read [CONTRIBUTING.md](CONTRIBUTING.md) (note
the **AI-usage disclosure** requirement) and our
[Code of Conduct](CODE_OF_CONDUCT.md). Report vulnerabilities privately per
[SECURITY.md](SECURITY.md).