MODULES declares the module set a deployment runs, one entry per module as
`<id>@<version>=<install manifest URL>`, and the container arrives at it by
itself (MODULE_SYSTEM.md §2.7.2 decision 4). A module already unpacked at the
declared version is a no-op that makes NO network call, so a restart with the
network down comes up unchanged; anything else goes through install.js — same
allowlist, same sha256, same inspect-then-extract — and install() now takes an
`expect: {id, version}` so a URL resolving to another module or version is
refused while it is still only a manifest.
Resolution runs inside start(), between the seed and the require of app.js: the
seed is where the host allowlist setting comes from, and the require is what
scans the volume. That buys it the database, so a compose-installed module gets
the same provenance columns an admin install writes.
A failure is logged and carried, never fatal — an unreachable release host must
not take the site down. The declaration owns what is on the volume; the row owns
whether a module runs, so uninstalling a declared module returns its files at
the next start and leaves it disabled. The admin list gains that as a fourth
source (declared / declaredVersion / declaredError), because a declared module
that failed to resolve has no row, no directory and nothing mounted.
Deferring the app require moved core's schema ahead of the volume scan, and the
module schema-fragment replay was wired to core's schema — so every installed
module silently got no tables. Invisible to the suite (each one stubs the loader
or the pool) and to a smoke on a database that already had the tables; found by
booting against an empty one. ensureSchema() now takes `replayModules: false`
for the one caller that scans later, server.js replays them itself after the
require, and a bootOrder test pins the five steps in the only order they work in.
741 server tests (+18), 187 client (+5); manifest unchanged at 166 public + 2
internal, OpenAPI byte-identical.
Co-Authored-By: Claude <noreply@anthropic.com>
758 lines
42 KiB
Markdown
758 lines
42 KiB
Markdown
# Runic Gateway Website
|
|
|
|
[](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
|
|
[](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
|
|
[](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
|
|
[](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
|
|
[](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
|
|
[](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
|
|
[](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
|
|
|
|
Public site, wiki, and protected admin panel for a game community — a full-stack app
|
|
in one repo. Everything specific to a *particular* game lives in an installable
|
|
module, not here. Branding is instance-configurable via `BRAND_*` (see
|
|
[Branding](#branding)); **UOMysticmoon**, an Ultima Online shard, is the first
|
|
instance, and its game half is
|
|
[RunicGateway/Module-uo](https://gitea.whitlocktech.com/RunicGateway/Module-uo).
|
|
|
|
A full-stack app in one repo:
|
|
|
|
- **Backend** — Node.js + Express REST API (layered `router → controller → model → db`), MariaDB, a provider-agnostic session layer (JWT cookie for web, bearer tokens for mobile, pluggable SSO).
|
|
- **Frontend** — React + Vite single-page app (public site, wiki, and the admin panel), dark "gothic" theme (Cinzel + Georgia).
|
|
- **Deploy** — Docker Compose (app + MariaDB) behind a reverse proxy (Pangolin, Nginx, Caddy, Traefik, …). Express serves the built SPA in production.
|
|
- **Modules** — the game-specific half of a site is a module dropped onto a volume: it adds routes, database tables, nav entries and whole SPA pages without this repo knowing anything about the game. See [Modules](#modules).
|
|
|
|
The design reference is [BACKEND_DESIGN.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/BACKEND_DESIGN.md) (API contract, schema, security), in the [**RunicGateway/docs**](https://gitea.whitlocktech.com/RunicGateway/docs) repo — where all project documentation now lives.
|
|
|
|
---
|
|
|
|
## Contents
|
|
|
|
- [Architecture](#architecture)
|
|
- [Tech stack](#tech-stack)
|
|
- [Project structure](#project-structure)
|
|
- [Prerequisites](#prerequisites)
|
|
- [Setup & run](#setup--run)
|
|
- [Option A — Docker Compose (full stack)](#option-a--docker-compose-full-stack)
|
|
- [Option B — Local development (hot reload)](#option-b--local-development-hot-reload)
|
|
- [Option C — Production build without Docker](#option-c--production-build-without-docker)
|
|
- [First admin & site mode](#first-admin--site-mode)
|
|
- [Pages & routes](#pages--routes)
|
|
- [API endpoints](#api-endpoints)
|
|
- [API documentation (Swagger)](#api-documentation-swagger)
|
|
- [Modules](#modules)
|
|
- [Environment variables](#environment-variables)
|
|
- [Security](#security)
|
|
- [Logging](#logging)
|
|
- [Deployment behind a reverse proxy](#deployment-behind-a-reverse-proxy)
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
How the pieces fit together — the React SPA and native app talk to one Express backend
|
|
(`router → controller → model → db`), which persists to MariaDB. Anything that knows
|
|
what game this site is about lives in an installed module, on the right of the diagram.
|
|
|
|
```mermaid
|
|
flowchart TB
|
|
%% ---------- Clients ----------
|
|
subgraph clients["Clients"]
|
|
browser["Browser<br/>React + Vite SPA<br/>(public · wiki · admin)"]
|
|
mobile["Native mobile app<br/>(bearer tokens)"]
|
|
end
|
|
|
|
idp["SSO providers<br/>Google · Discord · custom OIDC"]
|
|
discord["Discord"]
|
|
|
|
%% ---------- Website (one repo) ----------
|
|
subgraph website["website/ — Node app (one repo)"]
|
|
direction TB
|
|
|
|
subgraph backend["server/ — Express backend"]
|
|
direction TB
|
|
mw["Middleware<br/>helmet · siteMode · noindex<br/>rateLimit · loginProtection · botScore · validate"]
|
|
router["Router /api/v1<br/>auth (web · mobile · sso) · public · admin · player"]
|
|
ctrl["Controllers"]
|
|
auth["Session layer (auth/)<br/>sessionService · JWT/cookie · bearer · SSO+PKCE"]
|
|
model["Models (.model + .db)<br/>raw parameterized SQL — no ORM"]
|
|
sse["SSE fan-out<br/>public stream (allowlist) · admin stream (sensitive)"]
|
|
loader["modules/loader.js<br/>scans the volume · mounts · registries · lifecycle"]
|
|
secret["secretBox.js<br/>AES-256-GCM secrets at rest"]
|
|
end
|
|
|
|
bot["bot/<br/>Discord bot"]
|
|
end
|
|
|
|
db[("MariaDB<br/>users · posts · wiki · settings · activity<br/>mobileSessions · authProviders · userIdentities<br/>installed_modules · <module>_*")]
|
|
|
|
%% ---------- Module side ----------
|
|
subgraph modside["modules/<id>/ — installed, not built (e.g. Module-uo)"]
|
|
direction TB
|
|
modsrv["server/ — routers, models, schema fragment<br/>reaches core only through ctx"]
|
|
modcli["client/dist/entry.js — prebuilt ESM chunk<br/>React shared via window.__rg"]
|
|
end
|
|
|
|
game["The game<br/>whatever the module talks to<br/>(for Module-uo: a ServUO shard,<br/>via the uo-link sidecar)"]
|
|
|
|
%% ---------- Edges ----------
|
|
browser <-->|"same-origin JSON + SSE (cookie)"| mw
|
|
mobile -->|"REST (bearer access/refresh)"| mw
|
|
browser -.->|"OAuth redirect + PKCE"| idp
|
|
auth -.->|"token exchange"| idp
|
|
|
|
mw --> router --> ctrl
|
|
ctrl --> auth
|
|
ctrl --> model
|
|
ctrl --> sse
|
|
auth --> model
|
|
model <--> db
|
|
auth -. reads/writes secrets .-> secret
|
|
sse -->|"live events"| browser
|
|
bot -->|"messages"| discord
|
|
bot <--> db
|
|
|
|
loader -->|"mounts under /api/v1/<tier>/<prefix>"| router
|
|
loader -->|"require() + register(ctx, api)"| modsrv
|
|
modsrv -->|"ctx.db · ctx.push · ctx.activity …"| model
|
|
modsrv <--> game
|
|
browser -->|"<script type=module> injected by htmlShell"| modcli
|
|
|
|
%% ---------- Styling ----------
|
|
classDef ext fill:#2d2233,stroke:#7a5c94,color:#e8dff0;
|
|
classDef store fill:#1f2d2a,stroke:#4c8c7d,color:#dff0ea;
|
|
classDef mod fill:#2d2620,stroke:#94764c,color:#f0e6d8;
|
|
class idp,discord,game ext;
|
|
class db store;
|
|
class modsrv,modcli mod;
|
|
```
|
|
|
|
- **One backend, layered.** Every request flows `middleware → router → controller → model → db`.
|
|
Web browsers authenticate with an httpOnly JWT cookie; the native app uses short-lived bearer
|
|
access tokens plus rotated refresh tokens; SSO (Google/Discord/OIDC) is link-only and PKCE-guarded.
|
|
All three surfaces produce the *same* session via the session layer.
|
|
- **Core knows nothing about any game.** Routes, tables, nav entries, SPA pages and push streams for
|
|
a specific game arrive from a module the operator installed. Core provides the seams; the module
|
|
fills them. See [Modules](#modules).
|
|
- **A module that fails must never take the site down.** The loader catches failures across a
|
|
module's whole lifecycle and marks that one module `startup_failed`; the site comes up with its
|
|
routes and nav absent, and the admin panel says why.
|
|
- **Sensitive events stay private.** Events fan out to browsers over two SSE channels — a public
|
|
allowlist stream and an admin-only stream that adds staff audit / cheat / login events. Which
|
|
event kinds are public is decided by the module that publishes them, and core enforces the split.
|
|
|
|
---
|
|
|
|
## Tech stack
|
|
|
|
| Layer | Tech |
|
|
|---|---|
|
|
| Backend | Node.js 20+, Express 4, `mariadb` driver (parameterized SQL, no ORM) |
|
|
| Auth | Session service over JWT: httpOnly cookie (web) + bearer access/refresh tokens (mobile), bcrypt hashing, optional TOTP 2FA (`speakeasy` + `qrcode`), pluggable OAuth2/OIDC SSO (built-in Google & Discord + generic) |
|
|
| Database | MariaDB 11 (own container) |
|
|
| Frontend | React 18, Vite 5, React Router 6 |
|
|
| Email | Nodemailer via Gmail OAuth2 (configured in admin), with a `mailto:` fallback |
|
|
| API docs | OpenAPI 3.0 via `swagger-autogen`, served with `swagger-ui-express` at `/api/docs` |
|
|
| Deploy | Docker Compose, any reverse proxy (Pangolin, Nginx, Caddy, Traefik, …) |
|
|
|
|
---
|
|
|
|
## Project structure
|
|
|
|
```
|
|
website/
|
|
├─ server/ Express API
|
|
│ ├─ src/
|
|
│ │ ├─ server.js bootstrap: ensure schema → seed → listen (0.0.0.0)
|
|
│ │ ├─ app.js middleware + static SPA + routes
|
|
│ │ ├─ auth/ session layer: session.service · token (JWT/cookies) · session.middleware · ssoState (PKCE/CSRF) · providers/ (base · oauth2 · google · discord · genericOidc · registry)
|
|
│ │ ├─ router/v1/ auth (web · mobile · sso) / public / admin / player route groups
|
|
│ │ ├─ model/ users · posts · wiki · settings · activity · mobileSessions · authProviders · userIdentities · modules (.model + .db)
|
|
│ │ ├─ modules/ loader (scan · validate · mount) · registries (the seams) · lifecycle (boot/shutdown + reconcile)
|
|
│ │ ├─ middleware/ siteMode · noindex · rateLimit · loginProtection · botScore · validate
|
|
│ │ └─ utils/ auth (compat facade) · totp (2FA) · secretBox (AES-GCM secrets) · db (pool) · mailer · logger · htmlShell
|
|
│ ├─ db/ schema.sql + seed.js
|
|
│ ├─ swagger/ swagger.js (generator config) · swagger-output.json (generated, core only) · docsSpec.js (merges module fragments at request time)
|
|
│ └─ .env.example
|
|
├─ client/ React + Vite SPA
|
|
│ ├─ src/
|
|
│ │ ├─ routes/public/ Portal, Website, News, Screenshots, FiveOnFriday, Newsletter(+Issue), Status, About, Maintenance
|
|
│ │ ├─ routes/wiki/ Wiki landing + WikiArticle
|
|
│ │ ├─ routes/admin/ AdminLogin (password + TOTP + SSO buttons), AdminLayout, views/ (Dashboard, Posts, Wiki, Settings, Activity, Bot Activity, Authentication, Users, Account) + editors
|
|
│ │ ├─ components/ SiteHeader, SiteFooter, layout, guards, Modal, ProviderIcon (inline SSO SVGs), …
|
|
│ │ ├─ contexts/ AuthContext, SiteContext
|
|
│ │ ├─ modules/ the client registry: routes · nav · slots · feature gates · window.__rg
|
|
│ │ ├─ api/client.js fetch wrapper (sends cookies)
|
|
│ │ └─ styles/theme.css design tokens
|
|
│ └─ public/assets/img/ hero image
|
|
├─ modules/ installed modules, one directory each — a Docker bind mount; empty here
|
|
├─ Dockerfile builds client → serves via Express
|
|
├─ docker-compose.yml app + MariaDB
|
|
├─ .env.example root env (used by Compose)
|
|
└─ package.json workspace scripts
|
|
```
|
|
|
|
---
|
|
|
|
## Prerequisites
|
|
|
|
- **Node.js 20+** and npm (Node 22/24 are fine).
|
|
- **Docker Desktop** (for MariaDB, and for the full Compose deploy).
|
|
|
|
---
|
|
|
|
## Setup & run
|
|
|
|
### Option A — Docker Compose (full stack)
|
|
|
|
`docker-compose.yml` is **production-shaped**: it *pulls* the prebuilt `app` and `bot` images from
|
|
the Gitea container registry (published by `.gitea/workflows/build-images.yml` on every merge to
|
|
`main`) — it never builds. Each image already bundles the server deps and the built React client,
|
|
which Express serves. MariaDB runs in its own container; tables + defaults + the first admin are
|
|
created automatically on first boot.
|
|
|
|
```bash
|
|
cp .env.example .env
|
|
# Edit .env and set at least:
|
|
# DB_PASSWORD, DB_ROOT_PASSWORD (any strong values)
|
|
# JWT_SECRET (a long random string)
|
|
# ADMIN_USERNAME, ADMIN_PASSWORD (your first admin login)
|
|
|
|
docker compose pull && docker compose up -d # IMAGE_TAG defaults to `latest`
|
|
# pin a specific build (reproducible deploy / rollback):
|
|
IMAGE_TAG=sha-042a151 docker compose pull && docker compose up -d
|
|
```
|
|
|
|
- App: **http://localhost:3000** (binds `0.0.0.0`)
|
|
- Health check: `GET http://localhost:3000/api/health` → `{ "status": "ok" }`
|
|
- Logs: `docker compose logs -f app` (and `./logs/app.log` on the host)
|
|
- Stop: `docker compose down` (add `-v` to also wipe the database + uploads volumes)
|
|
- Modules: installed into `./modules` on the host (bind-mounted to `/app/modules`), never baked into
|
|
the image — an operator adds one to a pull-only deployment without building anything. Adding or
|
|
removing one takes a `docker compose restart app`; the scan is synchronous at startup. See
|
|
[`modules/README.md`](modules/README.md).
|
|
|
|
**Build the images locally instead of pulling** (offline, or to test an unmerged change) — overlay
|
|
the dev file, which adds `build:` back:
|
|
|
|
```bash
|
|
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d --build
|
|
```
|
|
|
|
Keeping `build:` out of the base file means a production host can only ever pull — it can never
|
|
accidentally build.
|
|
|
|
### Option B — Local development (hot reload)
|
|
|
|
Run the API and the Vite dev server separately. The Vite server proxies `/api` and `/uploads`
|
|
to the backend, so the SPA stays same-origin (cookies work).
|
|
|
|
**1. Start a MariaDB the backend can reach** (published on `localhost:3306`):
|
|
|
|
```bash
|
|
docker run -d --name rg-db -p 3306:3306 -e MARIADB_DATABASE=runic_gateway -e MARIADB_USER=runic -e MARIADB_PASSWORD=devpass -e MARIADB_ROOT_PASSWORD=rootpass mariadb:11
|
|
```
|
|
|
|
**2. Configure + start the backend** (terminal 1):
|
|
|
|
```bash
|
|
cp server/.env.example server/.env
|
|
# Set DB_HOST=127.0.0.1, DB_PORT=3306, DB_USER=runic, DB_PASSWORD=devpass,
|
|
# JWT_SECRET=<anything>, ADMIN_USERNAME=admin, ADMIN_PASSWORD=<your password>
|
|
npm run install-server
|
|
npm run server # nodemon → http://localhost:3000
|
|
```
|
|
|
|
**3. Start the frontend** (terminal 2):
|
|
|
|
```bash
|
|
npm run install-client
|
|
npm run client # Vite → http://localhost:5173
|
|
```
|
|
|
|
Develop at **http://localhost:5173** (hot reload). On Windows, the Vite proxy targets
|
|
`127.0.0.1:3000` to avoid the IPv6-`localhost` pitfall.
|
|
|
|
> Tip: `npm run install-all` installs both server and client deps in one go.
|
|
|
|
### Option C — Production build without Docker
|
|
|
|
Build the SPA and let Express serve it on a single port (still needs a MariaDB + `server/.env`):
|
|
|
|
```bash
|
|
npm run install-all
|
|
npm run build # → client/dist
|
|
npm start # node server → serves API + SPA at http://localhost:3000
|
|
```
|
|
|
|
---
|
|
|
|
## First admin & site mode
|
|
|
|
- On first boot, if the `users` table is empty and `ADMIN_USERNAME` / `ADMIN_PASSWORD` are set,
|
|
the first admin is created automatically. You can also run `npm run seed`. After it exists you
|
|
may blank those env vars.
|
|
- The site **starts in `maintenance` mode**: public visitors see the polished "coming soon" page;
|
|
the admin login and panel are always reachable.
|
|
- Sign in at **`/admin/login`**, then flip **Maintenance → Live** from the Dashboard. A logged-in
|
|
admin can preview the live site even while it's in maintenance.
|
|
|
|
---
|
|
|
|
## Pages & routes
|
|
|
|
**Public** (gated by site mode):
|
|
|
|
| Route | Page |
|
|
|---|---|
|
|
| `/` | Portal landing (hero + destinations) |
|
|
| `/site` | Website index (section cards) |
|
|
| `/site/news` | News feed |
|
|
| `/site/screenshots` | Screenshot gallery |
|
|
| `/site/five-on-friday` | Five on Friday |
|
|
| `/site/newsletter` · `/site/newsletter/:id` | Newsletter list + issue |
|
|
| `/site/about` · `/site/status` | About · Site status |
|
|
| `/wiki` · `/wiki/:slug` | Wiki landing + article (auto table-of-contents) |
|
|
|
|
**Admin** (cookie auth, `noindex`):
|
|
|
|
| Route | View |
|
|
|---|---|
|
|
| `/admin/login` | Sign in |
|
|
| `/admin` | Dashboard (mode toggle, stats, recent activity) |
|
|
| `/admin/posts` | Posts CRUD + publish + image upload |
|
|
| `/admin/wiki` | Wiki pages CRUD |
|
|
| `/admin/settings` | Site settings |
|
|
| `/admin/activity` | Activity log |
|
|
| `/admin/bot-activity` | Bot activity — banned IPs + recent scoring events, emergency unban (admin only) |
|
|
| `/admin/auth-providers` | Authentication — enable/configure SSO providers: built-in Google & Discord + custom OIDC/OAuth2 (admin only) |
|
|
| `/admin/users` | User management |
|
|
| `/admin/account` | Account security (self-service TOTP two-factor + linked SSO accounts) |
|
|
|
|
**Player** (any signed-in account, `noindex`): `/player` and its self-service views. Staff are a
|
|
superset of players and reach these too.
|
|
|
|
An installed module adds its own pages under `/<id>/*`, `/admin/<id>/*` and `/player/<id>/*` — for
|
|
Module-uo that is `/uo/shard`, `/admin/uo/link`, `/player/uo/characters` and the rest. Core does not
|
|
know their names; they arrive with the module and are interleaved into the nav.
|
|
|
|
---
|
|
|
|
## API endpoints
|
|
|
|
| Group | Base | Auth |
|
|
|---|---|---|
|
|
| Auth (web) | `/api/v1/auth` (`login`, `login/totp`, `logout`, `me`) | cookie |
|
|
| Auth (mobile) | `/api/v1/auth/mobile` (`login`, `refresh`, `logout`) | bearer (access + refresh tokens) |
|
|
| SSO | `/api/v1/auth` (`providers` — public discovery; `sso/:provider/start`, `sso/:provider/link`, `sso/:provider/callback`) | redirect flow |
|
|
| Public | `/api/v1/public` (`settings`, `status`, `posts/:category`, `posts/:category/:idOrSlug`, `wiki`, `wiki/:slug`, `contact`) | none |
|
|
| Admin | `/api/v1/admin` (`dashboard`, `site-mode`, `posts`, `posts/upload`, `wiki`, `settings`, `activity`, `bot-activity`, `bot-activity/unban`, `auth/providers` (CRUD), `users`, `account`, `account/totp/*`, `account/identities`) | cookie (admin) |
|
|
| Player | `/api/v1/player` (`me`, credentials, 2FA, identities, appeals) | cookie/bearer (any signed-in account) |
|
|
| Modules | `/api/v1/public/modules` — id, name, version and capabilities of the modules currently serving | none |
|
|
|
|
**Module routes are not in this table**, because they are not core's. An installed module mounts
|
|
under `/api/v1/public/<prefix>`, `/api/v1/admin/<prefix>` and `/api/v1/player/<prefix>`; which
|
|
prefixes exist depends on what is installed. Module-uo, for instance, serves 72 routes under
|
|
`/shard`, `/atlas` and `/uo-link` — see its own
|
|
[`routes.manifest.json`](https://gitea.whitlocktech.com/RunicGateway/Module-uo/src/branch/main/routes.manifest.json).
|
|
On a running instance, `/api/docs` lists everything, core and modules together.
|
|
|
|
Post categories (URL form): `news`, `five-on-friday`, `newsletter`, `screenshots`.
|
|
`authMethod` on a session ∈ `local · totp · mobile · google · discord · oidc`.
|
|
See [BACKEND_DESIGN.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/BACKEND_DESIGN.md) §4 for the full contract, or the interactive Swagger
|
|
docs below for a per-endpoint reference (parameters, request bodies, response codes).
|
|
|
|
---
|
|
|
|
## API documentation (Swagger)
|
|
|
|
The full API is documented as an **OpenAPI 3.0** spec and served with **Swagger UI**:
|
|
|
|
| URL | What |
|
|
|---|---|
|
|
| `http://localhost:3000/api/docs` | Interactive Swagger UI (try-it-out, auth) |
|
|
| `http://localhost:3000/api/docs.json` | Raw OpenAPI 3.0 spec (JSON) |
|
|
|
|
Every endpoint is tagged and grouped (Auth, Auth · Mobile, Auth · SSO, Public, and the Admin
|
|
groups) with its summary, parameters, request body, security requirement, and the response codes it
|
|
actually returns (`400` validation, `401`/`403` auth, `404`, `409` conflicts, `429` rate limits, …).
|
|
|
|
**Authentication in the UI** — click **Authorize** and provide either:
|
|
|
|
- `cookieAuth` — the session cookie (name `rg_token`, configurable via `COOKIE_NAME`; set automatically in the browser after
|
|
`POST /api/v1/auth/login`), or
|
|
- `bearerAuth` — a mobile access token from `POST /api/v1/auth/mobile/login` (sent as
|
|
`Authorization: Bearer <token>`).
|
|
|
|
**Regenerating the spec** — the spec is generated from `#swagger.*` annotations next to each route
|
|
(`server/src/router/**`) plus the shared definitions in `server/swagger/swagger.js`
|
|
([swagger-autogen](https://github.com/davibaltar/swagger-autogen)). The output
|
|
`server/swagger/swagger-output.json` is committed so the docs work with no build step. After adding
|
|
or changing a route, regenerate it:
|
|
|
|
```bash
|
|
cd server
|
|
npm run swagger # → server/swagger/swagger-output.json
|
|
```
|
|
|
|
If the generated spec is missing, the server logs a warning and simply disables `/api/docs` (it does
|
|
not crash).
|
|
|
|
**The committed spec is core only, and the served one is not.** swagger-autogen is *static
|
|
analysis* — it parses `src/app.js` as text and follows the literal `app.use(…)` chain — so it can
|
|
see neither an installed module (which arrives on a volume long after the image was built, and
|
|
mounts through a call no parser can follow) nor an extension slot (whose router is created empty and
|
|
filled later). Both are handled by merging a **fragment**:
|
|
|
|
- **Extension slots** contribute at generation time, from `server/swagger/slotSpecs.js`, so they are
|
|
in the committed file.
|
|
- **Modules** contribute at request time, from the `swagger-fragment.json` each one ships, merged by
|
|
`server/swagger/docsSpec.js`. So `/api/docs.json` on a running instance describes more than
|
|
`npm run swagger` produces here, and `swagger-output.json` stays reproducible on any machine
|
|
regardless of what is installed.
|
|
|
|
**Core wins every key collision** — a module cannot redefine a core path, tag or schema by shipping
|
|
one with the same name; the collision is logged and the module's version dropped.
|
|
|
|
One thing worth knowing if you edit an annotation: swagger-autogen **reports a broken one and then
|
|
succeeds anyway**, dropping it. `npm run swagger` now captures those diagnostics and fails, which is
|
|
how two annotations that had been silently documenting an empty request body were found. If it
|
|
rejects yours, the usual causes are an object literal a brace short, or a `"` or backtick inside a
|
|
single-quoted description (it re-quotes both to `'` before evaluating).
|
|
|
|
### The route manifest (frozen URL surface)
|
|
|
|
`server/routes.manifest.json` is a generated, sorted `{ method, path }` list of every route the two
|
|
Express listeners actually expose. It is **not** documentation — it is the machine-checkable freeze of
|
|
the URL surface, so that carving the router files up by business capability
|
|
(`docs/website/API_V2_PLAN.md`) can be proved to move no URL instead of merely claiming it.
|
|
|
|
```bash
|
|
cd server
|
|
npm run routes:manifest # → routes.manifest.json + routes.guards.json
|
|
npm run routes:manifest -- --check # exit 1 if either file is stale (what CI runs)
|
|
```
|
|
|
|
The generator walks the live Express stack (runtime introspection, not source parsing — a route's path
|
|
sits on the line *after* `router.get(`, which defeats greps) and keeps only
|
|
`/api/**` and `/.well-known/**` plus the internal listener. The SPA catch-all, `/uploads`, `/brand`
|
|
and installed modules' `/modules/<id>` chunks are filesystem-conditional static mounts, not API
|
|
contract, so they are excluded and the output depends neither on whether the client has been built
|
|
nor on which modules are mounted.
|
|
|
|
Two generated files, two very different meanings:
|
|
|
|
| File | Meaning of a diff |
|
|
|---|---|
|
|
| `routes.manifest.json` | **Contract change.** A URL moved. Justify it in the PR description; never let one ride along in a "mechanical" refactor. |
|
|
| `routes.guards.json` | **Review aid.** Per route: handler count + the *named* middleware on its mount chain. Names are a hint only — `requireRole(...)` returns an anonymous arrow and cannot be seen — but a vanished `requireAuth` is unambiguous. |
|
|
|
|
Unlike the Swagger spec, the manifest is annotation-free: `swagger-output.json` documents intent (only
|
|
annotated routes appear), the manifest records reality.
|
|
|
|
---
|
|
|
|
## Modules
|
|
|
|
**Everything specific to a game is a module.** Core has no idea what an "account", a "character" or
|
|
a "shard" is; it provides seams, and a module fills them. That is what makes one image able to run a
|
|
site for any game rather than for Ultima Online in particular.
|
|
|
|
The design of record is
|
|
[MODULE_SYSTEM.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_SYSTEM.md);
|
|
the normative contract — the one to read before writing a module — is
|
|
[MODULE_API.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_API.md).
|
|
The worked example is [RunicGateway/Module-uo](https://gitea.whitlocktech.com/RunicGateway/Module-uo),
|
|
which is where everything this README used to describe under *Shard integration (uo-link)* now
|
|
lives: the sidecar client, the ingest dispatcher, account linking, the town crier, the spawn atlas,
|
|
and every page that renders them.
|
|
|
|
### An operator never builds anything
|
|
|
|
That constraint shapes the whole design. Installing a module is the WordPress-plugin experience — an
|
|
admin-panel action, or a directory dropped onto the `modules/` volume — because production runs a
|
|
prebuilt, pull-only image with no toolchain in it. So a module ships **assembled**: its client half
|
|
is a prebuilt ESM chunk that resolves React from a `window.__rg` global core owns (an import map
|
|
would have to be inline, and the CSP is `script-src 'self'`), and its one runtime dependency travels
|
|
inside the tarball.
|
|
|
|
```
|
|
modules/
|
|
└─ uo/ one directory per module; the id is the directory name
|
|
├─ module.json id, version, coreApi range, mounts, extensions, capabilities
|
|
├─ swagger-fragment.json merged into /api/docs.json while the module is running
|
|
├─ server/ routers, models, and an idempotent schema.sql fragment
|
|
└─ client/dist/entry.js the prebuilt chunk, injected by utils/htmlShell.js
|
|
```
|
|
|
|
`modules/` is a bind mount in `docker-compose.yml`, so placing a directory there by hand is a
|
|
supported install. The directory is tracked in git (via its README) on purpose: Docker recreates a
|
|
*missing* bind-mount source as `root:root`, and the container is uid 1000.
|
|
|
|
### Three ways in, and none of them is a build
|
|
|
|
| | How | Where it fits |
|
|
|---|---|---|
|
|
| **Admin panel** | Admin → Modules, paste the URL of a release's install manifest | The click path. Installs, upgrades, disables, uninstalls and purges, with a restart button — no shell on the box |
|
|
| **`MODULES`** | Declare the set in the environment; the container resolves it at every start | The compose-managed host. The running set is a line in a file you version-control, not the residue of past clicks |
|
|
| **By hand** | `tar -xf module-uo-0.3.0.tar.gz -C ./modules && mv modules/module-uo-0.3.0 modules/uo`, then restart | Development, and any host where the other two do not fit |
|
|
|
|
`MODULES` takes one entry per module, whitespace- or comma-separated:
|
|
|
|
```
|
|
MODULES=uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json
|
|
```
|
|
|
|
The id and the version are written out rather than discovered inside the manifest so that **the
|
|
no-op case needs no network**: a module already unpacked at the declared version is answered by
|
|
reading its own `module.json`, so a restart with the internet down brings the site up exactly as it
|
|
was. Only a missing or different version is fetched, and it goes through the same
|
|
verify-and-unpack path — allowlisted `https` host, sha256 from the manifest, whole-archive
|
|
inspection before anything is written — that the admin panel uses. A version that cannot be
|
|
resolved is logged and shown on the admin screen; **it never stops the site from starting**.
|
|
|
|
The declaration owns what is *on the volume*, never what runs. A module disabled from the admin
|
|
panel gets its files back at the next start and stays disabled, because the row and the variable are
|
|
answering different questions.
|
|
|
|
### What a module gets, and what it may not do
|
|
|
|
At boot, `app.js` scans the volume synchronously, validates each `module.json`, and calls the
|
|
module's `register(ctx, api)`:
|
|
|
|
- **`ctx` is everything core hands over** — the database, the logger, settings, the session reader,
|
|
push, the secret box, the middleware, the rate-limit factory, the activity log, and **express
|
|
itself**. A module lives outside `server/`, so Node's resolver never reaches core's
|
|
`node_modules`; anything it must share has to be handed to it, or there would be two Expresses and
|
|
two Reacts in one process.
|
|
- **`api` is everything it may register** — routes (one prefix per tier), an extension slot fill,
|
|
notification streams, a news-announce leg, a post hook, and `onBoot`/`onShutdown`.
|
|
- **It may not reach into core's tree**, mount outside its declared prefixes, or create tables
|
|
outside its `<id>_` prefix. Each of those is checked, in the module's CI and again by the loader.
|
|
|
|
Two things are guaranteed regardless of what a module does. **A failure never takes the site down**:
|
|
the loader catches everything from `require` to `onBoot`, marks that module `startup_failed`, and
|
|
the site comes up with its routes and nav absent and the reason on the admin screen. And **no URL of
|
|
core's may move** — a module that displaced one is caught by the frozen route manifest, which is
|
|
generated from a real core with the module loaded.
|
|
|
|
### What is running right now
|
|
|
|
```
|
|
GET /api/v1/public/modules
|
|
{ "modules": [ { "id": "uo", "name": "Ultima Online", "version": "0.3.0",
|
|
"capabilities": ["shard", "atlas", "market", …] } ] }
|
|
```
|
|
|
|
Anonymous, database-free, never site-mode gated, and **`started` modules only** — a module that is
|
|
disabled or failed is absent, exactly as its routes and its nav already are. Clients feature-detect
|
|
against it; they do not use it to decide what to load (the HTML shell injects each chunk's tag).
|
|
|
|
---
|
|
|
|
## Environment variables
|
|
|
|
Copy `.env.example` (Compose) or `server/.env.example` (local) and fill in. **`.env` is git-ignored.**
|
|
|
|
| Var | Default | Notes |
|
|
|---|---|---|
|
|
| `NODE_ENV` | `production` | |
|
|
| `PORT` | `3000` | server listens on `0.0.0.0:PORT` |
|
|
| `UPLOAD_DIR` | `<server>/uploads` | where post images are written (`/app/uploads`, volume-mounted, in Compose) |
|
|
| `MODULES_DIR` | `<repo>/modules` | where installed modules are scanned from (`/app/modules`, bind-mounted, in Compose) |
|
|
| `MODULES` | — | the module set this deployment runs, resolved at every start: `<id>@<version>=<install manifest URL>`, whitespace/comma separated. Already at the declared version = no network. A failure is logged and shown in Admin → Modules, never fatal. See [Modules](#modules) |
|
|
| `MODULE_SOURCE_HOSTS` | `gitea.whitlocktech.com` | **bootstrap only** — seeds the `module_source_hosts` setting on first boot; after that the setting is authoritative and is edited in Admin → Modules |
|
|
| `DB_HOST` / `DB_PORT` | `db` / `3306` | `db` in Compose; `127.0.0.1` for local dev |
|
|
| `DB_NAME` / `DB_USER` / `DB_PASSWORD` | `runic_gateway` / `runic` / — | app database credentials |
|
|
| `DB_ROOT_PASSWORD` | — | MariaDB root (Compose only) |
|
|
| `JWT_SECRET` | — | **required** — long random string; signs session, mobile, and SSO-flow tokens |
|
|
| `JWT_EXPIRES_IN` | `1d` | web session token + cookie lifetime |
|
|
| `COOKIE_SECURE` | `auto` | `auto` = Secure only over HTTPS (works on LAN HTTP + proxy HTTPS) |
|
|
| `COOKIE_NAME` | `rg_token` | changing it on a live instance invalidates existing sessions |
|
|
| `BRAND_*` | Runic Gateway | instance branding (name, tagline, colors, logo/hero/favicon) — see [Branding](#branding) |
|
|
| `SECRET_ENC_KEY` | — | **required in prod** — key for AES-256-GCM encryption of stored OAuth client secrets. Dev falls back to a key derived from `JWT_SECRET` (with a warning) |
|
|
| `APP_BASE_URL` | — | public base URL, used to build the SSO OAuth `redirect_uri` (`${APP_BASE_URL}/api/v1/auth/sso/:provider/callback`). Set in prod to match what you register with Google/Discord; if unset it is derived from the request (fine for local dev) |
|
|
| `MOBILE_ACCESS_TTL` | `15m` | mobile bearer **access** token lifetime (short-lived) |
|
|
| `MOBILE_REFRESH_TTL_DAYS` | `30` | mobile **refresh** token lifetime (long-lived, rotated on use) |
|
|
| `TRUST_PROXY` | `1` | reverse-proxy trust for correct `req.ip` / `req.secure` (rate limiting, backoff, bot-ban). Pin to the proxy hop's LAN IP in prod. A blanket `true` is rejected (coerced to `1`) to block `X-Forwarded-For` spoofing |
|
|
| `DEBUG_TRUST_PROXY` | `0` | `1` logs raw peer address + `X-Forwarded-For` + resolved `req.ip` per request (to verify/refresh the proxy IP). Noisy — leave off |
|
|
| `TOTP_ISSUER` | `BRAND_NAME` | label shown in authenticator apps for optional per-user 2FA |
|
|
| `TOTP_CHALLENGE_TTL` | `5m` | lifetime of the short-lived post-password "awaiting code" step |
|
|
| `ADMIN_USERNAME` / `ADMIN_PASSWORD` | — | first-admin bootstrap (first boot only) |
|
|
| _Email_ | — | configured in Admin → Settings → Email (Gmail OAuth2), not via env; recipient = `contact_email` setting |
|
|
| `CLIENT_ORIGIN` | `http://localhost:5173` | enables CORS in dev only |
|
|
| `LOG_LEVEL` / `FILE_LOG_LEVEL` | `info` / `debug` | console / file verbosity |
|
|
| `LOG_TO_FILE` / `LOG_DIR` / `LOG_FILE` | `true` / `<server>/logs` / `app.log` | log file (bind-mounted to `./logs` in Docker) |
|
|
| `ANNOUNCE_POLL_MS` | `15000` | how often the news-announcement dispatcher sweeps `announce_jobs` for legs that are due or retrying. Which legs exist is up to what has registered one — Discord is core's; a module may add its own |
|
|
|
|
---
|
|
|
|
## Branding
|
|
|
|
Instance identity is data, not code — set via `BRAND_*` env vars, so one prebuilt
|
|
image can run as any community. With none set, everything renders as **Runic Gateway**.
|
|
|
|
| Var | What |
|
|
|---|---|
|
|
| `BRAND_NAME` / `BRAND_SHORT_NAME` | display name (full / short-in-prose) |
|
|
| `BRAND_TAGLINE` / `BRAND_DESCRIPTION` | tagline + meta/OG description |
|
|
| `BRAND_CONTACT_EMAIL` / `BRAND_URL` | contact + canonical URL (for OG/absolute links) |
|
|
| `BRAND_ACCENT_COLOR` | theme `--accent` (web) + Discord embed color |
|
|
| `BRAND_LOGO` / `BRAND_HERO` / `BRAND_FAVICON` | image paths under the `/brand` mount, or absolute URLs |
|
|
|
|
**How it flows:** text/colors reach the SPA at runtime through the public settings
|
|
API (`SiteContext`), so no rebuild is needed; the server templates `index.html`
|
|
`<title>`/meta/OG/favicon at boot; emails, TOTP issuer, and the Discord bot read
|
|
`BRAND_*` directly. The admin-editable **site title** and **contact email**
|
|
settings override `BRAND_NAME` / `BRAND_CONTACT_EMAIL` when set. Image assets are
|
|
delivered from the `./brand` bind-mount (see `brand/README.md`).
|
|
|
|
**UOMysticmoon** is the first instance — [`.env.uomysticmoon.example`](.env.uomysticmoon.example)
|
|
holds the exact `BRAND_*` + infra (`DB_NAME`/`DB_USER`/`COOKIE_NAME`) pinning to
|
|
run this repo as UOMysticmoon.
|
|
|
|
---
|
|
|
|
## Security
|
|
|
|
**Session & authorization**
|
|
|
|
- All auth flows go through one **session service** (`server/src/auth/`): controllers call
|
|
`sessionService.createSession(user, authMethod)` and middleware calls `validateSession()`, so web
|
|
cookies, mobile bearer tokens, and SSO all produce the *same* authenticated session model.
|
|
`utils/auth.js` remains a thin backward-compat facade.
|
|
- JWT in an httpOnly, `SameSite=Lax` cookie (`Secure` auto-detected), bcrypt password hashing.
|
|
- Admin routes are **re-validated against the database on every request**, so a demoted or deleted
|
|
user loses access immediately instead of keeping their old role until the token expires.
|
|
- **Role-based authorization** — admin-only endpoints (users, site mode, settings, auth providers)
|
|
are gated by a `requireRole` check, so a lower-privilege editor can't reach them.
|
|
|
|
**Mobile bearer auth**
|
|
|
|
- Native clients use `/api/v1/auth/mobile/*`: a short-lived **access token** (bearer JWT, validated
|
|
by the same middleware as the cookie) plus a long-lived, **server-stored, revocable refresh
|
|
token** that is **rotated on every refresh** (a replayed refresh token is single-use). Refresh
|
|
tokens are stored **hashed** (never in the clear); logout revokes one or all. Mobile login reuses
|
|
the same bot-scoring + backoff defenses as web, with single-request TOTP.
|
|
|
|
**Single sign-on (OAuth2 / OIDC)**
|
|
|
|
- Pluggable providers — built-in **Google** and **Discord** (endpoints fixed in code; admins supply
|
|
only client id/secret) plus fully-configurable **custom OIDC/OAuth2** providers, managed from the
|
|
**Authentication** admin panel. Only `enabled` + fully-configured providers are shown to users.
|
|
- **Link-only** by policy: an SSO login succeeds *only* if the external identity is already linked to
|
|
an existing account (linked by the user from **Account**). External identities are **never
|
|
auto-provisioned** — no one gains access without an account you created.
|
|
- The redirect flow is CSRF-protected with a signed, httpOnly, short-lived transaction cookie plus
|
|
**PKCE**; OAuth client secrets are **encrypted at rest** (AES-256-GCM) and never returned to any
|
|
client. SSO logins go through the same `sessionService`, so login/activity logging, RBAC, and bot
|
|
protection are identical to a local login.
|
|
|
|
**Login hardening**
|
|
|
|
- **Optional per-user TOTP two-factor** (opt-in, self-service on `/admin/account`). When enabled,
|
|
the password step issues only a short-lived, non-session `stage:'totp'` challenge; a session
|
|
cookie is granted only after the second factor verifies.
|
|
- **Login throttling** — `express-slow-down` + a hard rate cap + a separate per-IP exponential
|
|
backoff, with generic error messages that don't reveal whether the username exists.
|
|
- **Honeypot** field on the login form; submissions that fill it are treated as bots.
|
|
- **Bot-scoring + automatic IP ban** — weighted scoring of CMS-scanner paths and junk 404s (with a
|
|
periodic sweep of stale entries) bans hostile scanners; failed logins and honeypot hits feed the
|
|
score. Admins get visibility into this on the **Bot Activity** panel: currently banned IPs and a
|
|
recent-events feed (in-memory, most-recent-first), plus a logged emergency **unban** for false
|
|
positives — read + unban only, not a scoring-config surface.
|
|
|
|
**Uploads & input**
|
|
|
|
- Uploaded file extensions are derived from the **validated mimetype**, not the client-supplied
|
|
filename (prevents a disguised-extension upload).
|
|
- `express-validator` on all writes; usernames are validated **and** uniqueness-checked on update.
|
|
|
|
**Platform**
|
|
|
|
- `helmet`, admin routes `noindex` + `robots.txt` disallow, `trust proxy` for correct client IPs
|
|
behind a reverse proxy (see `TRUST_PROXY`), first admin seeded from env (no hardcoded credentials),
|
|
`.env` git-ignored. Passwords and request bodies are never logged. Email sends through Gmail
|
|
OAuth2 configured in the admin (refresh token stored AES-GCM-encrypted, never in env); the
|
|
contact form falls back to a `mailto:` link when unconfigured.
|
|
|
|
---
|
|
|
|
## Logging
|
|
|
|
Every log line goes to **both the console and a log file**, timestamped and leveled
|
|
(`error` / `warn` / `info` / `debug`):
|
|
|
|
```
|
|
2026-06-26T18:55:01.123Z INFO [server] listening on http://0.0.0.0:3000 ...
|
|
2026-06-26T18:55:09.880Z INFO [http] 192.168.1.40 admin POST /api/v1/auth/login 200 12 ms - 48 bytes
|
|
2026-06-26T18:55:14.402Z WARN [auth] login failed {"username":"root","ip":"192.168.1.40"}
|
|
2026-06-26T18:55:20.110Z ERROR [error] GET /api/v1/public/wiki -> 500 ... {"stack":"..."}
|
|
```
|
|
|
|
Captured: startup config banner, schema/seed steps, **HTTP access logs** (real client IP via
|
|
`trust proxy`, the authenticated admin, method/URL/status/time/size), login success/failure,
|
|
rate-limit hits, site-mode changes, all errors with stack traces, and graceful shutdown. Console
|
|
verbosity is `LOG_LEVEL`; the file keeps the fuller `FILE_LOG_LEVEL` record. In Docker the file is
|
|
bind-mounted to `./logs/app.log` and `docker compose logs -f app` shows the console stream.
|
|
|
|
---
|
|
|
|
## Deployment behind a reverse proxy
|
|
|
|
`docker compose up -d --build` exposes the `app` container on `0.0.0.0:3000` (no `127.0.0.1`
|
|
binding) so a reverse proxy — Pangolin, Nginx, Caddy, Traefik, etc. — can reach it. Point the
|
|
proxy at `app:3000` (or the host's `:3000` if the proxy runs outside Compose) and terminate TLS
|
|
there. Because `COOKIE_SECURE` defaults to `auto`, the admin login works both directly via the
|
|
LAN IP over HTTP **and** through the proxy over HTTPS — no config change needed. MariaDB stays on
|
|
the private Compose network (no published port by default); data persists in the `dbdata` volume,
|
|
uploads in `uploads`.
|
|
|
|
Set `TRUST_PROXY` so Express reads the real client IP from the proxy's `X-Forwarded-For` header
|
|
(see [Environment variables](#environment-variables)) — required for rate limiting, bot scoring,
|
|
and correct logging. Forward the standard `X-Forwarded-For` and `X-Forwarded-Proto` headers from
|
|
your proxy.
|
|
|
|
Minimal proxy examples:
|
|
|
|
```nginx
|
|
# Nginx
|
|
location / {
|
|
proxy_pass http://app:3000;
|
|
proxy_set_header Host $host;
|
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
proxy_set_header X-Forwarded-Proto $scheme;
|
|
}
|
|
```
|
|
|
|
```caddy
|
|
# Caddy — Caddyfile (automatic HTTPS; forwards X-Forwarded-* by default)
|
|
your.domain {
|
|
reverse_proxy app:3000
|
|
}
|
|
```
|
|
|
|
**Pangolin:** create a resource targeting `app:3000`; it forwards the required headers and
|
|
terminates HTTPS out of the box, so no extra configuration is needed.
|
|
|
|
---
|
|
|
|
## License
|
|
|
|
Runic Gateway is free software, licensed under the **GNU General Public License
|
|
v3.0 or later** — see [LICENSE.md](LICENSE.md).
|
|
|
|
Copyright (C) 2026 Runic Gateway
|
|
|
|
This program is free software: you can redistribute it and/or modify it under
|
|
the terms of the GNU General Public License as published by the Free Software
|
|
Foundation, either version 3 of the License, or (at your option) any later
|
|
version. It is distributed WITHOUT ANY WARRANTY; without even the implied
|
|
warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
General Public License for more details.
|
|
|
|
Contributions are welcome — please read [CONTRIBUTING.md](CONTRIBUTING.md) (note
|
|
the **AI-usage disclosure** requirement) and our
|
|
[Code of Conduct](CODE_OF_CONDUCT.md). Report vulnerabilities privately per
|
|
[SECURITY.md](SECURITY.md).
|