From 872352389d4fb1b1a6a5af88e618c5fb8d2b8d1d Mon Sep 17 00:00:00 2001 From: wtclaude Date: Tue, 11 Aug 2026 23:05:56 -0500 Subject: [PATCH] =?UTF-8?q?docs(website):=20close=20phase=203=20=E2=80=94?= =?UTF-8?q?=20slice=205,=20the=20fragment=20obligation,=20and=20the=20rust?= =?UTF-8?q?=20dry=20run?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Records the slice that closes the extraction, and adds the two documents phase 3 owed: docs/modules/uo/ and the module-rust dry run. **The slice found an obligation neither repo had built.** MODULE_API.md §2.8 and §6.1a settle the OpenAPI fragment in detail — a module ships one, core merges the fragments of started modules into /api/docs.json. Neither half existed, so the 72 URLs module-uo serves were in no spec at all. §2.8 and §6.1a now record what was built, including the four things settled while building it: the filename is fixed rather than declared, a module namespaces what it DEFINES and references core's shared schemas by core's name, the generator derives its prefixes from the module's own register() call, and swagger-autogen's diagnostics have to be captured because it reports a broken annotation and then prints Success. **§5.3 gains the design decision the frozen manifest actually made:** it is a SUBTRACTION, not a prefix filter. Generating the manifest without the module and then with it answers "what does the module serve" AND "did core lose anything", and the second is the one §1.2 promises to the shipped Android app. A module that shadowed a core route cannot appear as an addition anywhere. **BACKEND_DESIGN.md §4.0.1** is new: /api/docs.json is assembled per request, the two generated artifacts are core's alone, and the route count was still 228. **docs/modules/** is new, per §2.10 (module documentation aggregates here, not in module repos): docs/modules/uo/README.md orients a reader on what module-uo serves, owns and needs from an operator, and links out to the feature docs that already existed rather than restating them. **docs/modules/rust-dryrun.md** is phase 3's fourth acceptance criterion. A written, deliberately unimplemented module for Rust — chosen because it wipes monthly, runs several servers rather than one shard, identifies by Steam, and ships RCON so there is no sidecar to write. The contract generalises: same manifest, same seven registration calls, same schema rules, and six of the UI kit's seven members wanted by a game with nothing in common with the one the kit was curated from. It found one real gap — **a module cannot register an identity provider**, and "Sign in with Steam" is what a Rust community expects. Recorded as the first candidate for a future MODULE_API_VERSION bump rather than bolted on: an identity provider participates in session creation, and §2.7's link-only SSO policy has to survive it. Also: website-README.md refreshed from the repo (it was several changes stale), and three settled decisions added (18-20). Pairs with Module-uo#6 and website#141. Co-Authored-By: Claude --- README.md | 14 +- modules/rust-dryrun.md | 262 +++++++++++++++++++++++++++ modules/uo/README.md | 97 ++++++++++ website/BACKEND_DESIGN.md | 39 +++- website/MODULE_API.md | 69 +++++++ website/MODULE_SYSTEM.md | 143 ++++++++++++++- website/website-README.md | 372 +++++++++++++++++++++++++------------- 7 files changed, 866 insertions(+), 130 deletions(-) create mode 100644 modules/rust-dryrun.md create mode 100644 modules/uo/README.md diff --git a/README.md b/README.md index 9609db4..2bb803a 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,8 @@ so they live in one place, independent of either codebase. ## Layout ``` -website/ docs from the shard website (Node/Express + MariaDB + React/Vite) +website/ docs from the website core (Node/Express + MariaDB + React/Vite) +modules/ docs for installable game modules — one directory per module id link/ docs from the ServUO bridge (C# plugin + Rust sidecar + Node WS) android/ docs from the native Android client (Kotlin + Jetpack Compose) installer/ docs for the installer that deploys a shard's bridge components @@ -35,6 +36,17 @@ sidecar as a service, and hands you the values the website needs. | [website-README.md](website/website-README.md) | Snapshot of the website repo's README (setup/run reference) | | [PROJECT_TREE.md](website/PROJECT_TREE.md) | Auto-generated snapshot of the repo's tracked file layout | +### `modules/` + +Documentation for installable game modules aggregates here rather than in each module's repo +([MODULE_SYSTEM.md](website/MODULE_SYSTEM.md) §2.10). The website core knows nothing about any +particular game; a module is what makes it a site *for* one. + +| Doc | What it covers | +|---|---| +| [uo/](modules/uo/README.md) | **module-uo** — the Ultima Online module: what it serves, what it owns, and what an operator needs | +| [rust-dryrun.md](modules/rust-dryrun.md) | A written, deliberately unimplemented `module-rust` — the test that the module contract generalises past the game it was extracted from | + ### `link/` | Doc | What it covers | |---|---| diff --git a/modules/rust-dryrun.md b/modules/rust-dryrun.md new file mode 100644 index 0000000..8c9a401 --- /dev/null +++ b/modules/rust-dryrun.md @@ -0,0 +1,262 @@ +# `module-rust` — a dry run + +**Phase 3's fourth acceptance criterion** ([`../website/MODULE_SYSTEM.md`](../website/MODULE_SYSTEM.md) +§2.7). A written design for a module serving a **Rust** (Facepunch) community, taken far enough to +find out whether the contract generalises past the game it was extracted from — **and deliberately +not implemented.** A contract validated only against the module it was carved out of has not been +validated. + +The exercise is honest if it finds something. It found four things, one of which is a gap in the +contract that a real second module would hit on its first day. They are in *Findings* at the end; the +design comes first, because a finding is only worth anything with the design that produced it. + +Rust was chosen because it is unlike Ultima Online in the ways most likely to break assumptions: +it **wipes** every month, a community runs **several servers** rather than one shard, its identity +is **Steam**, and its server speaks a **protocol nobody has to write** — RCON over WebSocket, built +in. If the contract survives that, "game-agnostic" means something. + +> Nothing here re-specifies the contract. [`../website/MODULE_API.md`](../website/MODULE_API.md) is +> normative; this document only *uses* it. + +--- + +## 1. The manifest + +```json +{ + "id": "rust", + "name": "Rust", + "version": "0.1.0", + "coreApi": "^1.3.0", + "server": "server/index.js", + "client": { "entry": "client/dist/entry.js" }, + "schema": "server/db/schema.sql", + "purge": "server/db/purge.sql", + "mounts": { + "public": ["/rust"], + "admin": ["/rust"], + "player": ["/rust"] + }, + "extensions": ["admin.users.detail"], + "capabilities": ["servers", "wipes", "leaderboards", "map", "killfeed", "teams"] +} +``` + +One prefix per tier, named for the module rather than for a feature — the opposite of module-uo's +`/shard` + `/atlas` + `/uo-link`, and the better choice for anything new. Module-uo's prefixes are +what they are because §1.2 froze the URLs core already served; a module written today has no such +debt and should claim one obvious segment. It also sidesteps the collision surface entirely: +`/shard` is a word a second game might want, `/rust` is not. + +## 2. The server half + +```js +module.exports = function register(ctx, api) { + core.init(ctx) + + const publicRouter = require('./router/public/rust.router') + const adminRouter = require('./router/admin/rust.router') + const playerRouter = require('./router/player/rust.router') + const userExt = require('./router/admin/usersRust.router') + + api.registerRoutes({ + public: { '/rust': publicRouter }, + admin: { '/rust': adminRouter }, + player: { '/rust': playerRouter }, + }) + + api.registerExtension('admin.users.detail', userExt) + + api.registerNotificationStreams([ + { id: 'rust.wipe', label: 'Server wipes', + description: 'A server wiped — new map, new seed, everything reset.', + personal: false, requiresLinkedAccount: false }, + { id: 'rust.raid', label: 'Base raided', + description: 'Your base took damage while you were offline.', + personal: true, requiresLinkedAccount: true }, + ]) + + api.registerAnnounceLeg({ + leg: 'rust.ingame', + label: 'In-game chat', + dispatch: (post) => rcon.say(`[NEWS] ${post.title} — ${ctx.site.baseUrl}/news/${post.slug}`), + classify: (result) => (result.ok ? { outcome: 'done' } : { outcome: 'retry', error: result.error }), + }) + + api.onBoot(async () => { await rcon.connectAll() }) + api.onShutdown(async () => { await rcon.closeAll() }) +} +``` + +Everything above is a call the contract already has, used the way module-uo uses it. Two details are +worth pointing at: + +- **`rust.wipe` and `rust.raid` are namespaced**, with no grandfathering request. Module-uo's seven + bare stream ids are allowlisted because they were in `notification_subs` before the rule existed + (§6.5); a new module gets the rule, and the rule is exactly right. +- **The announce leg goes to in-game chat over RCON**, which is a one-shot delivery with retry — + `registerAnnounceLeg`, not `registerPostHook`. The distinction §2.4 draws holds up on a game that + has nothing in common with the one it was drawn for. + +### Tables + +`rust_servers`, `rust_wipes`, `rust_players`, `rust_player_stats`, `rust_teams`, `rust_events`, +`rust_bans`, `rust_maps`. All `rust_`-prefixed, all in one idempotent `schema.sql` fragment. + +**Every table that holds gameplay data carries a `wipe_id`.** That is the whole shape of the game in +one column: a leaderboard means "since the last wipe", a base means "on this map", and a player's +stats are per-wipe with an all-time rollup kept separately. It has no bearing on the contract — +core replays the fragment and never looks inside — but it is the first thing a UO-shaped mental +model gets wrong, and it is worth writing down for whoever builds this. + +### Talking to the game + +**No sidecar.** Rust ships RCON over WebSocket, so the module dials the server directly with the +token an admin saved, encrypted at rest through `ctx.secretBox`. + +This is the sharpest test of whether the module system's boundary is drawn in the right place, and it +passes: core has no opinion about how a module reaches its game. What core owns is that the module +never blocks a request on it, that its secrets are encrypted, and that a game being down degrades to +a page saying so. The *shard-dials-out* invariant that shapes +[`../link/PLAN.md`](../link/PLAN.md) is a property of ServUO — a game engine with no remote-control +surface, whose plugin must not stall on a socket — not of the platform. A game that ships RCON +already answers the question the sidecar exists to answer. + +The Integration Kit ([`MODULE_SYSTEM.md`](../website/MODULE_SYSTEM.md) §2.11) should say this +plainly, or its second reader will build a sidecar they did not need. + +## 3. The client half + +```js +registry.registerRoutes('rust', { + public: [ + { path: 'servers', element: }, + { path: 'servers/:id', element: }, + { path: 'servers/:id/map', element: }, + { path: 'leaderboards', element: }, + { path: 'wipes', element: }, + ], + player: [ + { path: 'account', element: }, + { path: 'stats', element: }, + ], + admin: [ + { path: 'servers', element: , gate: { roles: ['admin'] } }, + { path: 'ops', element: , gate: { roles: ['admin', 'moderator'] } }, + ], +}) + +registry.registerNav('rust', { + area: 'public', + items: [ + { label: 'Servers', to: '/rust/servers', group: 'Play', order: 10, icon: ServerIcon }, + { label: 'Leaderboards', to: '/rust/leaderboards', group: 'Play', order: 20, icon: TrophyIcon, + feature: 'leaderboards' }, + { label: 'Wipe schedule', to: '/rust/wipes', group: 'Play', order: 30, icon: CalendarIcon }, + ], +}) + +registry.registerFeatureProvider('rust', 'rust', useRustFeatures) +registry.registerExtension('rust', 'admin.users.detail', LinkedSteamAccounts) +``` + +`Play` is a group core does not have; §3.3 appends an unknown group rather than dropping the items, +so this works and lands at the end of the nav — where an operator can move it, because a module row +is an ordinary row once it is interleaved. + +The pages need `PublicLayout`, `PageHeader`, the three `PageState` components, `useAsync` and +`useAuth`: **six of the kit's seven members**, and the seventh (`useSite`) on the wipe-schedule page +for the site's timezone. A second game, unrelated to the first, wanting exactly what the kit +contains is the strongest evidence available that §3.4 was curated at the right altitude. + +The map view is the one page that wants something the kit does not have — a pan/zoom canvas. It +bundles one, which is the answer §3.4 already gives ("everything else a module bundles itself"), and +it costs the chunk about 40 KB. + +--- + +## Findings + +### 1. A module cannot register an identity provider — and Rust's identity is Steam + +The gap. Module-uo proves account ownership with an in-game `[link` command that issues a one-time +code; the website confirms it with the sidecar. Nothing about that needs core's auth layer, so +nothing in `api` ever needed to touch it. + +**Rust's answer is Steam OpenID**, and every Rust community expects "Sign in with Steam". Core has an +SSO layer — `authProviders`, `userIdentities`, OAuth2/OIDC providers behind a registry — and +[`MODULE_API.md`](../website/MODULE_API.md) §2.4 offers a module no way in. There is no +`registerAuthProvider`, and §2.7 forbids reaching for one. + +A Rust module can still ship: it can copy the UO shape, issuing a code in-game and matching it on +the website. That works, it is one screen worse, and it leaves the community's obvious expectation +unmet. + +**This is not a defect in what was built** — it is the boundary of what was specified, found exactly +where a dry run is supposed to find it. It is worth stating precisely, because whoever adds it has to +answer a question the current policy already has an opinion about: **SSO is link-only by design**, an +external identity must already be linked to an existing account, and identities are never +auto-provisioned. A Steam provider a module registers must inherit that, not route around it. It is +also *not* a small addition: an identity provider participates in session creation, which is the one +part of core a module must never be able to weaken. + +Recommendation: leave it out of v1 of the contract and record it here as the first candidate for +`MODULE_API_VERSION` 1.4 or 2.0, specified deliberately rather than bolted on when someone needs it. + +### 2. "One module, one game" is not the same as "one module, one server" + +A UO community runs one shard. A Rust community runs four or five, wipes them on different +schedules, and every page is a per-server view. + +The contract is silent on this, and silence turns out to be right: multiplicity lives entirely in the +module's own tables and route parameters (`/rust/servers/:id`). Core's mount prefixes, capabilities +and state machine are per-**module**, and none of them wanted to be per-server. + +Worth recording only because it looks like a problem until you try it — and because it is the shape +that would have broken a contract designed around "the shard" as a singular noun. The phase-2 +inversions that removed core's opinions about game content (the push catalog, the announce legs, +`mapEvent` dropped) are why it does not. + +### 3. The event catalog generalises; the *retention* assumption does not + +`rust_events` is the twin of `shard_events`, and the ingest shape carries over unchanged. + +What does not carry over is that a UO event log grows forever while a Rust one is **truncated every +wipe**. That is module-internal — but it lands on something core does own: the schema fragment +**must not** be where that truncation happens. §2.6's leading-verb allowlist (`CREATE`, `ALTER`, +`INSERT`, `UPDATE`) already forbids `DELETE` and `TRUNCATE` in a fragment, precisely because the +fragment is replayed on **every boot** and would empty the table each restart. + +So a wipe is a runtime operation on a module route, not a schema one. The allowlist was written for a +different reason — the phase-2 note says "the file replays every boot, so TRUNCATE/DELETE would empty +a table each restart" — and it correctly forbids the first mistake this module's author would make. +A rule that catches a case it was not written for is a rule at the right altitude. + +### 4. `capabilities` earns its keep the moment there are two modules + +With one module, `capabilities` reads like decoration — core never interprets one, and the SPA knows +what it registered. With two, it is the only thing a *client* can ask. + +The Android app is the case: it feature-detects against `GET /api/v1/public/modules` and must render +a site whose module it has never heard of. `["servers", "wipes", "leaderboards"]` tells it there is +nothing shard-shaped here without it having to know what `rust` means, and §2.9's rule — treat an +unknown capability as absent, never infer a route from one — is what keeps that from becoming a +second, worse route table. + +No change needed. Recorded because the design decision looked over-engineered with one module and is +load-bearing with two. + +--- + +## Verdict + +**The contract generalises.** A second game, chosen for how little it shares with the first, is +served by the same `module.json`, the same seven registration calls, the same schema-fragment rules, +the same client registry and the same UI kit — with one genuine gap (identity providers), one +non-issue that looks like a gap (multiple servers), and two places where a rule written for one +reason turns out to cover another. + +The gap is worth having found before something was built on top of it, which is what a dry run is +for. What it does **not** establish is that someone outside this org could build this module from the +documentation alone — that is the Integration Kit's acceptance test (§2.11), and it stays untested +until a person who did not write any of this does it. diff --git a/modules/uo/README.md b/modules/uo/README.md new file mode 100644 index 0000000..85a70f1 --- /dev/null +++ b/modules/uo/README.md @@ -0,0 +1,97 @@ +# module-uo — the Ultima Online module + +Everything specific to Ultima Online that the website serves. Code: +[RunicGateway/Module-uo](https://gitea.whitlocktech.com/RunicGateway/Module-uo). Module id `uo`, +installed at `modules/uo/` on the website's modules volume. + +This page is **orientation**: what the module is, what it serves, what it owns, and what an operator +has to know. It is not a second copy of the contract — [`../../website/MODULE_API.md`](../../website/MODULE_API.md) +is normative for anything about how a module and core fit together, and the module's own repo is +authoritative for its file layout. Where this page and either of those disagree, they win. + +Module documentation aggregates here rather than in module repos +([`MODULE_SYSTEM.md`](../../website/MODULE_SYSTEM.md) §2.10), so the feature docs that describe what +these routes *mean* are the ones that already existed and did not move: + +| Doc | What it covers | +|---|---| +| [`SHARD_VISIBILITY.md`](../../website/SHARD_VISIBILITY.md) | Who sees which shard data — the admin-configurable audience framework | +| [`SPAWN_ATLAS.md`](../../website/SPAWN_ATLAS.md) | The bestiary / spawn atlas, parsed from the shard's own ServUO tree | +| [`MARKETPLACE.md`](../../website/MARKETPLACE.md) | The player-vendor index | +| [`CLILOCS.md`](../../website/CLILOCS.md) | UO's id → name table | +| [`UOFIDDLER.md`](../../website/UOFIDDLER.md) | Operator runbook for extracting the cliloc table and creature art | +| [`../../link/PLAN.md`](../../link/PLAN.md), [`../../link/INTEGRATION.md`](../../link/INTEGRATION.md) | The wire protocol this module speaks to the sidecar | + +--- + +## What it serves + +**72 URLs**, frozen in the module's own +[`routes.manifest.json`](https://gitea.whitlocktech.com/RunicGateway/Module-uo/src/branch/main/routes.manifest.json) +and documented in its +[`swagger-fragment.json`](https://gitea.whitlocktech.com/RunicGateway/Module-uo/src/branch/main/swagger-fragment.json), +which core merges into `/api/docs.json` while the module is running. + +| Mount | Tier | What | +|---|---|---| +| `/api/v1/public/shard` | public | Status, activity feed, economy, presence, houses/IDOC, champs, guilds, governors, points boards, the player-vendor market, the ruleset, and the SSE event stream | +| `/api/v1/public/atlas` | public | The bestiary: creatures, regions, landmarks, champion altars, and what is loaded | +| `/api/v1/admin/shard` | admin | Shard ops (kick/ban/broadcast/pages), the visibility config, atlas and cliloc imports, market admin, account links | +| `/api/v1/admin/uo-link` | admin | The sidecar connection config, live status, and the town crier | +| `/api/v1/player/shard` | player | A player's own linked accounts: rosters, character sheets, vendors, sales | +| `/api/v1/admin/users/:id/shard/*` | admin | Six routes filling core's `admin.users.detail` **extension slot** — a module's routes hanging off a *core* resource, since core owns the user | + +Every URL is byte-identical to the one core served before the extraction. That is the whole point of +moving the code and not the paths: the shipped Android app calls +`POST /api/v1/admin/shard/kick`, the Discord bot reads `/api/v1/public/shard/*`, and neither knows a +module answers now. + +**In the SPA**: twelve public pages under `/uo/*`, the admin views under `/admin/uo/*`, the player +views under `/player/uo/*`, three extension-slot fills, and the nav rows for all of them, interleaved +into core's nav so an operator can reorder, relabel or hide them like any other row. + +## What it owns + +- **27 database tables** — 26 `shard_*` plus `uo_link_config`. Created by an idempotent + `schema.sql` fragment core replays on every boot, after its own schema. The `shard_`/`uo_link_` + prefixes are **grandfathered** ([`MODULE_API.md`](../../website/MODULE_API.md) §6.5): the rule for + a new module is `_`, and these predate it. +- **Seven push notification streams** and the announce leg `towncrier`, likewise grandfathered — + they are stored in `notification_subs` and `announce_job_legs.leg` and read by the shipped Android + app, so renaming them would be a data migration plus a client break. +- **Two settings rows**: `game_account_signup` and the shard's protocol pin. +- **The public-safety filter.** Which shard event kinds may reach the *public* SSE stream is decided + here, not in core — the kinds, the streams and the filter are one file that moves together. + +## For an operator + +**Installing.** A release is `module-uo-.tar.gz` plus a manifest carrying its `sha256`. +Unpack it as `modules/uo/` on the website's modules volume (or use the admin Modules screen when +phase 4 lands) and restart. **You never build anything** — the client chunk is prebuilt and the one +runtime dependency ships inside the tarball. + +**Connecting it to a shard.** The module needs the +[uo-link sidecar](https://gitea.whitlocktech.com/RunicGateway/link) running next to the ServUO +shard. Deploy that with the [installer](https://gitea.whitlocktech.com/RunicGateway/installer) — +[`../../installer/INSTALL.md`](../../installer/INSTALL.md) is the operator guide — and paste the four +values it prints into **Admin → Shard**. The token is encrypted at rest and write-only in the API. + +**Nothing requires the shard to exist.** With no sidecar configured the site renders normally and +shows the shard offline. That is the same bargain the module system makes one level up: a module +that fails to load never takes the site down. + +**Environment variables** — four, all optional, all read by the module and documented in its README: +`UOLINK_BASE_URL`, `UOLINK_WS_URL`, `UOLINK_PROTOCOL`, `TOWNCRIER_DURATION_SEC`. They live in the +Compose `.env`, because that is what reaches the container. + +## Compatibility + +`module.json` declares a `coreApi` semver range, checked at boot against core's +`MODULE_API_VERSION`. A mismatch fails **loudly** — the module is marked `startup_failed` and the +site comes up without it. This is a separate number from `PROTOCOL_VERSION`, which versions the +shard wire and says nothing about a website module. + +The module's CI clones core at a **pinned** ref ([`MODULE_API.md`](../../website/MODULE_API.md) §5.3) +to generate its frozen manifest. Bumping that pin is a deliberate commit that says which core the +module was last proved against — not a tracking reference that turns core's unrelated changes into +red Xes here. diff --git a/website/BACKEND_DESIGN.md b/website/BACKEND_DESIGN.md index 59e4ab1..d463eb2 100644 --- a/website/BACKEND_DESIGN.md +++ b/website/BACKEND_DESIGN.md @@ -731,8 +731,15 @@ are authoritative, and they answer different questions: | Artifact | Source of truth for | Generated by | |---|---|---| -| `server/routes.manifest.json` — mirrored as [api-route-inventory.json](./api-route-inventory.json) | **What URLs exist.** 228 public routes + 2 on the internal listener, sorted, method + path only. | `npm run routes:manifest`, by walking the live Express stack | -| `server/swagger/swagger-output.json` — served at `/api/docs` | **What each route means.** Parameters, bodies, response codes, security. | `npm run swagger`, from `#swagger.*` annotations | +| `server/routes.manifest.json` — mirrored as [api-route-inventory.json](./api-route-inventory.json) | **What URLs CORE serves.** 158 public routes + 2 on the internal listener, sorted, method + path only. | `npm run routes:manifest`, by walking the live Express stack | +| `server/swagger/swagger-output.json` — merged into `/api/docs` | **What each core route means.** Parameters, bodies, response codes, security. | `npm run swagger`, from `#swagger.*` annotations | + +Both are **core's**. An installed module's routes are in neither: they are in that module's own +frozen manifest and its `swagger-fragment.json`, in its own repo, and core merges the fragment into +`/api/docs.json` at request time (§4.0.1). So on a running instance the served document describes +more than the committed one does, which is the intended arrangement rather than a drift — +`swagger-output.json` has to regenerate identically on any machine, whatever happens to be +installed on it. The split is deliberate: Swagger is annotation-derived, so an unannotated route is invisible in it and it churns whenever a description is reworded — it documents *intent*. The manifest is introspection- @@ -764,6 +771,34 @@ authenticated endpoints silently. Names are a hint only — `requireRole(...)` r arrow and cannot be observed — but a *missing* `requireAuth` is unambiguous, and the server test suite asserts every `/admin/**` and `/player/**` route still carries it. +#### 4.0.1 `/api/docs.json` is assembled per request + +`GET /api/docs.json` and the Swagger UI at `/api/docs` do not serve `swagger-output.json` directly. +`swagger/docsSpec.js` merges the `swagger-fragment.json` of every **started** module over it first, +cached on the module loader's state version and rebuilt when a module's state moves. + +It exists because swagger-autogen is static analysis: it parses `src/app.js` as text and follows the +literal `app.use(…)` chain, which reaches neither an installed module (required by a filesystem loop, +from a volume that had nothing on it when the image was built) nor an extension slot (whose router is +created empty by `declareSlot()` and filled later). Slots are handled at generation time by +`swagger/slotSpecs.js` and are therefore *in* the committed file; modules cannot be, because core +never has their sources. + +Three rules, all from [`MODULE_API.md`](MODULE_API.md) §6.1a: + +- **`started` only.** A `registered`, `disabled` or `startup_failed` module's paths are absent — + documenting a route that answers 503 or 404 sends a client somewhere it cannot go. +- **Core wins every key collision**, in all three merged sections (`paths`, `tags`, + `components.schemas`); the collision is logged and the module's version dropped. This is what makes + the naming rule work: a module namespaces the schemas it *defines* (`UoShardStatus`) and references + core's shared ones (`Error`, `ValidationError`) by core's name, and both resolve in the merged + document. +- **A bad fragment costs that module its paths and nothing else.** Missing, unreadable or not JSON is + logged and skipped; `/api/docs.json` still answers with everything else. + +The committed spec is never mutated — it is a `require()`d JSON module, so an in-place merge would be +permanent for the life of the process *and* cumulative across rebuilds. + ### /auth (auth/index.js → the capability routers in §2) No group gate — `/auth` is where an anonymous caller becomes authenticated. The authenticated parts diff --git a/website/MODULE_API.md b/website/MODULE_API.md index 0755c6b..95451bd 100644 --- a/website/MODULE_API.md +++ b/website/MODULE_API.md @@ -435,6 +435,33 @@ fragments of started modules into `/api/docs.json`; the full reasoning and the c §6.1a. In short: fully-qualified paths, namespaced schema keys, module CI fails if a registered route has no path in the fragment, and core wins every key collision. +**Built in phase 3 slice 5** (module-uo#6, website#141). Four things settled while building it, all +of which a second module inherits: + +- **The filename is fixed here, not declared in `module.json`.** `swagger-fragment.json` in the + bundle root, like `module.json` itself — so a module cannot point core at some other file, and + core's loader has one path to check. A module that ships none is simply absent from the merged + document: whether it registered routes without documenting them is the *module's* CI to answer, + where the routes are known. Core cannot tell a module with no routes from one that forgot. +- **Namespace what you DEFINE; reference core's by core's name.** `UoShardStatus` is defined by the + module; `#/components/schemas/Error` and `ValidationError` are referenced and **not** redefined. + Both resolve in the merged document, which is the only place both halves exist — and shipping a + copy of `Error` would be a collision core drops, arriving at the same result the expensive way. + This is the practical form of "core wins": it makes the two cases feel different in the source, + which is where the mistake would otherwise be invisible. +- **Generate the fragment from the module's own registrations.** swagger-autogen needs a *file* and + cannot follow `api.registerRoutes`, so the module's generator runs its own `register()` against a + recording `api` and resolves each router back to its source through `require.cache`. A mount prefix + then exists in exactly one place. The two values it cannot derive — the tier base paths and the + slot's mount, both §2.4's — are checked against a real core by the §5.3 job rather than trusted. +- **swagger-autogen reports a broken annotation and then succeeds.** It `console.error`s "Syntax + error" or "out of structure", drops that annotation, and prints `Success`. Both repos' generators + now capture those diagnostics and fail on them, which found six annotations documenting less than + they claimed. Two ways one breaks: an object literal a brace short, and a `"` or backtick inside a + single-quoted description (the tool re-quotes both to `'` before evaluating, ending the string + early). A third, which nothing but a rendered page catches: an escaped apostrophe survives + literally, because the annotation is not evaluated as JavaScript. + ### 2.9 What core publishes about a module `GET /api/v1/public/modules` — anonymous, database-free, never site-mode gated. @@ -1088,6 +1115,29 @@ and runs core's own `routeManifest.js`. Nothing else proves the URLs a module cl actually serves — a manifest frozen by hand goes stale silently, and the failure it would have caught is a route that moved. +**Built in slice 5, and what it does with that core is a SUBTRACTION.** The job generates the +manifest without the module and then with it; the difference is what the module serves. Filtering the +combined manifest by the module's prefixes would have answered only "what does the module serve". +Subtracting also answers **"did core lose anything"** — and a module that shadowed or displaced a +core route cannot appear as an addition anywhere, so that is the only way to see it. Three checks +come out of one diff: + +1. the added routes match the module's committed `routes.manifest.json`; +2. **no route of core's was removed or changed**, which is `MODULE_SYSTEM.md` §1.2's promise; +3. every added route has an operation in `swagger-fragment.json`, and every operation is an added + route — §2.8's coverage requirement, answered against a running app rather than against a table. + +The third is why this job matters beyond the manifest: everything else in a module's repo compares +two strings that live in that repo. This compares a URL the module registers against a URL a real +Express app reports serving, which is the only thing that can catch a fragment that is internally +consistent and describes nothing. + +Also learned here: **copy the module into the core checkout, never symlink it.** The loader filters +its scan with `entry.isDirectory()`, which reports a link as a link and skips it silently — the +manifest then comes out with no module routes and the diff looks like a module that registered +nothing. And the client chunk must be **built before** the copy: `client.entry` is validated during +the manifest step of the scan, so a missing chunk is a load failure, not a warning. + Pinning the ref rather than tracking `edge` is what keeps this from being a source of unexplained red Xes: core moves for reasons that have nothing to do with the module, and a bump is then a deliberate commit that says which core the module was last proved against. @@ -1142,6 +1192,25 @@ shipping one with the same name; the collision is logged and the module's versio `swagger-output.json` itself stays exactly what core's own routes generate, so `npm run swagger` remains reproducible on any machine regardless of what is installed. +Built as `server/swagger/docsSpec.js` (website#141). Three properties that are contract rather than +implementation, because each one is a way the obvious version is wrong: + +- **The committed spec is never mutated.** It is a `require()`d JSON module, so a merge in place + would be permanent for the life of the process *and* cumulative across rebuilds — a module's paths + outliving its own uninstall. Every rebuild starts from a structural copy. +- **The Swagger UI is built per request too**, not bound once while `app.js` is still being required. + Bound at require time it would show core's routes for the life of the process while + `/api/docs.json` showed the merged set — two documents at two URLs, disagreeing. +- **A bad fragment costs that module its paths and nothing else.** Missing, unreadable or not JSON is + logged and skipped; the document still answers. That is §4.4's bargain — one module's failure is + never the site's — and a docs page that 500s is strictly worse than one missing a module's routes. + +`started` only, matching `clientEntryUrls()` rather than `clientChunks()`: the document is built when +it is asked for, at which point the state is known, and documenting a module that 503s every one of +those paths sends a client somewhere it cannot go. The cache key is a new +`modules.version()` — a counter the loader bumps on every state *change*, which says nothing about +which module moved or where to. + ### 6.2 The client contract is much larger than §2.1 says §2.1 lists three client registration calls and nothing else, implying React and the router are all a diff --git a/website/MODULE_SYSTEM.md b/website/MODULE_SYSTEM.md index aa329e6..29fbe7a 100644 --- a/website/MODULE_SYSTEM.md +++ b/website/MODULE_SYSTEM.md @@ -730,7 +730,9 @@ navigate a module it does not contain, on a deployment that builds nothing — a while no existing URL has moved. The exit criterion held: `routes.manifest.json` went 229 → 230 across the whole phase, and the one added line is PR 6's deliberate `GET /api/v1/public/modules`. -**Phase 3 — Extract `module-uo`.** Moves out of `website/`: the 8 model directories and their 25 +**Phase 3 — Extract `module-uo`. COMPLETE 2026-08-11**, in six slices; the record of each is in +§2.7.1 and all four acceptance criteria are met (the table at the end of slice 5). Moves out of +`website/`: the 8 model directories and their 25 tables; the nine UO `utils/` files plus `newsGump.js`; the 13 router/controller files; `scripts/importSpawnAtlas.js` and `db/spawnAtlas.art.json`; `usersShard.controller.js` **minus `getUser`** (§1.9); the shard-derived half of `notificationStreams.js` and the town-crier leg of @@ -782,7 +784,7 @@ Each slice is one `module-uo` PR (adds), one `website` PR (deletes), and one `do | 2 | **Client extension slots** | Core only, and the one slice that adds rather than moves: the client twin of `declareSlot`/`registerExtension` (API §3.7), the `site.footer.status` and `admin.users.detail` slots, and core filling both itself. `module-uo` untouched. | | 3 | **The whole client half** | **35 files / 5,332 lines** (measured; the 51/~3,700 above was counted differently) — all twelve public pages (`Shard`, `ShardActivity`, `Rules`, `Atlas`, `AtlasCreature`, `ChampSpawns`, `Market`, `MarketVendor`, `Governors`, `Guilds`, `Houses`, `Leaderboards`) under `/uo/*`, every admin and player view under `/admin/uo/*` and `/player/uo/*`, `PlayersOnline`, `VendorSales`, `CharacterStats`, `GameAccounts`, the `data/` and `lib/` UO leaves, the public nav rows, the feature provider, and all three slot fills. | | 4 | **De-UO core's copy** | `About`, `Screenshots`, `Website`, `Status`, `Wiki`, `SiteFooter`'s prose, `heroLayout`'s defaults, `brand.js`'s tagline + description, `db/seed.js`'s wiki copy, two user-visible `NavEditor` strings, the comments in `navOverrides.js`, and the 190 dead lines of `api/client.js`'s `shard`/`atlas` namespaces — plus the two settings rows core seeded for module-uo, and the §5.2 CI check that keeps all of it out. `README.md` deliberately deferred to slice 5. | -| 5 | **Close the phase** | `module-uo`'s frozen route manifest and release workflow; `docs/modules/uo/` and `docs/modules/rust-dryrun.md` | +| 5 | **Close the phase** | `module-uo`'s frozen route manifest and release workflow; `docs/modules/uo/` and `docs/modules/rust-dryrun.md` — **plus the OpenAPI fragment on both sides**, which §2.8/§6.1a had settled and neither repo had built, and the `README.md` slice 4 deferred | ##### Why the server half cannot be sliced — found 2026-08-11, before writing any of it @@ -1229,6 +1231,140 @@ learned by getting things wrong first is worth recording: core's README is a rewrite that belongs with the phase-closing documentation pass rather than half-done inside a code slice. +#### Slice 5 — closing the phase, and the obligation nobody had noticed + +Module-uo#6 + website#141 + docs#140 (2026-08-11). The slice table words this one as packaging and +documentation: the module's frozen manifest and release workflow, `docs/modules/uo/` and the +`module-rust` dry run. It is also where a **contract obligation that had never been built on either +side** surfaced, and closing it was the larger half. + +##### The OpenAPI fragment existed only on paper + +[`MODULE_API.md`](MODULE_API.md) §2.8 and §6.1a settle it in detail: a module ships +`swagger-fragment.json`, core merges the fragments of *started* modules into `/api/docs.json` at +request time, core wins every collision. **Neither half was written.** Module-uo's 417 `#swagger` +annotations came across in slice 1 and went nowhere; core's `swagger/mergeSpec.js` named the +request-time caller in its own file header and that caller did not exist. The result was that the 72 +URLs module-uo serves were **in no OpenAPI spec at all** — this repo's standing rule (never ship a +route that isn't in the spec) broken by the extraction rather than by a route. + +It was folded into this slice rather than deferred to Phase 4, because Phase 3 closing with 72 +undocumented routes would have made the phase's own exit criteria untrue. + +**The generator derives its prefixes rather than listing them.** `swaggerFragment.js` runs the +module's own `register()` against a recording `api` and asks `require.cache` which file each router +object came from — so a mount prefix lives in `server/index.js` and nowhere else, and swagger-autogen +(which needs a *file*, and cannot follow `api.registerRoutes`) gets pointed at the right one. The two +values it cannot derive, the tier base paths and the extension slot's mount, are §2.4's normative +table — and they are not taken on trust: the frozen-manifest job checks every generated path against +a real core. + +**Schemas are namespaced; core's are referenced by core's name.** The 31 UO schemas moved out of +core's `swagger.js` as `Uo…`, because core wins a collision and an un-namespaced `ShardStatus` from a +second game's module would lose to or clobber this one. But the annotations keep pointing at +`#/components/schemas/Error` and `ValidationError` **without** redefining them: those resolve in the +merged document, which is the only place both halves exist. Shipping a copy would be a collision core +correctly drops. Both directions verified against a live merge — 197 paths, no dangling `$ref`. + +##### The frozen manifest is a subtraction, not a filter + +§5.3 settles that the module's CI clones core at a pinned ref. What it does with that core is the +design decision this slice made: generate the manifest **without** the module and then **with** it, +and take the difference. + +Filtering the combined manifest by the module's prefixes would have answered "what does the module +serve". Subtracting answers that *and* "did core lose anything" — and the second is the one §1.2 +promises to the shipped Android app and the Discord bot. A module that shadowed or displaced a core +route cannot show up as an addition anywhere; it shows up here as a **removal**. Result: 72 routes +added, **0 removed**. + +That job is also the only place in the module's repo where a claim meets ground truth. Everything +else there compares two strings in the same repository; this compares a URL the module registers +against a URL a real Express app reports serving, which is what makes the fragment's coverage check +meaningful rather than self-referential. + +##### Releases: the version is declared, not computed + +`link` and `installer` both compute the next version from conventional-commit subjects. This module +does not, because it already has one authoritative version — `module.json`'s, which is what core +records in `installed_modules`, shows on the admin screen, and sits beside the `coreApi` range a bump +has to be weighed against. Two sources for one number is how they drift. So: a merge to `main` that +leaves `module.json` at a version with no release yet publishes one, and bumping the version is an +ordinary reviewed PR. + +The workflow **never writes to a branch** — it tags and publishes — so `main` needs no push +exception. That is the installer's model, adopted for the reason it was adopted there. The bundle is +assembled from an **include** list rather than an exclude list, because an exclude list ships +whatever it forgot. + +##### Six dropped annotations across two repos, and the tool that says `Success` + +swagger-autogen reports an annotation it cannot parse and **then prints `Success` in green**, having +skipped it. Nothing was listening in either repo. Making its diagnostics fatal immediately found: + +- **core**: `POST /api/v1/admin/invites` and `POST /api/v1/auth/invite/:token/accept`, both + documented with an **empty request body**, both since the day they were written; +- **module-uo**: the same brace-short mistake twice more, plus two descriptions whose inner quoting + the tool cannot survive — it re-quotes `"` and a backtick to `'` before evaluating, so either + inside a single-quoted description ends the string early. A typed, described query parameter had + been silently demoted to an untyped one. + +This is the same class as §6.1's silent drop, with one difference: the tool did say something. + +**And a seventh defect only a browser could show.** Nineteen descriptions carried an escaped +apostrophe — correct JavaScript, and wrong here, because swagger-autogen does not evaluate the +annotation as JS. The backslash survived into the fragment and Swagger UI rendered it verbatim to a +reader, mid-sentence. The fragment was valid JSON, the paths were right, every test passed. Only +opening the page found it: the §7.7 lesson, in a seam that has nothing to do with the client chunk. + +##### The last of core's UO copy + +`README.md`'s 48 mentions (the architecture diagram is now core + a module + "the game", and +*Shard integration (uo-link)* is now *Modules*), the four orphan tags and 31 orphan schemas in +`swagger.js`, `info.description`'s "a private Ultima Online shard", and `TOWNCRIER_DURATION_SEC` + +`UOLINK_*` in the two `.env.example`s — module-read, never core-read, and now documented in the +module's own README rather than half-copied in core's. + +Auditing the tag list for the four orphans turned up the same defect pointing the other way: five +tags **used** by core routes and never declared (`Admin · Email`, `Admin · Invites`, +`Admin · Moderation`, `Admin · Pages`, `Auth · Me`). + +##### The dry run + +[`../modules/rust-dryrun.md`](../modules/rust-dryrun.md) — Phase 3's fourth acceptance criterion, and +the only one that could fail in an interesting way. A written `module-rust`, for a game chosen +because it shares almost nothing with UO: it **wipes** monthly, a community runs several **servers** +rather than one shard, its identity is **Steam**, and its server ships **RCON** so there is no +sidecar to write. + +The contract generalises — same manifest, same seven registration calls, same schema-fragment rules, +same client registry, and **six of the UI kit's seven members** wanted by a game with nothing in +common with the one the kit was curated from. + +It found one real gap: **a module cannot register an identity provider**, and "Sign in with Steam" is +what a Rust community expects. A Rust module can still ship by copying UO's in-game-code flow, one +screen worse. Recorded as the first candidate for a future `MODULE_API_VERSION` bump rather than +bolted on now — an identity provider participates in session creation, which is the one part of core +a module must never be able to weaken, and §2.7's link-only policy has to survive it. + +It also found that two rules written for one reason cover another: §2.6's leading-verb allowlist +(written because a fragment replays every boot) correctly forbids the wipe-truncation a Rust author +would put in their schema, and `capabilities` — decoration with one module — is the only thing the +Android app can ask about a module it has never heard of. + +##### Phase 3 is complete + +Six slices, 2026-08-11. Core is 158 routes and knows nothing about any game; module-uo is 72 routes, +27 tables, 40 server files and 35 client files in its own repo, releasable, and documented in +[`../modules/uo/`](../modules/uo/README.md). Every acceptance criterion in §2.7 is met: + +| # | Criterion | Where it is proved | +|---|---|---| +| 1 | No UO identifier in core | `npm run check:modules`, first step of server-tests | +| 2 | Zero internal imports from module into core | `npm run check:imports`, module CI | +| 3 | Route manifest diff is only the deliberate move | core 158 + module 72, **0 core routes removed** | +| 4 | A written `module-rust` dry run | [`../modules/rust-dryrun.md`](../modules/rust-dryrun.md) | + **Phase 4 — Delivery.** The admin-panel Modules screen (install, enable, disable, retry, purge, `startup_failed` with its recorded reason) and the Docker-environment path from §2.5. Deliberately last, so loader, packaging, schema and chunk-loading problems are not all being debugged at once. @@ -1381,3 +1517,6 @@ row for it — when it has content, not while it is an empty repo. | 15 | Criterion 1's grep reads **code, not prose**; core's UO copy is rewritten in its own slice instead | API §5.2, §2.7.1 | | 16 | `module-uo`'s CI checks core out at a **pinned ref** to generate its frozen route manifest | API §5.3 | | 17 | The `module-rust` dry run lands as `docs/modules/rust-dryrun.md`; the Integration Kit links to it | §2.7.1, §2.11 | +| 18 | A module's frozen manifest is the **difference** between a core without it and the same core with it — never a prefix filter | API §5.3 | +| 19 | A module's release version is **declared** in `module.json`, not computed from commit subjects; the workflow tags and publishes and never writes to a branch | §2.7.1 | +| 20 | A module namespaces the schemas it **defines** and references core's shared ones by core's name | API §2.8, §6.1a | diff --git a/website/website-README.md b/website/website-README.md index 31c7fe1..0d21c09 100644 --- a/website/website-README.md +++ b/website/website-README.md @@ -1,18 +1,28 @@ # Runic Gateway Website -Public site, wiki, and protected admin panel for a private Ultima Online shard — a -full-stack app in one repo. Branding is instance-configurable via `BRAND_*` (see -[Branding](#branding)); **UOMysticmoon** is the first instance. +[![Bugs](https://sonar.whitlocktech.com/api/project_badges/measure?project=runic-gateway-website&metric=bugs&token=sqb_d3593f26ac5663cd3e666039b7038f3248e8df50)](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website) +[![Code Smells](https://sonar.whitlocktech.com/api/project_badges/measure?project=runic-gateway-website&metric=code_smells&token=sqb_d3593f26ac5663cd3e666039b7038f3248e8df50)](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website) +[![Duplicated Lines (%)](https://sonar.whitlocktech.com/api/project_badges/measure?project=runic-gateway-website&metric=duplicated_lines_density&token=sqb_d3593f26ac5663cd3e666039b7038f3248e8df50)](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website) +[![Lines of Code](https://sonar.whitlocktech.com/api/project_badges/measure?project=runic-gateway-website&metric=ncloc&token=sqb_d3593f26ac5663cd3e666039b7038f3248e8df50)](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website) +[![Security Hotspots](https://sonar.whitlocktech.com/api/project_badges/measure?project=runic-gateway-website&metric=security_hotspots&token=sqb_d3593f26ac5663cd3e666039b7038f3248e8df50)](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website) +[![Security Rating](https://sonar.whitlocktech.com/api/project_badges/measure?project=runic-gateway-website&metric=security_rating&token=sqb_d3593f26ac5663cd3e666039b7038f3248e8df50)](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website) +[![Vulnerabilities](https://sonar.whitlocktech.com/api/project_badges/measure?project=runic-gateway-website&metric=vulnerabilities&token=sqb_d3593f26ac5663cd3e666039b7038f3248e8df50)](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website) + +Public site, wiki, and protected admin panel for a game community — a full-stack app +in one repo. Everything specific to a *particular* game lives in an installable +module, not here. Branding is instance-configurable via `BRAND_*` (see +[Branding](#branding)); **UOMysticmoon**, an Ultima Online shard, is the first +instance, and its game half is +[RunicGateway/Module-uo](https://gitea.whitlocktech.com/RunicGateway/Module-uo). A full-stack app in one repo: - **Backend** — Node.js + Express REST API (layered `router → controller → model → db`), MariaDB, a provider-agnostic session layer (JWT cookie for web, bearer tokens for mobile, pluggable SSO). - **Frontend** — React + Vite single-page app (public site, wiki, and the admin panel), dark "gothic" theme (Cinzel + Georgia). -- **Deploy** — Docker Compose (app + MariaDB) behind a Pangolin reverse proxy. Express serves the built SPA in production. -- **Shard link** — a live bridge to the in-game ServUO shard through the **uo-link** sidecar ([RunicGateway/link](https://gitea.whitlocktech.com/RunicGateway/link)): the site ingests a live event feed and makes server-side REST calls to show shard status, economy, staff presence, IDOCs, live activity, and per-character sheets. See [Shard integration (uo-link)](#shard-integration-uo-link). -- **Moderation appeals** — a player whose linked Discord identity was banned or muted (per the bot's `mod_actions` log) can open an appeal from the player portal; staff claim and resolve appeals from an admin queue, and approving a ban/mute appeal best-effort reverses it in Discord automatically. See [MODERATION_APPEALS.md](MODERATION_APPEALS.md). +- **Deploy** — Docker Compose (app + MariaDB) behind a reverse proxy (Pangolin, Nginx, Caddy, Traefik, …). Express serves the built SPA in production. +- **Modules** — the game-specific half of a site is a module dropped onto a volume: it adds routes, database tables, nav entries and whole SPA pages without this repo knowing anything about the game. See [Modules](#modules). -The design reference is [BACKEND_DESIGN.md](BACKEND_DESIGN.md) (API contract, schema, security). +The design reference is [BACKEND_DESIGN.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/BACKEND_DESIGN.md) (API contract, schema, security), in the [**RunicGateway/docs**](https://gitea.whitlocktech.com/RunicGateway/docs) repo — where all project documentation now lives. --- @@ -30,20 +40,19 @@ The design reference is [BACKEND_DESIGN.md](BACKEND_DESIGN.md) (API contract, sc - [Pages & routes](#pages--routes) - [API endpoints](#api-endpoints) - [API documentation (Swagger)](#api-documentation-swagger) -- [Shard integration (uo-link)](#shard-integration-uo-link) +- [Modules](#modules) - [Environment variables](#environment-variables) - [Security](#security) - [Logging](#logging) -- [Deployment behind Pangolin](#deployment-behind-pangolin) +- [Deployment behind a reverse proxy](#deployment-behind-a-reverse-proxy) --- ## Architecture How the pieces fit together — the React SPA and native app talk to one Express backend -(`router → controller → model → db`), which persists to MariaDB and bridges to the live -game world only through the **uo-link** sidecar. The shard itself is never internet-facing. -See [ARCHITECTURE.md](ARCHITECTURE.md) for the fuller write-up. +(`router → controller → model → db`), which persists to MariaDB. Anything that knows +what game this site is about lives in an installed module, on the right of the diagram. ```mermaid flowchart TB @@ -63,32 +72,29 @@ flowchart TB subgraph backend["server/ — Express backend"] direction TB mw["Middleware
helmet · siteMode · noindex
rateLimit · loginProtection · botScore · validate"] - router["Router /api/v1
auth (web · mobile · sso) · public · admin"] + router["Router /api/v1
auth (web · mobile · sso) · public · admin · player"] ctrl["Controllers"] auth["Session layer (auth/)
sessionService · JWT/cookie · bearer · SSO+PKCE"] model["Models (.model + .db)
raw parameterized SQL — no ORM"] sse["SSE fan-out
public stream (allowlist) · admin stream (sensitive)"] - - subgraph shardutil["Shard integration (utils/)"] - ingest["shardIngest.js
WS ingest dispatcher"] - restcli["uoLinkClient.js
REST client (never throws)"] - end - + loader["modules/loader.js
scans the volume · mounts · registries · lifecycle"] secret["secretBox.js
AES-256-GCM secrets at rest"] end bot["bot/
Discord bot"] end - db[("MariaDB
users · posts · wiki · settings · activity
mobileSessions · authProviders · userIdentities
uoLinkConfig · shard_online/economy/houses/events")] + db[("MariaDB
users · posts · wiki · settings · activity
mobileSessions · authProviders · userIdentities
installed_modules · <module>_*")] - %% ---------- Shard side ---------- - subgraph shardside["Game shard (never internet-facing)"] + %% ---------- Module side ---------- + subgraph modside["modules/<id>/  — installed, not built (e.g. Module-uo)"] direction TB - sidecar["uo-link sidecar
(Rust) — the only bridge exposed"] - servuo["ServUO shard
(C# plugin)"] + modsrv["server/ — routers, models, schema fragment
reaches core only through ctx"] + modcli["client/dist/entry.js — prebuilt ESM chunk
React shared via window.__rg"] end + game["The game
whatever the module talks to
(for Module-uo: a ServUO shard,
via the uo-link sidecar)"] + %% ---------- Edges ---------- browser <-->|"same-origin JSON + SSE (cookie)"| mw mobile -->|"REST (bearer access/refresh)"| mw @@ -98,40 +104,42 @@ flowchart TB mw --> router --> ctrl ctrl --> auth ctrl --> model - ctrl --> restcli ctrl --> sse auth --> model model <--> db auth -. reads/writes secrets .-> secret - restcli -. reads config/token .-> secret - ingest --> model - ingest --> sse sse -->|"live events"| browser bot -->|"messages"| discord bot <--> db - restcli -->|"REST: /char /roster /economy /history · /link/confirm · /towncrier"| sidecar - sidecar -->|"WebSocket live event feed (bearer + X-UOLink-Version)"| ingest - servuo -->|"loopback TCP 127.0.0.1:7788
newline-delimited JSON (shard dials out)"| sidecar + loader -->|"mounts under /api/v1/<tier>/<prefix>"| router + loader -->|"require() + register(ctx, api)"| modsrv + modsrv -->|"ctx.db · ctx.push · ctx.activity …"| model + modsrv <--> game + browser -->|"<script type=module> injected by htmlShell"| modcli %% ---------- Styling ---------- classDef ext fill:#2d2233,stroke:#7a5c94,color:#e8dff0; classDef store fill:#1f2d2a,stroke:#4c8c7d,color:#dff0ea; - classDef bridge fill:#2d2620,stroke:#94764c,color:#f0e6d8; - class idp,discord ext; + classDef mod fill:#2d2620,stroke:#94764c,color:#f0e6d8; + class idp,discord,game ext; class db store; - class sidecar,servuo bridge; + class modsrv,modcli mod; ``` - **One backend, layered.** Every request flows `middleware → router → controller → model → db`. Web browsers authenticate with an httpOnly JWT cookie; the native app uses short-lived bearer access tokens plus rotated refresh tokens; SSO (Google/Discord/OIDC) is link-only and PKCE-guarded. All three surfaces produce the *same* session via the session layer. -- **The shard is never reachable.** The ServUO shard *dials out* over loopback TCP to the uo-link - sidecar; only the sidecar is exposed, and only the backend talks to it. The REST client - (`uoLinkClient.js`) never throws, so the site degrades gracefully when the shard is down. -- **Sensitive events stay private.** Ingested game events fan out to browsers over two SSE channels — - a public allowlist stream and an admin-only stream that adds staff audit / cheat / login events. +- **Core knows nothing about any game.** Routes, tables, nav entries, SPA pages and push streams for + a specific game arrive from a module the operator installed. Core provides the seams; the module + fills them. See [Modules](#modules). +- **A module that fails must never take the site down.** The loader catches failures across a + module's whole lifecycle and marks that one module `startup_failed`; the site comes up with its + routes and nav absent, and the admin panel says why. +- **Sensitive events stay private.** Events fan out to browsers over two SSE channels — a public + allowlist stream and an admin-only stream that adds staff audit / cheat / login events. Which + event kinds are public is decided by the module that publishes them, and core enforces the split. --- @@ -145,25 +153,26 @@ flowchart TB | Frontend | React 18, Vite 5, React Router 6 | | Email | Nodemailer via Gmail OAuth2 (configured in admin), with a `mailto:` fallback | | API docs | OpenAPI 3.0 via `swagger-autogen`, served with `swagger-ui-express` at `/api/docs` | -| Deploy | Docker Compose, Pangolin reverse proxy | +| Deploy | Docker Compose, any reverse proxy (Pangolin, Nginx, Caddy, Traefik, …) | --- ## Project structure ``` -UOMSITE/ +website/ ├─ server/ Express API │ ├─ src/ │ │ ├─ server.js bootstrap: ensure schema → seed → listen (0.0.0.0) │ │ ├─ app.js middleware + static SPA + routes │ │ ├─ auth/ session layer: session.service · token (JWT/cookies) · session.middleware · ssoState (PKCE/CSRF) · providers/ (base · oauth2 · google · discord · genericOidc · registry) -│ │ ├─ router/v1/ auth (web · mobile · sso) / public / admin route groups -│ │ ├─ model/ users · posts · wiki · settings · activity · mobileSessions · authProviders · userIdentities (.model + .db) +│ │ ├─ router/v1/ auth (web · mobile · sso) / public / admin / player route groups +│ │ ├─ model/ users · posts · wiki · settings · activity · mobileSessions · authProviders · userIdentities · modules (.model + .db) +│ │ ├─ modules/ loader (scan · validate · mount) · registries (the seams) · lifecycle (boot/shutdown + reconcile) │ │ ├─ middleware/ siteMode · noindex · rateLimit · loginProtection · botScore · validate -│ │ └─ utils/ auth (compat facade) · totp (2FA) · secretBox (AES-GCM secrets) · db (pool) · mailer · logger +│ │ └─ utils/ auth (compat facade) · totp (2FA) · secretBox (AES-GCM secrets) · db (pool) · mailer · logger · htmlShell │ ├─ db/ schema.sql + seed.js -│ ├─ swagger/ swagger.js (OpenAPI generator config) + swagger-output.json (generated spec) +│ ├─ swagger/ swagger.js (generator config) · swagger-output.json (generated, core only) · docsSpec.js (merges module fragments at request time) │ └─ .env.example ├─ client/ React + Vite SPA │ ├─ src/ @@ -172,9 +181,11 @@ UOMSITE/ │ │ ├─ routes/admin/ AdminLogin (password + TOTP + SSO buttons), AdminLayout, views/ (Dashboard, Posts, Wiki, Settings, Activity, Bot Activity, Authentication, Users, Account) + editors │ │ ├─ components/ SiteHeader, SiteFooter, layout, guards, Modal, ProviderIcon (inline SSO SVGs), … │ │ ├─ contexts/ AuthContext, SiteContext +│ │ ├─ modules/ the client registry: routes · nav · slots · feature gates · window.__rg │ │ ├─ api/client.js fetch wrapper (sends cookies) │ │ └─ styles/theme.css design tokens │ └─ public/assets/img/ hero image +├─ modules/ installed modules, one directory each — a Docker bind mount; empty here ├─ Dockerfile builds client → serves via Express ├─ docker-compose.yml app + MariaDB ├─ .env.example root env (used by Compose) @@ -216,6 +227,10 @@ IMAGE_TAG=sha-042a151 docker compose pull && docker compose up -d - Health check: `GET http://localhost:3000/api/health` → `{ "status": "ok" }` - Logs: `docker compose logs -f app` (and `./logs/app.log` on the host) - Stop: `docker compose down` (add `-v` to also wipe the database + uploads volumes) +- Modules: installed into `./modules` on the host (bind-mounted to `/app/modules`), never baked into + the image — an operator adds one to a pull-only deployment without building anything. Adding or + removing one takes a `docker compose restart app`; the scan is synchronous at startup. See + [`modules/README.md`](modules/README.md). **Build the images locally instead of pulling** (offline, or to test an unmerged change) — overlay the dev file, which adds `build:` back: @@ -296,7 +311,7 @@ npm start # node server → serves API + SPA at http://localhost:3 | `/site/screenshots` | Screenshot gallery | | `/site/five-on-friday` | Five on Friday | | `/site/newsletter` · `/site/newsletter/:id` | Newsletter list + issue | -| `/site/about` · `/site/status` | About · Shard status | +| `/site/about` · `/site/status` | About · Site status | | `/wiki` · `/wiki/:slug` | Wiki landing + article (auto table-of-contents) | **Admin** (cookie auth, `noindex`): @@ -314,6 +329,13 @@ npm start # node server → serves API + SPA at http://localhost:3 | `/admin/users` | User management | | `/admin/account` | Account security (self-service TOTP two-factor + linked SSO accounts) | +**Player** (any signed-in account, `noindex`): `/player` and its self-service views. Staff are a +superset of players and reach these too. + +An installed module adds its own pages under `//*`, `/admin//*` and `/player//*` — for +Module-uo that is `/uo/shard`, `/admin/uo/link`, `/player/uo/characters` and the rest. Core does not +know their names; they arrive with the module and are interleaved into the nav. + --- ## API endpoints @@ -325,13 +347,19 @@ npm start # node server → serves API + SPA at http://localhost:3 | SSO | `/api/v1/auth` (`providers` — public discovery; `sso/:provider/start`, `sso/:provider/link`, `sso/:provider/callback`) | redirect flow | | Public | `/api/v1/public` (`settings`, `status`, `posts/:category`, `posts/:category/:idOrSlug`, `wiki`, `wiki/:slug`, `contact`) | none | | Admin | `/api/v1/admin` (`dashboard`, `site-mode`, `posts`, `posts/upload`, `wiki`, `settings`, `activity`, `bot-activity`, `bot-activity/unban`, `auth/providers` (CRUD), `users`, `account`, `account/totp/*`, `account/identities`) | cookie (admin) | -| Public · Shard | `/api/v1/public/shard` (`status`, `feed`, `economy`, `online`, `idoc`, `stream`) | none | -| Player · Shard | `/api/v1/player/shard` (`link`, `accounts`, `roster/:account`, `vendors/:account`, `char/:serial`, `sales`) | cookie/bearer (player) | -| Admin · Shard | `/api/v1/admin/shard` (self linking, same as player) · `/api/v1/admin/uo-link` (`config`, `towncrier`, `stream`) | cookie (staff / admin) | +| Player | `/api/v1/player` (`me`, credentials, 2FA, identities, appeals) | cookie/bearer (any signed-in account) | +| Modules | `/api/v1/public/modules` — id, name, version and capabilities of the modules currently serving | none | + +**Module routes are not in this table**, because they are not core's. An installed module mounts +under `/api/v1/public/`, `/api/v1/admin/` and `/api/v1/player/`; which +prefixes exist depends on what is installed. Module-uo, for instance, serves 72 routes under +`/shard`, `/atlas` and `/uo-link` — see its own +[`routes.manifest.json`](https://gitea.whitlocktech.com/RunicGateway/Module-uo/src/branch/main/routes.manifest.json). +On a running instance, `/api/docs` lists everything, core and modules together. Post categories (URL form): `news`, `five-on-friday`, `newsletter`, `screenshots`. `authMethod` on a session ∈ `local · totp · mobile · google · discord · oidc`. -See [BACKEND_DESIGN.md](BACKEND_DESIGN.md) §4 for the full contract, or the interactive Swagger +See [BACKEND_DESIGN.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/BACKEND_DESIGN.md) §4 for the full contract, or the interactive Swagger docs below for a per-endpoint reference (parameters, request bodies, response codes). --- @@ -367,88 +395,132 @@ cd server npm run swagger # → server/swagger/swagger-output.json ``` -`swagger.js` post-processes the generator's output in two ways before writing it: - -- **Trailing slashes are stripped from path keys.** swagger-autogen builds a path by - string-concatenating the mount prefix with the route argument, so a capability router mounted at - `/users` whose collection route is `router.get('/')` would document as `/api/v1/admin/users/` — - a URL no client calls, while dropping the one they all do. Express is indifferent (non-strict - routing treats the two as one route), but the published spec is a contract. -- **Path keys are sorted.** The generator emits them in router-traversal order, so moving a route - between files rewrote most of this ~5k-line committed artifact even when the API was provably - unchanged. Sorting keeps the diff proportional to the change. OpenAPI attaches no meaning to path - order, and `scripts/routeManifest.js` already sorts for the same reason. - If the generated spec is missing, the server logs a warning and simply disables `/api/docs` (it does not crash). +**The committed spec is core only, and the served one is not.** swagger-autogen is *static +analysis* — it parses `src/app.js` as text and follows the literal `app.use(…)` chain — so it can +see neither an installed module (which arrives on a volume long after the image was built, and +mounts through a call no parser can follow) nor an extension slot (whose router is created empty and +filled later). Both are handled by merging a **fragment**: + +- **Extension slots** contribute at generation time, from `server/swagger/slotSpecs.js`, so they are + in the committed file. +- **Modules** contribute at request time, from the `swagger-fragment.json` each one ships, merged by + `server/swagger/docsSpec.js`. So `/api/docs.json` on a running instance describes more than + `npm run swagger` produces here, and `swagger-output.json` stays reproducible on any machine + regardless of what is installed. + +**Core wins every key collision** — a module cannot redefine a core path, tag or schema by shipping +one with the same name; the collision is logged and the module's version dropped. + +One thing worth knowing if you edit an annotation: swagger-autogen **reports a broken one and then +succeeds anyway**, dropping it. `npm run swagger` now captures those diagnostics and fails, which is +how two annotations that had been silently documenting an empty request body were found. If it +rejects yours, the usual causes are an object literal a brace short, or a `"` or backtick inside a +single-quoted description (it re-quotes both to `'` before evaluating). + +### The route manifest (frozen URL surface) + +`server/routes.manifest.json` is a generated, sorted `{ method, path }` list of every route the two +Express listeners actually expose. It is **not** documentation — it is the machine-checkable freeze of +the URL surface, so that carving the router files up by business capability +(`docs/website/API_V2_PLAN.md`) can be proved to move no URL instead of merely claiming it. + +```bash +cd server +npm run routes:manifest # → routes.manifest.json + routes.guards.json +npm run routes:manifest -- --check # exit 1 if either file is stale (what CI runs) +``` + +The generator walks the live Express stack (runtime introspection, not source parsing — a route's path +sits on the line *after* `router.get(`, which defeats greps) and keeps only +`/api/**` and `/.well-known/**` plus the internal listener. The SPA catch-all, `/uploads`, `/brand` +and installed modules' `/modules/` chunks are filesystem-conditional static mounts, not API +contract, so they are excluded and the output depends neither on whether the client has been built +nor on which modules are mounted. + +Two generated files, two very different meanings: + +| File | Meaning of a diff | +|---|---| +| `routes.manifest.json` | **Contract change.** A URL moved. Justify it in the PR description; never let one ride along in a "mechanical" refactor. | +| `routes.guards.json` | **Review aid.** Per route: handler count + the *named* middleware on its mount chain. Names are a hint only — `requireRole(...)` returns an anonymous arrow and cannot be seen — but a vanished `requireAuth` is unambiguous. | + +Unlike the Swagger spec, the manifest is annotation-free: `swagger-output.json` documents intent (only +annotated routes appear), the manifest records reality. + --- -## Shard integration (uo-link) +## Modules -The site is wired to the live in-game world through **uo-link**, a standalone sidecar service that -runs next to the ServUO shard. Its source lives in a separate repo: -**[RunicGateway/link](https://gitea.whitlocktech.com/RunicGateway/link)**. uo-link speaks the shard's internals and -exposes a small, authenticated HTTP + WebSocket API; this website is a *client* of it. The shard -itself is never exposed to the internet — only the sidecar is, and only the website's backend talks -to it. +**Everything specific to a game is a module.** Core has no idea what an "account", a "character" or +a "shard" is; it provides seams, and a module fills them. That is what makes one image able to run a +site for any game rather than for Ultima Online in particular. -### How it works +The design of record is +[MODULE_SYSTEM.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_SYSTEM.md); +the normative contract — the one to read before writing a module — is +[MODULE_API.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_API.md). +The worked example is [RunicGateway/Module-uo](https://gitea.whitlocktech.com/RunicGateway/Module-uo), +which is where everything this README used to describe under *Shard integration (uo-link)* now +lives: the sidecar client, the ingest dispatcher, account linking, the town crier, the spawn atlas, +and every page that renders them. + +### An operator never builds anything + +That constraint shapes the whole design. Installing a module is the WordPress-plugin experience — an +admin-panel action, or a directory dropped onto the `modules/` volume — because production runs a +prebuilt, pull-only image with no toolchain in it. So a module ships **assembled**: its client half +is a prebuilt ESM chunk that resolves React from a `window.__rg` global core owns (an import map +would have to be inline, and the CSP is `script-src 'self'`), and its one runtime dependency travels +inside the tarball. ``` -ServUO shard ──▶ uo-link sidecar (RunicGateway/link) ──▶ website backend ──▶ browser - REST + WebSocket, bearer-auth ingest + REST same-origin JSON/SSE +modules/ +└─ uo/ one directory per module; the id is the directory name + ├─ module.json id, version, coreApi range, mounts, extensions, capabilities + ├─ swagger-fragment.json merged into /api/docs.json while the module is running + ├─ server/ routers, models, and an idempotent schema.sql fragment + └─ client/dist/entry.js the prebuilt chunk, injected by utils/htmlShell.js ``` -- **Connection is admin-managed, not env.** The sidecar's base URL, WebSocket URL, shared-secret - token, and protocol version are stored in the database (`uoLinkConfig`), edited from the - **Admin → Shard** panel. The token is **encrypted at rest** (AES-256-GCM) and is **write-only** in - the API — it is never returned to any client and never sent to the browser. Every call the backend - makes carries `Authorization: Bearer ` and an `X-UOLink-Version` header (a protocol - mismatch fails fast with `409` instead of being mis-parsed). -- **Live ingest (WebSocket).** When enabled, the backend opens an outbound WebSocket to the sidecar - and receives a stream of game events — `mob.login`/`logout`, `char.vitals`, `economy.supply`, - `vendor.sale`, `player.death`/`murdered`, `house.decay` (IDOC), staff `audit.*`/`cheat.*`, - `link.request`, and `server.hello`/`shutdown`. A single dispatcher (`utils/shardIngest.js`) routes - each event: state-changing kinds update `shard_online` / `shard_economy` / `shard_houses`; notable - kinds are appended to an append-only `shard_events` log; high-frequency kinds (vitals, supply - ticks) only update state and are not logged. A changed boot id on `server.hello` is detected as a - restart and stale "online" rows are cleared. On reconnect the backend backfills missed events via - the sidecar's `/history`. -- **Live round-trips (REST).** For point-in-time reads the backend calls the sidecar directly — - `/char/serial/:serial`, `/roster/:account`, `/vendors/:account`, `/economy`, `/history` — plus - commands `/link/confirm` and `/towncrier`. The REST client (`utils/uoLinkClient.js`) **never - throws**: every call returns `{ ok, data, status }`, so a shard that is down or mid-restart - degrades to a `503`/retry banner instead of a 500. -- **Fan-out to the browser.** Ingested events are pushed to browsers over **Server-Sent Events**. - Two channels exist: a **public** stream carrying only a safe allowlist of kinds, and an - **admin-only** stream that also includes sensitive kinds (staff audit, cheat detection, login - attempts, IPs). Sensitive kinds can never leak onto the public channel. +`modules/` is a bind mount in `docker-compose.yml`, so placing a directory there by hand is a +supported install. The directory is tracked in git (via its README) on purpose: Docker recreates a +*missing* bind-mount source as `root:root`, and the container is uid 1000. -### Account linking +### What a module gets, and what it may not do -A player (or staff member) proves ownership of a game account without sharing any game credentials: +At boot, `app.js` scans the volume synchronously, validates each `module.json`, and calls the +module's `register(ctx, api)`: -1. In game, the player runs **`[link`** and receives a one-time code. -2. On the website (Player portal, or Admin → Account for staff) they enter the code. -3. The backend confirms the code with the sidecar (`POST /link/confirm`), which permanently tags the - game account with the website user id, and mirrors the link locally in `shard_account_links`. +- **`ctx` is everything core hands over** — the database, the logger, settings, the session reader, + push, the secret box, the middleware, the rate-limit factory, the activity log, and **express + itself**. A module lives outside `server/`, so Node's resolver never reaches core's + `node_modules`; anything it must share has to be handed to it, or there would be two Expresses and + two Reacts in one process. +- **`api` is everything it may register** — routes (one prefix per tier), an extension slot fill, + notification streams, a news-announce leg, a post hook, and `onBoot`/`onShutdown`. +- **It may not reach into core's tree**, mount outside its declared prefixes, or create tables + outside its `_` prefix. Each of those is checked, in the module's CI and again by the loader. -That mirror is the authorization basis for character reads: roster/vendor/character-sheet endpoints -are **ownership-checked** so a user only sees accounts they linked. **Admins may view any -character**; players and editor/moderator staff are limited to their own linked accounts. +Two things are guaranteed regardless of what a module does. **A failure never takes the site down**: +the loader catches everything from `require` to `onBoot`, marks that module `startup_failed`, and +the site comes up with its routes and nav absent and the reason on the admin screen. And **no URL of +core's may move** — a module that displaced one is caught by the frozen route manifest, which is +generated from a real core with the module loaded. -### What each audience sees +### What is running right now -| Surface | Endpoints | Who | Data | -|---|---|---|---| -| **Public** | `/api/v1/public/shard/*` (`status`, `feed`, `economy`, `online`, `idoc`, `stream`) | anyone | Shard up/down, gold-supply series, IDOC houses, a curated live feed, and **"Staff online"** — only players whose account is linked to a **staff** user (admin/editor/moderator), shown by name. Their in-game **map location is only included for admin/moderator viewers** — for players and the public it is stripped from the payload entirely (server-enforced, not just hidden in the UI). Linked *players* are never listed publicly; no vitals or account are exposed. | -| **Player** | `/api/v1/player/shard/*` (`link`, `accounts`, `roster/:account`, `vendors/:account`, `char/:serial`, `sales`) | logged-in player | Their own linked accounts: character rosters, character sheets, player-vendor snapshots, and recent vendor sales. | -| **Admin** | `/api/v1/admin/shard/*` (self-linking, same as player) · `/api/v1/admin/uo-link/*` (`config`, `towncrier`, `stream`) | staff / admin | Staff link their own accounts like players; **admins** additionally read *any* character's data, edit the sidecar connection config, publish/remove **town-crier** messages, and subscribe to the full event stream (incl. audit/cheat). | +``` +GET /api/v1/public/modules +{ "modules": [ { "id": "uo", "name": "Ultima Online", "version": "0.3.0", + "capabilities": ["shard", "atlas", "market", …] } ] } +``` -The sidecar URL and token are set once in **Admin → Shard**; if uo-link is not configured (or the -shard is offline), every shard surface degrades gracefully — the public page still renders, showing -the shard as offline. +Anonymous, database-free, never site-mode gated, and **`started` modules only** — a module that is +disabled or failed is absent, exactly as its routes and its nav already are. Clients feature-detect +against it; they do not use it to decide what to load (the HTML shell injects each chunk's tag). --- @@ -461,12 +533,13 @@ Copy `.env.example` (Compose) or `server/.env.example` (local) and fill in. **`. | `NODE_ENV` | `production` | | | `PORT` | `3000` | server listens on `0.0.0.0:PORT` | | `UPLOAD_DIR` | `/uploads` | where post images are written (`/app/uploads`, volume-mounted, in Compose) | +| `MODULES_DIR` | `/modules` | where installed modules are scanned from (`/app/modules`, bind-mounted, in Compose) | | `DB_HOST` / `DB_PORT` | `db` / `3306` | `db` in Compose; `127.0.0.1` for local dev | | `DB_NAME` / `DB_USER` / `DB_PASSWORD` | `runic_gateway` / `runic` / — | app database credentials | | `DB_ROOT_PASSWORD` | — | MariaDB root (Compose only) | | `JWT_SECRET` | — | **required** — long random string; signs session, mobile, and SSO-flow tokens | | `JWT_EXPIRES_IN` | `1d` | web session token + cookie lifetime | -| `COOKIE_SECURE` | `auto` | `auto` = Secure only over HTTPS (works on LAN HTTP + Pangolin HTTPS) | +| `COOKIE_SECURE` | `auto` | `auto` = Secure only over HTTPS (works on LAN HTTP + proxy HTTPS) | | `COOKIE_NAME` | `rg_token` | changing it on a live instance invalidates existing sessions | | `BRAND_*` | Runic Gateway | instance branding (name, tagline, colors, logo/hero/favicon) — see [Branding](#branding) | | `SECRET_ENC_KEY` | — | **required in prod** — key for AES-256-GCM encryption of stored OAuth client secrets. Dev falls back to a key derived from `JWT_SECRET` (with a warning) | @@ -482,15 +555,14 @@ Copy `.env.example` (Compose) or `server/.env.example` (local) and fill in. **`. | `CLIENT_ORIGIN` | `http://localhost:5173` | enables CORS in dev only | | `LOG_LEVEL` / `FILE_LOG_LEVEL` | `info` / `debug` | console / file verbosity | | `LOG_TO_FILE` / `LOG_DIR` / `LOG_FILE` | `true` / `/logs` / `app.log` | log file (bind-mounted to `./logs` in Docker) | -| `ANNOUNCE_POLL_MS` | `15000` | how often the news-announcement dispatcher sweeps `announce_job_legs` for due/retry legs (whichever are registered — Discord is core's, the town crier is module-uo's) | -| `TOWNCRIER_DURATION_SEC` | `3600` | how long a news post's in-game town-crier message stays up (≤ `86400`) | +| `ANNOUNCE_POLL_MS` | `15000` | how often the news-announcement dispatcher sweeps `announce_jobs` for legs that are due or retrying. Which legs exist is up to what has registered one — Discord is core's; a module may add its own | --- ## Branding Instance identity is data, not code — set via `BRAND_*` env vars, so one prebuilt -image can run as any shard. With none set, everything renders as **Runic Gateway**. +image can run as any community. With none set, everything renders as **Runic Gateway**. | Var | What | |---|---| @@ -507,7 +579,7 @@ API (`SiteContext`), so no rebuild is needed; the server templates `index.html` settings override `BRAND_NAME` / `BRAND_CONTACT_EMAIL` when set. Image assets are delivered from the `./brand` bind-mount (see `brand/README.md`). -**UOMysticmoon** is the first instance — [`.env.uomysticmoon.example`](https://gitea.whitlocktech.com/RunicGateway/website/src/branch/main/.env.uomysticmoon.example) +**UOMysticmoon** is the first instance — [`.env.uomysticmoon.example`](.env.uomysticmoon.example) holds the exact `BRAND_*` + infra (`DB_NAME`/`DB_USER`/`COOKIE_NAME`) pinning to run this repo as UOMysticmoon. @@ -571,7 +643,7 @@ run this repo as UOMysticmoon. **Platform** - `helmet`, admin routes `noindex` + `robots.txt` disallow, `trust proxy` for correct client IPs - behind Pangolin (see `TRUST_PROXY`), first admin seeded from env (no hardcoded credentials), + behind a reverse proxy (see `TRUST_PROXY`), first admin seeded from env (no hardcoded credentials), `.env` git-ignored. Passwords and request bodies are never logged. Email sends through Gmail OAuth2 configured in the admin (refresh token stored AES-GCM-encrypted, never in env); the contact form falls back to a `mailto:` link when unconfigured. @@ -598,10 +670,60 @@ bind-mounted to `./logs/app.log` and `docker compose logs -f app` shows the cons --- -## Deployment behind Pangolin +## Deployment behind a reverse proxy `docker compose up -d --build` exposes the `app` container on `0.0.0.0:3000` (no `127.0.0.1` -binding) so Pangolin can reach it. Point a Pangolin resource at `app:3000`. Because `COOKIE_SECURE` -defaults to `auto`, the admin login works both directly via the LAN IP over HTTP **and** through -Pangolin over HTTPS — no config change needed. MariaDB stays on the private Compose network -(no published port by default); data persists in the `dbdata` volume, uploads in `uploads`. +binding) so a reverse proxy — Pangolin, Nginx, Caddy, Traefik, etc. — can reach it. Point the +proxy at `app:3000` (or the host's `:3000` if the proxy runs outside Compose) and terminate TLS +there. Because `COOKIE_SECURE` defaults to `auto`, the admin login works both directly via the +LAN IP over HTTP **and** through the proxy over HTTPS — no config change needed. MariaDB stays on +the private Compose network (no published port by default); data persists in the `dbdata` volume, +uploads in `uploads`. + +Set `TRUST_PROXY` so Express reads the real client IP from the proxy's `X-Forwarded-For` header +(see [Environment variables](#environment-variables)) — required for rate limiting, bot scoring, +and correct logging. Forward the standard `X-Forwarded-For` and `X-Forwarded-Proto` headers from +your proxy. + +Minimal proxy examples: + +```nginx +# Nginx +location / { + proxy_pass http://app:3000; + proxy_set_header Host $host; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; +} +``` + +```caddy +# Caddy — Caddyfile (automatic HTTPS; forwards X-Forwarded-* by default) +your.domain { + reverse_proxy app:3000 +} +``` + +**Pangolin:** create a resource targeting `app:3000`; it forwards the required headers and +terminates HTTPS out of the box, so no extra configuration is needed. + +--- + +## License + +Runic Gateway is free software, licensed under the **GNU General Public License +v3.0 or later** — see [LICENSE.md](LICENSE.md). + + Copyright (C) 2026 Runic Gateway + + This program is free software: you can redistribute it and/or modify it under + the terms of the GNU General Public License as published by the Free Software + Foundation, either version 3 of the License, or (at your option) any later + version. It is distributed WITHOUT ANY WARRANTY; without even the implied + warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + +Contributions are welcome — please read [CONTRIBUTING.md](CONTRIBUTING.md) (note +the **AI-usage disclosure** requirement) and our +[Code of Conduct](CODE_OF_CONDUCT.md). Report vulnerabilities privately per +[SECURITY.md](SECURITY.md).