Merge pull request 'docs(website): close phase 3 — slice 5, the fragment obligation, and the rust dry run' (#140) from docs/module-close-phase3 into main
Reviewed-on: #140
This commit is contained in:
14
README.md
14
README.md
@@ -7,7 +7,8 @@ so they live in one place, independent of either codebase.
|
||||
## Layout
|
||||
|
||||
```
|
||||
website/ docs from the shard website (Node/Express + MariaDB + React/Vite)
|
||||
website/ docs from the website core (Node/Express + MariaDB + React/Vite)
|
||||
modules/ docs for installable game modules — one directory per module id
|
||||
link/ docs from the ServUO bridge (C# plugin + Rust sidecar + Node WS)
|
||||
android/ docs from the native Android client (Kotlin + Jetpack Compose)
|
||||
installer/ docs for the installer that deploys a shard's bridge components
|
||||
@@ -35,6 +36,17 @@ sidecar as a service, and hands you the values the website needs.
|
||||
| [website-README.md](website/website-README.md) | Snapshot of the website repo's README (setup/run reference) |
|
||||
| [PROJECT_TREE.md](website/PROJECT_TREE.md) | Auto-generated snapshot of the repo's tracked file layout |
|
||||
|
||||
### `modules/`
|
||||
|
||||
Documentation for installable game modules aggregates here rather than in each module's repo
|
||||
([MODULE_SYSTEM.md](website/MODULE_SYSTEM.md) §2.10). The website core knows nothing about any
|
||||
particular game; a module is what makes it a site *for* one.
|
||||
|
||||
| Doc | What it covers |
|
||||
|---|---|
|
||||
| [uo/](modules/uo/README.md) | **module-uo** — the Ultima Online module: what it serves, what it owns, and what an operator needs |
|
||||
| [rust-dryrun.md](modules/rust-dryrun.md) | A written, deliberately unimplemented `module-rust` — the test that the module contract generalises past the game it was extracted from |
|
||||
|
||||
### `link/`
|
||||
| Doc | What it covers |
|
||||
|---|---|
|
||||
|
||||
262
modules/rust-dryrun.md
Normal file
262
modules/rust-dryrun.md
Normal file
@@ -0,0 +1,262 @@
|
||||
# `module-rust` — a dry run
|
||||
|
||||
**Phase 3's fourth acceptance criterion** ([`../website/MODULE_SYSTEM.md`](../website/MODULE_SYSTEM.md)
|
||||
§2.7). A written design for a module serving a **Rust** (Facepunch) community, taken far enough to
|
||||
find out whether the contract generalises past the game it was extracted from — **and deliberately
|
||||
not implemented.** A contract validated only against the module it was carved out of has not been
|
||||
validated.
|
||||
|
||||
The exercise is honest if it finds something. It found four things, one of which is a gap in the
|
||||
contract that a real second module would hit on its first day. They are in *Findings* at the end; the
|
||||
design comes first, because a finding is only worth anything with the design that produced it.
|
||||
|
||||
Rust was chosen because it is unlike Ultima Online in the ways most likely to break assumptions:
|
||||
it **wipes** every month, a community runs **several servers** rather than one shard, its identity
|
||||
is **Steam**, and its server speaks a **protocol nobody has to write** — RCON over WebSocket, built
|
||||
in. If the contract survives that, "game-agnostic" means something.
|
||||
|
||||
> Nothing here re-specifies the contract. [`../website/MODULE_API.md`](../website/MODULE_API.md) is
|
||||
> normative; this document only *uses* it.
|
||||
|
||||
---
|
||||
|
||||
## 1. The manifest
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "rust",
|
||||
"name": "Rust",
|
||||
"version": "0.1.0",
|
||||
"coreApi": "^1.3.0",
|
||||
"server": "server/index.js",
|
||||
"client": { "entry": "client/dist/entry.js" },
|
||||
"schema": "server/db/schema.sql",
|
||||
"purge": "server/db/purge.sql",
|
||||
"mounts": {
|
||||
"public": ["/rust"],
|
||||
"admin": ["/rust"],
|
||||
"player": ["/rust"]
|
||||
},
|
||||
"extensions": ["admin.users.detail"],
|
||||
"capabilities": ["servers", "wipes", "leaderboards", "map", "killfeed", "teams"]
|
||||
}
|
||||
```
|
||||
|
||||
One prefix per tier, named for the module rather than for a feature — the opposite of module-uo's
|
||||
`/shard` + `/atlas` + `/uo-link`, and the better choice for anything new. Module-uo's prefixes are
|
||||
what they are because §1.2 froze the URLs core already served; a module written today has no such
|
||||
debt and should claim one obvious segment. It also sidesteps the collision surface entirely:
|
||||
`/shard` is a word a second game might want, `/rust` is not.
|
||||
|
||||
## 2. The server half
|
||||
|
||||
```js
|
||||
module.exports = function register(ctx, api) {
|
||||
core.init(ctx)
|
||||
|
||||
const publicRouter = require('./router/public/rust.router')
|
||||
const adminRouter = require('./router/admin/rust.router')
|
||||
const playerRouter = require('./router/player/rust.router')
|
||||
const userExt = require('./router/admin/usersRust.router')
|
||||
|
||||
api.registerRoutes({
|
||||
public: { '/rust': publicRouter },
|
||||
admin: { '/rust': adminRouter },
|
||||
player: { '/rust': playerRouter },
|
||||
})
|
||||
|
||||
api.registerExtension('admin.users.detail', userExt)
|
||||
|
||||
api.registerNotificationStreams([
|
||||
{ id: 'rust.wipe', label: 'Server wipes',
|
||||
description: 'A server wiped — new map, new seed, everything reset.',
|
||||
personal: false, requiresLinkedAccount: false },
|
||||
{ id: 'rust.raid', label: 'Base raided',
|
||||
description: 'Your base took damage while you were offline.',
|
||||
personal: true, requiresLinkedAccount: true },
|
||||
])
|
||||
|
||||
api.registerAnnounceLeg({
|
||||
leg: 'rust.ingame',
|
||||
label: 'In-game chat',
|
||||
dispatch: (post) => rcon.say(`[NEWS] ${post.title} — ${ctx.site.baseUrl}/news/${post.slug}`),
|
||||
classify: (result) => (result.ok ? { outcome: 'done' } : { outcome: 'retry', error: result.error }),
|
||||
})
|
||||
|
||||
api.onBoot(async () => { await rcon.connectAll() })
|
||||
api.onShutdown(async () => { await rcon.closeAll() })
|
||||
}
|
||||
```
|
||||
|
||||
Everything above is a call the contract already has, used the way module-uo uses it. Two details are
|
||||
worth pointing at:
|
||||
|
||||
- **`rust.wipe` and `rust.raid` are namespaced**, with no grandfathering request. Module-uo's seven
|
||||
bare stream ids are allowlisted because they were in `notification_subs` before the rule existed
|
||||
(§6.5); a new module gets the rule, and the rule is exactly right.
|
||||
- **The announce leg goes to in-game chat over RCON**, which is a one-shot delivery with retry —
|
||||
`registerAnnounceLeg`, not `registerPostHook`. The distinction §2.4 draws holds up on a game that
|
||||
has nothing in common with the one it was drawn for.
|
||||
|
||||
### Tables
|
||||
|
||||
`rust_servers`, `rust_wipes`, `rust_players`, `rust_player_stats`, `rust_teams`, `rust_events`,
|
||||
`rust_bans`, `rust_maps`. All `rust_`-prefixed, all in one idempotent `schema.sql` fragment.
|
||||
|
||||
**Every table that holds gameplay data carries a `wipe_id`.** That is the whole shape of the game in
|
||||
one column: a leaderboard means "since the last wipe", a base means "on this map", and a player's
|
||||
stats are per-wipe with an all-time rollup kept separately. It has no bearing on the contract —
|
||||
core replays the fragment and never looks inside — but it is the first thing a UO-shaped mental
|
||||
model gets wrong, and it is worth writing down for whoever builds this.
|
||||
|
||||
### Talking to the game
|
||||
|
||||
**No sidecar.** Rust ships RCON over WebSocket, so the module dials the server directly with the
|
||||
token an admin saved, encrypted at rest through `ctx.secretBox`.
|
||||
|
||||
This is the sharpest test of whether the module system's boundary is drawn in the right place, and it
|
||||
passes: core has no opinion about how a module reaches its game. What core owns is that the module
|
||||
never blocks a request on it, that its secrets are encrypted, and that a game being down degrades to
|
||||
a page saying so. The *shard-dials-out* invariant that shapes
|
||||
[`../link/PLAN.md`](../link/PLAN.md) is a property of ServUO — a game engine with no remote-control
|
||||
surface, whose plugin must not stall on a socket — not of the platform. A game that ships RCON
|
||||
already answers the question the sidecar exists to answer.
|
||||
|
||||
The Integration Kit ([`MODULE_SYSTEM.md`](../website/MODULE_SYSTEM.md) §2.11) should say this
|
||||
plainly, or its second reader will build a sidecar they did not need.
|
||||
|
||||
## 3. The client half
|
||||
|
||||
```js
|
||||
registry.registerRoutes('rust', {
|
||||
public: [
|
||||
{ path: 'servers', element: <Servers /> },
|
||||
{ path: 'servers/:id', element: <ServerDetail /> },
|
||||
{ path: 'servers/:id/map', element: <MapView /> },
|
||||
{ path: 'leaderboards', element: <Leaderboards /> },
|
||||
{ path: 'wipes', element: <WipeSchedule /> },
|
||||
],
|
||||
player: [
|
||||
{ path: 'account', element: <LinkedAccount /> },
|
||||
{ path: 'stats', element: <MyStats /> },
|
||||
],
|
||||
admin: [
|
||||
{ path: 'servers', element: <AdminServers />, gate: { roles: ['admin'] } },
|
||||
{ path: 'ops', element: <AdminOps />, gate: { roles: ['admin', 'moderator'] } },
|
||||
],
|
||||
})
|
||||
|
||||
registry.registerNav('rust', {
|
||||
area: 'public',
|
||||
items: [
|
||||
{ label: 'Servers', to: '/rust/servers', group: 'Play', order: 10, icon: ServerIcon },
|
||||
{ label: 'Leaderboards', to: '/rust/leaderboards', group: 'Play', order: 20, icon: TrophyIcon,
|
||||
feature: 'leaderboards' },
|
||||
{ label: 'Wipe schedule', to: '/rust/wipes', group: 'Play', order: 30, icon: CalendarIcon },
|
||||
],
|
||||
})
|
||||
|
||||
registry.registerFeatureProvider('rust', 'rust', useRustFeatures)
|
||||
registry.registerExtension('rust', 'admin.users.detail', LinkedSteamAccounts)
|
||||
```
|
||||
|
||||
`Play` is a group core does not have; §3.3 appends an unknown group rather than dropping the items,
|
||||
so this works and lands at the end of the nav — where an operator can move it, because a module row
|
||||
is an ordinary row once it is interleaved.
|
||||
|
||||
The pages need `PublicLayout`, `PageHeader`, the three `PageState` components, `useAsync` and
|
||||
`useAuth`: **six of the kit's seven members**, and the seventh (`useSite`) on the wipe-schedule page
|
||||
for the site's timezone. A second game, unrelated to the first, wanting exactly what the kit
|
||||
contains is the strongest evidence available that §3.4 was curated at the right altitude.
|
||||
|
||||
The map view is the one page that wants something the kit does not have — a pan/zoom canvas. It
|
||||
bundles one, which is the answer §3.4 already gives ("everything else a module bundles itself"), and
|
||||
it costs the chunk about 40 KB.
|
||||
|
||||
---
|
||||
|
||||
## Findings
|
||||
|
||||
### 1. A module cannot register an identity provider — and Rust's identity is Steam
|
||||
|
||||
The gap. Module-uo proves account ownership with an in-game `[link` command that issues a one-time
|
||||
code; the website confirms it with the sidecar. Nothing about that needs core's auth layer, so
|
||||
nothing in `api` ever needed to touch it.
|
||||
|
||||
**Rust's answer is Steam OpenID**, and every Rust community expects "Sign in with Steam". Core has an
|
||||
SSO layer — `authProviders`, `userIdentities`, OAuth2/OIDC providers behind a registry — and
|
||||
[`MODULE_API.md`](../website/MODULE_API.md) §2.4 offers a module no way in. There is no
|
||||
`registerAuthProvider`, and §2.7 forbids reaching for one.
|
||||
|
||||
A Rust module can still ship: it can copy the UO shape, issuing a code in-game and matching it on
|
||||
the website. That works, it is one screen worse, and it leaves the community's obvious expectation
|
||||
unmet.
|
||||
|
||||
**This is not a defect in what was built** — it is the boundary of what was specified, found exactly
|
||||
where a dry run is supposed to find it. It is worth stating precisely, because whoever adds it has to
|
||||
answer a question the current policy already has an opinion about: **SSO is link-only by design**, an
|
||||
external identity must already be linked to an existing account, and identities are never
|
||||
auto-provisioned. A Steam provider a module registers must inherit that, not route around it. It is
|
||||
also *not* a small addition: an identity provider participates in session creation, which is the one
|
||||
part of core a module must never be able to weaken.
|
||||
|
||||
Recommendation: leave it out of v1 of the contract and record it here as the first candidate for
|
||||
`MODULE_API_VERSION` 1.4 or 2.0, specified deliberately rather than bolted on when someone needs it.
|
||||
|
||||
### 2. "One module, one game" is not the same as "one module, one server"
|
||||
|
||||
A UO community runs one shard. A Rust community runs four or five, wipes them on different
|
||||
schedules, and every page is a per-server view.
|
||||
|
||||
The contract is silent on this, and silence turns out to be right: multiplicity lives entirely in the
|
||||
module's own tables and route parameters (`/rust/servers/:id`). Core's mount prefixes, capabilities
|
||||
and state machine are per-**module**, and none of them wanted to be per-server.
|
||||
|
||||
Worth recording only because it looks like a problem until you try it — and because it is the shape
|
||||
that would have broken a contract designed around "the shard" as a singular noun. The phase-2
|
||||
inversions that removed core's opinions about game content (the push catalog, the announce legs,
|
||||
`mapEvent` dropped) are why it does not.
|
||||
|
||||
### 3. The event catalog generalises; the *retention* assumption does not
|
||||
|
||||
`rust_events` is the twin of `shard_events`, and the ingest shape carries over unchanged.
|
||||
|
||||
What does not carry over is that a UO event log grows forever while a Rust one is **truncated every
|
||||
wipe**. That is module-internal — but it lands on something core does own: the schema fragment
|
||||
**must not** be where that truncation happens. §2.6's leading-verb allowlist (`CREATE`, `ALTER`,
|
||||
`INSERT`, `UPDATE`) already forbids `DELETE` and `TRUNCATE` in a fragment, precisely because the
|
||||
fragment is replayed on **every boot** and would empty the table each restart.
|
||||
|
||||
So a wipe is a runtime operation on a module route, not a schema one. The allowlist was written for a
|
||||
different reason — the phase-2 note says "the file replays every boot, so TRUNCATE/DELETE would empty
|
||||
a table each restart" — and it correctly forbids the first mistake this module's author would make.
|
||||
A rule that catches a case it was not written for is a rule at the right altitude.
|
||||
|
||||
### 4. `capabilities` earns its keep the moment there are two modules
|
||||
|
||||
With one module, `capabilities` reads like decoration — core never interprets one, and the SPA knows
|
||||
what it registered. With two, it is the only thing a *client* can ask.
|
||||
|
||||
The Android app is the case: it feature-detects against `GET /api/v1/public/modules` and must render
|
||||
a site whose module it has never heard of. `["servers", "wipes", "leaderboards"]` tells it there is
|
||||
nothing shard-shaped here without it having to know what `rust` means, and §2.9's rule — treat an
|
||||
unknown capability as absent, never infer a route from one — is what keeps that from becoming a
|
||||
second, worse route table.
|
||||
|
||||
No change needed. Recorded because the design decision looked over-engineered with one module and is
|
||||
load-bearing with two.
|
||||
|
||||
---
|
||||
|
||||
## Verdict
|
||||
|
||||
**The contract generalises.** A second game, chosen for how little it shares with the first, is
|
||||
served by the same `module.json`, the same seven registration calls, the same schema-fragment rules,
|
||||
the same client registry and the same UI kit — with one genuine gap (identity providers), one
|
||||
non-issue that looks like a gap (multiple servers), and two places where a rule written for one
|
||||
reason turns out to cover another.
|
||||
|
||||
The gap is worth having found before something was built on top of it, which is what a dry run is
|
||||
for. What it does **not** establish is that someone outside this org could build this module from the
|
||||
documentation alone — that is the Integration Kit's acceptance test (§2.11), and it stays untested
|
||||
until a person who did not write any of this does it.
|
||||
97
modules/uo/README.md
Normal file
97
modules/uo/README.md
Normal file
@@ -0,0 +1,97 @@
|
||||
# module-uo — the Ultima Online module
|
||||
|
||||
Everything specific to Ultima Online that the website serves. Code:
|
||||
[RunicGateway/Module-uo](https://gitea.whitlocktech.com/RunicGateway/Module-uo). Module id `uo`,
|
||||
installed at `modules/uo/` on the website's modules volume.
|
||||
|
||||
This page is **orientation**: what the module is, what it serves, what it owns, and what an operator
|
||||
has to know. It is not a second copy of the contract — [`../../website/MODULE_API.md`](../../website/MODULE_API.md)
|
||||
is normative for anything about how a module and core fit together, and the module's own repo is
|
||||
authoritative for its file layout. Where this page and either of those disagree, they win.
|
||||
|
||||
Module documentation aggregates here rather than in module repos
|
||||
([`MODULE_SYSTEM.md`](../../website/MODULE_SYSTEM.md) §2.10), so the feature docs that describe what
|
||||
these routes *mean* are the ones that already existed and did not move:
|
||||
|
||||
| Doc | What it covers |
|
||||
|---|---|
|
||||
| [`SHARD_VISIBILITY.md`](../../website/SHARD_VISIBILITY.md) | Who sees which shard data — the admin-configurable audience framework |
|
||||
| [`SPAWN_ATLAS.md`](../../website/SPAWN_ATLAS.md) | The bestiary / spawn atlas, parsed from the shard's own ServUO tree |
|
||||
| [`MARKETPLACE.md`](../../website/MARKETPLACE.md) | The player-vendor index |
|
||||
| [`CLILOCS.md`](../../website/CLILOCS.md) | UO's id → name table |
|
||||
| [`UOFIDDLER.md`](../../website/UOFIDDLER.md) | Operator runbook for extracting the cliloc table and creature art |
|
||||
| [`../../link/PLAN.md`](../../link/PLAN.md), [`../../link/INTEGRATION.md`](../../link/INTEGRATION.md) | The wire protocol this module speaks to the sidecar |
|
||||
|
||||
---
|
||||
|
||||
## What it serves
|
||||
|
||||
**72 URLs**, frozen in the module's own
|
||||
[`routes.manifest.json`](https://gitea.whitlocktech.com/RunicGateway/Module-uo/src/branch/main/routes.manifest.json)
|
||||
and documented in its
|
||||
[`swagger-fragment.json`](https://gitea.whitlocktech.com/RunicGateway/Module-uo/src/branch/main/swagger-fragment.json),
|
||||
which core merges into `/api/docs.json` while the module is running.
|
||||
|
||||
| Mount | Tier | What |
|
||||
|---|---|---|
|
||||
| `/api/v1/public/shard` | public | Status, activity feed, economy, presence, houses/IDOC, champs, guilds, governors, points boards, the player-vendor market, the ruleset, and the SSE event stream |
|
||||
| `/api/v1/public/atlas` | public | The bestiary: creatures, regions, landmarks, champion altars, and what is loaded |
|
||||
| `/api/v1/admin/shard` | admin | Shard ops (kick/ban/broadcast/pages), the visibility config, atlas and cliloc imports, market admin, account links |
|
||||
| `/api/v1/admin/uo-link` | admin | The sidecar connection config, live status, and the town crier |
|
||||
| `/api/v1/player/shard` | player | A player's own linked accounts: rosters, character sheets, vendors, sales |
|
||||
| `/api/v1/admin/users/:id/shard/*` | admin | Six routes filling core's `admin.users.detail` **extension slot** — a module's routes hanging off a *core* resource, since core owns the user |
|
||||
|
||||
Every URL is byte-identical to the one core served before the extraction. That is the whole point of
|
||||
moving the code and not the paths: the shipped Android app calls
|
||||
`POST /api/v1/admin/shard/kick`, the Discord bot reads `/api/v1/public/shard/*`, and neither knows a
|
||||
module answers now.
|
||||
|
||||
**In the SPA**: twelve public pages under `/uo/*`, the admin views under `/admin/uo/*`, the player
|
||||
views under `/player/uo/*`, three extension-slot fills, and the nav rows for all of them, interleaved
|
||||
into core's nav so an operator can reorder, relabel or hide them like any other row.
|
||||
|
||||
## What it owns
|
||||
|
||||
- **27 database tables** — 26 `shard_*` plus `uo_link_config`. Created by an idempotent
|
||||
`schema.sql` fragment core replays on every boot, after its own schema. The `shard_`/`uo_link_`
|
||||
prefixes are **grandfathered** ([`MODULE_API.md`](../../website/MODULE_API.md) §6.5): the rule for
|
||||
a new module is `<id>_`, and these predate it.
|
||||
- **Seven push notification streams** and the announce leg `towncrier`, likewise grandfathered —
|
||||
they are stored in `notification_subs` and `announce_job_legs.leg` and read by the shipped Android
|
||||
app, so renaming them would be a data migration plus a client break.
|
||||
- **Two settings rows**: `game_account_signup` and the shard's protocol pin.
|
||||
- **The public-safety filter.** Which shard event kinds may reach the *public* SSE stream is decided
|
||||
here, not in core — the kinds, the streams and the filter are one file that moves together.
|
||||
|
||||
## For an operator
|
||||
|
||||
**Installing.** A release is `module-uo-<version>.tar.gz` plus a manifest carrying its `sha256`.
|
||||
Unpack it as `modules/uo/` on the website's modules volume (or use the admin Modules screen when
|
||||
phase 4 lands) and restart. **You never build anything** — the client chunk is prebuilt and the one
|
||||
runtime dependency ships inside the tarball.
|
||||
|
||||
**Connecting it to a shard.** The module needs the
|
||||
[uo-link sidecar](https://gitea.whitlocktech.com/RunicGateway/link) running next to the ServUO
|
||||
shard. Deploy that with the [installer](https://gitea.whitlocktech.com/RunicGateway/installer) —
|
||||
[`../../installer/INSTALL.md`](../../installer/INSTALL.md) is the operator guide — and paste the four
|
||||
values it prints into **Admin → Shard**. The token is encrypted at rest and write-only in the API.
|
||||
|
||||
**Nothing requires the shard to exist.** With no sidecar configured the site renders normally and
|
||||
shows the shard offline. That is the same bargain the module system makes one level up: a module
|
||||
that fails to load never takes the site down.
|
||||
|
||||
**Environment variables** — four, all optional, all read by the module and documented in its README:
|
||||
`UOLINK_BASE_URL`, `UOLINK_WS_URL`, `UOLINK_PROTOCOL`, `TOWNCRIER_DURATION_SEC`. They live in the
|
||||
Compose `.env`, because that is what reaches the container.
|
||||
|
||||
## Compatibility
|
||||
|
||||
`module.json` declares a `coreApi` semver range, checked at boot against core's
|
||||
`MODULE_API_VERSION`. A mismatch fails **loudly** — the module is marked `startup_failed` and the
|
||||
site comes up without it. This is a separate number from `PROTOCOL_VERSION`, which versions the
|
||||
shard wire and says nothing about a website module.
|
||||
|
||||
The module's CI clones core at a **pinned** ref ([`MODULE_API.md`](../../website/MODULE_API.md) §5.3)
|
||||
to generate its frozen manifest. Bumping that pin is a deliberate commit that says which core the
|
||||
module was last proved against — not a tracking reference that turns core's unrelated changes into
|
||||
red Xes here.
|
||||
@@ -731,8 +731,15 @@ are authoritative, and they answer different questions:
|
||||
|
||||
| Artifact | Source of truth for | Generated by |
|
||||
|---|---|---|
|
||||
| `server/routes.manifest.json` — mirrored as [api-route-inventory.json](./api-route-inventory.json) | **What URLs exist.** 228 public routes + 2 on the internal listener, sorted, method + path only. | `npm run routes:manifest`, by walking the live Express stack |
|
||||
| `server/swagger/swagger-output.json` — served at `/api/docs` | **What each route means.** Parameters, bodies, response codes, security. | `npm run swagger`, from `#swagger.*` annotations |
|
||||
| `server/routes.manifest.json` — mirrored as [api-route-inventory.json](./api-route-inventory.json) | **What URLs CORE serves.** 158 public routes + 2 on the internal listener, sorted, method + path only. | `npm run routes:manifest`, by walking the live Express stack |
|
||||
| `server/swagger/swagger-output.json` — merged into `/api/docs` | **What each core route means.** Parameters, bodies, response codes, security. | `npm run swagger`, from `#swagger.*` annotations |
|
||||
|
||||
Both are **core's**. An installed module's routes are in neither: they are in that module's own
|
||||
frozen manifest and its `swagger-fragment.json`, in its own repo, and core merges the fragment into
|
||||
`/api/docs.json` at request time (§4.0.1). So on a running instance the served document describes
|
||||
more than the committed one does, which is the intended arrangement rather than a drift —
|
||||
`swagger-output.json` has to regenerate identically on any machine, whatever happens to be
|
||||
installed on it.
|
||||
|
||||
The split is deliberate: Swagger is annotation-derived, so an unannotated route is invisible in it and
|
||||
it churns whenever a description is reworded — it documents *intent*. The manifest is introspection-
|
||||
@@ -764,6 +771,34 @@ authenticated endpoints silently. Names are a hint only — `requireRole(...)` r
|
||||
arrow and cannot be observed — but a *missing* `requireAuth` is unambiguous, and the server test suite
|
||||
asserts every `/admin/**` and `/player/**` route still carries it.
|
||||
|
||||
#### 4.0.1 `/api/docs.json` is assembled per request
|
||||
|
||||
`GET /api/docs.json` and the Swagger UI at `/api/docs` do not serve `swagger-output.json` directly.
|
||||
`swagger/docsSpec.js` merges the `swagger-fragment.json` of every **started** module over it first,
|
||||
cached on the module loader's state version and rebuilt when a module's state moves.
|
||||
|
||||
It exists because swagger-autogen is static analysis: it parses `src/app.js` as text and follows the
|
||||
literal `app.use(…)` chain, which reaches neither an installed module (required by a filesystem loop,
|
||||
from a volume that had nothing on it when the image was built) nor an extension slot (whose router is
|
||||
created empty by `declareSlot()` and filled later). Slots are handled at generation time by
|
||||
`swagger/slotSpecs.js` and are therefore *in* the committed file; modules cannot be, because core
|
||||
never has their sources.
|
||||
|
||||
Three rules, all from [`MODULE_API.md`](MODULE_API.md) §6.1a:
|
||||
|
||||
- **`started` only.** A `registered`, `disabled` or `startup_failed` module's paths are absent —
|
||||
documenting a route that answers 503 or 404 sends a client somewhere it cannot go.
|
||||
- **Core wins every key collision**, in all three merged sections (`paths`, `tags`,
|
||||
`components.schemas`); the collision is logged and the module's version dropped. This is what makes
|
||||
the naming rule work: a module namespaces the schemas it *defines* (`UoShardStatus`) and references
|
||||
core's shared ones (`Error`, `ValidationError`) by core's name, and both resolve in the merged
|
||||
document.
|
||||
- **A bad fragment costs that module its paths and nothing else.** Missing, unreadable or not JSON is
|
||||
logged and skipped; `/api/docs.json` still answers with everything else.
|
||||
|
||||
The committed spec is never mutated — it is a `require()`d JSON module, so an in-place merge would be
|
||||
permanent for the life of the process *and* cumulative across rebuilds.
|
||||
|
||||
### /auth (auth/index.js → the capability routers in §2)
|
||||
|
||||
No group gate — `/auth` is where an anonymous caller becomes authenticated. The authenticated parts
|
||||
|
||||
@@ -435,6 +435,33 @@ fragments of started modules into `/api/docs.json`; the full reasoning and the c
|
||||
§6.1a. In short: fully-qualified paths, namespaced schema keys, module CI fails if a registered
|
||||
route has no path in the fragment, and core wins every key collision.
|
||||
|
||||
**Built in phase 3 slice 5** (module-uo#6, website#141). Four things settled while building it, all
|
||||
of which a second module inherits:
|
||||
|
||||
- **The filename is fixed here, not declared in `module.json`.** `swagger-fragment.json` in the
|
||||
bundle root, like `module.json` itself — so a module cannot point core at some other file, and
|
||||
core's loader has one path to check. A module that ships none is simply absent from the merged
|
||||
document: whether it registered routes without documenting them is the *module's* CI to answer,
|
||||
where the routes are known. Core cannot tell a module with no routes from one that forgot.
|
||||
- **Namespace what you DEFINE; reference core's by core's name.** `UoShardStatus` is defined by the
|
||||
module; `#/components/schemas/Error` and `ValidationError` are referenced and **not** redefined.
|
||||
Both resolve in the merged document, which is the only place both halves exist — and shipping a
|
||||
copy of `Error` would be a collision core drops, arriving at the same result the expensive way.
|
||||
This is the practical form of "core wins": it makes the two cases feel different in the source,
|
||||
which is where the mistake would otherwise be invisible.
|
||||
- **Generate the fragment from the module's own registrations.** swagger-autogen needs a *file* and
|
||||
cannot follow `api.registerRoutes`, so the module's generator runs its own `register()` against a
|
||||
recording `api` and resolves each router back to its source through `require.cache`. A mount prefix
|
||||
then exists in exactly one place. The two values it cannot derive — the tier base paths and the
|
||||
slot's mount, both §2.4's — are checked against a real core by the §5.3 job rather than trusted.
|
||||
- **swagger-autogen reports a broken annotation and then succeeds.** It `console.error`s "Syntax
|
||||
error" or "out of structure", drops that annotation, and prints `Success`. Both repos' generators
|
||||
now capture those diagnostics and fail on them, which found six annotations documenting less than
|
||||
they claimed. Two ways one breaks: an object literal a brace short, and a `"` or backtick inside a
|
||||
single-quoted description (the tool re-quotes both to `'` before evaluating, ending the string
|
||||
early). A third, which nothing but a rendered page catches: an escaped apostrophe survives
|
||||
literally, because the annotation is not evaluated as JavaScript.
|
||||
|
||||
### 2.9 What core publishes about a module
|
||||
|
||||
`GET /api/v1/public/modules` — anonymous, database-free, never site-mode gated.
|
||||
@@ -1088,6 +1115,29 @@ and runs core's own `routeManifest.js`. Nothing else proves the URLs a module cl
|
||||
actually serves — a manifest frozen by hand goes stale silently, and the failure it would have caught
|
||||
is a route that moved.
|
||||
|
||||
**Built in slice 5, and what it does with that core is a SUBTRACTION.** The job generates the
|
||||
manifest without the module and then with it; the difference is what the module serves. Filtering the
|
||||
combined manifest by the module's prefixes would have answered only "what does the module serve".
|
||||
Subtracting also answers **"did core lose anything"** — and a module that shadowed or displaced a
|
||||
core route cannot appear as an addition anywhere, so that is the only way to see it. Three checks
|
||||
come out of one diff:
|
||||
|
||||
1. the added routes match the module's committed `routes.manifest.json`;
|
||||
2. **no route of core's was removed or changed**, which is `MODULE_SYSTEM.md` §1.2's promise;
|
||||
3. every added route has an operation in `swagger-fragment.json`, and every operation is an added
|
||||
route — §2.8's coverage requirement, answered against a running app rather than against a table.
|
||||
|
||||
The third is why this job matters beyond the manifest: everything else in a module's repo compares
|
||||
two strings that live in that repo. This compares a URL the module registers against a URL a real
|
||||
Express app reports serving, which is the only thing that can catch a fragment that is internally
|
||||
consistent and describes nothing.
|
||||
|
||||
Also learned here: **copy the module into the core checkout, never symlink it.** The loader filters
|
||||
its scan with `entry.isDirectory()`, which reports a link as a link and skips it silently — the
|
||||
manifest then comes out with no module routes and the diff looks like a module that registered
|
||||
nothing. And the client chunk must be **built before** the copy: `client.entry` is validated during
|
||||
the manifest step of the scan, so a missing chunk is a load failure, not a warning.
|
||||
|
||||
Pinning the ref rather than tracking `edge` is what keeps this from being a source of unexplained red
|
||||
Xes: core moves for reasons that have nothing to do with the module, and a bump is then a deliberate
|
||||
commit that says which core the module was last proved against.
|
||||
@@ -1142,6 +1192,25 @@ shipping one with the same name; the collision is logged and the module's versio
|
||||
`swagger-output.json` itself stays exactly what core's own routes generate, so `npm run swagger`
|
||||
remains reproducible on any machine regardless of what is installed.
|
||||
|
||||
Built as `server/swagger/docsSpec.js` (website#141). Three properties that are contract rather than
|
||||
implementation, because each one is a way the obvious version is wrong:
|
||||
|
||||
- **The committed spec is never mutated.** It is a `require()`d JSON module, so a merge in place
|
||||
would be permanent for the life of the process *and* cumulative across rebuilds — a module's paths
|
||||
outliving its own uninstall. Every rebuild starts from a structural copy.
|
||||
- **The Swagger UI is built per request too**, not bound once while `app.js` is still being required.
|
||||
Bound at require time it would show core's routes for the life of the process while
|
||||
`/api/docs.json` showed the merged set — two documents at two URLs, disagreeing.
|
||||
- **A bad fragment costs that module its paths and nothing else.** Missing, unreadable or not JSON is
|
||||
logged and skipped; the document still answers. That is §4.4's bargain — one module's failure is
|
||||
never the site's — and a docs page that 500s is strictly worse than one missing a module's routes.
|
||||
|
||||
`started` only, matching `clientEntryUrls()` rather than `clientChunks()`: the document is built when
|
||||
it is asked for, at which point the state is known, and documenting a module that 503s every one of
|
||||
those paths sends a client somewhere it cannot go. The cache key is a new
|
||||
`modules.version()` — a counter the loader bumps on every state *change*, which says nothing about
|
||||
which module moved or where to.
|
||||
|
||||
### 6.2 The client contract is much larger than §2.1 says
|
||||
|
||||
§2.1 lists three client registration calls and nothing else, implying React and the router are all a
|
||||
|
||||
@@ -730,7 +730,9 @@ navigate a module it does not contain, on a deployment that builds nothing — a
|
||||
while no existing URL has moved. The exit criterion held: `routes.manifest.json` went 229 → 230 across
|
||||
the whole phase, and the one added line is PR 6's deliberate `GET /api/v1/public/modules`.
|
||||
|
||||
**Phase 3 — Extract `module-uo`.** Moves out of `website/`: the 8 model directories and their 25
|
||||
**Phase 3 — Extract `module-uo`. COMPLETE 2026-08-11**, in six slices; the record of each is in
|
||||
§2.7.1 and all four acceptance criteria are met (the table at the end of slice 5). Moves out of
|
||||
`website/`: the 8 model directories and their 25
|
||||
tables; the nine UO `utils/` files plus `newsGump.js`; the 13 router/controller files;
|
||||
`scripts/importSpawnAtlas.js` and `db/spawnAtlas.art.json`; `usersShard.controller.js` **minus
|
||||
`getUser`** (§1.9); the shard-derived half of `notificationStreams.js` and the town-crier leg of
|
||||
@@ -782,7 +784,7 @@ Each slice is one `module-uo` PR (adds), one `website` PR (deletes), and one `do
|
||||
| 2 | **Client extension slots** | Core only, and the one slice that adds rather than moves: the client twin of `declareSlot`/`registerExtension` (API §3.7), the `site.footer.status` and `admin.users.detail` slots, and core filling both itself. `module-uo` untouched. |
|
||||
| 3 | **The whole client half** | **35 files / 5,332 lines** (measured; the 51/~3,700 above was counted differently) — all twelve public pages (`Shard`, `ShardActivity`, `Rules`, `Atlas`, `AtlasCreature`, `ChampSpawns`, `Market`, `MarketVendor`, `Governors`, `Guilds`, `Houses`, `Leaderboards`) under `/uo/*`, every admin and player view under `/admin/uo/*` and `/player/uo/*`, `PlayersOnline`, `VendorSales`, `CharacterStats`, `GameAccounts`, the `data/` and `lib/` UO leaves, the public nav rows, the feature provider, and all three slot fills. |
|
||||
| 4 | **De-UO core's copy** | `About`, `Screenshots`, `Website`, `Status`, `Wiki`, `SiteFooter`'s prose, `heroLayout`'s defaults, `brand.js`'s tagline + description, `db/seed.js`'s wiki copy, two user-visible `NavEditor` strings, the comments in `navOverrides.js`, and the 190 dead lines of `api/client.js`'s `shard`/`atlas` namespaces — plus the two settings rows core seeded for module-uo, and the §5.2 CI check that keeps all of it out. `README.md` deliberately deferred to slice 5. |
|
||||
| 5 | **Close the phase** | `module-uo`'s frozen route manifest and release workflow; `docs/modules/uo/` and `docs/modules/rust-dryrun.md` |
|
||||
| 5 | **Close the phase** | `module-uo`'s frozen route manifest and release workflow; `docs/modules/uo/` and `docs/modules/rust-dryrun.md` — **plus the OpenAPI fragment on both sides**, which §2.8/§6.1a had settled and neither repo had built, and the `README.md` slice 4 deferred |
|
||||
|
||||
##### Why the server half cannot be sliced — found 2026-08-11, before writing any of it
|
||||
|
||||
@@ -1229,6 +1231,140 @@ learned by getting things wrong first is worth recording:
|
||||
core's README is a rewrite that belongs with the phase-closing documentation pass rather than
|
||||
half-done inside a code slice.
|
||||
|
||||
#### Slice 5 — closing the phase, and the obligation nobody had noticed
|
||||
|
||||
Module-uo#6 + website#141 + docs#140 (2026-08-11). The slice table words this one as packaging and
|
||||
documentation: the module's frozen manifest and release workflow, `docs/modules/uo/` and the
|
||||
`module-rust` dry run. It is also where a **contract obligation that had never been built on either
|
||||
side** surfaced, and closing it was the larger half.
|
||||
|
||||
##### The OpenAPI fragment existed only on paper
|
||||
|
||||
[`MODULE_API.md`](MODULE_API.md) §2.8 and §6.1a settle it in detail: a module ships
|
||||
`swagger-fragment.json`, core merges the fragments of *started* modules into `/api/docs.json` at
|
||||
request time, core wins every collision. **Neither half was written.** Module-uo's 417 `#swagger`
|
||||
annotations came across in slice 1 and went nowhere; core's `swagger/mergeSpec.js` named the
|
||||
request-time caller in its own file header and that caller did not exist. The result was that the 72
|
||||
URLs module-uo serves were **in no OpenAPI spec at all** — this repo's standing rule (never ship a
|
||||
route that isn't in the spec) broken by the extraction rather than by a route.
|
||||
|
||||
It was folded into this slice rather than deferred to Phase 4, because Phase 3 closing with 72
|
||||
undocumented routes would have made the phase's own exit criteria untrue.
|
||||
|
||||
**The generator derives its prefixes rather than listing them.** `swaggerFragment.js` runs the
|
||||
module's own `register()` against a recording `api` and asks `require.cache` which file each router
|
||||
object came from — so a mount prefix lives in `server/index.js` and nowhere else, and swagger-autogen
|
||||
(which needs a *file*, and cannot follow `api.registerRoutes`) gets pointed at the right one. The two
|
||||
values it cannot derive, the tier base paths and the extension slot's mount, are §2.4's normative
|
||||
table — and they are not taken on trust: the frozen-manifest job checks every generated path against
|
||||
a real core.
|
||||
|
||||
**Schemas are namespaced; core's are referenced by core's name.** The 31 UO schemas moved out of
|
||||
core's `swagger.js` as `Uo…`, because core wins a collision and an un-namespaced `ShardStatus` from a
|
||||
second game's module would lose to or clobber this one. But the annotations keep pointing at
|
||||
`#/components/schemas/Error` and `ValidationError` **without** redefining them: those resolve in the
|
||||
merged document, which is the only place both halves exist. Shipping a copy would be a collision core
|
||||
correctly drops. Both directions verified against a live merge — 197 paths, no dangling `$ref`.
|
||||
|
||||
##### The frozen manifest is a subtraction, not a filter
|
||||
|
||||
§5.3 settles that the module's CI clones core at a pinned ref. What it does with that core is the
|
||||
design decision this slice made: generate the manifest **without** the module and then **with** it,
|
||||
and take the difference.
|
||||
|
||||
Filtering the combined manifest by the module's prefixes would have answered "what does the module
|
||||
serve". Subtracting answers that *and* "did core lose anything" — and the second is the one §1.2
|
||||
promises to the shipped Android app and the Discord bot. A module that shadowed or displaced a core
|
||||
route cannot show up as an addition anywhere; it shows up here as a **removal**. Result: 72 routes
|
||||
added, **0 removed**.
|
||||
|
||||
That job is also the only place in the module's repo where a claim meets ground truth. Everything
|
||||
else there compares two strings in the same repository; this compares a URL the module registers
|
||||
against a URL a real Express app reports serving, which is what makes the fragment's coverage check
|
||||
meaningful rather than self-referential.
|
||||
|
||||
##### Releases: the version is declared, not computed
|
||||
|
||||
`link` and `installer` both compute the next version from conventional-commit subjects. This module
|
||||
does not, because it already has one authoritative version — `module.json`'s, which is what core
|
||||
records in `installed_modules`, shows on the admin screen, and sits beside the `coreApi` range a bump
|
||||
has to be weighed against. Two sources for one number is how they drift. So: a merge to `main` that
|
||||
leaves `module.json` at a version with no release yet publishes one, and bumping the version is an
|
||||
ordinary reviewed PR.
|
||||
|
||||
The workflow **never writes to a branch** — it tags and publishes — so `main` needs no push
|
||||
exception. That is the installer's model, adopted for the reason it was adopted there. The bundle is
|
||||
assembled from an **include** list rather than an exclude list, because an exclude list ships
|
||||
whatever it forgot.
|
||||
|
||||
##### Six dropped annotations across two repos, and the tool that says `Success`
|
||||
|
||||
swagger-autogen reports an annotation it cannot parse and **then prints `Success` in green**, having
|
||||
skipped it. Nothing was listening in either repo. Making its diagnostics fatal immediately found:
|
||||
|
||||
- **core**: `POST /api/v1/admin/invites` and `POST /api/v1/auth/invite/:token/accept`, both
|
||||
documented with an **empty request body**, both since the day they were written;
|
||||
- **module-uo**: the same brace-short mistake twice more, plus two descriptions whose inner quoting
|
||||
the tool cannot survive — it re-quotes `"` and a backtick to `'` before evaluating, so either
|
||||
inside a single-quoted description ends the string early. A typed, described query parameter had
|
||||
been silently demoted to an untyped one.
|
||||
|
||||
This is the same class as §6.1's silent drop, with one difference: the tool did say something.
|
||||
|
||||
**And a seventh defect only a browser could show.** Nineteen descriptions carried an escaped
|
||||
apostrophe — correct JavaScript, and wrong here, because swagger-autogen does not evaluate the
|
||||
annotation as JS. The backslash survived into the fragment and Swagger UI rendered it verbatim to a
|
||||
reader, mid-sentence. The fragment was valid JSON, the paths were right, every test passed. Only
|
||||
opening the page found it: the §7.7 lesson, in a seam that has nothing to do with the client chunk.
|
||||
|
||||
##### The last of core's UO copy
|
||||
|
||||
`README.md`'s 48 mentions (the architecture diagram is now core + a module + "the game", and
|
||||
*Shard integration (uo-link)* is now *Modules*), the four orphan tags and 31 orphan schemas in
|
||||
`swagger.js`, `info.description`'s "a private Ultima Online shard", and `TOWNCRIER_DURATION_SEC` +
|
||||
`UOLINK_*` in the two `.env.example`s — module-read, never core-read, and now documented in the
|
||||
module's own README rather than half-copied in core's.
|
||||
|
||||
Auditing the tag list for the four orphans turned up the same defect pointing the other way: five
|
||||
tags **used** by core routes and never declared (`Admin · Email`, `Admin · Invites`,
|
||||
`Admin · Moderation`, `Admin · Pages`, `Auth · Me`).
|
||||
|
||||
##### The dry run
|
||||
|
||||
[`../modules/rust-dryrun.md`](../modules/rust-dryrun.md) — Phase 3's fourth acceptance criterion, and
|
||||
the only one that could fail in an interesting way. A written `module-rust`, for a game chosen
|
||||
because it shares almost nothing with UO: it **wipes** monthly, a community runs several **servers**
|
||||
rather than one shard, its identity is **Steam**, and its server ships **RCON** so there is no
|
||||
sidecar to write.
|
||||
|
||||
The contract generalises — same manifest, same seven registration calls, same schema-fragment rules,
|
||||
same client registry, and **six of the UI kit's seven members** wanted by a game with nothing in
|
||||
common with the one the kit was curated from.
|
||||
|
||||
It found one real gap: **a module cannot register an identity provider**, and "Sign in with Steam" is
|
||||
what a Rust community expects. A Rust module can still ship by copying UO's in-game-code flow, one
|
||||
screen worse. Recorded as the first candidate for a future `MODULE_API_VERSION` bump rather than
|
||||
bolted on now — an identity provider participates in session creation, which is the one part of core
|
||||
a module must never be able to weaken, and §2.7's link-only policy has to survive it.
|
||||
|
||||
It also found that two rules written for one reason cover another: §2.6's leading-verb allowlist
|
||||
(written because a fragment replays every boot) correctly forbids the wipe-truncation a Rust author
|
||||
would put in their schema, and `capabilities` — decoration with one module — is the only thing the
|
||||
Android app can ask about a module it has never heard of.
|
||||
|
||||
##### Phase 3 is complete
|
||||
|
||||
Six slices, 2026-08-11. Core is 158 routes and knows nothing about any game; module-uo is 72 routes,
|
||||
27 tables, 40 server files and 35 client files in its own repo, releasable, and documented in
|
||||
[`../modules/uo/`](../modules/uo/README.md). Every acceptance criterion in §2.7 is met:
|
||||
|
||||
| # | Criterion | Where it is proved |
|
||||
|---|---|---|
|
||||
| 1 | No UO identifier in core | `npm run check:modules`, first step of server-tests |
|
||||
| 2 | Zero internal imports from module into core | `npm run check:imports`, module CI |
|
||||
| 3 | Route manifest diff is only the deliberate move | core 158 + module 72, **0 core routes removed** |
|
||||
| 4 | A written `module-rust` dry run | [`../modules/rust-dryrun.md`](../modules/rust-dryrun.md) |
|
||||
|
||||
**Phase 4 — Delivery.** The admin-panel Modules screen (install, enable, disable, retry, purge,
|
||||
`startup_failed` with its recorded reason) and the Docker-environment path from §2.5. Deliberately
|
||||
last, so loader, packaging, schema and chunk-loading problems are not all being debugged at once.
|
||||
@@ -1381,3 +1517,6 @@ row for it — when it has content, not while it is an empty repo.
|
||||
| 15 | Criterion 1's grep reads **code, not prose**; core's UO copy is rewritten in its own slice instead | API §5.2, §2.7.1 |
|
||||
| 16 | `module-uo`'s CI checks core out at a **pinned ref** to generate its frozen route manifest | API §5.3 |
|
||||
| 17 | The `module-rust` dry run lands as `docs/modules/rust-dryrun.md`; the Integration Kit links to it | §2.7.1, §2.11 |
|
||||
| 18 | A module's frozen manifest is the **difference** between a core without it and the same core with it — never a prefix filter | API §5.3 |
|
||||
| 19 | A module's release version is **declared** in `module.json`, not computed from commit subjects; the workflow tags and publishes and never writes to a branch | §2.7.1 |
|
||||
| 20 | A module namespaces the schemas it **defines** and references core's shared ones by core's name | API §2.8, §6.1a |
|
||||
|
||||
@@ -1,18 +1,28 @@
|
||||
# Runic Gateway Website
|
||||
|
||||
Public site, wiki, and protected admin panel for a private Ultima Online shard — a
|
||||
full-stack app in one repo. Branding is instance-configurable via `BRAND_*` (see
|
||||
[Branding](#branding)); **UOMysticmoon** is the first instance.
|
||||
[](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
|
||||
[](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
|
||||
[](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
|
||||
[](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
|
||||
[](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
|
||||
[](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
|
||||
[](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
|
||||
|
||||
Public site, wiki, and protected admin panel for a game community — a full-stack app
|
||||
in one repo. Everything specific to a *particular* game lives in an installable
|
||||
module, not here. Branding is instance-configurable via `BRAND_*` (see
|
||||
[Branding](#branding)); **UOMysticmoon**, an Ultima Online shard, is the first
|
||||
instance, and its game half is
|
||||
[RunicGateway/Module-uo](https://gitea.whitlocktech.com/RunicGateway/Module-uo).
|
||||
|
||||
A full-stack app in one repo:
|
||||
|
||||
- **Backend** — Node.js + Express REST API (layered `router → controller → model → db`), MariaDB, a provider-agnostic session layer (JWT cookie for web, bearer tokens for mobile, pluggable SSO).
|
||||
- **Frontend** — React + Vite single-page app (public site, wiki, and the admin panel), dark "gothic" theme (Cinzel + Georgia).
|
||||
- **Deploy** — Docker Compose (app + MariaDB) behind a Pangolin reverse proxy. Express serves the built SPA in production.
|
||||
- **Shard link** — a live bridge to the in-game ServUO shard through the **uo-link** sidecar ([RunicGateway/link](https://gitea.whitlocktech.com/RunicGateway/link)): the site ingests a live event feed and makes server-side REST calls to show shard status, economy, staff presence, IDOCs, live activity, and per-character sheets. See [Shard integration (uo-link)](#shard-integration-uo-link).
|
||||
- **Moderation appeals** — a player whose linked Discord identity was banned or muted (per the bot's `mod_actions` log) can open an appeal from the player portal; staff claim and resolve appeals from an admin queue, and approving a ban/mute appeal best-effort reverses it in Discord automatically. See [MODERATION_APPEALS.md](MODERATION_APPEALS.md).
|
||||
- **Deploy** — Docker Compose (app + MariaDB) behind a reverse proxy (Pangolin, Nginx, Caddy, Traefik, …). Express serves the built SPA in production.
|
||||
- **Modules** — the game-specific half of a site is a module dropped onto a volume: it adds routes, database tables, nav entries and whole SPA pages without this repo knowing anything about the game. See [Modules](#modules).
|
||||
|
||||
The design reference is [BACKEND_DESIGN.md](BACKEND_DESIGN.md) (API contract, schema, security).
|
||||
The design reference is [BACKEND_DESIGN.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/BACKEND_DESIGN.md) (API contract, schema, security), in the [**RunicGateway/docs**](https://gitea.whitlocktech.com/RunicGateway/docs) repo — where all project documentation now lives.
|
||||
|
||||
---
|
||||
|
||||
@@ -30,20 +40,19 @@ The design reference is [BACKEND_DESIGN.md](BACKEND_DESIGN.md) (API contract, sc
|
||||
- [Pages & routes](#pages--routes)
|
||||
- [API endpoints](#api-endpoints)
|
||||
- [API documentation (Swagger)](#api-documentation-swagger)
|
||||
- [Shard integration (uo-link)](#shard-integration-uo-link)
|
||||
- [Modules](#modules)
|
||||
- [Environment variables](#environment-variables)
|
||||
- [Security](#security)
|
||||
- [Logging](#logging)
|
||||
- [Deployment behind Pangolin](#deployment-behind-pangolin)
|
||||
- [Deployment behind a reverse proxy](#deployment-behind-a-reverse-proxy)
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
How the pieces fit together — the React SPA and native app talk to one Express backend
|
||||
(`router → controller → model → db`), which persists to MariaDB and bridges to the live
|
||||
game world only through the **uo-link** sidecar. The shard itself is never internet-facing.
|
||||
See [ARCHITECTURE.md](ARCHITECTURE.md) for the fuller write-up.
|
||||
(`router → controller → model → db`), which persists to MariaDB. Anything that knows
|
||||
what game this site is about lives in an installed module, on the right of the diagram.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
@@ -63,32 +72,29 @@ flowchart TB
|
||||
subgraph backend["server/ — Express backend"]
|
||||
direction TB
|
||||
mw["Middleware<br/>helmet · siteMode · noindex<br/>rateLimit · loginProtection · botScore · validate"]
|
||||
router["Router /api/v1<br/>auth (web · mobile · sso) · public · admin"]
|
||||
router["Router /api/v1<br/>auth (web · mobile · sso) · public · admin · player"]
|
||||
ctrl["Controllers"]
|
||||
auth["Session layer (auth/)<br/>sessionService · JWT/cookie · bearer · SSO+PKCE"]
|
||||
model["Models (.model + .db)<br/>raw parameterized SQL — no ORM"]
|
||||
sse["SSE fan-out<br/>public stream (allowlist) · admin stream (sensitive)"]
|
||||
|
||||
subgraph shardutil["Shard integration (utils/)"]
|
||||
ingest["shardIngest.js<br/>WS ingest dispatcher"]
|
||||
restcli["uoLinkClient.js<br/>REST client (never throws)"]
|
||||
end
|
||||
|
||||
loader["modules/loader.js<br/>scans the volume · mounts · registries · lifecycle"]
|
||||
secret["secretBox.js<br/>AES-256-GCM secrets at rest"]
|
||||
end
|
||||
|
||||
bot["bot/<br/>Discord bot"]
|
||||
end
|
||||
|
||||
db[("MariaDB<br/>users · posts · wiki · settings · activity<br/>mobileSessions · authProviders · userIdentities<br/>uoLinkConfig · shard_online/economy/houses/events")]
|
||||
db[("MariaDB<br/>users · posts · wiki · settings · activity<br/>mobileSessions · authProviders · userIdentities<br/>installed_modules · <module>_*")]
|
||||
|
||||
%% ---------- Shard side ----------
|
||||
subgraph shardside["Game shard (never internet-facing)"]
|
||||
%% ---------- Module side ----------
|
||||
subgraph modside["modules/<id>/ — installed, not built (e.g. Module-uo)"]
|
||||
direction TB
|
||||
sidecar["uo-link sidecar<br/>(Rust) — the only bridge exposed"]
|
||||
servuo["ServUO shard<br/>(C# plugin)"]
|
||||
modsrv["server/ — routers, models, schema fragment<br/>reaches core only through ctx"]
|
||||
modcli["client/dist/entry.js — prebuilt ESM chunk<br/>React shared via window.__rg"]
|
||||
end
|
||||
|
||||
game["The game<br/>whatever the module talks to<br/>(for Module-uo: a ServUO shard,<br/>via the uo-link sidecar)"]
|
||||
|
||||
%% ---------- Edges ----------
|
||||
browser <-->|"same-origin JSON + SSE (cookie)"| mw
|
||||
mobile -->|"REST (bearer access/refresh)"| mw
|
||||
@@ -98,40 +104,42 @@ flowchart TB
|
||||
mw --> router --> ctrl
|
||||
ctrl --> auth
|
||||
ctrl --> model
|
||||
ctrl --> restcli
|
||||
ctrl --> sse
|
||||
auth --> model
|
||||
model <--> db
|
||||
auth -. reads/writes secrets .-> secret
|
||||
restcli -. reads config/token .-> secret
|
||||
ingest --> model
|
||||
ingest --> sse
|
||||
sse -->|"live events"| browser
|
||||
bot -->|"messages"| discord
|
||||
bot <--> db
|
||||
|
||||
restcli -->|"REST: /char /roster /economy /history · /link/confirm · /towncrier"| sidecar
|
||||
sidecar -->|"WebSocket live event feed (bearer + X-UOLink-Version)"| ingest
|
||||
servuo -->|"loopback TCP 127.0.0.1:7788<br/>newline-delimited JSON (shard dials out)"| sidecar
|
||||
loader -->|"mounts under /api/v1/<tier>/<prefix>"| router
|
||||
loader -->|"require() + register(ctx, api)"| modsrv
|
||||
modsrv -->|"ctx.db · ctx.push · ctx.activity …"| model
|
||||
modsrv <--> game
|
||||
browser -->|"<script type=module> injected by htmlShell"| modcli
|
||||
|
||||
%% ---------- Styling ----------
|
||||
classDef ext fill:#2d2233,stroke:#7a5c94,color:#e8dff0;
|
||||
classDef store fill:#1f2d2a,stroke:#4c8c7d,color:#dff0ea;
|
||||
classDef bridge fill:#2d2620,stroke:#94764c,color:#f0e6d8;
|
||||
class idp,discord ext;
|
||||
classDef mod fill:#2d2620,stroke:#94764c,color:#f0e6d8;
|
||||
class idp,discord,game ext;
|
||||
class db store;
|
||||
class sidecar,servuo bridge;
|
||||
class modsrv,modcli mod;
|
||||
```
|
||||
|
||||
- **One backend, layered.** Every request flows `middleware → router → controller → model → db`.
|
||||
Web browsers authenticate with an httpOnly JWT cookie; the native app uses short-lived bearer
|
||||
access tokens plus rotated refresh tokens; SSO (Google/Discord/OIDC) is link-only and PKCE-guarded.
|
||||
All three surfaces produce the *same* session via the session layer.
|
||||
- **The shard is never reachable.** The ServUO shard *dials out* over loopback TCP to the uo-link
|
||||
sidecar; only the sidecar is exposed, and only the backend talks to it. The REST client
|
||||
(`uoLinkClient.js`) never throws, so the site degrades gracefully when the shard is down.
|
||||
- **Sensitive events stay private.** Ingested game events fan out to browsers over two SSE channels —
|
||||
a public allowlist stream and an admin-only stream that adds staff audit / cheat / login events.
|
||||
- **Core knows nothing about any game.** Routes, tables, nav entries, SPA pages and push streams for
|
||||
a specific game arrive from a module the operator installed. Core provides the seams; the module
|
||||
fills them. See [Modules](#modules).
|
||||
- **A module that fails must never take the site down.** The loader catches failures across a
|
||||
module's whole lifecycle and marks that one module `startup_failed`; the site comes up with its
|
||||
routes and nav absent, and the admin panel says why.
|
||||
- **Sensitive events stay private.** Events fan out to browsers over two SSE channels — a public
|
||||
allowlist stream and an admin-only stream that adds staff audit / cheat / login events. Which
|
||||
event kinds are public is decided by the module that publishes them, and core enforces the split.
|
||||
|
||||
---
|
||||
|
||||
@@ -145,25 +153,26 @@ flowchart TB
|
||||
| Frontend | React 18, Vite 5, React Router 6 |
|
||||
| Email | Nodemailer via Gmail OAuth2 (configured in admin), with a `mailto:` fallback |
|
||||
| API docs | OpenAPI 3.0 via `swagger-autogen`, served with `swagger-ui-express` at `/api/docs` |
|
||||
| Deploy | Docker Compose, Pangolin reverse proxy |
|
||||
| Deploy | Docker Compose, any reverse proxy (Pangolin, Nginx, Caddy, Traefik, …) |
|
||||
|
||||
---
|
||||
|
||||
## Project structure
|
||||
|
||||
```
|
||||
UOMSITE/
|
||||
website/
|
||||
├─ server/ Express API
|
||||
│ ├─ src/
|
||||
│ │ ├─ server.js bootstrap: ensure schema → seed → listen (0.0.0.0)
|
||||
│ │ ├─ app.js middleware + static SPA + routes
|
||||
│ │ ├─ auth/ session layer: session.service · token (JWT/cookies) · session.middleware · ssoState (PKCE/CSRF) · providers/ (base · oauth2 · google · discord · genericOidc · registry)
|
||||
│ │ ├─ router/v1/ auth (web · mobile · sso) / public / admin route groups
|
||||
│ │ ├─ model/ users · posts · wiki · settings · activity · mobileSessions · authProviders · userIdentities (.model + .db)
|
||||
│ │ ├─ router/v1/ auth (web · mobile · sso) / public / admin / player route groups
|
||||
│ │ ├─ model/ users · posts · wiki · settings · activity · mobileSessions · authProviders · userIdentities · modules (.model + .db)
|
||||
│ │ ├─ modules/ loader (scan · validate · mount) · registries (the seams) · lifecycle (boot/shutdown + reconcile)
|
||||
│ │ ├─ middleware/ siteMode · noindex · rateLimit · loginProtection · botScore · validate
|
||||
│ │ └─ utils/ auth (compat facade) · totp (2FA) · secretBox (AES-GCM secrets) · db (pool) · mailer · logger
|
||||
│ │ └─ utils/ auth (compat facade) · totp (2FA) · secretBox (AES-GCM secrets) · db (pool) · mailer · logger · htmlShell
|
||||
│ ├─ db/ schema.sql + seed.js
|
||||
│ ├─ swagger/ swagger.js (OpenAPI generator config) + swagger-output.json (generated spec)
|
||||
│ ├─ swagger/ swagger.js (generator config) · swagger-output.json (generated, core only) · docsSpec.js (merges module fragments at request time)
|
||||
│ └─ .env.example
|
||||
├─ client/ React + Vite SPA
|
||||
│ ├─ src/
|
||||
@@ -172,9 +181,11 @@ UOMSITE/
|
||||
│ │ ├─ routes/admin/ AdminLogin (password + TOTP + SSO buttons), AdminLayout, views/ (Dashboard, Posts, Wiki, Settings, Activity, Bot Activity, Authentication, Users, Account) + editors
|
||||
│ │ ├─ components/ SiteHeader, SiteFooter, layout, guards, Modal, ProviderIcon (inline SSO SVGs), …
|
||||
│ │ ├─ contexts/ AuthContext, SiteContext
|
||||
│ │ ├─ modules/ the client registry: routes · nav · slots · feature gates · window.__rg
|
||||
│ │ ├─ api/client.js fetch wrapper (sends cookies)
|
||||
│ │ └─ styles/theme.css design tokens
|
||||
│ └─ public/assets/img/ hero image
|
||||
├─ modules/ installed modules, one directory each — a Docker bind mount; empty here
|
||||
├─ Dockerfile builds client → serves via Express
|
||||
├─ docker-compose.yml app + MariaDB
|
||||
├─ .env.example root env (used by Compose)
|
||||
@@ -216,6 +227,10 @@ IMAGE_TAG=sha-042a151 docker compose pull && docker compose up -d
|
||||
- Health check: `GET http://localhost:3000/api/health` → `{ "status": "ok" }`
|
||||
- Logs: `docker compose logs -f app` (and `./logs/app.log` on the host)
|
||||
- Stop: `docker compose down` (add `-v` to also wipe the database + uploads volumes)
|
||||
- Modules: installed into `./modules` on the host (bind-mounted to `/app/modules`), never baked into
|
||||
the image — an operator adds one to a pull-only deployment without building anything. Adding or
|
||||
removing one takes a `docker compose restart app`; the scan is synchronous at startup. See
|
||||
[`modules/README.md`](modules/README.md).
|
||||
|
||||
**Build the images locally instead of pulling** (offline, or to test an unmerged change) — overlay
|
||||
the dev file, which adds `build:` back:
|
||||
@@ -296,7 +311,7 @@ npm start # node server → serves API + SPA at http://localhost:3
|
||||
| `/site/screenshots` | Screenshot gallery |
|
||||
| `/site/five-on-friday` | Five on Friday |
|
||||
| `/site/newsletter` · `/site/newsletter/:id` | Newsletter list + issue |
|
||||
| `/site/about` · `/site/status` | About · Shard status |
|
||||
| `/site/about` · `/site/status` | About · Site status |
|
||||
| `/wiki` · `/wiki/:slug` | Wiki landing + article (auto table-of-contents) |
|
||||
|
||||
**Admin** (cookie auth, `noindex`):
|
||||
@@ -314,6 +329,13 @@ npm start # node server → serves API + SPA at http://localhost:3
|
||||
| `/admin/users` | User management |
|
||||
| `/admin/account` | Account security (self-service TOTP two-factor + linked SSO accounts) |
|
||||
|
||||
**Player** (any signed-in account, `noindex`): `/player` and its self-service views. Staff are a
|
||||
superset of players and reach these too.
|
||||
|
||||
An installed module adds its own pages under `/<id>/*`, `/admin/<id>/*` and `/player/<id>/*` — for
|
||||
Module-uo that is `/uo/shard`, `/admin/uo/link`, `/player/uo/characters` and the rest. Core does not
|
||||
know their names; they arrive with the module and are interleaved into the nav.
|
||||
|
||||
---
|
||||
|
||||
## API endpoints
|
||||
@@ -325,13 +347,19 @@ npm start # node server → serves API + SPA at http://localhost:3
|
||||
| SSO | `/api/v1/auth` (`providers` — public discovery; `sso/:provider/start`, `sso/:provider/link`, `sso/:provider/callback`) | redirect flow |
|
||||
| Public | `/api/v1/public` (`settings`, `status`, `posts/:category`, `posts/:category/:idOrSlug`, `wiki`, `wiki/:slug`, `contact`) | none |
|
||||
| Admin | `/api/v1/admin` (`dashboard`, `site-mode`, `posts`, `posts/upload`, `wiki`, `settings`, `activity`, `bot-activity`, `bot-activity/unban`, `auth/providers` (CRUD), `users`, `account`, `account/totp/*`, `account/identities`) | cookie (admin) |
|
||||
| Public · Shard | `/api/v1/public/shard` (`status`, `feed`, `economy`, `online`, `idoc`, `stream`) | none |
|
||||
| Player · Shard | `/api/v1/player/shard` (`link`, `accounts`, `roster/:account`, `vendors/:account`, `char/:serial`, `sales`) | cookie/bearer (player) |
|
||||
| Admin · Shard | `/api/v1/admin/shard` (self linking, same as player) · `/api/v1/admin/uo-link` (`config`, `towncrier`, `stream`) | cookie (staff / admin) |
|
||||
| Player | `/api/v1/player` (`me`, credentials, 2FA, identities, appeals) | cookie/bearer (any signed-in account) |
|
||||
| Modules | `/api/v1/public/modules` — id, name, version and capabilities of the modules currently serving | none |
|
||||
|
||||
**Module routes are not in this table**, because they are not core's. An installed module mounts
|
||||
under `/api/v1/public/<prefix>`, `/api/v1/admin/<prefix>` and `/api/v1/player/<prefix>`; which
|
||||
prefixes exist depends on what is installed. Module-uo, for instance, serves 72 routes under
|
||||
`/shard`, `/atlas` and `/uo-link` — see its own
|
||||
[`routes.manifest.json`](https://gitea.whitlocktech.com/RunicGateway/Module-uo/src/branch/main/routes.manifest.json).
|
||||
On a running instance, `/api/docs` lists everything, core and modules together.
|
||||
|
||||
Post categories (URL form): `news`, `five-on-friday`, `newsletter`, `screenshots`.
|
||||
`authMethod` on a session ∈ `local · totp · mobile · google · discord · oidc`.
|
||||
See [BACKEND_DESIGN.md](BACKEND_DESIGN.md) §4 for the full contract, or the interactive Swagger
|
||||
See [BACKEND_DESIGN.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/BACKEND_DESIGN.md) §4 for the full contract, or the interactive Swagger
|
||||
docs below for a per-endpoint reference (parameters, request bodies, response codes).
|
||||
|
||||
---
|
||||
@@ -367,88 +395,132 @@ cd server
|
||||
npm run swagger # → server/swagger/swagger-output.json
|
||||
```
|
||||
|
||||
`swagger.js` post-processes the generator's output in two ways before writing it:
|
||||
|
||||
- **Trailing slashes are stripped from path keys.** swagger-autogen builds a path by
|
||||
string-concatenating the mount prefix with the route argument, so a capability router mounted at
|
||||
`/users` whose collection route is `router.get('/')` would document as `/api/v1/admin/users/` —
|
||||
a URL no client calls, while dropping the one they all do. Express is indifferent (non-strict
|
||||
routing treats the two as one route), but the published spec is a contract.
|
||||
- **Path keys are sorted.** The generator emits them in router-traversal order, so moving a route
|
||||
between files rewrote most of this ~5k-line committed artifact even when the API was provably
|
||||
unchanged. Sorting keeps the diff proportional to the change. OpenAPI attaches no meaning to path
|
||||
order, and `scripts/routeManifest.js` already sorts for the same reason.
|
||||
|
||||
If the generated spec is missing, the server logs a warning and simply disables `/api/docs` (it does
|
||||
not crash).
|
||||
|
||||
**The committed spec is core only, and the served one is not.** swagger-autogen is *static
|
||||
analysis* — it parses `src/app.js` as text and follows the literal `app.use(…)` chain — so it can
|
||||
see neither an installed module (which arrives on a volume long after the image was built, and
|
||||
mounts through a call no parser can follow) nor an extension slot (whose router is created empty and
|
||||
filled later). Both are handled by merging a **fragment**:
|
||||
|
||||
- **Extension slots** contribute at generation time, from `server/swagger/slotSpecs.js`, so they are
|
||||
in the committed file.
|
||||
- **Modules** contribute at request time, from the `swagger-fragment.json` each one ships, merged by
|
||||
`server/swagger/docsSpec.js`. So `/api/docs.json` on a running instance describes more than
|
||||
`npm run swagger` produces here, and `swagger-output.json` stays reproducible on any machine
|
||||
regardless of what is installed.
|
||||
|
||||
**Core wins every key collision** — a module cannot redefine a core path, tag or schema by shipping
|
||||
one with the same name; the collision is logged and the module's version dropped.
|
||||
|
||||
One thing worth knowing if you edit an annotation: swagger-autogen **reports a broken one and then
|
||||
succeeds anyway**, dropping it. `npm run swagger` now captures those diagnostics and fails, which is
|
||||
how two annotations that had been silently documenting an empty request body were found. If it
|
||||
rejects yours, the usual causes are an object literal a brace short, or a `"` or backtick inside a
|
||||
single-quoted description (it re-quotes both to `'` before evaluating).
|
||||
|
||||
### The route manifest (frozen URL surface)
|
||||
|
||||
`server/routes.manifest.json` is a generated, sorted `{ method, path }` list of every route the two
|
||||
Express listeners actually expose. It is **not** documentation — it is the machine-checkable freeze of
|
||||
the URL surface, so that carving the router files up by business capability
|
||||
(`docs/website/API_V2_PLAN.md`) can be proved to move no URL instead of merely claiming it.
|
||||
|
||||
```bash
|
||||
cd server
|
||||
npm run routes:manifest # → routes.manifest.json + routes.guards.json
|
||||
npm run routes:manifest -- --check # exit 1 if either file is stale (what CI runs)
|
||||
```
|
||||
|
||||
The generator walks the live Express stack (runtime introspection, not source parsing — a route's path
|
||||
sits on the line *after* `router.get(`, which defeats greps) and keeps only
|
||||
`/api/**` and `/.well-known/**` plus the internal listener. The SPA catch-all, `/uploads`, `/brand`
|
||||
and installed modules' `/modules/<id>` chunks are filesystem-conditional static mounts, not API
|
||||
contract, so they are excluded and the output depends neither on whether the client has been built
|
||||
nor on which modules are mounted.
|
||||
|
||||
Two generated files, two very different meanings:
|
||||
|
||||
| File | Meaning of a diff |
|
||||
|---|---|
|
||||
| `routes.manifest.json` | **Contract change.** A URL moved. Justify it in the PR description; never let one ride along in a "mechanical" refactor. |
|
||||
| `routes.guards.json` | **Review aid.** Per route: handler count + the *named* middleware on its mount chain. Names are a hint only — `requireRole(...)` returns an anonymous arrow and cannot be seen — but a vanished `requireAuth` is unambiguous. |
|
||||
|
||||
Unlike the Swagger spec, the manifest is annotation-free: `swagger-output.json` documents intent (only
|
||||
annotated routes appear), the manifest records reality.
|
||||
|
||||
---
|
||||
|
||||
## Shard integration (uo-link)
|
||||
## Modules
|
||||
|
||||
The site is wired to the live in-game world through **uo-link**, a standalone sidecar service that
|
||||
runs next to the ServUO shard. Its source lives in a separate repo:
|
||||
**[RunicGateway/link](https://gitea.whitlocktech.com/RunicGateway/link)**. uo-link speaks the shard's internals and
|
||||
exposes a small, authenticated HTTP + WebSocket API; this website is a *client* of it. The shard
|
||||
itself is never exposed to the internet — only the sidecar is, and only the website's backend talks
|
||||
to it.
|
||||
**Everything specific to a game is a module.** Core has no idea what an "account", a "character" or
|
||||
a "shard" is; it provides seams, and a module fills them. That is what makes one image able to run a
|
||||
site for any game rather than for Ultima Online in particular.
|
||||
|
||||
### How it works
|
||||
The design of record is
|
||||
[MODULE_SYSTEM.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_SYSTEM.md);
|
||||
the normative contract — the one to read before writing a module — is
|
||||
[MODULE_API.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_API.md).
|
||||
The worked example is [RunicGateway/Module-uo](https://gitea.whitlocktech.com/RunicGateway/Module-uo),
|
||||
which is where everything this README used to describe under *Shard integration (uo-link)* now
|
||||
lives: the sidecar client, the ingest dispatcher, account linking, the town crier, the spawn atlas,
|
||||
and every page that renders them.
|
||||
|
||||
### An operator never builds anything
|
||||
|
||||
That constraint shapes the whole design. Installing a module is the WordPress-plugin experience — an
|
||||
admin-panel action, or a directory dropped onto the `modules/` volume — because production runs a
|
||||
prebuilt, pull-only image with no toolchain in it. So a module ships **assembled**: its client half
|
||||
is a prebuilt ESM chunk that resolves React from a `window.__rg` global core owns (an import map
|
||||
would have to be inline, and the CSP is `script-src 'self'`), and its one runtime dependency travels
|
||||
inside the tarball.
|
||||
|
||||
```
|
||||
ServUO shard ──▶ uo-link sidecar (RunicGateway/link) ──▶ website backend ──▶ browser
|
||||
REST + WebSocket, bearer-auth ingest + REST same-origin JSON/SSE
|
||||
modules/
|
||||
└─ uo/ one directory per module; the id is the directory name
|
||||
├─ module.json id, version, coreApi range, mounts, extensions, capabilities
|
||||
├─ swagger-fragment.json merged into /api/docs.json while the module is running
|
||||
├─ server/ routers, models, and an idempotent schema.sql fragment
|
||||
└─ client/dist/entry.js the prebuilt chunk, injected by utils/htmlShell.js
|
||||
```
|
||||
|
||||
- **Connection is admin-managed, not env.** The sidecar's base URL, WebSocket URL, shared-secret
|
||||
token, and protocol version are stored in the database (`uoLinkConfig`), edited from the
|
||||
**Admin → Shard** panel. The token is **encrypted at rest** (AES-256-GCM) and is **write-only** in
|
||||
the API — it is never returned to any client and never sent to the browser. Every call the backend
|
||||
makes carries `Authorization: Bearer <token>` and an `X-UOLink-Version` header (a protocol
|
||||
mismatch fails fast with `409` instead of being mis-parsed).
|
||||
- **Live ingest (WebSocket).** When enabled, the backend opens an outbound WebSocket to the sidecar
|
||||
and receives a stream of game events — `mob.login`/`logout`, `char.vitals`, `economy.supply`,
|
||||
`vendor.sale`, `player.death`/`murdered`, `house.decay` (IDOC), staff `audit.*`/`cheat.*`,
|
||||
`link.request`, and `server.hello`/`shutdown`. A single dispatcher (`utils/shardIngest.js`) routes
|
||||
each event: state-changing kinds update `shard_online` / `shard_economy` / `shard_houses`; notable
|
||||
kinds are appended to an append-only `shard_events` log; high-frequency kinds (vitals, supply
|
||||
ticks) only update state and are not logged. A changed boot id on `server.hello` is detected as a
|
||||
restart and stale "online" rows are cleared. On reconnect the backend backfills missed events via
|
||||
the sidecar's `/history`.
|
||||
- **Live round-trips (REST).** For point-in-time reads the backend calls the sidecar directly —
|
||||
`/char/serial/:serial`, `/roster/:account`, `/vendors/:account`, `/economy`, `/history` — plus
|
||||
commands `/link/confirm` and `/towncrier`. The REST client (`utils/uoLinkClient.js`) **never
|
||||
throws**: every call returns `{ ok, data, status }`, so a shard that is down or mid-restart
|
||||
degrades to a `503`/retry banner instead of a 500.
|
||||
- **Fan-out to the browser.** Ingested events are pushed to browsers over **Server-Sent Events**.
|
||||
Two channels exist: a **public** stream carrying only a safe allowlist of kinds, and an
|
||||
**admin-only** stream that also includes sensitive kinds (staff audit, cheat detection, login
|
||||
attempts, IPs). Sensitive kinds can never leak onto the public channel.
|
||||
`modules/` is a bind mount in `docker-compose.yml`, so placing a directory there by hand is a
|
||||
supported install. The directory is tracked in git (via its README) on purpose: Docker recreates a
|
||||
*missing* bind-mount source as `root:root`, and the container is uid 1000.
|
||||
|
||||
### Account linking
|
||||
### What a module gets, and what it may not do
|
||||
|
||||
A player (or staff member) proves ownership of a game account without sharing any game credentials:
|
||||
At boot, `app.js` scans the volume synchronously, validates each `module.json`, and calls the
|
||||
module's `register(ctx, api)`:
|
||||
|
||||
1. In game, the player runs **`[link`** and receives a one-time code.
|
||||
2. On the website (Player portal, or Admin → Account for staff) they enter the code.
|
||||
3. The backend confirms the code with the sidecar (`POST /link/confirm`), which permanently tags the
|
||||
game account with the website user id, and mirrors the link locally in `shard_account_links`.
|
||||
- **`ctx` is everything core hands over** — the database, the logger, settings, the session reader,
|
||||
push, the secret box, the middleware, the rate-limit factory, the activity log, and **express
|
||||
itself**. A module lives outside `server/`, so Node's resolver never reaches core's
|
||||
`node_modules`; anything it must share has to be handed to it, or there would be two Expresses and
|
||||
two Reacts in one process.
|
||||
- **`api` is everything it may register** — routes (one prefix per tier), an extension slot fill,
|
||||
notification streams, a news-announce leg, a post hook, and `onBoot`/`onShutdown`.
|
||||
- **It may not reach into core's tree**, mount outside its declared prefixes, or create tables
|
||||
outside its `<id>_` prefix. Each of those is checked, in the module's CI and again by the loader.
|
||||
|
||||
That mirror is the authorization basis for character reads: roster/vendor/character-sheet endpoints
|
||||
are **ownership-checked** so a user only sees accounts they linked. **Admins may view any
|
||||
character**; players and editor/moderator staff are limited to their own linked accounts.
|
||||
Two things are guaranteed regardless of what a module does. **A failure never takes the site down**:
|
||||
the loader catches everything from `require` to `onBoot`, marks that module `startup_failed`, and
|
||||
the site comes up with its routes and nav absent and the reason on the admin screen. And **no URL of
|
||||
core's may move** — a module that displaced one is caught by the frozen route manifest, which is
|
||||
generated from a real core with the module loaded.
|
||||
|
||||
### What each audience sees
|
||||
### What is running right now
|
||||
|
||||
| Surface | Endpoints | Who | Data |
|
||||
|---|---|---|---|
|
||||
| **Public** | `/api/v1/public/shard/*` (`status`, `feed`, `economy`, `online`, `idoc`, `stream`) | anyone | Shard up/down, gold-supply series, IDOC houses, a curated live feed, and **"Staff online"** — only players whose account is linked to a **staff** user (admin/editor/moderator), shown by name. Their in-game **map location is only included for admin/moderator viewers** — for players and the public it is stripped from the payload entirely (server-enforced, not just hidden in the UI). Linked *players* are never listed publicly; no vitals or account are exposed. |
|
||||
| **Player** | `/api/v1/player/shard/*` (`link`, `accounts`, `roster/:account`, `vendors/:account`, `char/:serial`, `sales`) | logged-in player | Their own linked accounts: character rosters, character sheets, player-vendor snapshots, and recent vendor sales. |
|
||||
| **Admin** | `/api/v1/admin/shard/*` (self-linking, same as player) · `/api/v1/admin/uo-link/*` (`config`, `towncrier`, `stream`) | staff / admin | Staff link their own accounts like players; **admins** additionally read *any* character's data, edit the sidecar connection config, publish/remove **town-crier** messages, and subscribe to the full event stream (incl. audit/cheat). |
|
||||
```
|
||||
GET /api/v1/public/modules
|
||||
{ "modules": [ { "id": "uo", "name": "Ultima Online", "version": "0.3.0",
|
||||
"capabilities": ["shard", "atlas", "market", …] } ] }
|
||||
```
|
||||
|
||||
The sidecar URL and token are set once in **Admin → Shard**; if uo-link is not configured (or the
|
||||
shard is offline), every shard surface degrades gracefully — the public page still renders, showing
|
||||
the shard as offline.
|
||||
Anonymous, database-free, never site-mode gated, and **`started` modules only** — a module that is
|
||||
disabled or failed is absent, exactly as its routes and its nav already are. Clients feature-detect
|
||||
against it; they do not use it to decide what to load (the HTML shell injects each chunk's tag).
|
||||
|
||||
---
|
||||
|
||||
@@ -461,12 +533,13 @@ Copy `.env.example` (Compose) or `server/.env.example` (local) and fill in. **`.
|
||||
| `NODE_ENV` | `production` | |
|
||||
| `PORT` | `3000` | server listens on `0.0.0.0:PORT` |
|
||||
| `UPLOAD_DIR` | `<server>/uploads` | where post images are written (`/app/uploads`, volume-mounted, in Compose) |
|
||||
| `MODULES_DIR` | `<repo>/modules` | where installed modules are scanned from (`/app/modules`, bind-mounted, in Compose) |
|
||||
| `DB_HOST` / `DB_PORT` | `db` / `3306` | `db` in Compose; `127.0.0.1` for local dev |
|
||||
| `DB_NAME` / `DB_USER` / `DB_PASSWORD` | `runic_gateway` / `runic` / — | app database credentials |
|
||||
| `DB_ROOT_PASSWORD` | — | MariaDB root (Compose only) |
|
||||
| `JWT_SECRET` | — | **required** — long random string; signs session, mobile, and SSO-flow tokens |
|
||||
| `JWT_EXPIRES_IN` | `1d` | web session token + cookie lifetime |
|
||||
| `COOKIE_SECURE` | `auto` | `auto` = Secure only over HTTPS (works on LAN HTTP + Pangolin HTTPS) |
|
||||
| `COOKIE_SECURE` | `auto` | `auto` = Secure only over HTTPS (works on LAN HTTP + proxy HTTPS) |
|
||||
| `COOKIE_NAME` | `rg_token` | changing it on a live instance invalidates existing sessions |
|
||||
| `BRAND_*` | Runic Gateway | instance branding (name, tagline, colors, logo/hero/favicon) — see [Branding](#branding) |
|
||||
| `SECRET_ENC_KEY` | — | **required in prod** — key for AES-256-GCM encryption of stored OAuth client secrets. Dev falls back to a key derived from `JWT_SECRET` (with a warning) |
|
||||
@@ -482,15 +555,14 @@ Copy `.env.example` (Compose) or `server/.env.example` (local) and fill in. **`.
|
||||
| `CLIENT_ORIGIN` | `http://localhost:5173` | enables CORS in dev only |
|
||||
| `LOG_LEVEL` / `FILE_LOG_LEVEL` | `info` / `debug` | console / file verbosity |
|
||||
| `LOG_TO_FILE` / `LOG_DIR` / `LOG_FILE` | `true` / `<server>/logs` / `app.log` | log file (bind-mounted to `./logs` in Docker) |
|
||||
| `ANNOUNCE_POLL_MS` | `15000` | how often the news-announcement dispatcher sweeps `announce_job_legs` for due/retry legs (whichever are registered — Discord is core's, the town crier is module-uo's) |
|
||||
| `TOWNCRIER_DURATION_SEC` | `3600` | how long a news post's in-game town-crier message stays up (≤ `86400`) |
|
||||
| `ANNOUNCE_POLL_MS` | `15000` | how often the news-announcement dispatcher sweeps `announce_jobs` for legs that are due or retrying. Which legs exist is up to what has registered one — Discord is core's; a module may add its own |
|
||||
|
||||
---
|
||||
|
||||
## Branding
|
||||
|
||||
Instance identity is data, not code — set via `BRAND_*` env vars, so one prebuilt
|
||||
image can run as any shard. With none set, everything renders as **Runic Gateway**.
|
||||
image can run as any community. With none set, everything renders as **Runic Gateway**.
|
||||
|
||||
| Var | What |
|
||||
|---|---|
|
||||
@@ -507,7 +579,7 @@ API (`SiteContext`), so no rebuild is needed; the server templates `index.html`
|
||||
settings override `BRAND_NAME` / `BRAND_CONTACT_EMAIL` when set. Image assets are
|
||||
delivered from the `./brand` bind-mount (see `brand/README.md`).
|
||||
|
||||
**UOMysticmoon** is the first instance — [`.env.uomysticmoon.example`](https://gitea.whitlocktech.com/RunicGateway/website/src/branch/main/.env.uomysticmoon.example)
|
||||
**UOMysticmoon** is the first instance — [`.env.uomysticmoon.example`](.env.uomysticmoon.example)
|
||||
holds the exact `BRAND_*` + infra (`DB_NAME`/`DB_USER`/`COOKIE_NAME`) pinning to
|
||||
run this repo as UOMysticmoon.
|
||||
|
||||
@@ -571,7 +643,7 @@ run this repo as UOMysticmoon.
|
||||
**Platform**
|
||||
|
||||
- `helmet`, admin routes `noindex` + `robots.txt` disallow, `trust proxy` for correct client IPs
|
||||
behind Pangolin (see `TRUST_PROXY`), first admin seeded from env (no hardcoded credentials),
|
||||
behind a reverse proxy (see `TRUST_PROXY`), first admin seeded from env (no hardcoded credentials),
|
||||
`.env` git-ignored. Passwords and request bodies are never logged. Email sends through Gmail
|
||||
OAuth2 configured in the admin (refresh token stored AES-GCM-encrypted, never in env); the
|
||||
contact form falls back to a `mailto:` link when unconfigured.
|
||||
@@ -598,10 +670,60 @@ bind-mounted to `./logs/app.log` and `docker compose logs -f app` shows the cons
|
||||
|
||||
---
|
||||
|
||||
## Deployment behind Pangolin
|
||||
## Deployment behind a reverse proxy
|
||||
|
||||
`docker compose up -d --build` exposes the `app` container on `0.0.0.0:3000` (no `127.0.0.1`
|
||||
binding) so Pangolin can reach it. Point a Pangolin resource at `app:3000`. Because `COOKIE_SECURE`
|
||||
defaults to `auto`, the admin login works both directly via the LAN IP over HTTP **and** through
|
||||
Pangolin over HTTPS — no config change needed. MariaDB stays on the private Compose network
|
||||
(no published port by default); data persists in the `dbdata` volume, uploads in `uploads`.
|
||||
binding) so a reverse proxy — Pangolin, Nginx, Caddy, Traefik, etc. — can reach it. Point the
|
||||
proxy at `app:3000` (or the host's `:3000` if the proxy runs outside Compose) and terminate TLS
|
||||
there. Because `COOKIE_SECURE` defaults to `auto`, the admin login works both directly via the
|
||||
LAN IP over HTTP **and** through the proxy over HTTPS — no config change needed. MariaDB stays on
|
||||
the private Compose network (no published port by default); data persists in the `dbdata` volume,
|
||||
uploads in `uploads`.
|
||||
|
||||
Set `TRUST_PROXY` so Express reads the real client IP from the proxy's `X-Forwarded-For` header
|
||||
(see [Environment variables](#environment-variables)) — required for rate limiting, bot scoring,
|
||||
and correct logging. Forward the standard `X-Forwarded-For` and `X-Forwarded-Proto` headers from
|
||||
your proxy.
|
||||
|
||||
Minimal proxy examples:
|
||||
|
||||
```nginx
|
||||
# Nginx
|
||||
location / {
|
||||
proxy_pass http://app:3000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
```
|
||||
|
||||
```caddy
|
||||
# Caddy — Caddyfile (automatic HTTPS; forwards X-Forwarded-* by default)
|
||||
your.domain {
|
||||
reverse_proxy app:3000
|
||||
}
|
||||
```
|
||||
|
||||
**Pangolin:** create a resource targeting `app:3000`; it forwards the required headers and
|
||||
terminates HTTPS out of the box, so no extra configuration is needed.
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
Runic Gateway is free software, licensed under the **GNU General Public License
|
||||
v3.0 or later** — see [LICENSE.md](LICENSE.md).
|
||||
|
||||
Copyright (C) 2026 Runic Gateway
|
||||
|
||||
This program is free software: you can redistribute it and/or modify it under
|
||||
the terms of the GNU General Public License as published by the Free Software
|
||||
Foundation, either version 3 of the License, or (at your option) any later
|
||||
version. It is distributed WITHOUT ANY WARRANTY; without even the implied
|
||||
warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
||||
General Public License for more details.
|
||||
|
||||
Contributions are welcome — please read [CONTRIBUTING.md](CONTRIBUTING.md) (note
|
||||
the **AI-usage disclosure** requirement) and our
|
||||
[Code of Conduct](CODE_OF_CONDUCT.md). Report vulnerabilities privately per
|
||||
[SECURITY.md](SECURITY.md).
|
||||
|
||||
Reference in New Issue
Block a user