From 0cc561334ce389285446bec686a3ab3381e0d10f Mon Sep 17 00:00:00 2001 From: wtclaude Date: Mon, 10 Aug 2026 01:44:29 -0500 Subject: [PATCH] docs(website): add the module system design of record Turns Runic Gateway from a UO-specific platform into a game-agnostic one: game-specific routes, tables, screens and nav leave the core website and become an installable module. Operators install the base site, install the module for their game, and restart -- they never build anything. Verified against the working trees rather than written from the draft. The draft's load-bearing assumptions that did not survive: * one flat module route prefix cannot coexist with URL preservation, since UO routes span three access tiers -- modules own a named slot per tier * there is no server-side nav list, and navOverrides.js refuses to have one, so nav registration is client-side * the nav feature-gate mechanism is itself the shard visibility system * there is no migration system to model a module runner on -- schema.sql is replayed idempotently every boot, so modules ship fragments * boot/shutdown holds eight UO call sites with no hook to receive them * notificationStreams, pushDispatch and announceWorker are entangled, not moves * six UO routes are nested under the core users resource * the Discord bot has no UO logic at all * the installer never contacts the website and its Bundle is hardcoded to two components, so delivery is website-side * routeManifest and swagger walk app.js with no DB, so modules mount synchronously from the filesystem * production is a prebuilt pull-only image and CSP forbids inline script, which together decide how the client half loads Ten decisions are recorded as settled in Part 3. Co-Authored-By: Claude --- website/MODULE_SYSTEM.md | 464 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 464 insertions(+) create mode 100644 website/MODULE_SYSTEM.md diff --git a/website/MODULE_SYSTEM.md b/website/MODULE_SYSTEM.md new file mode 100644 index 0000000..cc3bf85 --- /dev/null +++ b/website/MODULE_SYSTEM.md @@ -0,0 +1,464 @@ +# The Module System — design of record + +**Status:** approved design, not yet implemented. Every decision in Part 3 has been settled with the +org lead; Part 1 records what was verified against the working trees on 2026-08-10, including the +places the original draft was wrong. + +**Goal.** Turn Runic Gateway from a UO/ServUO-specific platform into a game-agnostic one. The core +architecture is unchanged — sidecar → website → browser. What changes is that game-specific +behaviour (routes, tables, screens, nav) leaves the core website and becomes an installable +**module**. An operator installs the base site, installs the module for their game, and restarts. + +**The model is WordPress plugins, not a build system.** An operator never compiles anything to +deploy a module. The module's own CI publishes it prebuilt; the operator drops it in and enables it. +This single constraint drives most of Part 2. + +**Out of scope.** The sidecar's per-game protocol adapters. `link/`, `servuo-plugins/` and +`installer/` are the shard side and stay independent of this work — the installer runs on the shard +host and by design never contacts the website (`installer/src/cli.rs:162`). Also out of scope: the +Android app, which gets its own plan covering module discovery and multi-server profiles; this plan +only owes it the capability endpoint in §2.5. + +--- + +## Part 1 — What is actually there + +### 1.1 The parts that are already clean + +The extraction is closer to a folder move than a teardown, and that is not an assumption: + +- **Models.** `server/src/model/` holds 31 directories; exactly 8 are UO — `shardAtlas/`, + `shardClilocs/`, `shardEvents/`, `shardLinks/`, `shardMarket/`, `shardState/`, `shardVisibility/`, + `uoLinkConfig/`. No mixing with `users/`, `posts/`, `pages/`, `wiki/`, `settings/`. +- **Routers.** All 13 UO router/controller files are single-purpose, with no shared code: + `admin/shard.router.js`, `admin/shardAtlas|shardClilocs|shardOps|shardVisibility.controller.js`, + `admin/uoLink.router.js` + `.controller.js`, `public/atlas.router.js` + `.controller.js`, + `public/shard.router.js` + `.controller.js`, `player/shard.router.js` + `.controller.js`. +- **Mount points.** `router/v1/{public,admin,player}/index.js` are pure mount tables that declare no + routes of their own. Module mounting drops straight in with no restructuring. +- **The API surface is small and knowable.** The nine UO `utils/` files import only four things from + core: `settings.model`, `logger`, `auth`, `pushDispatch`. That is the empirical basis for §2.1 — + the contract is derived from what the real code uses, not designed speculatively. + +### 1.2 Route prefixes: one flat module prefix is impossible + +A module cannot be handed a single pre-scoped router at, say, `/api/v1/game/uo`, because the existing +UO URLs live under three different access tiers — `/api/v1/public/shard/*`, +`/api/v1/admin/shard/*`, `/api/v1/player/shard/*` — and those URLs are protected by +`routeManifest.test.js` and consumed by three shipped clients (SPA, Android app, Discord bot). + +**Resolved:** a module owns a *named slot inside each tier*. It still only ever holds a pre-scoped +`express.Router()` and structurally cannot reach above its mount point; it simply holds one per tier. + +```json +"mounts": { + "public": ["/shard", "/atlas"], + "admin": ["/shard", "/uo-link"], + "player": ["/shard"] +} +``` + +The loader rejects a prefix collision between two modules, or between a module and core, at +registration time. That check is unavoidable; the per-request routing boundary is not left to the +module's good behaviour. + +### 1.3 There is no server-side nav list, and there must not be one + +`server/src/utils/navOverrides.js` (lines 13–21) refuses this explicitly: + +> **What this module cannot check, deliberately: whether a `to` exists.** The three base NAV arrays +> are client constants (SiteHeader.jsx, AdminLayout.jsx, PlayerPortalLayout.jsx). Shipping a copy of +> them to the server would create a second source of truth for navigation that drifts the first time +> a route is added… + +Confirmed: `export const NAV` lives in `client/src/components/SiteHeader.jsx:23`, +`client/src/routes/admin/AdminLayout.jsx:56`, `client/src/routes/player/PlayerPortalLayout.jsx:41`. +The server validates override *shape* and nothing else. + +**Resolved:** nav registration is **client-side**, performed by the module's own client bundle +against a core-provided registry. No server nav API is introduced, and +`THEMING_AND_NAV.md`'s override model is untouched. The resulting pipeline is: + +> registered defaults (core + modules) → role/feature filtering → admin overrides → rendered nav + +### 1.4 Module nav items interleave into *core* groups + +Appending a "UO" group is not enough. Today's UO items sit inside core groups in `AdminLayout.jsx`: +group **Moderation** holds `/admin/shard-ops` and `/admin/houses`; group **System** holds +`/admin/shard`, `/admin/shard-visibility`, `/admin/shard-atlas`; the unnamed footer group holds +`/admin/characters`. `MOD_PATHS` (line 109) additionally hardcodes two UO paths as +moderator-visible. + +**Resolved:** nav registration takes a target group and order (`{ group: 'Moderation', order: 30 }`), +and `MOD_PATHS` becomes a `roles`-derived computation rather than a path allowlist. + +### 1.5 The public nav's feature-gating mechanism is itself a shard system + +Ten of the sixteen entries in `SiteHeader.jsx`'s NAV carry a `feature:` key (`status`, `champs`, +`guilds`, `governors`, `houses`, `ruleset`, `atlas`, `leaderboards`, `market`), resolved by +`useShardFeatures()` against `/api/v1/public/shard/features` — the shard visibility system. +Extracting the module removes the provider that core's own nav filter depends on. + +**Resolved:** core keeps a generic feature-flag context with a **registerable provider**; the module +registers its `useShardFeatures` for its own namespace. No core nav item carries a `feature` today, +so with no module installed the filter is a correct no-op. + +### 1.6 There is no migration system to model a module migration runner on + +`server/db/schema.sql` is a single idempotent file — 1,380 lines, 67 tables — replayed in full on +every boot by `ensureSchema()` (`src/utils/db.js:48`), split on `;` and executed statement by +statement. Schema evolution uses `ALTER TABLE … ADD COLUMN IF NOT EXISTS` / `MODIFY COLUMN` +(from line 1322). There is **no version table, no runner, no migrations directory**. + +A module-scoped migration runner would therefore be the *first* migration system in the codebase, +and would leave core and modules on two different schema models. + +**Resolved:** modules ship a `schema.sql` fragment, replayed idempotently by the same +`ensureSchema()` immediately after core's. Forward-only falls out for free — it is all an idempotent +replay can be. Install and upgrade become the same operation. Uninstall stops the fragment being +replayed; **purge** is a separate, explicit, destructive admin action that runs the module's +`purge.sql`. A real migration runner, covering core *and* modules together, is a legitimate future +workstream; it is not a prerequisite for this one. + +Twenty-five of the 67 tables move with the module: the 24 `shard_*` tables plus `uo_link_config`. + +### 1.7 Boot and shutdown is a lifecycle gap + +`server/src/server.js` holds eight UO call sites that routes, nav and schema do not cover: + +| Line | Call | +| --- | --- | +| 92 | `shardAtlas.refreshOnBoot()` | +| 99 | `shardClilocs.refreshOnBoot()` | +| 107 | `shardMarket.refreshDisplayNames()` (conditional on the cliloc import result) | +| 130 | `uoLinkSocket.start()` | +| 131 | `checkUoLink()` — plus the whole function at 147–169 | +| 179 | `uoLinkSocket.stop()` | +| 180 | `shardBroadcast.closeAll()` | +| 7–19 | five top-level `require`s of UO modules | + +**Resolved:** the API surface includes `onBoot(ctx)` and `onShutdown()`, each individually +try/caught by the loader per §2.4. + +### 1.8 Three core files are genuinely entangled + +Everything else is a folder move. These are not: + +1. **`src/config/notificationStreams.js`** — the push stream catalog. `mapShardEvent()` and most of + `STREAMS` are shard-derived, and it imports `PUBLIC_KINDS` from `utils/shardBroadcast`. Push + *infrastructure* is core; this *catalog* is module content. + → `registerNotificationStreams({ streams, mapEvent })`. +2. **`src/utils/pushDispatch.js`** — core infrastructure, but `fromShardEvent()` (line 112) requires + the `shardLinks` model (line 21) and `mapShardEvent` (line 23). + → invert: `publish()` stays core, `fromShardEvent` moves into the module and calls it. +3. **`src/utils/announceWorker.js`** — the news dispatcher, with two delivery legs: Discord (core) + and town crier (module, via `uoLinkClient.postTownCrier`, line 36). + → `registerAnnounceLeg({ leg, dispatch, classify })`. + +`src/utils/newsGump.js` is module-side (news → in-game gump) and moves whole. + +### 1.9 A fourth mount shape: module routes under a core resource + +`router/v1/admin/users.router.js` mounts `usersShard.controller.js` at six UO sub-paths of a **core** +resource — `/:id/shard/accounts|sales|houses|online|standing` and `DELETE /:id/shard/link/:account`. +And `GET /api/v1/admin/users/:id` (line 159) is itself served by `usersShard.getUser`, which is core +semantics that ended up in the UO controller by proximity. + +**Resolved, two parts:** (a) `getUser` moves back into `admin.controller.js`; (b) core declares a +narrow **extension slot** on `/admin/users/:id` that the module mounts into, so core never learns +what "shard" means and all six URLs are preserved. Only core may declare an extension slot; a module +may not invent one. + +### 1.10 The Discord bot has no UO logic + +The draft listed the bot's "UO-specific event/moderation logic" as an extraction candidate. Grepping +`website/bot/src` for `uo|ultima|shard|towncrier|governor|vendor` returns **zero matches**. The bot's +only site coupling is `src/site/siteApiClient.js`. There is nothing to extract. + +### 1.11 The installer is not, and will not become, the delivery path + +Two independent reasons, and the decision is that website and installer stay independent: + +1. **The installer runs on the shard host and never contacts the website** — `src/cli.rs:162`: *"The + installer never contacts your website, never deletes anything from your…"*. A website module is a + website-host artifact. +2. **`Bundle` is hardcoded to exactly two components.** `installer/src/bundle.rs` declares + `pub link: LinkComponent` and `pub overlay: OverlayComponent`, both non-`Option`, alongside a + single top-level `protocol: u32` and `SUPPORTED_SCHEMA: u32 = 1`. A third artifact type would be + a schema-2 bump — and the sidecar/overlay protocol number has nothing to say about a website + module anyway. + +Note also that **there is no SHA256SUMS trust anchor** anywhere in the installer, contrary to the +draft. The real model is a per-asset `sha256` field inside a bundle JSON fetched anonymously over +HTTPS from the `bundles` branch. No signatures. The *shape* is worth reusing; the name was wrong. + +### 1.12 Modules must mount synchronously, from the filesystem + +`server/scripts/routeManifest.js:38` and `server/swagger/swagger.js:29` both walk the Express stack by +`require`-ing `src/app.js` **with no database connection** — the manifest script deliberately points +the pool at a dead port. A DB-driven async loader would make module routes invisible to both, +silently breaking the frozen-URL-surface test and shipping undocumented routes. + +**Resolved:** the **filesystem is the mounting source of truth.** `app.js` synchronously scans +`modules/*/module.json` at require time and mounts what it finds. The `installed_modules` row carries +state and metadata (version, installed-at, `startup_failed` reason, admin enable/disable) and is +reconciled against the filesystem once the DB is up. A module disabled in the DB is skipped by a +one-line dispatch guard rather than being unmounted, so the URL surface stays deterministic and +generatable. + +### 1.13 The client seam is a route registry, not an admin-panel loader + +`client/src/App.jsx` is a flat 235-line static route table, and UO routes appear in all three areas — +public (`/site/shard`, `/site/shard/activity`, `/site/governors`, `/site/houses`, `/site/atlas`, +`/site/atlas/:slug`, `/site/market`, `/site/market/vendors/:serial`, plus champs, guilds, rules, +leaderboards), admin (`shard`, `shard-visibility`, `shard-atlas`, `shard-ops`, `houses`, +`characters`, `characters/:serial`) and player. Nav is one consumer of that registry, not the +mechanism itself. + +### 1.14 Production is a prebuilt, pull-only image — and that is the binding constraint + +`website/Dockerfile` bakes `client/dist` at image build time, and `docker-compose.yml` has no +`build:` stanza at all (deliberately: *"a production host can only ever pull, never accidentally +build"*). Combined with the requirement that **an operator must never build anything to deploy a +module**, this rules out build-time inclusion of module client code, which the draft had as its +default. + +It also rules out import maps as the shared-dependency mechanism: `config/csp.js:49` sets +`'script-src': ["'self'"]` with no `'unsafe-inline'`, and an import map must be an inline +`