RunicGateway/docs
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request 'docs(website): the module API contract, validated by the atlas spike (phase 1)' (#124) from docs/module-api into main

Browse Source 
Reviewed-on: #124
Reviewed-by: Colby Whitlock <whitlocktech@gmail.com>
This commit is contained in:
Colby Whitlock
2026-08-10 11:16:46 +00:00
parent 07d8390a17 037f4abad2
commit 27bfdc9152
3 changed files with 725 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@@ -24,6 +24,8 @@ sidecar as a service, and hands you the values the website needs.
| [BACKEND_DESIGN.md](website/BACKEND_DESIGN.md) | API contract, DB schema, security model | | [BACKEND_DESIGN.md](website/BACKEND_DESIGN.md) | API contract, DB schema, security model |
| [HERO_EDITOR.md](website/HERO_EDITOR.md) | Hero canvas editor feature spec | | [HERO_EDITOR.md](website/HERO_EDITOR.md) | Hero canvas editor feature spec |
| [THEMING_AND_NAV.md](website/THEMING_AND_NAV.md) | Admin-configurable theme, brand assets and navigation — build contract | | [THEMING_AND_NAV.md](website/THEMING_AND_NAV.md) | Admin-configurable theme, brand assets and navigation — build contract |
| [MODULE_SYSTEM.md](website/MODULE_SYSTEM.md) | Making the site game-agnostic: game logic becomes an installable module — design of record |
| [MODULE_API.md](website/MODULE_API.md) | The module ↔ core contract: `ctx`, the `register*` calls, the client registry and the loader's obligations |
| [WIKI_UPGRADE.md](website/WIKI_UPGRADE.md) | Wiki subsystem upgrade notes | | [WIKI_UPGRADE.md](website/WIKI_UPGRADE.md) | Wiki subsystem upgrade notes |
| [SHARD_VISIBILITY.md](website/SHARD_VISIBILITY.md) | Who sees which shard data — the admin-configurable audience framework | | [SHARD_VISIBILITY.md](website/SHARD_VISIBILITY.md) | Who sees which shard data — the admin-configurable audience framework |
| [SPAWN_ATLAS.md](website/SPAWN_ATLAS.md) | The bestiary / spawn atlas: what the shard contains, parsed from its own ServUO tree | | [SPAWN_ATLAS.md](website/SPAWN_ATLAS.md) | The bestiary / spawn atlas: what the shard contains, parsed from its own ServUO tree |

699
website/MODULE_API.md Normal file
View File

@@ -0,0 +1,699 @@
# The Module API — the contract
**Status:** Phase 1 deliverable of [MODULE_SYSTEM.md](MODULE_SYSTEM.md), **validated by the atlas
spike** — Part 7 records what the spike proved, what it changed in this contract, and the three
artifacts in it that are not design. This document is the
normative contract between the core website and an installed module. `MODULE_SYSTEM.md` decides
*what* the module system is; this decides *exactly what a module may call, what it must provide, and
what core promises not to break*.
Everything below is derived from what the UO code actually does today, re-read against the working
tree on 2026-08-10. Where the survey contradicted `MODULE_SYSTEM.md`, the contradiction is recorded
in Part 6 rather than quietly resolved — four of them, one of which (OpenAPI, §6.1) needs a decision
before Phase 2 starts.
**The one rule everything else serves:** a module reaches core *only* through the members named in
this document. Zero `require`/`import` from a module to a core file, enforced in CI (§5.1). A core
refactor that leaves this contract intact cannot break a module; anything a module needs that is not
here extends the contract first, in this file, before the module is written against it.
---
## Part 1 — Versioning
### 1.1 `MODULE_API_VERSION`
Core exports a single integer-major semver string from `server/src/modules/version.js`:
```js
const MODULE_API_VERSION = '1.0.0'
```
Every `module.json` declares a `coreApi` semver **range**. The loader checks it at boot, before it
requires a line of module code, and a mismatch fails that module loudly into `startup_failed`
(§4.4) with the two versions in the reason. It never silently proceeds.
| Change | Bump |
| --- | --- |
| A member is added to `ctx`, or a new `register*` call appears | minor |
| A member is removed or its signature changes | major |
| Behaviour of an existing member changes without a signature change | major |
| A core-internal refactor behind an unchanged member | none |
This is a **separate number from `PROTOCOL_VERSION`**, which versions the shard wire and has nothing
to say about a website module. It is also separate from the module's own version.
### 1.2 What is *not* contract
Core's internal file layout, table names, middleware ordering, the `api` client object's shape, and
every component under `client/src/components/` except the ones named in §3.4. A module that reaches
any of these is out of contract even if it happens to work.
---
## Part 2 — The server contract
### 2.1 `module.json`
Read synchronously by the loader from `modules/<id>/module.json`. Unknown top-level keys are
rejected rather than ignored, so a typo is a loud failure and not a silently-inert setting.
```json
{
"id": "uo",
"name": "Ultima Online",
"version": "1.0.0",
"coreApi": "^1.0.0",
"server": "server/index.js",
"client": { "entry": "client/dist/entry.js" },
"schema": "server/db/schema.sql",
"purge": "server/db/purge.sql",
"mounts": {
"public": ["/shard", "/atlas"],
"admin": ["/shard", "/uo-link"],
"player": ["/shard"]
},
"extensions": ["admin.users.detail"],
"capabilities": ["shard", "atlas", "market"]
}
```
| Key | Required | Meaning |
| --- | --- | --- |
| `id` | yes | `^[a-z][a-z0-9-]{1,31}$`. The directory name, the `installed_modules` key, the URL segment, the `window.__rg` registry key. Must equal the directory it was read from. |
| `name` | yes | Human label for the admin Modules screen. |
| `version` | yes | Semver. Recorded in `installed_modules`; shown on failure. |
| `coreApi` | yes | Semver range checked against `MODULE_API_VERSION` (§1.1). |
| `server` | no | Entry point, relative to the module root. Absent ⇒ client-only module. |
| `client.entry` | no | Prebuilt ESM chunk, relative to the module root. Absent ⇒ server-only module. |
| `schema` | no | Idempotent SQL fragment (§2.6). |
| `purge` | no | Destructive teardown (§2.6). Required if `schema` is present. |
| `mounts` | no | Declared prefixes per tier (§2.3). Declaration is the contract; the loader compares it against what the module actually registers and rejects a mismatch. |
| `extensions` | no | Core extension slots this module mounts into (§2.4). |
| `capabilities` | no | Opaque strings published by `GET /api/v1/public/modules`, for clients (the SPA, the Android app) to feature-detect against. |
### 2.2 The entry point
`server/index.js` exports a single function. It is called once, synchronously, during `app.js`
require — **not** after the database is up.
```js
module.exports = function register(ctx, api) { /* … */ }
```
It must not `await`, must not touch the database, and must not throw for a reason that a retry would
fix. Everything that needs a live database belongs in `onBoot` (§2.5). This constraint is not
stylistic: `scripts/routeManifest.js` and `swagger/swagger.js` both require `app.js` with the pool
pointed at a dead port, and a module that queried at registration time would hang both.
### 2.3 `ctx` — what core hands the module
Every member below exists because a UO file uses it today. Nothing is speculative, and nothing that
module-uo does not need is on the list.
| Member | Signature | Backed by | First real caller |
| --- | --- | --- | --- |
| `ctx.express` | the `express` namespace | core's `node_modules` | every module router (§7.2) |
| `ctx.validator` | the `express-validator` namespace | core's `node_modules` | `atlas.router.js` |
| `ctx.db.query` | `(sql, params?) => Promise<rows>` | `utils/db` | every `*.db.js` |
| `ctx.db.pool` | mariadb pool | `utils/db` | `shardAtlas.db.js` (streamed import) |
| `ctx.log` | `(namespace) => { error, warn, info, debug }`, each `(msg, meta?)` | `utils/logger` | all nine UO utils |
| `ctx.settings.get` | `(key) => Promise<string\|null>` | `model/settings` | `shardAtlas.model` |
| `ctx.settings.set` | `(key, value, updatedBy?) => Promise<void>` | `model/settings` | `shardAtlas.model:60` |
| `ctx.settings.getInstanceName` | `() => Promise<string>` | `model/settings` | `shardIngest.js:84` |
| `ctx.auth.getUserFromRequest` | `(req) => { id, username, role } \| null` | `utils/auth` | `shardVisibility.js` |
| `ctx.push.publish` | `(streamId, { ref?, ownerUserId? }) => Promise<void>` | `utils/pushDispatch:92` | `shardIngest.js:22` |
| `ctx.secretBox` | `{ encrypt(s), decrypt(s) }` | `utils/secretBox` | `uoLinkConfig.model` |
| `ctx.middleware` | `{ requireAuth, requireRole, siteMode, validate, noindex }` | `auth/session.middleware`, `middleware/*` | every UO router |
| `ctx.uploads` | `{ upload, UPLOAD_DIR, MIME_EXT }` | `admin/imageUpload.js` | atlas art import |
| `ctx.posts` | `{ listAll, getById, linkAnnounceJob, markAnnounced }` | `model/posts` | `newsGump.js:108`, `announceWorker.js:58` |
| `ctx.paths.moduleRoot` | absolute path to `modules/<id>/` | loader | atlas art, cliloc files |
| `ctx.moduleId` | the id from `module.json` | loader | log tags, table checks |
Three narrowings from `MODULE_SYSTEM.md` §2.1, all deliberate:
- **`ctx.auth` is one function, not `utils/auth`.** The facade also re-exports `signToken`,
`setAuthCookie` and the TOTP challenge primitives. Minting sessions is core's job; a module that
needs an identity needs to *read* one.
- **`ctx.settings` is three functions, not the model.** The model exports 24 names, most of them
registration/game-signup/app-links policy that is core's business.
- **`ctx.posts` is four functions.** `create`/`update`/`remove` are the CMS, not a module's.
And one addition the spike forced: **`ctx.express` and `ctx.validator`**. A module lives at
`<repo>/modules/<id>/`, outside `server/`, so Node's resolver never reaches `server/node_modules` and
`require('express')` from a module simply fails — which is how this was found. Even where it
resolved, a second express in the process is a second `Router` prototype. Core owns one express, as
it owns one React (§7.2).
`ctx` is frozen (`Object.freeze`, one level deep) before it is handed over. That is a guard against
accident, not against a hostile module — per `MODULE_SYSTEM.md` §2.2 the boundary is organisational,
not a security boundary.
### 2.4 `api` — what the module registers
The second argument. Every call is synchronous, idempotent-free (calling twice is an error), and
validated at once rather than at first use.
```js
api.registerRoutes({ public: {...}, admin: {...}, player: {...} })
api.registerExtension(slot, router)
api.registerNotificationStreams({ streams, mapEvent })
api.registerAnnounceLeg({ leg, dispatch, classify })
api.onBoot(async (ctx) => {})
api.onShutdown(async () => {})
```
**`registerRoutes(mounts)`** — one `express.Router()` per prefix per tier:
```js
api.registerRoutes({
public: { '/shard': shardRouter, '/atlas': atlasRouter },
admin: { '/shard': adminShardRouter, '/uo-link': uoLinkRouter },
player: { '/shard': playerShardRouter },
})
```
The keys must match `module.json`'s `mounts` exactly. Prefixes are validated `^/[a-z0-9][a-z0-9-]*$`
— one segment, no nesting, no parameters — and rejected on collision with core's own mount table or
with another module's, at registration time. The router is mounted *inside* the tier, so it
structurally cannot reach above its prefix.
**The tier gate is already applied.** A router registered under `admin` sits behind
`noindex, isLoggedIn, requireRole('admin','editor','moderator')` from `router/v1/admin/index.js`;
under `player`, behind `noindex, requireAuth`; under `public`, behind nothing, by design. A module
adds per-route gates on top of that and never re-implements the tier gate.
**`registerExtension(slot, router)`** — the §1.9 case: module routes hanging off a *core* resource.
Only core may declare a slot; a module may only fill one. Exactly one slot exists in v1:
| Slot | Mounted at | Declared by |
| --- | --- | --- |
| `admin.users.detail` | `/api/v1/admin/users/:id` | `router/v1/admin/users.router.js` |
The router receives `req.params.id` from the parent (`mergeParams: true`). Two modules filling the
same slot is a collision and is rejected; core's own routes on the resource always win a path
conflict.
**`registerNotificationStreams({ streams, mapEvent })`** — §1.8's push catalog.
`streams` is an array of `{ id, label, description, scope }` appended to core's catalog (ids are
namespaced `<moduleId>.<name>` and rejected otherwise); `mapEvent(event) => streamId | null` is
called by core's dispatcher for events the module's own code publishes.
**`registerAnnounceLeg({ leg, dispatch, classify })`** — §1.8's news dispatcher.
`leg` is a namespaced id, `dispatch(post) => Promise<void>` delivers, `classify(post) => boolean`
decides whether this leg wants the post. A leg that throws is retried by core's existing per-leg
retry and never blocks another leg.
**`onBoot(fn)` / `onShutdown(fn)`** — §2.5.
### 2.5 Lifecycle
```
require(module) → register(ctx, api) → [routes mounted, app.js require returns]
↓ (server.js, after ensureSchema + seed)
onBoot(ctx) → started
↓ (SIGINT/SIGTERM)
onShutdown()
```
`onBoot` is where the eight `server.js` UO call sites go (`MODULE_SYSTEM.md` §1.7): the atlas and
cliloc refreshes, the market display-name backfill, `uoLinkSocket.start()`, the sidecar probe.
It runs **after** `ensureSchema()` (so the module's own tables exist) and after `seedDefaults()`,
and **before** the HTTP listener binds — a module that must not serve traffic before it has warmed
its cache gets that for free.
`onShutdown` runs before the server closes, in reverse registration order, with a 5-second budget
per module; exceeding it is logged and skipped rather than hanging the process.
Both are individually try/caught. An `onBoot` that throws marks that module `startup_failed`
(§4.4) and the site still comes up — its routes stay mounted but its dispatch guard rejects them
with 503, because a module that failed to warm up serving half-initialised data is worse than a
module that says it is down.
### 2.6 Schema fragments
`schema` is an idempotent `.sql` file replayed by the same `ensureSchema()` that replays core's,
immediately after it, statement by statement, split the same way. It is subject to the same rules
core's file already follows: `CREATE TABLE IF NOT EXISTS`, `ALTER TABLE … ADD COLUMN IF NOT EXISTS`,
no `--` inside a string literal, no `DROP`.
**Table names are namespaced and collision-checked.** New tables must be prefixed `<id>_`. The
loader extracts every `CREATE TABLE IF NOT EXISTS <name>` from the fragment and rejects the module
if a name collides with a core table or with another module's — a wrong `DROP`-free fragment can
still silently adopt someone else's table otherwise.
**module-uo is grandfathered.** Its 27 tables are named `shard_*` (26) and `uo_link_config` (1), and
renaming them is a data migration this workstream explicitly does not do (`MODULE_SYSTEM.md` §1.6
puts the count at 25; the working tree says 27 — see §6.4). They are registered in the loader as an
explicit legacy allowlist keyed to `id: "uo"`, so the prefix rule holds for every module written
after this one.
`purge` is the destructive counterpart, run **only** by the explicit admin purge action, never by
uninstall. Required whenever `schema` is present: a module that can create tables and cannot drop
them leaves an operator with orphaned data and no supported way to remove it.
### 2.7 What a module must not do
- `require` anything outside its own directory except node built-ins and its own `dependencies`.
- Mutate `ctx`, `req.user`, or any object core handed it.
- Register an Express error handler, or any middleware at the app level.
- Read `process.env` for core configuration. Its own config is a `settings` key or its own table.
- Call `process.exit`, install signal handlers, or start a listener.
- Write outside `ctx.paths.moduleRoot` and the upload directory.
### 2.8 The OpenAPI fragment
Every module that registers routes ships `swagger-fragment.json` in its bundle root. Core merges the
fragments of started modules into `/api/docs.json`; the full reasoning and the collision rules are
§6.1a. In short: fully-qualified paths, namespaced schema keys, module CI fails if a registered
route has no path in the fragment, and core wins every key collision.
---
## Part 3 — The client contract
### 3.1 How the chunk gets there
Exactly as `MODULE_SYSTEM.md` §2.6 resolved, and Phase 1's spike is what proves it:
1. Module CI builds `client/dist/entry.js` with Vite in **library mode**, `react`, `react-dom`,
`react-dom/client` and `react-router-dom` declared **external**.
2. Core serves the module directory statically at `/modules/<id>/` — same-origin, so
`script-src 'self'` (`config/csp.js:49`) admits it with no nonce and no inline.
3. `utils/htmlShell.js` injects `<script type="module" src="/modules/<id>/entry.js">` at the
`</head>` rewrite it already performs (line 111), for each **started** module.
4. Before that tag, core has published `window.__rg` (§3.2) from its own bundle. The module's
externals resolve against it.
There is exactly one React instance and core owns it. A module that bundles its own React will
produce two copies of the hook dispatcher and fail at the first `useState`; the externals config in
§3.5 is what prevents it.
### 3.2 `window.__rg`
Populated by core's `main.jsx` **before** it renders, and frozen afterwards.
```js
window.__rg = {
version: '1.0.0', // MODULE_API_VERSION — the same number as the server's
react, // the React namespace
reactDom, // react-dom/client
router, // react-router-dom namespace
jsxRuntime, // react/jsx-runtime — see below
registry, // §3.3
ui, // §3.4
api, // §3.5
}
```
`jsxRuntime` is not decoration. A module's bundler compiles every `.jsx` file to imports from
`react/jsx-runtime` under the modern automatic runtime, and those have to resolve to *core's* React
like every other import. Without it on the global a module would have to build with
`jsxRuntime: 'classic'`; with it, the module uses the default its tooling already assumes.
A module entry checks `window.__rg.version` against its own `coreApi` range and refuses to register
on a mismatch, logging once — the client-side twin of §1.1, and the reason `version` is here at all.
### 3.3 `registry` — what the module registers
```js
registry.registerRoutes(id, { public: [...], admin: [...], player: [...] })
registry.registerNav(id, { area, items })
registry.registerFeatureProvider(id, namespace, hook)
```
**`registerRoutes`** — arrays of `{ path, element, gate? }`. Paths are relative to the module's
namespace and core prefixes them (`MODULE_SYSTEM.md` §2.8):
| Area | Rendered at | Wrapped in |
| --- | --- | --- |
| `public` | `/<id>/<path>` | `MaintenanceGate` |
| `admin` | `/admin/<id>/<path>` | `RequireAuth` + `AdminLayout` |
| `player` | `/player/<id>/<path>` | `RequirePlayer` + `PlayerPortalLayout` |
`gate` is an optional `{ roles: [...] }`, applied by core as the existing `RoleGate`. A module cannot
supply its own auth wrapper — that is the one place where the client boundary is load-bearing, since
the sidebar and the route table must agree about who may see what.
`App.jsx` stops being a flat static table and becomes core's routes plus
`registry.routesFor(area)`. Registration happens at entry-script evaluation, which is before
`createRoot().render()`, so nothing renders against a half-populated registry.
**`registerNav`** — items interleave into *core* groups (`MODULE_SYSTEM.md` §1.4):
```js
registry.registerNav('uo', {
area: 'admin',
items: [
{ label: 'Shard ops', to: '/admin/uo/shard-ops', group: 'Moderation', order: 30,
roles: ['admin', 'moderator'] },
{ label: 'Shard', to: '/admin/uo/link', group: 'System', order: 10 },
],
})
```
`group` names an existing core group; an unknown group name appends a new group at the end rather
than dropping the item. `order` sorts within the group, core items keeping their current positions.
`feature` (public area only) names a flag resolved by §3.3's provider.
`MOD_PATHS` in `AdminLayout.jsx:109` — today a hardcoded allowlist of two UO paths — becomes a
computation over each item's `roles`, so moderator visibility follows from the registration instead
of from a second list that has to be kept in sync.
The pipeline is unchanged from `THEMING_AND_NAV.md`, with one new first step:
> registered defaults (core **+ modules**) → role/feature filtering → admin overrides → rendered nav
**`registerFeatureProvider`** — core keeps a generic flag context; the module supplies the hook that
fills its namespace (`useShardFeatures` for `uo`). With no module installed the filter is a correct
no-op, because no core nav item carries a `feature` today.
### 3.4 `ui` — the shared component kit
**This is the largest addition Phase 1 makes to the plan, and it is not optional** (§6.2; approved
2026-08-10). The atlas
pages alone import five core modules that are not React and not the router: `PublicLayout`,
`PageHeader`, `Loading` / `ErrorState` / `EmptyState`, and `useAsync`. Without a shared kit a module
either reaches into core's tree (violating the zero-import rule) or ships its own copies, which
means a module page that does not look like the site it is installed in — and drifts further every
time core's layout changes.
The kit is **curated and closed**, not a re-export of `components/`:
| Export | From | Why it is in the kit |
| --- | --- | --- |
| `PublicLayout` | `components/PublicLayout.jsx` | the public chrome; a module page without it is a bare page |
| `AdminPage` | `routes/admin/…` | the admin content frame |
| `PageHeader` | `components/PageHeader.jsx` | title/subtitle furniture |
| `Loading`, `ErrorState`, `EmptyState` | `components/PageState.jsx` | the three states every data page has |
| `useAsync` | `lib/useAsync.js` | the fetch/loading/error hook every data page uses |
| `useAuth`, `useSite` | `contexts/*` | read-only access to session and site settings |
Everything else — tables, chips, tabs, the tiptap editor, dnd-kit — a module bundles itself.
Adding to the kit is a **minor** `MODULE_API_VERSION` bump; changing a kit component's props is a
**major** one. That is a real constraint on core and it is the price of the boundary being worth
anything.
### 3.5 `api` — the request primitive
`client/src/api/client.js` is one 518-line object, and it already carries module namespaces:
`api.atlas` (line 196) and `api.shard` are UO bindings living in core's client. They move out with
the module.
Core exposes the primitive, not the object:
```js
window.__rg.api = { request, ApiError, BASE } // request(path, { method, body, headers, raw })
```
`request` is `client.js`'s existing `req` — same-origin `/api/v1`, `credentials: 'include'`, JSON
in/out, throwing `ApiError(status, message, body)`. A module builds its own namespace over it and
owns the paths it calls, which is correct: it owns the routes at the other end.
### 3.6 Vite library-mode build
The module's `vite.config.js`, and the four externals are the whole contract:
```js
export default defineConfig({
plugins: [react()],
build: {
lib: { entry: 'src/entry.jsx', formats: ['es'], fileName: () => 'entry.js' },
outDir: 'dist',
modulePreload: { polyfill: false }, // same reason as core: no inline bootstrap under CSP
rollupOptions: {
external: ['react', 'react-dom', 'react-dom/client', 'react-router-dom'],
output: { paths: { /* rewritten to window.__rg by the shim below */ } },
},
},
})
```
Rollup's `external` alone emits bare `import 'react'` specifiers, which the browser cannot resolve
without an import map — and CSP forbids the inline `<script type="importmap">` that would provide
one (`MODULE_SYSTEM.md` §1.14). The module therefore ships a two-line shim module that re-exports
from the global, and aliases the four externals to it:
```js
// src/shim/react.js
export default window.__rg.react
export const { useState, useEffect, useMemo, useCallback, useRef, createElement, Fragment } = window.__rg.react
```
**This is the highest-risk mechanical detail in the whole plan and it is exactly what the Phase 1
spike exists to prove.** If it does not hold, §2.6 of the design of record is wrong and the client
half needs rethinking before Phase 2 builds on it.
---
## Part 4 — The loader's obligations
### 4.1 Synchronous, filesystem-sourced
`app.js` scans `modules/*/module.json` with `fs.readdirSync` at require time and mounts what it
finds (`MODULE_SYSTEM.md` §1.12). The database is not consulted. `MODULES_DIR` defaults to
`<repo>/modules` and is overridable by env for tests and for the Docker volume mount.
### 4.2 Order
Alphabetical by `id`, deterministically. There is no dependency resolution between modules (§2.0 of
the design of record puts it out of scope) and alphabetical order is the honest way to say so — any
other order would imply a precedence that is not being computed.
### 4.3 Validation, in this order
1. `module.json` parses; no unknown keys; `id` matches the directory.
2. `coreApi` satisfied by `MODULE_API_VERSION`.
3. Declared `mounts` prefixes are well-formed and collide with nothing.
4. Declared `extensions` slots all exist.
5. `schema`/`purge` files exist and are readable; table names are namespaced or allowlisted.
6. `require(server)` succeeds and exports a function.
7. `register(ctx, api)` returns without throwing, and registers exactly what `module.json` declared.
A failure at any step is that module's failure and nobody else's.
### 4.4 `startup_failed` is a state, not a crash
Per `MODULE_SYSTEM.md` §2.4, the loader try/catches the **entire** lifecycle — require, validation,
registration, schema replay, `onBoot` — and any failure marks the module `startup_failed` with the
reason recorded in `installed_modules`, visible in the admin panel, recoverable without shell
access. The site comes up.
Two sub-cases differ, and the difference matters:
| Failure before routes are mounted | Failure after (schema, `onBoot`) |
| --- | --- |
| routes and nav are simply absent | routes stay mounted; the dispatch guard returns **503** |
The second is what keeps the URL surface deterministic and generatable: `routes.manifest.json` must
not depend on whether a module's boot hook happened to succeed on the machine that generated it.
### 4.5 The disabled guard
A module disabled in `installed_modules` is *mounted and guarded*, never unmounted — a one-line
`if (!enabled) return res.status(404)` ahead of its tier mount. Same reason: the URL surface is a
property of the filesystem, not of a database row.
---
## Part 5 — Enforcement
### 5.1 Zero internal imports (CI, module repo)
The acceptance test for the whole contract. In the module's own CI:
```
grep -rE "require\(['\"]\.\./\.\./|from ['\"]\.\./\.\./\.\./" server/ client/src/
```
— refined to "no relative path that escapes the module root", plus a check that the only bare
specifiers in the client bundle are the four declared externals. A hit fails the build.
### 5.2 Zero UO identifiers in core (CI, website repo)
Phase 3's acceptance criterion 1: no `shard`, `uoLink`, `cliloc`, `atlas` or `towncrier` outside
`modules/`, as a CI grep test rather than a review promise.
### 5.3 Zero-line route manifest diff (CI, both repos)
`npm run routes:manifest -- --check` in core; the module generates and freezes its own manifest in
its own repo, using the same script pointed at a core+module app. Phase 2 must produce a zero-line
diff in core's; Phase 3 moves the UO entries out of core's and into module-uo's, which is the one
diff the whole workstream is allowed.
---
## Part 6 — Amendments to MODULE_SYSTEM.md
Four things the survey found that the design of record gets wrong or does not cover. The first
needs a decision.
### 6.1 OpenAPI generation does not survive a dynamic loader — **settled: fragment merge**
`MODULE_SYSTEM.md` §1.12 treats `scripts/routeManifest.js` and `swagger/swagger.js` as the same
problem, because both require `app.js` with no database. They are not the same problem.
- **`routeManifest.js` walks the live Express stack** (`app._router.stack`, line 175). It is runtime
introspection and a filesystem-scanning loader is invisible to it in the best way: whatever got
mounted, it sees.
- **`swagger/swagger.js` is static analysis.** `swagger-autogen` is handed `routes = ['./src/app.js']`
(line 29) and *parses the source text*, following `app.use(...)` to the required file. A
`require(path.join(dir, manifest.server))` inside a `for` loop is not statically resolvable. Module
routes will be **absent from `swagger-output.json`** — silently, with no error.
That collides directly with CLAUDE.md's standing rule: *never ship a route that isn't in the OpenAPI
spec*. Three ways out:
| Option | How | Cost |
| --- | --- | --- |
| **A. Fragment merge** ✅ | Module CI runs swagger-autogen against its own `server/index.js` and ships `swagger-fragment.json` in the bundle. Core deep-merges the fragments of started modules into `/api/docs.json` at request time. | one merge helper in core (~40 lines); the module owns its own spec, which matches "one repo, one bundle" |
| B. Glob the modules dir | Core's `swagger.js` adds `modules/*/server/index.js` to `routes` when present. | core's *committed* spec then depends on which modules the developer had checked out — a spec that differs per machine |
| C. Hand-write module paths into core's spec | — | a second source of truth; drifts on the first module release |
**Decided 2026-08-10: A.** It is the only one that keeps the spec correct on an operator's box,
where core is a prebuilt image and the module arrived afterwards. The obligation this puts on a
module is §2.8; the one it puts on core is Phase 2 item 2.
### 6.1a The obligation, on each side
**Module:** a `swagger-fragment.json` in the bundle root, generated by its own CI with the same
`swagger-autogen` tooling pointed at its own entry point, carrying only `paths`, `tags` and
`components.schemas`. Its paths must be fully qualified (`/api/v1/public/atlas/creatures`), because
the module knows its own mount prefixes and core does not re-derive them. Its `components.schemas`
keys are namespaced (`UoAtlasCreature`, not `AtlasCreature`) so two modules cannot collide in the
merged spec. CI fails the module build if a route it registers has no path in its fragment — the
per-module form of "never ship a route that isn't in the spec".
**Core:** `/api/docs.json` merges the fragments of **started** modules over its own committed spec at
request time (cached, invalidated on a module state change). Merge is shallow-per-section and
**core always wins a key collision** — a module cannot redefine a core path, tag or schema by
shipping one with the same name; the collision is logged and the module's version dropped.
`swagger-output.json` itself stays exactly what core's own routes generate, so `npm run swagger`
remains reproducible on any machine regardless of what is installed.
### 6.2 The client contract is much larger than §2.1 says
§2.1 lists three client registration calls and nothing else, implying React and the router are all a
module needs. The atlas pages disprove it: `Atlas.jsx` imports `PublicLayout`, `PageHeader`,
`PageState`'s three states, `useAsync` and `api` — five core modules beyond React, on the *smallest*
UO page. Hence §3.4's curated UI kit and §3.5's request primitive. This is an addition to the
contract, not a change of direction, but it makes core's component API a versioned surface, which it
has never been before.
### 6.3 `shardVisibility` is module-owned, and core's nav depends on it
`utils/shardVisibility.js` is a UO util that provides `requireFeature` and `project` — and the atlas
routes, the spike target, are gated by it (`atlas.router.js:25`). So the spike carries
`shardVisibility` plus the `shardVisibility` and `shardLinks` models with it, not just the atlas
files. Two consequences:
- On the **server** this is fine: `requireFeature` becomes module-internal middleware, and only
`PUBLIC_KINDS` crosses the boundary — already handled by §1.8's `registerNotificationStreams`
inversion.
- During the **spike** there will be two copies of `shardVisibility` in one process (the module's,
and core's for the not-yet-extracted shard routes) with two independent 5-second caches over the
same table. Functionally identical, and a spike-only artifact that Phase 3 resolves by moving the
original. Recorded so it is not mistaken for a design flaw.
### 6.4 Two counts in §1.6 and §2.7 are off
- **27 UO tables, not 25**: 26 `shard_*` plus `uo_link_config`. §1.6 says "the 24 `shard_*` tables
plus `uo_link_config`".
- **The atlas spike is 6 routes, not 5**: `/creatures`, `/creatures/:slug`, `/regions`,
`/landmarks`, `/champions`, `/meta`.
Neither changes a decision; both are corrected here rather than left to be tripped over when the
extraction is counted against the plan.
---
## Part 7 — What the spike proved
The Phase 1 spike ran on `website` branch `spike/module-atlas`, cut from `edge` and **deliberately
never merged** — it is the evidence, not the implementation. Phase 2 rebuilds the loader properly.
### 7.1 The exit criteria
| Criterion | Result |
| --- | --- |
| No internal-file imports from the module into core | **pass** — the module's only non-builtin requires are `ctx.express` / `ctx.validator`; the built client chunk contains **zero** bare import specifiers |
| `npm run routes:manifest` produces a zero-line diff | **pass** — `routes.manifest.json` *and* `routes.guards.json` are byte-identical with the six atlas routes now served by the module |
| The chunk loads under the enforced CSP | **pass** — `/uo/atlas` and `/uo/atlas/:slug` render from `/modules/uo/entry.js` under `script-src 'self'`, with **zero** violation reports at `/api/csp-report` and a clean console |
| Everything still passes | **pass** — 729 core tests, 81 module tests |
Also verified end to end against the real database: the schema fragment replayed after core's
(`schema ensured for module "uo"`), `onBoot` ran the atlas refresh, the module reached `started`, and
the six API URLs answered 200 unchanged at `/api/v1/public/atlas/*`.
### 7.2 One express, one React — the same rule, twice
The single biggest thing the spike changed. `MODULE_SYSTEM.md` §2.6 got the client half right — one
React, shared via a global — and said nothing about the server, where the identical problem exists
and bites harder:
- A module lives at `<repo>/modules/<id>/`. Node's resolver walks *up* from there and never reaches
`server/node_modules`, so `require('express')` inside a module **fails outright**. This was the
first error the spike hit.
- Installing express into the module would fix resolution and break something worse: two `Router`
prototypes, two sets of `instanceof` checks — and it would mean the operator running `npm install`
in a module directory, which is the build the whole plan exists to avoid.
Hence `ctx.express` and `ctx.validator`. The rule generalises: **anything shared between core and a
module is owned by core and handed over — never resolved by the module.** On the client that is
`react`, `react-dom/client`, `react-router-dom` and `react/jsx-runtime`; on the server it is
`express` and `express-validator`.
Two mechanical traps inside that, both cheap once known and both silent otherwise:
- **Vite's object-form `resolve.alias` does PREFIX matching.** A `react` key also rewrites
`react/jsx-runtime` into `src/shim/react.js/jsx-runtime`, a path that cannot exist. Use the array
form with anchored regexes (`/^react$/`).
- **`external` alone is not enough for an ESM library build.** Rollup then emits bare
`import 'react'`, which the browser cannot resolve without an import map — and CSP forbids the
inline `<script type="importmap">` that would supply one. Each shared dependency needs a two-line
alias shim that re-exports from `window.__rg`. `output.globals` does not help: it applies to
iife/umd output only.
### 7.3 §6.1 confirmed empirically, not just predicted
Regenerating the OpenAPI spec after the move deleted **361 lines** — all six atlas paths — from
`swagger-output.json`, with `Swagger-autogen: Success` and no warning of any kind. The route manifest
kept all six in the same run. That is the static-analysis-versus-runtime split of §6.1 happening for
real, and it is exactly the silent failure the fragment merge exists to prevent. Core's committed
spec is correct as regenerated — it describes core's own routes — and the six paths come back via
module-uo's fragment when Phase 2 item 2 lands.
### 7.4 The loader's failure guarantees are tested, not asserted
`server/test/moduleLoader.test.js` — 17 tests over the paths nobody exercises by hand: an entry point
that throws, a `coreApi` mismatch, an unknown manifest key, an id that disagrees with its directory,
two modules claiming one prefix, a module claiming a core prefix, registering an undeclared prefix
and declaring an unregistered one, a fragment naming a core table, an unprefixed table, a schema with
no purge, an `onBoot` that throws, an `onShutdown` that hangs and one that throws, a double
registration, and a probe asserting `ctx` exposes exactly the documented surface and is frozen.
The property under test throughout is the same: **the failing module fails alone.**
### 7.5 Spike artifacts that are NOT design
Three things in the branch are consequences of stopping at six routes, and Phase 3 removes all three.
They are recorded so nobody reads them as intended shape:
1. **Core reaches into the module twice** — `admin/shardAtlas.controller.js` and
`test/atlasController.test.js` require the module's model directly. The five admin atlas routes
live at `/admin/shard/atlas/*`, inside the `/shard` prefix core still owns, so the module cannot
take them without colliding or moving a URL. Phase 3 moves the whole `/shard` admin prefix at once
and the imports go with it.
2. **Two copies of `shardVisibility`** — the module's (as `utils/visibility.js`) and core's, for the
shard routes not yet extracted. Two five-second caches over the same table; functionally
identical. Predicted in §6.3, observed exactly as described.
3. **The module has no `swagger-fragment.json`** — §6.1a's obligation needs core's merge helper on
the other side of it, which is Phase 2.
### 7.6 A finding for Phase 2's loader
`scan()` is lazy — requiring the loader does not run it. That is deliberate (app.js decides when
modules are discovered) but it is a sharp edge: a caller that requires the loader and reads nothing
gets an empty, *silent* module list. It cost one confusing test failure during the spike. Phase 2
should either make the trigger explicit in the name or scan at require time and let app.js order the
require.

30
website/MODULE_SYSTEM.md
View File

@@ -4,6 +4,10 @@
org lead; Part 1 records what was verified against the working trees on 2026-08-10, including the org lead; Part 1 records what was verified against the working trees on 2026-08-10, including the
places the original draft was wrong. places the original draft was wrong.
**The normative contract is [`MODULE_API.md`](MODULE_API.md)** (Phase 1). This document decides what
the module system *is*; that one decides exactly what a module may call. Where the two differ, that
one wins — its Part 6 lists the four places it amends this document.
**Goal.** Turn Runic Gateway from a UO/ServUO-specific platform into a game-agnostic one. The core **Goal.** Turn Runic Gateway from a UO/ServUO-specific platform into a game-agnostic one. The core
architecture is unchanged — sidecar → website → browser. What changes is that game-specific architecture is unchanged — sidecar → website → browser. What changes is that game-specific
behaviour (routes, tables, screens, nav) leaves the core website and becomes an installable behaviour (routes, tables, screens, nav) leaves the core website and becomes an installable
@@ -120,7 +124,9 @@ replayed; **purge** is a separate, explicit, destructive admin action that runs
`purge.sql`. A real migration runner, covering core *and* modules together, is a legitimate future `purge.sql`. A real migration runner, covering core *and* modules together, is a legitimate future
workstream; it is not a prerequisite for this one. workstream; it is not a prerequisite for this one.
Twenty-five of the 67 tables move with the module: the 24 `shard_*` tables plus `uo_link_config`. Twenty-seven of the tables move with the module: the 26 `shard_*` tables plus `uo_link_config`.
(This said 25 when written; the working tree was recounted in Phase 1 — see
[`MODULE_API.md`](MODULE_API.md) §6.4.)
### 1.7 Boot and shutdown is a lifecycle gap ### 1.7 Boot and shutdown is a lifecycle gap
@@ -378,11 +384,20 @@ Phase 1 prototypes exactly this before anything is committed to it (§2.7).
Nothing else can be trusted until the first of these is done. Nothing else can be trusted until the first of these is done.
**Phase 1 — API contract + spike (blocking).** Merge this document. Write the contract at **Phase 1 — API contract + spike (blocking).** Merge this document. Write the contract at
`docs/website/MODULE_API.md`. Then a throwaway spike on an unmerged branch moving [`docs/website/MODULE_API.md`](MODULE_API.md) — **done**; it amends this document in four places,
**`/api/v1/public/atlas/*`** behind the proposed surface — the smallest honest test: five routes, listed in its Part 6, one of which (OpenAPI generation, §6.1 there) needs a decision before Phase 2
DB-backed, no sidecar, no SSE, one boot hook. The spike must *also* prove the §2.6 chunk load end to starts. Then a throwaway spike on an unmerged branch moving **`/api/v1/public/atlas/*`** behind the
end, since that is the highest-risk decision in the plan. Exit criteria: no internal-file imports, proposed surface — the smallest honest test: six routes, DB-backed, no sidecar, no SSE, one boot
`npm run routes:manifest` produces a zero-line diff, and the chunk loads under the enforced CSP. hook. The spike must *also* prove the §2.6 chunk load end to end, since that is the highest-risk
decision in the plan. Exit criteria: no internal-file imports, `npm run routes:manifest` produces a
zero-line diff, and the chunk loads under the enforced CSP.
**Phase 1 is complete.** The spike ran on `website` branch `spike/module-atlas` (cut from `edge`,
never merged) and **met all three exit criteria** — see [`MODULE_API.md`](MODULE_API.md) Part 7. §2.6
survives intact: the prebuilt chunk loads and renders under `script-src 'self'` with zero violation
reports. The one thing it changed is that §2.6's one-React rule turns out to have a server-side twin
nobody had written down — a module cannot resolve core's `express` either, so core hands that over
too (API §7.2).
**Phase 2 — Core scaffolding, no behaviour change.** One PR each, in order: **Phase 2 — Core scaffolding, no behaviour change.** One PR each, in order:
@@ -499,6 +514,9 @@ Conventional Commits, the AI-disclosure trailer, branches cut from an up-to-date
| 2 | Website and installer stay independent; delivery is website-side only | §1.11, §2.5 | | 2 | Website and installer stay independent; delivery is website-side only | §1.11, §2.5 |
| 3 | Core declares an extension slot on `/admin/users/:id`; all six URLs preserved | §1.9 | | 3 | Core declares an extension slot on `/admin/users/:id`; all six URLs preserved | §1.9 |
| 4 | Phase 1 spike targets `/api/v1/public/atlas/*` | §2.7 | | 4 | Phase 1 spike targets `/api/v1/public/atlas/*` | §2.7 |
| 4a | The contract lives in [`MODULE_API.md`](MODULE_API.md); it is normative where the two differ | §2.7 |
| 4b | Modules ship an OpenAPI **fragment**; core merges started modules' fragments into `/api/docs.json` | API §6.1 |
| 4c | Core exposes a **curated, closed** UI kit + request primitive on `window.__rg`, versioned by `MODULE_API_VERSION` | API §3.4 |
| 5 | Install surfaces: admin panel and the Docker environment; never a build step | §2.5 | | 5 | Install surfaces: admin panel and the Docker environment; never a build step | §2.5 |
| 6 | One repo, one bundle — server and client halves version together | §2.3 | | 6 | One repo, one bundle — server and client halves version together | §2.3 |
| 7 | Android app is a separate plan; core owes it `/api/v1/public/modules` | §2.5, §2.7 | | 7 | Android app is a separate plan; core owes it `/api/v1/public/modules` | §2.5, §2.7 |