Phase 2 PR 4 of MODULE_SYSTEM.md §2.7. Records what §1.8's three entangled files and §1.9's extension slot actually became, and four decisions taken with them. MODULE_API.md §2.4: - registerNotificationStreams takes the catalog ALONE. `mapEvent` was a leftover from before §1.8's push inversion was settled — a module owns fromShardEvent and calls publish() with an id it resolved, so core never needs a second route to the same place. It follows that the public-safety filter is module-internal, which is the right home: the kinds, the streams and the filter become one file that moves together. - the entry shape is two booleans, not a single `scope` — that object is the body of GET /auth/me/notifications/streams and a shipped Android client reads both. - registerAnnounceLeg gains `label`, so a module's leg renders in the admin panel with no client change; and legs became `announce_job_legs` ROWS, because a module cannot ALTER a core table and a registered leg had nowhere to live. - every call STAGES; nothing commits until the module as a whole is known good. New §6.5 — the eight grandfathered names (seven stream ids, one leg id), allowed to `uo` alone by explicit allowlist, the same shape as the loader's legacy table prefixes. Grandfathering rather than dropping the rule is what keeps it real for every module written after this one. New §6.6 — an extension slot is invisible to static analysis, so core needs §6.1's fragment merge for its OWN slot fills, a phase earlier than the plan expected. Moving the six users-detail routes behind the slot deleted 407 lines from swagger-output.json while printing `Success`. Which slots and where each hangs are both derived — from the registry and from the live express stack — because a written-down copy drifts. MODULE_SYSTEM.md §1.8 / §1.9 record the outcome per file, and §2.7 the progress. BACKEND_DESIGN.md and website-README.md follow the moved files; the PUBLIC_KINDS reference at §"visibility" named the wrong file and is corrected to utils/shardBroadcast.js. Co-Authored-By: Claude <noreply@anthropic.com>
43 KiB
The Module System — design of record
Status: approved design, not yet implemented. Every decision in Part 3 has been settled with the org lead; Part 1 records what was verified against the working trees on 2026-08-10, including the places the original draft was wrong.
The normative contract is MODULE_API.md (Phase 1). This document decides what
the module system is; that one decides exactly what a module may call. Where the two differ, that
one wins — its Part 6 lists the four places it amends this document.
Goal. Turn Runic Gateway from a UO/ServUO-specific platform into a game-agnostic one. The core architecture is unchanged — sidecar → website → browser. What changes is that game-specific behaviour (routes, tables, screens, nav) leaves the core website and becomes an installable module. An operator installs the base site, installs the module for their game, and restarts.
The model is WordPress plugins, not a build system. An operator never compiles anything to deploy a module. The module's own CI publishes it prebuilt; the operator drops it in and enables it. This single constraint drives most of Part 2.
Out of scope. The sidecar's per-game protocol adapters. link/, servuo-plugins/ and
installer/ are the shard side and stay independent of this work — the installer runs on the shard
host and by design never contacts the website (installer/src/cli.rs:162). Also out of scope: the
Android app, which gets its own plan covering module discovery and multi-server profiles; this plan
only owes it the capability endpoint in §2.5.
Part 1 — What is actually there
1.1 The parts that are already clean
The extraction is closer to a folder move than a teardown, and that is not an assumption:
- Models.
server/src/model/holds 31 directories; exactly 8 are UO —shardAtlas/,shardClilocs/,shardEvents/,shardLinks/,shardMarket/,shardState/,shardVisibility/,uoLinkConfig/. No mixing withusers/,posts/,pages/,wiki/,settings/. - Routers. All 13 UO router/controller files are single-purpose, with no shared code:
admin/shard.router.js,admin/shardAtlas|shardClilocs|shardOps|shardVisibility.controller.js,admin/uoLink.router.js+.controller.js,public/atlas.router.js+.controller.js,public/shard.router.js+.controller.js,player/shard.router.js+.controller.js. - Mount points.
router/v1/{public,admin,player}/index.jsare pure mount tables that declare no routes of their own. Module mounting drops straight in with no restructuring. - The API surface is small and knowable. The nine UO
utils/files import only four things from core:settings.model,logger,auth,pushDispatch. That is the empirical basis for §2.1 — the contract is derived from what the real code uses, not designed speculatively.
1.2 Route prefixes: one flat module prefix is impossible
A module cannot be handed a single pre-scoped router at, say, /api/v1/game/uo, because the existing
UO URLs live under three different access tiers — /api/v1/public/shard/*,
/api/v1/admin/shard/*, /api/v1/player/shard/* — and those URLs are protected by
routeManifest.test.js and consumed by three shipped clients (SPA, Android app, Discord bot).
Resolved: a module owns a named slot inside each tier. It still only ever holds a pre-scoped
express.Router() and structurally cannot reach above its mount point; it simply holds one per tier.
"mounts": {
"public": ["/shard", "/atlas"],
"admin": ["/shard", "/uo-link"],
"player": ["/shard"]
}
The loader rejects a prefix collision between two modules, or between a module and core, at registration time. That check is unavoidable; the per-request routing boundary is not left to the module's good behaviour.
1.3 There is no server-side nav list, and there must not be one
server/src/utils/navOverrides.js (lines 13–21) refuses this explicitly:
What this module cannot check, deliberately: whether a
toexists. The three base NAV arrays are client constants (SiteHeader.jsx, AdminLayout.jsx, PlayerPortalLayout.jsx). Shipping a copy of them to the server would create a second source of truth for navigation that drifts the first time a route is added…
Confirmed: export const NAV lives in client/src/components/SiteHeader.jsx:23,
client/src/routes/admin/AdminLayout.jsx:56, client/src/routes/player/PlayerPortalLayout.jsx:41.
The server validates override shape and nothing else.
Resolved: nav registration is client-side, performed by the module's own client bundle
against a core-provided registry. No server nav API is introduced, and
THEMING_AND_NAV.md's override model is untouched. The resulting pipeline is:
registered defaults (core + modules) → role/feature filtering → admin overrides → rendered nav
1.4 Module nav items interleave into core groups
Appending a "UO" group is not enough. Today's UO items sit inside core groups in AdminLayout.jsx:
group Moderation holds /admin/shard-ops and /admin/houses; group System holds
/admin/shard, /admin/shard-visibility, /admin/shard-atlas; the unnamed footer group holds
/admin/characters. MOD_PATHS (line 109) additionally hardcodes two UO paths as
moderator-visible.
Resolved: nav registration takes a target group and order ({ group: 'Moderation', order: 30 }),
and MOD_PATHS becomes a roles-derived computation rather than a path allowlist.
1.5 The public nav's feature-gating mechanism is itself a shard system
Ten of the sixteen entries in SiteHeader.jsx's NAV carry a feature: key (status, champs,
guilds, governors, houses, ruleset, atlas, leaderboards, market), resolved by
useShardFeatures() against /api/v1/public/shard/features — the shard visibility system.
Extracting the module removes the provider that core's own nav filter depends on.
Resolved: core keeps a generic feature-flag context with a registerable provider; the module
registers its useShardFeatures for its own namespace. No core nav item carries a feature today,
so with no module installed the filter is a correct no-op.
1.6 There is no migration system to model a module migration runner on
server/db/schema.sql is a single idempotent file — 1,380 lines, 67 tables — replayed in full on
every boot by ensureSchema() (src/utils/db.js:48), split on ; and executed statement by
statement. Schema evolution uses ALTER TABLE … ADD COLUMN IF NOT EXISTS / MODIFY COLUMN
(from line 1322). There is no version table, no runner, no migrations directory.
A module-scoped migration runner would therefore be the first migration system in the codebase, and would leave core and modules on two different schema models.
Resolved: modules ship a schema.sql fragment, replayed idempotently by the same
ensureSchema() immediately after core's. Forward-only falls out for free — it is all an idempotent
replay can be. Install and upgrade become the same operation. Uninstall stops the fragment being
replayed; purge is a separate, explicit, destructive admin action that runs the module's
purge.sql. A real migration runner, covering core and modules together, is a legitimate future
workstream; it is not a prerequisite for this one.
Twenty-seven of the tables move with the module: the 26 shard_* tables plus uo_link_config.
(This said 25 when written; the working tree was recounted in Phase 1 — see
MODULE_API.md §6.4.)
1.7 Boot and shutdown is a lifecycle gap
server/src/server.js holds eight UO call sites that routes, nav and schema do not cover:
| Line | Call |
|---|---|
| 92 | shardAtlas.refreshOnBoot() |
| 99 | shardClilocs.refreshOnBoot() |
| 107 | shardMarket.refreshDisplayNames() (conditional on the cliloc import result) |
| 130 | uoLinkSocket.start() |
| 131 | checkUoLink() — plus the whole function at 147–169 |
| 179 | uoLinkSocket.stop() |
| 180 | shardBroadcast.closeAll() |
| 7–19 | five top-level requires of UO modules |
Resolved: the API surface includes onBoot(ctx) and onShutdown(), each individually
try/caught by the loader per §2.4.
1.8 Three core files are genuinely entangled
Everything else is a folder move. These are not:
src/config/notificationStreams.js— the push stream catalog.mapShardEvent()and most ofSTREAMSare shard-derived, and it importsPUBLIC_KINDSfromutils/shardBroadcast. Push infrastructure is core; this catalog is module content. →registerNotificationStreams(streams).src/utils/pushDispatch.js— core infrastructure, butfromShardEvent()(line 112) requires theshardLinksmodel (line 21) andmapShardEvent(line 23). → invert:publish()stays core,fromShardEventmoves into the module and calls it.src/utils/announceWorker.js— the news dispatcher, with two delivery legs: Discord (core) and town crier (module, viauoLinkClient.postTownCrier, line 36). →registerAnnounceLeg({ leg, label, dispatch, classify }).
src/utils/newsGump.js is module-side (news → in-game gump) and moves whole.
Done in Phase 2 PR 4, with core still the only registrant — the registries are
src/modules/registries.js and core goes through them by the same door a module will
(registerCore(), called explicitly from app.js before modules.load()). What each of the three
became:
- Split in two.
config/coreStreams.jsis core's one stream (news.post, produced by the website's own posts path);config/shardStreams.jsis the other seven plusmapShardEventand the public-safety filter, and moves to module-uo whole.registerNotificationStreamslost itsmapEventhalf — seeMODULE_API.md§2.4 for why that was a leftover, and what follows for the public/personal split. - Inverted.
pushDispatch.jsispublish+isAllowedEndpointand nothing else;utils/shardPush.jsholdsfromShardEventand is whatshardIngestnow calls. - Legs became registrations, and per-leg rows. The
towncrier_*/discord_*column groups onannounce_jobscould never have held a module's leg — a module cannotALTERa core table — so they becameannounce_job_legs, backfilled and dropped in the same idempotent replay. The worker no longer contains the word "towncrier": it iterates whatever is registered.
The residue in core is a one-time backfill block in schema.sql, deletable once every deployment has
booted it, and the two lines of registerCore() that Phase 3 turns into module-uo's register().
1.9 A fourth mount shape: module routes under a core resource
router/v1/admin/users.router.js mounts usersShard.controller.js at six UO sub-paths of a core
resource — /:id/shard/accounts|sales|houses|online|standing and DELETE /:id/shard/link/:account.
And GET /api/v1/admin/users/:id (line 159) is itself served by usersShard.getUser, which is core
semantics that ended up in the UO controller by proximity.
Resolved, two parts: (a) getUser moves back into admin.controller.js; (b) core declares a
narrow extension slot on /admin/users/:id that the module mounts into, so core never learns
what "shard" means and all six URLs are preserved. Only core may declare an extension slot; a module
may not invent one.
Both done in Phase 2 PR 4, with core filling its own slot: the six paths are
router/v1/admin/usersShard.router.js, registered into admin.users.detail by registerCore(), and
Phase 3 changes the registrant rather than the routes. The slot's router is created at declare time
and filled later, because users.router.js is required while app.js is still being built. It is
mounted last on the resource, so core wins any path conflict by first-match.
One consequence was not foreseen and is worth the warning: a slot is invisible to static analysis.
There is no literal mount for swagger-autogen to follow, so the move silently deleted all six paths
from swagger-output.json while printing Success. The OpenAPI build now merges a generated
fragment per filled slot — MODULE_API.md §6.6.
1.10 The Discord bot has no UO logic
The draft listed the bot's "UO-specific event/moderation logic" as an extraction candidate. Grepping
website/bot/src for uo|ultima|shard|towncrier|governor|vendor returns zero matches. The bot's
only site coupling is src/site/siteApiClient.js. There is nothing to extract.
1.11 The installer is not, and will not become, the delivery path
Two independent reasons, and the decision is that website and installer stay independent:
- The installer runs on the shard host and never contacts the website —
src/cli.rs:162: "The installer never contacts your website, never deletes anything from your…". A website module is a website-host artifact. Bundleis hardcoded to exactly two components.installer/src/bundle.rsdeclarespub link: LinkComponentandpub overlay: OverlayComponent, both non-Option, alongside a single top-levelprotocol: u32andSUPPORTED_SCHEMA: u32 = 1. A third artifact type would be a schema-2 bump — and the sidecar/overlay protocol number has nothing to say about a website module anyway.
Note also that there is no SHA256SUMS trust anchor anywhere in the installer, contrary to the
draft. The real model is a per-asset sha256 field inside a bundle JSON fetched anonymously over
HTTPS from the bundles branch. No signatures. The shape is worth reusing; the name was wrong.
1.12 Modules must mount synchronously, from the filesystem
server/scripts/routeManifest.js:38 and server/swagger/swagger.js:29 both walk the Express stack by
require-ing src/app.js with no database connection — the manifest script deliberately points
the pool at a dead port. A DB-driven async loader would make module routes invisible to both,
silently breaking the frozen-URL-surface test and shipping undocumented routes.
Resolved: the filesystem is the mounting source of truth. app.js synchronously scans
modules/*/module.json at require time and mounts what it finds. The installed_modules row carries
state and metadata (version, installed-at, startup_failed reason, admin enable/disable) and is
reconciled against the filesystem once the DB is up. A module disabled in the DB is skipped by a
one-line dispatch guard rather than being unmounted, so the URL surface stays deterministic and
generatable.
1.13 The client seam is a route registry, not an admin-panel loader
client/src/App.jsx is a flat 235-line static route table, and UO routes appear in all three areas —
public (/site/shard, /site/shard/activity, /site/governors, /site/houses, /site/atlas,
/site/atlas/:slug, /site/market, /site/market/vendors/:serial, plus champs, guilds, rules,
leaderboards), admin (shard, shard-visibility, shard-atlas, shard-ops, houses,
characters, characters/:serial) and player. Nav is one consumer of that registry, not the
mechanism itself.
1.14 Production is a prebuilt, pull-only image — and that is the binding constraint
website/Dockerfile bakes client/dist at image build time, and docker-compose.yml has no
build: stanza at all (deliberately: "a production host can only ever pull, never accidentally
build"). Combined with the requirement that an operator must never build anything to deploy a
module, this rules out build-time inclusion of module client code, which the draft had as its
default.
It also rules out import maps as the shared-dependency mechanism: config/csp.js:49 sets
'script-src': ["'self'"] with no 'unsafe-inline', and an import map must be an inline
<script type="importmap">.
Resolved — see §2.6. The path that survives all three constraints is: the module's CI ships a
prebuilt ESM chunk, core hands it React through a global rather than an import map, and
htmlShell.js:111 injects a same-origin <script type="module" src>, which 'self' already
allows.
Part 2 — The plan
2.0 Scope and non-goals
Out of scope unless Phase 1 turns up a concrete reason otherwise: hot module reload; sandboxing beyond the boundary stated in §2.2; inter-module dependency resolution; a module marketplace or discovery UI; automatic data rollback beyond the forward-only model in §1.6. Install and uninstall require a controlled restart — never a rebuild.
Added: no installer changes at all (§1.11), and no Android changes in this workstream beyond the one consequence recorded in §2.8.
2.1 The API surface, derived from real dependencies
Taken from what the UO code actually imports today. Nothing speculative — if module-uo does not use it, it is not on the list.
Server — the ctx handed to a module's entry point
| Member | Backed by | Why it is here |
|---|---|---|
ctx.db |
utils/db (query, pool) |
every *.db.js |
ctx.settings |
model/settings/settings.model |
shardIngest.js:20 |
ctx.log(namespace) |
utils/logger |
all nine UO utils |
ctx.auth |
utils/auth |
shardVisibility.js:26 |
ctx.push.publish() |
utils/pushDispatch |
shardIngest.js:22 |
ctx.secretBox |
utils/secretBox |
uoLinkConfig model |
ctx.middleware |
requireAuth, requireRole, siteMode, validate |
every UO router |
ctx.uploads |
admin/imageUpload.js |
atlas art import |
ctx.posts |
model/posts/posts.model |
newsGump.js, announce legs |
Server — what a module registers
registerRoutes(mounts) (§1.2) · registerExtension(slot, router) (§1.9) ·
registerNotificationStreams({ streams, mapEvent }) (§1.8) ·
registerAnnounceLeg({ leg, dispatch, classify }) (§1.8) · onBoot(ctx) / onShutdown() (§1.7).
Client — what a module registers
registerRoutes({ public, admin, player }) (§1.13) ·
registerNav({ nav, group, order, feature }) (§1.3, §1.4) ·
registerFeatureProvider(namespace, hook) (§1.5).
The acceptance test for the whole contract: module-uo runs with zero require/import
reaching outside its own directory. Any gap extends the surface before extraction proceeds.
2.2 What the module boundary is, and is not
A module runs in the same Node process with full access. The boundary is a code-organisation and distribution boundary, not a security boundary — which is fine for a self-hosted operator installing software they chose, the same trust category as running its schema fragment. What makes it worth having is that modules interact with core through a defined surface, so a core refactor cannot silently break a module. Hence the zero-internal-imports rule above, enforced in CI rather than by review.
2.3 Module packaging — one repo, one bundle
RunicGateway/Module-uo — https://gitea.whitlocktech.com/RunicGateway/Module-uo.git, note the
capital M, matching Android-app's casing rather than the lowercase directory name. The repo
exists but is empty as of 2026-08-10: no branches, no initial commit. Its first commit needs the
usual scaffolding — README.md, LICENSE.md (GPL-3.0-or-later), CONTRIBUTING.md with the
AI-disclosure clause, the PR template, and CI.
Server and client halves live side by side and version together, so a route and the screen that calls it can never be mismatched:
RunicGateway/Module-uo
module.json id, version, coreApi range, mounts, extensions
server/ routers, controllers, models, utils
server/db/schema.sql fragment replayed by ensureSchema()
server/db/purge.sql destructive, only ever run by an explicit purge
client/src/ route components, nav registrations, feature provider
client/dist/ PREBUILT ESM chunk, published by module CI
Release artifact: module-uo-<version>.tar.gz plus a manifest carrying its sha256.
The module's id is uo — that is what appears in module.json, in installed_modules, in the
modules/<id>/ path and in the URL segment. Module-uo is the repository; module-uo elsewhere in
this document names the module and its artifact, not the repo.
module.json declares a coreApi semver range, checked at boot against a MODULE_API_VERSION
constant in core; a mismatch fails loudly rather than silently. This is a separate number from
PROTOCOL_VERSION, which versions the shard wire and says nothing about a website module.
2.4 The module state machine
installed → enabled → started, with disabled and startup_failed as recoverable states.
A module that fails to load must never take the site down. The loader catches failures across the
module's entire lifecycle — require, schema fragment, router construction, registration calls,
onBoot — not merely those that surface after a router object was returned. Any failure at any point
marks that one module startup_failed, records the reason, and the site comes up with that module's
routes and nav absent. startup_failed is recoverable from the admin panel — disable, retry, or roll
back to the previous version — with no shell access to the box.
Where the states live. One installed_modules row per module, keyed by its id, with the machine
held in a single state column carrying all five values — the shape this section already describes,
rather than a policy flag beside a runtime one. The table also carries name/version for the admin
screen, failure_stage + failure_reason for MODULE_API.md §4.4's recorded
reason, source + sha256 for the
install provenance of §2.5 below (both null for a directory placed on the volume by hand, which stays
supported), and installed_at / started_at / updated_at. Full column list in
BACKEND_DESIGN.md §3.
The row is a record of what happened, never the source of truth for what is mounted. The loader
scans the filesystem at require time, before the database is reachable (API §4.1), so the URL surface
is a property of the volume and not of a row here. What the row decides is whether a mounted module
answers (disabled ⇒ its guard 404s, API §4.5) and what the admin panel shows after a failure.
This is also why
routes.manifest.json can be generated against a dead database.
disabled is the only state a boot leaves alone. Every boot resets each non-disabled row to
enabled, clearing any recorded failure, and the load that follows writes this boot's outcome —
started or startup_failed. Three consequences, all deliberate:
- A
startup_failedmodule is retried on every restart. An operator who fixes the underlying cause — a truncated file, a missing dependency, a database that was not up yet — gets the module back by restarting, with no admin-panel visit. The cost is that a deterministically broken module re-records its failure each boot, which is the honest thing for it to do. - A stale reason can never be shown against a running module, because every non-failing transition clears the failure columns.
- Disabling is an operator decision, not an outcome, so it survives restarts untouched — and a
module the operator switched off is neither started nor re-recorded as failed if it happens to be
broken.
installedis likewise transient: it is the gap between an install writing the row and the restart that resolves it.
A re-install or an upgrade refreshes name/version/provenance and deliberately leaves state
alone: upgrading an enabled module must not silently switch it off, and re-installing a disabled one
must not silently switch it on.
2.5 Install, uninstall, purge
Modules live on a mounted volume, not in the image — the same treatment uploads already gets in
docker-compose.yml. That is what makes the WordPress model work against a pull-only image.
Install: admin selects the module → bundle downloaded from the module repo's release and verified
against its sha256 → unpacked into modules/<id>/ on the volume → installed_modules row written →
restart. On boot the loader scans the filesystem (§1.12), validates prefixes, mounts, replays the
schema fragment, runs onBoot, and each module reaches started or startup_failed.
Nothing is compiled at any point. The operator restarts; they never build.
Uninstall (default, non-destructive): row set to disabled, directory removed, restart. The
module's tables and data are retained. Purge is a separate, explicit, destructive action that
runs purge.sql; it is never bundled into uninstall.
Surfaces: the admin panel, and the Docker environment under website/ — a declarative module set
resolved at container start from the mounted volume, so a compose-managed host is not driven by
clicking. Both paths write the same installed_modules row and neither requires a build step.
2.6 How the client half loads
This is the piece §1.14 constrains hardest. Three requirements had to hold at once: the operator
builds nothing, production pulls a prebuilt image, and script-src 'self' forbids inline script.
- The module's CI builds its client half with Vite in library mode, declaring
react,react-domandreact-router-domas externals. The module never bundles its own React — there is exactly one React instance, owned by core. - Core exposes the shared dependencies on a global before mount —
window.__rg = { react, reactDom, router, registry }— and the module's externals resolve to it. A global, not an import map, precisely because an import map must be inline and CSP forbids that. htmlShell.jsinjects the module's entry script. It already rewrites</head>(utils/htmlShell.js:111), so this is an extension of a working mechanism, not a new one. The tag is<script type="module" src="/modules/uo/entry.js">— same-origin, so'self'passes with no nonce and no inline.- The SPA reads
/api/v1/public/modulesto learn what to load, then registers routes, nav and its feature provider throughwindow.__rg.registry.
Phase 1 prototypes exactly this before anything is committed to it (§2.7).
2.7 Phases
Phase 0 — unblock CI and scaffold the repo. Land the one-line pr-checks.yml trigger fix on
website main (§2.9), cut edge from main, and give Module-uo its initial commit (§2.3).
Nothing else can be trusted until the first of these is done.
Phase 1 — API contract + spike (blocking). Merge this document. Write the contract at
docs/website/MODULE_API.md — done; it amends this document in four places,
listed in its Part 6, one of which (OpenAPI generation, §6.1 there) needs a decision before Phase 2
starts. Then a throwaway spike on an unmerged branch moving /api/v1/public/atlas/* behind the
proposed surface — the smallest honest test: six routes, DB-backed, no sidecar, no SSE, one boot
hook. The spike must also prove the §2.6 chunk load end to end, since that is the highest-risk
decision in the plan. Exit criteria: no internal-file imports, npm run routes:manifest produces a
zero-line diff, and the chunk loads under the enforced CSP.
Phase 1 is complete. The spike ran on website branch spike/module-atlas (cut from edge,
never merged) and met all three exit criteria — see MODULE_API.md Part 7. §2.6
survives intact: the prebuilt chunk loads and renders under script-src 'self' with zero violation
reports. The one thing it changed is that §2.6's one-React rule turns out to have a server-side twin
nobody had written down — a module cannot resolve core's express either, so core hands that over
too (API §7.2).
Phase 2 — Core scaffolding, no behaviour change. One PR each, in order:
installed_modulestable + the §2.4 state machine.src/modules/loader.js— synchronous filesystem scan, manifest validation, prefix-collision rejection, per-module try/catch across the whole load path, mounting into the tier routers.ensureSchema()extended to replay module fragments after core's.- The three de-entanglement registries (§1.8), with core still the only registrant.
- Boot/shutdown hook dispatch in
server.js, likewise. GET /api/v1/public/modules— installed ids, versions and capabilities, shaped like the existing branding/site-settings endpoint. The SPA needs it to know what to load; the Android plan consumes the same endpoint.- Client
src/modules/registry.js, thewindow.__rgshared-dependency global, and thehtmlShellscript injection — empty registry, no visible change. MOD_PATHS→roles-derived (§1.4); the generic feature-provider seam (§1.5).docker-compose.ymlgains themodulesvolume.
Exit criterion: routes.manifest.json diff is zero lines and every existing test passes. If Phase 2
changes one URL, it is wrong.
Progress: PRs 1-4 done.
-
PR 1 —
installed_modulesand the state machine, with the stored shape and the boot rules settled in §2.4 above. -
PR 2 —
server/src/modules/loader.js: the filesystem scan, manifest validation, prefix and table-name collision rejection, per-module try/catch and the tier mount, behind the §4.5 dispatch guard. Two decisions landed with it, both recorded inMODULE_API.md: the load trigger is one explicitmodules.load(tierRouters)call inapp.js, never a lazy scan (API §7.6); and the "does core own this prefix" check probes the live tier routers rather than a hardcoded table, so it cannot drift when core adds a capability router (API §4.3). The three de-entanglement registries and the two lifecycle hooks thrownot available until phase 2 PR 4/5rather than no-op — an accepting stub would let a module believe it had registered something. 28 tests, all on the failure paths. -
PR 3 — schema fragment replay.
ensureSchema()replays each installed module's fragment after core's, with the statement splitter extracted toutils/sqlStatements.jsso both are split by the same code. The decision that shaped it, recorded inMODULE_API.md§2.6: the fragment is validated at load time and executed later, split on whether a database is needed to know the answer — a fragment breaking a stated rule never mounts, while a failure only the server could report (a bad column type) is post-mount and 503s. The rules are enforced as a leading-verb allowlist (CREATE,ALTER,INSERT,UPDATE) rather than theDROPdenylist §2.6 words them as, because the file is replayed on every boot. Found while wiring it:npm run seedcallsensureSchema()without ever requiringapp.js, so the replay has to tolerate an unscanned loader. -
PR 4 — the three de-entanglement registries,
src/modules/registries.js. Core's own streams, its Discord announce leg and its users-detail routes all go through them, so the seams are exercised on every boot before a module depends on them; §1.8 and §1.9 above record what each became. Four decisions landed with it, all recorded inMODULE_API.md: announce legs became a child table rather than waiting for Phase 3 (§2.4 — a module cannot alter a core table, so a registered leg had nowhere to live);mapEventdropped from the stream registry (§2.4 — a leftover from before the push inversion was settled); core registers through the same staging area a module uses; and core's six shard sub-paths moved behind the slot now rather than in Phase 3.Registering is validate-then-commit: the loader stages a module's claims and the second pass commits them, so a module that throws halfway through
register()— or fails a later validation step — leaves nothing behind. That is the registry-side twin of PR 2's second-pass mount rule.Two build tools needed teaching, both because a mechanism this PR introduced is one they had never seen.
scripts/routeManifest.jscould not decode a parameterised mount: its unwinder expected a group shape express does not emit, and the branch had never run. It threw rather than guessing, which is exactly what it is for. Andswagger-autogencould not follow a route into an extension slot, deleting 407 lines while reporting success; the fix is the fragment merge core owed anyway (MODULE_API.md§6.6).
There is still no module on the volume and no boot wiring, so this changes nothing an operator or a
client can see: 884 tests pass and routes.manifest.json is unchanged at 229 routes. The two
lines of OpenAPI that do move are the retry endpoint's summary and its leg, which is no longer a
fixed enum because the leg set is whatever has been registered.
Phase 3 — Extract module-uo. Moves out of website/: the 8 model directories and their 25
tables; the nine UO utils/ files plus newsGump.js; the 13 router/controller files;
scripts/importSpawnAtlas.js and db/spawnAtlas.art.json; usersShard.controller.js minus
getUser (§1.9); the shard-derived half of notificationStreams.js and the town-crier leg of
announceWorker.js; and on the client, roughly twenty route components, their nav registrations and
useShardFeatures.
Acceptance, all four required:
- Zero UO identifiers in core — no
shard,uoLink,cliloc,atlasortowncrieroutsidemodules/. Enforced by a CI grep test, not by review. - Zero internal-file imports from
module-uointo core. routes.manifest.jsonAPI diff is zero lines, except the deliberateGET /admin/users/:idownership move, which changes no URL. After extraction the core manifest no longer contains UO routes —module-uogenerates and freezes its own in its own repo.- A written
module-rustdry run — manifest, mounts, nav entries, one notification stream — not implemented, to prove the contract generalises before more is built on it.
Phase 4 — Delivery. The admin-panel Modules screen (install, enable, disable, retry, purge,
startup_failed with its recorded reason) and the Docker-environment path from §2.5. Deliberately
last, so loader, packaging, schema and chunk-loading problems are not all being debugged at once.
Phase 5 — The Integration Kit. RunicGateway/Integration-kit, the instruction book for building
a module for a game that is not UO — the website module, the sidecar and why it exists, and the
game-side plugin that feeds it. Scaffolded when Phase 2 lands, written against Phase 3's extraction,
finished alongside Phase 4. Full shape and its acceptance test in §2.11.
2.8 SPA URL namespacing — a deliberate break
Decision: module pages are namespaced, and old paths are not redirected. The site is not public yet, so bookmarks, inbound links and configured nav overrides carry no real weight. This buys a visible boundary in the URL rather than a hidden one.
The rule is that a module owns one path segment wherever it appears:
| Today | After |
|---|---|
/site/shard, /site/atlas, /site/market, /site/governors, … |
/uo/shard, /uo/atlas, /uo/market, /uo/governors, … |
/admin/shard-ops, /admin/shard, /admin/shard-visibility |
/admin/uo/shard-ops, /admin/uo/link, /admin/uo/visibility |
| player shard screens | /player/uo/… |
API URLs are not affected — they keep their exact paths per §1.2, so the Android app and the Discord bot need no change for the API.
Two consequences, both accepted:
- Saved nav-override rows are keyed by
to(utils/navOverrides.js), so any storednav_public/nav_admin/nav_playercustomisation stops applying and must be redone. No migration is written. android-app/.../ui/navigation/NavPaths.ktmaps SPA paths to native screens and holds ten/site/*constants that will no longer resolve. That is one small Android PR, folded into the separate Android module plan. App Links verification itself is unaffected — the manifest's intent filters only cover/mobile/callbackandauth/callback.
2.9 Branch strategy — edge, then one cutover
All website work lands on an edge branch and reaches main as a single cutover at the end,
the same shape used for protocol v3 and the Android theming workstream. Nothing
half-extracted is ever on main: a core that has grown a module loader but not yet lost its UO code
is a coherent state, and a core mid-extraction is not.
edge does not exist on website today — the protocol v3 cutover landed and the branch was cleaned
up, so it is cut fresh from main. Module-uo develops on its own main from its first commit;
it has no cutover to perform, since nothing depends on it until the website cutover lands.
Phase 0, and it blocks everything: the CI trigger. website/.gitea/workflows/pr-checks.yml
declares:
on:
pull_request:
branches: [main]
So a PR into edge runs no checks at all — no server tests, no client build, no bot install.
This is the same trap that let all nine Android M12 phase PRs merge with zero CI. It matters more
here than it did there, because Phase 2's exit criterion is a CI result: a zero-line
routes.manifest.json diff and a passing test suite. Running the whole workstream blind and
discovering the breakage at cutover is the expensive version of this.
The fix is one line — branches: [main, edge] — and it must land on website main before the
first module PR, not alongside it. build-images.yml is untouched: it triggers on push to main, so
images are published and production rolls at the cutover and at no point before it, which is correct.
2.10 Process obligations
Every server-side PR runs npm run swagger, npm run routes:manifest (the diff is reviewed, not
merely regenerated) and npm test, and carries a matching edit to BACKEND_DESIGN.md. Module
documentation aggregates in this repo under docs/modules/<id>/ rather than living in module repos.
Conventional Commits, the AI-disclosure trailer, branches cut from an up-to-date main.
2.11 The Integration Kit — the instruction book for building a module
RunicGateway/Integration-kit — https://gitea.whitlocktech.com/RunicGateway/Integration-kit.git,
empty as of 2026-08-10: no branches, no initial commit, exactly where Module-uo was at the start
of Phase 0. Its first commit needs the same scaffolding as any other repo here — README.md,
LICENSE.md (GPL-3.0-or-later), CONTRIBUTING.md with the AI-disclosure clause, the PR template.
Who it is for. Everything else in this plan is written for someone changing this system. The kit is written for someone building a new one: a person who wants Runic Gateway to front a game that is not Ultima Online, starting from nothing. It is the only document in the project whose audience is outside the org, and that changes how it is written — it explains and motivates rather than records decisions.
The job spans all three layers of the data path, which is why it is one book and not a page in each repo:
- The website module.
module.json, the server entry point and whatctxhands you, theregister*calls, the schema fragment, the prebuilt client chunk and the shared-dependency rule, packaging and release CI. The bulk of it. - The sidecar — what it is and, more importantly, why. The shard is never network-reachable; the shard dials out and the sidecar is the listener; the wire is a versioned compatibility contract rather than a build dependency; only the website's backend talks to it. A new game needs its own sidecar or an adapter into the existing one, and neither can be designed by someone who has been handed the message list and none of the reasoning.
- The game-side plugin — how a shard feeds the sidecar without ever letting the sidecar stall the
game: the bounded drop-oldest queue, the dedicated writer thread, world reads only on the game's
own thread.
servuo-plugins/is the worked example; the constraints are general, and a plugin that ignores them takes the game down when the sidecar wedges.
The rule that keeps it from rotting: the kit never re-specifies a contract.
MODULE_API.md stays normative for the module surface, and
../link/PLAN.md + ../link/INTEGRATION.md for the wire
protocol. The kit teaches — worked examples, the reasoning, the order to do things in, the mistakes
that cost time — and links out for the authority. Where it must show a member list it quotes with a
pointer, never a copy. A guide that restates a contract diverges from it silently, and a reader who
follows the divergent copy gets a module that fails validation for reasons the guide cannot explain.
It cannot be written before the contract is proven, so it trails the implementation rather than leading it:
- Scaffolded once Phase 2 lands — repo, license, CI, and an outline. By then a loader exists to describe and a real module to point at.
- Written against Phase 3's extraction, using
Module-uoas the worked example throughout. A kit whose examples are invented is a kit whose examples do not compile. - Phase 3's fourth acceptance criterion, the written
module-rustdry run, is really this book's first chapter — and doubles as the honest test that the contract generalises past its first module.
Acceptance: someone builds a trivial working module for a second game by following the kit alone, without reading core's source. Until that has happened it is a draft, however finished it looks.
The org landing page (RunicGateway/.profile) and the workspace's CLAUDE.md repo table both gain a
row for it — when it has content, not while it is an empty repo.
Part 3 — Settled decisions
| # | Decision | Where |
|---|---|---|
| 1 | Modules ship idempotent schema.sql fragments; no migration runner is built |
§1.6 |
| 2 | Website and installer stay independent; delivery is website-side only | §1.11, §2.5 |
| 3 | Core declares an extension slot on /admin/users/:id; all six URLs preserved |
§1.9 |
| 4 | Phase 1 spike targets /api/v1/public/atlas/* |
§2.7 |
| 4a | The contract lives in MODULE_API.md; it is normative where the two differ |
§2.7 |
| 4b | Modules ship an OpenAPI fragment; core merges started modules' fragments into /api/docs.json |
API §6.1 |
| 4c | Core exposes a curated, closed UI kit + request primitive on window.__rg, versioned by MODULE_API_VERSION |
API §3.4 |
| 5 | Install surfaces: admin panel and the Docker environment; never a build step | §2.5 |
| 6 | One repo, one bundle — server and client halves version together | §2.3 |
| 7 | Android app is a separate plan; core owes it /api/v1/public/modules |
§2.5, §2.7 |
| 8 | SPA pages namespaced: /uo/*, /admin/uo/*, /player/uo/* |
§2.8 |
| 9 | Clean break — no redirects, no nav-override migration; site is not public yet | §2.8 |
| 10 | Client half loads as a prebuilt ESM chunk with React shared via a core global | §2.6 |
| 11 | Website work lands on edge and reaches main as one cutover at the end |
§2.9 |
| 12 | The module repo is RunicGateway/Module-uo; the module id is uo |
§2.3 |
| 13 | RunicGateway/Integration-kit is the module-builder's instruction book — module + sidecar + game plugin, teaching only, never re-specifying a contract |
§2.11 |