Records Phase 2 PR 8 of the module system: MODULE_SYSTEM.md gains the PR entry in 2.7 and the built-it notes on 1.4 and 1.5; MODULE_API.md 3.3 is amended where building it settled something the draft left open or got wrong. The amendments to 3.3: - The pipeline arrow had role/feature filtering BEFORE admin overrides. The code has always been the other way round, deliberately - the filter runs last so it stays a boundary an override cannot cross (THEMING_AND_NAV 7). - `feature` was documented as public-area-only. It applies in all three areas: core's admin and player navs still carry no flags, but a module row that declares a gate and has it silently ignored is a trap. - How a provider is found: by the module that registered the row, not by a prefix parsed out of the flag name. Core's own rows resolve against the owner id `core`, which core registers useShardFlags under. - What a provider hook returns, and that every unknown fails OPEN. - Why calling one hook per provider in a loop is legal, and why the enumerator is a module export rather than a member of registry. - Six details the interleave settled: unordered rows append rather than defaulting to 0; an ungrouped admin row gets its own trailing group rather than joining core's; a module-created group is a legal override destination; a colliding `to` is dropped with a warning; and why the interleave must precede the override merge. MOD_PATHS' replacement is recorded in both files, including the defect the derivation fixed: the moderator redirect was a third hardcoded list that disagreed with MOD_PATHS about /admin/houses. THEMING_AND_NAV 7's "moderator confinement" note is amended to match. Code: website PR 8 (client-only; 160 client tests, manifest and OpenAPI unchanged), verified with the 7.7 browser smoke. Co-Authored-By: Claude <noreply@anthropic.com>
1090 lines
65 KiB
Markdown
1090 lines
65 KiB
Markdown
# The Module API — the contract
|
|
|
|
**Status:** Phase 1 deliverable of [MODULE_SYSTEM.md](MODULE_SYSTEM.md), **validated by the atlas
|
|
spike** — Part 7 records what the spike proved, what it changed in this contract, and the three
|
|
artifacts in it that are not design. This document is the
|
|
normative contract between the core website and an installed module. `MODULE_SYSTEM.md` decides
|
|
*what* the module system is; this decides *exactly what a module may call, what it must provide, and
|
|
what core promises not to break*.
|
|
|
|
Everything below is derived from what the UO code actually does today, re-read against the working
|
|
tree on 2026-08-10. Where the survey contradicted `MODULE_SYSTEM.md`, the contradiction is recorded
|
|
in Part 6 rather than quietly resolved — four of them, one of which (OpenAPI, §6.1) needs a decision
|
|
before Phase 2 starts.
|
|
|
|
**The one rule everything else serves:** a module reaches core *only* through the members named in
|
|
this document. Zero `require`/`import` from a module to a core file, enforced in CI (§5.1). A core
|
|
refactor that leaves this contract intact cannot break a module; anything a module needs that is not
|
|
here extends the contract first, in this file, before the module is written against it.
|
|
|
|
---
|
|
|
|
## Part 1 — Versioning
|
|
|
|
### 1.1 `MODULE_API_VERSION`
|
|
|
|
Core exports a single integer-major semver string from `server/src/modules/version.js`:
|
|
|
|
```js
|
|
const MODULE_API_VERSION = '1.0.0'
|
|
```
|
|
|
|
Every `module.json` declares a `coreApi` semver **range**. The loader checks it at boot, before it
|
|
requires a line of module code, and a mismatch fails that module loudly into `startup_failed`
|
|
(§4.4) with the two versions in the reason. It never silently proceeds.
|
|
|
|
| Change | Bump |
|
|
| --- | --- |
|
|
| A member is added to `ctx`, or a new `register*` call appears | minor |
|
|
| A member is removed or its signature changes | major |
|
|
| Behaviour of an existing member changes without a signature change | major |
|
|
| A core-internal refactor behind an unchanged member | none |
|
|
|
|
This is a **separate number from `PROTOCOL_VERSION`**, which versions the shard wire and has nothing
|
|
to say about a website module. It is also separate from the module's own version.
|
|
|
|
### 1.2 What is *not* contract
|
|
|
|
Core's internal file layout, table names, middleware ordering, the `api` client object's shape, and
|
|
every component under `client/src/components/` except the ones named in §3.4. A module that reaches
|
|
any of these is out of contract even if it happens to work.
|
|
|
|
---
|
|
|
|
## Part 2 — The server contract
|
|
|
|
### 2.1 `module.json`
|
|
|
|
Read synchronously by the loader from `modules/<id>/module.json`. Unknown top-level keys are
|
|
rejected rather than ignored, so a typo is a loud failure and not a silently-inert setting.
|
|
|
|
```json
|
|
{
|
|
"id": "uo",
|
|
"name": "Ultima Online",
|
|
"version": "1.0.0",
|
|
"coreApi": "^1.0.0",
|
|
"server": "server/index.js",
|
|
"client": { "entry": "client/dist/entry.js" },
|
|
"schema": "server/db/schema.sql",
|
|
"purge": "server/db/purge.sql",
|
|
"mounts": {
|
|
"public": ["/shard", "/atlas"],
|
|
"admin": ["/shard", "/uo-link"],
|
|
"player": ["/shard"]
|
|
},
|
|
"extensions": ["admin.users.detail"],
|
|
"capabilities": ["shard", "atlas", "market"]
|
|
}
|
|
```
|
|
|
|
| Key | Required | Meaning |
|
|
| --- | --- | --- |
|
|
| `id` | yes | `^[a-z][a-z0-9-]{1,31}$`. The directory name, the `installed_modules` key, the URL segment, the `window.__rg` registry key. Must equal the directory it was read from. |
|
|
| `name` | yes | Human label for the admin Modules screen. |
|
|
| `version` | yes | Semver. Recorded in `installed_modules`; shown on failure. |
|
|
| `coreApi` | yes | Semver range checked against `MODULE_API_VERSION` (§1.1). |
|
|
| `server` | no | Entry point, relative to the module root. Absent ⇒ client-only module. |
|
|
| `client.entry` | no | Prebuilt ESM chunk, relative to the module root, and **in a subdirectory** — the directory it sits in is what gets served (§3.1). Absent ⇒ server-only module; present-but-empty is rejected, since it claims a client half and delivers none. |
|
|
| `schema` | no | Idempotent SQL fragment (§2.6). |
|
|
| `purge` | no | Destructive teardown (§2.6). Required if `schema` is present. |
|
|
| `mounts` | no | Declared prefixes per tier (§2.3). Declaration is the contract; the loader compares it against what the module actually registers and rejects a mismatch. |
|
|
| `extensions` | no | Core extension slots this module mounts into (§2.4). |
|
|
| `capabilities` | no | Opaque strings published by `GET /api/v1/public/modules` (§2.9), for clients (the SPA, the Android app) to feature-detect against. Published only while the module is `started`. |
|
|
|
|
### 2.2 The entry point
|
|
|
|
`server/index.js` exports a single function. It is called once, synchronously, during `app.js`
|
|
require — **not** after the database is up.
|
|
|
|
```js
|
|
module.exports = function register(ctx, api) { /* … */ }
|
|
```
|
|
|
|
It must not `await`, must not touch the database, and must not throw for a reason that a retry would
|
|
fix. Everything that needs a live database belongs in `onBoot` (§2.5). This constraint is not
|
|
stylistic: `scripts/routeManifest.js` and `swagger/swagger.js` both require `app.js` with the pool
|
|
pointed at a dead port, and a module that queried at registration time would hang both.
|
|
|
|
### 2.3 `ctx` — what core hands the module
|
|
|
|
Every member below exists because a UO file uses it today. Nothing is speculative, and nothing that
|
|
module-uo does not need is on the list.
|
|
|
|
| Member | Signature | Backed by | First real caller |
|
|
| --- | --- | --- | --- |
|
|
| `ctx.express` | the `express` namespace | core's `node_modules` | every module router (§7.2) |
|
|
| `ctx.validator` | the `express-validator` namespace | core's `node_modules` | `atlas.router.js` |
|
|
| `ctx.db.query` | `(sql, params?) => Promise<rows>` | `utils/db` | every `*.db.js` |
|
|
| `ctx.db.pool` | mariadb pool | `utils/db` | `shardAtlas.db.js` (streamed import) |
|
|
| `ctx.log` | `(namespace) => { error, warn, info, debug }`, each `(msg, meta?)` | `utils/logger` | all nine UO utils |
|
|
| `ctx.settings.get` | `(key) => Promise<string\|null>` | `model/settings` | `shardAtlas.model` |
|
|
| `ctx.settings.set` | `(key, value, updatedBy?) => Promise<void>` | `model/settings` | `shardAtlas.model:60` |
|
|
| `ctx.settings.getInstanceName` | `() => Promise<string>` | `model/settings` | `shardIngest.js:84` |
|
|
| `ctx.auth.getUserFromRequest` | `(req) => { id, username, role } \| null` | `utils/auth` | `shardVisibility.js` |
|
|
| `ctx.push.publish` | `(streamId, { ref?, ownerUserId? }) => Promise<void>` | `utils/pushDispatch:92` | `shardIngest.js:22` |
|
|
| `ctx.secretBox` | `{ encrypt(s), decrypt(s) }` | `utils/secretBox` | `uoLinkConfig.model` |
|
|
| `ctx.middleware` | `{ requireAuth, requireRole, siteMode, validate, noindex }` | `auth/session.middleware`, `middleware/*` | every UO router |
|
|
| `ctx.uploads` | `{ upload, UPLOAD_DIR, MIME_EXT }` | `admin/imageUpload.js` | atlas art import |
|
|
| `ctx.posts` | `{ listAll, getById, linkAnnounceJob, markAnnounced }` | `model/posts` | `newsGump.js:108`, `announceWorker.js:58` |
|
|
| `ctx.paths.moduleRoot` | absolute path to `modules/<id>/` | loader | atlas art, cliloc files |
|
|
| `ctx.moduleId` | the id from `module.json` | loader | log tags, table checks |
|
|
|
|
Three narrowings from `MODULE_SYSTEM.md` §2.1, all deliberate:
|
|
|
|
- **`ctx.auth` is one function, not `utils/auth`.** The facade also re-exports `signToken`,
|
|
`setAuthCookie` and the TOTP challenge primitives. Minting sessions is core's job; a module that
|
|
needs an identity needs to *read* one.
|
|
- **`ctx.settings` is three functions, not the model.** The model exports 24 names, most of them
|
|
registration/game-signup/app-links policy that is core's business.
|
|
- **`ctx.posts` is four functions.** `create`/`update`/`remove` are the CMS, not a module's.
|
|
|
|
And one addition the spike forced: **`ctx.express` and `ctx.validator`**. A module lives at
|
|
`<repo>/modules/<id>/`, outside `server/`, so Node's resolver never reaches `server/node_modules` and
|
|
`require('express')` from a module simply fails — which is how this was found. Even where it
|
|
resolved, a second express in the process is a second `Router` prototype. Core owns one express, as
|
|
it owns one React (§7.2).
|
|
|
|
`ctx` is frozen (`Object.freeze`, one level deep) before it is handed over. That is a guard against
|
|
accident, not against a hostile module — per `MODULE_SYSTEM.md` §2.2 the boundary is organisational,
|
|
not a security boundary.
|
|
|
|
### 2.4 `api` — what the module registers
|
|
|
|
The second argument. Every call is synchronous, idempotent-free (calling twice is an error), and
|
|
validated at once rather than at first use.
|
|
|
|
```js
|
|
api.registerRoutes({ public: {...}, admin: {...}, player: {...} })
|
|
api.registerExtension(slot, router)
|
|
api.registerNotificationStreams(streams)
|
|
api.registerAnnounceLeg({ leg, label, dispatch, classify })
|
|
api.onBoot(async (ctx) => {})
|
|
api.onShutdown(async () => {})
|
|
```
|
|
|
|
**Every call STAGES; nothing is committed until the module as a whole is known good.** A claim's
|
|
shape is checked at the call, so a malformed one throws with the registrant's own stack; whether the
|
|
name is *taken* can only be answered once the batch is complete, and is checked when the loader
|
|
commits it in its second pass. The consequence is the one that matters: a module that registers two
|
|
streams and then throws — or fails `checkDeclared` after `register()` returns — has left nothing
|
|
behind. A half-registered catalog would be worse than a missing one, because it is a subscribable
|
|
stream nothing will ever publish to. This is the registry-side twin of §4.3's second-pass mount rule,
|
|
and both exist for the same reason.
|
|
|
|
**`registerRoutes(mounts)`** — one `express.Router()` per prefix per tier:
|
|
|
|
```js
|
|
api.registerRoutes({
|
|
public: { '/shard': shardRouter, '/atlas': atlasRouter },
|
|
admin: { '/shard': adminShardRouter, '/uo-link': uoLinkRouter },
|
|
player: { '/shard': playerShardRouter },
|
|
})
|
|
```
|
|
|
|
The keys must match `module.json`'s `mounts` exactly. Prefixes are validated `^/[a-z0-9][a-z0-9-]*$`
|
|
— one segment, no nesting, no parameters — and rejected on collision with core's own mount table or
|
|
with another module's, at registration time. The router is mounted *inside* the tier, so it
|
|
structurally cannot reach above its prefix.
|
|
|
|
**The tier gate is already applied.** A router registered under `admin` sits behind
|
|
`noindex, isLoggedIn, requireRole('admin','editor','moderator')` from `router/v1/admin/index.js`;
|
|
under `player`, behind `noindex, requireAuth`; under `public`, behind nothing, by design. A module
|
|
adds per-route gates on top of that and never re-implements the tier gate.
|
|
|
|
**`registerExtension(slot, router)`** — the §1.9 case: module routes hanging off a *core* resource.
|
|
Only core may declare a slot; a module may only fill one. Exactly one slot exists in v1:
|
|
|
|
| Slot | Mounted at | Declared by |
|
|
| --- | --- | --- |
|
|
| `admin.users.detail` | `/api/v1/admin/users/:id` | `router/v1/admin/users.router.js` |
|
|
|
|
The router receives `req.params.id` from the parent (`mergeParams: true`). Two modules filling the
|
|
same slot is a collision and is rejected; core's own routes on the resource always win a path
|
|
conflict.
|
|
|
|
**`registerNotificationStreams(streams)`** — §1.8's push catalog.
|
|
An array of `{ id, label, description, personal, requiresLinkedAccount }` appended to core's catalog.
|
|
Ids are namespaced `<moduleId>.<name>` and rejected otherwise, save for the seven grandfathered ones
|
|
in §6.4.
|
|
|
|
Two amendments this signature carries, both settled 2026-08-10 with PR 4:
|
|
|
|
- **`mapEvent` is gone.** The earlier signature took `{ streams, mapEvent }`, with core's dispatcher
|
|
calling `mapEvent(event) => streamId`. That was a leftover from before §1.8's push inversion was
|
|
settled: the module owns `fromShardEvent` outright and calls `ctx.push.publish(streamId, …)` with
|
|
an id it has already resolved, so core never needs a second way to get there. What core wants from
|
|
a module here is the catalog — for the subscribe endpoint, for validating a subscription write, and
|
|
for the personal/linked-account gate. It follows that the public-safety filter (a sensitive event
|
|
kind can never produce a *public* push) is module-internal; that is the right home, because the
|
|
kinds, the streams and the filter are then one file that moves together, rather than a rule in core
|
|
about data only the module defines.
|
|
- **Two booleans, not one `scope`.** The entry shape above is the response body of
|
|
`GET /auth/me/notifications/streams`, which a shipped Android client already reads
|
|
(`NotificationsDto.kt`). `scope` was never the wire shape.
|
|
|
|
**`registerAnnounceLeg({ leg, label, dispatch, classify })`** — §1.8's news dispatcher.
|
|
`leg` is a namespaced id, `label` is what the admin panel shows, `dispatch(post) => Promise<result>`
|
|
delivers, and `classify(result) => { outcome, error }` maps the client's result to
|
|
`done` / `retry` / `terminal`. A leg that throws is caught, classified as a retry, and never blocks
|
|
another leg.
|
|
|
|
`label` is an addition: the panel used to hold a client-side `{ towncrier, discord }` label table,
|
|
which would have left a module's leg rendering as a bare id. It comes from the registration so a
|
|
module needs no client change.
|
|
|
|
**Legs are rows, not columns.** `announce_jobs` carried a `towncrier_*` and a `discord_*` column
|
|
group until PR 4; a module cannot `ALTER` a core table, so a registered leg had nowhere to live. The
|
|
per-leg state moved to `announce_job_legs (job_id, leg, status, attempts, last_error,
|
|
next_attempt_at)` and `leg` is a stored value. The parent `status` rollup is over *all* the job's
|
|
legs — done when every leg delivered, failed when every leg gave up, partial in between; and `done`
|
|
when a job has no legs at all, since nothing is left to deliver.
|
|
|
|
**`onBoot(fn)` / `onShutdown(fn)`** — §2.5.
|
|
|
|
### 2.5 Lifecycle
|
|
|
|
```
|
|
require(module) → register(ctx, api) → [routes mounted, app.js require returns]
|
|
↓ (server.js, after ensureSchema + seed)
|
|
onBoot(ctx) → started
|
|
↓ (SIGINT/SIGTERM)
|
|
onShutdown()
|
|
```
|
|
|
|
`onBoot` is where the eight `server.js` UO call sites go (`MODULE_SYSTEM.md` §1.7): the atlas and
|
|
cliloc refreshes, the market display-name backfill, `uoLinkSocket.start()`, the sidecar probe.
|
|
It runs **after** `ensureSchema()` (so the module's own tables exist) and after `seedDefaults()`,
|
|
and **before** the HTTP listener binds — a module that must not serve traffic before it has warmed
|
|
its cache gets that for free.
|
|
|
|
`onShutdown` runs before anything core owns is closed — the database pool, the push dispatcher and
|
|
the SSE fan-out are all still open, because a module's `onShutdown` is the only chance it gets to
|
|
flush through them. Reverse registration order, with a 5-second budget per module; exceeding it is
|
|
logged and the hook abandoned rather than hanging the process. Abandoned, not cancelled: nothing can
|
|
stop a promise that is still running, but the process is exiting anyway and the alternative is a host
|
|
where `systemctl stop` waits for SIGKILL.
|
|
|
|
**`onBoot` has no budget, deliberately.** Shutdown races the process being killed; boot does not. A
|
|
slow `onBoot` delays the listener binding, which is the guarantee two paragraphs up rather than a
|
|
problem to be timed out, and core's own boot steps are awaited exactly the same way.
|
|
|
|
Both hooks are optional, and both are individually try/caught. An `onBoot` that throws marks that
|
|
module `startup_failed` (§4.4) and the site still comes up — its routes stay mounted but its dispatch
|
|
guard rejects them with 503, because a module that failed to warm up serving half-initialised data is
|
|
worse than a module that says it is down. A module with no `onBoot` at all still reaches `started`:
|
|
having nothing to warm up is not the same as never having started, and the row has to agree with the
|
|
guard about whether the module is serving. A module whose `onBoot` threw gets **no** `onShutdown` — it
|
|
is part-way through a warm-up it never finished, and handing it a half-built world to tear down is
|
|
worse than not closing cleanly.
|
|
|
|
`onBoot` receives the same frozen `ctx` object `register()` was given, not a second one built to look
|
|
like it.
|
|
|
|
**What a boot does to `installed_modules`** (`MODULE_SYSTEM.md` §2.4). The dispatch is the second half
|
|
of a reconcile, and the order of its four steps is the design:
|
|
|
|
1. Clear the last boot's outcomes, so what is on display afterwards is what *this* boot did.
|
|
`disabled` rows are left alone — that is an operator decision, not an outcome.
|
|
2. Write a row for every module found on the volume, with null provenance if it has none. A directory
|
|
placed on the volume by hand is a supported install (§2.5 of the design of record) and without a
|
|
row it could be neither disabled nor reported.
|
|
3. Mark any row whose directory is **not** on the volume `startup_failed` (stage `require`). Step 1
|
|
has just reset it to `enabled`, and a row claiming to be enabled for a module that is not there is
|
|
the one state that is simply untrue. A plain uninstall leaves `disabled`, which step 1 never
|
|
touches, so this catches only a directory deleted by hand.
|
|
4. Write down the outcome each module already carries — disabled by the operator, or failed during
|
|
load or schema replay, both of which happen before the database is reachable — and only then
|
|
dispatch `onBoot`.
|
|
|
|
**The operator's switch wins over everything, including a failure.** A module whose row says
|
|
`disabled` is guarded (§4.5), is not booted, and does **not** have its failure re-recorded:
|
|
overwriting a deliberate `disabled` with an outcome would silently switch it back on at the next
|
|
boot.
|
|
|
|
**A bookkeeping failure is not a boot failure.** Every database write in the reconcile is individually
|
|
caught. A row that will not update is bad — the admin panel shows the wrong thing — but it is
|
|
strictly less bad than a site that will not start, and it must not stop the modules behind it from
|
|
booting.
|
|
|
|
Dispatch and reconcile live in `server/src/modules/lifecycle.js`, not in the loader: `routeManifest.js`
|
|
and `swagger.js` both require `app.js` against a dead pool (§4.1), so the loader may not reach the
|
|
database. The two halves meet at exactly one place — `loader.setState()` — so the in-memory record
|
|
the dispatch guard reads and the row the admin panel reads are moved together and cannot disagree.
|
|
|
|
### 2.6 Schema fragments
|
|
|
|
`schema` is an idempotent `.sql` file replayed by the same `ensureSchema()` that replays core's,
|
|
immediately after it, statement by statement, split the same way. It is subject to the same rules
|
|
core's file already follows: `CREATE TABLE IF NOT EXISTS`, `ALTER TABLE … ADD COLUMN IF NOT EXISTS`,
|
|
no `--` inside a string literal, no `DROP`.
|
|
|
|
"Split the same way" is shared code, not a shared description: `utils/sqlStatements.js` holds the
|
|
splitter and both callers use it. It is its own file rather than an export of `utils/db.js` because
|
|
the loader validates fragments at require time and must not pull the mariadb pool into `app.js`'s
|
|
require chain to do it.
|
|
|
|
**The rules above are enforced at LOAD time, not at replay time** (PR 3). Everything §2.6 states
|
|
about the SQL is knowable by reading the file, so a fragment that breaks a rule costs the module its
|
|
mount entirely (§4.4's left-hand column) rather than mounting and then 503ing with tables half
|
|
created. What is left for the replay is the class of failure only the database can report — an
|
|
unknown column type, a bad foreign key — and those are post-mount and answer 503.
|
|
|
|
**The check is a leading-verb allowlist: `CREATE`, `ALTER`, `INSERT`, `UPDATE`.** Those are the four
|
|
core's own `schema.sql` uses. It is an allowlist rather than the `DROP` denylist this section words
|
|
it as because a fragment is **replayed on every boot**: `TRUNCATE` and `DELETE` would empty a table
|
|
at every restart, `RENAME` would fail at the second one, and `GRANT`/`SET`/`USE` are core's business.
|
|
A denylist only ever bans what somebody thought of. It is a leading-verb check and claims no more:
|
|
`ALTER TABLE x DROP COLUMN y` passes it, and catching that needs a SQL parser — a large dependency
|
|
for a rule whose job is stopping the obvious foot-gun early. A `CREATE TABLE` missing `IF NOT EXISTS`
|
|
is rejected on the same grounds: it succeeds exactly once and fails every boot after, presenting to
|
|
an operator as a module that broke on restart.
|
|
|
|
**The replay is outside `ensureSchema()`'s retry loop.** Core's schema is retried ten times while the
|
|
database comes up; a fragment that throws is one module's failure, not a signal the database is not
|
|
ready, and retrying core's whole schema over one module's bad SQL would turn a 503'd module into a
|
|
two-minute boot. Partial application is accepted rather than compensated for — MariaDB self-commits
|
|
each DDL statement, so no transaction could roll back the tables created before the failing one, and
|
|
the idempotence rule is what makes re-running a corrected fragment safe.
|
|
|
|
**One caller replays nothing, deliberately.** `db/seed.js` (`npm run seed`) calls `ensureSchema()`
|
|
standalone without requiring `app.js`, so no scan has happened and `fragments()`'s §7.6 throw would
|
|
break seeding outright. The replay asks `isLoaded()` and logs the skip. That is the only sanctioned
|
|
use of that predicate: everywhere else, reading the module list before `load()` still throws, because
|
|
a booting server quietly getting no module tables is precisely what §7.6 exists to prevent.
|
|
|
|
**Table names are namespaced and collision-checked.** New tables must be prefixed `<id>_`. The
|
|
loader extracts every `CREATE TABLE IF NOT EXISTS <name>` from the fragment and rejects the module
|
|
if a name collides with a core table or with another module's — a wrong `DROP`-free fragment can
|
|
still silently adopt someone else's table otherwise.
|
|
|
|
**module-uo is grandfathered.** Its 27 tables are named `shard_*` (26) and `uo_link_config` (1), and
|
|
renaming them is a data migration this workstream explicitly does not do (`MODULE_SYSTEM.md` §1.6
|
|
puts the count at 25; the working tree says 27 — see §6.4). They are registered in the loader as an
|
|
explicit legacy allowlist keyed to `id: "uo"`, so the prefix rule holds for every module written
|
|
after this one.
|
|
|
|
`purge` is the destructive counterpart, run **only** by the explicit admin purge action, never by
|
|
uninstall. Required whenever `schema` is present: a module that can create tables and cannot drop
|
|
them leaves an operator with orphaned data and no supported way to remove it.
|
|
|
|
### 2.7 What a module must not do
|
|
|
|
- `require` anything outside its own directory except node built-ins and its own `dependencies`.
|
|
- Mutate `ctx`, `req.user`, or any object core handed it.
|
|
- Register an Express error handler, or any middleware at the app level.
|
|
- Read `process.env` for core configuration. Its own config is a `settings` key or its own table.
|
|
- Call `process.exit`, install signal handlers, or start a listener.
|
|
- Write outside `ctx.paths.moduleRoot` and the upload directory.
|
|
|
|
### 2.8 The OpenAPI fragment
|
|
|
|
Every module that registers routes ships `swagger-fragment.json` in its bundle root. Core merges the
|
|
fragments of started modules into `/api/docs.json`; the full reasoning and the collision rules are
|
|
§6.1a. In short: fully-qualified paths, namespaced schema keys, module CI fails if a registered
|
|
route has no path in the fragment, and core wins every key collision.
|
|
|
|
### 2.9 What core publishes about a module
|
|
|
|
`GET /api/v1/public/modules` — anonymous, database-free, never site-mode gated.
|
|
|
|
```json
|
|
{ "modules": [ { "id": "uo", "name": "Ultima Online", "version": "1.0.0",
|
|
"capabilities": ["shard", "atlas", "market"] } ] }
|
|
```
|
|
|
|
Four fields, in the loader's scan order (§4.2). What is *not* there is the design:
|
|
|
|
- **Only `started` modules appear.** The endpoint answers what this backend is serving. A module that
|
|
is `disabled` or `startup_failed` is **absent**, which is the same answer §4.4 already gives for its
|
|
routes and its nav — a client renders a site without that capability rather than one advertising a
|
|
capability that 503s. `installed` and `registered` are likewise absent: neither is serving yet.
|
|
- **No `state`, no `failure_stage`, no `failure_reason`.** Where a module broke and how far it got is
|
|
operator detail for the admin Modules screen. An anonymous visitor is not told that something is
|
|
broken, and the message — which is an exception string from inside core — never leaves the server.
|
|
- **No `client` chunk URL.** `utils/htmlShell.js` injects a `<script type="module">` per started
|
|
module (§3.1.3), so the browser is handed the tag rather than a URL to fetch. This endpoint is for
|
|
**feature detection**, not for loading. `MODULE_SYSTEM.md` §2.6 step 4 predates that resolution and
|
|
is amended to match (§6.7).
|
|
- **An empty `modules` array is a real answer** — a core with nothing installed. The one thing that is
|
|
not an answer is the §7.6 guard: reading the list before `modules.load()` ran is a **500**, never
|
|
`[]`, because a caller cannot tell an empty list from a mis-ordered boot.
|
|
|
|
`capabilities` are opaque to core: it never interprets one, and two modules may declare the same
|
|
string. A client must treat an unknown capability as absent and must not infer a route from one — the
|
|
mount prefixes are `module.json`'s business (§2.3), not the capability list's.
|
|
|
|
The endpoint owns the `/modules` prefix on the public tier, which is *why* it is a router of its own
|
|
rather than a fifth singleton beside `/settings` and `/version`. The loader's collision probe reads
|
|
the live tier stack and skips root-mounted layers (a `use('/', …)` matches every path), so a route
|
|
declared inside the root-mounted site router would be invisible to it — a real `use('/modules', …)`
|
|
layer is what makes "no module may claim `/modules`" enforced rather than merely intended.
|
|
|
|
---
|
|
|
|
## Part 3 — The client contract
|
|
|
|
### 3.1 How the chunk gets there
|
|
|
|
Exactly as `MODULE_SYSTEM.md` §2.6 resolved, and Phase 1's spike is what proves it:
|
|
|
|
1. Module CI builds `client/dist/entry.js` with Vite in **library mode**, `react`, `react-dom`,
|
|
`react-dom/client` and `react-router-dom` declared **external**.
|
|
2. Core serves the **directory the entry sits in** statically at `/modules/<id>/` — same-origin, so
|
|
`script-src 'self'` (`config/csp.js:49`) admits it with no nonce and no inline.
|
|
3. `utils/htmlShell.js` injects `<script type="module" src="/modules/<id>/entry.js">` **before
|
|
`</body>`**, for each **started** module.
|
|
4. Before that tag, core has published `window.__rg` (§3.2) from its own bundle. The module's
|
|
externals resolve against it.
|
|
5. Core renders **on `DOMContentLoaded`**, which is after every one of those scripts, so the routes a
|
|
module registers are present in the first render.
|
|
|
|
There is exactly one React instance and core owns it. A module that bundles its own React will
|
|
produce two copies of the hook dispatcher and fail at the first `useState`; the externals config in
|
|
§3.5 is what prevents it.
|
|
|
|
Four of those five steps carry a constraint that is easy to get wrong and impossible to notice in a
|
|
unit test. All four are normative.
|
|
|
|
**The static root is the entry's directory, never the module root.** One `express.static` over a
|
|
module root publishes its server source, its `module.json` and its schema fragment. The loader
|
|
therefore **rejects an entry sitting directly in the module root** — an entry must be in a
|
|
subdirectory — rather than leaving the rule to whoever writes the mount. The mount sits behind the
|
|
module's own state guard, so a chunk is `503` while the module is `startup_failed` and `404` while it
|
|
is `disabled`, exactly as its API routes are: the browser must not be running the client half of
|
|
something the server half has stopped serving. Anything else under `/modules` is a `404`, not the SPA
|
|
shell — answering a `<script src>` with an HTML page turns a missing file into a MIME-type refusal
|
|
with a `200` in the network tab. And because a library build emits an **unhashed** `entry.js`, chunks
|
|
are served `Cache-Control: no-cache`: revalidation is what stops an upgraded module serving
|
|
yesterday's code out of the disk cache.
|
|
|
|
**The injection point is `</body>`, and that is a contract, not a formatting choice.** Module scripts
|
|
are deferred and execute in **document order**, so core's bundle — which publishes `window.__rg` —
|
|
has to come first or every import in every module chunk resolves against `undefined`. Injecting into
|
|
`</head>` happens to work today only because Vite hoists core's entry script into `<head>`; that is a
|
|
bundler's emit decision, and if it ever changed, every module in the wild would break with nothing in
|
|
core having been edited. Last in the body is after core's script wherever core's script is.
|
|
|
|
**Core's render waits for `DOMContentLoaded`, and the readyState check is `'complete'`, not
|
|
`'loading'`.** A deferred script runs *after* the document is parsed, so by the time core's bundle
|
|
executes `document.readyState` is already `'interactive'` and `DOMContentLoaded` has not fired yet.
|
|
A `readyState === 'loading'` test therefore mounts immediately, before any module chunk has
|
|
evaluated, and a module's routes are missing from the first render — which is indistinguishable from
|
|
a module that failed to load: its URL falls through to core's catch-all and redirects home. This was
|
|
found by loading a real chunk in a browser, not by a test, and it is why PR 7's verification includes
|
|
one (§7.7).
|
|
|
|
### 3.2 `window.__rg`
|
|
|
|
Populated by core's `main.jsx` **before** it renders, and frozen afterwards.
|
|
|
|
```js
|
|
window.__rg = {
|
|
version: '1.0.0', // MODULE_API_VERSION — the same number as the server's
|
|
react, // the React namespace
|
|
reactDom, // react-dom/client
|
|
router, // react-router-dom namespace
|
|
jsxRuntime, // react/jsx-runtime — see below
|
|
registry, // §3.3
|
|
ui, // §3.4
|
|
api, // §3.5
|
|
}
|
|
```
|
|
|
|
`jsxRuntime` is not decoration. A module's bundler compiles every `.jsx` file to imports from
|
|
`react/jsx-runtime` under the modern automatic runtime, and those have to resolve to *core's* React
|
|
like every other import. Without it on the global a module would have to build with
|
|
`jsxRuntime: 'classic'`; with it, the module uses the default its tooling already assumes.
|
|
|
|
A module entry checks `window.__rg.version` against its own `coreApi` range and refuses to register
|
|
on a mismatch, logging once — the client-side twin of §1.1, and the reason `version` is here at all.
|
|
|
|
### 3.3 `registry` — what the module registers
|
|
|
|
```js
|
|
registry.registerRoutes(id, { public: [...], admin: [...], player: [...] })
|
|
registry.registerNav(id, { area, items })
|
|
registry.registerFeatureProvider(id, namespace, hook)
|
|
```
|
|
|
|
**`registerRoutes`** — arrays of `{ path, element, gate? }`. Paths are relative to the module's
|
|
namespace and core prefixes them (`MODULE_SYSTEM.md` §2.8):
|
|
|
|
| Area | Rendered at | Wrapped in |
|
|
| --- | --- | --- |
|
|
| `public` | `/<id>/<path>` | `MaintenanceGate` |
|
|
| `admin` | `/admin/<id>/<path>` | `RequireAuth` + `AdminLayout` |
|
|
| `player` | `/player/<id>/<path>` | `RequirePlayer` + `PlayerPortalLayout` |
|
|
|
|
`gate` is an optional `{ roles: [...] }`, applied by core as the existing `RoleGate`. A module cannot
|
|
supply its own auth wrapper — that is the one place where the client boundary is load-bearing, since
|
|
the sidebar and the route table must agree about who may see what.
|
|
|
|
`App.jsx` stops being a flat static table and becomes core's routes plus
|
|
`registry.routesFor(area)`. Registration happens at entry-script evaluation, which is before
|
|
`createRoot().render()`, so nothing renders against a half-populated registry.
|
|
|
|
**`registerNav`** — items interleave into *core* groups (`MODULE_SYSTEM.md` §1.4):
|
|
|
|
```js
|
|
registry.registerNav('uo', {
|
|
area: 'admin',
|
|
items: [
|
|
{ label: 'Shard ops', to: '/admin/uo/shard-ops', group: 'Moderation', order: 30,
|
|
roles: ['admin', 'moderator'] },
|
|
{ label: 'Shard', to: '/admin/uo/link', group: 'System', order: 10 },
|
|
],
|
|
})
|
|
```
|
|
|
|
`group` names an existing core group; an unknown group name appends a new group at the end rather
|
|
than dropping the item. `order` sorts within the group, core items keeping their current positions.
|
|
`feature` names a flag resolved by the provider below.
|
|
|
|
Six details settled when this was built (Phase 2 PR 8, `client/src/modules/nav.js`):
|
|
|
|
- **`order` on a flat nav is a position among core's rows**, which are keyed by their index; an
|
|
explicit `order` beats a core row that merely sits at that index. A row with **no** `order`
|
|
appends after the coded rows rather than defaulting to 0 — otherwise "I didn't ask for a
|
|
position" would mean "put me first", which is the one place a module could take over the header
|
|
without asking for anything.
|
|
- **A row with no `group` on the admin sidebar gets a trailing untitled group of its own**, not a
|
|
place in one of core's untitled groups. Those are Dashboard at the top and Account at the bottom;
|
|
a module page belongs beside neither, and core does not invent a display title out of a module id.
|
|
- **A group a module created is itself a legal override destination.** It falls out of building the
|
|
destination set from the merged base nav, and is recorded so it is not mistaken for an accident.
|
|
- **A row whose `to` collides with an existing row is dropped, with a console warning.** `to` is the
|
|
key the override layer stores under and React renders by, so two rows sharing one would give an
|
|
admin a single editor row that silently moves both. Core's row wins, since that is the one any
|
|
stored override was written against.
|
|
- **`feature` is not public-area-only.** An earlier draft of this section said it was, on the
|
|
grounds that core's admin and player navs carry no flags. They still do not — but a module row
|
|
that declares a gate and has it silently ignored is a trap, so the gate is applied in all three
|
|
areas and the field means one thing everywhere.
|
|
- **The interleave happens before the override merge, and that ordering is load-bearing.** The merge
|
|
is keyed by `to` and drops any key its base array does not declare, so module rows appended
|
|
afterwards would be unorderable, unrelabellable and unhideable — a visible regression for every
|
|
operator who has ever edited a nav, the day the UO rows leave core.
|
|
|
|
Moderator confinement, `MOD_PATHS` in `AdminLayout.jsx:109` — a hardcoded allowlist of five paths —
|
|
becomes a computation over each item's `roles`, so moderator visibility follows from the
|
|
registration instead of from a second list that has to be kept in sync. It moved to
|
|
`client/src/lib/adminNav.js`, plain JS so the test runner can reach it, along with the redirect that
|
|
confines a moderator who deep-links. **The redirect derives from the base nav, never the
|
|
override-merged one**: an override is presentation and must not move an authorization boundary in
|
|
either direction — hiding a row must not also bar someone from the page, and un-hiding one must not
|
|
admit them to it.
|
|
|
|
Deriving it changed what a moderator sees, in both cases toward what the server already permitted:
|
|
**Dashboard**, whose `roles` had always named moderator while `MOD_PATHS` omitted it, and **My
|
|
Characters**, which is ungated self-service. It also fixed a defect the two lists had between them —
|
|
`/admin/houses` was on the sidebar and not in the redirect's own third list, so a moderator clicking
|
|
Houses in their own nav was bounced back to Moderation.
|
|
|
|
The pipeline is unchanged from `THEMING_AND_NAV.md`, with one new first step:
|
|
|
|
> registered defaults (core **+ modules**) → admin overrides → role/feature filtering → rendered nav
|
|
|
|
An earlier draft of this line had the last two the other way round. The filter runs **last** and
|
|
that is deliberate — it is what keeps it a boundary an override cannot cross (`THEMING_AND_NAV.md`
|
|
§7), and both layouts have always been written that way.
|
|
|
|
**`registerFeatureProvider`** — core keeps a generic flag context; the module supplies the hook that
|
|
fills its namespace (`useShardFeatures` for `uo`). With no module installed the filter is a correct
|
|
no-op, because no core nav item carries a `feature` today.
|
|
|
|
**The namespace comes from the registration, not from the string.** A row's `feature` is resolved by
|
|
the provider its own module registered, so a module author writes `feature: 'status'` exactly as it
|
|
reads today: nothing parses a prefix, and a typo'd namespace is not a thing that can exist. Core's
|
|
own rows carry no `moduleId` and resolve against the owner id **`core`** — which is what core
|
|
registers `useShardFlags` under (`main.jsx`), the client twin of the server's `registries.registerCore()`.
|
|
So the ten shard-gated rows in the public header already run through the module seam rather than
|
|
beside it, and Phase 3 deletes core's registration instead of rewriting the header.
|
|
|
|
A provider hook returns **a Set-like of the flags this viewer may see, or `null`** while the answer
|
|
is in flight. Every unknown — no provider, a null answer, a provider that returned something without
|
|
a `has`, a malformed row — **shows the link**. This is presentation and the server is the gate, so a
|
|
UI mistake that hides a page from someone entitled to it is worse in every case than one that shows
|
|
a link which then 403s.
|
|
|
|
Core calls every registered provider's hook unconditionally, in a fixed order, at the top of the
|
|
context component. That is legal because the rules of hooks require the same hooks in the same order
|
|
on every render of a component, not a statically known list: registration completes before the first
|
|
render (§3.1), nothing unregisters, and the provider list is snapshotted per component instance
|
|
anyway. `registry.featureProviders()` is a module export and deliberately **not** a member of the
|
|
`registry` object handed to modules — a module asks for a namespace it knows the name of, and has no
|
|
business enumerating what everyone else registered.
|
|
|
|
### 3.4 `ui` — the shared component kit
|
|
|
|
**This is the largest addition Phase 1 makes to the plan, and it is not optional** (§6.2; approved
|
|
2026-08-10). The atlas
|
|
pages alone import five core modules that are not React and not the router: `PublicLayout`,
|
|
`PageHeader`, `Loading` / `ErrorState` / `EmptyState`, and `useAsync`. Without a shared kit a module
|
|
either reaches into core's tree (violating the zero-import rule) or ships its own copies, which
|
|
means a module page that does not look like the site it is installed in — and drifts further every
|
|
time core's layout changes.
|
|
|
|
The kit is **curated and closed**, not a re-export of `components/`:
|
|
|
|
| Export | From | Why it is in the kit |
|
|
| --- | --- | --- |
|
|
| `PublicLayout` | `components/PublicLayout.jsx` | the public chrome; a module page without it is a bare page |
|
|
| `PageHeader` | `components/PageHeader.jsx` | title/subtitle furniture |
|
|
| `Loading`, `ErrorState`, `EmptyState` | `components/PageState.jsx` | the three states every data page has |
|
|
| `useAsync` | `lib/useAsync.js` | the fetch/loading/error hook every data page uses |
|
|
| `useAuth`, `useSite` | `contexts/*` | read-only access to session and site settings |
|
|
|
|
Everything else — tables, chips, tabs, the tiptap editor, dnd-kit — a module bundles itself.
|
|
Adding to the kit is a **minor** `MODULE_API_VERSION` bump; changing a kit component's props is a
|
|
**major** one. That is a real constraint on core and it is the price of the boundary being worth
|
|
anything.
|
|
|
|
The kit is those **seven** members. An earlier draft of this table listed an eighth, `AdminPage`, and
|
|
core has no such component — admin views are plain markup inside `AdminLayout`. It was struck in
|
|
Phase 2 PR 7 rather than satisfied by inventing a core component with no consumer until Phase 3;
|
|
adding it later costs a minor bump, which is the case this versioning exists for.
|
|
|
|
### 3.5 `api` — the request primitive
|
|
|
|
`client/src/api/client.js` is one 518-line object, and it already carries module namespaces:
|
|
`api.atlas` (line 196) and `api.shard` are UO bindings living in core's client. They move out with
|
|
the module.
|
|
|
|
Core exposes the primitive, not the object:
|
|
|
|
```js
|
|
window.__rg.api = { request, ApiError, BASE } // request(path, { method, body, headers, raw })
|
|
```
|
|
|
|
`request` is `client.js`'s existing `req` — same-origin `/api/v1`, `credentials: 'include'`, JSON
|
|
in/out, throwing `ApiError(status, message, body)`. A module builds its own namespace over it and
|
|
owns the paths it calls, which is correct: it owns the routes at the other end.
|
|
|
|
### 3.6 Vite library-mode build
|
|
|
|
The module's `vite.config.js`, and the four externals are the whole contract:
|
|
|
|
```js
|
|
export default defineConfig({
|
|
plugins: [react()],
|
|
build: {
|
|
lib: { entry: 'src/entry.jsx', formats: ['es'], fileName: () => 'entry.js' },
|
|
outDir: 'dist',
|
|
modulePreload: { polyfill: false }, // same reason as core: no inline bootstrap under CSP
|
|
rollupOptions: {
|
|
external: ['react', 'react-dom', 'react-dom/client', 'react-router-dom'],
|
|
output: { paths: { /* rewritten to window.__rg by the shim below */ } },
|
|
},
|
|
},
|
|
})
|
|
```
|
|
|
|
Rollup's `external` alone emits bare `import 'react'` specifiers, which the browser cannot resolve
|
|
without an import map — and CSP forbids the inline `<script type="importmap">` that would provide
|
|
one (`MODULE_SYSTEM.md` §1.14). The module therefore ships a two-line shim module that re-exports
|
|
from the global, and aliases the four externals to it:
|
|
|
|
```js
|
|
// src/shim/react.js
|
|
export default window.__rg.react
|
|
export const { useState, useEffect, useMemo, useCallback, useRef, createElement, Fragment } = window.__rg.react
|
|
```
|
|
|
|
**This is the highest-risk mechanical detail in the whole plan and it is exactly what the Phase 1
|
|
spike exists to prove.** If it does not hold, §2.6 of the design of record is wrong and the client
|
|
half needs rethinking before Phase 2 builds on it.
|
|
|
|
---
|
|
|
|
## Part 4 — The loader's obligations
|
|
|
|
### 4.1 Synchronous, filesystem-sourced
|
|
|
|
`app.js` scans `modules/*/module.json` with `fs.readdirSync` at require time and mounts what it
|
|
finds (`MODULE_SYSTEM.md` §1.12). The database is not consulted. `MODULES_DIR` defaults to
|
|
`<repo>/modules` and is overridable by env for tests and for the Docker volume mount.
|
|
|
|
**The trigger is one explicit call, and there is no lazy self-scan** (§7.6). `app.js` calls
|
|
|
|
```js
|
|
modules.load({ public: publicRouter, admin: adminRouter, player: playerRouter })
|
|
```
|
|
|
|
exactly once, and every accessor throws until it has run rather than answering with an empty list —
|
|
"no modules installed" is a real state, and a caller must not be handed it by accident. Its position
|
|
in `app.js` is load-bearing in both directions: **after** `app.use('/api', apiRouter)`, so every core
|
|
prefix is already on the tier routers when §4.3 asks them what core owns and so first-match-wins
|
|
means a module cannot shadow a core route; **before** the `/api` 404, so a module route reaches its
|
|
handler instead of the catch-all.
|
|
|
|
**Mounting is a second pass** over the modules that survived validation, not part of the scan loop.
|
|
Otherwise the first module's layers sit on the tier router while the second is being validated,
|
|
indistinguishable from core's — the second module would be told it collided with *core*, naming the
|
|
wrong culprit, and the module-versus-module check would be unreachable.
|
|
|
|
### 4.2 Order
|
|
|
|
Alphabetical by `id`, deterministically. There is no dependency resolution between modules (§2.0 of
|
|
the design of record puts it out of scope) and alphabetical order is the honest way to say so — any
|
|
other order would imply a precedence that is not being computed.
|
|
|
|
### 4.3 Validation, in this order
|
|
|
|
1. `module.json` parses; no unknown keys; `id` matches the directory.
|
|
2. `coreApi` satisfied by `MODULE_API_VERSION`.
|
|
3. Declared `mounts` prefixes are well-formed and collide with nothing.
|
|
4. Declared `extensions` slots all exist.
|
|
5. `schema`/`purge` files exist and are readable; table names are namespaced or allowlisted.
|
|
6. `require(server)` succeeds and exports a function.
|
|
7. `register(ctx, api)` returns without throwing, and registers exactly what `module.json` declared.
|
|
|
|
A failure at any step is that module's failure and nobody else's.
|
|
|
|
**Step 3 asks the live tier routers, not a list.** Whether core owns a prefix is answered by probing
|
|
the tier router's own stack with express's `layer.match()`, skipping root-mounted (`fast_slash`)
|
|
layers — `public/index.js` ends with `use('/', siteRouter)` and `admin/index.js` with the dashboard
|
|
router, and both match every path, so counting them would report every prefix as taken and no module
|
|
could ever mount. A hardcoded prefix table was tried in the spike and was already one prefix stale
|
|
when it was written; deriving it means the check cannot drift the first time core adds a capability
|
|
router, and needs no second declaration of the mount table.
|
|
|
|
### 4.4 `startup_failed` is a state, not a crash
|
|
|
|
Per `MODULE_SYSTEM.md` §2.4, the loader try/catches the **entire** lifecycle — require, validation,
|
|
registration, schema replay, `onBoot` — and any failure marks the module `startup_failed` with the
|
|
reason recorded in `installed_modules`, visible in the admin panel, recoverable without shell
|
|
access. The site comes up.
|
|
|
|
Two sub-cases differ, and the difference matters:
|
|
|
|
| Failure before routes are mounted | Failure after (schema, `onBoot`) |
|
|
| --- | --- |
|
|
| routes and nav are simply absent | routes stay mounted; the dispatch guard returns **503** |
|
|
|
|
The second is what keeps the URL surface deterministic and generatable: `routes.manifest.json` must
|
|
not depend on whether a module's boot hook happened to succeed on the machine that generated it.
|
|
|
|
**Every failure is recorded against the step that produced it**, in `failure_stage`, so the admin
|
|
panel can say *where* a module broke and not only what the message was. The stages are §4.3's seven
|
|
validation steps plus `boot`:
|
|
|
|
| Stage | The step that failed |
|
|
| --- | --- |
|
|
| `manifest` | `module.json` unparseable, an unknown key, a bad or mismatched `id`, no `version` |
|
|
| `core_api` | `coreApi` missing, or not satisfied by `MODULE_API_VERSION` |
|
|
| `mounts` | a malformed prefix, or one already owned by core or another module |
|
|
| `extensions` | a declared slot that does not exist |
|
|
| `schema` | a fragment breaking a §2.6 rule at load, or a statement the database rejected at replay |
|
|
| `require` | the entry point threw, or did not export a function — also a row whose directory is gone |
|
|
| `register` | `register()` threw, a claim was malformed, or what it registered ≠ what it declared |
|
|
| `boot` | `onBoot` threw |
|
|
|
|
The four steps that share one function label themselves; the rest are inferred from how far the load
|
|
had got, and an unlabelled throw is recorded against the step that was running rather than guessed
|
|
at. Every non-failing transition clears both the stage and the reason, so a running module can never
|
|
show the failure it had two boots ago.
|
|
|
|
### 4.5 The disabled guard
|
|
|
|
A module disabled in `installed_modules` is *mounted and guarded*, never unmounted — a one-line
|
|
`if (!enabled) return res.status(404)` ahead of its tier mount. Same reason: the URL surface is a
|
|
property of the filesystem, not of a database row.
|
|
|
|
---
|
|
|
|
## Part 5 — Enforcement
|
|
|
|
### 5.1 Zero internal imports (CI, module repo)
|
|
|
|
The acceptance test for the whole contract. In the module's own CI:
|
|
|
|
```
|
|
grep -rE "require\(['\"]\.\./\.\./|from ['\"]\.\./\.\./\.\./" server/ client/src/
|
|
```
|
|
|
|
— refined to "no relative path that escapes the module root", plus a check that the only bare
|
|
specifiers in the client bundle are the four declared externals. A hit fails the build.
|
|
|
|
### 5.2 Zero UO identifiers in core (CI, website repo)
|
|
|
|
Phase 3's acceptance criterion 1: no `shard`, `uoLink`, `cliloc`, `atlas` or `towncrier` outside
|
|
`modules/`, as a CI grep test rather than a review promise.
|
|
|
|
### 5.3 Zero-line route manifest diff (CI, both repos)
|
|
|
|
`npm run routes:manifest -- --check` in core; the module generates and freezes its own manifest in
|
|
its own repo, using the same script pointed at a core+module app. Phase 2 must produce a zero-line
|
|
diff in core's; Phase 3 moves the UO entries out of core's and into module-uo's, which is the one
|
|
diff the whole workstream is allowed.
|
|
|
|
---
|
|
|
|
## Part 6 — Amendments to MODULE_SYSTEM.md
|
|
|
|
Things the survey found that the design of record gets wrong or does not cover, plus what
|
|
implementation has since amended. The first needed a decision and has one.
|
|
|
|
### 6.1 OpenAPI generation does not survive a dynamic loader — **settled: fragment merge**
|
|
|
|
`MODULE_SYSTEM.md` §1.12 treats `scripts/routeManifest.js` and `swagger/swagger.js` as the same
|
|
problem, because both require `app.js` with no database. They are not the same problem.
|
|
|
|
- **`routeManifest.js` walks the live Express stack** (`app._router.stack`, line 175). It is runtime
|
|
introspection and a filesystem-scanning loader is invisible to it in the best way: whatever got
|
|
mounted, it sees.
|
|
- **`swagger/swagger.js` is static analysis.** `swagger-autogen` is handed `routes = ['./src/app.js']`
|
|
(line 29) and *parses the source text*, following `app.use(...)` to the required file. A
|
|
`require(path.join(dir, manifest.server))` inside a `for` loop is not statically resolvable. Module
|
|
routes will be **absent from `swagger-output.json`** — silently, with no error.
|
|
|
|
That collides directly with CLAUDE.md's standing rule: *never ship a route that isn't in the OpenAPI
|
|
spec*. Three ways out:
|
|
|
|
| Option | How | Cost |
|
|
| --- | --- | --- |
|
|
| **A. Fragment merge** ✅ | Module CI runs swagger-autogen against its own `server/index.js` and ships `swagger-fragment.json` in the bundle. Core deep-merges the fragments of started modules into `/api/docs.json` at request time. | one merge helper in core (~40 lines); the module owns its own spec, which matches "one repo, one bundle" |
|
|
| B. Glob the modules dir | Core's `swagger.js` adds `modules/*/server/index.js` to `routes` when present. | core's *committed* spec then depends on which modules the developer had checked out — a spec that differs per machine |
|
|
| C. Hand-write module paths into core's spec | — | a second source of truth; drifts on the first module release |
|
|
|
|
**Decided 2026-08-10: A.** It is the only one that keeps the spec correct on an operator's box,
|
|
where core is a prebuilt image and the module arrived afterwards. The obligation this puts on a
|
|
module is §2.8; the one it puts on core is Phase 2 item 2.
|
|
|
|
### 6.1a The obligation, on each side
|
|
|
|
**Module:** a `swagger-fragment.json` in the bundle root, generated by its own CI with the same
|
|
`swagger-autogen` tooling pointed at its own entry point, carrying only `paths`, `tags` and
|
|
`components.schemas`. Its paths must be fully qualified (`/api/v1/public/atlas/creatures`), because
|
|
the module knows its own mount prefixes and core does not re-derive them. Its `components.schemas`
|
|
keys are namespaced (`UoAtlasCreature`, not `AtlasCreature`) so two modules cannot collide in the
|
|
merged spec. CI fails the module build if a route it registers has no path in its fragment — the
|
|
per-module form of "never ship a route that isn't in the spec".
|
|
|
|
**Core:** `/api/docs.json` merges the fragments of **started** modules over its own committed spec at
|
|
request time (cached, invalidated on a module state change). Merge is shallow-per-section and
|
|
**core always wins a key collision** — a module cannot redefine a core path, tag or schema by
|
|
shipping one with the same name; the collision is logged and the module's version dropped.
|
|
`swagger-output.json` itself stays exactly what core's own routes generate, so `npm run swagger`
|
|
remains reproducible on any machine regardless of what is installed.
|
|
|
|
### 6.2 The client contract is much larger than §2.1 says
|
|
|
|
§2.1 lists three client registration calls and nothing else, implying React and the router are all a
|
|
module needs. The atlas pages disprove it: `Atlas.jsx` imports `PublicLayout`, `PageHeader`,
|
|
`PageState`'s three states, `useAsync` and `api` — five core modules beyond React, on the *smallest*
|
|
UO page. Hence §3.4's curated UI kit and §3.5's request primitive. This is an addition to the
|
|
contract, not a change of direction, but it makes core's component API a versioned surface, which it
|
|
has never been before.
|
|
|
|
### 6.3 `shardVisibility` is module-owned, and core's nav depends on it
|
|
|
|
`utils/shardVisibility.js` is a UO util that provides `requireFeature` and `project` — and the atlas
|
|
routes, the spike target, are gated by it (`atlas.router.js:25`). So the spike carries
|
|
`shardVisibility` plus the `shardVisibility` and `shardLinks` models with it, not just the atlas
|
|
files. Two consequences:
|
|
|
|
- On the **server** this is fine: `requireFeature` becomes module-internal middleware, and only
|
|
`PUBLIC_KINDS` crosses the boundary — already handled by §1.8's `registerNotificationStreams`
|
|
inversion.
|
|
- During the **spike** there will be two copies of `shardVisibility` in one process (the module's,
|
|
and core's for the not-yet-extracted shard routes) with two independent 5-second caches over the
|
|
same table. Functionally identical, and a spike-only artifact that Phase 3 resolves by moving the
|
|
original. Recorded so it is not mistaken for a design flaw.
|
|
|
|
### 6.4 Two counts in §1.6 and §2.7 are off
|
|
|
|
- **27 UO tables, not 25**: 26 `shard_*` plus `uo_link_config`. §1.6 says "the 24 `shard_*` tables
|
|
plus `uo_link_config`".
|
|
- **The atlas spike is 6 routes, not 5**: `/creatures`, `/creatures/:slug`, `/regions`,
|
|
`/landmarks`, `/champions`, `/meta`.
|
|
|
|
Neither changes a decision; both are corrected here rather than left to be tripped over when the
|
|
extraction is counted against the plan.
|
|
|
|
### 6.5 Grandfathered names, and why the prefix rules survive them
|
|
|
|
§2.4 requires a module's stream ids and announce legs to carry its module id. Eight names predate the
|
|
module system and cannot take it:
|
|
|
|
| Kind | Names | Why they cannot be renamed |
|
|
| --- | --- | --- |
|
|
| Streams | `server.status`, `idoc.warning`, `champ.start`, `governor.election`, `vendor.sale`, `house.idoc`, `account.login` | stored in `notification_subs` rows; read by a shipped Android client |
|
|
| Announce leg | `towncrier` | a stored value in `announce_job_legs.leg` and the body of the retry endpoint |
|
|
|
|
They are allowed to **`uo` alone**, by an explicit per-module allowlist — the same shape and the same
|
|
reasoning as the loader's `LEGACY_TABLE_PREFIXES` for module-uo's 27 tables. Grandfathering by
|
|
allowlist rather than dropping the rule is what keeps the rule real for every module written after
|
|
this one; the alternative leaves the first name collision to be discovered by a module silently
|
|
adopting someone else's stream.
|
|
|
|
### 6.6 An extension slot is invisible to static analysis — core needs the merge too
|
|
|
|
§6.1 settled the fragment merge for *modules*. PR 4 found that core needs the identical machinery for
|
|
its **own** slot fills, one phase earlier than the plan expected.
|
|
|
|
A slot's router is created by `registries.declareSlot()` and filled later, so there is no literal
|
|
`use(require(...))` for swagger-autogen to follow. Moving the six `/admin/users/:id/shard/*` routes
|
|
behind `admin.users.detail` therefore deleted 407 lines from `swagger-output.json` — with
|
|
`Swagger-autogen: Success` and no warning. Same failure as §7.4, different cause, and it would have
|
|
shipped six undocumented core routes against CLAUDE.md's standing rule.
|
|
|
|
`npm run swagger` now has a second step (`swagger/slotSpecs.js`): for each **filled** slot, generate a
|
|
fragment by pointing swagger-autogen at that router's own file, re-root its paths at the prefix the
|
|
router actually hangs at, and merge. Two things are derived rather than written down, because a
|
|
written-down copy drifts:
|
|
|
|
- **which** slots — from `registries.filledSlots()`;
|
|
- **where** each hangs — by finding the slot's own router object in the live express stack, decoding
|
|
the mount prefixes above it with `scripts/routeManifest.js`'s own `mountPath`, so the manifest and
|
|
the spec can never disagree about what a mount decodes to.
|
|
|
|
An empty fragment is a hard build failure, because an empty fragment is exactly what the silent drop
|
|
looks like. The merge itself is `swagger/mergeSpec.js` — the ~40-line helper §6.1a already owed core
|
|
for module fragments, written here and proved against core's own slot before a module depends on it.
|
|
This is **build**-time and lands in the committed spec, because slot routes are core's; a module's
|
|
fragment is still merged at **request** time into `/api/docs.json` (§6.1a), and `swagger-output.json`
|
|
stays reproducible on any machine regardless of what is installed.
|
|
|
|
`registerExtension` therefore takes a third, **core-only** argument: the file its router is generated
|
|
from. A module needs no equivalent — it ships a prebuilt `swagger-fragment.json`, because core never
|
|
has its sources to analyse.
|
|
|
|
### 6.7 `/api/v1/public/modules` is not the client's load trigger
|
|
|
|
`MODULE_SYSTEM.md` §2.6 step 4 says "the SPA reads `/api/v1/public/modules` to learn what to load,
|
|
then registers routes, nav and its feature provider". Step 3 of the same list resolved the loading
|
|
question a different way, and §3.1.3 here is the normative version: `htmlShell.js` injects a
|
|
`<script type="module" src="/modules/<id>/entry.js">` per **started** module, so the browser is handed
|
|
the tag by the document and never fetches a URL the endpoint told it about. Nothing waits on an API
|
|
round trip to start loading, which is also why the tag can be in `<head>`.
|
|
|
|
What the endpoint is for is **feature detection**: capabilities for the SPA and the Android app, which
|
|
has no chunk to load at all. §2.9 is the shape. The two statements were only ever in tension because
|
|
§2.6 was written before the CSP constraint forced the injected-tag design; step 4 should read "the SPA
|
|
reads `/api/v1/public/modules` to feature-detect", and the registration it describes happens when the
|
|
injected chunk executes and calls `window.__rg.registry` (§3.3).
|
|
|
|
---
|
|
|
|
## Part 7 — What the spike proved
|
|
|
|
The Phase 1 spike ran on `website` branch `spike/module-atlas`, cut from `edge` and **deliberately
|
|
never merged** — it is the evidence, not the implementation. Phase 2 rebuilds the loader properly.
|
|
|
|
### 7.1 The exit criteria
|
|
|
|
| Criterion | Result |
|
|
| --- | --- |
|
|
| No internal-file imports from the module into core | **pass** — the module's only non-builtin requires are `ctx.express` / `ctx.validator`; the built client chunk contains **zero** bare import specifiers |
|
|
| `npm run routes:manifest` produces a zero-line diff | **pass** — `routes.manifest.json` *and* `routes.guards.json` are byte-identical with the six atlas routes now served by the module |
|
|
| The chunk loads under the enforced CSP | **pass** — `/uo/atlas` and `/uo/atlas/:slug` render from `/modules/uo/entry.js` under `script-src 'self'`, with **zero** violation reports at `/api/csp-report` and a clean console |
|
|
| Everything still passes | **pass** — 729 core tests, 81 module tests |
|
|
|
|
Also verified end to end against the real database: the schema fragment replayed after core's
|
|
(`schema ensured for module "uo"`), `onBoot` ran the atlas refresh, the module reached `started`, and
|
|
the six API URLs answered 200 unchanged at `/api/v1/public/atlas/*`.
|
|
|
|
### 7.2 One express, one React — the same rule, twice
|
|
|
|
The single biggest thing the spike changed. `MODULE_SYSTEM.md` §2.6 got the client half right — one
|
|
React, shared via a global — and said nothing about the server, where the identical problem exists
|
|
and bites harder:
|
|
|
|
- A module lives at `<repo>/modules/<id>/`. Node's resolver walks *up* from there and never reaches
|
|
`server/node_modules`, so `require('express')` inside a module **fails outright**. This was the
|
|
first error the spike hit.
|
|
- Installing express into the module would fix resolution and break something worse: two `Router`
|
|
prototypes, two sets of `instanceof` checks — and it would mean the operator running `npm install`
|
|
in a module directory, which is the build the whole plan exists to avoid.
|
|
|
|
Hence `ctx.express` and `ctx.validator`. The rule generalises: **anything shared between core and a
|
|
module is owned by core and handed over — never resolved by the module.** On the client that is
|
|
`react`, `react-dom/client`, `react-router-dom` and `react/jsx-runtime`; on the server it is
|
|
`express` and `express-validator`.
|
|
|
|
Two mechanical traps inside that, both cheap once known and both silent otherwise:
|
|
|
|
- **Vite's object-form `resolve.alias` does PREFIX matching.** A `react` key also rewrites
|
|
`react/jsx-runtime` into `src/shim/react.js/jsx-runtime`, a path that cannot exist. Use the array
|
|
form with anchored regexes (`/^react$/`).
|
|
- **`external` alone is not enough for an ESM library build.** Rollup then emits bare
|
|
`import 'react'`, which the browser cannot resolve without an import map — and CSP forbids the
|
|
inline `<script type="importmap">` that would supply one. Each shared dependency needs a two-line
|
|
alias shim that re-exports from `window.__rg`. `output.globals` does not help: it applies to
|
|
iife/umd output only.
|
|
|
|
### 7.3 §6.1 confirmed empirically, not just predicted
|
|
|
|
Regenerating the OpenAPI spec after the move deleted **361 lines** — all six atlas paths — from
|
|
`swagger-output.json`, with `Swagger-autogen: Success` and no warning of any kind. The route manifest
|
|
kept all six in the same run. That is the static-analysis-versus-runtime split of §6.1 happening for
|
|
real, and it is exactly the silent failure the fragment merge exists to prevent. Core's committed
|
|
spec is correct as regenerated — it describes core's own routes — and the six paths come back via
|
|
module-uo's fragment when Phase 2 item 2 lands.
|
|
|
|
### 7.4 The loader's failure guarantees are tested, not asserted
|
|
|
|
`server/test/moduleLoader.test.js` — 17 tests over the paths nobody exercises by hand: an entry point
|
|
that throws, a `coreApi` mismatch, an unknown manifest key, an id that disagrees with its directory,
|
|
two modules claiming one prefix, a module claiming a core prefix, registering an undeclared prefix
|
|
and declaring an unregistered one, a fragment naming a core table, an unprefixed table, a schema with
|
|
no purge, an `onBoot` that throws, an `onShutdown` that hangs and one that throws, a double
|
|
registration, and a probe asserting `ctx` exposes exactly the documented surface and is frozen.
|
|
|
|
The property under test throughout is the same: **the failing module fails alone.**
|
|
|
|
### 7.5 Spike artifacts that are NOT design
|
|
|
|
Three things in the branch are consequences of stopping at six routes, and Phase 3 removes all three.
|
|
They are recorded so nobody reads them as intended shape:
|
|
|
|
1. **Core reaches into the module twice** — `admin/shardAtlas.controller.js` and
|
|
`test/atlasController.test.js` require the module's model directly. The five admin atlas routes
|
|
live at `/admin/shard/atlas/*`, inside the `/shard` prefix core still owns, so the module cannot
|
|
take them without colliding or moving a URL. Phase 3 moves the whole `/shard` admin prefix at once
|
|
and the imports go with it.
|
|
2. **Two copies of `shardVisibility`** — the module's (as `utils/visibility.js`) and core's, for the
|
|
shard routes not yet extracted. Two five-second caches over the same table; functionally
|
|
identical. Predicted in §6.3, observed exactly as described.
|
|
3. **The module has no `swagger-fragment.json`** — §6.1a's obligation needs core's merge helper on
|
|
the other side of it, which is Phase 2.
|
|
|
|
### 7.6 A finding for Phase 2's loader — **settled: an explicit `load()`**
|
|
|
|
`scan()` was lazy — requiring the loader did not run it. That was deliberate (app.js decides when
|
|
modules are discovered) but it is a sharp edge: a caller that requires the loader and reads nothing
|
|
gets an empty, *silent* module list. It cost one confusing test failure during the spike.
|
|
|
|
**Phase 2 PR 2 made the trigger explicit** rather than scanning at require time. `app.js` calls
|
|
`modules.load(tierRouters)` once, and `list()` throws until it has. Scanning on require was the
|
|
alternative and was rejected for two reasons: the loader now needs the tier routers *handed to it*
|
|
for the §4.3 collision check, which a require-time side effect cannot receive; and it would make the
|
|
ordering constraint invisible, enforced by where a `require` sits rather than by an argument that is
|
|
missing if it is wrong.
|
|
|
|
### 7.7 The client half has to be verified in a browser — **the timing bug no test could see**
|
|
|
|
Phase 2 PR 7 built the delivery mechanism: the static mount, the injected tag, `window.__rg`, the
|
|
registry, and core's consumption of it. Everything above is unit-tested, and the tests all passed
|
|
against a build that **did not work in a browser**.
|
|
|
|
The smoke that found it is worth repeating whenever this seam changes, and it is four steps: write a
|
|
throwaway `modules/<id>/` with a hand-written ESM `entry.js` — no bundler needed, since
|
|
`window.__rg.react.createElement` is enough to render a page — point `MODULES_DIR` at it, boot the
|
|
server against the built client, and load the module's URL in a real browser with the console open.
|
|
|
|
What it caught was step 5 of §3.1: core mounted before any module chunk had evaluated, because
|
|
`document.readyState` during a deferred script is `'interactive'` and not `'loading'`. The page
|
|
redirected home — the same thing a module that failed to load does — with **no error anywhere**:
|
|
the chunk had fetched, executed, and registered its route into a registry nothing read again. No unit
|
|
test in this repo can see it. There is no DOM in the server or client test runner, and the ordering
|
|
being asserted is the browser's, not the code's.
|
|
|
|
Two smaller things the same run confirmed, both worth keeping in the loop when re-running it: the
|
|
chunk executes under the **enforced** `script-src 'self'` with no CSP report, which is the property
|
|
§3.6 called the highest-risk detail in the plan; and the shell is read **once at boot**
|
|
(`htmlShell.init`), so rebuilding the client without restarting the server serves an `index.html`
|
|
pointing at a hashed bundle that no longer exists — core never runs, `window.__rg` is undefined, and
|
|
the failure looks exactly like a contract violation in the module.
|