Compare commits
22 Commits
b5277d827c
...
docs/modul
| Author | SHA1 | Date | |
|---|---|---|---|
| f3a6231084 | |||
| f402395fa5 | |||
| 7548c20820 | |||
| 749233d378 | |||
| 12e4eaad10 | |||
| 8a7b099c2d | |||
| d66ee832c5 | |||
| f740530eb8 | |||
| b43b40e71b | |||
| 17d8608a7f | |||
| fdede79909 | |||
| 6a39ef63c6 | |||
| e733ac0c9c | |||
| 004e217806 | |||
| 30589f1fa5 | |||
| 58b435be70 | |||
| bcd3a750e7 | |||
| 63e6c2b5d1 | |||
| 510d10b297 | |||
| d7054fd1f2 | |||
| 9355f2aec4 | |||
| bb0e6a02fe |
@@ -105,6 +105,12 @@ server/
|
||||
NOT under /shard: nothing here
|
||||
touches the sidecar, and unlike
|
||||
/shard it IS site-mode gated
|
||||
modules.router.js (1) /public/modules — the installed-module
|
||||
list a client feature-detects against.
|
||||
A real prefix and not a fifth singleton
|
||||
below, so the module loader's
|
||||
collision probe (which skips
|
||||
root-mounted layers) sees it
|
||||
site.router.js (4) /settings /status /version /contact —
|
||||
the group-root singletons; declares no
|
||||
router-level middleware
|
||||
@@ -365,7 +371,7 @@ A DB read never yields a usable reset link. See §4 `/auth/password/*`.
|
||||
| col | type | notes |
|
||||
|---|---|---|
|
||||
| user_id | INT NOT NULL FK→users(id) ON DELETE CASCADE | |
|
||||
| stream_id | VARCHAR(64) NOT NULL | an id from the catalog (`config/notificationStreams.js`), validated on write |
|
||||
| stream_id | VARCHAR(64) NOT NULL | an id from the catalog (`modules/registries.js` — core's plus every installed module's), validated on write |
|
||||
| created_at | DATETIME | |
|
||||
|
||||
`PRIMARY KEY(user_id, stream_id)`. Subscriptions are per-user (applied to every device); a PUT
|
||||
@@ -690,6 +696,14 @@ who fixes the cause needs no admin-panel visit), a running module can never disp
|
||||
and `disabled` — the one operator *decision* rather than outcome — survives untouched. A re-install or
|
||||
upgrade refreshes the metadata and leaves `state` alone.
|
||||
|
||||
The write happens in one place, `src/modules/lifecycle.js`, on the boot path after `ensureSchema()`
|
||||
and before the listener binds: it resets the last boot's outcomes, writes a row for every module found
|
||||
on the volume (with NULL provenance for a hand-placed directory), marks any row whose directory is
|
||||
**gone** `startup_failed`, and then runs each surviving module's `onBoot` and records what happened. A
|
||||
`disabled` row is guarded, not booted, and never has its failure re-recorded — an outcome must not
|
||||
overwrite the operator's decision. Every one of those writes is individually caught: a row that will
|
||||
not update is worse reporting, never a failed boot.
|
||||
|
||||
Design of record: [`MODULE_SYSTEM.md`](MODULE_SYSTEM.md) §2.4; the loader's obligations are
|
||||
[`MODULE_API.md`](MODULE_API.md) Part 4.
|
||||
|
||||
@@ -707,7 +721,7 @@ are authoritative, and they answer different questions:
|
||||
|
||||
| Artifact | Source of truth for | Generated by |
|
||||
|---|---|---|
|
||||
| `server/routes.manifest.json` — mirrored as [api-route-inventory.json](./api-route-inventory.json) | **What URLs exist.** 226 public routes + 2 on the internal listener, sorted, method + path only. | `npm run routes:manifest`, by walking the live Express stack |
|
||||
| `server/routes.manifest.json` — mirrored as [api-route-inventory.json](./api-route-inventory.json) | **What URLs exist.** 228 public routes + 2 on the internal listener, sorted, method + path only. | `npm run routes:manifest`, by walking the live Express stack |
|
||||
| `server/swagger/swagger-output.json` — served at `/api/docs` | **What each route means.** Parameters, bodies, response codes, security. | `npm run swagger`, from `#swagger.*` annotations |
|
||||
|
||||
The split is deliberate: Swagger is annotation-derived, so an unannotated route is invisible in it and
|
||||
@@ -721,9 +735,16 @@ slashes from generated path keys — see *Regenerating the spec* in the website
|
||||
domain split makes that necessary.
|
||||
|
||||
Scope: the manifest keeps `/api/**` and `/.well-known/**` from the public app plus everything on the
|
||||
internal listener. The SPA catch-all, `/uploads` and `/brand` are filesystem-conditional static
|
||||
mounts — not API contract, and including them would make the output depend on whether CI had built
|
||||
the client.
|
||||
internal listener. The SPA catch-all, `/uploads`, `/brand` and `/modules` are filesystem-conditional
|
||||
static mounts — not API contract, and including them would make the output depend on whether CI had
|
||||
built the client, or on which modules happened to be on the volume of the machine that generated it.
|
||||
|
||||
`/modules/<id>/` is the last of those and the newest: an installed module's prebuilt client chunk,
|
||||
served from the directory its `client.entry` sits in and never from the module root, behind the
|
||||
module's own state guard (`503` when it failed to start, `404` when disabled) and with
|
||||
`Cache-Control: no-cache`, because Vite's library build emits an unhashed `entry.js`. Anything else
|
||||
under `/modules` is a `404` rather than the SPA shell. The full contract is
|
||||
[`MODULE_API.md`](MODULE_API.md) §3.1.
|
||||
|
||||
A third generated file, `server/routes.guards.json`, is a **review aid and not a contract**: per route,
|
||||
the middleware handler count plus the *named* middleware on its mount chain. It exists because a
|
||||
@@ -793,14 +814,18 @@ registers device endpoints (`/auth/me/devices`); nothing is pushed unless subscr
|
||||
self-hosted **ntfy** endpoint (`utils/pushDispatch`); the app wakes and pulls the real, ownership-
|
||||
checked content over the authenticated API. Two producers fan out through the one publisher: the shard
|
||||
ingest dispatcher (`utils/shardIngest`, beside the SSE broadcast) for shard-derived streams, and the
|
||||
create/publish-post path for `news.post`. The stream catalog + event→stream mapping is
|
||||
`config/notificationStreams.js`. Security invariants:
|
||||
create/publish-post path for `news.post`. The catalog is assembled at boot by
|
||||
`modules/registries.js` from core's own streams (`config/coreStreams.js` — just `news.post`) plus
|
||||
each installed module's; the shard streams and their event→stream mapping are
|
||||
`config/shardStreams.js`, which belongs to module-uo and moves out with it
|
||||
(MODULE_SYSTEM.md §1.8). Security invariants:
|
||||
- **Same public/admin split as the SSE feed.** Public streams are drawn *only* from the SSE
|
||||
`PUBLIC_KINDS` allowlist; a sensitive kind (audit/cheat/IP/login-attempt) can never produce a public
|
||||
push.
|
||||
- **Personal streams are owner-keyed.** `vendor.sale` / `house.idoc` / `account.login` are delivered
|
||||
only to the *owning* user's devices, resolved via `shardLinks` (the same ownership check as
|
||||
`/player/shard/*`).
|
||||
`/player/shard/*`) in `utils/shardPush.js`. `utils/pushDispatch.js` itself only publishes to a
|
||||
stream id someone else resolved — it has no idea what a shard event is.
|
||||
- **SSRF guard.** A device `endpoint` is a client-supplied URL the server POSTs to, so registration and
|
||||
every publish validate it is HTTPS, non-private/loopback, and (when configured) on the shard's ntfy
|
||||
allow-set (`NTFY_BASE_URL` / `NTFY_ALLOWED_ORIGINS`).
|
||||
@@ -886,6 +911,7 @@ from the per-route **siteMode** middleware (§5), never from an auth gate.
|
||||
| GET | `/settings` | whitelisted public keys, derived `registration`/`gameAccountSignup` flags, the per-shard **`brand`** block (name, `accent` color, logo/hero/favicon) a client themes itself from — one image runs as any shard, asset fields may be site-relative paths (resolve against the base URL); these are **effective** values, so an admin theme (`theme_visual`) beats `BRAND_ACCENT_COLOR` and an uploaded `brand_assets` asset beats its `BRAND_*` path — an optional **`theme`** block, the resolved CSS custom properties for that admin theme (absent when the instance was never themed, which is what makes it render from the shipped stylesheet unchanged) — and a **`push`** block `{ ntfyUrl }` (M7): the client-facing ntfy relay URL the app's embedded distributor registers its device topic against, from `NTFY_PUBLIC_URL` / first `NTFY_ALLOWED_ORIGINS` (never the internal `NTFY_BASE_URL`); `null` when push isn't configured for the shard. |
|
||||
| GET | `/status` | status message + current mode, **plus a `version` block** (`{ service:'runic-gateway', api, server }`) so a client first-run probe recognizes the backend and can run a version-mismatch guard |
|
||||
| GET | `/version` | lightweight, **DB-free** backend identity/version (`{ service, api, server }`) — the canonical target for the version guard and a cheap liveness check |
|
||||
| GET | `/modules` | `{ modules: [{ id, name, version, capabilities }] }` — the modules this backend is currently **serving**, in scan order (module system, `MODULE_API.md` §2.9). A module that is disabled or failed to load is **absent**, not listed with a state: its routes and nav are absent too, so the client renders a site without that capability rather than advertising one that 503s. The recorded failure stage and reason are admin-panel detail and are never published here. `capabilities` are opaque strings the module declares — feature-detect against them and treat an unknown one as absent. Like `/status` and `/version` it is **DB-free and not site-mode gated**, so a client can still feature-detect during maintenance. It is *not* how a module's client chunk loads — `htmlShell` injects a `<script type="module">` per started module. |
|
||||
| GET | `/posts/:category` | published only; `category` ∈ news\|five-on-friday\|newsletter\|screenshots |
|
||||
| GET | `/posts/:category/:idOrSlug` | single published post |
|
||||
| GET | `/wiki` | list of pages (slug + title) |
|
||||
@@ -1130,7 +1156,7 @@ it picks, it fails open on one side.)
|
||||
| Nav | `GET /public/shard/features` returns only what the caller may reach, so the SPA never renders a link that would 403. Presentation only. |
|
||||
|
||||
Config reads are cached ~5s, so admin changes take effect within seconds **including on already-open
|
||||
streams**. `PUBLIC_KINDS` still exists and is still exported (`notificationStreams.js`) but is now
|
||||
streams**. `PUBLIC_KINDS` still exists and is still exported (`utils/shardBroadcast.js`) but is now
|
||||
**derived** from the kind map rather than hand-maintained, so the two cannot drift.
|
||||
|
||||
**`PUBLIC_KINDS` is a module-load constant and must not be used to answer "may this caller read this
|
||||
|
||||
@@ -26,9 +26,20 @@ here extends the contract first, in this file, before the module is written agai
|
||||
Core exports a single integer-major semver string from `server/src/modules/version.js`:
|
||||
|
||||
```js
|
||||
const MODULE_API_VERSION = '1.0.0'
|
||||
const MODULE_API_VERSION = '1.1.0'
|
||||
```
|
||||
|
||||
The client half carries the same number (`client/src/modules/version.js`) and a test asserts the two
|
||||
agree. Duplicated rather than fetched because the value has to be on `window.__rg` before the first
|
||||
module chunk evaluates, which is earlier than any network round trip could answer.
|
||||
|
||||
**1.1.0 — Phase 3 slice 1.** `ctx` gained `activity.log`, `users.getById`, `site.baseUrl`, and
|
||||
`middleware.rateLimit` + `middleware.accountChangeLimiter`; `api` gained `registerPostHook`.
|
||||
Additions only. Each exists because module-uo's extraction needed it and none could be vendored — an
|
||||
admin action a module performs belongs in core's one audit log, the extension slot needs the user its
|
||||
prefix names, §2.7 forbids a module reading core's `APP_BASE_URL`, a second rate-limit store is a
|
||||
limit enforced by two counters, and core's CMS was calling a UO file directly.
|
||||
|
||||
Every `module.json` declares a `coreApi` semver **range**. The loader checks it at boot, before it
|
||||
requires a line of module code, and a mismatch fails that module loudly into `startup_failed`
|
||||
(§4.4) with the two versions in the reason. It never silently proceeds.
|
||||
@@ -85,12 +96,12 @@ rejected rather than ignored, so a typo is a loud failure and not a silently-ine
|
||||
| `version` | yes | Semver. Recorded in `installed_modules`; shown on failure. |
|
||||
| `coreApi` | yes | Semver range checked against `MODULE_API_VERSION` (§1.1). |
|
||||
| `server` | no | Entry point, relative to the module root. Absent ⇒ client-only module. |
|
||||
| `client.entry` | no | Prebuilt ESM chunk, relative to the module root. Absent ⇒ server-only module. |
|
||||
| `client.entry` | no | Prebuilt ESM chunk, relative to the module root, and **in a subdirectory** — the directory it sits in is what gets served (§3.1). Absent ⇒ server-only module; present-but-empty is rejected, since it claims a client half and delivers none. |
|
||||
| `schema` | no | Idempotent SQL fragment (§2.6). |
|
||||
| `purge` | no | Destructive teardown (§2.6). Required if `schema` is present. |
|
||||
| `mounts` | no | Declared prefixes per tier (§2.3). Declaration is the contract; the loader compares it against what the module actually registers and rejects a mismatch. |
|
||||
| `extensions` | no | Core extension slots this module mounts into (§2.4). |
|
||||
| `capabilities` | no | Opaque strings published by `GET /api/v1/public/modules`, for clients (the SPA, the Android app) to feature-detect against. |
|
||||
| `capabilities` | no | Opaque strings published by `GET /api/v1/public/modules` (§2.9), for clients (the SPA, the Android app) to feature-detect against. Published only while the module is `started`. |
|
||||
|
||||
### 2.2 The entry point
|
||||
|
||||
@@ -128,6 +139,11 @@ module-uo does not need is on the list.
|
||||
| `ctx.uploads` | `{ upload, UPLOAD_DIR, MIME_EXT }` | `admin/imageUpload.js` | atlas art import |
|
||||
| `ctx.posts` | `{ listAll, getById, linkAnnounceJob, markAnnounced }` | `model/posts` | `newsGump.js:108`, `announceWorker.js:58` |
|
||||
| `ctx.paths.moduleRoot` | absolute path to `modules/<id>/` | loader | atlas art, cliloc files |
|
||||
| `ctx.activity.log` | `({ req, action, detail }) => Promise<void>` | `model/activity` | every admin UO controller (1.1.0) |
|
||||
| `ctx.users.getById` | `(id) => Promise<user\|null>` | `model/users` | `usersShard.controller` (1.1.0) |
|
||||
| `ctx.site.baseUrl` | getter, string with no trailing slash | `APP_BASE_URL` | `shardAnnounce` (1.1.0) |
|
||||
| `ctx.middleware.rateLimit` | `(options) => middleware` | `middleware/rateLimit` | the market search (1.1.0) |
|
||||
| `ctx.middleware.accountChangeLimiter` | middleware | `middleware/rateLimit` | `player/shard.router` (1.1.0) |
|
||||
| `ctx.moduleId` | the id from `module.json` | loader | log tags, table checks |
|
||||
|
||||
Three narrowings from `MODULE_SYSTEM.md` §2.1, all deliberate:
|
||||
@@ -157,12 +173,22 @@ validated at once rather than at first use.
|
||||
```js
|
||||
api.registerRoutes({ public: {...}, admin: {...}, player: {...} })
|
||||
api.registerExtension(slot, router)
|
||||
api.registerNotificationStreams({ streams, mapEvent })
|
||||
api.registerAnnounceLeg({ leg, dispatch, classify })
|
||||
api.registerNotificationStreams(streams)
|
||||
api.registerAnnounceLeg({ leg, label, dispatch, classify })
|
||||
api.registerPostHook({ onSaved, onDeleted })
|
||||
api.onBoot(async (ctx) => {})
|
||||
api.onShutdown(async () => {})
|
||||
```
|
||||
|
||||
**Every call STAGES; nothing is committed until the module as a whole is known good.** A claim's
|
||||
shape is checked at the call, so a malformed one throws with the registrant's own stack; whether the
|
||||
name is *taken* can only be answered once the batch is complete, and is checked when the loader
|
||||
commits it in its second pass. The consequence is the one that matters: a module that registers two
|
||||
streams and then throws — or fails `checkDeclared` after `register()` returns — has left nothing
|
||||
behind. A half-registered catalog would be worse than a missing one, because it is a subscribable
|
||||
stream nothing will ever publish to. This is the registry-side twin of §4.3's second-pass mount rule,
|
||||
and both exist for the same reason.
|
||||
|
||||
**`registerRoutes(mounts)`** — one `express.Router()` per prefix per tier:
|
||||
|
||||
```js
|
||||
@@ -194,15 +220,60 @@ The router receives `req.params.id` from the parent (`mergeParams: true`). Two m
|
||||
same slot is a collision and is rejected; core's own routes on the resource always win a path
|
||||
conflict.
|
||||
|
||||
**`registerNotificationStreams({ streams, mapEvent })`** — §1.8's push catalog.
|
||||
`streams` is an array of `{ id, label, description, scope }` appended to core's catalog (ids are
|
||||
namespaced `<moduleId>.<name>` and rejected otherwise); `mapEvent(event) => streamId | null` is
|
||||
called by core's dispatcher for events the module's own code publishes.
|
||||
**`registerNotificationStreams(streams)`** — §1.8's push catalog.
|
||||
An array of `{ id, label, description, personal, requiresLinkedAccount }` appended to core's catalog.
|
||||
Ids are namespaced `<moduleId>.<name>` and rejected otherwise, save for the seven grandfathered ones
|
||||
in §6.4.
|
||||
|
||||
**`registerAnnounceLeg({ leg, dispatch, classify })`** — §1.8's news dispatcher.
|
||||
`leg` is a namespaced id, `dispatch(post) => Promise<void>` delivers, `classify(post) => boolean`
|
||||
decides whether this leg wants the post. A leg that throws is retried by core's existing per-leg
|
||||
retry and never blocks another leg.
|
||||
Two amendments this signature carries, both settled 2026-08-10 with PR 4:
|
||||
|
||||
- **`mapEvent` is gone.** The earlier signature took `{ streams, mapEvent }`, with core's dispatcher
|
||||
calling `mapEvent(event) => streamId`. That was a leftover from before §1.8's push inversion was
|
||||
settled: the module owns `fromShardEvent` outright and calls `ctx.push.publish(streamId, …)` with
|
||||
an id it has already resolved, so core never needs a second way to get there. What core wants from
|
||||
a module here is the catalog — for the subscribe endpoint, for validating a subscription write, and
|
||||
for the personal/linked-account gate. It follows that the public-safety filter (a sensitive event
|
||||
kind can never produce a *public* push) is module-internal; that is the right home, because the
|
||||
kinds, the streams and the filter are then one file that moves together, rather than a rule in core
|
||||
about data only the module defines.
|
||||
- **Two booleans, not one `scope`.** The entry shape above is the response body of
|
||||
`GET /auth/me/notifications/streams`, which a shipped Android client already reads
|
||||
(`NotificationsDto.kt`). `scope` was never the wire shape.
|
||||
|
||||
**`registerAnnounceLeg({ leg, label, dispatch, classify })`** — §1.8's news dispatcher.
|
||||
`leg` is a namespaced id, `label` is what the admin panel shows, `dispatch(post) => Promise<result>`
|
||||
delivers, and `classify(result) => { outcome, error }` maps the client's result to
|
||||
`done` / `retry` / `terminal`. A leg that throws is caught, classified as a retry, and never blocks
|
||||
another leg.
|
||||
|
||||
`label` is an addition: the panel used to hold a client-side `{ towncrier, discord }` label table,
|
||||
which would have left a module's leg rendering as a bare id. It comes from the registration so a
|
||||
module needs no client change.
|
||||
|
||||
**Legs are rows, not columns.** `announce_jobs` carried a `towncrier_*` and a `discord_*` column
|
||||
group until PR 4; a module cannot `ALTER` a core table, so a registered leg had nowhere to live. The
|
||||
per-leg state moved to `announce_job_legs (job_id, leg, status, attempts, last_error,
|
||||
next_attempt_at)` and `leg` is a stored value. The parent `status` rollup is over *all* the job's
|
||||
legs — done when every leg delivered, failed when every leg gave up, partial in between; and `done`
|
||||
when a job has no legs at all, since nothing is left to deliver.
|
||||
|
||||
**`registerPostHook({ onSaved, onDeleted })`** — added in API 1.1.0. Core's CMS is the only writer of
|
||||
posts, and a module may need to mirror one somewhere core knows nothing about. `onSaved` receives
|
||||
`{ post, transition }` — the same transition `registerAnnounceLeg` fires on — and `onDeleted`
|
||||
receives `{ post, id }`. Both are optional; a registration with neither is refused, since it is a
|
||||
subscription that can never fire. One hook per registrant.
|
||||
|
||||
Every hook is awaited and none may throw past core: a subscriber's failure is logged and costs
|
||||
neither another subscriber nor the save itself. A sidecar hiccup breaking a post edit would be a
|
||||
worse bug than a stale mirror.
|
||||
|
||||
**It is deliberately not part of `registerAnnounceLeg`**, which fires on the same transition. A leg
|
||||
is a one-shot *delivery* with retry and classification; a post hook maintains idempotent *state*, has
|
||||
to run on delete as well as save, and refreshes silently on an edit. Overloading the leg would have
|
||||
meant a `dispatch` that must not be retried and a `classify` that means nothing.
|
||||
|
||||
Before it existed, core's post controller required `utils/newsGump` directly — core's publish path
|
||||
naming a UO file, and the last thing binding core to the module.
|
||||
|
||||
**`onBoot(fn)` / `onShutdown(fn)`** — §2.5.
|
||||
|
||||
@@ -222,13 +293,59 @@ It runs **after** `ensureSchema()` (so the module's own tables exist) and after
|
||||
and **before** the HTTP listener binds — a module that must not serve traffic before it has warmed
|
||||
its cache gets that for free.
|
||||
|
||||
`onShutdown` runs before the server closes, in reverse registration order, with a 5-second budget
|
||||
per module; exceeding it is logged and skipped rather than hanging the process.
|
||||
`onShutdown` runs before anything core owns is closed — the database pool, the push dispatcher and
|
||||
the SSE fan-out are all still open, because a module's `onShutdown` is the only chance it gets to
|
||||
flush through them. Reverse registration order, with a 5-second budget per module; exceeding it is
|
||||
logged and the hook abandoned rather than hanging the process. Abandoned, not cancelled: nothing can
|
||||
stop a promise that is still running, but the process is exiting anyway and the alternative is a host
|
||||
where `systemctl stop` waits for SIGKILL.
|
||||
|
||||
Both are individually try/caught. An `onBoot` that throws marks that module `startup_failed`
|
||||
(§4.4) and the site still comes up — its routes stay mounted but its dispatch guard rejects them
|
||||
with 503, because a module that failed to warm up serving half-initialised data is worse than a
|
||||
module that says it is down.
|
||||
**`onBoot` has no budget, deliberately.** Shutdown races the process being killed; boot does not. A
|
||||
slow `onBoot` delays the listener binding, which is the guarantee two paragraphs up rather than a
|
||||
problem to be timed out, and core's own boot steps are awaited exactly the same way.
|
||||
|
||||
Both hooks are optional, and both are individually try/caught. An `onBoot` that throws marks that
|
||||
module `startup_failed` (§4.4) and the site still comes up — its routes stay mounted but its dispatch
|
||||
guard rejects them with 503, because a module that failed to warm up serving half-initialised data is
|
||||
worse than a module that says it is down. A module with no `onBoot` at all still reaches `started`:
|
||||
having nothing to warm up is not the same as never having started, and the row has to agree with the
|
||||
guard about whether the module is serving. A module whose `onBoot` threw gets **no** `onShutdown` — it
|
||||
is part-way through a warm-up it never finished, and handing it a half-built world to tear down is
|
||||
worse than not closing cleanly.
|
||||
|
||||
`onBoot` receives the same frozen `ctx` object `register()` was given, not a second one built to look
|
||||
like it.
|
||||
|
||||
**What a boot does to `installed_modules`** (`MODULE_SYSTEM.md` §2.4). The dispatch is the second half
|
||||
of a reconcile, and the order of its four steps is the design:
|
||||
|
||||
1. Clear the last boot's outcomes, so what is on display afterwards is what *this* boot did.
|
||||
`disabled` rows are left alone — that is an operator decision, not an outcome.
|
||||
2. Write a row for every module found on the volume, with null provenance if it has none. A directory
|
||||
placed on the volume by hand is a supported install (§2.5 of the design of record) and without a
|
||||
row it could be neither disabled nor reported.
|
||||
3. Mark any row whose directory is **not** on the volume `startup_failed` (stage `require`). Step 1
|
||||
has just reset it to `enabled`, and a row claiming to be enabled for a module that is not there is
|
||||
the one state that is simply untrue. A plain uninstall leaves `disabled`, which step 1 never
|
||||
touches, so this catches only a directory deleted by hand.
|
||||
4. Write down the outcome each module already carries — disabled by the operator, or failed during
|
||||
load or schema replay, both of which happen before the database is reachable — and only then
|
||||
dispatch `onBoot`.
|
||||
|
||||
**The operator's switch wins over everything, including a failure.** A module whose row says
|
||||
`disabled` is guarded (§4.5), is not booted, and does **not** have its failure re-recorded:
|
||||
overwriting a deliberate `disabled` with an outcome would silently switch it back on at the next
|
||||
boot.
|
||||
|
||||
**A bookkeeping failure is not a boot failure.** Every database write in the reconcile is individually
|
||||
caught. A row that will not update is bad — the admin panel shows the wrong thing — but it is
|
||||
strictly less bad than a site that will not start, and it must not stop the modules behind it from
|
||||
booting.
|
||||
|
||||
Dispatch and reconcile live in `server/src/modules/lifecycle.js`, not in the loader: `routeManifest.js`
|
||||
and `swagger.js` both require `app.js` against a dead pool (§4.1), so the loader may not reach the
|
||||
database. The two halves meet at exactly one place — `loader.setState()` — so the in-memory record
|
||||
the dispatch guard reads and the row the admin panel reads are moved together and cannot disagree.
|
||||
|
||||
### 2.6 Schema fragments
|
||||
|
||||
@@ -237,6 +354,40 @@ immediately after it, statement by statement, split the same way. It is subject
|
||||
core's file already follows: `CREATE TABLE IF NOT EXISTS`, `ALTER TABLE … ADD COLUMN IF NOT EXISTS`,
|
||||
no `--` inside a string literal, no `DROP`.
|
||||
|
||||
"Split the same way" is shared code, not a shared description: `utils/sqlStatements.js` holds the
|
||||
splitter and both callers use it. It is its own file rather than an export of `utils/db.js` because
|
||||
the loader validates fragments at require time and must not pull the mariadb pool into `app.js`'s
|
||||
require chain to do it.
|
||||
|
||||
**The rules above are enforced at LOAD time, not at replay time** (PR 3). Everything §2.6 states
|
||||
about the SQL is knowable by reading the file, so a fragment that breaks a rule costs the module its
|
||||
mount entirely (§4.4's left-hand column) rather than mounting and then 503ing with tables half
|
||||
created. What is left for the replay is the class of failure only the database can report — an
|
||||
unknown column type, a bad foreign key — and those are post-mount and answer 503.
|
||||
|
||||
**The check is a leading-verb allowlist: `CREATE`, `ALTER`, `INSERT`, `UPDATE`.** Those are the four
|
||||
core's own `schema.sql` uses. It is an allowlist rather than the `DROP` denylist this section words
|
||||
it as because a fragment is **replayed on every boot**: `TRUNCATE` and `DELETE` would empty a table
|
||||
at every restart, `RENAME` would fail at the second one, and `GRANT`/`SET`/`USE` are core's business.
|
||||
A denylist only ever bans what somebody thought of. It is a leading-verb check and claims no more:
|
||||
`ALTER TABLE x DROP COLUMN y` passes it, and catching that needs a SQL parser — a large dependency
|
||||
for a rule whose job is stopping the obvious foot-gun early. A `CREATE TABLE` missing `IF NOT EXISTS`
|
||||
is rejected on the same grounds: it succeeds exactly once and fails every boot after, presenting to
|
||||
an operator as a module that broke on restart.
|
||||
|
||||
**The replay is outside `ensureSchema()`'s retry loop.** Core's schema is retried ten times while the
|
||||
database comes up; a fragment that throws is one module's failure, not a signal the database is not
|
||||
ready, and retrying core's whole schema over one module's bad SQL would turn a 503'd module into a
|
||||
two-minute boot. Partial application is accepted rather than compensated for — MariaDB self-commits
|
||||
each DDL statement, so no transaction could roll back the tables created before the failing one, and
|
||||
the idempotence rule is what makes re-running a corrected fragment safe.
|
||||
|
||||
**One caller replays nothing, deliberately.** `db/seed.js` (`npm run seed`) calls `ensureSchema()`
|
||||
standalone without requiring `app.js`, so no scan has happened and `fragments()`'s §7.6 throw would
|
||||
break seeding outright. The replay asks `isLoaded()` and logs the skip. That is the only sanctioned
|
||||
use of that predicate: everywhere else, reading the module list before `load()` still throws, because
|
||||
a booting server quietly getting no module tables is precisely what §7.6 exists to prevent.
|
||||
|
||||
**Table names are namespaced and collision-checked.** New tables must be prefixed `<id>_`. The
|
||||
loader extracts every `CREATE TABLE IF NOT EXISTS <name>` from the fragment and rejects the module
|
||||
if a name collides with a core table or with another module's — a wrong `DROP`-free fragment can
|
||||
@@ -268,6 +419,42 @@ fragments of started modules into `/api/docs.json`; the full reasoning and the c
|
||||
§6.1a. In short: fully-qualified paths, namespaced schema keys, module CI fails if a registered
|
||||
route has no path in the fragment, and core wins every key collision.
|
||||
|
||||
### 2.9 What core publishes about a module
|
||||
|
||||
`GET /api/v1/public/modules` — anonymous, database-free, never site-mode gated.
|
||||
|
||||
```json
|
||||
{ "modules": [ { "id": "uo", "name": "Ultima Online", "version": "1.0.0",
|
||||
"capabilities": ["shard", "atlas", "market"] } ] }
|
||||
```
|
||||
|
||||
Four fields, in the loader's scan order (§4.2). What is *not* there is the design:
|
||||
|
||||
- **Only `started` modules appear.** The endpoint answers what this backend is serving. A module that
|
||||
is `disabled` or `startup_failed` is **absent**, which is the same answer §4.4 already gives for its
|
||||
routes and its nav — a client renders a site without that capability rather than one advertising a
|
||||
capability that 503s. `installed` and `registered` are likewise absent: neither is serving yet.
|
||||
- **No `state`, no `failure_stage`, no `failure_reason`.** Where a module broke and how far it got is
|
||||
operator detail for the admin Modules screen. An anonymous visitor is not told that something is
|
||||
broken, and the message — which is an exception string from inside core — never leaves the server.
|
||||
- **No `client` chunk URL.** `utils/htmlShell.js` injects a `<script type="module">` per started
|
||||
module (§3.1.3), so the browser is handed the tag rather than a URL to fetch. This endpoint is for
|
||||
**feature detection**, not for loading. `MODULE_SYSTEM.md` §2.6 step 4 predates that resolution and
|
||||
is amended to match (§6.7).
|
||||
- **An empty `modules` array is a real answer** — a core with nothing installed. The one thing that is
|
||||
not an answer is the §7.6 guard: reading the list before `modules.load()` ran is a **500**, never
|
||||
`[]`, because a caller cannot tell an empty list from a mis-ordered boot.
|
||||
|
||||
`capabilities` are opaque to core: it never interprets one, and two modules may declare the same
|
||||
string. A client must treat an unknown capability as absent and must not infer a route from one — the
|
||||
mount prefixes are `module.json`'s business (§2.3), not the capability list's.
|
||||
|
||||
The endpoint owns the `/modules` prefix on the public tier, which is *why* it is a router of its own
|
||||
rather than a fifth singleton beside `/settings` and `/version`. The loader's collision probe reads
|
||||
the live tier stack and skips root-mounted layers (a `use('/', …)` matches every path), so a route
|
||||
declared inside the root-mounted site router would be invisible to it — a real `use('/modules', …)`
|
||||
layer is what makes "no module may claim `/modules`" enforced rather than merely intended.
|
||||
|
||||
---
|
||||
|
||||
## Part 3 — The client contract
|
||||
@@ -278,17 +465,50 @@ Exactly as `MODULE_SYSTEM.md` §2.6 resolved, and Phase 1's spike is what proves
|
||||
|
||||
1. Module CI builds `client/dist/entry.js` with Vite in **library mode**, `react`, `react-dom`,
|
||||
`react-dom/client` and `react-router-dom` declared **external**.
|
||||
2. Core serves the module directory statically at `/modules/<id>/` — same-origin, so
|
||||
2. Core serves the **directory the entry sits in** statically at `/modules/<id>/` — same-origin, so
|
||||
`script-src 'self'` (`config/csp.js:49`) admits it with no nonce and no inline.
|
||||
3. `utils/htmlShell.js` injects `<script type="module" src="/modules/<id>/entry.js">` at the
|
||||
`</head>` rewrite it already performs (line 111), for each **started** module.
|
||||
3. `utils/htmlShell.js` injects `<script type="module" src="/modules/<id>/entry.js">` **before
|
||||
`</body>`**, for each **started** module.
|
||||
4. Before that tag, core has published `window.__rg` (§3.2) from its own bundle. The module's
|
||||
externals resolve against it.
|
||||
5. Core renders **on `DOMContentLoaded`**, which is after every one of those scripts, so the routes a
|
||||
module registers are present in the first render.
|
||||
|
||||
There is exactly one React instance and core owns it. A module that bundles its own React will
|
||||
produce two copies of the hook dispatcher and fail at the first `useState`; the externals config in
|
||||
§3.5 is what prevents it.
|
||||
|
||||
Four of those five steps carry a constraint that is easy to get wrong and impossible to notice in a
|
||||
unit test. All four are normative.
|
||||
|
||||
**The static root is the entry's directory, never the module root.** One `express.static` over a
|
||||
module root publishes its server source, its `module.json` and its schema fragment. The loader
|
||||
therefore **rejects an entry sitting directly in the module root** — an entry must be in a
|
||||
subdirectory — rather than leaving the rule to whoever writes the mount. The mount sits behind the
|
||||
module's own state guard, so a chunk is `503` while the module is `startup_failed` and `404` while it
|
||||
is `disabled`, exactly as its API routes are: the browser must not be running the client half of
|
||||
something the server half has stopped serving. Anything else under `/modules` is a `404`, not the SPA
|
||||
shell — answering a `<script src>` with an HTML page turns a missing file into a MIME-type refusal
|
||||
with a `200` in the network tab. And because a library build emits an **unhashed** `entry.js`, chunks
|
||||
are served `Cache-Control: no-cache`: revalidation is what stops an upgraded module serving
|
||||
yesterday's code out of the disk cache.
|
||||
|
||||
**The injection point is `</body>`, and that is a contract, not a formatting choice.** Module scripts
|
||||
are deferred and execute in **document order**, so core's bundle — which publishes `window.__rg` —
|
||||
has to come first or every import in every module chunk resolves against `undefined`. Injecting into
|
||||
`</head>` happens to work today only because Vite hoists core's entry script into `<head>`; that is a
|
||||
bundler's emit decision, and if it ever changed, every module in the wild would break with nothing in
|
||||
core having been edited. Last in the body is after core's script wherever core's script is.
|
||||
|
||||
**Core's render waits for `DOMContentLoaded`, and the readyState check is `'complete'`, not
|
||||
`'loading'`.** A deferred script runs *after* the document is parsed, so by the time core's bundle
|
||||
executes `document.readyState` is already `'interactive'` and `DOMContentLoaded` has not fired yet.
|
||||
A `readyState === 'loading'` test therefore mounts immediately, before any module chunk has
|
||||
evaluated, and a module's routes are missing from the first render — which is indistinguishable from
|
||||
a module that failed to load: its URL falls through to core's catch-all and redirects home. This was
|
||||
found by loading a real chunk in a browser, not by a test, and it is why PR 7's verification includes
|
||||
one (§7.7).
|
||||
|
||||
### 3.2 `window.__rg`
|
||||
|
||||
Populated by core's `main.jsx` **before** it renders, and frozen afterwards.
|
||||
@@ -354,20 +574,82 @@ registry.registerNav('uo', {
|
||||
|
||||
`group` names an existing core group; an unknown group name appends a new group at the end rather
|
||||
than dropping the item. `order` sorts within the group, core items keeping their current positions.
|
||||
`feature` (public area only) names a flag resolved by §3.3's provider.
|
||||
`feature` names a flag resolved by the provider below.
|
||||
|
||||
`MOD_PATHS` in `AdminLayout.jsx:109` — today a hardcoded allowlist of two UO paths — becomes a
|
||||
computation over each item's `roles`, so moderator visibility follows from the registration instead
|
||||
of from a second list that has to be kept in sync.
|
||||
Six details settled when this was built (Phase 2 PR 8, `client/src/modules/nav.js`):
|
||||
|
||||
- **`order` on a flat nav is a position among core's rows**, which are keyed by their index; an
|
||||
explicit `order` beats a core row that merely sits at that index. A row with **no** `order`
|
||||
appends after the coded rows rather than defaulting to 0 — otherwise "I didn't ask for a
|
||||
position" would mean "put me first", which is the one place a module could take over the header
|
||||
without asking for anything.
|
||||
- **A row with no `group` on the admin sidebar gets a trailing untitled group of its own**, not a
|
||||
place in one of core's untitled groups. Those are Dashboard at the top and Account at the bottom;
|
||||
a module page belongs beside neither, and core does not invent a display title out of a module id.
|
||||
- **A group a module created is itself a legal override destination.** It falls out of building the
|
||||
destination set from the merged base nav, and is recorded so it is not mistaken for an accident.
|
||||
- **A row whose `to` collides with an existing row is dropped, with a console warning.** `to` is the
|
||||
key the override layer stores under and React renders by, so two rows sharing one would give an
|
||||
admin a single editor row that silently moves both. Core's row wins, since that is the one any
|
||||
stored override was written against.
|
||||
- **`feature` is not public-area-only.** An earlier draft of this section said it was, on the
|
||||
grounds that core's admin and player navs carry no flags. They still do not — but a module row
|
||||
that declares a gate and has it silently ignored is a trap, so the gate is applied in all three
|
||||
areas and the field means one thing everywhere.
|
||||
- **The interleave happens before the override merge, and that ordering is load-bearing.** The merge
|
||||
is keyed by `to` and drops any key its base array does not declare, so module rows appended
|
||||
afterwards would be unorderable, unrelabellable and unhideable — a visible regression for every
|
||||
operator who has ever edited a nav, the day the UO rows leave core.
|
||||
|
||||
Moderator confinement, `MOD_PATHS` in `AdminLayout.jsx:109` — a hardcoded allowlist of five paths —
|
||||
becomes a computation over each item's `roles`, so moderator visibility follows from the
|
||||
registration instead of from a second list that has to be kept in sync. It moved to
|
||||
`client/src/lib/adminNav.js`, plain JS so the test runner can reach it, along with the redirect that
|
||||
confines a moderator who deep-links. **The redirect derives from the base nav, never the
|
||||
override-merged one**: an override is presentation and must not move an authorization boundary in
|
||||
either direction — hiding a row must not also bar someone from the page, and un-hiding one must not
|
||||
admit them to it.
|
||||
|
||||
Deriving it changed what a moderator sees, in both cases toward what the server already permitted:
|
||||
**Dashboard**, whose `roles` had always named moderator while `MOD_PATHS` omitted it, and **My
|
||||
Characters**, which is ungated self-service. It also fixed a defect the two lists had between them —
|
||||
`/admin/houses` was on the sidebar and not in the redirect's own third list, so a moderator clicking
|
||||
Houses in their own nav was bounced back to Moderation.
|
||||
|
||||
The pipeline is unchanged from `THEMING_AND_NAV.md`, with one new first step:
|
||||
|
||||
> registered defaults (core **+ modules**) → role/feature filtering → admin overrides → rendered nav
|
||||
> registered defaults (core **+ modules**) → admin overrides → role/feature filtering → rendered nav
|
||||
|
||||
An earlier draft of this line had the last two the other way round. The filter runs **last** and
|
||||
that is deliberate — it is what keeps it a boundary an override cannot cross (`THEMING_AND_NAV.md`
|
||||
§7), and both layouts have always been written that way.
|
||||
|
||||
**`registerFeatureProvider`** — core keeps a generic flag context; the module supplies the hook that
|
||||
fills its namespace (`useShardFeatures` for `uo`). With no module installed the filter is a correct
|
||||
no-op, because no core nav item carries a `feature` today.
|
||||
|
||||
**The namespace comes from the registration, not from the string.** A row's `feature` is resolved by
|
||||
the provider its own module registered, so a module author writes `feature: 'status'` exactly as it
|
||||
reads today: nothing parses a prefix, and a typo'd namespace is not a thing that can exist. Core's
|
||||
own rows carry no `moduleId` and resolve against the owner id **`core`** — which is what core
|
||||
registers `useShardFlags` under (`main.jsx`), the client twin of the server's `registries.registerCore()`.
|
||||
So the ten shard-gated rows in the public header already run through the module seam rather than
|
||||
beside it, and Phase 3 deletes core's registration instead of rewriting the header.
|
||||
|
||||
A provider hook returns **a Set-like of the flags this viewer may see, or `null`** while the answer
|
||||
is in flight. Every unknown — no provider, a null answer, a provider that returned something without
|
||||
a `has`, a malformed row — **shows the link**. This is presentation and the server is the gate, so a
|
||||
UI mistake that hides a page from someone entitled to it is worse in every case than one that shows
|
||||
a link which then 403s.
|
||||
|
||||
Core calls every registered provider's hook unconditionally, in a fixed order, at the top of the
|
||||
context component. That is legal because the rules of hooks require the same hooks in the same order
|
||||
on every render of a component, not a statically known list: registration completes before the first
|
||||
render (§3.1), nothing unregisters, and the provider list is snapshotted per component instance
|
||||
anyway. `registry.featureProviders()` is a module export and deliberately **not** a member of the
|
||||
`registry` object handed to modules — a module asks for a namespace it knows the name of, and has no
|
||||
business enumerating what everyone else registered.
|
||||
|
||||
### 3.4 `ui` — the shared component kit
|
||||
|
||||
**This is the largest addition Phase 1 makes to the plan, and it is not optional** (§6.2; approved
|
||||
@@ -383,7 +665,6 @@ The kit is **curated and closed**, not a re-export of `components/`:
|
||||
| Export | From | Why it is in the kit |
|
||||
| --- | --- | --- |
|
||||
| `PublicLayout` | `components/PublicLayout.jsx` | the public chrome; a module page without it is a bare page |
|
||||
| `AdminPage` | `routes/admin/…` | the admin content frame |
|
||||
| `PageHeader` | `components/PageHeader.jsx` | title/subtitle furniture |
|
||||
| `Loading`, `ErrorState`, `EmptyState` | `components/PageState.jsx` | the three states every data page has |
|
||||
| `useAsync` | `lib/useAsync.js` | the fetch/loading/error hook every data page uses |
|
||||
@@ -394,6 +675,11 @@ Adding to the kit is a **minor** `MODULE_API_VERSION` bump; changing a kit compo
|
||||
**major** one. That is a real constraint on core and it is the price of the boundary being worth
|
||||
anything.
|
||||
|
||||
The kit is those **seven** members. An earlier draft of this table listed an eighth, `AdminPage`, and
|
||||
core has no such component — admin views are plain markup inside `AdminLayout`. It was struck in
|
||||
Phase 2 PR 7 rather than satisfied by inventing a core component with no consumer until Phase 3;
|
||||
adding it later costs a minor bump, which is the case this versioning exists for.
|
||||
|
||||
### 3.5 `api` — the request primitive
|
||||
|
||||
`client/src/api/client.js` is one 518-line object, and it already carries module namespaces:
|
||||
@@ -412,27 +698,33 @@ owns the paths it calls, which is correct: it owns the routes at the other end.
|
||||
|
||||
### 3.6 Vite library-mode build
|
||||
|
||||
The module's `vite.config.js`, and the four externals are the whole contract:
|
||||
The module's `vite.config.js`, and the shared-dependency aliases are the whole contract:
|
||||
|
||||
```js
|
||||
export default defineConfig({
|
||||
plugins: [react()],
|
||||
plugins: [react(), assertSharedNotBundled()],
|
||||
resolve: {
|
||||
// ARRAY form with ANCHORED regexes. The object form does PREFIX matching, so
|
||||
// a `react` key silently also rewrites `react/jsx-runtime`.
|
||||
alias: [
|
||||
{ find: /^react$/, replacement: shim('react') },
|
||||
{ find: /^react\/jsx-runtime$/, replacement: shim('jsx-runtime') },
|
||||
{ find: /^react\/jsx-dev-runtime$/, replacement: shim('jsx-runtime') },
|
||||
{ find: /^react-dom$/, replacement: shim('react-dom') },
|
||||
{ find: /^react-dom\/client$/, replacement: shim('react-dom') },
|
||||
{ find: /^react-router-dom$/, replacement: shim('react-router-dom') },
|
||||
],
|
||||
},
|
||||
build: {
|
||||
lib: { entry: 'src/entry.jsx', formats: ['es'], fileName: () => 'entry.js' },
|
||||
outDir: 'dist',
|
||||
modulePreload: { polyfill: false }, // same reason as core: no inline bootstrap under CSP
|
||||
rollupOptions: {
|
||||
external: ['react', 'react-dom', 'react-dom/client', 'react-router-dom'],
|
||||
output: { paths: { /* rewritten to window.__rg by the shim below */ } },
|
||||
},
|
||||
rollupOptions: { external: [] }, // deliberately empty — see below
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
Rollup's `external` alone emits bare `import 'react'` specifiers, which the browser cannot resolve
|
||||
without an import map — and CSP forbids the inline `<script type="importmap">` that would provide
|
||||
one (`MODULE_SYSTEM.md` §1.14). The module therefore ships a two-line shim module that re-exports
|
||||
from the global, and aliases the four externals to it:
|
||||
Each aliased specifier resolves to a two-line shim that re-exports from the global:
|
||||
|
||||
```js
|
||||
// src/shim/react.js
|
||||
@@ -440,9 +732,35 @@ export default window.__rg.react
|
||||
export const { useState, useEffect, useMemo, useCallback, useRef, createElement, Fragment } = window.__rg.react
|
||||
```
|
||||
|
||||
**This is the highest-risk mechanical detail in the whole plan and it is exactly what the Phase 1
|
||||
spike exists to prove.** If it does not hold, §2.6 of the design of record is wrong and the client
|
||||
half needs rethinking before Phase 2 builds on it.
|
||||
**Corrected 2026-08-11, Phase 3 slice 0: `external` and the aliases do not compose, and this section
|
||||
used to show both.** Rollup asks `external` *before* Vite's alias resolver runs, so a specifier
|
||||
listed there is marked external and never aliased. The chunk then emits bare `import 'react'`
|
||||
specifiers, which the browser cannot resolve without an import map — and CSP forbids the inline
|
||||
`<script type="importmap">` that would provide one (`MODULE_SYSTEM.md` §1.14). Slice 0 shipped with
|
||||
both, built cleanly, and emitted exactly that chunk. So: **alias only, and `external` empty.**
|
||||
|
||||
What `external` was there to guard is real — an alias that misses means a second React welded into
|
||||
the chunk, which loads fine and then throws about an invalid hook call somewhere unrelated. That is
|
||||
guarded instead by a **resolution-time plugin that fails the build if a shared dependency resolves
|
||||
into `node_modules`**. Two things about it are contract, because both were wrong first:
|
||||
|
||||
- **It hooks `transform`, not `load`.** `load` is first-wins, so an earlier plugin returning the
|
||||
module's contents means the guard is never called for it. Written against `load` it sat in the
|
||||
build doing nothing, and a deliberately-broken alias produced a 24 kB chunk with react-router
|
||||
bundled and a green build.
|
||||
- **Its forbidden-package list is stated, not derived from the alias list.** Deriving it "so the two
|
||||
cannot disagree" means deleting an alias also deletes the guard against what that alias prevented
|
||||
— which is exactly when it is needed. What may not be bundled is a fact about `window.__rg`; a
|
||||
test asserts the aliases stay inside it.
|
||||
|
||||
The module's own boundary checks are `scripts/checkImports.js` (§5.1) and `scripts/checkExternals.js`,
|
||||
which asks the **built chunk** whether any bare specifier survived. That question cannot be asked of
|
||||
source: `import { useState } from 'react'` is correct in every file, and which React it becomes is
|
||||
decided here.
|
||||
|
||||
**This is the highest-risk mechanical detail in the whole plan.** The Phase 1 spike proved the
|
||||
approach; slice 0 proved the configuration, in a browser, under the enforced `script-src 'self'`,
|
||||
by checking each imported binding is **identity-equal** to the one core published.
|
||||
|
||||
---
|
||||
|
||||
@@ -452,7 +770,31 @@ half needs rethinking before Phase 2 builds on it.
|
||||
|
||||
`app.js` scans `modules/*/module.json` with `fs.readdirSync` at require time and mounts what it
|
||||
finds (`MODULE_SYSTEM.md` §1.12). The database is not consulted. `MODULES_DIR` defaults to
|
||||
`<repo>/modules` and is overridable by env for tests and for the Docker volume mount.
|
||||
`<repo>/modules` and is overridable by env for tests and for the Docker mount. Under Compose it is
|
||||
set to `/app/modules`, where `./modules` is bind-mounted read-write (`MODULE_SYSTEM.md` §2.5); the
|
||||
image itself carries that directory empty and owned by the container user, and `.dockerignore`
|
||||
excludes any local one so a module can never be baked in.
|
||||
|
||||
**A missing modules directory is not an error.** The scan catches and returns, because "no modules
|
||||
installed" is the normal state of bare core and the loader must not make the mount mandatory to boot.
|
||||
|
||||
**The trigger is one explicit call, and there is no lazy self-scan** (§7.6). `app.js` calls
|
||||
|
||||
```js
|
||||
modules.load({ public: publicRouter, admin: adminRouter, player: playerRouter })
|
||||
```
|
||||
|
||||
exactly once, and every accessor throws until it has run rather than answering with an empty list —
|
||||
"no modules installed" is a real state, and a caller must not be handed it by accident. Its position
|
||||
in `app.js` is load-bearing in both directions: **after** `app.use('/api', apiRouter)`, so every core
|
||||
prefix is already on the tier routers when §4.3 asks them what core owns and so first-match-wins
|
||||
means a module cannot shadow a core route; **before** the `/api` 404, so a module route reaches its
|
||||
handler instead of the catch-all.
|
||||
|
||||
**Mounting is a second pass** over the modules that survived validation, not part of the scan loop.
|
||||
Otherwise the first module's layers sit on the tier router while the second is being validated,
|
||||
indistinguishable from core's — the second module would be told it collided with *core*, naming the
|
||||
wrong culprit, and the module-versus-module check would be unreachable.
|
||||
|
||||
### 4.2 Order
|
||||
|
||||
@@ -472,6 +814,14 @@ other order would imply a precedence that is not being computed.
|
||||
|
||||
A failure at any step is that module's failure and nobody else's.
|
||||
|
||||
**Step 3 asks the live tier routers, not a list.** Whether core owns a prefix is answered by probing
|
||||
the tier router's own stack with express's `layer.match()`, skipping root-mounted (`fast_slash`)
|
||||
layers — `public/index.js` ends with `use('/', siteRouter)` and `admin/index.js` with the dashboard
|
||||
router, and both match every path, so counting them would report every prefix as taken and no module
|
||||
could ever mount. A hardcoded prefix table was tried in the spike and was already one prefix stale
|
||||
when it was written; deriving it means the check cannot drift the first time core adds a capability
|
||||
router, and needs no second declaration of the mount table.
|
||||
|
||||
### 4.4 `startup_failed` is a state, not a crash
|
||||
|
||||
Per `MODULE_SYSTEM.md` §2.4, the loader try/catches the **entire** lifecycle — require, validation,
|
||||
@@ -488,6 +838,26 @@ Two sub-cases differ, and the difference matters:
|
||||
The second is what keeps the URL surface deterministic and generatable: `routes.manifest.json` must
|
||||
not depend on whether a module's boot hook happened to succeed on the machine that generated it.
|
||||
|
||||
**Every failure is recorded against the step that produced it**, in `failure_stage`, so the admin
|
||||
panel can say *where* a module broke and not only what the message was. The stages are §4.3's seven
|
||||
validation steps plus `boot`:
|
||||
|
||||
| Stage | The step that failed |
|
||||
| --- | --- |
|
||||
| `manifest` | `module.json` unparseable, an unknown key, a bad or mismatched `id`, no `version` |
|
||||
| `core_api` | `coreApi` missing, or not satisfied by `MODULE_API_VERSION` |
|
||||
| `mounts` | a malformed prefix, or one already owned by core or another module |
|
||||
| `extensions` | a declared slot that does not exist |
|
||||
| `schema` | a fragment breaking a §2.6 rule at load, or a statement the database rejected at replay |
|
||||
| `require` | the entry point threw, or did not export a function — also a row whose directory is gone |
|
||||
| `register` | `register()` threw, a claim was malformed, or what it registered ≠ what it declared |
|
||||
| `boot` | `onBoot` threw |
|
||||
|
||||
The four steps that share one function label themselves; the rest are inferred from how far the load
|
||||
had got, and an unlabelled throw is recorded against the step that was running rather than guessed
|
||||
at. Every non-failing transition clears both the stage and the reason, so a running module can never
|
||||
show the failure it had two boots ago.
|
||||
|
||||
### 4.5 The disabled guard
|
||||
|
||||
A module disabled in `installed_modules` is *mounted and guarded*, never unmounted — a one-line
|
||||
@@ -514,6 +884,25 @@ specifiers in the client bundle are the four declared externals. A hit fails the
|
||||
Phase 3's acceptance criterion 1: no `shard`, `uoLink`, `cliloc`, `atlas` or `towncrier` outside
|
||||
`modules/`, as a CI grep test rather than a review promise.
|
||||
|
||||
**Settled 2026-08-11: the grep reads code, not prose.** It covers four things, and each of them is a
|
||||
thing a module owns:
|
||||
|
||||
1. **File and directory names** under `server/src/`, `server/scripts/`, `server/db/` and `client/src/`.
|
||||
2. **Import and require specifiers** — the path in `require('…')` / `from '…'`.
|
||||
3. **Route path literals** — the string arguments to `.get`/`.post`/`.put`/`.patch`/`.delete`/`.use`.
|
||||
4. **Declared identifiers** — function, const, class and property names.
|
||||
|
||||
It does **not** read comments or string content generally, and that is not a loophole. Core's
|
||||
marketing copy legitimately says "shard" — `About.jsx`, `Screenshots.jsx`, `SiteFooter.jsx`,
|
||||
`heroLayout.js` — and a literal word grep would turn each of those into a CI failure while proving
|
||||
nothing about the boundary. Worse, it would forbid a core comment from ever using the word as an
|
||||
example, which is the sort of rule people work around rather than obey. The boundary this test exists
|
||||
to defend is *structural*: core must not name a module's files, import them, route to them, or
|
||||
declare their symbols. It can talk about them in English.
|
||||
|
||||
Core's UO-flavoured default copy is dealt with directly instead, as `MODULE_SYSTEM.md` §2.7.1's
|
||||
slice 4 — a rewrite with its own review, not an exemption.
|
||||
|
||||
### 5.3 Zero-line route manifest diff (CI, both repos)
|
||||
|
||||
`npm run routes:manifest -- --check` in core; the module generates and freezes its own manifest in
|
||||
@@ -521,12 +910,22 @@ its own repo, using the same script pointed at a core+module app. Phase 2 must p
|
||||
diff in core's; Phase 3 moves the UO entries out of core's and into module-uo's, which is the one
|
||||
diff the whole workstream is allowed.
|
||||
|
||||
**Settled 2026-08-11: `module-uo`'s CI checks core out at a pinned ref.** The module's workflow
|
||||
clones `RunicGateway/website` at a ref recorded in the module repo, drops itself in as `modules/uo`,
|
||||
and runs core's own `routeManifest.js`. Nothing else proves the URLs a module claims are the URLs it
|
||||
actually serves — a manifest frozen by hand goes stale silently, and the failure it would have caught
|
||||
is a route that moved.
|
||||
|
||||
Pinning the ref rather than tracking `edge` is what keeps this from being a source of unexplained red
|
||||
Xes: core moves for reasons that have nothing to do with the module, and a bump is then a deliberate
|
||||
commit that says which core the module was last proved against.
|
||||
|
||||
---
|
||||
|
||||
## Part 6 — Amendments to MODULE_SYSTEM.md
|
||||
|
||||
Four things the survey found that the design of record gets wrong or does not cover. The first
|
||||
needs a decision.
|
||||
Things the survey found that the design of record gets wrong or does not cover, plus what
|
||||
implementation has since amended. The first needed a decision and has one.
|
||||
|
||||
### 6.1 OpenAPI generation does not survive a dynamic loader — **settled: fragment merge**
|
||||
|
||||
@@ -605,6 +1004,69 @@ files. Two consequences:
|
||||
Neither changes a decision; both are corrected here rather than left to be tripped over when the
|
||||
extraction is counted against the plan.
|
||||
|
||||
### 6.5 Grandfathered names, and why the prefix rules survive them
|
||||
|
||||
§2.4 requires a module's stream ids and announce legs to carry its module id. Eight names predate the
|
||||
module system and cannot take it:
|
||||
|
||||
| Kind | Names | Why they cannot be renamed |
|
||||
| --- | --- | --- |
|
||||
| Streams | `server.status`, `idoc.warning`, `champ.start`, `governor.election`, `vendor.sale`, `house.idoc`, `account.login` | stored in `notification_subs` rows; read by a shipped Android client |
|
||||
| Announce leg | `towncrier` | a stored value in `announce_job_legs.leg` and the body of the retry endpoint |
|
||||
|
||||
They are allowed to **`uo` alone**, by an explicit per-module allowlist — the same shape and the same
|
||||
reasoning as the loader's `LEGACY_TABLE_PREFIXES` for module-uo's 27 tables. Grandfathering by
|
||||
allowlist rather than dropping the rule is what keeps the rule real for every module written after
|
||||
this one; the alternative leaves the first name collision to be discovered by a module silently
|
||||
adopting someone else's stream.
|
||||
|
||||
### 6.6 An extension slot is invisible to static analysis — core needs the merge too
|
||||
|
||||
§6.1 settled the fragment merge for *modules*. PR 4 found that core needs the identical machinery for
|
||||
its **own** slot fills, one phase earlier than the plan expected.
|
||||
|
||||
A slot's router is created by `registries.declareSlot()` and filled later, so there is no literal
|
||||
`use(require(...))` for swagger-autogen to follow. Moving the six `/admin/users/:id/shard/*` routes
|
||||
behind `admin.users.detail` therefore deleted 407 lines from `swagger-output.json` — with
|
||||
`Swagger-autogen: Success` and no warning. Same failure as §7.4, different cause, and it would have
|
||||
shipped six undocumented core routes against CLAUDE.md's standing rule.
|
||||
|
||||
`npm run swagger` now has a second step (`swagger/slotSpecs.js`): for each **filled** slot, generate a
|
||||
fragment by pointing swagger-autogen at that router's own file, re-root its paths at the prefix the
|
||||
router actually hangs at, and merge. Two things are derived rather than written down, because a
|
||||
written-down copy drifts:
|
||||
|
||||
- **which** slots — from `registries.filledSlots()`;
|
||||
- **where** each hangs — by finding the slot's own router object in the live express stack, decoding
|
||||
the mount prefixes above it with `scripts/routeManifest.js`'s own `mountPath`, so the manifest and
|
||||
the spec can never disagree about what a mount decodes to.
|
||||
|
||||
An empty fragment is a hard build failure, because an empty fragment is exactly what the silent drop
|
||||
looks like. The merge itself is `swagger/mergeSpec.js` — the ~40-line helper §6.1a already owed core
|
||||
for module fragments, written here and proved against core's own slot before a module depends on it.
|
||||
This is **build**-time and lands in the committed spec, because slot routes are core's; a module's
|
||||
fragment is still merged at **request** time into `/api/docs.json` (§6.1a), and `swagger-output.json`
|
||||
stays reproducible on any machine regardless of what is installed.
|
||||
|
||||
`registerExtension` therefore takes a third, **core-only** argument: the file its router is generated
|
||||
from. A module needs no equivalent — it ships a prebuilt `swagger-fragment.json`, because core never
|
||||
has its sources to analyse.
|
||||
|
||||
### 6.7 `/api/v1/public/modules` is not the client's load trigger
|
||||
|
||||
`MODULE_SYSTEM.md` §2.6 step 4 says "the SPA reads `/api/v1/public/modules` to learn what to load,
|
||||
then registers routes, nav and its feature provider". Step 3 of the same list resolved the loading
|
||||
question a different way, and §3.1.3 here is the normative version: `htmlShell.js` injects a
|
||||
`<script type="module" src="/modules/<id>/entry.js">` per **started** module, so the browser is handed
|
||||
the tag by the document and never fetches a URL the endpoint told it about. Nothing waits on an API
|
||||
round trip to start loading, which is also why the tag can be in `<head>`.
|
||||
|
||||
What the endpoint is for is **feature detection**: capabilities for the SPA and the Android app, which
|
||||
has no chunk to load at all. §2.9 is the shape. The two statements were only ever in tension because
|
||||
§2.6 was written before the CSP constraint forced the injected-tag design; step 4 should read "the SPA
|
||||
reads `/api/v1/public/modules` to feature-detect", and the registration it describes happens when the
|
||||
injected chunk executes and calls `window.__rg.registry` (§3.3).
|
||||
|
||||
---
|
||||
|
||||
## Part 7 — What the spike proved
|
||||
@@ -690,10 +1152,40 @@ They are recorded so nobody reads them as intended shape:
|
||||
3. **The module has no `swagger-fragment.json`** — §6.1a's obligation needs core's merge helper on
|
||||
the other side of it, which is Phase 2.
|
||||
|
||||
### 7.6 A finding for Phase 2's loader
|
||||
### 7.6 A finding for Phase 2's loader — **settled: an explicit `load()`**
|
||||
|
||||
`scan()` is lazy — requiring the loader does not run it. That is deliberate (app.js decides when
|
||||
`scan()` was lazy — requiring the loader did not run it. That was deliberate (app.js decides when
|
||||
modules are discovered) but it is a sharp edge: a caller that requires the loader and reads nothing
|
||||
gets an empty, *silent* module list. It cost one confusing test failure during the spike. Phase 2
|
||||
should either make the trigger explicit in the name or scan at require time and let app.js order the
|
||||
require.
|
||||
gets an empty, *silent* module list. It cost one confusing test failure during the spike.
|
||||
|
||||
**Phase 2 PR 2 made the trigger explicit** rather than scanning at require time. `app.js` calls
|
||||
`modules.load(tierRouters)` once, and `list()` throws until it has. Scanning on require was the
|
||||
alternative and was rejected for two reasons: the loader now needs the tier routers *handed to it*
|
||||
for the §4.3 collision check, which a require-time side effect cannot receive; and it would make the
|
||||
ordering constraint invisible, enforced by where a `require` sits rather than by an argument that is
|
||||
missing if it is wrong.
|
||||
|
||||
### 7.7 The client half has to be verified in a browser — **the timing bug no test could see**
|
||||
|
||||
Phase 2 PR 7 built the delivery mechanism: the static mount, the injected tag, `window.__rg`, the
|
||||
registry, and core's consumption of it. Everything above is unit-tested, and the tests all passed
|
||||
against a build that **did not work in a browser**.
|
||||
|
||||
The smoke that found it is worth repeating whenever this seam changes, and it is four steps: write a
|
||||
throwaway `modules/<id>/` with a hand-written ESM `entry.js` — no bundler needed, since
|
||||
`window.__rg.react.createElement` is enough to render a page — point `MODULES_DIR` at it, boot the
|
||||
server against the built client, and load the module's URL in a real browser with the console open.
|
||||
|
||||
What it caught was step 5 of §3.1: core mounted before any module chunk had evaluated, because
|
||||
`document.readyState` during a deferred script is `'interactive'` and not `'loading'`. The page
|
||||
redirected home — the same thing a module that failed to load does — with **no error anywhere**:
|
||||
the chunk had fetched, executed, and registered its route into a registry nothing read again. No unit
|
||||
test in this repo can see it. There is no DOM in the server or client test runner, and the ordering
|
||||
being asserted is the browser's, not the code's.
|
||||
|
||||
Two smaller things the same run confirmed, both worth keeping in the loop when re-running it: the
|
||||
chunk executes under the **enforced** `script-src 'self'` with no CSP report, which is the property
|
||||
§3.6 called the highest-risk detail in the plan; and the shell is read **once at boot**
|
||||
(`htmlShell.init`), so rebuilding the client without restarting the server serves an `index.html`
|
||||
pointing at a hashed bundle that no longer exists — core never runs, `window.__rg` is undefined, and
|
||||
the failure looks exactly like a contract violation in the module.
|
||||
|
||||
@@ -1,12 +1,13 @@
|
||||
# The Module System — design of record
|
||||
|
||||
**Status:** approved design, not yet implemented. Every decision in Part 3 has been settled with the
|
||||
org lead; Part 1 records what was verified against the working trees on 2026-08-10, including the
|
||||
places the original draft was wrong.
|
||||
**Status:** approved design, **in implementation** — Phase 2's core scaffolding is landing on the
|
||||
website `edge` branch, PRs 1–7 of 9 done (§2.7 tracks what each settled). Every decision in Part 3
|
||||
has been settled with the org lead; Part 1 records what was verified against the working trees on
|
||||
2026-08-10, including the places the original draft was wrong.
|
||||
|
||||
**The normative contract is [`MODULE_API.md`](MODULE_API.md)** (Phase 1). This document decides what
|
||||
the module system *is*; that one decides exactly what a module may call. Where the two differ, that
|
||||
one wins — its Part 6 lists the four places it amends this document.
|
||||
one wins — its Part 6 lists every place it amends this document.
|
||||
|
||||
**Goal.** Turn Runic Gateway from a UO/ServUO-specific platform into a game-agnostic one. The core
|
||||
architecture is unchanged — sidecar → website → browser. What changes is that game-specific
|
||||
@@ -96,6 +97,14 @@ moderator-visible.
|
||||
**Resolved:** nav registration takes a target group and order (`{ group: 'Moderation', order: 30 }`),
|
||||
and `MOD_PATHS` becomes a `roles`-derived computation rather than a path allowlist.
|
||||
|
||||
Built in Phase 2 PR 8. Two things it turned up that this section did not predict. The interleave has
|
||||
to happen **before** the admin-override merge and not after it, because that merge drops any `to`
|
||||
its base array does not declare — appending module rows afterwards would leave them uneditable in
|
||||
Admin → Navigation, which today's UO rows are not. And there was a **third** hardcoded list: the
|
||||
redirect that confines a moderator checked three path prefixes, while `MOD_PATHS` listed five paths,
|
||||
and they disagreed about `/admin/houses` — a moderator who clicked Houses in their own sidebar was
|
||||
bounced straight back to Moderation. One derivation cannot disagree with itself.
|
||||
|
||||
### 1.5 The public nav's feature-gating mechanism is itself a shard system
|
||||
|
||||
Ten of the sixteen entries in `SiteHeader.jsx`'s NAV carry a `feature:` key (`status`, `champs`,
|
||||
@@ -107,6 +116,12 @@ Extracting the module removes the provider that core's own nav filter depends on
|
||||
registers its `useShardFeatures` for its own namespace. No core nav item carries a `feature` today,
|
||||
so with no module installed the filter is a correct no-op.
|
||||
|
||||
Built in Phase 2 PR 8, and core registers into it **now** rather than at extraction: `useShardFlags`
|
||||
goes in under the owner id `core`, so the ten rows above are already resolved through the seam and
|
||||
`SiteHeader` runs one mechanism instead of two. Which provider answers a row is decided by the
|
||||
module that registered it, not by a prefix parsed out of the flag name, so those ten keep the exact
|
||||
strings they carry today and Phase 3 moves them without a rename.
|
||||
|
||||
### 1.6 There is no migration system to model a module migration runner on
|
||||
|
||||
`server/db/schema.sql` is a single idempotent file — 1,380 lines, 67 tables — replayed in full on
|
||||
@@ -153,16 +168,36 @@ Everything else is a folder move. These are not:
|
||||
1. **`src/config/notificationStreams.js`** — the push stream catalog. `mapShardEvent()` and most of
|
||||
`STREAMS` are shard-derived, and it imports `PUBLIC_KINDS` from `utils/shardBroadcast`. Push
|
||||
*infrastructure* is core; this *catalog* is module content.
|
||||
→ `registerNotificationStreams({ streams, mapEvent })`.
|
||||
→ `registerNotificationStreams(streams)`.
|
||||
2. **`src/utils/pushDispatch.js`** — core infrastructure, but `fromShardEvent()` (line 112) requires
|
||||
the `shardLinks` model (line 21) and `mapShardEvent` (line 23).
|
||||
→ invert: `publish()` stays core, `fromShardEvent` moves into the module and calls it.
|
||||
3. **`src/utils/announceWorker.js`** — the news dispatcher, with two delivery legs: Discord (core)
|
||||
and town crier (module, via `uoLinkClient.postTownCrier`, line 36).
|
||||
→ `registerAnnounceLeg({ leg, dispatch, classify })`.
|
||||
→ `registerAnnounceLeg({ leg, label, dispatch, classify })`.
|
||||
|
||||
`src/utils/newsGump.js` is module-side (news → in-game gump) and moves whole.
|
||||
|
||||
**Done in Phase 2 PR 4**, with core still the only registrant — the registries are
|
||||
`src/modules/registries.js` and core goes through them by the same door a module will
|
||||
(`registerCore()`, called explicitly from `app.js` before `modules.load()`). What each of the three
|
||||
became:
|
||||
|
||||
1. Split in two. `config/coreStreams.js` is core's one stream (`news.post`, produced by the website's
|
||||
own posts path); `config/shardStreams.js` is the other seven plus `mapShardEvent` and the
|
||||
public-safety filter, and moves to module-uo whole. `registerNotificationStreams` lost its
|
||||
`mapEvent` half — see [`MODULE_API.md`](MODULE_API.md) §2.4 for why that was a leftover, and what
|
||||
follows for the public/personal split.
|
||||
2. Inverted. `pushDispatch.js` is `publish` + `isAllowedEndpoint` and nothing else;
|
||||
`utils/shardPush.js` holds `fromShardEvent` and is what `shardIngest` now calls.
|
||||
3. Legs became registrations, and per-leg **rows**. The `towncrier_*` / `discord_*` column groups on
|
||||
`announce_jobs` could never have held a module's leg — a module cannot `ALTER` a core table — so
|
||||
they became `announce_job_legs`, backfilled and dropped in the same idempotent replay. The worker
|
||||
no longer contains the word "towncrier": it iterates whatever is registered.
|
||||
|
||||
The residue in core is a one-time backfill block in `schema.sql`, deletable once every deployment has
|
||||
booted it, and the two lines of `registerCore()` that Phase 3 turns into module-uo's `register()`.
|
||||
|
||||
### 1.9 A fourth mount shape: module routes under a core resource
|
||||
|
||||
`router/v1/admin/users.router.js` mounts `usersShard.controller.js` at six UO sub-paths of a **core**
|
||||
@@ -175,6 +210,17 @@ narrow **extension slot** on `/admin/users/:id` that the module mounts into, so
|
||||
what "shard" means and all six URLs are preserved. Only core may declare an extension slot; a module
|
||||
may not invent one.
|
||||
|
||||
**Both done in Phase 2 PR 4**, with core filling its own slot: the six paths are
|
||||
`router/v1/admin/usersShard.router.js`, registered into `admin.users.detail` by `registerCore()`, and
|
||||
Phase 3 changes the registrant rather than the routes. The slot's router is created at declare time
|
||||
and filled later, because `users.router.js` is required while `app.js` is still being built. It is
|
||||
mounted **last** on the resource, so core wins any path conflict by first-match.
|
||||
|
||||
One consequence was not foreseen and is worth the warning: **a slot is invisible to static analysis.**
|
||||
There is no literal mount for `swagger-autogen` to follow, so the move silently deleted all six paths
|
||||
from `swagger-output.json` while printing `Success`. The OpenAPI build now merges a generated
|
||||
fragment per filled slot — [`MODULE_API.md`](MODULE_API.md) §6.6.
|
||||
|
||||
### 1.10 The Discord bot has no UO logic
|
||||
|
||||
The draft listed the bot's "UO-specific event/moderation logic" as an extraction candidate. Grepping
|
||||
@@ -235,8 +281,9 @@ It also rules out import maps as the shared-dependency mechanism: `config/csp.js
|
||||
|
||||
**Resolved** — see §2.6. The path that survives all three constraints is: the module's CI ships a
|
||||
**prebuilt ESM chunk**, core hands it React through a **global** rather than an import map, and
|
||||
`htmlShell.js:111` injects a **same-origin** `<script type="module" src>`, which `'self'` already
|
||||
allows.
|
||||
`htmlShell.js` injects a **same-origin** `<script type="module" src>`, which `'self'` already
|
||||
allows. Verified in a browser against the enforced policy in Phase 2 PR 7, not only reasoned about
|
||||
([`MODULE_API.md`](MODULE_API.md) §7.7).
|
||||
|
||||
---
|
||||
|
||||
@@ -372,10 +419,21 @@ A re-install or an upgrade refreshes `name`/`version`/provenance and deliberatel
|
||||
alone: upgrading an enabled module must not silently switch it off, and re-installing a disabled one
|
||||
must not silently switch it on.
|
||||
|
||||
**A row whose directory is gone is marked `startup_failed`** (stage `require`, reason "module
|
||||
directory not present on the volume"), settled with PR 5. The boot reset above has just moved it to
|
||||
`enabled`, and a row claiming to be enabled for a module that is not on the volume is the one state
|
||||
that is simply untrue — it would be read that way by the admin panel and by
|
||||
`GET /api/v1/public/modules` alike. This catches only a directory deleted by hand: an uninstall
|
||||
leaves the row `disabled`, which the reset never touches.
|
||||
|
||||
### 2.5 Install, uninstall, purge
|
||||
|
||||
Modules live on a **mounted volume**, not in the image — the same treatment `uploads` already gets in
|
||||
`docker-compose.yml`. That is what makes the WordPress model work against a pull-only image.
|
||||
Modules live on a **mounted volume**, not in the image. That is what makes the WordPress model work
|
||||
against a pull-only image. *(Settled in Phase 2 PR 9: it is a **bind mount** of `./modules`, not the
|
||||
named volume this sentence originally reached for by analogy with `uploads` — hand-placing a module
|
||||
directory is a supported install below, and a named volume would route it through `docker cp`. The
|
||||
image's copy is excluded by `.dockerignore`, so a module in a builder's working tree can never ship
|
||||
inside an image; see `modules/README.md` in the website repo.)*
|
||||
|
||||
**Install:** admin selects the module → bundle downloaded from the module repo's release and verified
|
||||
against its `sha256` → unpacked into `modules/<id>/` on the volume → `installed_modules` row written →
|
||||
@@ -403,12 +461,20 @@ builds nothing, production pulls a prebuilt image, and `script-src 'self'` forbi
|
||||
2. **Core exposes the shared dependencies on a global** before mount — `window.__rg = { react,
|
||||
reactDom, router, registry }` — and the module's externals resolve to it. A global, not an import
|
||||
map, precisely because an import map must be inline and CSP forbids that.
|
||||
3. **`htmlShell.js` injects the module's entry script.** It already rewrites `</head>`
|
||||
(`utils/htmlShell.js:111`), so this is an extension of a working mechanism, not a new one. The tag
|
||||
is `<script type="module" src="/modules/uo/entry.js">` — same-origin, so `'self'` passes with no
|
||||
nonce and no inline.
|
||||
4. **The SPA reads `/api/v1/public/modules`** to learn what to load, then registers routes, nav and
|
||||
its feature provider through `window.__rg.registry`.
|
||||
3. **`htmlShell.js` injects the module's entry script.** It already rewrites the shell it serves, so
|
||||
this is an extension of a working mechanism, not a new one. The tag is
|
||||
`<script type="module" src="/modules/uo/entry.js">` — same-origin, so `'self'` passes with no
|
||||
nonce and no inline. *(Amended in Phase 2 PR 7: the tag is injected before `</body>`, not at the
|
||||
`</head>` rewrite this step assumed. Module scripts execute in document order and core's bundle
|
||||
has to run first, so the injection must be after core's own script tag wherever a bundler chooses
|
||||
to put it — [`MODULE_API.md`](MODULE_API.md) §3.1, which also states the static mount's root, its
|
||||
state guard and its cache policy.)*
|
||||
4. **The SPA reads `/api/v1/public/modules`** to feature-detect against what this backend is
|
||||
serving. Registration happens when the injected chunk executes and calls `window.__rg.registry` —
|
||||
it is not gated on this call. *(Amended by [`MODULE_API.md`](MODULE_API.md) §6.7: this step
|
||||
originally said the SPA reads the endpoint "to learn what to load", which step 3 above had
|
||||
already answered a different way. Nothing waits on an API round trip to start loading. The
|
||||
endpoint's shape is API §2.9.)*
|
||||
|
||||
Phase 1 prototypes exactly this before anything is committed to it (§2.7).
|
||||
|
||||
@@ -419,7 +485,7 @@ Phase 1 prototypes exactly this before anything is committed to it (§2.7).
|
||||
Nothing else can be trusted until the first of these is done.
|
||||
|
||||
**Phase 1 — API contract + spike (blocking).** Merge this document. Write the contract at
|
||||
[`docs/website/MODULE_API.md`](MODULE_API.md) — **done**; it amends this document in four places,
|
||||
[`docs/website/MODULE_API.md`](MODULE_API.md) — **done**; the places it amends this document are
|
||||
listed in its Part 6, one of which (OpenAPI generation, §6.1 there) needs a decision before Phase 2
|
||||
starts. Then a throwaway spike on an unmerged branch moving **`/api/v1/public/atlas/*`** behind the
|
||||
proposed surface — the smallest honest test: six routes, DB-backed, no sidecar, no SSE, one boot
|
||||
@@ -442,20 +508,227 @@ too (API §7.2).
|
||||
3. `ensureSchema()` extended to replay module fragments after core's.
|
||||
4. The three de-entanglement registries (§1.8), with core still the only registrant.
|
||||
5. Boot/shutdown hook dispatch in `server.js`, likewise.
|
||||
6. `GET /api/v1/public/modules` — installed ids, versions and capabilities, shaped like the existing
|
||||
branding/site-settings endpoint. The SPA needs it to know what to load; the Android plan consumes
|
||||
the same endpoint.
|
||||
7. Client `src/modules/registry.js`, the `window.__rg` shared-dependency global, and the
|
||||
`htmlShell` script injection — empty registry, no visible change.
|
||||
6. `GET /api/v1/public/modules` — ids, names, versions and capabilities of the modules currently
|
||||
**serving**, shaped like the existing branding/site-settings endpoints (anonymous, database-free,
|
||||
not site-mode gated). The SPA and the Android plan both feature-detect against it; it is not what
|
||||
loads a client chunk ([`MODULE_API.md`](MODULE_API.md) §2.9 and §6.7).
|
||||
7. Client `src/modules/registry.js`, the `window.__rg` shared-dependency global, the chunk's static
|
||||
mount and the `htmlShell` script injection — empty registry, no visible change.
|
||||
8. `MOD_PATHS` → `roles`-derived (§1.4); the generic feature-provider seam (§1.5).
|
||||
9. `docker-compose.yml` gains the `modules` volume.
|
||||
9. `docker-compose.yml` gains the `modules` mount.
|
||||
|
||||
Exit criterion: `routes.manifest.json` diff is zero lines and every existing test passes. If Phase 2
|
||||
changes one URL, it is wrong.
|
||||
Exit criterion: no **existing** URL moves and every existing test passes. If Phase 2 changes one URL,
|
||||
it is wrong. PR 6 is the single deliberate exception in the phase and it *adds*: `routes.manifest.json`
|
||||
gains exactly one line, `GET /api/v1/public/modules`, and nothing else in the file moves. Every other
|
||||
PR in Phase 2 produces a zero-line diff.
|
||||
|
||||
**Progress: PR 1 done** — `installed_modules` and the state machine, with the stored shape and the
|
||||
boot rules settled in §2.4 above. No loader, no routes, no boot wiring yet, so it changes nothing an
|
||||
operator or a client can see.
|
||||
**Progress: complete — all nine PRs landed.**
|
||||
|
||||
- **PR 1** — `installed_modules` and the state machine, with the stored shape and the boot rules
|
||||
settled in §2.4 above.
|
||||
- **PR 2** — `server/src/modules/loader.js`: the filesystem scan, manifest validation, prefix and
|
||||
table-name collision rejection, per-module try/catch and the tier mount, behind the §4.5 dispatch
|
||||
guard. Two decisions landed with it, both recorded in [`MODULE_API.md`](MODULE_API.md): the load
|
||||
trigger is **one explicit `modules.load(tierRouters)` call in `app.js`**, never a lazy scan
|
||||
(API §7.6); and the "does core own this prefix" check **probes the live tier routers** rather than
|
||||
a hardcoded table, so it cannot drift when core adds a capability router (API §4.3). The three
|
||||
de-entanglement registries and the two lifecycle hooks throw `not available until phase 2 PR 4/5`
|
||||
rather than no-op — an accepting stub would let a module believe it had registered something.
|
||||
28 tests, all on the failure paths.
|
||||
- **PR 3** — schema fragment replay. `ensureSchema()` replays each installed module's fragment after
|
||||
core's, with the statement splitter extracted to `utils/sqlStatements.js` so both are split by the
|
||||
same code. The decision that shaped it, recorded in [`MODULE_API.md`](MODULE_API.md) §2.6: the
|
||||
fragment is **validated at load time and executed later**, split on whether a database is needed to
|
||||
know the answer — a fragment breaking a stated rule never mounts, while a failure only the server
|
||||
could report (a bad column type) is post-mount and 503s. The rules are enforced as a **leading-verb
|
||||
allowlist** (`CREATE`, `ALTER`, `INSERT`, `UPDATE`) rather than the `DROP` denylist §2.6 words them
|
||||
as, because the file is replayed on **every boot**. Found while wiring it: `npm run seed` calls
|
||||
`ensureSchema()` without ever requiring `app.js`, so the replay has to tolerate an unscanned loader.
|
||||
|
||||
- **PR 4** — the three de-entanglement registries, `src/modules/registries.js`. Core's own streams,
|
||||
its Discord announce leg and its users-detail routes all go through them, so the seams are
|
||||
exercised on every boot before a module depends on them; §1.8 and §1.9 above record what each
|
||||
became. Four decisions landed with it, all recorded in [`MODULE_API.md`](MODULE_API.md): announce
|
||||
legs became a **child table** rather than waiting for Phase 3 (§2.4 — a module cannot alter a core
|
||||
table, so a registered leg had nowhere to live); **`mapEvent` dropped** from the stream registry
|
||||
(§2.4 — a leftover from before the push inversion was settled); **core registers through the same
|
||||
staging area a module uses**; and core's six shard sub-paths **moved behind the slot now** rather
|
||||
than in Phase 3.
|
||||
|
||||
Registering is **validate-then-commit**: the loader stages a module's claims and the second pass
|
||||
commits them, so a module that throws halfway through `register()` — or fails a later validation
|
||||
step — leaves nothing behind. That is the registry-side twin of PR 2's second-pass mount rule.
|
||||
|
||||
Two build tools needed teaching, both because a mechanism this PR introduced is one they had never
|
||||
seen. `scripts/routeManifest.js` could not decode a **parameterised mount**: its unwinder expected
|
||||
a group shape express does not emit, and the branch had never run. It threw rather than guessing,
|
||||
which is exactly what it is for. And `swagger-autogen` could not follow a route into an extension
|
||||
**slot**, deleting 407 lines while reporting success; the fix is the fragment merge core owed
|
||||
anyway ([`MODULE_API.md`](MODULE_API.md) §6.6).
|
||||
|
||||
There is still no module on the volume and no boot wiring, so this changes nothing an operator or a
|
||||
client can see: **884 tests pass** and `routes.manifest.json` is unchanged at 229 routes. The two
|
||||
lines of OpenAPI that do move are the retry endpoint's summary and its `leg`, which is no longer a
|
||||
fixed enum because the leg set is whatever has been registered.
|
||||
|
||||
- **PR 5** — boot/shutdown dispatch and the `installed_modules` reconcile, `src/modules/lifecycle.js`.
|
||||
`api.onBoot`/`api.onShutdown` stop throwing, `server.js` gains one call on each side, and the
|
||||
§2.4 machine finally runs against real outcomes — which is what makes §4.5's `disabled` 404 leg
|
||||
reachable for the first time. Four decisions landed with it, all recorded in
|
||||
[`MODULE_API.md`](MODULE_API.md) §2.5 and §4.4: **the loader classifies its failures** by §4.3 step,
|
||||
so `failure_stage` says where a module broke instead of being a column nothing filled; **a row whose
|
||||
directory is gone is marked failed** rather than left claiming `enabled` (§2.4 above); **core's eight
|
||||
UO boot call sites stay in `server.js`** until Phase 3, because unlike a registered announce leg a
|
||||
boot call site already has somewhere to live and moving it now would be extraction done early in a
|
||||
phase whose exit criterion is that nothing changes; and **`onBoot` gets no timeout** — shutdown races
|
||||
a SIGKILL and boot does not, and a slow `onBoot` delaying the listener is the contract's promise to
|
||||
a module that must warm up before it serves.
|
||||
|
||||
The dispatch lives outside the loader for the reason the schema replay does: the loader is required
|
||||
by `app.js` against a dead pool, and this half is database-first. They meet at one function,
|
||||
`loader.setState()`, so the in-memory record the dispatch guard reads and the row the admin panel
|
||||
reads cannot drift apart.
|
||||
|
||||
Still nothing on the volume: **900 tests pass**, `routes.manifest.json` is unchanged at 229 routes
|
||||
and the OpenAPI spec regenerates byte-identical.
|
||||
|
||||
- **PR 6** — `GET /api/v1/public/modules`, the first module-system URL a client can see. Four
|
||||
decisions, all recorded in [`MODULE_API.md`](MODULE_API.md) §2.9: **`started` modules only**, so a
|
||||
disabled or failed module is absent exactly as its routes and nav already are, and no visitor is
|
||||
told that something is broken; **no `state`, `failure_stage` or `failure_reason`** on the public
|
||||
surface — those are the admin screen's, and the reason is an exception string from inside core;
|
||||
**no `client` chunk URL**, because `htmlShell` hands the browser the tag rather than a URL to fetch
|
||||
(which amends §2.6 step 4 above — see API §6.7); and **no `siteMode` gate and no database**, the
|
||||
same class as `/public/status` and `/public/version`, so a client can still feature-detect while
|
||||
the site is in maintenance.
|
||||
|
||||
It is a **capability router of its own** rather than a fifth singleton in `site.router.js`, and that
|
||||
is the load-bearing part. The loader's prefix-collision probe reads the live tier stack and skips
|
||||
root-mounted layers, because a `use('/', …)` matches every path — so a route declared inside the
|
||||
root-mounted site router is invisible to it. Mounting `use('/modules', …)` is what makes "no module
|
||||
may ever claim `/modules`" a rule the loader enforces rather than a convention a reviewer has to
|
||||
remember.
|
||||
|
||||
**910 tests pass** (+9, every one of them on the boundary: what must *not* appear). The route
|
||||
inventory goes 229 → 230 (228 public + 2 internal) and moves by exactly the one added route;
|
||||
`routes.guards.json` records it with an empty `gates` list, which is itself the assertion that the
|
||||
endpoint is ungated. The OpenAPI spec gains the operation and the `PublicModules`/`PublicModule`
|
||||
schemas. The published mirror [`api-route-inventory.json`](./api-route-inventory.json) is refreshed
|
||||
to match.
|
||||
|
||||
- **PR 7** — the client half's delivery: `client/src/modules/registry.js`, `window.__rg`
|
||||
(`modules/shared.js`), the chunk's static mount and the `htmlShell` injection, with `App.jsx`
|
||||
reading `routesFor` for all three areas. The registry is empty on a bare core, so nothing an
|
||||
operator can see changes. Four decisions, all recorded in [`MODULE_API.md`](MODULE_API.md) §3.1 and
|
||||
§3.4: **routes now, nav in PR 8** — PR 7 is "a module chunk loads and renders its page", PR 8 is
|
||||
"it appears in the nav", which keeps the nav interleave and its override merge in one reviewable
|
||||
change; **the script tag is injected before `</body>`**, not into `</head>`, so the ordering that
|
||||
the whole client contract rests on comes from document structure rather than from Vite's choice to
|
||||
hoist core's entry into `<head>`; **the static mount is rooted at the entry's directory, behind the
|
||||
module's state guard, with `no-cache`** — a mount rooted at the module root would publish server
|
||||
source, `module.json` and the schema fragment, so an entry in the module root is rejected outright;
|
||||
and **the UI kit ships its seven real members**, with `AdminPage` struck from the contract rather
|
||||
than invented in core to satisfy a table.
|
||||
|
||||
The verification that mattered was **not a test**. Everything passed against a build that did not
|
||||
work in a browser: core mounted before any module chunk had evaluated, because `document.readyState`
|
||||
during a deferred script is `'interactive'`, not `'loading'`. A module's routes were missing from
|
||||
the first render and its URL redirected home — indistinguishable from a module that failed to load,
|
||||
and with nothing logged anywhere. It was found by loading a hand-written chunk in Chrome, and that
|
||||
smoke is now written down as part of the contract ([`MODULE_API.md`](MODULE_API.md) §7.7) because no
|
||||
test in this repo can see it. The same run confirmed the property §3.6 called the highest-risk
|
||||
detail in the plan: the chunk executes under the **enforced** `script-src 'self'`, resolving core's
|
||||
React and UI kit off the global, with zero CSP reports.
|
||||
|
||||
**933 server tests** (+23) and **123 client tests** (+14) pass; `routes.manifest.json` is unchanged
|
||||
at 230 routes and the OpenAPI spec regenerates byte-identical — `/modules/<id>/` is a
|
||||
filesystem-conditional static mount, not API surface, for the same reason `/uploads` and `/brand`
|
||||
are not in the manifest.
|
||||
|
||||
- **PR 8** — the nav half PR 7 deferred, and the two seams §1.4 and §1.5 asked for: `withModuleNav`
|
||||
(`client/src/modules/nav.js`) interleaving module rows into core's three navs, `MOD_PATHS` and the
|
||||
moderator redirect replaced by a `roles`-derived computation in `client/src/lib/adminNav.js`, and
|
||||
the generic feature-provider seam (`modules/features.jsx` + `modules/featureGate.js`) that core
|
||||
registers its own `useShardFlags` into. Four decisions, all recorded in
|
||||
[`MODULE_API.md`](MODULE_API.md) §3.3.
|
||||
|
||||
**The interleave happens before the admin-override merge**, which is the decision the rest follow
|
||||
from: the merge is keyed by `to` and drops any key its base array does not declare, so module rows
|
||||
appended after it would be unorderable, unrelabellable and unhideable — and today's UO rows are
|
||||
all three of those things, so appending would make the extraction a visible regression for every
|
||||
operator who has ever edited their nav. Doing it first means a module row is an ordinary row to
|
||||
everything downstream: nothing in `navOverrides.js`, `NavEditor.jsx` or the layouts knows a module
|
||||
exists. **Moderator visibility derives purely from `roles`**, which moves two rows the old
|
||||
allowlist withheld — Dashboard, whose `roles` had always named moderator, and My Characters, which
|
||||
is ungated self-service — both toward what the server already permitted. **A row's `feature` is
|
||||
resolved by the provider its own module registered**, so the namespace comes from the registration
|
||||
rather than from a parsed string prefix. And **core registers through the same seam**, under the
|
||||
owner id `core`, so `SiteHeader` holds one mechanism instead of two and Phase 3 is a deletion.
|
||||
|
||||
The PR also fixed a defect that predates the module system: the moderator redirect was a **third**
|
||||
hardcoded list, and it disagreed with `MOD_PATHS` about `/admin/houses`, so a moderator who
|
||||
clicked Houses in their own sidebar was bounced back to Moderation. The derived allow-list is
|
||||
computed from the **base** nav rather than the merged one, so an override — which is presentation
|
||||
— cannot move that boundary in either direction.
|
||||
|
||||
**933 server tests** (unchanged — this PR is client-only) and **160 client tests** (+37) pass;
|
||||
`routes.manifest.json` is unchanged at 230 routes and the OpenAPI spec regenerates byte-identical.
|
||||
The [§7.7](MODULE_API.md#77-the-client-half-has-to-be-verified-in-a-browser--the-timing-bug-no-test-could-see)
|
||||
browser smoke was re-run, since this is the seam that rule exists for: a throwaway module
|
||||
registering nav in all three areas and a provider granting one flag and withholding another. It
|
||||
confirmed, in Chrome with the console open, that the row lands inside core's Moderation group
|
||||
rather than in an appended block, that the withheld row does not render while the granted one
|
||||
does, that a moderator reaches both `/admin/houses` and the module's own admin page, and that an
|
||||
admin can relabel a module row in Admin → Navigation and have it persist and apply — the whole
|
||||
point of merging before the override layer. Zero CSP reports, zero console errors.
|
||||
|
||||
- **PR 9** — the mount itself, which closes the phase: `docker-compose.yml` gains `./modules` at
|
||||
`/app/modules`, the `Dockerfile` creates that directory node-owned, `.dockerignore` keeps any local
|
||||
one out of the image and `modules/README.md` documents the directory for whoever opens it.
|
||||
|
||||
It is a **bind mount, not the named volume** §2.5 assumed by analogy with `uploads`. Placing a
|
||||
module directory by hand is a supported install, and a named volume makes that a `docker cp` into a
|
||||
running container — the one install path an operator without the admin panel has, routed through
|
||||
the least discoverable mechanism Docker offers. A bind mount makes it `tar -xf … -C ./modules`, and
|
||||
makes the installed set something an operator can *see*. §2.5 is amended above; nothing else about
|
||||
install, uninstall or purge changes.
|
||||
|
||||
Two consequences worth stating, because both are silent failures rather than errors. The directory
|
||||
is **tracked** — via its README, the same shape `brand/` already uses — because Docker recreates a
|
||||
missing bind-mount source as `root:root`, and the container runs as uid 1000: delete `modules/` from
|
||||
a checkout and the next install fails on a permission error that names no cause. And the mount is
|
||||
**read-write**, since §2.5's install unpacks into it from inside the container; deferring that to
|
||||
Phase 4 would have bought nothing, as a mount-mode change is a redeploy either way.
|
||||
|
||||
`.dockerignore` matters more than it looks. `COPY . .` would otherwise bake whatever module the
|
||||
builder had checked out into every image — and because Docker seeds a *fresh* named volume from
|
||||
the image's contents, that module could have appeared on a production deployment that never
|
||||
installed it. The exclusion is what makes "modules live on a mount, never in the image" true rather
|
||||
than merely intended.
|
||||
|
||||
Verified against a **running container**, which is the only thing that can check any of the above —
|
||||
a compose file that parses proves nothing about ownership, and nothing about what the image
|
||||
contains. The image carries an empty, node-owned `/app/modules` despite a module sitting in the
|
||||
build context. A module on the mount loads, mounts, runs `onBoot` and reaches `started`;
|
||||
`/api/v1/public/modules` lists it; its chunk serves from the entry's directory alone, with the
|
||||
module's own server source and `module.json` both `404`. The [§7.7](MODULE_API.md#77-the-client-half-has-to-be-verified-in-a-browser--the-timing-bug-no-test-could-see)
|
||||
browser smoke was re-run against the containerised stack rather than a working tree: in Chrome, the
|
||||
page renders on first paint inside core's `PublicLayout`, drawing React and the UI kit off
|
||||
`window.__rg`, with its nav row interleaved into core's public nav — under the enforced
|
||||
`script-src 'self'`, with zero CSP reports and no console errors. Removing the directory by hand and
|
||||
restarting reconciles the row to `startup_failed`/`require` exactly as §2.4 says, and leaves core
|
||||
healthy with no script injected and `{"modules":[]}` published.
|
||||
|
||||
**933 server tests** and **160 client tests** pass, both unchanged — this PR ships no application
|
||||
code. `routes.manifest.json` stays at 230 routes and the OpenAPI spec regenerates byte-identical.
|
||||
The one source change is a comment: `scripts/routeManifest.js` enumerated the filesystem-conditional
|
||||
mounts it excludes and had never been told about `/modules`. Its filter is an allowlist, so the
|
||||
behaviour was always right and only the explanation was stale.
|
||||
|
||||
**Phase 2 is complete.** Core can discover, validate, mount, migrate, boot, publish, serve and
|
||||
navigate a module it does not contain, on a deployment that builds nothing — and it does all of that
|
||||
while no existing URL has moved. The exit criterion held: `routes.manifest.json` went 229 → 230 across
|
||||
the whole phase, and the one added line is PR 6's deliberate `GET /api/v1/public/modules`.
|
||||
|
||||
**Phase 3 — Extract `module-uo`.** Moves out of `website/`: the 8 model directories and their 25
|
||||
tables; the nine UO `utils/` files plus `newsGump.js`; the 13 router/controller files;
|
||||
@@ -473,12 +746,203 @@ Acceptance, all four required:
|
||||
ownership move, which changes no URL. After extraction the core manifest no longer contains UO
|
||||
routes — `module-uo` generates and freezes its own in its own repo.
|
||||
4. **A written `module-rust` dry run** — manifest, mounts, nav entries, one notification stream — not
|
||||
implemented, to prove the contract generalises before more is built on it.
|
||||
implemented, to prove the contract generalises before more is built on it. It lands as
|
||||
`docs/modules/rust-dryrun.md`, where §2.10 already aggregates module documentation; Phase 5's
|
||||
Integration Kit links to it rather than copying it, per the kit's own never-re-specify rule
|
||||
(§2.11).
|
||||
|
||||
#### 2.7.1 Phase 3's shape — settled 2026-08-11
|
||||
|
||||
Measured against `edge` at the close of Phase 2, the surface is **72 server files / ~9,700 lines**,
|
||||
**51 client files / ~3,700 lines**, and **32 of core's 82 server test files**. The counts in the
|
||||
paragraph above were written in Phase 0 against a smaller tree and are superseded by the slice table
|
||||
below.
|
||||
|
||||
**The finding that sets the order: the two halves are independent.** Because §1.2 preserves API URLs
|
||||
exactly, core's client keeps calling `/api/v1/public/shard/status` after that route is served by the
|
||||
module, and a module page calls the same URL while core still serves it. Nothing forces a feature's
|
||||
server and client halves to move together, so the extraction is **server-first, then client**, sliced
|
||||
by feature — which keeps each PR inside one layer and one review's worth of context.
|
||||
|
||||
**Merge order across the two repos: `module-uo` first, then `website`.** The loader's `ownedByCore`
|
||||
probe means a module cannot *load* while core still owns its prefix — but `module-uo`'s own CI never
|
||||
loads it into core, so its PR merges perfectly well beforehand. Taking that order means `edge` serves
|
||||
the feature from core right up to the moment core drops it, and there is never a window where the
|
||||
branch is missing a feature outright. The reverse order would break `edge` at every slice boundary
|
||||
for the length of a review. Verification is unaffected either way: a slice is proved by running the
|
||||
*pair* together locally — the module branch checked out into `website/modules/uo`, the deletion
|
||||
branch checked out in `website/` — before either merges.
|
||||
|
||||
Each slice is one `module-uo` PR (adds), one `website` PR (deletes), and one `docs` PR:
|
||||
|
||||
| # | Slice | Moves |
|
||||
| --- | --- | --- |
|
||||
| 0 | **The bundle skeleton** | `module.json`, both `package.json`s, `server/index.js` registering nothing, the Vite library build + the four shared-dep shims, CI armed, the §5.1 zero-internal-imports check. `website` untouched. |
|
||||
| 1 | **The whole server half** | 40 files / ~9,674 lines, 25 of core's 82 test files, 27 of its 68 tables — every UO model, util, router and controller, `config/shardStreams.js`, `scripts/importSpawnAtlas.js` and the art JSON. **One merge, five commits** (below). |
|
||||
| 2 | **Public pages** | `Shard`, `ShardActivity`, `Rules`, `Atlas`, `AtlasCreature`, `ChampSpawns`, `Market`, `MarketVendor`, `Governors`, `Guilds`, `Houses`, `Leaderboards`, `PlayersOnline`, `VendorSales`, `data/cityCrests.js`, `lib/shardEvents.js`, `lib/useShardFeed.js`, and their public nav rows — under `/uo/*` per §2.8 |
|
||||
| 3 | **Admin + player pages** | `ShardAdmin`, `ShardOps`, `ShardVisibility`, `SpawnAtlas`, `HousesAdmin`, `AdminCharacter(s)`, `PlayerCharacter(s)`, `GameAccounts`, `CharacterSheet`, `CharacterStats`, `ShardAccountActions`, `CreateGameAccountForm`, `useShardFeatures` and the `useShardFlags` feature provider — under `/admin/uo/*` and `/player/uo/*` |
|
||||
| 4 | **De-UO core's copy** | `About`, `Screenshots`, `Website`, `SiteFooter`, `heroLayout`'s defaults, `api/client.js`'s `shard`/`atlas` namespaces, and the comments in `navOverrides.js` — plus the §5.2 CI grep that keeps them out |
|
||||
| 5 | **Close the phase** | `module-uo`'s frozen route manifest and release workflow; `docs/modules/uo/` and `docs/modules/rust-dryrun.md` |
|
||||
|
||||
##### Why the server half cannot be sliced — found 2026-08-11, before writing any of it
|
||||
|
||||
The table above used to run to ten slices, with the server half split five ways by feature. It does
|
||||
not divide, and the reason is that two contract rules compose:
|
||||
|
||||
- **A mount prefix is claimed whole.** `ownedByCore` probes the live tier router and `registerRoutes`
|
||||
validates single-segment prefixes, so `/admin/shard` moves as one unit — and it is a single
|
||||
386-line router carrying 25 routes that span atlas, clilocs, shard-ops, visibility, market *and*
|
||||
account links.
|
||||
- **A model cannot be shared across the boundary** (§5.1), so a model moves with the *last* route
|
||||
that consumes it.
|
||||
|
||||
Take the closure and every prefix is in it:
|
||||
|
||||
```
|
||||
/public/atlas ──shardAtlas── /admin/shard ──shardState,shardEvents,shardMarket── /public/shard
|
||||
│ │
|
||||
shardClilocs,shardLinks uoLinkConfig
|
||||
│ │
|
||||
/player/shard /admin/uo-link
|
||||
```
|
||||
|
||||
Landing any one of the old slices alone would either strand core importing `modules/uo/` — which is
|
||||
precisely what acceptance criterion 2 forbids — or delete routes core is still serving.
|
||||
|
||||
**Giving the admin routes their own prefixes would divide it, and is rejected.** `/admin/atlas` and
|
||||
`/admin/clilocs` alongside a slimmer `/admin/shard` would make the closure fall apart. It also
|
||||
changes API URLs, which §1.2 promises not to do — and not hypothetically: the shipped Android app
|
||||
calls `POST /api/v1/admin/shard/kick`, `/ban`, `/unban`, `/broadcast` and the three `/pages` routes
|
||||
(`data/api/AdminApi.kt`). A prefix rename is a client break, and the API surface is frozen for
|
||||
exactly this reason.
|
||||
|
||||
**So slice 1 is one PR per repo, structured as five commits** along the old slice lines, reviewable
|
||||
one at a time while landing atomically: atlas + clilocs · the live shard · market · account links and
|
||||
the `admin.users.detail` slot · the town-crier leg. The alternative considered was stacked PRs into a
|
||||
per-repo integration branch; it buys PR-level granularity for ten extra PRs and two long-lived
|
||||
branches, and commits give most of the same reading order for none of it.
|
||||
|
||||
**The client half is unaffected** and still slices cleanly: the client registry takes routes per
|
||||
*area*, with no prefix atomicity and no shared models — which is the same asymmetry that let the two
|
||||
halves be separated in the first place.
|
||||
|
||||
**Criterion 1 is a grep over code, not over prose** — see [API §5.2](MODULE_API.md#52-zero-uo-identifiers-in-core-ci-website-repo)
|
||||
for what that means precisely. Core's marketing copy says "shard" in a dozen places, and a literal
|
||||
word grep would have made every one of them a CI failure while proving nothing about the boundary.
|
||||
Slice 4 rewrites that copy anyway, because a core that still reads as a UO site is not the
|
||||
game-agnostic platform this workstream is for — but it is a deliberate piece of work with its own
|
||||
review, not an exemption hidden in a grep pattern.
|
||||
|
||||
**One small gap in the kit, deliberately not closed.** The UO client views import almost exactly the
|
||||
seven §3.4 members — plus `lib/format.js`, a pure leaf formatter. The module **vendors a copy**
|
||||
rather than core adding an eighth member: the kit is closed on purpose, and a function with no
|
||||
props and no layout cannot drift the way a component can. The same is not true of `PublicLayout`,
|
||||
which is why that one is in the kit.
|
||||
|
||||
#### Slice 0 — the bundle skeleton (Module-uo#2, 2026-08-11)
|
||||
|
||||
`module.json`, an entry point taking `(ctx, api)`, the Vite library build, four shims, and both
|
||||
boundary checks. It **registers nothing**, and core is untouched — what it proves is the delivery
|
||||
path itself, before a single UO file moves into it. 29 server tests and 9 client tests, both new.
|
||||
|
||||
Verified against a real core rather than asserted: the module loads, mounts its zero routes, reaches
|
||||
`started` and is published by `/api/v1/public/modules`; its chunk serves from the entry's directory
|
||||
with `Cache-Control: no-cache` while its server source, `module.json` and `package.json` all 404;
|
||||
and in Chrome, under the enforced `script-src 'self'`, the chunk reports every shared dependency
|
||||
**identity-equal** to core's, with zero CSP reports.
|
||||
|
||||
**Three findings, each of which had produced a green build that was wrong.** The first amends the
|
||||
contract and is written up at [API §3.6](MODULE_API.md#36-vite-library-mode-build): `external` and
|
||||
the aliases do not compose, so `external` is now empty and a resolution-time build guard replaces
|
||||
it. The second is that guard's own two failures — hooking `load` (first-wins, so it never ran) and
|
||||
deriving its forbidden list from the alias list (so deleting an alias deleted the guard). Both were
|
||||
found by breaking an alias on purpose and checking the build actually went red, which is the only
|
||||
way a guard's absence is visible.
|
||||
|
||||
The third is about the boundary check itself and generalises past this repo. **`checkImports.js`
|
||||
failed on its own documentation** — the comment naming `require("../../etc/passwd")` as an example
|
||||
of what to catch, and the entry point's comment explaining why a module must never
|
||||
`require('express')`. A check that cannot survive being described is one people stop writing
|
||||
comments around, so it strips comments and template literals with a character walk rather than a
|
||||
regexp (a URL in a string contains a comment opener; a comment contains quotes) and carries its own
|
||||
test suite. The same applies to slice 4's §5.2 grep, which will be read by a codebase that discusses
|
||||
modules constantly.
|
||||
|
||||
#### Slice 1 — the whole server half (Module-uo#3 + website#137, 2026-08-11)
|
||||
|
||||
40 files, ~9,674 lines, 27 of 68 tables, 25 of 82 test files. Core no longer contains anything that
|
||||
knows what a shard is. Three commits per repo, readable in order.
|
||||
|
||||
**The acceptance criterion held exactly.** Core's `routes.manifest.json` goes 228 → 158 public
|
||||
routes, and the 70 that left reappear byte-identical once the module is loaded — proved by generating
|
||||
the manifest against core+module and diffing it against the pre-extraction file: zero missing, zero
|
||||
added, and `routes.guards.json` identical across all 228, so no auth gate moved either.
|
||||
|
||||
**`server/core.js` is the port mechanism and the shape is the finding.** Ported code requires its
|
||||
dependencies at file scope, which runs before `register()` and therefore before any `ctx` exists — so
|
||||
every member of that file is a stable function resolving `ctx` when *called*, and nothing may be
|
||||
destructured off `ctx` at init either, because core is free to hand over a getter. That kept the port
|
||||
to a one-line import change per file instead of a signature change per function. Its consequence:
|
||||
**require order is load-bearing.** A router does `const express = core.express` at its own file
|
||||
scope, so `core.init(ctx)` must run before the first `require` under `router/`, and the module's
|
||||
entry point requires its routers inside `register()` for exactly that reason.
|
||||
|
||||
**The contract grew to 1.1.0**, four members, none of which could be avoided:
|
||||
`ctx.activity.log` (an admin action a module performs belongs in core's *one* audit log — a module
|
||||
with its own is a second place to look, which means a place nobody looks), `ctx.users.getById`,
|
||||
`ctx.site.baseUrl`, and `ctx.middleware.rateLimit` + `accountChangeLimiter`. The rate-limit split is
|
||||
worth restating: a module states its own window and cap because it knows what its endpoints cost, and
|
||||
takes the plumbing from core so there is one `express-rate-limit` in the process and one place a
|
||||
breach is logged.
|
||||
|
||||
**`registerPostHook` is the fourth registry and the last coupling removed** — see
|
||||
[API §2.4](MODULE_API.md#24-api--what-the-module-registers).
|
||||
|
||||
**What was vendored, and what deliberately was not.** `deriveExcerpt` came across as nine lines of
|
||||
pure text handling; core's **sanitiser** sitting beside it did not, because a second copy of a
|
||||
security control diverges silently the moment either is fixed. That is the line: pure leaf helpers
|
||||
may be copied, controls may not.
|
||||
|
||||
**Two defects the extraction exposed, both in core.** The loader matched `CREATE TABLE` against the
|
||||
**raw** fragment, so a schema file whose header says "every CREATE TABLE carries IF NOT EXISTS" was
|
||||
rejected for a prefix violation on a table called `carries` — the same class as slice 0's boundary
|
||||
check failing on its own documentation, and now fixed on both scans by reading split statements. And
|
||||
the atlas art map resolved `../../../db/data`, correct in core and pointing outside `server/` in the
|
||||
module: a path that happens to resolve is exactly what survives a green suite, because the
|
||||
absent-file branch returns `{}` and looks like the normal case. It was caught by the integration run,
|
||||
not by tests.
|
||||
|
||||
**One deliberate behaviour change.** `uoLinkSocket.start()` and the sidecar health probe used to run
|
||||
*after* the listener bound and now run before it, because `onBoot` does. `start()` returns as soon as
|
||||
the reconnecting client is armed, but the probe is a real HTTP call, so it is fired and **not**
|
||||
awaited — an unreachable sidecar must not hold the site closed. Reporting that the bridge is down is
|
||||
diagnostics; being up is not a precondition for serving a page.
|
||||
|
||||
**One test stayed that looked like it should move.** `playerRouteAccess.test.js` guards a real past
|
||||
bug — an admin 403'd off their own characters — through a now-module-owned URL, but the *guarantee*
|
||||
is core's: `/player/*` is role-agnostic self-service. It stays and asserts that through
|
||||
`/player/appeals`. Moving it would have left core with no test of its own tier rule, which is
|
||||
precisely what regressed once before.
|
||||
|
||||
**A note for anyone running core's suite locally: remove `modules/uo` first.** With a module
|
||||
installed the manifest tests fail correctly — core's committed manifest is core-only, and the live
|
||||
stack has the module's routes on it.
|
||||
|
||||
**One thing to know before running a module locally: the loader skips a *symlinked* module directory
|
||||
silently.** `readdirSync(…, { withFileTypes: true }).filter(e => e.isDirectory())` reports a Windows
|
||||
junction as a symlink, so a module linked rather than copied into `modules/` is simply not there,
|
||||
with nothing logged. Not a defect for a real install — `modules/` is a bind mount of real
|
||||
directories (§2.5) — but it is the first thing to check when a module fails to appear.
|
||||
|
||||
**Phase 4 — Delivery.** The admin-panel Modules screen (install, enable, disable, retry, purge,
|
||||
`startup_failed` with its recorded reason) and the Docker-environment path from §2.5. Deliberately
|
||||
last, so loader, packaging, schema and chunk-loading problems are not all being debugged at once.
|
||||
|
||||
**Phase 5 — The Integration Kit.** `RunicGateway/Integration-kit`, the instruction book for building
|
||||
a module for a game that is not UO — the website module, the sidecar and why it exists, and the
|
||||
game-side plugin that feeds it. Scaffolded when Phase 2 lands, written against Phase 3's extraction,
|
||||
finished alongside Phase 4. Full shape and its acceptance test in §2.11.
|
||||
|
||||
### 2.8 SPA URL namespacing — a deliberate break
|
||||
|
||||
**Decision: module pages are namespaced, and old paths are not redirected.** The site is not public
|
||||
@@ -543,6 +1007,59 @@ merely regenerated) and `npm test`, and carries a matching edit to `BACKEND_DESI
|
||||
documentation aggregates in this repo under `docs/modules/<id>/` rather than living in module repos.
|
||||
Conventional Commits, the AI-disclosure trailer, branches cut from an up-to-date `main`.
|
||||
|
||||
### 2.11 The Integration Kit — the instruction book for building a module
|
||||
|
||||
**`RunicGateway/Integration-kit`** — `https://gitea.whitlocktech.com/RunicGateway/Integration-kit.git`,
|
||||
**empty as of 2026-08-10**: no branches, no initial commit, exactly where `Module-uo` was at the start
|
||||
of Phase 0. Its first commit needs the same scaffolding as any other repo here — `README.md`,
|
||||
`LICENSE.md` (GPL-3.0-or-later), `CONTRIBUTING.md` with the AI-disclosure clause, the PR template.
|
||||
|
||||
**Who it is for.** Everything else in this plan is written for someone changing *this* system. The kit
|
||||
is written for someone building a **new** one: a person who wants Runic Gateway to front a game that
|
||||
is not Ultima Online, starting from nothing. It is the only document in the project whose audience is
|
||||
outside the org, and that changes how it is written — it explains and motivates rather than records
|
||||
decisions.
|
||||
|
||||
The job spans all three layers of the data path, which is why it is one book and not a page in each
|
||||
repo:
|
||||
|
||||
1. **The website module.** `module.json`, the server entry point and what `ctx` hands you, the
|
||||
`register*` calls, the schema fragment, the prebuilt client chunk and the shared-dependency rule,
|
||||
packaging and release CI. The bulk of it.
|
||||
2. **The sidecar** — what it is and, more importantly, *why*. The shard is never network-reachable;
|
||||
the shard dials **out** and the sidecar is the listener; the wire is a versioned compatibility
|
||||
contract rather than a build dependency; only the website's backend talks to it. A new game needs
|
||||
its own sidecar or an adapter into the existing one, and neither can be designed by someone who has
|
||||
been handed the message list and none of the reasoning.
|
||||
3. **The game-side plugin** — how a shard feeds the sidecar without ever letting the sidecar stall the
|
||||
game: the bounded drop-oldest queue, the dedicated writer thread, world reads only on the game's
|
||||
own thread. `servuo-plugins/` is the worked example; the constraints are general, and a plugin that
|
||||
ignores them takes the game down when the sidecar wedges.
|
||||
|
||||
**The rule that keeps it from rotting: the kit never re-specifies a contract.**
|
||||
[`MODULE_API.md`](MODULE_API.md) stays normative for the module surface, and
|
||||
[`../link/PLAN.md`](../link/PLAN.md) + [`../link/INTEGRATION.md`](../link/INTEGRATION.md) for the wire
|
||||
protocol. The kit *teaches* — worked examples, the reasoning, the order to do things in, the mistakes
|
||||
that cost time — and links out for the authority. Where it must show a member list it quotes with a
|
||||
pointer, never a copy. A guide that restates a contract diverges from it silently, and a reader who
|
||||
follows the divergent copy gets a module that fails validation for reasons the guide cannot explain.
|
||||
|
||||
**It cannot be written before the contract is proven**, so it trails the implementation rather than
|
||||
leading it:
|
||||
|
||||
- **Scaffolded once Phase 2 lands** — repo, license, CI, and an outline. By then a loader exists to
|
||||
describe and a real module to point at.
|
||||
- **Written against Phase 3's extraction**, using `Module-uo` as the worked example throughout. A kit
|
||||
whose examples are invented is a kit whose examples do not compile.
|
||||
- Phase 3's fourth acceptance criterion, the written `module-rust` dry run, is really this book's
|
||||
first chapter — and doubles as the honest test that the contract generalises past its first module.
|
||||
|
||||
**Acceptance:** someone builds a trivial working module for a second game by following the kit alone,
|
||||
without reading core's source. Until that has happened it is a draft, however finished it looks.
|
||||
|
||||
The org landing page (`RunicGateway/.profile`) and the workspace's `CLAUDE.md` repo table both gain a
|
||||
row for it — when it has content, not while it is an empty repo.
|
||||
|
||||
---
|
||||
|
||||
## Part 3 — Settled decisions
|
||||
@@ -564,3 +1081,8 @@ Conventional Commits, the AI-disclosure trailer, branches cut from an up-to-date
|
||||
| 10 | Client half loads as a prebuilt ESM chunk with React shared via a core global | §2.6 |
|
||||
| 11 | Website work lands on `edge` and reaches `main` as one cutover at the end | §2.9 |
|
||||
| 12 | The module repo is `RunicGateway/Module-uo`; the module id is `uo` | §2.3 |
|
||||
| 13 | `RunicGateway/Integration-kit` is the module-builder's instruction book — module + sidecar + game plugin, teaching only, never re-specifying a contract | §2.11 |
|
||||
| 14 | Phase 3 extracts **server-first, then client**, sliced by feature; `module-uo` merges before `website` in each pair | §2.7.1 |
|
||||
| 15 | Criterion 1's grep reads **code, not prose**; core's UO copy is rewritten in its own slice instead | API §5.2, §2.7.1 |
|
||||
| 16 | `module-uo`'s CI checks core out at a **pinned ref** to generate its frozen route manifest | API §5.3 |
|
||||
| 17 | The `module-rust` dry run lands as `docs/modules/rust-dryrun.md`; the Integration Kit links to it | §2.7.1, §2.11 |
|
||||
|
||||
@@ -536,10 +536,20 @@ Three existing behaviors the merge must not disturb:
|
||||
no menu. `pruneNav` applies the gate inside a section and then drops one it
|
||||
leaves empty.
|
||||
|
||||
- **Moderator confinement.** `AdminLayout` restricts moderators to `MOD_PATHS` and
|
||||
redirects them out of anything else. Overrides apply before that filter, so a
|
||||
moderator can still end up with a legitimately short sidebar — but the redirect
|
||||
effect must keep working untouched.
|
||||
- **Moderator confinement.** `AdminLayout` restricts moderators to the rows their
|
||||
role carries and redirects them out of anything else. Overrides apply before
|
||||
that filter, so a moderator can still end up with a legitimately short sidebar
|
||||
— but the redirect effect must keep working untouched.
|
||||
|
||||
Amended by the module system's Phase 2 PR 8: this used to be a hardcoded
|
||||
`MOD_PATHS` allowlist plus a second, differently-worded prefix check in the
|
||||
redirect, and the two had drifted — `/admin/houses` was on the sidebar and not
|
||||
in the redirect, so a moderator who clicked Houses was bounced to Moderation.
|
||||
Both are now derived from each row's own `roles`
|
||||
([`adminNav.js`](../../website/client/src/lib/adminNav.js)), and the redirect
|
||||
derives from the **base** nav rather than the merged one, which is what keeps
|
||||
an override from moving the boundary in either direction. See
|
||||
[`MODULE_SYSTEM.md`](MODULE_SYSTEM.md) §1.4.
|
||||
- **Empty groups.** `AdminLayout` drops groups whose items all filtered out. An
|
||||
override that hides every item in a group must produce no orphaned header.
|
||||
|
||||
|
||||
@@ -781,6 +781,10 @@
|
||||
"method": "POST",
|
||||
"path": "/api/v1/public/contact"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/modules"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/pages/:id/preview/:token"
|
||||
|
||||
@@ -482,7 +482,7 @@ Copy `.env.example` (Compose) or `server/.env.example` (local) and fill in. **`.
|
||||
| `CLIENT_ORIGIN` | `http://localhost:5173` | enables CORS in dev only |
|
||||
| `LOG_LEVEL` / `FILE_LOG_LEVEL` | `info` / `debug` | console / file verbosity |
|
||||
| `LOG_TO_FILE` / `LOG_DIR` / `LOG_FILE` | `true` / `<server>/logs` / `app.log` | log file (bind-mounted to `./logs` in Docker) |
|
||||
| `ANNOUNCE_POLL_MS` | `15000` | how often the news-announcement dispatcher sweeps `announce_jobs` for due/retry legs (town crier + Discord) |
|
||||
| `ANNOUNCE_POLL_MS` | `15000` | how often the news-announcement dispatcher sweeps `announce_job_legs` for due/retry legs (whichever are registered — Discord is core's, the town crier is module-uo's) |
|
||||
| `TOWNCRIER_DURATION_SEC` | `3600` | how long a news post's in-game town-crier message stays up (≤ `86400`) |
|
||||
|
||||
---
|
||||
|
||||
Reference in New Issue
Block a user