diff --git a/website/BACKEND_DESIGN.md b/website/BACKEND_DESIGN.md index 97b600b..6a57392 100644 --- a/website/BACKEND_DESIGN.md +++ b/website/BACKEND_DESIGN.md @@ -690,6 +690,14 @@ who fixes the cause needs no admin-panel visit), a running module can never disp and `disabled` — the one operator *decision* rather than outcome — survives untouched. A re-install or upgrade refreshes the metadata and leaves `state` alone. +The write happens in one place, `src/modules/lifecycle.js`, on the boot path after `ensureSchema()` +and before the listener binds: it resets the last boot's outcomes, writes a row for every module found +on the volume (with NULL provenance for a hand-placed directory), marks any row whose directory is +**gone** `startup_failed`, and then runs each surviving module's `onBoot` and records what happened. A +`disabled` row is guarded, not booted, and never has its failure re-recorded — an outcome must not +overwrite the operator's decision. Every one of those writes is individually caught: a row that will +not update is worse reporting, never a failed boot. + Design of record: [`MODULE_SYSTEM.md`](MODULE_SYSTEM.md) §2.4; the loader's obligations are [`MODULE_API.md`](MODULE_API.md) Part 4. diff --git a/website/MODULE_API.md b/website/MODULE_API.md index 3313c90..61f3346 100644 --- a/website/MODULE_API.md +++ b/website/MODULE_API.md @@ -258,13 +258,59 @@ It runs **after** `ensureSchema()` (so the module's own tables exist) and after and **before** the HTTP listener binds — a module that must not serve traffic before it has warmed its cache gets that for free. -`onShutdown` runs before the server closes, in reverse registration order, with a 5-second budget -per module; exceeding it is logged and skipped rather than hanging the process. +`onShutdown` runs before anything core owns is closed — the database pool, the push dispatcher and +the SSE fan-out are all still open, because a module's `onShutdown` is the only chance it gets to +flush through them. Reverse registration order, with a 5-second budget per module; exceeding it is +logged and the hook abandoned rather than hanging the process. Abandoned, not cancelled: nothing can +stop a promise that is still running, but the process is exiting anyway and the alternative is a host +where `systemctl stop` waits for SIGKILL. -Both are individually try/caught. An `onBoot` that throws marks that module `startup_failed` -(§4.4) and the site still comes up — its routes stay mounted but its dispatch guard rejects them -with 503, because a module that failed to warm up serving half-initialised data is worse than a -module that says it is down. +**`onBoot` has no budget, deliberately.** Shutdown races the process being killed; boot does not. A +slow `onBoot` delays the listener binding, which is the guarantee two paragraphs up rather than a +problem to be timed out, and core's own boot steps are awaited exactly the same way. + +Both hooks are optional, and both are individually try/caught. An `onBoot` that throws marks that +module `startup_failed` (§4.4) and the site still comes up — its routes stay mounted but its dispatch +guard rejects them with 503, because a module that failed to warm up serving half-initialised data is +worse than a module that says it is down. A module with no `onBoot` at all still reaches `started`: +having nothing to warm up is not the same as never having started, and the row has to agree with the +guard about whether the module is serving. A module whose `onBoot` threw gets **no** `onShutdown` — it +is part-way through a warm-up it never finished, and handing it a half-built world to tear down is +worse than not closing cleanly. + +`onBoot` receives the same frozen `ctx` object `register()` was given, not a second one built to look +like it. + +**What a boot does to `installed_modules`** (`MODULE_SYSTEM.md` §2.4). The dispatch is the second half +of a reconcile, and the order of its four steps is the design: + +1. Clear the last boot's outcomes, so what is on display afterwards is what *this* boot did. + `disabled` rows are left alone — that is an operator decision, not an outcome. +2. Write a row for every module found on the volume, with null provenance if it has none. A directory + placed on the volume by hand is a supported install (§2.5 of the design of record) and without a + row it could be neither disabled nor reported. +3. Mark any row whose directory is **not** on the volume `startup_failed` (stage `require`). Step 1 + has just reset it to `enabled`, and a row claiming to be enabled for a module that is not there is + the one state that is simply untrue. A plain uninstall leaves `disabled`, which step 1 never + touches, so this catches only a directory deleted by hand. +4. Write down the outcome each module already carries — disabled by the operator, or failed during + load or schema replay, both of which happen before the database is reachable — and only then + dispatch `onBoot`. + +**The operator's switch wins over everything, including a failure.** A module whose row says +`disabled` is guarded (§4.5), is not booted, and does **not** have its failure re-recorded: +overwriting a deliberate `disabled` with an outcome would silently switch it back on at the next +boot. + +**A bookkeeping failure is not a boot failure.** Every database write in the reconcile is individually +caught. A row that will not update is bad — the admin panel shows the wrong thing — but it is +strictly less bad than a site that will not start, and it must not stop the modules behind it from +booting. + +Dispatch and reconcile live in `server/src/modules/lifecycle.js`, not in the loader: `routeManifest.js` +and `swagger.js` both require `app.js` against a dead pool (§4.1), so the loader may not reach the +database. The two halves meet at exactly one place — `loader.setState()` — so the in-memory record +the dispatch guard reads and the row the admin panel reads are moved together and cannot disagree. ### 2.6 Schema fragments @@ -584,6 +630,26 @@ Two sub-cases differ, and the difference matters: The second is what keeps the URL surface deterministic and generatable: `routes.manifest.json` must not depend on whether a module's boot hook happened to succeed on the machine that generated it. +**Every failure is recorded against the step that produced it**, in `failure_stage`, so the admin +panel can say *where* a module broke and not only what the message was. The stages are §4.3's seven +validation steps plus `boot`: + +| Stage | The step that failed | +| --- | --- | +| `manifest` | `module.json` unparseable, an unknown key, a bad or mismatched `id`, no `version` | +| `core_api` | `coreApi` missing, or not satisfied by `MODULE_API_VERSION` | +| `mounts` | a malformed prefix, or one already owned by core or another module | +| `extensions` | a declared slot that does not exist | +| `schema` | a fragment breaking a §2.6 rule at load, or a statement the database rejected at replay | +| `require` | the entry point threw, or did not export a function — also a row whose directory is gone | +| `register` | `register()` threw, a claim was malformed, or what it registered ≠ what it declared | +| `boot` | `onBoot` threw | + +The four steps that share one function label themselves; the rest are inferred from how far the load +had got, and an unlabelled throw is recorded against the step that was running rather than guessed +at. Every non-failing transition clears both the stage and the reason, so a running module can never +show the failure it had two boots ago. + ### 4.5 The disabled guard A module disabled in `installed_modules` is *mounted and guarded*, never unmounted — a one-line diff --git a/website/MODULE_SYSTEM.md b/website/MODULE_SYSTEM.md index 92f99bf..dd77124 100644 --- a/website/MODULE_SYSTEM.md +++ b/website/MODULE_SYSTEM.md @@ -403,6 +403,13 @@ A re-install or an upgrade refreshes `name`/`version`/provenance and deliberatel alone: upgrading an enabled module must not silently switch it off, and re-installing a disabled one must not silently switch it on. +**A row whose directory is gone is marked `startup_failed`** (stage `require`, reason "module +directory not present on the volume"), settled with PR 5. The boot reset above has just moved it to +`enabled`, and a row claiming to be enabled for a module that is not on the volume is the one state +that is simply untrue — it would be read that way by the admin panel and by +`GET /api/v1/public/modules` alike. This catches only a directory deleted by hand: an uninstall +leaves the row `disabled`, which the reset never touches. + ### 2.5 Install, uninstall, purge Modules live on a **mounted volume**, not in the image — the same treatment `uploads` already gets in @@ -484,7 +491,7 @@ too (API §7.2). Exit criterion: `routes.manifest.json` diff is zero lines and every existing test passes. If Phase 2 changes one URL, it is wrong. -**Progress: PRs 1-4 done.** +**Progress: PRs 1-5 done.** - **PR 1** — `installed_modules` and the state machine, with the stored shape and the boot rules settled in §2.4 above. @@ -533,6 +540,27 @@ client can see: **884 tests pass** and `routes.manifest.json` is unchanged at 22 lines of OpenAPI that do move are the retry endpoint's summary and its `leg`, which is no longer a fixed enum because the leg set is whatever has been registered. +- **PR 5** — boot/shutdown dispatch and the `installed_modules` reconcile, `src/modules/lifecycle.js`. + `api.onBoot`/`api.onShutdown` stop throwing, `server.js` gains one call on each side, and the + §2.4 machine finally runs against real outcomes — which is what makes §4.5's `disabled` 404 leg + reachable for the first time. Four decisions landed with it, all recorded in + [`MODULE_API.md`](MODULE_API.md) §2.5 and §4.4: **the loader classifies its failures** by §4.3 step, + so `failure_stage` says where a module broke instead of being a column nothing filled; **a row whose + directory is gone is marked failed** rather than left claiming `enabled` (§2.4 above); **core's eight + UO boot call sites stay in `server.js`** until Phase 3, because unlike a registered announce leg a + boot call site already has somewhere to live and moving it now would be extraction done early in a + phase whose exit criterion is that nothing changes; and **`onBoot` gets no timeout** — shutdown races + a SIGKILL and boot does not, and a slow `onBoot` delaying the listener is the contract's promise to + a module that must warm up before it serves. + + The dispatch lives outside the loader for the reason the schema replay does: the loader is required + by `app.js` against a dead pool, and this half is database-first. They meet at one function, + `loader.setState()`, so the in-memory record the dispatch guard reads and the row the admin panel + reads cannot drift apart. + + Still nothing on the volume: **900 tests pass**, `routes.manifest.json` is unchanged at 229 routes + and the OpenAPI spec regenerates byte-identical. + **Phase 3 — Extract `module-uo`.** Moves out of `website/`: the 8 model directories and their 25 tables; the nine UO `utils/` files plus `newsGump.js`; the 13 router/controller files; `scripts/importSpawnAtlas.js` and `db/spawnAtlas.art.json`; `usersShard.controller.js` **minus