From 4c9b2e7e9c7f771e6b6ff73b930664e47692f528 Mon Sep 17 00:00:00 2001 From: wtclaude Date: Wed, 12 Aug 2026 02:31:20 -0500 Subject: [PATCH] =?UTF-8?q?docs(website):=20settle=20phase=204's=20shape?= =?UTF-8?q?=20=E2=80=94=20delivery?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 3 closed with core at 158 routes and module-uo releasable; phase 4 is the first phase that is not about the boundary. It answers how a module gets onto a box an operator owns, and how it comes off again. Adds MODULE_SYSTEM.md §2.7.2 recording the measured starting state (the producer side is finished — module-uo's release.yml already publishes the tarball, the install manifest and SHA256SUMS, and module.json declares purge.sql — while core has the installed_modules provenance columns and nothing that fetches, verifies, unpacks or purges), the four org-lead decisions, the five slices, the unpack threat model and the acceptance table. The four decisions: 1. Restart is a button. Install, uninstall and re-enable only take effect at boot; recording a pending change and telling the operator to restart the container contradicts §2.4's "no shell access to the box", so an admin action runs the shutdown and exits, and the supervisor the shipped compose file already declares brings it back. 2. The install source is a pasted manifest URL, not a catalog — a catalog would make core's release cadence decide which modules exist. Safety is the declared sha256 plus an https host allowlist, which also stops the endpoint being an SSRF primitive. 3. Disable dispatches that one module's onShutdown before flipping the guard. As built, disable makes a module invisible rather than stopped — module-uo keeps its sidecar socket open and keeps ingesting into shard_* tables — which is wrong for the case the button exists for. Enable is not the mirror: there is no onBoot re-dispatch and the hooks were never promised re-entrant, so enable flips the row and offers the restart. 4. The declarative Docker set is an environment variable, resolved before the server starts, idempotent and offline-safe: an already-unpacked module at the declared version is a no-op. Amends §2.4 (disable is no longer a pure guard flip) and §2.5 (what "admin selects the module" and "restart" concretely mean). Records re-entrant lifecycle hooks as a second candidate for a future MODULE_API major bump, beside the identity-provider gap the rust dry run found. Co-Authored-By: Claude --- website/MODULE_SYSTEM.md | 122 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 122 insertions(+) diff --git a/website/MODULE_SYSTEM.md b/website/MODULE_SYSTEM.md index 29fbe7a..dd39603 100644 --- a/website/MODULE_SYSTEM.md +++ b/website/MODULE_SYSTEM.md @@ -400,6 +400,13 @@ is a property of the volume and not of a row here. What the row decides is wheth This is also why `routes.manifest.json` can be generated against a dead database. +*(Amended in Phase 4, §2.7.2 decision 3: **disable is not only a guard flip.** It dispatches that one +module's `onShutdown` first, so a module an admin switches off actually stops — releases its sockets, +closes its streams — rather than merely becoming unreachable while it keeps working in the +background. Enable is deliberately not the mirror image: there is no `onBoot` re-dispatch, so +enabling flips the row and the admin screen offers a restart. The claim this section makes is +unaffected — the row still decides only whether a mounted module answers, never what is mounted.)* + **`disabled` is the only state a boot leaves alone.** Every boot resets each non-disabled row to `enabled`, clearing any recorded failure, and the load that follows writes this boot's outcome — `started` or `startup_failed`. Three consequences, all deliberate: @@ -440,6 +447,12 @@ against its `sha256` → unpacked into `modules//` on the volume → `instal **restart**. On boot the loader scans the filesystem (§1.12), validates prefixes, mounts, replays the schema fragment, runs `onBoot`, and each module reaches `started` or `startup_failed`. +*(Made concrete in Phase 4, §2.7.2 decisions 1 and 2: "admin selects the module" is **pasting the URL +of a release's install manifest** — there is no catalog, because a catalog would make core's release +cadence decide which modules exist. And the **restart is a button on the same screen**, not an +instruction to go and restart the container: it runs `moduleLifecycle.shutdown()` and exits, and the +supervisor the shipped compose file already declares brings the process back.)* + Nothing is compiled at any point. The operator restarts; they never build. **Uninstall** (default, non-destructive): row set to `disabled`, directory removed, restart. The @@ -1368,12 +1381,121 @@ Six slices, 2026-08-11. Core is 158 routes and knows nothing about any game; mod **Phase 4 — Delivery.** The admin-panel Modules screen (install, enable, disable, retry, purge, `startup_failed` with its recorded reason) and the Docker-environment path from §2.5. Deliberately last, so loader, packaging, schema and chunk-loading problems are not all being debugged at once. +Its shape, and the four decisions it turned on, are in §2.7.2 below. **Phase 5 — The Integration Kit.** `RunicGateway/Integration-kit`, the instruction book for building a module for a game that is not UO — the website module, the sidecar and why it exists, and the game-side plugin that feeds it. Scaffolded when Phase 2 lands, written against Phase 3's extraction, finished alongside Phase 4. Full shape and its acceptance test in §2.11. +#### 2.7.2 Phase 4's shape — settled 2026-08-12 + +Phase 4 is the first phase that is not about the *boundary*. Phases 1–3 answered "can a module exist +and can core live without one"; this one answers "how does a module get onto a box an operator owns, +and how do they get it off again", which is the whole point of §1.14's WordPress constraint. + +**Measured starting state.** The producer side is finished and the consumer side is empty. +`module-uo`'s `release.yml` already publishes the three assets an installer wants — a +`module-uo-.tar.gz`, an install manifest `module-uo-.json` carrying +`{id, name, version, coreApi, artifact, url, sha256, size}`, and a `SHA256SUMS` — and `module.json` +already declares `purge: server/db/purge.sql`. Core already has the `installed_modules` columns +`source` and `sha256` (§2.4 carved them out for exactly this and nothing has ever written them +non-null), the live `stateGuard`, and `modules.model.js`'s `enable`/`disable`/`remove`. What core has +**none** of: any code that fetches, verifies, unpacks, removes or purges anything; any admin route; +any client screen; and any resolution of a declared module set at container start — +`docker-compose.yml`'s own comment still tells the operator to untar by hand and restart. + +##### Four decisions, settled by the org lead + +**1. Restart is a button, not an instruction.** Install, uninstall and re-enable only take effect at +boot, because §1.12 mounts synchronously from the filesystem and the loader is `load()`-once. Rather +than record a pending change and tell the operator to go and restart the container — which +contradicts §2.4's "recoverable from the admin panel, with no shell access to the box" — an admin +action runs `moduleLifecycle.shutdown()` and then exits the process cleanly, and the supervisor +brings it back. The shipped `docker-compose.yml` already declares `restart: unless-stopped` on `app`, +so on the supported deployment this is simply true. It is not universally true — a bare +`npm start --prefix server` does not come back — so the screen says what it is about to do and the +endpoint is not something a stray click reaches. + +**2. The install source is a pasted manifest URL, not a catalog.** The admin pastes the URL of a +release's `-.json`; core fetches it, downloads the `artifact` it names, verifies the +`sha256` it declares, and unpacks. There is no index to run and no core release needed to make a new +module installable — which is the WordPress model exactly, and the only shape that lets a module +outside this org be installed at all. The safety is not the catalog, it is the sha256 in the manifest +(module-uo's CHANGELOG already names it "the trust anchor") plus a **host allowlist**: `https` only, +against a configurable list defaulting to the Gitea host. This endpoint installs code that will run +in the server process on the next boot; it is admin-only, rate-limited and audit-logged, and the +allowlist is what stops a pasted URL from being an SSRF primitive as well. + +**3. Disable stops the module; enable asks for a restart.** As Phase 2 built it, `disable` flips one +field on the in-memory record and the `stateGuard` starts answering 404 — the module becomes +*invisible*, not *stopped*. For module-uo, concretely, `boot.js`'s `uoLinkSocket` stays connected to +the sidecar and keeps ingesting shard events into `shard_*` tables, and `shardBroadcast`'s SSE +streams stay open, because `onShutdown` is dispatched only from the signal handler. That is +acceptable for "I do not want this feature on the site today" and wrong for the case the button +exists for — "this module is misbehaving and I have no shell". So disable now dispatches **that one +module's** `onShutdown` before flipping the guard, and is a real kill switch. + +Enable cannot be its mirror. There is no `onBoot` re-dispatch path, and the contract has never +promised the hooks are re-entrant — no module author has written `onBoot` to be safe to run twice in +one process. Enable therefore flips the row and the screen offers the restart button from decision 1. +Making the hooks re-entrant is the better end state and is recorded as a candidate for a future +`MODULE_API_VERSION` major bump, beside the identity-provider gap §2.7's dry run found; it is not +worth a rewrite of every module's `boot.js` to save one restart. + +This is a **change to §2.4**, which described both transitions as pure row-and-guard flips. §2.4's +underlying claim survives untouched: the row still never decides what is *mounted*, only whether a +mounted module answers. + +**4. The Docker path is an environment variable.** §2.5 owes "a declarative module set resolved at +container start from the mounted volume, so a compose-managed host is not driven by clicking". That +set is declared in the compose file — the file such a host already edits and version-controls — +rather than in a second config file on the volume. Resolution runs before the server starts and is +**idempotent and offline-safe**: a module already unpacked at the declared version is a no-op, so a +restart with the network down brings the site up exactly as it was. Only a missing or +wrong-version module reaches out, and it reaches out through the same fetch-verify-unpack path the +admin panel uses. + +##### The slices + +Slice 0 is independent of the rest and lands first; 1–4 are ordered. Unlike Phase 3, most of this is +core-only, so the two-repo merge dance does not apply — `module-uo` is touched only by slice 0. + +| # | Slice | Repos | +| --- | --- | --- | +| 0 | **SonarQube for `module-uo`** — `sonar-project.properties` (project key `Module-uo`) plus the scan workflow, mirroring `website`'s. The module is 40 server + 35 client files of extracted code that has never been scanned. | `Module-uo`, `docs` | +| 1 | **The install service and the admin API** — fetch, verify, hardened unpack, remove, purge; `/api/v1/admin/modules` with list, install, enable, disable, uninstall, purge and restart; the per-module `onShutdown` dispatch decision 3 requires; the host allowlist. | `website`, `docs` | +| 2 | **The Modules screen** — `ModulesAdmin.jsx`, its nav row, its API bindings, and the §7.7 browser smoke against a real install of `module-uo` from a real release URL. | `website`, `docs` | +| 3 | **The declarative Docker path** — container-start resolution of decision 4's variable, the compose and image changes it needs, and the operator documentation. | `website`, `docs` | +| 4 | **Close the phase** — `BACKEND_DESIGN.md` §3, the `api-route-inventory.json` mirror, `docs/modules/uo/`'s install instructions, and the acceptance table. | `docs`, `website` | + +##### The unpack is the dangerous part, and it is not the download + +The download is a solved problem: an allowlisted `https` host, a declared `sha256`, and a size cap. +Unpacking is where the archive gets to choose filenames, and a tar entry may name an absolute path, +escape upward with `..`, be a symlink or hardlink pointing anywhere on the filesystem, or expand to +far more than it appeared to be. Core writes into a **bind-mounted directory it shares with the +host** (Phase 2 PR 9), so an escape is not confined to the container's own filesystem. + +The extractor therefore rejects rather than sanitises — an archive that needs correcting is an +archive that should not be trusted — on: any absolute path, any `..` segment, any entry that is not a +regular file or a directory (so no symlinks, hardlinks, devices or FIFOs), more than one top-level +directory, a top-level directory whose name is not the manifest's `id`, and a total unpacked size or +entry count over its cap. Unpacking goes to a temporary directory beside the target and is moved into +place only once the whole archive has been accepted, so a rejected or interrupted install never +leaves a half-module on the volume for the next boot's scan to find. `tar` is a real dependency +rather than a hand-rolled parser: the format has enough shape (long-name extensions, PAX headers, +sparse entries) that parsing it is exactly the kind of code this list is trying to defend against. + +##### What Phase 4 must prove + +| # | Criterion | How | +| --- | --- | --- | +| 1 | A module installs, starts and serves with no shell access and nothing built | Admin panel install from a release URL, restart, module `started`, its pages render | +| 2 | Uninstall leaves the data, purge removes it | Uninstall then reinstall recovers the rows; purge then reinstall does not | +| 3 | A hostile archive cannot write outside `modules//` | Unit tests over the rejection list above, each with a real crafted tar | +| 4 | A compose-declared module resolves at container start, offline | Container boots with the network down and an already-unpacked module, and comes up unchanged | + ### 2.8 SPA URL namespacing — a deliberate break **Decision: module pages are namespaced, and old paths are not redirected.** The site is not public