feat(modules): the declarative Docker path (phase 4, slice 3) #144

Merged
whitlocktech merged 1 commits from feature/module-docker-path into edge 2026-08-12 13:02:25 +00:00
Member

Phase 4 slice 3 of MODULE_SYSTEM.md §2.7.2 — decision 4. A compose-managed host declares the module set it runs, and the container arrives at it by itself.

MODULES=uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json

741 server tests (+18), 187 client (+5); manifest unchanged at 166 public + 2 internal, OpenAPI byte-identical.

Docs half: RunicGateway/docs#143 (merge that first).

  1. Resolution runs in the server process, in start() between the seed and the require of app.js — the seed is where the host-allowlist setting comes from, and the require is what scans the volume. That buys it the database, so a compose-installed module gets the same source/sha256 an admin install writes and is the same thing on the screen rather than a second feature that looks alike. A bare npm start with MODULES set behaves identically.
  2. <id>@<version>=<url>, with the version written out. The point is that the no-op case needs no network: "already at the declared version" is a module.json read on the volume, so a restart with the network down comes up exactly as it was. Only a missing or different version reaches out — same allowlist, same sha256, same inspect-then-extract, and install() now takes expect: {id, version} so a URL that resolves to another module or version is refused while it is still only a manifest, not after it is unpacked.
  3. A failure is loud and not fatal. An unreachable release host leaves the site serving without that module, or on the version already unpacked — never in a restart loop, which is what refusing to boot means under restart: unless-stopped.
  4. The declaration owns the volume; the row owns whether a module runs. Uninstalling a declared module from the panel returns its files at the next start and leaves it disabled — no tombstone, no precedence rule, because the two are answering different questions. The screen says so, since files reappearing unexplained is an afternoon of debugging.

Which makes the admin screen's §2.4 reconciliation four sources, not three: a declared module that failed to resolve has no row, no directory and nothing mounted, so without listing it the screen is identical to one where nobody asked for it. The declaration gets its own line rather than folding into the status label — a module can be running fine while its declared upgrade is failing, and one label would have to discard one of those facts.

The defect this slice created

Deferring the require('./app') is what makes any of this possible — the loader scans and mounts at require time, so that require is the last moment a module can reach this process. It also moved core's schema ahead of the scan, and the module schema-fragment replay was wired to core's schema. Every installed module silently got no tables.

It passed the whole suite (every suite here stubs either the loader or the pool) and passed the first browser smoke, because this machine's dev database already had the 27 shard_* tables. The only trace was one line — no module scan in this process — skipping schema fragment replay — which is correct output for npm run seed and means the opposite in a booting server. Found by booting against an empty database.

ensureSchema() now takes replayModules: false for the one caller that intends to scan later; server.js replays the fragments itself right after the require; and test/bootOrder.test.js reads server.js and asserts the five steps are in the one order they work in. It is a structural test and claims no more than that — worth having because nothing else here can see this.

What the smoke proved

Real published module-uo v0.3.0, real Gitea release, fresh empty database:

Claim Result
A declared module installs and mounts in the same boot 252,517 bytes, sha256 verified, unpacked, 37 schema statements, started, /api/v1/public/shard/status → 200
A restart at the declared version touches no network already at the declared version 0.3.0, zero requests, module up
Two unresolvable declarations do not stop the site both 404s logged as errors, site listening, uo still serving 0.3.0
Uninstall then restart returns the files and not the module directory back with provenance recorded, row still disabled, /api/v1/public/modules empty, shard routes 404

The new admin line was verified through the live API rather than the browser — the extension returned a 0×0 viewport and CDP screenshot errors for the whole session. GET /admin/modules carries declared / declaredVersion / declaredError in both shapes that matter; the rendering above it is one derived line, covered by the client suite. Stating it plainly: that is the one thing in this slice not proved in a browser.

Also here: docker-compose.yml, .env.example, server/.env.example and the README (a "three ways in, and none of them is a build" table), and the compose comment that still told operators to untar by hand.

AI disclosure

  • This contribution was AI-assisted (Claude Code).

🤖 Generated with Claude Code

Phase 4 slice 3 of [MODULE_SYSTEM.md §2.7.2](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_SYSTEM.md) — decision 4. A compose-managed host declares the module set it runs, and the container arrives at it by itself. ``` MODULES=uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json ``` **741 server tests (+18), 187 client (+5); manifest unchanged at 166 public + 2 internal, OpenAPI byte-identical.** Docs half: **RunicGateway/docs#143** (merge that first). ## The four decisions, all as recommended 1. **Resolution runs in the server process**, in `start()` between the seed and the `require` of `app.js` — the seed is where the host-allowlist setting comes from, and the require is what scans the volume. That buys it the database, so a compose-installed module gets the same `source`/`sha256` an admin install writes and is the same thing on the screen rather than a second feature that looks alike. A bare `npm start` with `MODULES` set behaves identically. 2. **`<id>@<version>=<url>`, with the version written out.** The point is that the no-op case needs no network: "already at the declared version" is a `module.json` read on the volume, so a restart with the network down comes up exactly as it was. Only a missing or different version reaches out — same allowlist, same sha256, same inspect-then-extract, and `install()` now takes `expect: {id, version}` so a URL that resolves to another module or version is refused *while it is still only a manifest*, not after it is unpacked. 3. **A failure is loud and not fatal.** An unreachable release host leaves the site serving without that module, or on the version already unpacked — never in a restart loop, which is what refusing to boot means under `restart: unless-stopped`. 4. **The declaration owns the volume; the row owns whether a module runs.** Uninstalling a declared module from the panel returns its files at the next start and leaves it `disabled` — no tombstone, no precedence rule, because the two are answering different questions. The screen says so, since files reappearing unexplained is an afternoon of debugging. Which makes the admin screen's §2.4 reconciliation **four** sources, not three: a declared module that failed to resolve has no row, no directory and nothing mounted, so without listing it the screen is identical to one where nobody asked for it. The declaration gets its own line rather than folding into the status label — a module can be running fine while its declared upgrade is failing, and one label would have to discard one of those facts. ## The defect this slice created Deferring the `require('./app')` is what makes any of this possible — the loader scans and mounts at require time, so that require is the last moment a module can reach this process. It also moved core's schema **ahead of the scan**, and the module schema-fragment replay was wired to core's schema. **Every installed module silently got no tables.** It passed the whole suite (every suite here stubs either the loader or the pool) and passed the first browser smoke, because this machine's dev database already had the 27 `shard_*` tables. The only trace was one line — `no module scan in this process — skipping schema fragment replay` — which is *correct* output for `npm run seed` and means the opposite in a booting server. Found by booting against an empty database. `ensureSchema()` now takes `replayModules: false` for the one caller that intends to scan later; `server.js` replays the fragments itself right after the require; and `test/bootOrder.test.js` reads `server.js` and asserts the five steps are in the one order they work in. It is a structural test and claims no more than that — worth having because nothing else here can see this. ## What the smoke proved Real published `module-uo` v0.3.0, real Gitea release, fresh empty database: | Claim | Result | | --- | --- | | A declared module installs and mounts in the **same boot** | 252,517 bytes, sha256 verified, unpacked, 37 schema statements, `started`, `/api/v1/public/shard/status` → 200 | | A restart at the declared version touches no network | `already at the declared version 0.3.0`, zero requests, module up | | Two unresolvable declarations do not stop the site | both 404s logged as errors, site listening, `uo` still serving 0.3.0 | | Uninstall then restart returns the files and not the module | directory back with provenance recorded, row still `disabled`, `/api/v1/public/modules` empty, shard routes 404 | The new admin line was verified through the live API rather than the browser — the extension returned a 0×0 viewport and CDP screenshot errors for the whole session. `GET /admin/modules` carries `declared` / `declaredVersion` / `declaredError` in both shapes that matter; the rendering above it is one derived line, covered by the client suite. Stating it plainly: that is the one thing in this slice not proved in a browser. Also here: `docker-compose.yml`, `.env.example`, `server/.env.example` and the README (a "three ways in, and none of them is a build" table), and the compose comment that still told operators to untar by hand. ### AI disclosure - [x] This contribution was AI-assisted (Claude Code). 🤖 Generated with [Claude Code](https://claude.com/claude-code)
wtclaude added 1 commit 2026-08-12 12:58:09 +00:00
feat(modules): the declarative Docker path (phase 4, slice 3)
Some checks failed
PR Checks / bot-install (pull_request) Successful in 23s
PR Checks / client-build (pull_request) Successful in 30s
PR Checks / server-tests (pull_request) Failing after 4m23s
9b16f39a52
MODULES declares the module set a deployment runs, one entry per module as
`<id>@<version>=<install manifest URL>`, and the container arrives at it by
itself (MODULE_SYSTEM.md §2.7.2 decision 4). A module already unpacked at the
declared version is a no-op that makes NO network call, so a restart with the
network down comes up unchanged; anything else goes through install.js — same
allowlist, same sha256, same inspect-then-extract — and install() now takes an
`expect: {id, version}` so a URL resolving to another module or version is
refused while it is still only a manifest.

Resolution runs inside start(), between the seed and the require of app.js: the
seed is where the host allowlist setting comes from, and the require is what
scans the volume. That buys it the database, so a compose-installed module gets
the same provenance columns an admin install writes.

A failure is logged and carried, never fatal — an unreachable release host must
not take the site down. The declaration owns what is on the volume; the row owns
whether a module runs, so uninstalling a declared module returns its files at
the next start and leaves it disabled. The admin list gains that as a fourth
source (declared / declaredVersion / declaredError), because a declared module
that failed to resolve has no row, no directory and nothing mounted.

Deferring the app require moved core's schema ahead of the volume scan, and the
module schema-fragment replay was wired to core's schema — so every installed
module silently got no tables. Invisible to the suite (each one stubs the loader
or the pool) and to a smoke on a database that already had the tables; found by
booting against an empty one. ensureSchema() now takes `replayModules: false`
for the one caller that scans later, server.js replays them itself after the
require, and a bootOrder test pins the five steps in the only order they work in.

741 server tests (+18), 187 client (+5); manifest unchanged at 166 public + 2
internal, OpenAPI byte-identical.

Co-Authored-By: Claude <noreply@anthropic.com>
whitlocktech merged commit 8bc09d8b53 into edge 2026-08-12 13:02:25 +00:00
whitlocktech deleted branch feature/module-docker-path 2026-08-12 13:02:26 +00:00
Sign in to join this conversation.
No description provided.