feat(modules): the declarative Docker path (phase 4, slice 3)
MODULES declares the module set a deployment runs, one entry per module as
`<id>@<version>=<install manifest URL>`, and the container arrives at it by
itself (MODULE_SYSTEM.md §2.7.2 decision 4). A module already unpacked at the
declared version is a no-op that makes NO network call, so a restart with the
network down comes up unchanged; anything else goes through install.js — same
allowlist, same sha256, same inspect-then-extract — and install() now takes an
`expect: {id, version}` so a URL resolving to another module or version is
refused while it is still only a manifest.
Resolution runs inside start(), between the seed and the require of app.js: the
seed is where the host allowlist setting comes from, and the require is what
scans the volume. That buys it the database, so a compose-installed module gets
the same provenance columns an admin install writes.
A failure is logged and carried, never fatal — an unreachable release host must
not take the site down. The declaration owns what is on the volume; the row owns
whether a module runs, so uninstalling a declared module returns its files at
the next start and leaves it disabled. The admin list gains that as a fourth
source (declared / declaredVersion / declaredError), because a declared module
that failed to resolve has no row, no directory and nothing mounted.
Deferring the app require moved core's schema ahead of the volume scan, and the
module schema-fragment replay was wired to core's schema — so every installed
module silently got no tables. Invisible to the suite (each one stubs the loader
or the pool) and to a smoke on a database that already had the tables; found by
booting against an empty one. ensureSchema() now takes `replayModules: false`
for the one caller that scans later, server.js replays them itself after the
require, and a bootOrder test pins the five steps in the only order they work in.
741 server tests (+18), 187 client (+5); manifest unchanged at 166 public + 2
internal, OpenAPI byte-identical.
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -30,6 +30,7 @@ const settings = require('../../../model/settings/settings.model')
|
||||
const loader = require('../../../modules/loader')
|
||||
const lifecycle = require('../../../modules/lifecycle')
|
||||
const install = require('../../../modules/install')
|
||||
const declared = require('../../../modules/declared')
|
||||
// A namespace import, like every other require in this file, and not
|
||||
// `const { runPurge } = …`: destructuring at require time captures the function
|
||||
// rather than the module, which makes it the one dependency here that cannot be
|
||||
@@ -42,8 +43,9 @@ const log = require('../../../utils/logger')('admin-modules')
|
||||
// The allowlist setting. Seeded from MODULE_SOURCE_HOSTS on first boot and
|
||||
// admin-managed from then on (decision 6) — db/seed.js writes it once and never
|
||||
// overwrites it, so changing the variable later does not silently reach in and
|
||||
// undo an operator's choice.
|
||||
const HOSTS_KEY = 'module_source_hosts'
|
||||
// undo an operator's choice. The key itself lives in install.js, which is now
|
||||
// read by boot-time declared-set resolution as well as by this controller.
|
||||
const HOSTS_KEY = install.HOSTS_SETTING
|
||||
|
||||
// A hostname, not a URL: no scheme, no path, no port, no wildcard. Deliberately
|
||||
// strict — every character allowed here is a character that can appear in the
|
||||
@@ -57,24 +59,29 @@ async function allowedHosts() {
|
||||
/**
|
||||
* One module, as the admin screen needs it.
|
||||
*
|
||||
* Three sources have to be reconciled, and which one answers which question is
|
||||
* Four sources have to be reconciled, and which one answers which question is
|
||||
* the whole of §2.4:
|
||||
*
|
||||
* - the ROW says what the operator decided and what the last boot recorded;
|
||||
* - the LOADER says what is mounted and answering right now;
|
||||
* - the VOLUME says whether there is still a directory there at all.
|
||||
* - the VOLUME says whether there is still a directory there at all;
|
||||
* - the DECLARATION (slice 3) says what this container's environment asks for,
|
||||
* which is the only one of the four an admin cannot change from this screen.
|
||||
*
|
||||
* They can legitimately disagree, and the screen has to show that rather than
|
||||
* pick a winner. A row `enabled` with a loader state of `disabled` is a module
|
||||
* the operator has just switched back on and which is waiting for a restart —
|
||||
* exactly the case decision 3 creates, and it would be a lie to render it as
|
||||
* either "running" or "off".
|
||||
* either "running" or "off". A declared module with no row and no directory is
|
||||
* the newest of those disagreements: MODULES asked for it and resolution could
|
||||
* not get it, so the screen carries the reason rather than showing nothing.
|
||||
*/
|
||||
function present(row, live, onVolume) {
|
||||
function present(row, live, onVolume, declaration = null) {
|
||||
const id = row ? row.id : live ? live.id : declaration.id
|
||||
return {
|
||||
id: row ? row.id : live.id,
|
||||
name: row ? row.name : live.name,
|
||||
version: row ? row.version : live.version,
|
||||
id,
|
||||
name: row ? row.name : live ? live.name : id,
|
||||
version: row ? row.version : live ? live.version : null,
|
||||
// What the database records.
|
||||
state: row ? row.state : null,
|
||||
failureStage: row ? row.failureStage : (live && live.stage) || null,
|
||||
@@ -93,7 +100,16 @@ function present(row, live, onVolume) {
|
||||
capabilities: live ? live.capabilities : [],
|
||||
// What is on the volume.
|
||||
onVolume,
|
||||
canPurge: onVolume && Boolean(install.purgeFile(row ? row.id : live.id)),
|
||||
canPurge: onVolume && Boolean(install.purgeFile(id)),
|
||||
// What the environment declares. `declaredVersion` is what MODULES pins, not
|
||||
// what is installed — they differ exactly while a resolution is failing, and
|
||||
// `declaredError` says why. Uninstalling a declared module from this screen
|
||||
// removes its directory and disables its row; the next boot puts the
|
||||
// directory back and leaves the row disabled, so the screen says so rather
|
||||
// than letting the files reappear unexplained.
|
||||
declared: Boolean(declaration),
|
||||
declaredVersion: declaration ? declaration.version : null,
|
||||
declaredError: declaration && declaration.action === 'failed' ? declaration.message : null,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -107,18 +123,31 @@ async function list(req, res) {
|
||||
// `npm run seed` never gets here, but a test harness might.
|
||||
const live = loader.isLoaded() ? loader.list() : []
|
||||
const byId = new Map(live.map((m) => [m.id, m]))
|
||||
const declaredById = new Map(declared.state().map((d) => [d.id, d]))
|
||||
|
||||
const seen = new Set()
|
||||
const out = []
|
||||
for (const row of rows) {
|
||||
seen.add(row.id)
|
||||
out.push(present(row, byId.get(row.id) || null, install.isInstalled(row.id)))
|
||||
out.push(
|
||||
present(row, byId.get(row.id) || null, install.isInstalled(row.id), declaredById.get(row.id)),
|
||||
)
|
||||
}
|
||||
// A directory on the volume that has no row yet — a hand-placed install
|
||||
// before its first boot. It has to be listed, or the screen would show
|
||||
// nothing for a module whose routes are already being served.
|
||||
for (const m of live) {
|
||||
if (!seen.has(m.id)) out.push(present(null, m, true))
|
||||
if (!seen.has(m.id)) {
|
||||
seen.add(m.id)
|
||||
out.push(present(null, m, true, declaredById.get(m.id)))
|
||||
}
|
||||
}
|
||||
// A module MODULES declares that has neither. Resolution failed and left
|
||||
// nothing behind — the case an operator most needs told, because from the
|
||||
// screen's other three sources it is indistinguishable from never having
|
||||
// asked for it.
|
||||
for (const d of declaredById.values()) {
|
||||
if (!seen.has(d.id)) out.push(present(null, null, install.isInstalled(d.id), d))
|
||||
}
|
||||
|
||||
return res.json({ modules: out, sourceHosts: await allowedHosts() })
|
||||
|
||||
Reference in New Issue
Block a user