1 Commits

Author SHA1 Message Date
3add0063bf feat(modules): installed_modules and the module state machine
All checks were successful
PR Checks / bot-install (pull_request) Successful in 17s
PR Checks / client-build (pull_request) Successful in 27s
PR Checks / server-tests (pull_request) Successful in 1m35s
Phase 2 PR 1 of the module system (docs/website/MODULE_SYSTEM.md 2.7). The
table and the state machine only: no loader, no routes, no boot wiring, so
nothing an operator or a client can see changes and the route manifest diff
is zero lines.

The five states of 2.4 live in one `state` column: installed -> enabled ->
started, with disabled and startup_failed as the recoverable ones. The row is
a record of what happened, never the source of truth for what is mounted --
the loader scans the filesystem at require time, before the database is
reachable (MODULE_API.md 4.1), which is what keeps routes.manifest.json
generatable against a dead database.

Two rules the model owns and the boot path will lean on:

- Every boot resets each non-disabled row to `enabled` and clears its
  recorded failure, so a startup_failed module is retried on the next restart
  and a fixed one recovers with no admin-panel visit. `disabled` is the one
  operator decision rather than outcome, so it survives untouched -- and a
  disabled module's failure is a no-op, never a re-enable.
- A failure carries the stage it happened at, and every non-failing
  transition clears it, so a running module can never show a stale reason.

An illegal move throws instead of writing a row that misrepresents the state,
except on the two boot-path softenings noted above, because one module's
failure must never become everybody's.

22 model tests over an in-memory fake; the SQL and the DDL were round-tripped
against a real MariaDB separately.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-10 06:35:57 -05:00
4 changed files with 580 additions and 0 deletions

View File

@@ -1318,6 +1318,49 @@ CREATE TABLE IF NOT EXISTS shard_atlas_pending (
CONSTRAINT chk_shard_atlas_pending_singleton CHECK (id = 1)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Installed modules (module system, docs/website/MODULE_SYSTEM.md §2.4). One row
-- per module the operator has installed onto the modules volume, keyed by the
-- module id from its module.json — the same id that names the directory, the URL
-- segment and the client registry key.
--
-- This table is a RECORD of what happened, never the source of truth for what is
-- mounted: the loader scans the filesystem at require time, before the database is
-- reachable (MODULE_API.md §4.1), so the URL surface is a property of the volume
-- and not of a row here. What the row decides is whether a mounted module answers
-- (`disabled` ⇒ its guard 404s, §4.5) and what the admin panel shows after a
-- failure.
--
-- `state` is the §2.4 machine in one column: installed → enabled → started, with
-- disabled and startup_failed as the recoverable states. `installed` is the
-- transient state between an install writing the row and the restart that starts
-- it. On every boot each non-disabled row is reset to `enabled` and re-attempted
-- (so a fixed module recovers on restart, with no panel visit needed), then the
-- load outcome writes `started` or `startup_failed`. Only `disabled` survives a
-- boot untouched — it is the operator's decision, not an outcome.
--
-- failure_stage/failure_reason are §4.4's recorded reason, one of the seven
-- validation steps of §4.3 plus `boot`. Both are cleared by every transition that
-- is not a failure, so a stale reason can never be shown against a running module.
--
-- source/sha256 are install provenance (§2.5): the release the bundle came from and
-- the digest that was verified before unpacking. Both NULL for a directory placed
-- on the volume by hand, which stays supported.
CREATE TABLE IF NOT EXISTS installed_modules (
id VARCHAR(32) NOT NULL PRIMARY KEY, -- module.json id; names the directory
name VARCHAR(128) NOT NULL, -- human label for the admin Modules screen
version VARCHAR(32) NOT NULL, -- module.json version (semver)
state ENUM('installed','enabled','disabled','started','startup_failed')
NOT NULL DEFAULT 'installed',
failure_stage VARCHAR(32) NULL, -- manifest|core_api|mounts|extensions|schema|require|register|boot
failure_reason TEXT NULL, -- the recorded reason, shown in the admin panel
source VARCHAR(255) NULL, -- release URL the bundle came from
sha256 CHAR(64) NULL, -- verified bundle digest
installed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
started_at DATETIME NULL, -- last successful start
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_installed_modules_state (state)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Migrations for databases created before the wiki upgrade. Each statement uses
-- IF NOT EXISTS so re-running on every boot is a harmless no-op. New installs get
-- these columns from the CREATE TABLE above; existing installs get them here.

View File

@@ -0,0 +1,59 @@
const { query } = require('../../utils/db')
// SQL for installed_modules — the module system's record of what is installed and
// what happened to it on the last boot (db/schema.sql, docs/website/MODULE_SYSTEM.md
// §2.4). Rows are keyed by module id. All state rules live in modules.model.js;
// this file only moves rows.
const COLS = `id, name, version, state, failure_stage, failure_reason,
source, sha256, installed_at, started_at, updated_at`
const listAll = () => query(`SELECT ${COLS} FROM installed_modules ORDER BY id`)
const getOne = (id) => query(`SELECT ${COLS} FROM installed_modules WHERE id = ?`, [id])
// Write (or refresh) the row for an installed module. A re-install or an upgrade
// updates the metadata and deliberately leaves `state` alone: upgrading an enabled
// module must not silently disable it, and re-installing a disabled one must not
// silently switch it back on. A brand-new row lands in `installed`, the transient
// state the next restart resolves.
const upsert = ({ id, name, version, source, sha256 }) =>
query(
`INSERT INTO installed_modules (id, name, version, source, sha256, state)
VALUES (?, ?, ?, ?, ?, 'installed')
ON DUPLICATE KEY UPDATE
name = VALUES(name),
version = VALUES(version),
source = VALUES(source),
sha256 = VALUES(sha256)`,
[id, name, version, source ?? null, sha256 ?? null],
)
// Move one row to a new state. `failureStage`/`failureReason` are written on every
// call — a non-failing transition passes nulls, which is what clears a stale reason
// off a module that has since come up. `stampStarted` sets started_at to now.
const setState = ({ id, state, failureStage = null, failureReason = null, stampStarted = false }) =>
query(
`UPDATE installed_modules
SET state = ?, failure_stage = ?, failure_reason = ?
${stampStarted ? ', started_at = CURRENT_TIMESTAMP' : ''}
WHERE id = ?`,
[state, failureStage, failureReason, id],
)
// Boot reset: every row the operator has not disabled goes back to `enabled` with
// no failure recorded, so the load that follows writes this boot's outcome rather
// than leaving the last one on display. `disabled` is untouched — it is a decision,
// not an outcome.
const resetForBoot = () =>
query(
`UPDATE installed_modules
SET state = 'enabled', failure_stage = NULL, failure_reason = NULL
WHERE state <> 'disabled'`,
)
// Drop the row entirely. Only the explicit purge does this (§2.5); a plain
// uninstall disables the module and keeps its row and its data.
const remove = (id) => query('DELETE FROM installed_modules WHERE id = ?', [id])
module.exports = { listAll, getOne, upsert, setState, resetForBoot, remove }

View File

@@ -0,0 +1,183 @@
// The module state machine (docs/website/MODULE_SYSTEM.md §2.4, MODULE_API.md §4.4).
//
// installed ──► enabled ──► started
// │ │
// │ └──► startup_failed ──┐
// │ │ (retry)
// └──────────────► disabled ◄─────────┘
//
// One row per installed module, one column holding the state. The rules that make
// the machine mean anything live here, not in the SQL:
//
// - `installed` is transient. An install writes the row; the restart that follows
// resolves it to `started` or `startup_failed` (§2.5).
// - `disabled` is the only state a boot leaves alone. It is the operator's
// decision; every other state is an outcome and is recomputed each boot by
// beginBoot(). That is what makes a fixed module recover on restart without
// anyone visiting the admin panel.
// - A failure is recorded with the stage it happened at, and every non-failing
// transition clears it — a running module can never show a stale reason.
//
// What this table does NOT decide is which routes exist. The loader scans the
// filesystem at require time, before the database is reachable (MODULE_API.md §4.1),
// so a disabled module is still mounted and simply guarded (§4.5). Keeping the URL
// surface a property of the volume is what lets routes.manifest.json be generated
// off a dead database.
const db = require('./modules.db')
const STATES = ['installed', 'enabled', 'disabled', 'started', 'startup_failed']
// The stage a failure happened at: MODULE_API.md §4.3's seven validation steps,
// plus `boot` for an onBoot hook that threw (§2.5).
const FAILURE_STAGES = [
'manifest',
'core_api',
'mounts',
'extensions',
'schema',
'require',
'register',
'boot',
]
class ModuleStateError extends Error {
constructor(code, message) {
super(message)
this.name = 'ModuleStateError'
this.code = code
}
}
// Legal moves, keyed by target state. Anything not listed is a bug in the caller
// and throws rather than writing a row that misrepresents what happened.
const ALLOWED_FROM = {
// Enabling is the recovery path as well as the first step: a disabled module the
// operator switches back on, and a startup_failed one they retry, both land here.
enabled: ['installed', 'enabled', 'disabled', 'startup_failed', 'started'],
// The operator may disable a module in any state, including one that is running.
disabled: STATES,
// Reached from `enabled` on a normal boot, and from `installed` on the first boot
// after an install (or for a directory placed on the volume by hand, whose row is
// written moments earlier in the same boot).
started: ['installed', 'enabled'],
// Failure always precedes `started` in the lifecycle; `started` is accepted so a
// late failure can still be recorded truthfully rather than dropped.
startup_failed: ['installed', 'enabled', 'started'],
}
// row → API shape.
function serialize(row) {
if (!row) return null
return {
id: row.id,
name: row.name,
version: row.version,
state: row.state,
failureStage: row.failure_stage ?? null,
failureReason: row.failure_reason ?? null,
source: row.source ?? null,
sha256: row.sha256 ?? null,
installedAt: row.installed_at ?? null,
startedAt: row.started_at ?? null,
updatedAt: row.updated_at ?? null,
}
}
async function list() {
const rows = await db.listAll()
return rows.map(serialize)
}
async function get(id) {
const rows = await db.getOne(id)
return serialize(rows[0])
}
// Record an install (or a re-install / upgrade). Metadata is refreshed; the state is
// left as it is, so upgrading an enabled module does not switch it off and
// re-installing a disabled one does not switch it on. A new row lands in `installed`.
async function recordInstalled({ id, name, version, source = null, sha256 = null }) {
if (!id || !name || !version) {
throw new ModuleStateError('invalid_module', 'id, name and version are required')
}
await db.upsert({ id, name, version, source, sha256 })
return get(id)
}
// Start of boot: clear the last boot's outcomes so what is on display after this
// boot is what this boot did. Leaves `disabled` rows alone (see the header).
// Returns the number of rows reset.
async function beginBoot() {
const res = await db.resetForBoot()
return res?.affectedRows ?? 0
}
// Apply one transition, after checking it is legal for the row's current state.
// A row that does not exist is not an error the caller can act on — a module can be
// present on the volume with no row at all — so it returns null and writes nothing.
async function transition(id, target, { failureStage = null, failureReason = null } = {}) {
const current = await get(id)
if (!current) return null
const allowed = ALLOWED_FROM[target]
if (!allowed.includes(current.state)) {
throw new ModuleStateError(
'illegal_transition',
`module '${id}': cannot move from '${current.state}' to '${target}'`,
)
}
await db.setState({
id,
state: target,
failureStage,
failureReason,
stampStarted: target === 'started',
})
return get(id)
}
const enable = (id) => transition(id, 'enabled')
const disable = (id) => transition(id, 'disabled')
const markStarted = (id) => transition(id, 'started')
// Record a failure at a named stage. Two deliberate softenings, both because this is
// called from the boot path where throwing would turn one module's failure into
// everybody's (MODULE_API.md §4.4 — the failing module fails alone):
//
// - a `disabled` row is a no-op. The operator switched it off; a broken module
// they already disabled is not news, and overwriting their decision with an
// outcome would silently re-enable it on the next boot.
// - an unrecognised stage is recorded as `require` rather than rejected, so a
// miscategorised failure still reaches the admin panel with its reason intact.
async function markStartupFailed(id, { stage, reason }) {
const current = await get(id)
if (!current || current.state === 'disabled') return current
return transition(id, 'startup_failed', {
failureStage: FAILURE_STAGES.includes(stage) ? stage : 'require',
failureReason: String(reason ?? 'unknown error').slice(0, 4000),
})
}
// Purge only (§2.5). A plain uninstall disables the module and keeps its row, so its
// data survives and the admin panel can still show what was there.
async function remove(id) {
await db.remove(id)
}
module.exports = {
STATES,
FAILURE_STAGES,
ModuleStateError,
list,
get,
recordInstalled,
beginBoot,
enable,
disable,
markStarted,
markStartupFailed,
remove,
}

View File

@@ -0,0 +1,295 @@
// Point the DB pool at a dead port before it's built; every modules.db method is
// monkeypatched below, and pool.close() at the end lets the process exit cleanly.
process.env.DB_HOST = '127.0.0.1'
process.env.DB_PORT = '59999'
const { test, beforeEach, after } = require('node:test')
const assert = require('node:assert/strict')
const pool = require('../src/utils/db')
after(() => pool.close())
// Unit-test the module state machine (docs/website/MODULE_SYSTEM.md §2.4) against an
// in-memory fake by monkeypatching modules.db (no DB). What is locked here is
// everything the boot path and the admin panel will lean on in later phases:
// - `installed` is transient and a re-install never overwrites the operator's
// enable/disable decision;
// - beginBoot() recomputes outcomes and leaves `disabled` alone — the property that
// makes a fixed module recover on a restart with no panel visit;
// - a failure carries its stage and reason, and every non-failing transition clears
// them, so a running module can never display a stale reason;
// - a disabled module's failure is a no-op, because the boot path must never turn
// one module's failure into a re-enable of a module the operator switched off;
// - an illegal move throws instead of writing a row that misrepresents the state.
const modulesDb = require('../src/model/modules/modules.db')
const modules = require('../src/model/modules/modules.model')
let rows // id → row, snake_case exactly as modules.db returns it
const saved = { ...modulesDb }
function reset() {
rows = new Map()
}
modulesDb.listAll = async () => [...rows.values()].sort((a, b) => a.id.localeCompare(b.id))
modulesDb.getOne = async (id) => (rows.has(id) ? [rows.get(id)] : [])
modulesDb.upsert = async ({ id, name, version, source, sha256 }) => {
const existing = rows.get(id)
if (existing) {
Object.assign(existing, { name, version, source: source ?? null, sha256: sha256 ?? null })
return { affectedRows: 1 }
}
rows.set(id, {
id,
name,
version,
state: 'installed',
failure_stage: null,
failure_reason: null,
source: source ?? null,
sha256: sha256 ?? null,
installed_at: '2026-08-10T00:00:00Z',
started_at: null,
updated_at: '2026-08-10T00:00:00Z',
})
return { affectedRows: 1 }
}
modulesDb.setState = async ({ id, state, failureStage, failureReason, stampStarted }) => {
const row = rows.get(id)
if (!row) return { affectedRows: 0 }
row.state = state
row.failure_stage = failureStage ?? null
row.failure_reason = failureReason ?? null
if (stampStarted) row.started_at = '2026-08-10T12:00:00Z'
return { affectedRows: 1 }
}
modulesDb.resetForBoot = async () => {
let n = 0
for (const row of rows.values()) {
if (row.state === 'disabled') continue
row.state = 'enabled'
row.failure_stage = null
row.failure_reason = null
n += 1
}
return { affectedRows: n }
}
modulesDb.remove = async (id) => ({ affectedRows: rows.delete(id) ? 1 : 0 })
after(() => Object.assign(modulesDb, saved))
beforeEach(reset)
// Install one module and put it in a given state, bypassing the machine so a test
// can start from any state without asserting its way there.
async function seed(id, state = 'installed', extra = {}) {
await modules.recordInstalled({ id, name: `Module ${id}`, version: '1.0.0', ...extra })
rows.get(id).state = state
return modules.get(id)
}
// ── recordInstalled ───────────────────────────────────────────────────
test('a new install lands in the transient installed state', async () => {
const mod = await modules.recordInstalled({
id: 'uo',
name: 'Ultima Online',
version: '1.2.0',
source: 'https://gitea.example/releases/module-uo-1.2.0.tar.gz',
sha256: 'a'.repeat(64),
})
assert.equal(mod.state, 'installed')
assert.equal(mod.version, '1.2.0')
assert.equal(mod.sha256, 'a'.repeat(64))
assert.equal(mod.startedAt, null)
assert.equal(mod.failureReason, null)
})
test('a hand-placed module records with no provenance', async () => {
const mod = await modules.recordInstalled({ id: 'uo', name: 'Ultima Online', version: '1.0.0' })
assert.equal(mod.source, null)
assert.equal(mod.sha256, null)
})
test('recordInstalled refuses a manifest missing id, name or version', async () => {
await assert.rejects(
() => modules.recordInstalled({ id: 'uo', version: '1.0.0' }),
(err) => err.code === 'invalid_module',
)
})
test('an upgrade refreshes metadata and leaves the operator decision alone', async () => {
await seed('uo', 'disabled')
const mod = await modules.recordInstalled({ id: 'uo', name: 'Ultima Online', version: '2.0.0' })
assert.equal(mod.version, '2.0.0')
assert.equal(mod.name, 'Ultima Online')
assert.equal(mod.state, 'disabled', 're-installing must not silently re-enable')
})
test('an upgrade of a started module does not switch it off', async () => {
await seed('uo', 'started')
const mod = await modules.recordInstalled({ id: 'uo', name: 'Module uo', version: '1.1.0' })
assert.equal(mod.state, 'started')
})
// ── the machine ───────────────────────────────────────────────────────
test('installed → enabled → started, stamping the start', async () => {
await seed('uo')
assert.equal((await modules.enable('uo')).state, 'enabled')
const started = await modules.markStarted('uo')
assert.equal(started.state, 'started')
assert.ok(started.startedAt, 'a successful start is stamped')
})
test('a first boot may start a module straight from installed', async () => {
await seed('uo')
assert.equal((await modules.markStarted('uo')).state, 'started')
})
test('a disabled module can be re-enabled', async () => {
await seed('uo', 'disabled')
assert.equal((await modules.enable('uo')).state, 'enabled')
})
test('a failed module is retried by enabling it, which clears the reason', async () => {
await seed('uo', 'enabled')
await modules.markStartupFailed('uo', { stage: 'boot', reason: 'atlas refresh threw' })
const retried = await modules.enable('uo')
assert.equal(retried.state, 'enabled')
assert.equal(retried.failureStage, null)
assert.equal(retried.failureReason, null)
})
test('a running module can be disabled', async () => {
await seed('uo', 'started')
assert.equal((await modules.disable('uo')).state, 'disabled')
})
test('a disabled module is never started — that would be a core bug, so it throws', async () => {
await seed('uo', 'disabled')
await assert.rejects(
() => modules.markStarted('uo'),
(err) => err.name === 'ModuleStateError' && err.code === 'illegal_transition',
)
assert.equal((await modules.get('uo')).state, 'disabled')
})
test('a transition on a module with no row writes nothing and returns null', async () => {
assert.equal(await modules.enable('ghost'), null)
assert.equal(await modules.markStarted('ghost'), null)
assert.equal(rows.size, 0)
})
// ── failures ──────────────────────────────────────────────────────────
test('a failure records its stage and reason', async () => {
await seed('uo', 'enabled')
const failed = await modules.markStartupFailed('uo', {
stage: 'core_api',
reason: "module 'uo' needs coreApi ^2.0.0, core provides 1.0.0",
})
assert.equal(failed.state, 'startup_failed')
assert.equal(failed.failureStage, 'core_api')
assert.match(failed.failureReason, /coreApi/)
})
test('an unrecognised stage is still recorded, as require', async () => {
await seed('uo', 'enabled')
const failed = await modules.markStartupFailed('uo', { stage: 'nonsense', reason: 'boom' })
assert.equal(failed.failureStage, 'require')
assert.equal(failed.failureReason, 'boom')
})
test('a missing reason still produces a displayable one', async () => {
await seed('uo', 'enabled')
const failed = await modules.markStartupFailed('uo', { stage: 'register' })
assert.equal(failed.failureReason, 'unknown error')
})
test('a runaway reason is truncated rather than refused', async () => {
await seed('uo', 'enabled')
const failed = await modules.markStartupFailed('uo', { stage: 'boot', reason: 'x'.repeat(9000) })
assert.equal(failed.failureReason.length, 4000)
})
test("a disabled module's failure is a no-op, not a re-enable", async () => {
await seed('uo', 'disabled')
const unchanged = await modules.markStartupFailed('uo', { stage: 'require', reason: 'broken' })
assert.equal(unchanged.state, 'disabled')
assert.equal(unchanged.failureReason, null)
})
test('starting successfully clears the previous failure', async () => {
await seed('uo', 'enabled')
await modules.markStartupFailed('uo', { stage: 'schema', reason: 'bad fragment' })
await modules.enable('uo')
const started = await modules.markStarted('uo')
assert.equal(started.failureStage, null)
assert.equal(started.failureReason, null)
})
// ── boot ──────────────────────────────────────────────────────────────
test('beginBoot recomputes outcomes and leaves disabled alone', async () => {
await seed('a', 'started')
await seed('b', 'startup_failed')
await seed('c', 'disabled')
await seed('d', 'installed')
rows.get('b').failure_reason = 'last boot blew up'
rows.get('b').failure_stage = 'boot'
assert.equal(await modules.beginBoot(), 3)
const byId = Object.fromEntries((await modules.list()).map((m) => [m.id, m]))
assert.equal(byId.a.state, 'enabled')
assert.equal(byId.b.state, 'enabled', 'a failed module is retried on the next boot')
assert.equal(byId.b.failureReason, null, 'and last boots reason is cleared')
assert.equal(byId.c.state, 'disabled', 'the operator decision survives a boot')
assert.equal(byId.d.state, 'enabled')
})
test('beginBoot keeps the start stamp of a module that was running', async () => {
await seed('uo', 'enabled')
await modules.markStarted('uo')
await modules.beginBoot()
assert.ok((await modules.get('uo')).startedAt, 'started_at is the last successful start')
})
// ── list / purge ──────────────────────────────────────────────────────
test('list returns every module, id-ordered and serialized', async () => {
await seed('zzz')
await seed('aaa')
const all = await modules.list()
assert.deepEqual(
all.map((m) => m.id),
['aaa', 'zzz'],
)
assert.deepEqual(Object.keys(all[0]).sort(), [
'failureReason',
'failureStage',
'id',
'installedAt',
'name',
'sha256',
'source',
'startedAt',
'state',
'updatedAt',
'version',
])
})
test('purge drops the row; uninstall is a disable and keeps it', async () => {
await seed('uo', 'started')
await modules.disable('uo')
assert.ok(await modules.get('uo'), 'uninstall keeps the row and its data')
await modules.remove('uo')
assert.equal(await modules.get('uo'), null)
})