feat(modules): install, uninstall, purge and restart (phase 4, slice 1)
All checks were successful
PR Checks / bot-install (pull_request) Successful in 17s
PR Checks / client-build (pull_request) Successful in 26s
PR Checks / server-tests (pull_request) Successful in 33s

The consumer half of a release module-uo's CI has been publishing since
phase 3 closed. Before this, core had the installed_modules provenance
columns and no code that could ever fill them: nothing fetched, verified,
unpacked, removed or purged anything, and there was no admin route at all.

Adds modules/archive.js, modules/install.js, schema.runPurge(),
lifecycle.stop(), loader.stopHook(), and /api/v1/admin/modules with eight
routes. 797 server tests (+76), manifest 158 -> 166 + 2 internal, OpenAPI
gains 8 operations and loses nothing.

Reject, never sanitise
----------------------
The download is the easy part: an https-only allowlist re-checked on every
redirect hop, a declared sha256 compared against the bytes that arrived, and
a byte cap. Unpacking is where the archive chooses the filenames, and core
writes into a directory bind-mounted from the host, so an escape is not
confined to the container.

archive.js inspects the whole archive before a byte is unpacked and refuses
absolute and drive-absolute paths, `..` segments, NUL bytes, backslashes,
anything that is not a regular file or a directory, more than one top-level
entry, and anything over the entry or byte caps. Refusing symlinks and
hardlinks outright is what keeps this off the majority of node-tar's
published advisories rather than depending on the library to contain them.

That two-pass shape is load-bearing, and it was measured rather than assumed:
extracting an archive whose fourth member escapes upward throws under
node-tar 7.5.22 -- and leaves the first three members on disk. The loader
scans that directory at require time on the next boot, so a half-unpacked
module is a module. Everything therefore happens in a scratch directory that
is removed on any failure, and the move into place is the last step.

`tar` is pinned to ^7.5.22 rather than the ^6 that installs by default: 6.x
is flagged critical, and reading the advisory list is what the file's header
now says out loud -- almost all of it is hardlink or symlink traversal and
PAX header interpretation differentials, which is exactly this feature's
threat model.

Two things the plan had wrong
-----------------------------
The bundle's top-level directory is `module-uo-<version>`, not the module id
-- so "the top-level name must equal the id" was checked against nothing real.
The extractor strips that level instead, because its name belongs to whoever
published the bundle and the directory it lands in is core's. What is checked
instead is the unpacked module.json: a manifest promising `uo` and delivering
something else is refused rather than installed under the name it promised.

And purge cannot be a follow-up action (decision 5): purge.sql lives inside
the directory uninstall deletes. It is offered in the uninstall flow and as a
standalone action on a still-installed module, and the standalone one refuses
unless the module is already disabled -- dropping tables under something that
is still serving leaves it answering out of a world that no longer exists.

Disable now means stopped
-------------------------
lifecycle.stop() dispatches that one module's onShutdown before flipping the
guard, so a module an operator switches off actually releases its sockets and
closes its streams instead of merely becoming unreachable. The hook runs
first and the state moves after it, because while onShutdown runs the module
is still `started` and that is the only state in which its routes and the
world it is tearing down agree. A hook that throws does not stop the disable
-- the opposite of the boot path's rule, and deliberately.

Enable is not its mirror and there is no start(id) beside it. There is no
onBoot re-dispatch and the hooks were never promised re-entrant, so enable
moves the row and the restart route starts it. A test pins that enable does
not touch the loader, because "fixing" it is a one-line change that would put
a module with closed sockets back on the nav.

Restart raises SIGTERM against its own process rather than calling the
shutdown path directly, so server.js's handler stays the one graceful-shutdown
path and this route cannot drift from it.

The allowlist bootstraps from MODULE_SOURCE_HOSTS into a settings row and is
admin-managed after that (decision 6); seedDefault is INSERT IGNORE, so
changing the variable on an existing deployment is a no-op by design. An empty
list forbids every install rather than allowing every host -- the safe
direction for a value someone might blank by accident.

Verified against the real v0.3.0 release
----------------------------------------
Not a fixture: fetched the published install manifest over the real Gitea
host and its redirect chain, verified the sha256, inspected and unpacked the
252,517-byte artifact to 82 files, and then booted core against the result --
the module registered its five mounts, seven streams and eight capabilities
and resolved its client chunk, with no scratch directory left behind.

Two defects this slice's own tooling caught, both of which had already been
written down as classes:
  - the controller destructured runPurge at require time, capturing the
    function rather than the module, which made the one dependency whose
    ORDER matters the one that could not be substituted;
  - two swagger annotations carried an apostrophe inside a quoted string,
    dropped silently by swagger-autogen before slice 5 taught it to fail loudly.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2026-08-12 03:09:45 -05:00
parent 2cb549e9e5
commit b30e82cde2
20 changed files with 3381 additions and 3 deletions

View File

@@ -0,0 +1,463 @@
// ── Admin · Modules: the delivery surface ──────────────────────────────────
//
// Phase 4, slice 1 of MODULE_SYSTEM.md §2.7.2. The controller is tested directly
// with a mock `res` and stubbed models — the same shape adminUsers.test.js uses —
// because what is interesting here is not the HTTP plumbing but the ORDER of
// operations and which of the three sources of truth answers which question.
//
// Two of these tests exist to pin decisions that are easy to "fix" back into
// being wrong:
//
// - **enable must not touch the loader.** Disable ran the module's onShutdown;
// there is no onBoot re-dispatch, so flipping the record back would put a
// module with closed sockets and cleared timers back on the nav.
// - **purge must run before the directory is removed.** purge.sql lives inside
// that directory. Reorder those two lines and the feature silently stops
// working, with a 200 and no data deleted.
//
// Point the DB at a closed port BEFORE requiring anything that builds the pool.
process.env.DB_HOST = '127.0.0.1'
process.env.DB_PORT = '59999'
const { test, beforeEach, after } = require('node:test')
const assert = require('node:assert/strict')
const ctrl = require('../src/router/v1/admin/modules.controller')
const modules = require('../src/model/modules/modules.model')
const activity = require('../src/model/activity/activity.model')
const settings = require('../src/model/settings/settings.model')
const loader = require('../src/modules/loader')
const lifecycle = require('../src/modules/lifecycle')
const install = require('../src/modules/install')
const schema = require('../src/modules/schema')
const db = require('../src/utils/db')
after(() => db.close())
function mockRes() {
return {
statusCode: 200,
body: null,
status(c) { this.statusCode = c; return this },
json(b) { this.body = b; return this },
}
}
const req = (extra = {}) => ({
user: { id: 1, username: 'admin' },
params: {},
query: {},
body: {},
...extra,
})
// Everything the controller reaches for, replaced wholesale per test. Restored
// from these originals rather than from a snapshot taken mid-run, so one test
// leaking a stub cannot quietly become another test's fixture.
const originals = {
modules: { ...modules },
activity: { log: activity.log },
settings: { get: settings.get, set: settings.set },
loader: { isLoaded: loader.isLoaded, list: loader.list },
lifecycle: { stop: lifecycle.stop },
install: {
install: install.install,
isInstalled: install.isInstalled,
purgeFile: install.purgeFile,
removeDir: install.removeDir,
},
schema: { runPurge: schema.runPurge },
}
let logged
beforeEach(() => {
Object.assign(modules, originals.modules)
Object.assign(activity, originals.activity)
Object.assign(settings, originals.settings)
Object.assign(loader, originals.loader)
Object.assign(lifecycle, originals.lifecycle)
Object.assign(install, originals.install)
Object.assign(schema, originals.schema)
logged = []
activity.log = async (entry) => { logged.push(entry) }
settings.get = async () => 'gitea.whitlocktech.com'
loader.isLoaded = () => true
loader.list = () => []
})
// ── list ───────────────────────────────────────────────────────────────────
test('list reconciles the row, the loader and the volume without picking a winner', async () => {
// The case §2.4 creates and decision 3 makes routine: the row says `enabled`
// because the operator just switched it back on, the loader still says
// `disabled` because its onShutdown has run and there is no way back without a
// restart. Rendering either one alone would be a lie.
modules.list = async () => [{
id: 'uo', name: 'UO', version: '1.0.0', state: 'enabled',
failureStage: null, failureReason: null, source: 'https://x/y.json', sha256: 'a'.repeat(64),
installedAt: null, startedAt: null,
}]
loader.list = () => [{ id: 'uo', name: 'UO', version: '1.0.0', state: 'disabled', stage: null, reason: null, capabilities: ['shard'] }]
install.isInstalled = () => true
install.purgeFile = () => '/modules/uo/server/db/purge.sql'
const res = mockRes()
await ctrl.list(req(), res)
const [m] = res.body.modules
assert.equal(m.state, 'enabled', 'what the operator decided')
assert.equal(m.liveState, 'disabled', 'what is actually answering')
assert.equal(m.onVolume, true)
assert.equal(m.canPurge, true)
assert.deepEqual(m.capabilities, ['shard'])
assert.deepEqual(res.body.sourceHosts, ['gitea.whitlocktech.com'])
})
test('list includes a module on the volume that has no row yet', async () => {
// A hand-placed directory before its first boot. §2.5 keeps that a supported
// install, and its routes are already being served — a screen showing nothing
// for it would be showing the wrong thing.
modules.list = async () => []
loader.list = () => [{ id: 'byhand', name: 'By Hand', version: '0.1.0', state: 'started', stage: null, reason: null, capabilities: [] }]
install.isInstalled = () => true
install.purgeFile = () => null
const res = mockRes()
await ctrl.list(req(), res)
assert.equal(res.body.modules.length, 1)
assert.equal(res.body.modules[0].id, 'byhand')
assert.equal(res.body.modules[0].state, null, 'no row means no recorded state, not a guessed one')
assert.equal(res.body.modules[0].liveState, 'started')
})
test('list survives a process where the loader never scanned', async () => {
loader.isLoaded = () => false
loader.list = () => { throw new Error('modules.list() before modules.load()') }
modules.list = async () => [{ id: 'uo', name: 'UO', version: '1', state: 'disabled' }]
install.isInstalled = () => false
install.purgeFile = () => null
const res = mockRes()
await ctrl.list(req(), res)
assert.equal(res.statusCode, 200)
assert.equal(res.body.modules[0].liveState, null)
})
// ── install ────────────────────────────────────────────────────────────────
test('install records provenance and says a restart is needed', async () => {
const calls = []
install.install = async ({ url, hosts }) => {
calls.push({ url, hosts })
return { id: 'uo', name: 'UO', version: '1.0.0', sha256: 'b'.repeat(64), source: url, replaced: false }
}
modules.recordInstalled = async (row) => { calls.push(row); return { ...row, state: 'installed' } }
const res = mockRes()
await ctrl.create(req({ body: { url: 'https://gitea.whitlocktech.com/x/uo.json' } }), res)
assert.equal(res.statusCode, 201)
assert.equal(res.body.restartRequired, true)
assert.deepEqual(calls[0].hosts, ['gitea.whitlocktech.com'], 'the allowlist comes from the setting')
// Provenance is written HERE and nowhere else — the boot reconcile records a
// module with null source/sha256 and leaves what it is not given.
assert.equal(calls[1].source, 'https://gitea.whitlocktech.com/x/uo.json')
assert.equal(calls[1].sha256, 'b'.repeat(64))
assert.equal(logged[0].action, 'module.install')
})
test('an install refusal is reported to the operator, with its own status', async () => {
install.install = async () => {
const err = new Error('"evil.net" is not an allowed module source host')
err.name = 'InstallError'
err.status = 400
throw err
}
const res = mockRes()
await ctrl.create(req({ body: { url: 'https://evil.net/x.json' } }), res)
assert.equal(res.statusCode, 400)
// The message is the useful part: the operator pasted a URL and needs to know
// what was wrong with what came back.
assert.match(res.body.message, /not an allowed module source host/)
assert.equal(logged.length, 0, 'a refused install is not an audit-log entry')
})
test('an unreachable host is a 502, not a 400', async () => {
install.install = async () => {
const err = new Error('could not reach x: timeout')
err.name = 'InstallError'
err.status = 502
throw err
}
const res = mockRes()
await ctrl.create(req({ body: { url: 'https://gitea.whitlocktech.com/x.json' } }), res)
assert.equal(res.statusCode, 502)
})
test('an unexpected failure is a 500 and does not leak its message', async () => {
install.install = async () => { throw new Error('ENOENT /some/internal/path') }
const res = mockRes()
await ctrl.create(req({ body: { url: 'https://gitea.whitlocktech.com/x.json' } }), res)
assert.equal(res.statusCode, 500)
assert.equal(res.body.message, 'Internal Server Error')
})
// ── enable / disable ───────────────────────────────────────────────────────
test('enable moves the row and does NOT touch the loader', async () => {
// The decision-3 invariant. Re-enabling cannot restart a module: its
// onShutdown has run, and MODULE_API.md has never promised onBoot is safe to
// run twice. Flipping the record would put it back on the nav with a
// torn-down world behind it.
let setStateCalled = false
loader.setState = () => { setStateCalled = true }
modules.enable = async (id) => ({ id, state: 'enabled' })
const res = mockRes()
await ctrl.enable(req({ params: { id: 'uo' } }), res)
assert.equal(res.body.module.state, 'enabled')
assert.equal(res.body.restartRequired, true)
assert.equal(setStateCalled, false, 'enable must not move the in-memory record')
assert.equal(logged[0].action, 'module.enable')
loader.setState = originals.loader.setState
})
test('enabling a module with no row is a 404', async () => {
modules.enable = async () => null
const res = mockRes()
await ctrl.enable(req({ params: { id: 'ghost' } }), res)
assert.equal(res.statusCode, 404)
})
test('an illegal transition is a 409, not a 500', async () => {
modules.enable = async () => {
const err = new Error("module 'uo': cannot move from 'x' to 'enabled'")
err.name = 'ModuleStateError'
throw err
}
const res = mockRes()
await ctrl.enable(req({ params: { id: 'uo' } }), res)
assert.equal(res.statusCode, 409)
})
test('disable stops the module and reports whether the hook ran', async () => {
const calls = []
modules.get = async (id) => ({ id, state: 'started' })
lifecycle.stop = async (id) => { calls.push(id); return { stopped: true, error: null } }
const res = mockRes()
await ctrl.disable(req({ params: { id: 'uo' } }), res)
assert.deepEqual(calls, ['uo'])
assert.equal(res.body.stopped, true)
// No restart: this is the one action that takes effect immediately, and it is
// the one an operator reaches for when something is going wrong.
assert.equal(res.body.restartRequired, undefined)
assert.equal(logged[0].action, 'module.disable')
})
test('a shutdown hook that failed is reported rather than swallowed', async () => {
modules.get = async (id) => ({ id, state: 'started' })
lifecycle.stop = async () => ({ stopped: false, error: 'socket would not close' })
const res = mockRes()
await ctrl.disable(req({ params: { id: 'uo' } }), res)
// It IS disabled either way; the operator should be told it did not close
// cleanly while they still have the logs in front of them.
assert.equal(res.statusCode, 200)
assert.match(res.body.shutdownError, /socket would not close/)
})
// ── uninstall and purge ────────────────────────────────────────────────────
test('uninstall purges BEFORE it removes the directory', async () => {
// The ordering that makes decision 5 work at all: purge.sql is a file inside
// the directory being deleted. Swap these two and the endpoint still answers
// 200 and deletes nothing.
const order = []
modules.get = async (id) => ({ id, state: 'started' })
install.isInstalled = () => true
install.purgeFile = () => '/modules/uo/server/db/purge.sql'
schema.runPurge = async () => { order.push('purge'); return 12 }
lifecycle.stop = async () => { order.push('stop'); return { stopped: true, error: null } }
install.removeDir = async () => { order.push('removeDir'); return true }
modules.remove = async () => { order.push('removeRow') }
const res = mockRes()
await ctrl.remove(req({ params: { id: 'uo' }, query: { purge: 'true' } }), res)
assert.deepEqual(order, ['purge', 'stop', 'removeDir', 'removeRow'])
assert.equal(res.body.purged, 12)
assert.equal(res.body.restartRequired, true)
assert.equal(logged[0].action, 'module.purge')
})
test('a plain uninstall keeps the row and does not purge', async () => {
const order = []
modules.get = async (id) => ({ id, state: 'started' })
install.isInstalled = () => true
schema.runPurge = async () => { order.push('purge'); return 1 }
lifecycle.stop = async () => { order.push('stop'); return { stopped: true, error: null } }
install.removeDir = async () => { order.push('removeDir'); return true }
modules.remove = async () => { order.push('removeRow') }
const res = mockRes()
await ctrl.remove(req({ params: { id: 'uo' } }), res)
// §2.5's default: the directory goes, the data stays, and the disabled row is
// what keeps the retained data visible and the module reinstallable.
assert.deepEqual(order, ['stop', 'removeDir'])
assert.equal(res.body.purged, null)
assert.equal(logged[0].action, 'module.uninstall')
})
test('asking to purge a module that ships no purge.sql refuses instead of pretending', async () => {
modules.get = async (id) => ({ id, state: 'started' })
install.isInstalled = () => true
install.purgeFile = () => null
let removed = false
install.removeDir = async () => { removed = true; return true }
const res = mockRes()
await ctrl.remove(req({ params: { id: 'uo' }, query: { purge: 'true' } }), res)
assert.equal(res.statusCode, 400)
assert.match(res.body.message, /ships no purge.sql/)
// Nothing happened. The operator asked for the module AND its data to go; the
// data cannot go, so doing half of it silently would be the worst answer.
assert.equal(removed, false)
})
test('uninstalling something that is neither on the volume nor in a row is a 404', async () => {
modules.get = async () => null
install.isInstalled = () => false
const res = mockRes()
await ctrl.remove(req({ params: { id: 'ghost' } }), res)
assert.equal(res.statusCode, 404)
})
test('standalone purge refuses while the module is still running', async () => {
// Dropping the tables under a module that is still serving leaves it answering
// out of a world that no longer exists. Disabling first is one click.
modules.get = async (id) => ({ id, state: 'started' })
let ran = false
schema.runPurge = async () => { ran = true; return 1 }
const res = mockRes()
await ctrl.purge(req({ params: { id: 'uo' } }), res)
assert.equal(res.statusCode, 409)
assert.match(res.body.message, /Disable this module before purging/)
assert.equal(ran, false)
})
test('standalone purge runs on a disabled module', async () => {
modules.get = async (id) => ({ id, state: 'disabled' })
install.purgeFile = () => '/modules/uo/server/db/purge.sql'
schema.runPurge = async () => 7
const res = mockRes()
await ctrl.purge(req({ params: { id: 'uo' } }), res)
assert.equal(res.body.purged, 7)
assert.equal(logged[0].action, 'module.purge')
})
// ── the allowlist ──────────────────────────────────────────────────────────
test('setSources stores a normalised list and audits the change', async () => {
let stored = null
settings.set = async (key, value) => { stored = { key, value } }
const res = mockRes()
await ctrl.setSources(req({ body: { hosts: 'Gitea.Example.com, releases.example.org' } }), res)
assert.deepEqual(res.body.sourceHosts, ['gitea.example.com', 'releases.example.org'])
assert.equal(stored.key, ctrl.HOSTS_KEY)
assert.equal(stored.value, 'gitea.example.com,releases.example.org')
// Before AND after: this setting decides what code the site will execute, so
// the audit entry has to say what it used to be.
assert.equal(logged[0].action, 'module.sources')
assert.deepEqual(logged[0].detail.before, ['gitea.whitlocktech.com'])
})
test('setSources refuses anything that is not a bare hostname', async () => {
let stored = false
settings.set = async () => { stored = true }
for (const bad of ['https://x.com', 'x.com/path', 'x.com:8443', '*.x.com', 'x_y.com']) {
const res = mockRes()
// eslint-disable-next-line no-await-in-loop
await ctrl.setSources(req({ body: { hosts: bad } }), res)
assert.equal(res.statusCode, 400, `${bad} should be refused`)
}
assert.equal(stored, false)
})
test('an empty allowlist is storable, and means no installs', async () => {
// Not a wildcard, and not an error: "nothing may be installed" is a position
// an operator is entitled to take.
let stored = null
settings.set = async (key, value) => { stored = value }
const res = mockRes()
await ctrl.setSources(req({ body: { hosts: '' } }), res)
assert.equal(res.statusCode, 200)
assert.deepEqual(res.body.sourceHosts, [])
assert.equal(stored, '')
})
// ── restart ────────────────────────────────────────────────────────────────
test('restart answers before it signals, and signals its own process', async () => {
// It raises SIGTERM rather than calling the shutdown path directly, so that
// server.js's handler stays the ONE graceful-shutdown path and this route
// cannot drift from it.
const originalKill = process.kill
const signals = []
process.kill = (pid, signal) => { signals.push({ pid, signal }) }
try {
const res = mockRes()
ctrl.restart(req(), res)
// Answered synchronously: once the signal lands there is no listener left to
// flush a response through, so the operator would be told nothing.
assert.equal(res.statusCode, 202)
assert.equal(res.body.restarting, true)
await new Promise((resolve) => setTimeout(resolve, 400))
assert.deepEqual(signals, [{ pid: process.pid, signal: 'SIGTERM' }])
assert.equal(logged[0].action, 'module.restart')
} finally {
process.kill = originalKill
}
})
test('a failure to write the audit entry does not cancel the restart', async () => {
const originalKill = process.kill
const signals = []
process.kill = (pid, signal) => { signals.push(signal) }
activity.log = async () => { throw new Error('database is gone') }
try {
ctrl.restart(req(), mockRes())
await new Promise((resolve) => setTimeout(resolve, 400))
assert.deepEqual(signals, ['SIGTERM'])
} finally {
process.kill = originalKill
}
})

View File

@@ -0,0 +1,252 @@
// modules/archive.js — the hardened bundle extractor (MODULE_SYSTEM.md §2.7.2).
//
// Every rejection case is exercised against a REAL archive rather than a mocked
// tar parser, because the thing being tested is what the parser reports for a
// given sequence of bytes. A fake that returns `{type: 'SymbolicLink'}` proves
// only that the `if` is spelled correctly.
//
// The hostile archives are written here as raw ustar headers instead of being
// produced with `tar`, for two reasons that both bit during this slice:
//
// 1. `ln -s` needs a privilege Windows does not hand out by default, so a
// symlink fixture built with the shell is a fixture that silently is not
// one, and the test passes for the wrong reason on the machine most of this
// work happens on.
// 2. GNU tar will not emit `../escape` or `/etc/passwd` as a member name — it
// strips them and tells you so. The archives worth defending against are
// exactly the ones a cooperative archiver refuses to produce.
//
// A ustar header is 512 bytes of fixed-offset fields, so writing one is less
// code than persuading a tool to misbehave.
const test = require('node:test')
const assert = require('node:assert/strict')
const fs = require('fs')
const os = require('os')
const path = require('path')
const zlib = require('zlib')
const archive = require('../src/modules/archive')
// ── A minimal ustar writer ─────────────────────────────────────────────────
const BLOCK = 512
function octal(value, width) {
// ustar numeric fields are NUL-terminated octal, right-aligned with zeros.
return Number(value).toString(8).padStart(width - 1, '0') + '\0'
}
/**
* One 512-byte header plus its padded data.
*
* @param {object} entry
* @param {string} entry.name member path
* @param {string} [entry.type] '0' file · '5' dir · '2' symlink · '1' hardlink
* · '3' char dev · '6' FIFO
* @param {string} [entry.linkname] target, for the link types
* @param {string} [entry.body] file contents
* @param {number} [entry.size] declared size — defaults to the body's, and
* may be set independently to build a header
* that lies about its payload
*/
function member({ name, type = '0', linkname = '', body = '', size = null }) {
const header = Buffer.alloc(BLOCK, 0)
const data = Buffer.from(body, 'utf8')
const declared = size === null ? data.length : size
header.write(name, 0, 100, 'utf8')
header.write(octal(0o644, 8), 100, 8, 'ascii') // mode
header.write(octal(0, 8), 108, 8, 'ascii') // uid
header.write(octal(0, 8), 116, 8, 'ascii') // gid
header.write(octal(declared, 12), 124, 12, 'ascii')
header.write(octal(0, 12), 136, 12, 'ascii') // mtime
header.write(' ', 148, 8, 'ascii') // checksum field is spaces while summing
header.write(type, 156, 1, 'ascii')
header.write(linkname, 157, 100, 'utf8')
header.write('ustar\0', 257, 6, 'ascii')
header.write('00', 263, 2, 'ascii')
let sum = 0
for (const byte of header) sum += byte
header.write(`${sum.toString(8).padStart(6, '0')}\0 `, 148, 8, 'ascii')
const padding = Buffer.alloc((BLOCK - (data.length % BLOCK)) % BLOCK, 0)
return Buffer.concat([header, data, padding])
}
/** Gzip a set of members into a .tar.gz on disk, and return its path. */
function writeArchive(dir, filename, members) {
const tarball = Buffer.concat([
...members.map(member),
Buffer.alloc(BLOCK * 2, 0), // two zero blocks end the archive
])
const file = path.join(dir, filename)
fs.writeFileSync(file, zlib.gzipSync(tarball))
return file
}
/** A well-formed bundle: one top-level directory, ordinary files inside it. */
function goodMembers(root = 'module-uo-1.0.0') {
return [
{ name: `${root}/`, type: '5' },
{ name: `${root}/module.json`, body: '{"id":"uo","name":"UO","version":"1.0.0"}' },
{ name: `${root}/server/`, type: '5' },
{ name: `${root}/server/index.js`, body: 'module.exports = () => {}\n' },
]
}
// One scratch directory for the whole file, removed at the end.
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'rg-archive-'))
test.after(() => fs.rmSync(tmp, { recursive: true, force: true }))
/** Assert that inspecting `members` fails, and that the message says why. */
async function rejects(name, members, matcher) {
const file = writeArchive(tmp, `${name}.tar.gz`, members)
await assert.rejects(
() => archive.inspect(file),
(err) => {
assert.equal(err.name, 'ArchiveError', `expected an ArchiveError, got ${err.name}: ${err.message}`)
assert.match(err.message, matcher)
return true
},
)
}
// ── What a good bundle does ────────────────────────────────────────────────
test('inspect accepts a well-formed bundle and reports its single root', async () => {
const file = writeArchive(tmp, 'good.tar.gz', goodMembers())
const stats = await archive.inspect(file)
assert.equal(stats.root, 'module-uo-1.0.0')
assert.equal(stats.entries, 4)
assert.ok(stats.bytes > 0)
})
test('extract strips the top level, so the bundle lands as the module id', async () => {
// The wrapper directory is the publisher's naming (module-uo's release
// workflow packs `module-uo-<version>/`); the directory it lands in is core's,
// and has to be the id the loader scans for.
const file = writeArchive(tmp, 'strip.tar.gz', goodMembers())
const dest = path.join(tmp, 'unpacked-uo')
await archive.unpack(file, dest)
assert.ok(fs.existsSync(path.join(dest, 'module.json')), 'module.json should be at the root of the destination')
assert.ok(fs.existsSync(path.join(dest, 'server', 'index.js')))
assert.ok(!fs.existsSync(path.join(dest, 'module-uo-1.0.0')), 'the wrapper directory should not survive')
})
test('extract refuses a destination that already exists', async () => {
const file = writeArchive(tmp, 'exists.tar.gz', goodMembers())
const dest = path.join(tmp, 'already-there')
fs.mkdirSync(dest)
await assert.rejects(() => archive.extract(file, dest), /already exists/)
})
// ── What a hostile bundle does ─────────────────────────────────────────────
test('an absolute member path is refused', async () => {
await rejects('absolute', [
{ name: 'mod/', type: '5' },
{ name: '/etc/cron.d/pwned', body: '* * * * * root sh\n' },
], /absolute/)
})
test('a drive-absolute member path is refused', async () => {
// Its own node-tar advisory, and invisible to a leading-slash check.
await rejects('drive', [
{ name: 'mod/', type: '5' },
{ name: 'C:\\Windows\\Temp\\pwned', body: 'x' },
], /backslash|drive-absolute/)
})
test('an upward-escaping member path is refused', async () => {
await rejects('escape', [
{ name: 'mod/', type: '5' },
{ name: 'mod/../../../etc/passwd', body: 'root::0:0\n' },
], /escapes upward/)
})
test('a symlink member is refused, and the message names the type', async () => {
await rejects('symlink', [
{ name: 'mod/', type: '5' },
{ name: 'mod/passwd', type: '2', linkname: '/etc/passwd' },
], /SymbolicLink/)
})
test('a hardlink member is refused', async () => {
// The single most-published node-tar escape primitive. Refusing the type
// outright is what keeps this file from depending on the library getting
// hardlink containment right.
await rejects('hardlink', [
{ name: 'mod/', type: '5' },
{ name: 'mod/shadow', type: '1', linkname: '../../../etc/shadow' },
], /Link/)
})
test('a device node is refused', async () => {
await rejects('device', [
{ name: 'mod/', type: '5' },
{ name: 'mod/zero', type: '3', linkname: '' },
], /may only contain files and directories/)
})
test('a FIFO is refused', async () => {
await rejects('fifo', [
{ name: 'mod/', type: '5' },
{ name: 'mod/pipe', type: '6' },
], /may only contain files and directories/)
})
test('two top-level directories are refused', async () => {
await rejects('two-roots', [
{ name: 'mod-a/', type: '5' },
{ name: 'mod-a/module.json', body: '{}' },
{ name: 'mod-b/', type: '5' },
{ name: 'mod-b/module.json', body: '{}' },
], /exactly one top-level directory, found 2/)
})
test('a bundle whose declared sizes exceed the cap is refused before it is read to the end', async () => {
// The header lies: it declares a gigabyte and carries nothing. That is the
// decompression-bomb shape, and the point is that inspect() decides on the
// DECLARED size without ever materialising the payload.
await rejects('bomb', [
{ name: 'mod/', type: '5' },
{ name: 'mod/big', size: archive.MAX_BYTES + 1, body: '' },
], /unpacks to more than/)
})
test('an empty archive is refused', async () => {
await rejects('empty', [], /empty/)
})
// ── The path check on its own ──────────────────────────────────────────────
//
// pathProblem is exported so the cases that are awkward to express as archive
// bytes can still be asserted directly.
test('pathProblem accepts ordinary bundle paths', () => {
for (const ok of ['mod/module.json', 'mod/server/router/x.js', 'mod/a.b-c_d/e.js']) {
assert.equal(archive.pathProblem(ok), null, `${ok} should be accepted`)
}
})
test('pathProblem rejects the escape shapes', () => {
assert.match(archive.pathProblem('/etc/passwd'), /absolute/)
assert.match(archive.pathProblem('C:/Windows/x'), /drive-absolute/)
assert.match(archive.pathProblem('mod/../../x'), /escapes upward/)
assert.match(archive.pathProblem('..'), /escapes upward/)
assert.match(archive.pathProblem('mod\\x'), /backslash/)
assert.match(archive.pathProblem('mod/\0/x'), /NUL byte/)
})
test('pathProblem does not reject a filename that merely contains two dots', () => {
// `..` is a SEGMENT, not a substring — a file called `version..js` is fine,
// and a check written with `includes('..')` would refuse it.
assert.equal(archive.pathProblem('mod/version..js'), null)
assert.equal(archive.pathProblem('mod/..hidden'), null)
})

View File

@@ -0,0 +1,448 @@
// modules/install.js — fetching, verifying and placing a module bundle.
//
// Phase 4, slice 1 of MODULE_SYSTEM.md §2.7.2. The rules under test are all
// refusals, and each one is a step an attacker would otherwise walk through:
// a non-https URL, a host that is not allowed, a REDIRECT to a host that is not
// allowed, a body larger than declared, a hash that does not match, an archive
// that does not agree with the manifest about what it is.
//
// `fetchImpl` is injected rather than a TLS server being stood up, for the same
// reason `replayFragments` takes a `query`: what is being tested is what this
// file decides about a response, and a real server would mostly test Node's
// certificate handling. The responses below are real `Response` objects with
// real bodies, so the streaming, the hashing and the byte cap are exercised for
// real — only the transport is stubbed.
process.env.DB_HOST = '127.0.0.1'
process.env.DB_PORT = '59999'
const crypto = require('crypto')
const fs = require('fs')
const os = require('os')
const path = require('path')
const zlib = require('zlib')
const { test, beforeEach, after } = require('node:test')
const assert = require('node:assert/strict')
const db = require('../src/utils/db')
after(() => db.close())
const HOSTS = ['releases.example.com']
const MANIFEST_URL = 'https://releases.example.com/mod/uo-1.0.0.json'
let tmpRoot
let install
/**
* A fresh install.js bound to a fresh modules directory.
*
* install.js reads `loader.dir()`, which is resolved once at require time from
* MODULES_DIR — so both have to be re-required per test, exactly as
* moduleLifecycle.test.js does.
*/
function freshInstall(dir) {
process.env.MODULES_DIR = dir
delete require.cache[require.resolve('../src/modules/loader')]
delete require.cache[require.resolve('../src/modules/install')]
// eslint-disable-next-line global-require
return require('../src/modules/install')
}
// ── Building a bundle to serve ─────────────────────────────────────────────
const BLOCK = 512
function octal(value, width) {
return Number(value).toString(8).padStart(width - 1, '0') + '\0'
}
function member({ name, type = '0', body = '' }) {
const header = Buffer.alloc(BLOCK, 0)
const data = Buffer.from(body, 'utf8')
header.write(name, 0, 100, 'utf8')
header.write(octal(0o644, 8), 100, 8, 'ascii')
header.write(octal(0, 8), 108, 8, 'ascii')
header.write(octal(0, 8), 116, 8, 'ascii')
header.write(octal(data.length, 12), 124, 12, 'ascii')
header.write(octal(0, 12), 136, 12, 'ascii')
header.write(' ', 148, 8, 'ascii')
header.write(type, 156, 1, 'ascii')
header.write('ustar\0', 257, 6, 'ascii')
header.write('00', 263, 2, 'ascii')
let sum = 0
for (const byte of header) sum += byte
header.write(`${sum.toString(8).padStart(6, '0')}\0 `, 148, 8, 'ascii')
const padding = Buffer.alloc((BLOCK - (data.length % BLOCK)) % BLOCK, 0)
return Buffer.concat([header, data, padding])
}
/** A bundle tarball whose root directory is named the way a release names it. */
function bundle({ id = 'uo', version = '1.0.0', extra = [] } = {}) {
const root = `module-${id}-${version}`
return zlib.gzipSync(Buffer.concat([
member({ name: `${root}/`, type: '5' }),
member({
name: `${root}/module.json`,
body: JSON.stringify({ id, name: 'Ultima Online', version, coreApi: '^1.0.0', server: 'server/index.js' }),
}),
member({ name: `${root}/server/`, type: '5' }),
member({ name: `${root}/server/index.js`, body: 'module.exports = () => {}\n' }),
...extra.map(member),
Buffer.alloc(BLOCK * 2, 0),
]))
}
const sha256 = (buf) => crypto.createHash('sha256').update(buf).digest('hex')
/**
* A fake transport serving one manifest and one artifact.
*
* `routes` maps an absolute URL to either a Buffer/string body or
* `{ status, location }` for a redirect, so a test can describe exactly what the
* remote host does without describing how it does it.
*/
function fakeFetch(routes) {
const seen = []
const impl = async (url) => {
const href = String(url)
seen.push(href)
const route = routes[href]
if (route === undefined) return new Response('not found', { status: 404 })
if (route && route.status) {
return new Response(null, {
status: route.status,
headers: route.location ? { location: route.location } : {},
})
}
return new Response(route, { status: 200 })
}
impl.seen = seen
return impl
}
/** The manifest a release publishes, with whatever a test wants to change. */
function manifestFor(tarball, overrides = {}) {
return JSON.stringify({
schema: 1,
id: 'uo',
name: 'Ultima Online',
version: '1.0.0',
coreApi: '^1.0.0',
artifact: 'uo-1.0.0.tar.gz',
url: 'https://releases.example.com/mod/uo-1.0.0.tar.gz',
sha256: sha256(tarball),
size: tarball.length,
...overrides,
})
}
/** The happy-path pair: a valid manifest and the artifact it describes. */
function goodRoutes(options = {}) {
const tarball = bundle(options)
return {
tarball,
routes: {
[MANIFEST_URL]: manifestFor(tarball, options.manifest || {}),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
},
}
}
beforeEach(() => {
tmpRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'rg-install-'))
install = freshInstall(tmpRoot)
})
// ── The allowlist ──────────────────────────────────────────────────────────
test('parseHosts accepts comma and whitespace separation, and lower-cases', () => {
assert.deepEqual(install.parseHosts('a.com, B.com\nc.com'), ['a.com', 'b.com', 'c.com'])
assert.deepEqual(install.parseHosts(''), [])
assert.deepEqual(install.parseHosts(null), [])
})
test('an empty allowlist forbids everything rather than allowing everything', () => {
// The direction matters: a setting someone blanks by accident must stop
// installs, not open the door to every host on the internet.
assert.throws(() => install.checkUrl('https://releases.example.com/x.json', []), /no module source hosts are allowed/)
})
test('only https may be installed from', () => {
// The sha256 is no help against a plaintext fetch: whoever can rewrite the
// artifact in flight can rewrite the manifest that declares its hash.
assert.throws(() => install.checkUrl('http://releases.example.com/x.json', HOSTS), /only https/)
assert.throws(() => install.checkUrl('file:///etc/passwd', HOSTS), /only https/)
})
test('a host that is not on the allowlist is refused, and the message says which are', () => {
assert.throws(
() => install.checkUrl('https://evil.example.net/x.json', HOSTS),
/"evil.example.net" is not an allowed module source host \(allowed: releases.example.com\)/,
)
})
test('the allowlist is matched on the host, not on a substring of the URL', () => {
// `https://evil.com/?x=releases.example.com` must not pass, and neither must a
// subdomain nobody listed.
assert.throws(() => install.checkUrl('https://evil.com/?releases.example.com', HOSTS), /not an allowed/)
assert.throws(() => install.checkUrl('https://sub.releases.example.com/x', HOSTS), /not an allowed/)
assert.throws(() => install.checkUrl('https://releases.example.com.evil.net/x', HOSTS), /not an allowed/)
})
// ── Redirects ──────────────────────────────────────────────────────────────
test('a redirect is followed, and re-checked against the allowlist', async () => {
const { tarball } = goodRoutes()
const impl = fakeFetch({
[MANIFEST_URL]: { status: 302, location: 'https://releases.example.com/cdn/uo-1.0.0.json' },
'https://releases.example.com/cdn/uo-1.0.0.json': manifestFor(tarball),
})
const manifest = await install.fetchManifest(MANIFEST_URL, HOSTS, impl)
assert.equal(manifest.id, 'uo')
assert.deepEqual(impl.seen, [MANIFEST_URL, 'https://releases.example.com/cdn/uo-1.0.0.json'])
})
test('a redirect to a host that is not allowed is refused', async () => {
// The hole the allowlist exists to close. `fetch`'s own redirect following
// would check the first hop and then go wherever it was pointed — which is
// why get() follows them by hand.
const impl = fakeFetch({
[MANIFEST_URL]: { status: 302, location: 'http://169.254.169.254/latest/meta-data/' },
})
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /only https/)
})
test('a redirect loop is bounded rather than followed forever', async () => {
const impl = fakeFetch({
[MANIFEST_URL]: { status: 302, location: MANIFEST_URL },
})
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /too many redirects/)
})
// ── The install manifest ───────────────────────────────────────────────────
test('a manifest that is not JSON is refused with a readable reason', async () => {
const impl = fakeFetch({ [MANIFEST_URL]: 'this is not json' })
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /not valid JSON/)
})
test('a manifest with an invalid module id is refused', async () => {
// The loader would refuse to scan a directory called `../etc`, but the point
// is that one is never created: the id becomes a path segment.
for (const id of ['../etc', 'UO', '', 'x', 'a'.repeat(40)]) {
const impl = fakeFetch({ [MANIFEST_URL]: JSON.stringify({ id, name: 'n', version: '1', sha256: 'a'.repeat(64) }) })
// eslint-disable-next-line no-await-in-loop
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /invalid module id/, `id ${JSON.stringify(id)}`)
}
})
test('a manifest with no usable sha256 is refused', async () => {
const impl = fakeFetch({
[MANIFEST_URL]: JSON.stringify({ id: 'uo', name: 'n', version: '1.0.0', sha256: 'not-a-hash' }),
})
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /no valid sha256/)
})
test('the artifact URL is resolved relative to the manifest when it carries no absolute one', async () => {
const { tarball } = goodRoutes()
const impl = fakeFetch({ [MANIFEST_URL]: manifestFor(tarball, { url: undefined }) })
const manifest = await install.fetchManifest(MANIFEST_URL, HOSTS, impl)
// Release assets sit beside their manifest, so a manifest that travelled
// without its absolute URL still resolves to the right place.
assert.equal(manifest.artifactUrl, 'https://releases.example.com/mod/uo-1.0.0.tar.gz')
})
// ── The install ────────────────────────────────────────────────────────────
test('a good bundle installs into modules/<id>, stripped of its wrapper', async () => {
const { routes, tarball } = goodRoutes()
const result = await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) })
assert.equal(result.id, 'uo')
assert.equal(result.version, '1.0.0')
assert.equal(result.sha256, sha256(tarball))
assert.equal(result.source, MANIFEST_URL)
assert.equal(result.replaced, false)
// The directory is named for the module id, not for the archive's root — the
// loader scans for the former and the publisher chose the latter.
const dir = path.join(tmpRoot, 'uo')
assert.ok(fs.existsSync(path.join(dir, 'module.json')))
assert.ok(fs.existsSync(path.join(dir, 'server', 'index.js')))
assert.ok(!fs.existsSync(path.join(tmpRoot, 'module-uo-1.0.0')))
})
test('a hash that does not match refuses the install and writes nothing', async () => {
const { routes } = goodRoutes()
// The bytes are fine; the manifest lies about them. Which is the same thing an
// artifact swapped after publication looks like.
routes[MANIFEST_URL] = manifestFor(Buffer.from('different'), {})
await assert.rejects(
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
/does not match the sha256/,
)
assert.deepEqual(fs.readdirSync(tmpRoot), [], 'nothing may be left on the volume')
})
test('a bundle whose module.json disagrees with the manifest is refused', async () => {
// A manifest promising `uo` and delivering something else would otherwise be
// installed into `modules/uo/` under a name it is not.
const tarball = bundle({ id: 'rust', version: '1.0.0' })
const routes = {
[MANIFEST_URL]: manifestFor(tarball),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
}
await assert.rejects(
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
/declares module id "rust" but the install manifest promised "uo"/,
)
assert.deepEqual(fs.readdirSync(tmpRoot), [])
})
test('a bundle whose version disagrees with the manifest is refused', async () => {
const tarball = bundle({ id: 'uo', version: '9.9.9' })
const routes = {
[MANIFEST_URL]: manifestFor(tarball),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
}
await assert.rejects(
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
/declares version "9.9.9"/,
)
})
test('an artifact whose length differs from the declared size is refused', async () => {
const tarball = bundle()
const routes = {
[MANIFEST_URL]: manifestFor(tarball, { size: tarball.length + 10 }),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
}
await assert.rejects(
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
/but the manifest declares/,
)
})
test('a hostile archive is refused, and nothing of it reaches the volume', async () => {
// The case node-tar alone does NOT cover: it throws on the escaping member,
// but only once it reaches it — the members before it are already on disk.
// archive.inspect() decides before extract() runs, and the unpack happens in a
// scratch directory that is removed either way.
const tarball = bundle({
extra: [
{ name: 'module-uo-1.0.0/../../ESCAPED.txt', body: 'escaped' },
],
})
const routes = {
[MANIFEST_URL]: manifestFor(tarball),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
}
await assert.rejects(
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
/escapes upward/,
)
assert.deepEqual(fs.readdirSync(tmpRoot), [], 'no scratch directory, no partial module, no escape')
})
test('an upgrade replaces the directory and reports that it did', async () => {
const first = goodRoutes()
await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(first.routes) })
fs.writeFileSync(path.join(tmpRoot, 'uo', 'STALE.txt'), 'from the old version')
const second = goodRoutes({ version: '2.0.0', manifest: { version: '2.0.0' } })
second.routes[MANIFEST_URL] = manifestFor(second.tarball, { version: '2.0.0' })
const result = await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(second.routes) })
assert.equal(result.replaced, true)
assert.equal(result.version, '2.0.0')
// A replace, not a merge: a file the previous version left behind must not
// survive into the new one, or an upgrade quietly keeps dead code loadable.
assert.ok(!fs.existsSync(path.join(tmpRoot, 'uo', 'STALE.txt')))
assert.deepEqual(fs.readdirSync(tmpRoot), ['uo'], 'the aside copy is cleaned up')
})
test('a failed upgrade leaves the previous version in place', async () => {
const first = goodRoutes()
await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(first.routes) })
const badTarball = bundle({ id: 'uo', version: '2.0.0', extra: [{ name: 'evil/', type: '5' }] })
await assert.rejects(() => install.install({
url: MANIFEST_URL,
hosts: HOSTS,
fetchImpl: fakeFetch({
[MANIFEST_URL]: manifestFor(badTarball, { version: '2.0.0' }),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': badTarball,
}),
}))
// The installed module is untouched: the swap is the last step, so a bundle
// rejected before it never got near the live directory.
assert.ok(fs.existsSync(path.join(tmpRoot, 'uo', 'module.json')))
assert.equal(JSON.parse(fs.readFileSync(path.join(tmpRoot, 'uo', 'module.json'), 'utf8')).version, '1.0.0')
assert.deepEqual(fs.readdirSync(tmpRoot), ['uo'])
})
// ── The volume ─────────────────────────────────────────────────────────────
test('moduleDir refuses an id that is not one', () => {
for (const id of ['../escape', 'a/b', '', 'UO', '.']) {
assert.throws(() => install.moduleDir(id), /invalid module id/, `id ${JSON.stringify(id)}`)
}
assert.equal(install.moduleDir('uo'), path.join(tmpRoot, 'uo'))
})
test('isInstalled and removeDir report what they did', async () => {
const { routes } = goodRoutes()
assert.equal(install.isInstalled('uo'), false)
await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) })
assert.equal(install.isInstalled('uo'), true)
assert.equal(await install.removeDir('uo'), true)
assert.equal(install.isInstalled('uo'), false)
// Removing what is not there is not an error — an uninstall of a module whose
// directory was already deleted by hand should still tidy up the row.
assert.equal(await install.removeDir('uo'), false)
})
test('purgeFile resolves the manifest declaration, and refuses one that escapes', async () => {
const dir = path.join(tmpRoot, 'uo')
fs.mkdirSync(path.join(dir, 'server', 'db'), { recursive: true })
fs.writeFileSync(path.join(dir, 'server', 'db', 'purge.sql'), 'DROP TABLE IF EXISTS x;')
const write = (purge) => fs.writeFileSync(
path.join(dir, 'module.json'),
JSON.stringify({ id: 'uo', name: 'UO', version: '1.0.0', purge }),
)
write('server/db/purge.sql')
assert.equal(install.purgeFile('uo'), path.join(dir, 'server', 'db', 'purge.sql'))
// The same containment rule the loader applies to client.entry: a manifest may
// not point core at a file outside the module it belongs to.
write('../../../../etc/passwd')
assert.equal(install.purgeFile('uo'), null)
// Declared but absent, and not declared at all, are both "nothing to run".
write('server/db/missing.sql')
assert.equal(install.purgeFile('uo'), null)
write(undefined)
assert.equal(install.purgeFile('uo'), null)
})
test('purgeFile is null for a module that is not on the volume', () => {
assert.equal(install.purgeFile('nothere'), null)
})

View File

@@ -125,6 +125,11 @@ function fakeModel(seed = []) {
if (!row || row.state === 'disabled') return
Object.assign(row, { state: 'startup_failed', failureStage: stage, failureReason: reason })
},
async disable(id) {
calls.push(`disable:${id}`)
const row = rows.get(id)
if (row) Object.assign(row, { state: 'disabled', failureStage: null, failureReason: null })
},
}
return model
}
@@ -374,3 +379,151 @@ test('a hook that throws does not stop the ones behind it', async () => {
await assert.doesNotReject(() => lifecycle.shutdown({ modules: loader }))
assert.deepEqual(noted(file), ['shutdown:zzz', 'shutdown:aaa'])
})
// ── stop(): the admin panel's Disable ──────────────────────────────────────
//
// Phase 4, §2.7.2 decision 3. Phase 2's disable moved a record and left the
// module running; the whole point of these tests is that it no longer does.
test('stop runs that one module\'s onShutdown and leaves the others alone', async () => {
const file = path.join(tmpRoot, 'log.txt')
writeModule('aaa', { boot: '', shutdown: '', log: file })
writeModule('bbb', { boot: '', shutdown: '', log: file })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
fs.writeFileSync(file, '')
const result = await lifecycle.stop('aaa', { modules: loader, model })
assert.deepEqual(result, { stopped: true, error: null })
// Only aaa. This is the difference from shutdown(), which runs everything.
assert.deepEqual(noted(file), ['shutdown:aaa'])
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
assert.equal(stateOf(loader, 'bbb').state, 'started', 'the other module keeps running')
assert.equal(model.rows.get('aaa').state, 'disabled')
})
test('stop moves the record only after the hook has run', async () => {
// While onShutdown runs, the module is still `started` — the only state in
// which its routes and the world it is tearing down agree with each other. The
// module reports its own view of itself, so a record moved too early shows up
// here as `disabled` instead of `started`.
const file = path.join(tmpRoot, 'log.txt')
const dir = path.join(tmpRoot, 'aaa')
fs.mkdirSync(dir, { recursive: true })
fs.writeFileSync(path.join(dir, 'module.json'), JSON.stringify({
id: 'aaa', name: 'A', version: '1.0.0', coreApi: '^1.0.0', server: 'index.js',
}))
fs.writeFileSync(path.join(dir, 'index.js'), `
module.exports = (ctx, api) => {
api.onBoot(async () => {})
api.onShutdown(async () => {
const loader = require(${JSON.stringify(require.resolve('../src/modules/loader'))})
const me = loader.list().find((m) => m.id === 'aaa')
require('fs').appendFileSync(${JSON.stringify(file)}, 'state-during-hook:' + me.state + ${JSON.stringify('\n')})
})
}`)
const loader = freshLoader(tmpRoot)
await lifecycle.boot({ modules: loader, model: fakeModel() })
await lifecycle.stop('aaa', { modules: loader, model: fakeModel([{ id: 'aaa', state: 'started' }]) })
assert.deepEqual(noted(file), ['state-during-hook:started'])
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
})
test('a hook that throws does not prevent the disable', async () => {
// The opposite of the boot path's rule, deliberately. There, a failure means
// the module never became safe to use; here, the operator has asked for it to
// stop answering and a module that could not close cleanly is a reason to log
// loudly, not a reason to leave it serving.
writeModule('aaa', { boot: '', shutdown: 'throw new Error("socket stuck")' })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
const result = await lifecycle.stop('aaa', { modules: loader, model })
assert.equal(result.stopped, false)
assert.match(result.error, /socket stuck/)
assert.equal(stateOf(loader, 'aaa').state, 'disabled', 'disabled anyway')
assert.equal(model.rows.get('aaa').state, 'disabled')
})
test('a hook that hangs costs its budget, and the module is still disabled', async () => {
writeModule('aaa', { boot: '', shutdown: 'await new Promise(() => {})' })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
const started = Date.now()
const result = await lifecycle.stop('aaa', { modules: loader, model, budgetMs: 50 })
assert.equal(result.stopped, false)
assert.match(result.error, /budget/)
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
assert.ok(Date.now() - started < 2000)
})
test('stopping a module with no onShutdown still disables it', async () => {
// A hookless module has nothing to run and must still stop answering, or the
// guard and the row disagree about what is serving.
writeModule('aaa', { boot: '' })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
const result = await lifecycle.stop('aaa', { modules: loader, model })
assert.deepEqual(result, { stopped: false, error: null })
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
assert.equal(model.rows.get('aaa').state, 'disabled')
})
test('stopping a module whose onBoot failed does not run its onShutdown', async () => {
// Same rule shutdownHooks() applies: a module that never finished warming up
// has a half-built world its onShutdown was not written for. It is still
// disabled — it just is not asked to tear anything down.
const file = path.join(tmpRoot, 'log.txt')
writeModule('aaa', { boot: 'throw new Error("no")', shutdown: '', log: file })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
const before = noted(file)
const result = await lifecycle.stop('aaa', { modules: loader, model })
assert.equal(result.stopped, false)
// Compared against what was already there rather than against an empty file:
// `noted` splits, so a blanked file reads as [''] and an emptiness assertion
// would pass for the wrong reason.
assert.deepEqual(noted(file), before, 'no new hook output')
assert.ok(!noted(file).includes('shutdown:aaa'))
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
})
test('stopping an unknown id writes the row and does not throw', async () => {
// A row can exist for a module that is not on the volume, and disabling it is
// exactly what an operator would do about that.
writeModule('aaa', { boot: '' })
const loader = freshLoader(tmpRoot)
const model = fakeModel([{ id: 'ghost', state: 'startup_failed' }])
await assert.doesNotReject(() => lifecycle.stop('ghost', { modules: loader, model }))
assert.equal(model.rows.get('ghost').state, 'disabled')
})
test('a row that will not write does not stop the module from being disabled', async () => {
// The same rule the boot path follows: bookkeeping failure is not the
// operation failing. The guard is what stops traffic, and it has already moved.
writeModule('aaa', { boot: '', shutdown: '' })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
model.disable = async () => { throw new Error('database is gone') }
await assert.doesNotReject(() => lifecycle.stop('aaa', { modules: loader, model }))
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
})

View File

@@ -25,7 +25,7 @@ const assert = require('node:assert/strict')
const express = require('express')
const db = require('../src/utils/db')
const { replayFragments } = require('../src/modules/schema')
const { replayFragments, runPurge } = require('../src/modules/schema')
const { splitStatements } = require('../src/utils/sqlStatements')
const { startApp } = require('./_helper')
@@ -245,3 +245,66 @@ test('replay is skipped, not thrown, when no scan happened in this process', asy
assert.deepEqual(rec.ran, [])
assert.equal(loader.isLoaded(), false)
})
// ── runPurge: the destructive twin ─────────────────────────────────────────
//
// Phase 4, slice 1. Same splitter, same pool, same serial execution as the
// replay above — pointed the other way. The two differences are deliberate and
// are what these tests are for.
test('purge runs every statement in the file, in order', async () => {
const file = path.join(tmpRoot, 'purge.sql')
fs.writeFileSync(file, [
'-- drop everything this module owns',
'DROP TABLE IF EXISTS mod_b;',
'DROP TABLE IF EXISTS mod_a;',
"DELETE FROM settings WHERE `key` = 'mod_thing';",
].join('\n'))
const rec = recorder()
const ran = await runPurge(file, { query: rec.query })
assert.equal(ran, 3)
assert.match(rec.ran[0], /DROP TABLE IF EXISTS mod_b/)
assert.match(rec.ran[2], /DELETE FROM settings/)
})
test('purge is not held to the fragment allowlist — DROP is the point of it', async () => {
// §2.6's leading-verb allowlist exists because a fragment replays on every
// boot. purge.sql never does, which is exactly why it is the one file a module
// may put a DROP in.
const file = path.join(tmpRoot, 'purge.sql')
fs.writeFileSync(file, 'DROP TABLE IF EXISTS mod_a;\nTRUNCATE TABLE mod_b;')
const rec = recorder()
assert.equal(await runPurge(file, { query: rec.query }), 2)
})
test('purge THROWS on failure, unlike the replay', async () => {
// A replay failure is one module failing to start, which the site survives by
// 503ing it. A purge failure is an operator's explicit destructive request not
// having happened — reporting success would leave them believing data is gone
// when it is not.
const file = path.join(tmpRoot, 'purge.sql')
fs.writeFileSync(file, 'DROP TABLE IF EXISTS mod_a;\nDROP TABLE mod_missing;')
const query = async (sql) => {
if (sql.includes('mod_missing')) throw new Error("Unknown table 'mod_missing'")
}
await assert.rejects(
() => runPurge(file, { query }),
// The message names WHICH statement stopped it, because a purge is not
// transactional — the ones before it have already committed and the operator
// needs to know where it got to.
/purge failed at statement 2 of 2: Unknown table 'mod_missing'/,
)
})
test('an empty purge file runs nothing and does not throw', async () => {
const file = path.join(tmpRoot, 'purge.sql')
fs.writeFileSync(file, '-- nothing to drop yet\n')
const rec = recorder()
assert.equal(await runPurge(file, { query: rec.query }), 0)
assert.deepEqual(rec.ran, [])
})