feat(modules): replay module schema fragments after core's
All checks were successful
PR Checks / bot-install (pull_request) Successful in 22s
PR Checks / client-build (pull_request) Successful in 28s
PR Checks / server-tests (pull_request) Successful in 10m9s

Phase 2, PR 3 of docs/website/MODULE_SYSTEM.md 2.7. ensureSchema() now replays
every installed module's schema fragment immediately after core's schema.sql,
per MODULE_API.md 2.6.

The work splits across two files on the line of whether a database is needed to
know the answer. loader.js VALIDATES a fragment at load time, before anything is
mounted, because every rule 2.6 states about the SQL is knowable by reading it;
a module that breaks one never mounts (4.4, left column). modules/schema.js
EXECUTES it, so the only failures there are the ones the database alone could
report, and those are post-mount and answer 503 (4.4, right column).

Validation is a leading-verb allowlist -- CREATE, ALTER, INSERT, UPDATE, the
four core's own schema.sql uses -- rather than the DROP denylist 2.6 words it
as. A fragment is replayed on every boot, so TRUNCATE and DELETE would empty a
table at each restart and RENAME would fail at the second one; a denylist only
ever bans what somebody thought of. A CREATE TABLE missing IF NOT EXISTS is
rejected for the same reason: it works once and fails every boot after, which
presents to an operator as a module that broke on restart.

The splitter moves to utils/sqlStatements.js so core's schema and a fragment are
split by literally the same code, which is what 2.6 promises. It is its own file
rather than an export of utils/db.js because the loader validates fragments at
require time and must not drag the mariadb pool into app.js's require chain.

The replay sits outside ensureSchema's wait-for-the-database retry loop: a
fragment that throws is one module's failure, not a signal the database is
coming up, and retrying core's whole schema nine more times over one module's
bad SQL would turn a 503'd module into a two-minute boot.

Found while wiring it: db/seed.js calls ensureSchema() standalone for
`npm run seed`, without ever requiring app.js, so the loader has not scanned and
fragments()'s 7.6 throw would have broken seeding outright. The replay asks
isLoaded() and logs the skip rather than swallowing it -- a booting server
quietly getting no module tables is the thing 7.6 exists to prevent.

Verification:

- 856 server tests pass, 14 new. moduleSchema.test.js injects the query fn, so
  the exact statements and their order are asserted with the pool at a dead port
  like every other suite.
- routes.manifest.json and routes.guards.json diffs are zero lines, 229 routes
  -- the phase 2 exit criterion. swagger-output.json regenerates byte-identical.
- Run for real against the local MariaDB with two fixture modules: a good
  fragment created its table, applied its ALTER and seeded its row; a fragment
  whose SQL passes validation but the server rejects (`id NOTATYPE`) marked only
  that module startup_failed, its route answering 503 while the other answered
  200; a second ensureSchema on the same database was a clean no-op.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2026-08-10 16:56:58 -05:00
parent 7780fb033b
commit 2892d01b24
6 changed files with 541 additions and 23 deletions

View File

@@ -284,7 +284,7 @@ test('registering the same thing twice is an error, not a silent overwrite', ()
assert.match(stateOf(loader, 'twice').reason, /registerRoutes\(\) called twice/)
})
// ── Schema fragment validation (the replay itself is PR 3) ─────────────────
// ── Schema fragment validation (the replay itself is moduleSchema.test.js) ──
test('a schema fragment declaring a core table is rejected', () => {
writeModule('thief', { schema: 'CREATE TABLE IF NOT EXISTS users (id INT);' })
@@ -332,6 +332,75 @@ test('a declared purge file that is not there is rejected', () => {
assert.match(stateOf(loader, 'gone').reason, /purge file "purge\.sql" is missing/)
})
test('a fragment may only use the four verbs core\'s own schema.sql uses', () => {
// An allowlist rather than a DROP denylist. §2.6 bans DROP, but this file is
// REPLAYED ON EVERY BOOT, so TRUNCATE and DELETE would empty a table at every
// restart and RENAME would fail at the second one — a denylist only ever bans
// what somebody thought of.
for (const [id, sql, verb] of [
['dropper', 'DROP TABLE dropper_x;', 'DROP'],
['nuker', 'TRUNCATE TABLE nuker_x;', 'TRUNCATE'],
['wiper', 'DELETE FROM wiper_x;', 'DELETE'],
['granter', 'GRANT ALL ON *.* TO app;', 'GRANT'],
]) {
writeModule(id, { schema: sql })
const reason = stateOf(freshLoader(tmpRoot), id).reason
assert.match(reason, new RegExp(`starts with "${verb}"`))
}
})
test('a fragment may INSERT and UPDATE its own seed data', () => {
// Core's schema.sql does both (INSERT IGNORE INTO settings, one UPDATE), so a
// module that seeds a lookup table the same way must not be rejected.
writeModule('seeder', {
schema: [
'CREATE TABLE IF NOT EXISTS seeder_kinds (id INT PRIMARY KEY, label VARCHAR(32));',
"INSERT IGNORE INTO seeder_kinds (id, label) VALUES (1, 'first');",
"UPDATE seeder_kinds SET label = 'first' WHERE id = 1;",
].join('\n'),
})
assert.equal(stateOf(freshLoader(tmpRoot), 'seeder').state, 'registered')
})
test('a CREATE TABLE without IF NOT EXISTS is rejected', () => {
// It succeeds exactly once and fails every boot after it, which presents as a
// module that worked until the first restart — the worst kind of bug to ship
// to an operator, and free to catch by reading the file.
writeModule('once', { schema: 'CREATE TABLE once_x (id INT);' })
assert.match(stateOf(freshLoader(tmpRoot), 'once').reason, /CREATE TABLE without IF NOT EXISTS/)
})
test('a fragment carrying an unreadable file fails the module, not the boot', () => {
writeModule('missing', { schema: 'CREATE TABLE IF NOT EXISTS missing_x (id INT);' })
fs.unlinkSync(path.join(tmpRoot, 'missing', 'schema.sql'))
writeModule('fine', oneRoute('/ok'))
const loader = freshLoader(tmpRoot)
assert.equal(stateOf(loader, 'missing').state, 'startup_failed')
assert.equal(stateOf(loader, 'fine').state, 'registered')
})
test('fragments() lists only registered modules that have one', () => {
writeModule('withdb', { schema: 'CREATE TABLE IF NOT EXISTS withdb_x (id INT);' })
writeModule('nodb', oneRoute('/plain'))
writeModule('broken', { schema: 'CREATE TABLE IF NOT EXISTS not_mine (id INT);' })
const loader = freshLoader(tmpRoot)
const frags = loader.fragments()
assert.deepEqual(frags.map((f) => f.id), ['withdb'])
// An absolute path, so the replay never has to know how a module dir is laid out.
assert.equal(frags[0].file, path.join(tmpRoot, 'withdb', 'schema.sql'))
})
test('fragments() before load() throws, like every other accessor', () => {
process.env.MODULES_DIR = tmpRoot
delete require.cache[require.resolve('../src/modules/loader')]
// eslint-disable-next-line global-require
const loader = require('../src/modules/loader')
assert.throws(() => loader.fragments(), /modules\.fragments\(\) before modules\.load\(\)/)
})
// ── Mounting and the dispatch guard ────────────────────────────────────────
test('a registered module answers on its prefix; a failed one is simply absent', async () => {