7 Commits

Author SHA1 Message Date
92c6972344 test(login): stop the backoff-guard test racing its own one-second lock
A single recordFailure() locks for BASE_MS * 2 ** 0 — exactly one second — and
the test then does a real HTTP round trip against it. On CI that round trip took
1,456 ms and the guard correctly answered 200, failing the run for a reason that
has nothing to do with what the test is about.

Five failures lock for sixteen seconds. The subject is the guard's answer while
locked out, which is unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-12 08:04:56 -05:00
9b16f39a52 feat(modules): the declarative Docker path (phase 4, slice 3)
Some checks failed
PR Checks / bot-install (pull_request) Successful in 23s
PR Checks / client-build (pull_request) Successful in 30s
PR Checks / server-tests (pull_request) Failing after 4m23s
MODULES declares the module set a deployment runs, one entry per module as
`<id>@<version>=<install manifest URL>`, and the container arrives at it by
itself (MODULE_SYSTEM.md §2.7.2 decision 4). A module already unpacked at the
declared version is a no-op that makes NO network call, so a restart with the
network down comes up unchanged; anything else goes through install.js — same
allowlist, same sha256, same inspect-then-extract — and install() now takes an
`expect: {id, version}` so a URL resolving to another module or version is
refused while it is still only a manifest.

Resolution runs inside start(), between the seed and the require of app.js: the
seed is where the host allowlist setting comes from, and the require is what
scans the volume. That buys it the database, so a compose-installed module gets
the same provenance columns an admin install writes.

A failure is logged and carried, never fatal — an unreachable release host must
not take the site down. The declaration owns what is on the volume; the row owns
whether a module runs, so uninstalling a declared module returns its files at
the next start and leaves it disabled. The admin list gains that as a fourth
source (declared / declaredVersion / declaredError), because a declared module
that failed to resolve has no row, no directory and nothing mounted.

Deferring the app require moved core's schema ahead of the volume scan, and the
module schema-fragment replay was wired to core's schema — so every installed
module silently got no tables. Invisible to the suite (each one stubs the loader
or the pool) and to a smoke on a database that already had the tables; found by
booting against an empty one. ensureSchema() now takes `replayModules: false`
for the one caller that scans later, server.js replays them itself after the
require, and a bootOrder test pins the five steps in the only order they work in.

741 server tests (+18), 187 client (+5); manifest unchanged at 166 public + 2
internal, OpenAPI byte-identical.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-12 07:57:08 -05:00
75f4d29e93 Merge pull request 'feat(admin): the Modules screen (phase 4, slice 2)' (#143) from feature/module-admin-screen into edge
Reviewed-on: #143
2026-08-12 08:52:23 +00:00
9083e4135a feat(admin): the Modules screen (phase 4, slice 2)
The screen slice 1's API was written for: install from a release URL, enable,
disable, uninstall, purge, and restart. Admin-only, matching the server, and
core's own screen because it is how a module reaches the volume at all.

182 client tests (+21), manifest and OpenAPI unchanged.

Everything that decides what a row SAYS and which buttons it offers is in
`lib/moduleAdmin.js` -- plain JS, so the DOM-less runner can reach it, the
same reason `lib/adminNav.js` is. The JSX renders what it returns.

Three sources of truth, and they are allowed to disagree
--------------------------------------------------------
The row records what the operator decided and what the last boot did; the
loader says what is mounted and answering; the volume says whether there is a
directory at all. Picking one and rendering it is simpler and lies. The case
that makes it concrete is the one decision 3 creates on purpose: disable a
module (its onShutdown runs) and enable it again, and the row says `enabled`
while the loader still says `disabled` because nothing can start it before a
restart. Neither "Running" nor "Disabled" is true; "Restart to start" is.

Two shapes that are deliberately unlike the rest of the panel: the restart is
a BANNER, because a restart is a property of the server rather than of a
module and an operator who installed three modules should restart once; and
purge is offered inside the uninstall flow as a second confirm, because
purge.sql lives inside the directory being deleted and there is no later.

What the browser found that no test could
-----------------------------------------
Installing over a row the previous boot had left `startup_failed` rendered
"Failed at the require stage: module directory not present on the volume" one
second after the files had been written to the volume -- and, because that
branch is not pending, it suppressed the restart banner the install had just
told the operator to use. Every unit test passed, because none of them had
modelled a stale row plus a fresh install.

The fix is a derivation rather than a special case: the loader scans the
volume once at require time, so a module that is on the volume now and has no
live record arrived after that scan, and everything the row says about it
predates the install. That check runs before the failure one.

The same class, one place further on: an upgrade leaves the old code loaded,
so the row's version is a promise about the next boot. `liveVersion` (slice 1)
lets the screen say "Restart to finish upgrading" instead of reporting the new
version as running.

Verified against a live server and the real published release: pasted the
v0.3.0 install-manifest URL, restarted, watched the module register its five
mounts and seven streams and its own nav rows appear in the sidebar. Disable
ran its onShutdown for real -- the uo-link WebSocket closed, its routes went
to 404, and it left /public/modules -- and enable then showed the decision-3
state with the banner. The restart button itself was exercised through its
endpoint rather than clicked, because a window.confirm wedges the browser
automation.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-12 03:49:02 -05:00
732927a6bb fix(modules): three defects a real install exposed (phase 4, slice 1)
Standing the slice-2 screen up against a live server and installing the
published module-uo v0.3.0 through it found three things, none of which any
unit test in this repo could have caught. Two of them are older than this
phase.

1. The boot refresh nulled every install's provenance
--------------------------------------------------------
`installed_modules.source` and `.sha256` exist so the admin panel can say
where a module came from. They never survived a restart.

`lifecycle.boot()` re-records every scanned module with no source and no
sha256 -- correctly, because a scan finds a directory and never where it came
from -- and `upsert` assigned both columns unconditionally. So an install's
provenance lasted exactly until the restart that install asked for, and the
screen then described a module installed from a URL as "placed on the volume
by hand". Verified live: install, restart, provenance gone.

Nothing could have caught it before now. Phase 4 wrote the first non-null
value these columns had ever had, so lifecycle.js's comment asserting that
"recordInstalled leaves what it is not given" described an intention rather
than the statement below it -- and modules.model.test.js's fake reproduced
the defect faithfully, assigning unconditionally just like the SQL.

Fixed with COALESCE(VALUES(col), col): a value overwrites, a NULL leaves what
is there. The fake now matches, and two tests pin both directions -- a boot
refresh must not wipe it, and a re-install from a new URL must still replace
it, or the column would become write-once and an upgrade would for ever show
where the first version came from.

2. The restart killed the server on Windows instead of stopping it
------------------------------------------------------------------
The route called `process.kill(process.pid, 'SIGTERM')` to reach server.js's
graceful-shutdown handler. That works on Linux. **Windows has no POSIX
signals, and Node documents SIGTERM there as unconditional termination of the
target process** -- so on a Windows host the restart killed the server
outright: no module onShutdown, no listener close, no pool close, no log
flush. Observed exactly that: the process was gone and the shutdown handler
had logged nothing at all.

`process.on('SIGTERM', ...)` is an ordinary EventEmitter listener, so
`process.emit('SIGTERM')` reaches the same handler on every platform without
involving the OS. One shutdown path, still; it just gets there by an event.

Deployment is Linux containers and would never have shown this. Development
is not, and neither is the smoke that found it.

The test was worse than useless: it stubbed `process.kill` and asserted it
had been called with SIGTERM, which is precisely the call whose MEANING
differs by platform. It now waits for the SIGTERM EVENT -- what server.js is
actually subscribed to -- so a pass here means the handler would run.

3. `present()` did not publish the running version
--------------------------------------------------
An upgrade writes new files and a new row while the old code stays loaded, so
the row's version is a promise about the next boot rather than a description
of this one. Adds `liveVersion` from the loader beside `liveState`, so the
screen can tell the two apart instead of reporting the new version as running.

723 server tests (+2), manifest and OpenAPI both unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-12 03:48:27 -05:00
50d719cf46 Merge pull request 'feat(modules): install, uninstall, purge and restart (phase 4, slice 1)' (#142) from feature/module-install-service into edge
Reviewed-on: #142
2026-08-12 08:31:38 +00:00
b30e82cde2 feat(modules): install, uninstall, purge and restart (phase 4, slice 1)
All checks were successful
PR Checks / bot-install (pull_request) Successful in 17s
PR Checks / client-build (pull_request) Successful in 26s
PR Checks / server-tests (pull_request) Successful in 33s
The consumer half of a release module-uo's CI has been publishing since
phase 3 closed. Before this, core had the installed_modules provenance
columns and no code that could ever fill them: nothing fetched, verified,
unpacked, removed or purged anything, and there was no admin route at all.

Adds modules/archive.js, modules/install.js, schema.runPurge(),
lifecycle.stop(), loader.stopHook(), and /api/v1/admin/modules with eight
routes. 797 server tests (+76), manifest 158 -> 166 + 2 internal, OpenAPI
gains 8 operations and loses nothing.

Reject, never sanitise
----------------------
The download is the easy part: an https-only allowlist re-checked on every
redirect hop, a declared sha256 compared against the bytes that arrived, and
a byte cap. Unpacking is where the archive chooses the filenames, and core
writes into a directory bind-mounted from the host, so an escape is not
confined to the container.

archive.js inspects the whole archive before a byte is unpacked and refuses
absolute and drive-absolute paths, `..` segments, NUL bytes, backslashes,
anything that is not a regular file or a directory, more than one top-level
entry, and anything over the entry or byte caps. Refusing symlinks and
hardlinks outright is what keeps this off the majority of node-tar's
published advisories rather than depending on the library to contain them.

That two-pass shape is load-bearing, and it was measured rather than assumed:
extracting an archive whose fourth member escapes upward throws under
node-tar 7.5.22 -- and leaves the first three members on disk. The loader
scans that directory at require time on the next boot, so a half-unpacked
module is a module. Everything therefore happens in a scratch directory that
is removed on any failure, and the move into place is the last step.

`tar` is pinned to ^7.5.22 rather than the ^6 that installs by default: 6.x
is flagged critical, and reading the advisory list is what the file's header
now says out loud -- almost all of it is hardlink or symlink traversal and
PAX header interpretation differentials, which is exactly this feature's
threat model.

Two things the plan had wrong
-----------------------------
The bundle's top-level directory is `module-uo-<version>`, not the module id
-- so "the top-level name must equal the id" was checked against nothing real.
The extractor strips that level instead, because its name belongs to whoever
published the bundle and the directory it lands in is core's. What is checked
instead is the unpacked module.json: a manifest promising `uo` and delivering
something else is refused rather than installed under the name it promised.

And purge cannot be a follow-up action (decision 5): purge.sql lives inside
the directory uninstall deletes. It is offered in the uninstall flow and as a
standalone action on a still-installed module, and the standalone one refuses
unless the module is already disabled -- dropping tables under something that
is still serving leaves it answering out of a world that no longer exists.

Disable now means stopped
-------------------------
lifecycle.stop() dispatches that one module's onShutdown before flipping the
guard, so a module an operator switches off actually releases its sockets and
closes its streams instead of merely becoming unreachable. The hook runs
first and the state moves after it, because while onShutdown runs the module
is still `started` and that is the only state in which its routes and the
world it is tearing down agree. A hook that throws does not stop the disable
-- the opposite of the boot path's rule, and deliberately.

Enable is not its mirror and there is no start(id) beside it. There is no
onBoot re-dispatch and the hooks were never promised re-entrant, so enable
moves the row and the restart route starts it. A test pins that enable does
not touch the loader, because "fixing" it is a one-line change that would put
a module with closed sockets back on the nav.

Restart raises SIGTERM against its own process rather than calling the
shutdown path directly, so server.js's handler stays the one graceful-shutdown
path and this route cannot drift from it.

The allowlist bootstraps from MODULE_SOURCE_HOSTS into a settings row and is
admin-managed after that (decision 6); seedDefault is INSERT IGNORE, so
changing the variable on an existing deployment is a no-op by design. An empty
list forbids every install rather than allowing every host -- the safe
direction for a value someone might blank by accident.

Verified against the real v0.3.0 release
----------------------------------------
Not a fixture: fetched the published install manifest over the real Gitea
host and its redirect chain, verified the sha256, inspected and unpacked the
252,517-byte artifact to 82 files, and then booted core against the result --
the module registered its five mounts, seven streams and eight capabilities
and resolved its client chunk, with no scratch directory left behind.

Two defects this slice's own tooling caught, both of which had already been
written down as classes:
  - the controller destructured runPurge at require time, capturing the
    function rather than the module, which made the one dependency whose
    ORDER matters the one that could not be substituted;
  - two swagger annotations carried an apostrophe inside a quoted string,
    dropped silently by swagger-autogen before slice 5 taught it to fail loudly.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-12 03:09:45 -05:00
38 changed files with 5463 additions and 19 deletions

View File

@@ -113,6 +113,20 @@ BOT_INTERNAL_KEY=change-me-to-a-long-random-string
# knows nothing about any of them. A module may read its own env vars, and they
# belong here because Compose passes this file to the container.
#
# MODULES declares the set this deployment runs, and the container arrives at it
# on its own — no admin panel, no `tar -xf` on the host. One entry per module,
# `<id>@<version>=<install manifest URL>`, whitespace- or comma-separated:
#
# MODULES=uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json
#
# A module already unpacked at the declared version is a no-op that never touches
# the network, so a restart with the internet down brings the site up exactly as
# it was; only a missing or different version is fetched, verified against the
# sha256 its manifest declares, and unpacked. A failure is logged and shown in
# Admin → Modules, and the site starts anyway. The variable owns what is on the
# volume, not what runs — a module disabled from the admin panel stays disabled.
# Leave it unset to install from the admin panel instead.
#
# RunicGateway/Module-uo, for example, reads UOLINK_BASE_URL / UOLINK_WS_URL /
# UOLINK_PROTOCOL as the defaults for its connection to a uo-link sidecar, and
# TOWNCRIER_DURATION_SEC for its news leg. Its README documents them; they are

View File

@@ -489,6 +489,32 @@ modules/
supported install. The directory is tracked in git (via its README) on purpose: Docker recreates a
*missing* bind-mount source as `root:root`, and the container is uid 1000.
### Three ways in, and none of them is a build
| | How | Where it fits |
|---|---|---|
| **Admin panel** | Admin → Modules, paste the URL of a release's install manifest | The click path. Installs, upgrades, disables, uninstalls and purges, with a restart button — no shell on the box |
| **`MODULES`** | Declare the set in the environment; the container resolves it at every start | The compose-managed host. The running set is a line in a file you version-control, not the residue of past clicks |
| **By hand** | `tar -xf module-uo-0.3.0.tar.gz -C ./modules && mv modules/module-uo-0.3.0 modules/uo`, then restart | Development, and any host where the other two do not fit |
`MODULES` takes one entry per module, whitespace- or comma-separated:
```
MODULES=uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json
```
The id and the version are written out rather than discovered inside the manifest so that **the
no-op case needs no network**: a module already unpacked at the declared version is answered by
reading its own `module.json`, so a restart with the internet down brings the site up exactly as it
was. Only a missing or different version is fetched, and it goes through the same
verify-and-unpack path — allowlisted `https` host, sha256 from the manifest, whole-archive
inspection before anything is written — that the admin panel uses. A version that cannot be
resolved is logged and shown on the admin screen; **it never stops the site from starting**.
The declaration owns what is *on the volume*, never what runs. A module disabled from the admin
panel gets its files back at the next start and stays disabled, because the row and the variable are
answering different questions.
### What a module gets, and what it may not do
At boot, `app.js` scans the volume synchronously, validates each `module.json`, and calls the
@@ -534,6 +560,8 @@ Copy `.env.example` (Compose) or `server/.env.example` (local) and fill in. **`.
| `PORT` | `3000` | server listens on `0.0.0.0:PORT` |
| `UPLOAD_DIR` | `<server>/uploads` | where post images are written (`/app/uploads`, volume-mounted, in Compose) |
| `MODULES_DIR` | `<repo>/modules` | where installed modules are scanned from (`/app/modules`, bind-mounted, in Compose) |
| `MODULES` | — | the module set this deployment runs, resolved at every start: `<id>@<version>=<install manifest URL>`, whitespace/comma separated. Already at the declared version = no network. A failure is logged and shown in Admin → Modules, never fatal. See [Modules](#modules) |
| `MODULE_SOURCE_HOSTS` | `gitea.whitlocktech.com` | **bootstrap only** — seeds the `module_source_hosts` setting on first boot; after that the setting is authoritative and is edited in Admin → Modules |
| `DB_HOST` / `DB_PORT` | `db` / `3306` | `db` in Compose; `127.0.0.1` for local dev |
| `DB_NAME` / `DB_USER` / `DB_PASSWORD` | `runic_gateway` / `runic` / — | app database credentials |
| `DB_ROOT_PASSWORD` | — | MariaDB root (Compose only) |

View File

@@ -41,6 +41,7 @@ import AuthProvidersAdmin from './routes/admin/views/AuthProvidersAdmin.jsx'
import UsersAdmin from './routes/admin/views/UsersAdmin.jsx'
import UserDetail from './routes/admin/views/UserDetail.jsx'
import InvitesAdmin from './routes/admin/views/InvitesAdmin.jsx'
import ModulesAdmin from './routes/admin/views/ModulesAdmin.jsx'
import AccountAdmin from './routes/admin/views/AccountAdmin.jsx'
import Moderation from './routes/admin/views/Moderation.jsx'
import ModerationUser from './routes/admin/views/ModerationUser.jsx'
@@ -169,6 +170,10 @@ export default function App() {
<Route path="users" element={<UsersAdmin />} />
<Route path="users/:id" element={<UserDetail />} />
<Route path="invites" element={<InvitesAdmin />} />
{/* Core's own screen, and it has to be: it is how a module reaches
the volume in the first place. Declared here with the rest of
core's routes, above the module-supplied ones below. */}
<Route path="modules" element={<ModulesAdmin />} />
<Route path="account" element={<AccountAdmin />} />
{/* Installed modules' admin pages, at /admin/<id>/…, already inside
RequireAuth + AdminLayout. A module cannot supply its own auth

View File

@@ -233,6 +233,20 @@ export const api = {
req('/admin/invites', { method: 'POST', body: { email, role, sendEmail } }),
revokeInvite: (id) => req(`/admin/invites/${id}`, { method: 'DELETE' }),
// Installed modules (MODULE_SYSTEM.md §2.7.2). `uninstallModule`'s purge flag
// is a query parameter rather than a body because it hangs off a DELETE, and
// it is spelled out at the call site rather than defaulted, so the
// destructive branch is never the one you get by forgetting an argument.
listModules: () => req('/admin/modules'),
installModule: (url) => req('/admin/modules', { method: 'POST', body: { url } }),
enableModule: (id) => req(`/admin/modules/${encodeURIComponent(id)}/enable`, { method: 'POST' }),
disableModule: (id) => req(`/admin/modules/${encodeURIComponent(id)}/disable`, { method: 'POST' }),
uninstallModule: (id, { purge } = {}) =>
req(`/admin/modules/${encodeURIComponent(id)}${purge ? '?purge=true' : ''}`, { method: 'DELETE' }),
purgeModule: (id) => req(`/admin/modules/${encodeURIComponent(id)}/purge`, { method: 'POST' }),
setModuleSources: (hosts) => req('/admin/modules/sources', { method: 'PUT', body: { hosts } }),
restartServer: () => req('/admin/modules/restart', { method: 'POST' }),
// ----- moderation dashboard (admin + moderator) -----
modSummary: () => req('/admin/moderation/stats/summary'),
modRecent: (params = {}) => {

View File

@@ -0,0 +1,252 @@
// What an admin should be told about one installed module, and what they may do
// to it — derived, not spelled out at each button.
//
// Phase 4, slice 2 of docs/website/MODULE_SYSTEM.md §2.7.2. Plain JS rather than
// a hook or a chunk of JSX, for the same reason `lib/adminNav.js` is: the test
// runner here has no DOM, and this is the part of the Modules screen that is
// actually worth testing.
//
// **The screen has four sources of truth and they are allowed to disagree**
// (MODULE_SYSTEM.md §2.4, and slice 3 for the fourth):
//
// state what the DATABASE row records — what the operator decided, and
// what the last boot ended up doing
// liveState what the LOADER has mounted in this process and is answering with
// onVolume whether there is still a directory there at all
// declared what this container's MODULES variable asks for — the only one of
// the four that no button on this screen can change
//
// Picking one and rendering it would be simpler and would lie. The case that
// makes this concrete is the one decision 3 creates on purpose: an operator
// disables a module (its onShutdown runs, its routes 404) and then enables it
// again. The row says `enabled`; the loader still says `disabled`, because
// there is no `onBoot` re-dispatch and nothing can start it before a restart.
// It is neither running nor off, and the honest thing to show is "enabled —
// restart to start it".
/**
* The one-line status of a module, and whether that status is waiting on a
* restart.
*
* Ordering matters here. The checks run most-alarming first, so a module whose
* directory has been deleted is described that way rather than by whatever its
* row happens to still say.
*
* @param {object} m a row from GET /admin/modules
* @returns {{ label: string, tone: 'ok'|'warn'|'bad'|'idle', pending: boolean, detail: string }}
*/
export function statusOf(m) {
// Declared by the environment and not there at all: no row, no directory,
// nothing mounted. Every other branch below reads one of those three, so
// without this the screen would describe a module it has never had as though
// a row had gone stale — and the one thing the operator needs, the reason
// resolution failed, would be nowhere.
if (m.declared && !m.onVolume && m.state === null) {
return {
label: 'Declared, not installed',
tone: 'bad',
pending: false,
detail: m.declaredError
? `MODULES asks for v${m.declaredVersion}; the last start could not install it: ${m.declaredError}`
: `MODULES asks for v${m.declaredVersion}. It will be installed when the server next starts.`,
}
}
// Gone from the volume, but still known. Either a hand-deleted directory (the
// boot reconcile marks that `startup_failed`) or an uninstall waiting for its
// restart. Both are "there is nothing to run here".
if (!m.onVolume) {
return {
label: m.state === 'disabled' ? 'Uninstalled' : 'Missing from the volume',
tone: m.state === 'disabled' ? 'idle' : 'bad',
pending: m.liveState !== null,
detail: m.state === 'disabled'
? 'The files are gone. Its data was kept, and reinstalling brings it back.'
: 'A row exists but there is no module directory. Reinstall it, or uninstall to clear the row.',
}
}
// **Installed since this process booted**, and this check has to come before
// the failure one. `liveState` is the loader's record, and the loader scans
// the volume once at require time — so a module that is on the volume NOW and
// has no live record was put there after the scan. Anything the row still says
// about it therefore predates the install and is stale by definition.
//
// Found by the §7.7 browser smoke, and no unit test here had modelled it:
// installing over a row left `startup_failed` by the previous boot rendered
// "Failed at the require stage: module directory not present on the volume"
// one second after the file had been written to the volume — and, because that
// branch is not pending, suppressed the restart banner the install had just
// told the operator to use.
if (m.liveState === null) {
return {
label: 'Restart to start',
tone: 'warn',
pending: true,
detail: 'Installed. It mounts when the server next starts.',
}
}
if (m.state === 'startup_failed' || m.liveState === 'startup_failed') {
return {
label: 'Failed to start',
tone: 'bad',
pending: false,
detail: m.failureReason
? `Failed at the ${m.failureStage || 'unknown'} stage: ${m.failureReason}`
: 'It failed to start and recorded no reason.',
}
}
if (m.state === 'disabled') {
return {
label: 'Disabled',
tone: 'idle',
pending: false,
detail: 'Stopped and switched off. Its routes answer 404 and it stays off across restarts.',
}
}
// The row has been switched on but the loader has not started it — the
// decision-3 case: disable ran its onShutdown, and nothing can start it again
// before a restart.
if (m.liveState !== 'started') {
return {
label: 'Restart to start',
tone: 'warn',
pending: true,
detail: m.liveState === 'disabled'
? 'Enabled, but still stopped in the running server — it cannot be restarted in place.'
: 'Enabled. It mounts when the server next starts.',
}
}
// Running, but not the version that is installed. An upgrade writes new files
// and a new row while the old code stays loaded, so the row's `version` is a
// promise about the next boot rather than a description of this one — and
// "Running v2.0.0" beside a process serving v1.0.0 is the same lie as the
// stale-failure one above, in a different place.
if (m.liveVersion && m.liveVersion !== m.version) {
return {
label: 'Restart to finish upgrading',
tone: 'warn',
pending: true,
detail: `v${m.version} is installed; v${m.liveVersion} is still running.`,
}
}
return {
label: 'Running',
tone: 'ok',
pending: false,
detail: 'Mounted and serving.',
}
}
/**
* What the environment's declaration means for this module, as one sentence — or
* null if nothing declares it.
*
* Kept out of `statusOf` on purpose. A module can be running perfectly while its
* declared upgrade is failing, and collapsing both into one label would have to
* pick which of the two is "the" status. This is a second line, beside the first.
*
* The sentence an operator most needs is the uninstall one: MODULES owns what is
* on the volume and the row owns whether it runs, so uninstalling a declared
* module puts its files back at the next start and leaves it switched off. Files
* reappearing unexplained is exactly the kind of thing that gets debugged for an
* afternoon.
*
* @param {object} m a row from GET /admin/modules
* @returns {{ text: string, tone: 'warn'|'idle' }|null}
*/
export function declarationNoteFor(m) {
if (!m.declared) return null
if (m.declaredError) {
return {
text: `MODULES asks for v${m.declaredVersion} and the last start could not install it: ${m.declaredError}`,
tone: 'warn',
}
}
if (!m.onVolume) {
return {
text:
`MODULES declares v${m.declaredVersion}, so its files come back when the server next starts`
+ (m.state === 'disabled' ? ' — switched off, until you enable it.' : '.'),
tone: 'warn',
}
}
return { text: `Declared by this deployment's MODULES variable at v${m.declaredVersion}.`, tone: 'idle' }
}
/**
* Which actions are offered for a module, and why the others are not.
*
* Returned as a map of `{ shown, reason }` rather than a list of shown actions,
* so a disabled button can say what would make it available. Every rule here
* mirrors one the server enforces — this is presentation, never the boundary.
*
* @param {object} m a row from GET /admin/modules
*/
export function actionsFor(m) {
const running = m.liveState === 'started'
const disabled = m.state === 'disabled'
return {
// Only offered while something is actually running: disabling a module that
// is already stopped has nothing to stop and no guard to flip.
disable: {
shown: !disabled && m.onVolume,
reason: disabled ? 'Already disabled.' : 'Nothing is running to stop.',
},
enable: {
shown: disabled && m.onVolume,
reason: 'Only a disabled module can be enabled.',
},
uninstall: {
shown: m.onVolume,
reason: 'There are no files left to remove.',
},
// The server refuses a standalone purge unless the module is disabled, so
// the button says so rather than offering a click that 409s.
purge: {
shown: m.onVolume && m.canPurge,
enabled: disabled,
reason: !m.canPurge
? 'This module ships no purge.sql, so its data cannot be deleted.'
: 'Disable it first, so nothing is serving out of the tables being dropped.',
},
// A row with no directory is the one thing an uninstall cannot tidy through
// the normal path — offer clearing it instead.
forget: {
shown: !m.onVolume && m.state !== null,
reason: 'The module is still installed.',
},
running,
}
}
/**
* Does anything on this list need a restart before it matches what is running?
*
* Drives the one banner at the top of the screen rather than a badge per row:
* the restart is a property of the SERVER, not of a module, and offering it
* five times would suggest otherwise.
*/
export const needsRestart = (modules) => modules.some((m) => statusOf(m).pending)
/**
* Split a hosts string the way the server will.
*
* Duplicated from `install.parseHosts` deliberately — it is four lines, and the
* alternative is an API round trip to preview what the field is going to mean.
* The server remains the one that decides; this only shows the operator how
* their typing will be read.
*/
export function parseHosts(value) {
return String(value || '')
.split(/[,\s]+/)
.map((h) => h.trim().toLowerCase())
.filter(Boolean)
}

View File

@@ -45,6 +45,7 @@ const IconPulse = () => <Icon><path d="M3 12h3l2 6 4-14 2 8h7" /></Icon>
const IconUser = () => <Icon><circle cx="12" cy="8" r="4" /><path d="M4 21a8 8 0 0 1 16 0" /></Icon>
const IconNav = () => <Icon><path d="M4 6h16M4 12h16M4 18h10" /><circle cx="18" cy="18" r="2.5" /></Icon>
const IconPalette = () => <Icon><path d="M12 3a9 9 0 1 0 0 18 2 2 0 0 0 1.6-3.2 2 2 0 0 1 1.6-3.2H18a3 3 0 0 0 3-3 9 9 0 0 0-9-8.6z" /><circle cx="7.5" cy="11.5" r="1" /><circle cx="10.5" cy="7.5" r="1" /><circle cx="15" cy="8.5" r="1" /></Icon>
const IconModules = () => <Icon><path d="M12 3l8 4.5-8 4.5-8-4.5z" /><path d="M4 12l8 4.5 8-4.5" /><path d="M4 16.5L12 21l8-4.5" /></Icon>
// Nav is grouped into collapsible categories. A group with no `title` renders
// its items ungrouped (Dashboard at top, Account at bottom). Each item's `roles`
@@ -83,6 +84,10 @@ export const NAV = [
{ to: '/admin/users', label: 'Users', icon: IconUsers, roles: ['admin'] },
{ to: '/admin/invites', label: 'Invites', icon: IconUsers, roles: ['admin'] },
{ to: '/admin/settings', label: 'Settings', icon: IconGear, roles: ['admin'] },
// Admin-only, matching the server: every route under /admin/modules
// re-gates to `admin` on top of the group's staff gate, because installing
// a module runs its code in this process.
{ to: '/admin/modules', label: 'Modules', icon: IconModules, roles: ['admin'] },
{ to: '/admin/appearance', label: 'Appearance', icon: IconPalette, roles: ['admin'] },
{ to: '/admin/navigation', label: 'Navigation', icon: IconNav, roles: ['admin'] },
{ to: '/admin/hero', label: 'Hero Editor', icon: IconHero, roles: ['admin'] },

View File

@@ -0,0 +1,424 @@
import { useCallback, useEffect, useState } from 'react'
import { Loading, ErrorState } from '../../../components/PageState.jsx'
import { dateTime } from '../../../lib/format.js'
import { statusOf, actionsFor, declarationNoteFor, needsRestart, parseHosts } from '../../../lib/moduleAdmin.js'
import { api } from '../../../api/client.js'
// Installed modules: install from a release URL, enable, disable, uninstall,
// purge, and restart the server so the changes take effect.
//
// Phase 4, slice 2 of docs/website/MODULE_SYSTEM.md §2.7.2. Everything that
// decides what a row SAYS and which buttons it offers lives in
// lib/moduleAdmin.js, which is plain JS and has tests; this file renders it.
//
// Two things about this screen are unlike the rest of the admin panel and are
// deliberate:
//
// 1. **Restart is a banner, not a per-row button.** A restart is a property of
// the server, not of a module. Offering it on five rows would suggest
// otherwise, and an operator who installed three modules should restart
// once.
// 2. **Disable is the only action that takes effect immediately.** Everything
// else is "true after the next boot", because the loader reads the volume
// at require time (§1.12). The buttons say which they are.
const TONE = {
ok: '#7fd0a4',
warn: 'var(--accent)',
bad: '#d98b84',
idle: 'var(--muted)',
}
const DANGER = { color: '#d98b84', borderColor: '#5b2020' }
function Pill({ tone, children }) {
return (
<span
className="badge"
style={{ color: TONE[tone] || 'var(--muted)', borderColor: 'var(--line)', background: 'var(--panel-flat)' }}
>
{children}
</span>
)
}
// ── Install ────────────────────────────────────────────────────────────────
function InstallForm({ sourceHosts, onInstalled }) {
const [url, setUrl] = useState('')
const [busy, setBusy] = useState(false)
const [error, setError] = useState('')
const [result, setResult] = useState(null)
async function submit(e) {
e.preventDefault()
setError('')
setResult(null)
if (!url.trim()) return setError('Paste the URL of a release install manifest.')
setBusy(true)
try {
const res = await api.admin.installModule(url.trim())
setResult(res)
setUrl('')
await onInstalled()
} catch (err) {
// The server's message is written to be read by whoever pasted the URL —
// which host was refused, which hash did not match, what the archive
// contained. Replacing it with something friendlier would throw away the
// only part that helps.
setError(err.message || 'Could not install that module.')
} finally {
setBusy(false)
}
}
return (
<div className="panel" style={{ padding: 22, marginBottom: 22 }}>
<div className="field-label" style={{ marginBottom: 10 }}>Install a module</div>
<form onSubmit={submit} style={{ display: 'flex', gap: 12, alignItems: 'flex-end', flexWrap: 'wrap' }}>
<label style={{ flex: '1 1 380px' }}>
<span className="field-label">Release install-manifest URL</span>
<input
type="url"
value={url}
onChange={(e) => setUrl(e.target.value)}
className="input"
placeholder="https://gitea.example.com/org/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json"
/>
</label>
<button type="submit" disabled={busy} className="btn btn-primary btn-sq">
{busy ? 'Installing…' : 'Install'}
</button>
</form>
<p className="sans" style={{ margin: '12px 0 0', fontSize: '0.8rem', color: 'var(--muted)' }}>
The bundle is downloaded, checked against the <code>sha256</code> its release published, and
unpacked onto the modules volume. It starts serving after a restart.{' '}
{sourceHosts.length === 0
? 'No source hosts are allowed yet — add one below before installing.'
: `Allowed hosts: ${sourceHosts.join(', ')}.`}
</p>
{error && <p className="sans" style={{ margin: '12px 0 0', color: TONE.bad, fontSize: '0.85rem' }}>{error}</p>}
{result && (
<p className="sans" style={{ margin: '12px 0 0', color: TONE.ok, fontSize: '0.85rem' }}>
{result.replaced ? 'Upgraded' : 'Installed'} {result.module?.name} v{result.module?.version}. Restart to load it.
</p>
)}
</div>
)
}
// ── The restart banner ─────────────────────────────────────────────────────
function RestartBanner({ onDone }) {
const [busy, setBusy] = useState(false)
const [sent, setSent] = useState(false)
async function restart() {
// Said plainly, because it is true and because the failure mode is bad: a
// deployment with no supervisor does not come back on its own.
const ok = window.confirm(
'Restart the server now?\n\n'
+ 'The site will be briefly unavailable. It comes back on its own only if something is '
+ 'supervising the process — the shipped Docker Compose file does. If you are running '
+ '`npm start` by hand, you will have to start it again yourself.',
)
if (!ok) return
setBusy(true)
try {
await api.admin.restartServer()
setSent(true)
// Nothing is coming back on this connection: the process is exiting. Give
// the supervisor a moment and then reload, which is what the operator was
// about to do anyway.
setTimeout(() => { if (onDone) onDone() }, 6000)
} catch {
// A failed request here is expected as often as not — the process can win
// the race and drop the socket before the response lands.
setSent(true)
setTimeout(() => { if (onDone) onDone() }, 6000)
} finally {
setBusy(false)
}
}
return (
<div className="panel" style={{ padding: 18, marginBottom: 22, borderColor: 'var(--accent)' }}>
<div style={{ display: 'flex', gap: 14, alignItems: 'center', flexWrap: 'wrap' }}>
<div style={{ flex: '1 1 320px' }}>
<div className="field-label" style={{ marginBottom: 4 }}>Restart needed</div>
<p className="sans" style={{ margin: 0, fontSize: '0.84rem', color: 'var(--muted)' }}>
{sent
? 'Restarting. This page will reload once the server is back.'
: 'Modules are read from disk when the server starts, so an install, an uninstall or a re-enable only takes effect after a restart.'}
</p>
</div>
<button type="button" className="btn btn-primary btn-sq" disabled={busy || sent} onClick={restart}>
{sent ? 'Restarting…' : 'Restart the server'}
</button>
</div>
</div>
)
}
// ── The source allowlist ───────────────────────────────────────────────────
function SourceHosts({ hosts, onSaved }) {
const [value, setValue] = useState(hosts.join(', '))
const [busy, setBusy] = useState(false)
const [error, setError] = useState('')
const [saved, setSaved] = useState(false)
useEffect(() => { setValue(hosts.join(', ')) }, [hosts])
async function save(e) {
e.preventDefault()
setError('')
setSaved(false)
setBusy(true)
try {
await api.admin.setModuleSources(value)
setSaved(true)
await onSaved()
} catch (err) {
setError(err.message || 'Could not save the allowlist.')
} finally {
setBusy(false)
}
}
const parsed = parseHosts(value)
return (
<div className="panel" style={{ padding: 22, marginTop: 22 }}>
<div className="field-label" style={{ marginBottom: 10 }}>Where modules may be installed from</div>
<form onSubmit={save} style={{ display: 'flex', gap: 12, alignItems: 'flex-end', flexWrap: 'wrap' }}>
<label style={{ flex: '1 1 380px' }}>
<span className="field-label">Allowed hosts</span>
<input
type="text"
value={value}
onChange={(e) => setValue(e.target.value)}
className="input"
placeholder="gitea.example.com, releases.example.org"
/>
</label>
<button type="submit" disabled={busy} className="btn btn-sq">{busy ? 'Saving…' : 'Save'}</button>
</form>
<p className="sans" style={{ margin: '12px 0 0', fontSize: '0.8rem', color: 'var(--muted)' }}>
Installing a module runs its code inside this server, so only hosts listed here may be
installed from — over HTTPS, and re-checked on every redirect. An empty list blocks all
installs.{' '}
{parsed.length > 0 && <>Will be saved as: <code>{parsed.join(', ')}</code>.</>}
</p>
{error && <p className="sans" style={{ margin: '10px 0 0', color: TONE.bad, fontSize: '0.85rem' }}>{error}</p>}
{saved && !error && <p className="sans" style={{ margin: '10px 0 0', color: TONE.ok, fontSize: '0.85rem' }}>Saved.</p>}
</div>
)
}
// ── One module ─────────────────────────────────────────────────────────────
function ModuleRow({ m, onChanged, onError }) {
const [busy, setBusy] = useState('')
const status = statusOf(m)
const actions = actionsFor(m)
const note = declarationNoteFor(m)
async function run(name, fn) {
setBusy(name)
try {
await fn()
await onChanged()
} catch (err) {
onError(err.message || `Could not ${name} ${m.id}.`)
} finally {
setBusy('')
}
}
const disable = () => run('disable', () => api.admin.disableModule(m.id))
const enable = () => run('enable', () => api.admin.enableModule(m.id))
function uninstall() {
// The purge choice is made HERE and only here, because purge.sql lives
// inside the directory the uninstall is about to delete — there is no
// "purge it later" (§2.7.2 decision 5). Two prompts rather than one, so
// "delete the data too" is never something you agree to by reflex.
if (!window.confirm(`Uninstall ${m.name}?\n\nIts files are removed. Its data is kept unless you ask otherwise next.`)) return
let purge = false
if (m.canPurge) {
purge = window.confirm(
`Also permanently delete ${m.name}'s data?\n\n`
+ 'This drops its tables and cannot be undone. This is the only moment it can be offered — '
+ 'the script that does it is part of the files being removed.\n\n'
+ 'OK deletes the data. Cancel keeps it.',
)
}
return run('uninstall', () => api.admin.uninstallModule(m.id, { purge }))
}
function purge() {
if (!window.confirm(`Permanently delete ${m.name}'s data?\n\nThis drops its tables and cannot be undone.`)) return
return run('purge', () => api.admin.purgeModule(m.id))
}
const forget = () => run('forget', () => api.admin.uninstallModule(m.id))
return (
<tr>
<td className="adm-td" style={{ color: 'var(--text)' }}>
<div style={{ fontWeight: 600 }}>{m.name}</div>
<div className="dim" style={{ fontSize: '0.76rem' }}>
{/* A declared module that has never installed has no version to show —
only the one MODULES asks for, which the status column carries. */}
{m.id}{m.version ? ` · v${m.version}` : ''}
</div>
{m.capabilities?.length > 0 && (
<div className="dim" style={{ fontSize: '0.72rem', marginTop: 2 }}>{m.capabilities.join(' · ')}</div>
)}
</td>
<td className="adm-td">
<Pill tone={status.tone}>{status.label}</Pill>
<div className="dim" style={{ fontSize: '0.74rem', marginTop: 4, maxWidth: 380 }}>{status.detail}</div>
{/* The environment's declaration, on its own line: a module can be
running fine while its declared upgrade is failing, and the status
above can only be one of those two things. */}
{note && (
<div
style={{
fontSize: '0.74rem',
marginTop: 4,
maxWidth: 380,
color: note.tone === 'warn' ? TONE.warn : 'var(--muted)',
}}
>
{note.text}
</div>
)}
</td>
<td className="adm-td dim" style={{ fontSize: '0.74rem' }}>
{/* Provenance. Null for a directory placed on the volume by hand, which
stays a supported install — so it is shown as that, not as missing.
A declared module can also reach a boot with no provenance: the
no-op path never fetches, so it has no sha256 to record and no
reason to write a row. Saying "by hand" there would be the one
wrong answer. */}
{m.source ? (
<>
<div style={{ wordBreak: 'break-all', maxWidth: 260 }}>{m.source}</div>
{m.sha256 && <div style={{ marginTop: 2 }}>sha256 {m.sha256.slice(0, 12)}…</div>}
</>
) : (
<span>{m.declared ? 'From the declared module set' : 'Placed on the volume by hand'}</span>
)}
{m.installedAt && <div style={{ marginTop: 2 }}>{dateTime(m.installedAt)}</div>}
</td>
<td className="adm-td" style={{ textAlign: 'right', whiteSpace: 'nowrap' }}>
<div style={{ display: 'inline-flex', gap: 6, flexWrap: 'wrap', justifyContent: 'flex-end' }}>
{actions.disable.shown && (
<button type="button" className="pill" style={{ fontSize: '0.72rem' }} disabled={Boolean(busy)} onClick={disable}>
{busy === 'disable' ? 'Stopping…' : 'Disable'}
</button>
)}
{actions.enable.shown && (
<button type="button" className="pill" style={{ fontSize: '0.72rem' }} disabled={Boolean(busy)} onClick={enable}>
{busy === 'enable' ? 'Enabling…' : 'Enable'}
</button>
)}
{actions.purge.shown && (
<button
type="button"
className="pill"
style={{ fontSize: '0.72rem', ...DANGER, opacity: actions.purge.enabled ? 1 : 0.45 }}
disabled={Boolean(busy) || !actions.purge.enabled}
title={actions.purge.enabled ? undefined : actions.purge.reason}
onClick={purge}
>
{busy === 'purge' ? 'Purging…' : 'Purge data'}
</button>
)}
{actions.uninstall.shown && (
<button type="button" className="pill" style={{ fontSize: '0.72rem', ...DANGER }} disabled={Boolean(busy)} onClick={uninstall}>
{busy === 'uninstall' ? 'Removing…' : 'Uninstall'}
</button>
)}
{actions.forget.shown && (
<button type="button" className="pill" style={{ fontSize: '0.72rem' }} disabled={Boolean(busy)} onClick={forget}>
{busy === 'forget' ? 'Clearing…' : 'Clear the row'}
</button>
)}
</div>
</td>
</tr>
)
}
// ── The screen ─────────────────────────────────────────────────────────────
export default function ModulesAdmin() {
const [data, setData] = useState(null)
const [error, setError] = useState('')
const [actionError, setActionError] = useState('')
const load = useCallback(async () => {
setError('')
try {
setData(await api.admin.listModules())
} catch {
setError('Could not load installed modules.')
}
}, [])
useEffect(() => { load() }, [load])
if (error) return <ErrorState message={error} />
if (!data) return <Loading />
const modules = data.modules || []
const sourceHosts = data.sourceHosts || []
return (
<section>
{needsRestart(modules) && <RestartBanner onDone={() => window.location.reload()} />}
<InstallForm sourceHosts={sourceHosts} onInstalled={load} />
{actionError && (
<p className="sans" style={{ margin: '0 0 14px', color: TONE.bad, fontSize: '0.85rem' }}>{actionError}</p>
)}
<div className="panel-flat">
<table className="adm-table">
<thead>
<tr>
<th className="adm-th">Module</th>
<th className="adm-th">Status</th>
<th className="adm-th">Installed from</th>
<th className="adm-th" />
</tr>
</thead>
<tbody>
{modules.length === 0 && (
<tr>
<td className="adm-td" colSpan={4} style={{ color: 'var(--muted)' }}>
No modules installed. Paste a release install-manifest URL above to add one.
</td>
</tr>
)}
{modules.map((m) => (
<ModuleRow key={m.id} m={m} onChanged={load} onError={setActionError} />
))}
</tbody>
</table>
</div>
<SourceHosts hosts={sourceHosts} onSaved={load} />
</section>
)
}

View File

@@ -140,3 +140,48 @@ test('DELETE self-service session revoke encodes the id and uses the DELETE meth
assert.equal(calls[0].opts.method, 'DELETE')
assert.match(calls[0].url, /\/auth\/me\/sessions\/a%20b%2Fc$/)
})
// ── admin: installed modules (MODULE_SYSTEM.md §2.7.2) ──────────────────
//
// These pin the URLs, because the destructive one differs from the harmless one
// by a query parameter and nothing else.
test('module actions hit the right paths and methods', async () => {
const cases = [
[() => api.admin.listModules(), 'GET', '/api/v1/admin/modules'],
[() => api.admin.installModule('https://x/y.json'), 'POST', '/api/v1/admin/modules'],
[() => api.admin.enableModule('uo'), 'POST', '/api/v1/admin/modules/uo/enable'],
[() => api.admin.disableModule('uo'), 'POST', '/api/v1/admin/modules/uo/disable'],
[() => api.admin.purgeModule('uo'), 'POST', '/api/v1/admin/modules/uo/purge'],
[() => api.admin.setModuleSources('a.com'), 'PUT', '/api/v1/admin/modules/sources'],
[() => api.admin.restartServer(), 'POST', '/api/v1/admin/modules/restart'],
]
for (const [call, method, url] of cases) {
calls = []
willReply({ body: {} })
await call()
assert.equal(calls[0].url, url)
assert.equal(calls[0].opts.method || 'GET', method)
}
})
test('uninstall only asks for a purge when it is told to', async () => {
// The difference between "remove the module" and "remove the module and drop
// every table it owns" is this query parameter, so a default that leaned the
// wrong way would be irreversible.
willReply({ body: {} })
await api.admin.uninstallModule('uo')
assert.equal(calls[0].url, '/api/v1/admin/modules/uo')
assert.equal(calls[0].opts.method, 'DELETE')
calls = []
willReply({ body: {} })
await api.admin.uninstallModule('uo', { purge: true })
assert.equal(calls[0].url, '/api/v1/admin/modules/uo?purge=true')
})
test('a module id is URL-encoded on the way into the path', async () => {
willReply({ body: {} })
await api.admin.disableModule('a b/c')
assert.equal(calls[0].url, '/api/v1/admin/modules/a%20b%2Fc/disable')
})

View File

@@ -0,0 +1,281 @@
import { test } from 'node:test'
import assert from 'node:assert/strict'
import { statusOf, actionsFor, declarationNoteFor, needsRestart, parseHosts } from '../src/lib/moduleAdmin.js'
// lib/moduleAdmin.js — what the Modules screen says about a module and what it
// lets you do to it. Phase 4, slice 2 of MODULE_SYSTEM.md §2.7.2.
//
// This is the part of the screen worth testing, and it is plain JS so this
// runner can reach it (there is no DOM here). What it encodes is §2.4's rule
// that the row, the loader and the volume are three sources of truth which are
// ALLOWED to disagree — so most of these cases are combinations that a screen
// picking one source would render as a lie.
/** A module as GET /admin/modules returns it, with the running case as default. */
const mod = (over = {}) => ({
id: 'uo',
name: 'Ultima Online',
version: '1.0.0',
state: 'started',
failureStage: null,
failureReason: null,
source: 'https://gitea.example.com/x/uo.json',
sha256: 'a'.repeat(64),
installedAt: null,
startedAt: null,
liveState: 'started',
liveVersion: '1.0.0',
capabilities: [],
onVolume: true,
canPurge: true,
declared: false,
declaredVersion: null,
declaredError: null,
...over,
})
// ── statusOf ───────────────────────────────────────────────────────────────
test('a mounted, started module is Running and needs nothing', () => {
const s = statusOf(mod())
assert.equal(s.label, 'Running')
assert.equal(s.tone, 'ok')
assert.equal(s.pending, false)
})
test('enabled in the row but disabled in the loader is "Restart to start"', () => {
// THE case decision 3 creates on purpose: disable ran the module's onShutdown,
// then the operator enabled it again. The row says enabled; nothing can start
// it before a restart. Showing either "Running" or "Disabled" would be false.
const s = statusOf(mod({ state: 'enabled', liveState: 'disabled' }))
assert.equal(s.label, 'Restart to start')
assert.equal(s.tone, 'warn')
assert.equal(s.pending, true)
assert.match(s.detail, /cannot be restarted in place/)
})
test('freshly installed and never booted into is also "Restart to start"', () => {
const s = statusOf(mod({ state: 'installed', liveState: null }))
assert.equal(s.label, 'Restart to start')
assert.equal(s.pending, true)
assert.match(s.detail, /mounts when the server next starts/)
})
test('a disabled module is Disabled, and that is not pending anything', () => {
// Disable takes effect immediately — it is the one action that does — so there
// is nothing for a restart banner to be about.
const s = statusOf(mod({ state: 'disabled', liveState: 'disabled' }))
assert.equal(s.label, 'Disabled')
assert.equal(s.pending, false)
})
test('a fresh install over a failed row is pending, not failed', () => {
// THE defect the §7.7 browser smoke found, and one no test here had modelled.
// Installing over a row the previous boot left `startup_failed` rendered
// "Failed at the require stage: module directory not present on the volume" a
// second after the files had been written — and suppressed the restart banner
// the install had just told the operator to use.
//
// `liveState === null` with the module on the volume means the loader's scan
// never saw it, so it arrived after boot and everything the row says predates
// it.
const s = statusOf(mod({
state: 'startup_failed',
liveState: null,
failureStage: 'require',
failureReason: 'module directory not present on the volume',
}))
assert.equal(s.label, 'Restart to start')
assert.equal(s.pending, true)
assert.doesNotMatch(s.detail, /not present on the volume/, 'the stale reason must not survive the install')
})
test('the restart banner appears for that install', () => {
// The second half of the same defect: the banner is driven by `pending`, so a
// row wrongly classified as failed silently removed the only way to act on it.
assert.equal(needsRestart([mod({ state: 'startup_failed', liveState: null })]), true)
})
test('an upgrade that has not been restarted into says so', () => {
// Same class as the stale-failure defect: the row is a promise about the next
// boot, not a description of this one. Reporting "Running v2.0.0" while the
// process is serving v1.0.0 would hide the only action that fixes it.
const s = statusOf(mod({ version: '2.0.0', liveVersion: '1.0.0' }))
assert.equal(s.label, 'Restart to finish upgrading')
assert.equal(s.pending, true)
assert.match(s.detail, /v2\.0\.0 is installed; v1\.0\.0 is still running/)
})
test('reinstalling the SAME version is not an upgrade in progress', () => {
assert.equal(statusOf(mod({ version: '1.0.0', liveVersion: '1.0.0' })).label, 'Running')
})
test('a failed module reports the stage and the reason it recorded', () => {
const s = statusOf(mod({
state: 'startup_failed',
liveState: 'startup_failed',
failureStage: 'schema',
failureReason: "Unknown column 'x' in 'field list'",
}))
assert.equal(s.label, 'Failed to start')
assert.equal(s.tone, 'bad')
assert.match(s.detail, /schema stage/)
assert.match(s.detail, /Unknown column/)
})
test('a failure with no recorded reason says so rather than showing a blank', () => {
const s = statusOf(mod({ state: 'startup_failed', liveState: 'startup_failed' }))
assert.match(s.detail, /recorded no reason/)
})
test('a row whose directory is gone by hand is bad, not merely disabled', () => {
// The boot reconcile marks this `startup_failed` because a row claiming to be
// enabled for a module that is not on the volume is simply untrue.
const s = statusOf(mod({ state: 'startup_failed', liveState: null, onVolume: false, failureStage: 'require', failureReason: 'module directory not present on the volume' }))
assert.equal(s.label, 'Missing from the volume')
assert.equal(s.tone, 'bad')
})
test('an uninstalled module reads as uninstalled, and says the data was kept', () => {
// Uninstall leaves the row `disabled` and the data alone — which is the whole
// point of keeping the row, so the screen has to say it.
const s = statusOf(mod({ state: 'disabled', liveState: 'disabled', onVolume: false }))
assert.equal(s.label, 'Uninstalled')
assert.equal(s.tone, 'idle')
assert.match(s.detail, /data was kept/i)
})
test('missing-from-the-volume beats every other status', () => {
// Ordering: a module with no files is described that way whatever its row
// still claims, because there is nothing there to be running.
for (const state of ['started', 'enabled', 'installed', 'startup_failed']) {
assert.match(statusOf(mod({ state, onVolume: false })).label, /Missing from the volume/)
}
})
// ── actionsFor ─────────────────────────────────────────────────────────────
test('a running module offers disable, uninstall and a blocked purge', () => {
const a = actionsFor(mod())
assert.equal(a.disable.shown, true)
assert.equal(a.enable.shown, false)
assert.equal(a.uninstall.shown, true)
assert.equal(a.purge.shown, true)
// Shown but not clickable: the server refuses a standalone purge on anything
// that is not disabled, so offering the click would only produce a 409.
assert.equal(a.purge.enabled, false)
assert.match(a.purge.reason, /Disable it first/)
})
test('a disabled module offers enable, and purge is now live', () => {
const a = actionsFor(mod({ state: 'disabled', liveState: 'disabled' }))
assert.equal(a.enable.shown, true)
assert.equal(a.disable.shown, false)
assert.equal(a.purge.enabled, true)
})
test('a module with no purge.sql never offers purge, and says why', () => {
const a = actionsFor(mod({ state: 'disabled', liveState: 'disabled', canPurge: false }))
assert.equal(a.purge.shown, false)
assert.match(a.purge.reason, /ships no purge.sql/)
})
test('a module with no files offers only clearing the row', () => {
const a = actionsFor(mod({ state: 'disabled', liveState: null, onVolume: false }))
assert.equal(a.uninstall.shown, false)
assert.equal(a.disable.shown, false)
assert.equal(a.enable.shown, false)
assert.equal(a.purge.shown, false, 'there is no purge.sql left to run')
assert.equal(a.forget.shown, true)
})
test('a directory with no row yet is actionable, and offers nothing to forget', () => {
// A hand-placed install before its first boot: it has no row, so `state` is
// null. Its routes are already being served, so it must be disableable.
const a = actionsFor(mod({ state: null, liveState: 'started' }))
assert.equal(a.disable.shown, true)
assert.equal(a.uninstall.shown, true)
assert.equal(a.forget.shown, false)
})
// ── needsRestart ───────────────────────────────────────────────────────────
test('the restart banner is driven by the list, not by any one module', () => {
// A restart is a property of the SERVER. One pending module is enough, and
// three do not mean three restarts.
assert.equal(needsRestart([mod(), mod({ id: 'b' })]), false)
assert.equal(needsRestart([mod(), mod({ id: 'b', state: 'installed', liveState: null })]), true)
assert.equal(needsRestart([]), false)
})
test('a disabled module does not ask for a restart', () => {
// Disable is immediate; a banner here would be asking for a restart that
// would change nothing.
assert.equal(needsRestart([mod({ state: 'disabled', liveState: 'disabled' })]), false)
})
test('a failed module does not ask for a restart either', () => {
// It is retried on every boot anyway, and the operator has to fix the cause
// first — a banner would suggest restarting is the remedy.
assert.equal(needsRestart([mod({ state: 'startup_failed', liveState: 'startup_failed' })]), false)
})
// ── parseHosts ─────────────────────────────────────────────────────────────
test('parseHosts previews exactly what the server will store', () => {
assert.deepEqual(parseHosts('A.com, b.com\n c.com'), ['a.com', 'b.com', 'c.com'])
assert.deepEqual(parseHosts(' '), [])
assert.deepEqual(parseHosts(undefined), [])
})
// ── the declaration (slice 3) ──────────────────────────────────────────────
test('a declared module that has never installed says so, with the reason', () => {
// No row, no directory, nothing mounted — invisible to the other three
// sources, so without this branch the screen would describe a module it has
// never had as a row gone stale.
const s = statusOf(mod({
state: null,
liveState: null,
liveVersion: null,
version: null,
onVolume: false,
declared: true,
declaredVersion: '0.3.0',
declaredError: 'could not reach releases.example.com',
}))
assert.equal(s.label, 'Declared, not installed')
assert.equal(s.tone, 'bad')
assert.equal(s.pending, false, 'a restart will not fix an unreachable host')
assert.match(s.detail, /could not reach releases.example.com/)
})
test('a declared module waiting for its first resolution is not reported as failed', () => {
const s = statusOf(mod({ state: null, liveState: null, version: null, onVolume: false, declared: true, declaredVersion: '0.3.0' }))
assert.match(s.detail, /installed when the server next starts/)
})
test('a running module whose declared upgrade is failing is still Running', () => {
// Both facts are true at once. The status is one label, so the declaration
// gets its own line rather than overwriting it.
const m = mod({ declared: true, declaredVersion: '2.0.0', declaredError: 'sha256 did not match' })
assert.equal(statusOf(m).label, 'Running')
const note = declarationNoteFor(m)
assert.equal(note.tone, 'warn')
assert.match(note.text, /sha256 did not match/)
})
test('uninstalling a declared module is told that its files come back', () => {
// The sentence that saves an afternoon: MODULES owns what is on the volume,
// the row owns whether it runs.
const note = declarationNoteFor(mod({ state: 'disabled', liveState: null, onVolume: false, declared: true, declaredVersion: '1.0.0' }))
assert.match(note.text, /come back when the server next starts/)
assert.match(note.text, /switched off/)
})
test('an ordinary declared module gets a quiet note, and an undeclared one none', () => {
assert.equal(declarationNoteFor(mod()), null)
const note = declarationNoteFor(mod({ declared: true, declaredVersion: '1.0.0' }))
assert.equal(note.tone, 'idle')
assert.match(note.text, /MODULES/)
})

View File

@@ -39,6 +39,24 @@ services:
# defaults to (<repo>/modules, and the repo is /app in the image), set
# explicitly because the bind mount below is what makes it meaningful.
MODULES_DIR: /app/modules
# WHICH modules this deployment runs (MODULE_SYSTEM.md §2.7.2 decision 4).
# One entry per module, `<id>@<version>=<install manifest URL>`, whitespace-
# or comma-separated. The container resolves this set for itself at every
# start: a module already unpacked at the declared version is left alone
# without a single network call — so a restart with the internet down comes
# up unchanged — and only a missing or different version is fetched,
# verified against the sha256 its release manifest declares, and unpacked.
# A failure is logged and surfaced in Admin → Modules; it never stops the
# site from starting.
#
# Uncomment to declare a set here, in the file this host version-controls,
# or leave it out and set MODULES in .env (env_file above) — or leave it
# unset entirely and install from the admin panel. What it declares is what
# is ON the volume, never whether a module runs: a module disabled from the
# admin panel gets its files back and stays disabled.
#
# MODULES: >-
# uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json
depends_on:
db:
condition: service_healthy
@@ -56,10 +74,12 @@ services:
# pull-only deployment without building anything. A bind mount rather than
# a named volume because placing a module directory by hand is a supported
# install — `tar -xf uo-1.0.0.tgz -C ./modules` then restart — and that has
# to be doable from the host, not through `docker cp`.
# to be doable from the host, not through `docker cp`. It is no longer the
# usual way in: declare MODULES above, or install from Admin → Modules.
#
# Read-WRITE: the admin panel's install/uninstall unpacks and removes
# directories here from inside the container.
# Read-WRITE: MODULES resolution at start, and the admin panel's
# install/uninstall, both unpack and remove directories here from inside
# the container.
#
# `modules/` is tracked (it ships a README) so the directory exists in the
# checkout with the operator's own ownership. Do not delete it — Docker

View File

@@ -129,3 +129,38 @@ ANNOUNCE_POLL_MS=15000
# NTFY_PUBLIC_URL=https://ntfy.example.com
# NTFY_ALLOWED_ORIGINS=https://ntfy.example.com
# NTFY_PUBLISH_TOKEN=
# Modules (MODULE_SYSTEM.md §2.5) — where installable modules live, and where
# they may be installed from.
# MODULES_DIR Directory the loader scans at require time. Defaults to
# <repo>/modules; docker-compose.yml sets it to /app/modules,
# which is the bind mount that makes it meaningful.
# MODULE_SOURCE_HOSTS BOOTSTRAP ONLY. Comma-separated hostnames the admin panel
# may install a module from, seeded into the `module_source_hosts`
# setting the first time the site boots without one. From then
# on the SETTING is authoritative and is edited in
# Admin → Modules — changing this variable on an existing
# deployment does nothing, deliberately, so a redeploy cannot
# silently undo an operator's choice. Installs are https-only
# and an empty list forbids all of them.
# MODULES The module set this deployment RUNS, resolved at every
# start (§2.7.2 decision 4). One entry per module, separated
# by whitespace or commas:
#
# <id>@<version>=<install manifest URL>
#
# A module already unpacked at the declared version is left
# alone WITHOUT touching the network, so a restart with no
# route to the internet comes up unchanged; only a missing or
# different version is fetched, through the same verify-and-
# unpack path (and the same host allowlist) the admin panel
# uses. A version that cannot be fetched is logged and shown
# in Admin → Modules — it never stops the site from starting.
#
# This variable owns what is ON the volume, not what runs: a
# module disabled from the admin panel stays disabled even
# though its files are put back. Leave it unset to manage
# modules entirely from the admin panel.
# MODULES_DIR=/app/modules
# MODULE_SOURCE_HOSTS=gitea.whitlocktech.com
# MODULES=uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json

View File

@@ -23,6 +23,17 @@ const DEFAULT_SETTINGS = {
site_title: brand.name,
// Android App Links opt-in — off until an admin enables it (docs/android/APP_LINKS.md).
mobile_app_links_enabled: 'false',
// Hosts a module may be installed from (MODULE_SYSTEM.md §2.7.2 decision 6).
//
// The environment BOOTSTRAPS this and does not own it: seedDefault is an
// INSERT IGNORE, so the variable supplies a sane default on a fresh install
// and never reaches back in to overwrite what an admin later chose in
// Admin → Modules. Changing MODULE_SOURCE_HOSTS on an existing deployment is
// therefore a no-op, which is the intended behaviour and not an oversight.
//
// An empty stored value forbids every install rather than allowing every host
// — the safe direction for a setting someone might blank by accident.
module_source_hosts: process.env.MODULE_SOURCE_HOSTS || 'gitea.whitlocktech.com',
}
// Starter wiki sections (editable later via the admin panel).

View File

@@ -27,6 +27,7 @@
"sanitize-html": "^2.17.5",
"speakeasy": "^2.0.0",
"swagger-ui-express": "^5.0.1",
"tar": "^7.5.22",
"ws": "^8.21.0"
},
"devDependencies": {
@@ -34,6 +35,18 @@
"swagger-autogen": "^2.23.7"
}
},
"node_modules/@isaacs/fs-minipass": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/@isaacs/fs-minipass/-/fs-minipass-4.0.1.tgz",
"integrity": "sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w==",
"license": "ISC",
"dependencies": {
"minipass": "^7.0.4"
},
"engines": {
"node": ">=18.0.0"
}
},
"node_modules/@scarf/scarf": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/@scarf/scarf/-/scarf-1.4.0.tgz",
@@ -330,6 +343,15 @@
"fsevents": "~2.3.2"
}
},
"node_modules/chownr": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/chownr/-/chownr-3.0.0.tgz",
"integrity": "sha512-+IxzY9BZOQd/XuYPRmrvEVjF/nqj5kgT4kEq7VofrDoM1MxoRjEWkrCC3EtLi59TVawxTAn+orJwFQcrqEN1+g==",
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=18"
}
},
"node_modules/cliui": {
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/cliui/-/cliui-6.0.0.tgz",
@@ -1488,6 +1510,27 @@
"url": "https://github.com/sponsors/isaacs"
}
},
"node_modules/minipass": {
"version": "7.1.3",
"resolved": "https://registry.npmjs.org/minipass/-/minipass-7.1.3.tgz",
"integrity": "sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==",
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=16 || 14 >=14.17"
}
},
"node_modules/minizlib": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/minizlib/-/minizlib-3.1.0.tgz",
"integrity": "sha512-KZxYo1BUkWD2TVFLr0MQoM8vUUigWD3LlD83a/75BqC+4qE0Hb1Vo5v1FgcfaNXvfXzr+5EhQ6ing/CaBijTlw==",
"license": "MIT",
"dependencies": {
"minipass": "^7.1.2"
},
"engines": {
"node": ">= 18"
}
},
"node_modules/morgan": {
"version": "1.11.0",
"resolved": "https://registry.npmjs.org/morgan/-/morgan-1.11.0.tgz",
@@ -2254,6 +2297,22 @@
"express": ">=4.0.0 || >=5.0.0-beta"
}
},
"node_modules/tar": {
"version": "7.5.22",
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.22.tgz",
"integrity": "sha512-MFO/QzvtAOmJbkhOaCTvbGcFN9L9b+JunIsDwaKljSOdcLMea3NJ1k9Usz/rjdfSXTq4dfzfeS7W4p4YOAAHeA==",
"license": "BlueOak-1.0.0",
"dependencies": {
"@isaacs/fs-minipass": "^4.0.0",
"chownr": "^3.0.0",
"minipass": "^7.1.2",
"minizlib": "^3.1.0",
"yallist": "^5.0.0"
},
"engines": {
"node": ">=18"
}
},
"node_modules/to-regex-range": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/to-regex-range/-/to-regex-range-5.0.1.tgz",
@@ -2414,6 +2473,15 @@
"integrity": "sha512-JKhqTOwSrqNA1NY5lSztJ1GrBiUodLMmIZuLiDaMRJ+itFd+ABVE8XBjOvIWL+rSqNDC74LCSFmlb/U4UZ4hJQ==",
"license": "ISC"
},
"node_modules/yallist": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/yallist/-/yallist-5.0.0.tgz",
"integrity": "sha512-YgvUTfwqyc7UXVMrB+SImsVYSmTS8X/tSrtdNZMImM+n7+QTriRXyXim0mBrTXNeqzVF0KWGgHPeiyViFFrNDw==",
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=18"
}
},
"node_modules/yargs": {
"version": "15.4.1",
"resolved": "https://registry.npmjs.org/yargs/-/yargs-15.4.1.tgz",

View File

@@ -38,6 +38,7 @@
"sanitize-html": "^2.17.5",
"speakeasy": "^2.0.0",
"swagger-ui-express": "^5.0.1",
"tar": "^7.5.22",
"ws": "^8.21.0"
},
"devDependencies": {

View File

@@ -427,6 +427,90 @@
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/modules",
"handlers": 2,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "POST",
"path": "/api/v1/admin/modules",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "DELETE",
"path": "/api/v1/admin/modules/:id",
"handlers": 5,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "POST",
"path": "/api/v1/admin/modules/:id/disable",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "POST",
"path": "/api/v1/admin/modules/:id/enable",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "POST",
"path": "/api/v1/admin/modules/:id/purge",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "POST",
"path": "/api/v1/admin/modules/restart",
"handlers": 2,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "PUT",
"path": "/api/v1/admin/modules/sources",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/pages",

View File

@@ -177,6 +177,38 @@
"method": "POST",
"path": "/api/v1/admin/moderation/user/:discordId/notes"
},
{
"method": "GET",
"path": "/api/v1/admin/modules"
},
{
"method": "POST",
"path": "/api/v1/admin/modules"
},
{
"method": "DELETE",
"path": "/api/v1/admin/modules/:id"
},
{
"method": "POST",
"path": "/api/v1/admin/modules/:id/disable"
},
{
"method": "POST",
"path": "/api/v1/admin/modules/:id/enable"
},
{
"method": "POST",
"path": "/api/v1/admin/modules/:id/purge"
},
{
"method": "POST",
"path": "/api/v1/admin/modules/restart"
},
{
"method": "PUT",
"path": "/api/v1/admin/modules/sources"
},
{
"method": "GET",
"path": "/api/v1/admin/pages"

View File

@@ -17,6 +17,23 @@ const getOne = (id) => query(`SELECT ${COLS} FROM installed_modules WHERE id = ?
// module must not silently disable it, and re-installing a disabled one must not
// silently switch it back on. A brand-new row lands in `installed`, the transient
// state the next restart resolves.
// Provenance is COALESCEd, and that is the whole difference between this
// working and not.
//
// `lifecycle.boot()` re-records every module it scanned with no source and no
// sha256 — a directory placed on the volume by hand genuinely has neither, and
// the boot has no way to know where one came from. With a plain
// `source = VALUES(source)` that refresh overwrote both columns with NULL on
// EVERY boot, so an admin-panel install's provenance survived exactly until the
// restart that install asked for. Nothing could catch it before Phase 4: until
// then no caller ever passed a non-null value, and lifecycle.js's comment
// asserting that "recordInstalled leaves what it is not given" was a description
// of an intention rather than of this statement.
//
// COALESCE makes that comment true: a value overwrites, a NULL leaves what is
// there. The cost is that re-placing a DIFFERENT bundle by hand over a row that
// was once installed from a URL keeps the old provenance — which is wrong but
// stale, and strictly better than the alternative, which was wrong and blank.
const upsert = ({ id, name, version, source, sha256 }) =>
query(
`INSERT INTO installed_modules (id, name, version, source, sha256, state)
@@ -24,8 +41,8 @@ const upsert = ({ id, name, version, source, sha256 }) =>
ON DUPLICATE KEY UPDATE
name = VALUES(name),
version = VALUES(version),
source = VALUES(source),
sha256 = VALUES(sha256)`,
source = COALESCE(VALUES(source), source),
sha256 = COALESCE(VALUES(sha256), sha256)`,
[id, name, version, source ?? null, sha256 ?? null],
)

View File

@@ -0,0 +1,221 @@
// ── The hardened bundle extractor ──────────────────────────────────────────
//
// Phase 4, slice 1 of docs/website/MODULE_SYSTEM.md §2.7.2. This is the part of
// the install path that handles input an attacker chose, and it is separated
// from install.js so that it can be tested against crafted archives without a
// network, a database or a filesystem layout.
//
// **The download is not the dangerous part.** An allowlisted host, a declared
// sha256 and a size cap between them settle where the bytes came from and that
// they are the bytes that were published. Unpacking is different: the ARCHIVE
// chooses the filenames, and core writes into a directory bind-mounted from the
// host (MODULE_SYSTEM.md §2.5), so an escape is not confined to the container.
//
// The rule this file follows is **reject, never sanitise.** node-tar will
// happily strip a leading `/` and drop a `..` for you, which turns a hostile
// archive into a slightly different archive that then gets installed. An archive
// that needs correcting is an archive that should not be trusted, so anything on
// the list below fails the whole install and nothing is written.
//
// - an absolute path, POSIX (`/etc/…`) or Windows (`C:\…`, `\\server\…`)
// - any `..` segment, anywhere
// - anything that is not a regular file or a directory — so no symlinks, no
// hardlinks, no devices, no FIFOs. A module bundle has no legitimate use for
// any of them, and every one is a documented escape primitive
// - more than one top-level entry
// - more entries, or more unpacked bytes, than the caps below
//
// That third rule deserves its own note, because it is why the `tar` dependency
// is pinned forward rather than merely present: **the majority of node-tar's
// published advisories are hardlink or symlink path traversal**, several of them
// through interpretation differences in PAX and GNU long-name headers rather
// than through anything the calling code did wrong. Refusing those entry types
// outright means this extractor is not relying on the library to get their
// containment right. The remaining advisories are parser denial-of-service, and
// those are what the caps and the two-pass shape address.
//
// Two passes, deliberately: `inspect()` reads the archive and decides, and only
// an archive that survived that is handed to `extract()`. A `filter` callback
// during extraction cannot reject — by the time it is asked about entry 400, the
// first 399 are already on disk.
const fs = require('fs')
const fsp = require('fs/promises')
const path = require('path')
const tar = require('tar')
// Caps. Generous against a real bundle (module-uo's is a few megabytes, a few
// hundred files) and small enough that a decompression bomb is refused rather
// than paged in. Both are checked DURING the inspect pass, not after it, so a
// hostile archive stops being read at the limit instead of at its end.
const MAX_BYTES = 128 * 1024 * 1024
const MAX_ENTRIES = 20000
// tar entry types worth naming. node-tar reports these as strings; anything not
// in this set is refused, including the ones no one has thought of, which is the
// point of an allowlist here rather than a list of the types known to be bad.
const ALLOWED_TYPES = new Set(['File', 'Directory'])
class ArchiveError extends Error {
constructor(message) {
super(message)
this.name = 'ArchiveError'
}
}
/**
* Is this entry path safe to unpack anywhere?
*
* Returns a reason string, or null when the path is fine. Written as a reason
* rather than a boolean because the operator pasting a URL needs to be told what
* was wrong with what they were served, and "the archive is invalid" is not that.
*/
function pathProblem(entryPath) {
const p = String(entryPath)
// NUL is not a path character. node-tar has had uncaught-exception advisories
// for NUL bytes inside PAX records, so this is checked here rather than left
// to the parser.
if (p.includes('\0')) return 'an entry path contains a NUL byte'
// A backslash is a path separator on the platform this may be unpacked on, so
// an entry that contains one is not the single path component it looks like.
if (p.includes('\\')) return `an entry path contains a backslash: "${p}"`
if (p.startsWith('/')) return `an entry path is absolute: "${p}"`
// Drive-relative and UNC. Both are the subject of their own node-tar
// advisories, and neither is refused by a leading-slash check.
if (/^[a-zA-Z]:/.test(p)) return `an entry path is drive-absolute: "${p}"`
const segments = p.split('/')
if (segments.includes('..')) return `an entry path escapes upward: "${p}"`
return null
}
/**
* Read an archive without unpacking it, and decide whether it may be unpacked.
*
* @param {string} file absolute path to the .tar.gz
* @returns {Promise<{root: string, entries: number, bytes: number}>} the single
* top-level directory name, and what is inside it.
* @throws {ArchiveError} on anything in this file's header list.
*/
async function inspect(file) {
const roots = new Set()
let entries = 0
let bytes = 0
// The first problem found, kept rather than thrown from inside the callback:
// throwing out of `onentry` escapes through the parser's stream machinery and
// arrives as an unhelpful wrapped error, when it arrives at all.
let problem = null
const note = (message) => {
if (!problem) problem = message
}
await tar.t({
file,
onentry(entry) {
if (problem) return
entries += 1
if (entries > MAX_ENTRIES) {
note(`the archive has more than ${MAX_ENTRIES} entries`)
return
}
if (!ALLOWED_TYPES.has(entry.type)) {
// The message names the type because "symbolic link" is a much more
// useful thing for an operator to read than "invalid entry".
note(`the archive contains a ${entry.type} ("${entry.path}") — a module bundle may only contain files and directories`)
return
}
const bad = pathProblem(entry.path)
if (bad) {
note(bad)
return
}
// A negative or absurd size is a parser-confusion primitive in its own
// right; clamping at zero keeps the running total honest.
bytes += Math.max(0, Number(entry.size) || 0)
if (bytes > MAX_BYTES) {
note(`the archive unpacks to more than ${Math.round(MAX_BYTES / 1024 / 1024)} MB`)
return
}
const [root] = String(entry.path).split('/')
if (root) roots.add(root)
},
})
if (problem) throw new ArchiveError(problem)
if (entries === 0) throw new ArchiveError('the archive is empty')
if (roots.size !== 1) {
// A bundle is one directory. More than one top-level entry means `strip: 1`
// below would silently merge or discard things, and a bundle that needs
// interpreting is not a bundle.
throw new ArchiveError(
`the archive must contain exactly one top-level directory, found ${roots.size}` +
(roots.size > 1 ? ` (${[...roots].slice(0, 4).join(', ')})` : ''),
)
}
return { root: [...roots][0], entries, bytes }
}
/**
* Unpack an inspected archive into `dest`, stripping its single top level.
*
* The top-level directory is stripped rather than kept, because its name belongs
* to whoever published the bundle — module-uo's release workflow packs
* `module-uo-<version>/`, not `uo/` — while the directory it lands in is core's
* decision and has to be the module id the loader scans for. What is inside the
* bundle is the module; what the wrapper is called is packaging.
*
* `dest` must not exist. Callers unpack to a temporary directory and move it
* into place, so a failure halfway through leaves nothing for the next boot's
* scan to find.
*/
async function extract(file, dest) {
if (fs.existsSync(dest)) throw new ArchiveError(`extraction target already exists: ${dest}`)
await fsp.mkdir(dest, { recursive: true })
await tar.x({
file,
cwd: dest,
strip: 1,
// Belt and braces on top of inspect(): the library's own refusal to write
// outside cwd, and its refusal to overwrite through a link. Neither is
// load-bearing here — inspect() has already rejected every entry that could
// exercise them — but a second lock on a door that is already locked costs
// nothing, and this is the door.
preservePaths: false,
// Reject rather than warn on anything the parser itself objects to.
strict: true,
})
return dest
}
/**
* Inspect, then extract. The only entry point install.js uses.
*/
async function unpack(file, dest) {
const stats = await inspect(file)
await extract(file, dest)
return stats
}
module.exports = {
ArchiveError,
inspect,
extract,
unpack,
pathProblem,
MAX_BYTES,
MAX_ENTRIES,
ALLOWED_TYPES,
}

View File

@@ -0,0 +1,220 @@
// ── The declared module set ────────────────────────────────────────────────
//
// Phase 4, slice 3 of docs/website/MODULE_SYSTEM.md §2.7.2 — decision 4. A
// compose-managed host is not driven by clicking: it declares which modules it
// runs, in the file it already edits and version-controls, and the container
// arrives at that set by itself.
//
// MODULES: uo@0.3.0=https://<host>/…/module-uo-0.3.0.json
//
// One entry per module, `<id>@<version>=<install manifest URL>`, separated by
// whitespace or commas. The id and the version are written out rather than left
// to be discovered inside the manifest for one reason: **the no-op case must not
// need the network.** A module already unpacked at the declared version is
// answered by reading its own `module.json` off the volume, so a restart with
// the network down brings the site up exactly as it was. Only a module that is
// missing, or unpacked at some other version, reaches out — and it reaches out
// through modules/install.js, the same fetch-verify-unpack path the admin panel
// uses, under the same host allowlist.
//
// Three things this file deliberately does not do:
//
// - **It does not decide whether a module RUNS.** Resolution owns what is on
// the volume; `installed_modules` owns whether a mounted module answers. An
// admin who uninstalls a declared module gets its directory back at the next
// boot with the row still `disabled`, so it stays off until they enable it.
// The two never fight because they are not answering the same question.
// - **It does not fail a boot.** A module publisher's host being unreachable
// must not take a shard's website down with it; core is built to serve with
// a module absent (§1.6). Every failure is logged loudly and kept for the
// admin screen, and the site comes up.
// - **It does not mount anything.** §1.12 makes the volume the mounting source
// of truth, read once at require time — which is why this runs before
// `require('./app')` in server.js and not from inside it.
//
// It runs on every boot, not only in Docker: a bare `npm start` with MODULES set
// resolves the same way. The Docker path is the reason it exists, not a special
// case in it.
const fs = require('fs')
const path = require('path')
const install = require('./install')
const log = require('../utils/logger')('modules')
// The variable an operator sets. Named next to MODULES_DIR, which is the other
// half of the same story: one says where modules live, the other says which.
const VAR = 'MODULES'
// Same id rule the loader enforces when scanning and install.js enforces when
// placing, restated here so a declaration cannot name something neither would
// accept.
const ID = /^[a-z][a-z0-9-]{1,31}$/
// The outcome of the last resolution, in memory, for the admin screen. Not a
// database row: a declaration is a fact about this process's environment, and
// writing it down would put it in front of the boot reconcile, which resets
// every non-disabled row (§2.4). The screen merges it as a fourth source
// alongside the row, the loader and the volume.
let results = []
/**
* Parse the declaration into entries.
*
* Malformed entries are collected rather than thrown: one operator typo should
* cost that module, not every module on the host. A duplicate id keeps the
* first — there is no sensible way to run two versions of one module, and
* silently preferring the last would make the outcome depend on the order of a
* list nobody reads as ordered.
*
* @param {string} value the raw variable
* @returns {{entries: Array<{id,version,url}>, errors: string[]}}
*/
function parse(value) {
const entries = []
const errors = []
const seen = new Set()
for (const token of String(value || '').split(/[,\s]+/).filter(Boolean)) {
const at = token.indexOf('@')
const eq = token.indexOf('=')
if (at < 1 || eq < at + 2) {
errors.push(`"${token}" is not <id>@<version>=<url>`)
continue
}
const id = token.slice(0, at)
const version = token.slice(at + 1, eq)
const url = token.slice(eq + 1)
if (!ID.test(id)) {
errors.push(`"${token}" names an invalid module id "${id}"`)
continue
}
if (!url) {
errors.push(`"${token}" has no install manifest URL`)
continue
}
if (seen.has(id)) {
errors.push(`"${id}" is declared more than once — keeping the first`)
continue
}
seen.add(id)
entries.push({ id, version, url })
}
return { entries, errors }
}
/**
* The version currently unpacked on the volume, or null.
*
* Read straight out of the module's own `module.json`, which is the same file
* the loader trusts for the same fact — and never from `installed_modules`,
* because the row records what was installed and this has to answer what is
* actually there. An unreadable manifest counts as absent: whatever is in that
* directory, it is not a module at the declared version.
*/
function installedVersion(id) {
try {
const manifest = JSON.parse(
fs.readFileSync(path.join(install.moduleDir(id), 'module.json'), 'utf8'),
)
return manifest && manifest.version ? String(manifest.version) : null
} catch {
return null
}
}
/**
* Bring the volume in line with the declaration.
*
* Never throws and never rejects. Returns one outcome per declared entry, and
* remembers them for `state()`.
*
* @param {object} args
* @param {string} [args.value] the raw variable (defaults to the environment)
* @param {string[]} args.hosts the install allowlist, already parsed
* @param {object} args.model modules.model, for recording provenance
* @param {object} [args.installImpl] injection seam, as everywhere else here
* @returns {Promise<Array<{id,version,url,action,message}>>}
*/
async function resolve({ value = process.env[VAR], hosts = [], model, installImpl = install } = {}) {
const { entries, errors } = parse(value)
results = []
for (const message of errors) log.error(`${VAR}: ${message}`)
if (!entries.length) return results
log.info(`${VAR} declares ${entries.length} module(s)`, {
modules: entries.map((e) => `${e.id}@${e.version}`).join(' '),
})
for (const entry of entries) {
const present = installedVersion(entry.id)
if (present === entry.version) {
// The offline path, and the common one: nothing is fetched, nothing is
// written, and a host with no route to the internet boots unchanged.
log.info(`module "${entry.id}" is already at the declared version ${entry.version}`)
results.push({ ...entry, action: 'noop', message: null })
continue
}
try {
// `expect` is the declaration itself, handed down so install.js can refuse
// a URL that resolves to another module or another version while it is
// still only a manifest — a check made after the unpack would be made with
// the undeclared module already on the volume.
const result = await installImpl.install({
url: entry.url,
hosts,
expect: { id: entry.id, version: entry.version },
})
// Provenance, written exactly as the admin route writes it — the whole
// point of resolving in-process rather than from a script that cannot
// reach the database. A module installed by the compose file and one
// installed by an admin are then indistinguishable on the screen, which
// is what makes this one feature and not two.
if (model) {
await model.recordInstalled({
id: result.id,
name: result.name,
version: result.version,
source: entry.url,
sha256: result.sha256,
})
}
log.warn(`installed declared module "${entry.id}" v${entry.version}`, {
from: present || 'nothing',
source: entry.url,
sha256: result.sha256,
})
results.push({ ...entry, action: 'installed', message: null })
} catch (err) {
// Loud, and then onward. The site serves; this module does not, or serves
// the version that was already there.
log.error(
`could not resolve declared module "${entry.id}@${entry.version}": ${err.message}` +
(present ? ` — leaving version ${present} in place` : ''),
)
results.push({ ...entry, action: 'failed', message: err.message })
}
}
return results
}
/** What the last resolution decided, for the admin screen. */
function state() {
return results.map((r) => ({ ...r }))
}
/** Test seam: forget the last resolution. */
function reset() {
results = []
}
module.exports = { VAR, parse, installedVersion, resolve, state, reset }

View File

@@ -0,0 +1,450 @@
// ── Installing and removing a module bundle ────────────────────────────────
//
// Phase 4, slice 1 of docs/website/MODULE_SYSTEM.md §2.7.2 — the consumer half
// of a release the module's own CI already publishes. Nothing here touches the
// database or the loader: this file moves bytes onto the volume and off it, and
// the caller (router/v1/admin/modules.controller.js) writes down what happened.
//
// The shape of an install, and why it is this shape:
//
// 1. The operator pastes the URL of a release's install manifest — a small
// JSON document naming the artifact, its sha256 and its size (decision 2).
// There is no catalog, because a catalog would make core's release cadence
// decide which modules can exist.
// 2. Every URL fetched — the manifest, the artifact, and every redirect hop —
// is checked against the admin-managed host allowlist (decision 6). That is
// what keeps a pasted URL from also being an SSRF primitive.
// 3. The artifact is streamed to a temporary file under a byte cap, hashed as
// it arrives, and compared against the manifest's `sha256`. The hash is the
// trust anchor; releases are unsigned and say so.
// 4. modules/archive.js inspects the archive in full before a single byte is
// unpacked, and only then unpacks it — into a TEMPORARY directory.
// 5. The unpacked tree's own `module.json` must agree with the manifest about
// what it is. A manifest that promises `uo` and delivers something else is
// refused rather than installed under the name it was promised.
// 6. Only then is anything moved into `modules/<id>/`, and the directory it
// replaces is kept aside until the move has succeeded.
//
// **Nothing is written into the modules directory until every check has passed**,
// and that is not belt-and-braces. Verified against node-tar 7.5.22 while writing
// this: extracting an archive whose fourth member escapes upward throws — and
// leaves the first three members on disk. A loader scans that directory at
// require time on the next boot; a half-unpacked module is a module.
//
// One thing this file deliberately does NOT do: mount anything. Installing puts
// a directory on the volume, and §1.12 means the volume is read at require time,
// so the module appears when the process restarts. The admin screen offers that
// restart (decision 1); it is not implied here.
const crypto = require('crypto')
const fs = require('fs')
const fsp = require('fs/promises')
const path = require('path')
const { Readable, Transform } = require('stream')
const { pipeline } = require('stream/promises')
const archive = require('./archive')
const loader = require('./loader')
const log = require('../utils/logger')('modules')
// An install manifest is a small JSON document. A megabyte of it is not a
// manifest, and reading it into memory unbounded is the one place this file
// would otherwise trust a remote length.
const MAX_MANIFEST_BYTES = 256 * 1024
// The artifact cap is the archive's own unpacked cap — a compressed bundle
// larger than what it is allowed to unpack to has nothing to offer.
const MAX_ARTIFACT_BYTES = archive.MAX_BYTES
const FETCH_TIMEOUT_MS = 30000
// Gitea serves a release asset through at least one redirect. Following them is
// necessary; following them blindly is how an allowlist gets bypassed, so each
// hop is re-checked and the chain is bounded.
const MAX_REDIRECTS = 5
// Same id rule the loader enforces when scanning, restated rather than imported
// so an install cannot put a directory on the volume that the loader would then
// refuse to look at.
const ID = /^[a-z][a-z0-9-]{1,31}$/
const SHA256 = /^[0-9a-f]{64}$/
class InstallError extends Error {
constructor(message, { status = 400 } = {}) {
super(message)
this.name = 'InstallError'
// Carried so the controller can answer 400 for "your URL is wrong" and 502
// for "the host you named misbehaved" without re-deriving it from the text.
this.status = status
}
}
// ── The allowlist ──────────────────────────────────────────────────────────
// The settings row the allowlist lives in (decision 6): seeded from
// MODULE_SOURCE_HOSTS on a fresh install and admin-managed from then on. The KEY
// lives here rather than in the admin controller because it is now read from two
// places — the controller, and the boot-time resolution of the declared module
// set (modules/declared.js), which has no route and no request.
const HOSTS_SETTING = 'module_source_hosts'
/**
* Parse the stored allowlist setting into hostnames.
*
* Comma or whitespace separated, case-insensitive, empty entries dropped. An
* empty list means nothing may be installed from anywhere — a refusal, never a
* wildcard. That direction matters: a setting an admin accidentally blanks
* should stop installs, not permit every host on the internet.
*/
function parseHosts(value) {
return String(value || '')
.split(/[,\s]+/)
.map((h) => h.trim().toLowerCase())
.filter(Boolean)
}
/**
* Check one URL against the allowlist, and return it parsed.
*
* `https` only. A plaintext fetch of code this process is going to execute is
* not something an operator should be able to opt into by typing a URL, and the
* sha256 does not help — whoever can rewrite the artifact in flight can rewrite
* the manifest that declares its hash.
*/
function checkUrl(raw, hosts) {
let url
try {
url = new URL(String(raw))
} catch {
throw new InstallError(`"${raw}" is not a valid URL`)
}
if (url.protocol !== 'https:') {
throw new InstallError(`only https URLs may be installed from (got "${url.protocol}")`)
}
if (!hosts.length) {
throw new InstallError(
'no module source hosts are allowed — set one in Admin → Modules before installing',
)
}
if (!hosts.includes(url.hostname.toLowerCase())) {
throw new InstallError(
`"${url.hostname}" is not an allowed module source host (allowed: ${hosts.join(', ')})`,
)
}
return url
}
// ── Fetching ───────────────────────────────────────────────────────────────
/**
* GET a URL, following redirects by hand so every hop is re-checked.
*
* `fetch`'s own redirect following would take the first hop off the allowlist
* and the rest wherever it was pointed, which is exactly the hole the allowlist
* exists to close.
*
* `fetchImpl` is the same injection seam `replayFragments({query})` and
* `lifecycle.boot({model})` use, and it exists for the same reason: the rules
* this file enforces — https only, allowlisted host, allowlisted REDIRECT host,
* bounded body, matching hash — are all decisions about a response, and testing
* them against a real TLS server would mean testing Node's certificate handling
* instead. Production never passes it.
*
* @returns {Promise<Response>} a response whose body has not been read.
*/
async function get(rawUrl, hosts, fetchImpl = fetch) {
let url = checkUrl(rawUrl, hosts)
for (let hop = 0; hop <= MAX_REDIRECTS; hop += 1) {
let res
try {
res = await fetchImpl(url, {
redirect: 'manual',
signal: AbortSignal.timeout(FETCH_TIMEOUT_MS),
headers: { accept: '*/*' },
})
} catch (err) {
throw new InstallError(`could not reach ${url.hostname}: ${err.message}`, { status: 502 })
}
if (res.status >= 300 && res.status < 400) {
const location = res.headers.get('location')
if (!location) throw new InstallError(`${url.hostname} redirected without a location`, { status: 502 })
// Resolved against the current URL, then re-checked — a relative redirect
// is normal and a cross-host one is the thing being guarded against.
url = checkUrl(new URL(location, url).toString(), hosts)
continue
}
if (!res.ok) {
throw new InstallError(`${url.href} returned ${res.status}`, { status: 502 })
}
return res
}
throw new InstallError(`too many redirects (more than ${MAX_REDIRECTS})`, { status: 502 })
}
/** Read a bounded response body as text. */
async function readText(res, cap, what) {
const declared = Number(res.headers.get('content-length'))
if (Number.isFinite(declared) && declared > cap) {
throw new InstallError(`${what} is larger than ${cap} bytes`)
}
const buf = Buffer.from(await res.arrayBuffer())
// Checked again after reading: content-length is the server's claim, not a
// limit it is obliged to honour.
if (buf.length > cap) throw new InstallError(`${what} is larger than ${cap} bytes`)
return buf.toString('utf8')
}
/**
* Stream a response body to a file, hashing as it goes and stopping at the cap.
*
* The hash is computed from the bytes that were written rather than by re-reading
* the file, so there is no window in which the file could differ from what was
* verified.
*/
async function download(res, dest, cap) {
const hash = crypto.createHash('sha256')
let bytes = 0
const body = Readable.fromWeb(res.body)
const counter = new Transform({
transform(chunk, _enc, cb) {
bytes += chunk.length
if (bytes > cap) {
cb(new InstallError(`the artifact is larger than ${Math.round(cap / 1024 / 1024)} MB`))
return
}
hash.update(chunk)
cb(null, chunk)
},
})
await pipeline(body, counter, fs.createWriteStream(dest))
return { sha256: hash.digest('hex'), bytes }
}
// ── The install manifest ───────────────────────────────────────────────────
/**
* Fetch and validate a release's install manifest.
*
* The shape is the one module-uo's release workflow already writes:
* `{schema, id, name, version, coreApi, artifact, url, sha256, size}`. Only the
* fields this side needs are required — a module publisher may carry more.
*/
async function fetchManifest(manifestUrl, hosts, fetchImpl = fetch) {
const res = await get(manifestUrl, hosts, fetchImpl)
const text = await readText(res, MAX_MANIFEST_BYTES, 'the install manifest')
let manifest
try {
manifest = JSON.parse(text)
} catch (err) {
throw new InstallError(`the install manifest is not valid JSON: ${err.message}`)
}
if (!manifest || typeof manifest !== 'object' || Array.isArray(manifest)) {
throw new InstallError('the install manifest is not a JSON object')
}
const { id, name, version, sha256 } = manifest
if (!ID.test(String(id || ''))) {
throw new InstallError(`the install manifest has an invalid module id: ${JSON.stringify(id)}`)
}
if (!name || !version) throw new InstallError('the install manifest is missing name or version')
if (!SHA256.test(String(sha256 || '').toLowerCase())) {
throw new InstallError('the install manifest has no valid sha256 for its artifact')
}
// `url` is the absolute artifact URL the publisher wrote; `artifact` is its
// filename. Prefer the URL, fall back to resolving the filename beside the
// manifest — which is where a release's assets sit — so a manifest that
// travelled without its absolute URL still installs.
const artifactUrl = new URL(manifest.url || manifest.artifact || '', manifestUrl).toString()
return {
id: String(id),
name: String(name),
version: String(version),
sha256: String(sha256).toLowerCase(),
size: Number(manifest.size) || null,
artifactUrl,
coreApi: manifest.coreApi ? String(manifest.coreApi) : null,
}
}
// ── Paths on the volume ────────────────────────────────────────────────────
/** Where a module with this id lives. Rejects an id that is not one. */
function moduleDir(id) {
if (!ID.test(String(id || ''))) throw new InstallError(`invalid module id: ${JSON.stringify(id)}`)
return path.join(loader.dir(), String(id))
}
/** Is there a directory for this module on the volume right now? */
function isInstalled(id) {
try {
return fs.statSync(moduleDir(id)).isDirectory()
} catch {
return false
}
}
/**
* The absolute path of a module's `purge.sql`, or null.
*
* Read from the module's own `module.json` on disk rather than from the loader,
* because purge has to work for a module that never loaded — a `startup_failed`
* one is exactly when an operator wants its tables gone.
*/
function purgeFile(id) {
try {
const manifest = JSON.parse(fs.readFileSync(path.join(moduleDir(id), 'module.json'), 'utf8'))
if (!manifest.purge) return null
const file = path.resolve(moduleDir(id), manifest.purge)
// The same containment check the loader applies to `client.entry`: a
// manifest may not point core at a file outside the module.
if (!file.startsWith(moduleDir(id) + path.sep)) return null
return fs.existsSync(file) ? file : null
} catch {
return null
}
}
// ── Install ────────────────────────────────────────────────────────────────
/**
* Download, verify, unpack and install a module from an install-manifest URL.
*
* Never leaves a partial module on the volume: everything happens under a
* scratch directory that is removed on any failure, and the move into place is
* the last step.
*
* `expect` is what the CALLER was promised, as opposed to what the manifest
* promises about itself — the declared module set (modules/declared.js) pins an
* id and a version in the environment, and a URL that resolves to something else
* has to be refused rather than installed. Checked against the manifest, before
* a byte is downloaded: catching it after the unpack would mean the undeclared
* module is already on the volume when the objection is raised. The admin panel
* passes nothing, because there a URL is the whole of what was asked for.
*
* @param {object} args
* @param {string} args.url the install manifest URL the admin pasted
* @param {string[]} args.hosts the allowlist, already parsed
* @param {{id?: string, version?: string}} [args.expect] what the caller pinned
* @returns {Promise<{id,name,version,sha256,source,bytes,replaced}>}
*/
async function install({ url, hosts, expect = null, fetchImpl = fetch }) {
const manifest = await fetchManifest(url, hosts, fetchImpl)
if (expect && expect.id && manifest.id !== expect.id) {
throw new InstallError(
`that URL installs the module "${manifest.id}", but "${expect.id}" was asked for`,
)
}
if (expect && expect.version && manifest.version !== expect.version) {
throw new InstallError(
`that URL installs ${manifest.id} v${manifest.version}, but v${expect.version} was asked for`,
)
}
const target = moduleDir(manifest.id)
const scratch = await fsp.mkdtemp(path.join(loader.dir(), `.install-${manifest.id}-`))
const tarball = path.join(scratch, 'bundle.tar.gz')
const unpacked = path.join(scratch, 'unpacked')
try {
const res = await get(manifest.artifactUrl, hosts, fetchImpl)
const { sha256, bytes } = await download(res, tarball, MAX_ARTIFACT_BYTES)
if (sha256 !== manifest.sha256) {
// The whole trust model in one comparison. Deliberately does not name the
// computed hash in a way that reads like a value to copy into the
// manifest — a mismatch means stop, not reconcile.
throw new InstallError(
`the downloaded artifact does not match the sha256 in the install manifest — refusing to install`,
)
}
if (manifest.size && bytes !== manifest.size) {
throw new InstallError(`the artifact is ${bytes} bytes but the manifest declares ${manifest.size}`)
}
await archive.unpack(tarball, unpacked)
// What the bundle says it is, checked against what the manifest promised.
let inner
try {
inner = JSON.parse(await fsp.readFile(path.join(unpacked, 'module.json'), 'utf8'))
} catch {
throw new InstallError('the bundle has no readable module.json at its root')
}
if (inner.id !== manifest.id) {
throw new InstallError(
`the bundle declares module id "${inner.id}" but the install manifest promised "${manifest.id}"`,
)
}
if (inner.version !== manifest.version) {
throw new InstallError(
`the bundle declares version "${inner.version}" but the install manifest promised "${manifest.version}"`,
)
}
// The swap. The outgoing directory is moved aside rather than deleted first,
// so a failed rename leaves the previous version recoverable instead of
// leaving no module at all — the same reasoning the installer repo applies
// to a ServUO tree.
const replaced = fs.existsSync(target)
const aside = `${target}.replaced-${Date.now()}`
if (replaced) await fsp.rename(target, aside)
try {
await fsp.rename(unpacked, target)
} catch (err) {
if (replaced) await fsp.rename(aside, target).catch(() => {})
throw err
}
if (replaced) await fsp.rm(aside, { recursive: true, force: true })
log.info(`installed module "${manifest.id}" v${manifest.version}`, {
source: url,
sha256: manifest.sha256,
bytes,
replaced,
})
return { ...manifest, source: url, bytes, replaced }
} finally {
await fsp.rm(scratch, { recursive: true, force: true }).catch(() => {})
}
}
/**
* Remove a module's directory from the volume.
*
* Returns whether there was one. Does not touch the database, does not run
* purge.sql, and does not stop the running module — the controller sequences
* all three, because the order matters and only it knows what the operator
* asked for.
*/
async function removeDir(id) {
const dir = moduleDir(id)
if (!fs.existsSync(dir)) return false
await fsp.rm(dir, { recursive: true, force: true })
log.info(`removed module directory for "${id}"`)
return true
}
module.exports = {
InstallError,
HOSTS_SETTING,
parseHosts,
checkUrl,
get,
fetchManifest,
install,
removeDir,
moduleDir,
isInstalled,
purgeFile,
MAX_MANIFEST_BYTES,
MAX_ARTIFACT_BYTES,
}

View File

@@ -101,10 +101,16 @@ async function boot({ modules, model } = {}) {
id: m.id,
name: m.name,
version: m.version,
// Null provenance is what a hand-placed directory looks like. An install
// performed through the admin panel (§2.5, a later phase) writes the row
// with its source and hash first; this refresh deliberately does not
// overwrite either, because recordInstalled leaves what it is not given.
// Null provenance is what a hand-placed directory looks like, and it is
// all this step can honestly say: a scan finds a directory, never where it
// came from. An install through the admin panel writes source and sha256
// first, and this refresh must not undo that.
//
// It used to. `upsert` assigned both columns unconditionally, so every
// boot nulled them and an install's provenance survived only until the
// restart it asked for. The statement now COALESCEs — see the note on
// modules.db.js's upsert. Nothing could have caught it before Phase 4:
// this was the only caller, and it has never had a value to pass.
}))
}
@@ -205,4 +211,67 @@ async function shutdown({ modules, budgetMs = SHUTDOWN_BUDGET_MS } = {}) {
}
}
module.exports = { boot, shutdown, SHUTDOWN_BUDGET_MS }
/**
* Stop ONE module and mark it disabled — the admin panel's Disable (§2.7.2
* decision 3).
*
* Phase 2 built disable as a pure state flip: the record moved to `disabled` and
* the dispatch guard started answering 404. That makes a module invisible, not
* stopped. Everything a module does that is not a response to a request — the
* sockets and timers its `onBoot` armed — carried on running, so an operator
* disabling a module *because* it was misbehaving got nothing until the next
* restart, which is the one thing the button was supposed to save them.
*
* So the hook runs first, and the state moves after it: while `onShutdown` is
* running the module is still `started`, which is the only state in which its
* own routes and the things it is tearing down are consistent with each other.
* The hook gets the same budget the exit path gives it.
*
* A hook that throws does NOT stop the disable. The operator asked for this
* module to stop answering; a module that could not close cleanly is a reason to
* log loudly, not a reason to leave it serving. That is the opposite of the boot
* path's rule, and deliberately: there, a failure means the module never became
* safe to use.
*
* **Enable is not the mirror of this, and there is no `start(id)` beside it.**
* There is no `onBoot` re-dispatch, and MODULE_API.md has never promised the
* hooks are re-entrant — no module author has written `onBoot` to be safe to run
* twice in one process. Re-enabling therefore moves the row and waits for a
* restart, which the admin screen offers.
*
* @returns {Promise<{stopped: boolean, error: string|null}>} whether a hook ran.
*/
async function stop(id, { modules, model, budgetMs = SHUTDOWN_BUDGET_MS } = {}) {
/* eslint-disable global-require */
const loader = modules || require('./loader')
const rows = model || require('../model/modules/modules.model')
/* eslint-enable global-require */
let error = null
let stopped = false
if (loader.isLoaded()) {
const target = loader.stopHook(id)
if (target) {
try {
await withBudget(target.hook, budgetMs)
stopped = true
log.info(`module "${id}" stopped by an operator`)
} catch (err) {
error = err.message
log.warn(`module "${id}" onShutdown failed or timed out while being disabled — disabling anyway`, {
error: err.message,
})
}
}
// Unconditional, and after the hook: this is what makes its routes and its
// client chunk answer 404 (§4.5). A module that is not loaded in this
// process has no record to move, and setState ignores an unknown id.
loader.setState(id, 'disabled')
}
await safe(`disabling module "${id}"`, () => rows.disable(id))
return { stopped, error }
}
module.exports = { boot, shutdown, stop, SHUTDOWN_BUDGET_MS }

View File

@@ -796,6 +796,31 @@ function shutdownHooks() {
.reverse()
}
/**
* One module's `onShutdown`, for stopping it on its own rather than at exit.
*
* Phase 4 (§2.7.2 decision 3) gave the admin panel's Disable a real meaning.
* Until then, disabling flipped this record's state and the dispatch guard began
* answering 404 — which made the module invisible without making it stop. A
* module's `onBoot` is where it opens its sockets and arms its timers, and none
* of that is reachable through a URL, so an operator disabling a misbehaving
* module got no relief from it at all until the next restart.
*
* `started` only, the same rule shutdownHooks() applies and for the same reason:
* a module whose `onBoot` threw has a half-built world its `onShutdown` was
* never written to tear down. Returns null when there is nothing to run — which
* covers "not started", "no hook", and "no such module", none of which is an
* error the caller can act on differently.
*
* @returns {{id: string, hook: Function}|null}
*/
function stopHook(id) {
assertLoaded('stopHook')
const record = modules.get(id)
if (!record || record.state !== 'started' || !record.hooks.onShutdown) return null
return { id: record.id, hook: record.hooks.onShutdown }
}
/**
* Every schema fragment waiting to be replayed, in scan order.
*
@@ -896,6 +921,7 @@ module.exports = {
fragments,
bootable,
shutdownHooks,
stopHook,
clientChunks,
clientEntryUrls,
specFragments,

View File

@@ -81,4 +81,48 @@ async function replayFragments({ query, modules } = {}) {
}
}
module.exports = { replayFragments }
/**
* Run one module's `purge.sql` — the destructive twin of the replay above
* (MODULE_SYSTEM.md §2.5, and §2.7.2 decision 5 for when it is offered).
*
* It lives beside replayFragments because they are the same operation pointed in
* opposite directions: a file of statements the module ships, split by the same
* splitter, run serially on the same pool. Keeping them together is what makes
* it obvious that the fragment's leading-verb allowlist does NOT apply here —
* `purge.sql` is the one file a module may put a DROP in, precisely because it
* is the one file that never runs on a boot.
*
* Unlike the replay, this **throws**. A replay failure is one module failing to
* start, which the site survives by 503ing that module; a purge failure is an
* operator's explicit destructive request not having happened, and reporting
* success for that would leave them believing data is gone when it is not.
*
* Statements run serially and are not wrapped in a transaction, for the reason
* the replay's header already gives: MariaDB commits DDL implicitly, so there is
* no rollback to have. A purge that fails halfway has dropped some tables — the
* error names the statement that stopped it, and running it again is safe
* because a purge script is required to be idempotent in the same way a fragment
* is (`DROP TABLE IF EXISTS`).
*
* @param {string} file absolute path to the module's purge.sql
* @param {object} [deps] injection seam for tests
* @returns {Promise<number>} how many statements ran
*/
async function runPurge(file, { query } = {}) {
// eslint-disable-next-line global-require
const run = query || require('../utils/db').query
const statements = splitStatements(fs.readFileSync(file, 'utf8'))
let ran = 0
for (const statement of statements) {
try {
await run(statement)
ran += 1
} catch (err) {
throw new Error(`purge failed at statement ${ran + 1} of ${statements.length}: ${err.message}`)
}
}
return ran
}
module.exports = { replayFragments, runPurge }

View File

@@ -30,6 +30,7 @@ const pagesRouter = require('./pages.router')
const emailRouter = require('./email.router')
const discordBotRouter = require('./discordBot.router')
const settingsRouter = require('./settings.router')
const modulesRouter = require('./modules.router')
const dashboardRouter = require('./dashboard.router')
const adminRouter = express.Router()
@@ -73,6 +74,11 @@ adminRouter.use('/pages', pagesRouter)
adminRouter.use('/email', emailRouter)
adminRouter.use('/discord-bot', discordBotRouter)
adminRouter.use('/settings', settingsRouter)
// The Modules screen (Phase 4). Core's, not a module's — and it has to be
// core's: it is how a module gets onto the volume in the first place. Mounted
// here alongside the other configuration capabilities, and admin-only per route
// rather than at this line, so the gate sits next to what it is guarding.
adminRouter.use('/modules', modulesRouter)
// The two singletons that own no path segment of their own: GET /dashboard and
// PUT /site-mode. Mounted at the group root, last, exactly where the residual

View File

@@ -0,0 +1,426 @@
// ── Admin: installed modules ───────────────────────────────────────────────
//
// Phase 4, slice 1 of docs/website/MODULE_SYSTEM.md §2.7.2. Admin-only, and more
// so than anything else in this directory: installing a module puts JavaScript on
// the volume that core will `require` into its own process on the next boot. That
// is the feature — it is what "an operator never builds anything" means (§1.14) —
// but it is worth being plain that this controller is remote code execution with
// an audit trail, not a settings screen.
//
// What guards it, in the order an attacker would meet them:
//
// 1. `requireRole('admin')` on every route, on top of the group's staff gate.
// 2. An `https`-only host allowlist, re-checked on every redirect hop, so a
// pasted URL cannot be pointed at the compose network or a metadata service
// (install.js).
// 3. The sha256 the release published, compared against the bytes that arrived.
// 4. A full inspection of the archive before a byte of it is unpacked, and an
// unpack into a scratch directory that is only moved into place once the
// bundle has agreed with the manifest about what it is (archive.js).
// 5. Every action here writes to core's one audit log.
//
// The one thing this file cannot do is mount anything. §1.12 makes the volume the
// mounting source of truth, read once at require time, so install and uninstall
// take effect on the next boot — which is why `restart` is a route here rather
// than a sentence in a tooltip (decision 1).
const modules = require('../../../model/modules/modules.model')
const activity = require('../../../model/activity/activity.model')
const settings = require('../../../model/settings/settings.model')
const loader = require('../../../modules/loader')
const lifecycle = require('../../../modules/lifecycle')
const install = require('../../../modules/install')
const declared = require('../../../modules/declared')
// A namespace import, like every other require in this file, and not
// `const { runPurge } = …`: destructuring at require time captures the function
// rather than the module, which makes it the one dependency here that cannot be
// substituted. That matters because the two tests worth having about purge are
// about the ORDER it runs in relative to the directory being removed.
const schema = require('../../../modules/schema')
const log = require('../../../utils/logger')('admin-modules')
// The allowlist setting. Seeded from MODULE_SOURCE_HOSTS on first boot and
// admin-managed from then on (decision 6) — db/seed.js writes it once and never
// overwrites it, so changing the variable later does not silently reach in and
// undo an operator's choice. The key itself lives in install.js, which is now
// read by boot-time declared-set resolution as well as by this controller.
const HOSTS_KEY = install.HOSTS_SETTING
// A hostname, not a URL: no scheme, no path, no port, no wildcard. Deliberately
// strict — every character allowed here is a character that can appear in the
// host of a URL this server will fetch and execute the contents of.
const HOSTNAME = /^[a-z0-9]([a-z0-9-]*[a-z0-9])?(\.[a-z0-9]([a-z0-9-]*[a-z0-9])?)*$/
async function allowedHosts() {
return install.parseHosts(await settings.get(HOSTS_KEY))
}
/**
* One module, as the admin screen needs it.
*
* Four sources have to be reconciled, and which one answers which question is
* the whole of §2.4:
*
* - the ROW says what the operator decided and what the last boot recorded;
* - the LOADER says what is mounted and answering right now;
* - the VOLUME says whether there is still a directory there at all;
* - the DECLARATION (slice 3) says what this container's environment asks for,
* which is the only one of the four an admin cannot change from this screen.
*
* They can legitimately disagree, and the screen has to show that rather than
* pick a winner. A row `enabled` with a loader state of `disabled` is a module
* the operator has just switched back on and which is waiting for a restart —
* exactly the case decision 3 creates, and it would be a lie to render it as
* either "running" or "off". A declared module with no row and no directory is
* the newest of those disagreements: MODULES asked for it and resolution could
* not get it, so the screen carries the reason rather than showing nothing.
*/
function present(row, live, onVolume, declaration = null) {
const id = row ? row.id : live ? live.id : declaration.id
return {
id,
name: row ? row.name : live ? live.name : id,
version: row ? row.version : live ? live.version : null,
// What the database records.
state: row ? row.state : null,
failureStage: row ? row.failureStage : (live && live.stage) || null,
failureReason: row ? row.failureReason : (live && live.reason) || null,
source: row ? row.source : null,
sha256: row ? row.sha256 : null,
installedAt: row ? row.installedAt : null,
startedAt: row ? row.startedAt : null,
// What is actually mounted in this process, and what it is answering.
liveState: live ? live.state : null,
// The version RUNNING, which is not always the version installed: an upgrade
// writes new files and a new row while the old code stays loaded until the
// restart. Without this the screen would report the new version as
// "Running", which is the same lie in a different place.
liveVersion: live ? live.version : null,
capabilities: live ? live.capabilities : [],
// What is on the volume.
onVolume,
canPurge: onVolume && Boolean(install.purgeFile(id)),
// What the environment declares. `declaredVersion` is what MODULES pins, not
// what is installed — they differ exactly while a resolution is failing, and
// `declaredError` says why. Uninstalling a declared module from this screen
// removes its directory and disables its row; the next boot puts the
// directory back and leaves the row disabled, so the screen says so rather
// than letting the files reappear unexplained.
declared: Boolean(declaration),
declaredVersion: declaration ? declaration.version : null,
declaredError: declaration && declaration.action === 'failed' ? declaration.message : null,
}
}
// GET /admin/modules — every module core knows about, from all three sources,
// plus the source allowlist the install form needs.
async function list(req, res) {
try {
const rows = await modules.list()
// The loader throws rather than returning [] before load() has run (§7.6),
// and this controller is reachable from a process where that is true —
// `npm run seed` never gets here, but a test harness might.
const live = loader.isLoaded() ? loader.list() : []
const byId = new Map(live.map((m) => [m.id, m]))
const declaredById = new Map(declared.state().map((d) => [d.id, d]))
const seen = new Set()
const out = []
for (const row of rows) {
seen.add(row.id)
out.push(
present(row, byId.get(row.id) || null, install.isInstalled(row.id), declaredById.get(row.id)),
)
}
// A directory on the volume that has no row yet — a hand-placed install
// before its first boot. It has to be listed, or the screen would show
// nothing for a module whose routes are already being served.
for (const m of live) {
if (!seen.has(m.id)) {
seen.add(m.id)
out.push(present(null, m, true, declaredById.get(m.id)))
}
}
// A module MODULES declares that has neither. Resolution failed and left
// nothing behind — the case an operator most needs told, because from the
// screen's other three sources it is indistinguishable from never having
// asked for it.
for (const d of declaredById.values()) {
if (!seen.has(d.id)) out.push(present(null, null, install.isInstalled(d.id), d))
}
return res.json({ modules: out, sourceHosts: await allowedHosts() })
} catch (err) {
log.error('list modules', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/modules — install (or upgrade) from an install-manifest URL.
async function create(req, res) {
const url = String(req.body.url || '').trim()
try {
const hosts = await allowedHosts()
const result = await install.install({ url, hosts })
// Provenance is written here and nowhere else: the boot reconcile records a
// module with NULL source/sha256 and leaves what it is not given, precisely
// so that a refresh cannot overwrite what an install knew (lifecycle.js).
const row = await modules.recordInstalled({
id: result.id,
name: result.name,
version: result.version,
source: result.source,
sha256: result.sha256,
})
await activity.log({
req,
userId: req.user.id,
action: 'module.install',
detail: { id: result.id, version: result.version, source: url, sha256: result.sha256, replaced: result.replaced },
})
log.warn('module installed — it will mount on the next restart', {
id: result.id,
version: result.version,
by: req.user.username,
})
return res.status(201).json({ module: row, restartRequired: true, replaced: result.replaced })
} catch (err) {
if (err.name === 'InstallError' || err.name === 'ArchiveError') {
// The operator pasted a URL and something about what came back was wrong.
// The message is the useful part and is written to be read by them.
log.warn('module install refused', { url, reason: err.message })
return res.status(err.status || 400).json({ message: err.message })
}
log.error('install module', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/modules/:id/enable — switch a module back on, for the next boot.
//
// Deliberately does NOT touch the loader's record. Disable ran the module's
// onShutdown (decision 3), and there is no onBoot re-dispatch to undo that: a
// module whose sockets were closed and timers cleared cannot be made to serve
// again by flipping a flag, and pretending otherwise would put it back on the
// nav with a torn-down world behind it. The row moves; the restart starts it.
async function enable(req, res) {
const { id } = req.params
try {
const row = await modules.enable(id)
if (!row) return res.status(404).json({ message: 'No such module.' })
await activity.log({ req, userId: req.user.id, action: 'module.enable', detail: { id } })
return res.json({ module: row, restartRequired: true })
} catch (err) {
if (err.name === 'ModuleStateError') return res.status(409).json({ message: err.message })
log.error('enable module', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/modules/:id/disable — stop it now.
//
// The one action on this screen that takes effect without a restart, and the
// reason it does is that it is the one an operator reaches for when something is
// going wrong. Its routes answer 404 from the moment this returns, and its
// onShutdown has already run.
async function disable(req, res) {
const { id } = req.params
try {
const current = await modules.get(id)
if (!current) return res.status(404).json({ message: 'No such module.' })
const { stopped, error } = await lifecycle.stop(id)
const row = await modules.get(id)
await activity.log({
req,
userId: req.user.id,
action: 'module.disable',
detail: { id, hookRan: stopped, hookError: error },
})
log.warn('module disabled by an operator', { id, hookRan: stopped, by: req.user.username })
// `shutdownError` is reported rather than swallowed: the module IS disabled
// either way, and an operator whose module could not close cleanly should be
// told so while they still have the logs to look at.
return res.json({ module: row, stopped, shutdownError: error })
} catch (err) {
log.error('disable module', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// DELETE /admin/modules/:id[?purge=true] — uninstall.
//
// Non-destructive by default (§2.5): the directory goes, the row stays
// `disabled`, and the module's tables and data are left alone.
//
// The purge option is here rather than as a follow-up action because it cannot
// be a follow-up action (decision 5): `purge.sql` is a file inside the directory
// this is about to delete, so after an uninstall there is nothing left to purge
// with. Ticking the box is the last moment the file exists.
//
// The order below is the whole of it, and each step depends on the one above:
// purge while the SQL is still readable, stop while the code is still loaded,
// then delete.
async function remove(req, res) {
const { id } = req.params
const purge = req.query.purge === 'true' || req.query.purge === '1'
try {
const current = await modules.get(id)
const onVolume = install.isInstalled(id)
if (!current && !onVolume) return res.status(404).json({ message: 'No such module.' })
let purged = null
if (purge) {
const file = install.purgeFile(id)
if (!file) {
return res.status(400).json({
message: 'This module ships no purge.sql, so its data cannot be deleted. Uninstall without purging instead.',
})
}
purged = await schema.runPurge(file)
}
// Stop it before its files vanish. A module whose directory is deleted out
// from under a running onShutdown is being asked to tear down a world whose
// code may already be half-unreadable — and its sockets would otherwise stay
// open until the restart, holding a connection on behalf of a module that no
// longer exists on disk.
await lifecycle.stop(id)
const removed = await install.removeDir(id)
// A purge leaves nothing: no directory, no tables, no data. Keeping a
// `disabled` row for that is a tombstone with nothing to offer and a Purge
// button that would fail. A plain uninstall keeps its row, which is what
// makes the retained data visible and reinstallable.
if (purge) await modules.remove(id)
await activity.log({
req,
userId: req.user.id,
action: purge ? 'module.purge' : 'module.uninstall',
detail: { id, purged, removed },
})
log.warn(`module ${purge ? 'uninstalled and purged' : 'uninstalled'}`, {
id,
statements: purged,
by: req.user.username,
})
return res.json({ id, removed, purged, restartRequired: true })
} catch (err) {
log.error('uninstall module', err)
return res.status(500).json({ message: err.message || 'Internal Server Error' })
}
}
// POST /admin/modules/:id/purge — drop a still-installed module's data.
//
// Refuses unless the module is already disabled, and that guard is the point:
// dropping the tables under a module that is still serving requests leaves it
// answering out of a world that no longer exists. Disabling first is one click
// and makes the destructive step happen against something that has stopped.
async function purge(req, res) {
const { id } = req.params
try {
const current = await modules.get(id)
if (!current) return res.status(404).json({ message: 'No such module.' })
if (current.state !== 'disabled') {
return res.status(409).json({
message: 'Disable this module before purging its data, so nothing is serving out of the tables being dropped.',
})
}
const file = install.purgeFile(id)
if (!file) {
return res.status(400).json({ message: 'This module ships no purge.sql, so its data cannot be deleted.' })
}
const statements = await schema.runPurge(file)
await activity.log({ req, userId: req.user.id, action: 'module.purge', detail: { id, statements } })
log.warn('module data purged', { id, statements, by: req.user.username })
return res.json({ id, purged: statements })
} catch (err) {
log.error('purge module', err)
return res.status(500).json({ message: err.message || 'Internal Server Error' })
}
}
// PUT /admin/modules/sources — the host allowlist.
async function setSources(req, res) {
const hosts = install.parseHosts(req.body.hosts)
const bad = hosts.find((h) => !HOSTNAME.test(h))
if (bad) return res.status(400).json({ message: `"${bad}" is not a valid hostname.` })
try {
const before = await allowedHosts()
await settings.set(HOSTS_KEY, hosts.join(','), req.user.id)
await activity.log({
req,
userId: req.user.id,
action: 'module.sources',
detail: { before, after: hosts },
})
log.warn('module source allowlist changed', { before, after: hosts, by: req.user.username })
return res.json({ sourceHosts: hosts })
} catch (err) {
log.error('set module sources', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/modules/restart — restart the server process.
//
// Decision 1. Install, uninstall and re-enable all only take effect at boot
// because §1.12 reads the volume at require time, and §2.4 promises recovery
// "with no shell access to the box" — which a banner saying "please restart your
// container" does not deliver.
//
// It reaches server.js's existing SIGTERM handler rather than doing the work
// itself: that handler stops the modules, the workers and the listeners in the
// right order and closes the pool and the log file before exiting 0, and going
// through it means there is exactly one graceful-shutdown path that this route
// cannot drift from.
//
// It gets there by EMITTING the event, not by signalling the process, and that
// is not a detail. `process.kill(process.pid, 'SIGTERM')` is what this did
// first, and it works on Linux — but **Windows has no POSIX signals, and Node
// documents SIGTERM there as unconditional termination of the target process**.
// So on a Windows host the restart killed the server outright: no module
// `onShutdown`, no pool close, no log flush. Verified by running it — the
// process was gone and the shutdown handler had logged nothing.
//
// `process.on('SIGTERM', …)` is an ordinary EventEmitter listener, so
// `process.emit('SIGTERM')` invokes exactly the same handler on every platform
// without involving the OS at all. Deployment is Linux containers and would
// never have shown this; development is not.
//
// What brings the process BACK is the supervisor, not this. The shipped
// docker-compose.yml declares `restart: unless-stopped` on `app`, which restarts
// on a clean exit as well as a crash. A bare `npm start` does not come back, and
// the screen says so before it asks.
function restart(req, res) {
log.warn('restart requested from the admin panel', { by: req.user.username })
// Logged and answered first. Once the signal is raised the response has no
// listener left to flush through, so the operator would be told nothing.
res.status(202).json({ restarting: true })
activity
.log({ req, userId: req.user.id, action: 'module.restart', detail: {} })
.catch((err) => log.error('failed to record the restart in the audit log', err))
.finally(() => {
// A beat, so the 202 is on the wire. `unref` so this timer is not itself
// something keeping the process alive.
setTimeout(() => process.emit('SIGTERM'), 250).unref()
})
}
module.exports = { list, create, enable, disable, remove, purge, setSources, restart, HOSTS_KEY }

View File

@@ -0,0 +1,143 @@
// Admin · Modules — install, enable, disable, uninstall, purge and restart.
//
// Mounted at /api/v1/admin/modules by admin/index.js, which has already applied
// `noindex, isLoggedIn, staffOnly`. Every route here re-gates to `admin`: an
// editor or moderator has no business installing code into the server process,
// and the group gate alone would let them.
//
// Route order matters in one place. `/restart` and `/sources` are declared
// BEFORE the `/:id/...` routes, because express matches in declaration order and
// a module whose id was `restart` would otherwise shadow — or be shadowed by —
// the literal path. The id pattern below makes that unreachable in practice; the
// ordering makes it unreachable by construction.
const express = require('express')
const { body, param, query } = require('express-validator')
const controller = require('./modules.controller')
const { requireRole } = require('../../../utils/auth')
const validate = require('../../../middleware/validate')
const modulesRouter = express.Router()
const adminOnly = requireRole('admin')
// The loader's own id rule (MODULE_API.md §2.1). Applied at the edge so a
// traversal-shaped id never reaches a path join, even though install.js checks
// it again — this one produces a 400 with a readable message, that one is the
// guarantee.
const ID = /^[a-z][a-z0-9-]{1,31}$/
modulesRouter.get(
'/',
// #swagger.tags = ['Admin · Modules']
// #swagger.summary = 'List installed modules, their live state, and the source allowlist'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Modules and the install source allowlist', content: { "application/json": { schema: { type: "object", properties: { modules: { type: "array", items: { type: "object", additionalProperties: true } }, sourceHosts: { type: "array", items: { type: "string" } } } } } } } */
adminOnly,
controller.list,
)
modulesRouter.post(
'/',
// #swagger.tags = ['Admin · Modules']
// #swagger.summary = 'Install or upgrade a module from a release install-manifest URL'
// #swagger.description = 'Downloads the artifact the manifest names, verifies its sha256, inspects the archive in full and unpacks it onto the modules volume. The module mounts on the next restart.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["url"], properties: { url: { type: "string", description: "https URL of the release install manifest, on an allowed host" } } } } } } */
/* #swagger.responses[201] = { description: 'Installed — restart to mount it', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[400] = { description: 'The URL, the manifest, the hash or the archive was refused', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[502] = { description: 'The source host could not be reached or answered badly', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
body('url').isString().trim().isLength({ min: 1, max: 2048 }),
validate,
controller.create,
)
modulesRouter.put(
'/sources',
// #swagger.tags = ['Admin · Modules']
// #swagger.summary = 'Replace the allowlist of hosts modules may be installed from'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["hosts"], properties: { hosts: { type: "string", description: "Comma- or space-separated hostnames. An empty list forbids all installs." } } } } } } */
/* #swagger.responses[200] = { description: 'The new allowlist', content: { "application/json": { schema: { type: "object", properties: { sourceHosts: { type: "array", items: { type: "string" } } } } } } } */
/* #swagger.responses[400] = { description: 'One of the entries is not a hostname', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
body('hosts').isString().isLength({ max: 2048 }),
validate,
controller.setSources,
)
modulesRouter.post(
'/restart',
// #swagger.tags = ['Admin · Modules']
// #swagger.summary = 'Restart the server process so module changes take effect'
// #swagger.description = 'Runs the same graceful shutdown a SIGTERM does. The process is brought back by the supervisor, which the shipped docker-compose.yml provides; a bare `npm start` will not come back.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[202] = { description: 'Shutting down', content: { "application/json": { schema: { type: "object", properties: { restarting: { type: "boolean" } } } } } } */
adminOnly,
controller.restart,
)
modulesRouter.post(
'/:id/enable',
// #swagger.tags = ['Admin · Modules']
// #swagger.summary = 'Enable a module (takes effect on the next restart)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Module id.' }
/* #swagger.responses[200] = { description: 'Enabled — restart to start it', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[404] = { description: 'No such module', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
param('id').matches(ID),
validate,
controller.enable,
)
modulesRouter.post(
'/:id/disable',
// #swagger.tags = ['Admin · Modules']
// #swagger.summary = 'Stop a module now — runs its onShutdown, then its routes answer 404'
// #swagger.description = 'The only module action that takes effect without a restart. Re-enabling needs one, because there is no onBoot re-dispatch.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Module id.' }
/* #swagger.responses[200] = { description: 'Disabled', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[404] = { description: 'No such module', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
param('id').matches(ID),
validate,
controller.disable,
)
modulesRouter.post(
'/:id/purge',
// #swagger.tags = ['Admin · Modules']
// #swagger.summary = "Run a disabled module’s purge.sql, dropping its tables and data"
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Module id.' }
/* #swagger.responses[200] = { description: 'Purged', content: { "application/json": { schema: { type: "object", properties: { id: { type: "string" }, purged: { type: "integer" } } } } } } */
/* #swagger.responses[400] = { description: 'The module ships no purge.sql', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[409] = { description: 'The module must be disabled first', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
param('id').matches(ID),
validate,
controller.purge,
)
modulesRouter.delete(
'/:id',
// #swagger.tags = ['Admin · Modules']
// #swagger.summary = 'Uninstall a module, optionally deleting its data too'
// #swagger.description = "Removes the module directory and leaves its row disabled. With purge=true it also runs purge.sql first — which is the only moment it can, since purge.sql lives inside the directory being deleted."
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Module id.' }
// #swagger.parameters['purge'] = { in: 'query', required: false, schema: { type: 'boolean' }, description: "Also run the module’s purge.sql and drop its row. Destructive and irreversible." }
/* #swagger.responses[200] = { description: 'Uninstalled — restart to unmount it', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[400] = { description: 'Purge was asked for and the module ships no purge.sql', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[404] = { description: 'No such module', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
param('id').matches(ID),
query('purge').optional().isIn(['true', 'false', '1', '0']),
validate,
controller.remove,
)
module.exports = modulesRouter

View File

@@ -1,8 +1,12 @@
require('dotenv').config()
const http = require('http')
const app = require('./app')
const internalApp = require('./internalApp')
// NOTE: `./app` and `./internalApp` are deliberately NOT required here. Requiring
// app.js runs `modules.load()`, which scans the volume and mounts whatever is on
// it (MODULE_API.md §4.1) — so the declared module set has to be resolved before
// that require, not before the listener. They are required inside start(), after
// resolveDeclaredModules(); everything else this file needs is safe to pull in
// now because none of it reaches the loader's scan.
const botScore = require('./middleware/botScore')
const announceWorker = require('./utils/announceWorker')
const { ensureSchema, close } = require('./utils/db')
@@ -11,6 +15,9 @@ const settings = require('./model/settings/settings.model')
const revokedSessions = require('./model/revokedSessions/revokedSessions.model')
const mobileAuthBridge = require('./model/mobileAuthBridge/mobileAuthBridge.model')
const moduleLifecycle = require('./modules/lifecycle')
const declaredModules = require('./modules/declared')
const moduleInstall = require('./modules/install')
const moduleModel = require('./model/modules/modules.model')
const createLogger = require('./utils/logger')
const { evaluateBotInternalKey } = require('./utils/botInternalKey')
const brand = require('./config/brand')
@@ -51,7 +58,9 @@ async function start() {
}
log.info('ensuring database schema...')
await ensureSchema()
// Core's schema only. Each installed module's fragment is replayed further
// down, after the volume has been scanned — see the require of ./app below.
await ensureSchema({ replayModules: false })
log.info('seeding defaults...')
await seedDefaults()
await createInitialAdminFromEnv()
@@ -77,6 +86,36 @@ async function start() {
const mode = await settings.get('site_mode')
log.info(`site mode: ${String(mode || 'live').toUpperCase()}`)
// Bring the modules volume in line with what MODULES declares (§2.7.2
// decision 4), and only then require the app — the loader scans and mounts at
// require time, so this is the last moment at which a module can be put on the
// volume and still be part of this process.
//
// After the schema and the seed, because the host allowlist it installs under
// is a settings row that the seed creates on a fresh instance. Never throws:
// an unreachable release host leaves the site serving without that module
// rather than taking the site down with it.
await declaredModules.resolve({
hosts: moduleInstall.parseHosts(await settings.get(moduleInstall.HOSTS_SETTING)),
model: moduleModel,
})
// Requiring app.js is what scans the volume and mounts what is on it. Every
// line above this one runs against a core that has no modules in it yet.
// eslint-disable-next-line global-require
const app = require('./app')
// eslint-disable-next-line global-require
const internalApp = require('./internalApp')
// Now that the scan has happened, replay each module's schema fragment
// (MODULE_API.md §2.6). This used to ride inside ensureSchema() and could,
// because app.js was required at the top of this file; resolving the declared
// set first moved the scan after it, and a booting server quietly getting no
// module tables is precisely what §7.6 warns about. Caught by the browser
// smoke rather than by a test: every suite here stubs one side or the other.
// eslint-disable-next-line global-require
await require('./modules/schema').replayFragments()
// Reconcile installed_modules with what the loader found on the volume at
// require time, then run each module's onBoot (MODULE_API.md §2.5).
//

View File

@@ -55,11 +55,18 @@ const SCHEMA_PATH = path.join(__dirname, '..', '..', 'db', 'schema.sql')
* below — the discovery, splitting and per-module failure handling all live in
* modules/schema.js, required lazily so that requiring the pool never drags the
* loader in with it.
*
* `replayModules: false` is for a caller that has not scanned the volume YET and
* intends to. server.js is the one: since slice 3 it resolves the declared
* module set before requiring app.js, which puts core's schema *before* the scan
* — so it replays the fragments itself, in the one place that knows the scan has
* happened. Left true everywhere else, so the ordinary caller cannot get module
* tables by accident and lose them by refactor.
*/
async function ensureSchema({ retries = 10, delayMs = 2000 } = {}) {
async function ensureSchema({ retries = 10, delayMs = 2000, replayModules = true } = {}) {
await ensureCoreSchema({ retries, delayMs })
// eslint-disable-next-line global-require
await require('../modules/schema').replayFragments()
if (replayModules) await require('../modules/schema').replayFragments()
}
/** Core's own schema.sql, with the wait-for-the-database retry. */

View File

@@ -2353,6 +2353,490 @@
}
}
},
"/api/v1/admin/modules": {
"get": {
"tags": [
"Admin · Modules"
],
"summary": "List installed modules, their live state, and the source allowlist",
"description": "",
"responses": {
"200": {
"description": "Modules and the install source allowlist",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"modules": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"sourceHosts": {
"type": "array",
"items": {
"type": "string"
}
}
}
}
}
}
},
"500": {
"description": "Internal Server Error"
}
},
"security": [
{
"cookieAuth": []
},
{
"bearerAuth": []
}
]
},
"post": {
"tags": [
"Admin · Modules"
],
"summary": "Install or upgrade a module from a release install-manifest URL",
"description": "Downloads the artifact the manifest names, verifies its sha256, inspects the archive in full and unpacks it onto the modules volume. The module mounts on the next restart.",
"responses": {
"201": {
"description": "Installed — restart to mount it",
"content": {
"application/json": {
"schema": {
"type": "object",
"additionalProperties": true
}
}
}
},
"400": {
"description": "The URL, the manifest, the hash or the archive was refused",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
}
}
}
},
"500": {
"description": "Internal Server Error"
},
"502": {
"description": "The source host could not be reached or answered badly",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
}
}
}
}
},
"security": [
{
"cookieAuth": []
},
{
"bearerAuth": []
}
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"type": "object",
"required": [
"url"
],
"properties": {
"url": {
"type": "string",
"description": "https URL of the release install manifest, on an allowed host"
}
}
}
}
}
}
}
},
"/api/v1/admin/modules/restart": {
"post": {
"tags": [
"Admin · Modules"
],
"summary": "Restart the server process so module changes take effect",
"description": "Runs the same graceful shutdown a SIGTERM does. The process is brought back by the supervisor, which the shipped docker-compose.yml provides; a bare `npm start` will not come back.",
"responses": {
"202": {
"description": "Shutting down",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"restarting": {
"type": "boolean"
}
}
}
}
}
}
},
"security": [
{
"cookieAuth": []
},
{
"bearerAuth": []
}
]
}
},
"/api/v1/admin/modules/sources": {
"put": {
"tags": [
"Admin · Modules"
],
"summary": "Replace the allowlist of hosts modules may be installed from",
"description": "",
"responses": {
"200": {
"description": "The new allowlist",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"sourceHosts": {
"type": "array",
"items": {
"type": "string"
}
}
}
}
}
}
},
"400": {
"description": "One of the entries is not a hostname",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
}
}
}
},
"500": {
"description": "Internal Server Error"
}
},
"security": [
{
"cookieAuth": []
},
{
"bearerAuth": []
}
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"type": "object",
"required": [
"hosts"
],
"properties": {
"hosts": {
"type": "string",
"description": "Comma- or space-separated hostnames. An empty list forbids all installs."
}
}
}
}
}
}
}
},
"/api/v1/admin/modules/{id}": {
"delete": {
"tags": [
"Admin · Modules"
],
"summary": "Uninstall a module, optionally deleting its data too",
"description": "Removes the module directory and leaves its row disabled. With purge=true it also runs purge.sql first — which is the only moment it can, since purge.sql lives inside the directory being deleted.",
"parameters": [
{
"name": "id",
"in": "path",
"required": true,
"schema": {
"type": "string"
},
"description": "Module id."
},
{
"name": "purge",
"in": "query",
"required": false,
"schema": {
"type": "boolean"
},
"description": "Also run the module’s purge.sql and drop its row. Destructive and irreversible."
}
],
"responses": {
"200": {
"description": "Uninstalled — restart to unmount it",
"content": {
"application/json": {
"schema": {
"type": "object",
"additionalProperties": true
}
}
}
},
"400": {
"description": "Purge was asked for and the module ships no purge.sql",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
}
}
}
},
"404": {
"description": "No such module",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
}
}
}
},
"500": {
"description": "Internal Server Error"
}
},
"security": [
{
"cookieAuth": []
},
{
"bearerAuth": []
}
]
}
},
"/api/v1/admin/modules/{id}/disable": {
"post": {
"tags": [
"Admin · Modules"
],
"summary": "Stop a module now — runs its onShutdown, then its routes answer 404",
"description": "The only module action that takes effect without a restart. Re-enabling needs one, because there is no onBoot re-dispatch.",
"parameters": [
{
"name": "id",
"in": "path",
"required": true,
"schema": {
"type": "string"
},
"description": "Module id."
}
],
"responses": {
"200": {
"description": "Disabled",
"content": {
"application/json": {
"schema": {
"type": "object",
"additionalProperties": true
}
}
}
},
"400": {
"description": "Bad Request"
},
"404": {
"description": "No such module",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
}
}
}
},
"500": {
"description": "Internal Server Error"
}
},
"security": [
{
"cookieAuth": []
},
{
"bearerAuth": []
}
]
}
},
"/api/v1/admin/modules/{id}/enable": {
"post": {
"tags": [
"Admin · Modules"
],
"summary": "Enable a module (takes effect on the next restart)",
"description": "",
"parameters": [
{
"name": "id",
"in": "path",
"required": true,
"schema": {
"type": "string"
},
"description": "Module id."
}
],
"responses": {
"200": {
"description": "Enabled — restart to start it",
"content": {
"application/json": {
"schema": {
"type": "object",
"additionalProperties": true
}
}
}
},
"400": {
"description": "Bad Request"
},
"404": {
"description": "No such module",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
}
}
}
},
"409": {
"description": "Conflict"
},
"500": {
"description": "Internal Server Error"
}
},
"security": [
{
"cookieAuth": []
},
{
"bearerAuth": []
}
]
}
},
"/api/v1/admin/modules/{id}/purge": {
"post": {
"tags": [
"Admin · Modules"
],
"summary": "Run a disabled module’s purge.sql, dropping its tables and data",
"description": "",
"parameters": [
{
"name": "id",
"in": "path",
"required": true,
"schema": {
"type": "string"
},
"description": "Module id."
}
],
"responses": {
"200": {
"description": "Purged",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"purged": {
"type": "integer"
}
}
}
}
}
},
"400": {
"description": "The module ships no purge.sql",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
}
}
}
},
"404": {
"description": "Not Found"
},
"409": {
"description": "The module must be disabled first",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
}
}
}
},
"500": {
"description": "Internal Server Error"
}
},
"security": [
{
"cookieAuth": []
},
{
"bearerAuth": []
}
]
}
},
"/api/v1/admin/pages": {
"get": {
"tags": [

View File

@@ -0,0 +1,528 @@
// ── Admin · Modules: the delivery surface ──────────────────────────────────
//
// Phase 4, slice 1 of MODULE_SYSTEM.md §2.7.2. The controller is tested directly
// with a mock `res` and stubbed models — the same shape adminUsers.test.js uses —
// because what is interesting here is not the HTTP plumbing but the ORDER of
// operations and which of the three sources of truth answers which question.
//
// Two of these tests exist to pin decisions that are easy to "fix" back into
// being wrong:
//
// - **enable must not touch the loader.** Disable ran the module's onShutdown;
// there is no onBoot re-dispatch, so flipping the record back would put a
// module with closed sockets and cleared timers back on the nav.
// - **purge must run before the directory is removed.** purge.sql lives inside
// that directory. Reorder those two lines and the feature silently stops
// working, with a 200 and no data deleted.
//
// Point the DB at a closed port BEFORE requiring anything that builds the pool.
process.env.DB_HOST = '127.0.0.1'
process.env.DB_PORT = '59999'
const { test, beforeEach, after } = require('node:test')
const assert = require('node:assert/strict')
const ctrl = require('../src/router/v1/admin/modules.controller')
const modules = require('../src/model/modules/modules.model')
const activity = require('../src/model/activity/activity.model')
const settings = require('../src/model/settings/settings.model')
const loader = require('../src/modules/loader')
const lifecycle = require('../src/modules/lifecycle')
const install = require('../src/modules/install')
const declared = require('../src/modules/declared')
const schema = require('../src/modules/schema')
const db = require('../src/utils/db')
after(() => db.close())
function mockRes() {
return {
statusCode: 200,
body: null,
status(c) { this.statusCode = c; return this },
json(b) { this.body = b; return this },
}
}
const req = (extra = {}) => ({
user: { id: 1, username: 'admin' },
params: {},
query: {},
body: {},
...extra,
})
// Everything the controller reaches for, replaced wholesale per test. Restored
// from these originals rather than from a snapshot taken mid-run, so one test
// leaking a stub cannot quietly become another test's fixture.
const originals = {
modules: { ...modules },
activity: { log: activity.log },
settings: { get: settings.get, set: settings.set },
loader: { isLoaded: loader.isLoaded, list: loader.list },
lifecycle: { stop: lifecycle.stop },
install: {
install: install.install,
isInstalled: install.isInstalled,
purgeFile: install.purgeFile,
removeDir: install.removeDir,
},
schema: { runPurge: schema.runPurge },
declared: { state: declared.state },
}
let logged
beforeEach(() => {
Object.assign(modules, originals.modules)
Object.assign(activity, originals.activity)
Object.assign(settings, originals.settings)
Object.assign(loader, originals.loader)
Object.assign(lifecycle, originals.lifecycle)
Object.assign(install, originals.install)
Object.assign(schema, originals.schema)
Object.assign(declared, originals.declared)
logged = []
activity.log = async (entry) => { logged.push(entry) }
settings.get = async () => 'gitea.whitlocktech.com'
loader.isLoaded = () => true
loader.list = () => []
})
// ── list ───────────────────────────────────────────────────────────────────
test('list reconciles the row, the loader and the volume without picking a winner', async () => {
// The case §2.4 creates and decision 3 makes routine: the row says `enabled`
// because the operator just switched it back on, the loader still says
// `disabled` because its onShutdown has run and there is no way back without a
// restart. Rendering either one alone would be a lie.
modules.list = async () => [{
id: 'uo', name: 'UO', version: '1.0.0', state: 'enabled',
failureStage: null, failureReason: null, source: 'https://x/y.json', sha256: 'a'.repeat(64),
installedAt: null, startedAt: null,
}]
loader.list = () => [{ id: 'uo', name: 'UO', version: '1.0.0', state: 'disabled', stage: null, reason: null, capabilities: ['shard'] }]
install.isInstalled = () => true
install.purgeFile = () => '/modules/uo/server/db/purge.sql'
const res = mockRes()
await ctrl.list(req(), res)
const [m] = res.body.modules
assert.equal(m.state, 'enabled', 'what the operator decided')
assert.equal(m.liveState, 'disabled', 'what is actually answering')
assert.equal(m.onVolume, true)
assert.equal(m.canPurge, true)
assert.deepEqual(m.capabilities, ['shard'])
assert.deepEqual(res.body.sourceHosts, ['gitea.whitlocktech.com'])
})
test('list includes a module on the volume that has no row yet', async () => {
// A hand-placed directory before its first boot. §2.5 keeps that a supported
// install, and its routes are already being served — a screen showing nothing
// for it would be showing the wrong thing.
modules.list = async () => []
loader.list = () => [{ id: 'byhand', name: 'By Hand', version: '0.1.0', state: 'started', stage: null, reason: null, capabilities: [] }]
install.isInstalled = () => true
install.purgeFile = () => null
const res = mockRes()
await ctrl.list(req(), res)
assert.equal(res.body.modules.length, 1)
assert.equal(res.body.modules[0].id, 'byhand')
assert.equal(res.body.modules[0].state, null, 'no row means no recorded state, not a guessed one')
assert.equal(res.body.modules[0].liveState, 'started')
})
test('list carries what MODULES declares, including a module it could not install', async () => {
// Slice 3's fourth source. A declared module that failed to resolve has no
// row, no directory and nothing mounted — so it is invisible to the other
// three, and the reason it is missing is the one thing the operator needs.
modules.list = async () => [{
id: 'uo', name: 'UO', version: '0.2.0', state: 'enabled',
failureStage: null, failureReason: null, source: null, sha256: null,
installedAt: null, startedAt: null,
}]
loader.list = () => [{ id: 'uo', name: 'UO', version: '0.2.0', state: 'started', stage: null, reason: null, capabilities: [] }]
install.isInstalled = (id) => id === 'uo'
install.purgeFile = () => null
declared.state = () => [
{ id: 'uo', version: '0.3.0', url: 'https://x/uo.json', action: 'failed', message: 'the host is down' },
{ id: 'market', version: '1.0.0', url: 'https://x/market.json', action: 'failed', message: 'not found' },
]
const res = mockRes()
await ctrl.list(req(), res)
const byId = Object.fromEntries(res.body.modules.map((m) => [m.id, m]))
// Running fine at 0.2.0 while the declared upgrade to 0.3.0 is failing: both
// facts survive, because collapsing them would have to discard one.
assert.equal(byId.uo.liveState, 'started')
assert.equal(byId.uo.declared, true)
assert.equal(byId.uo.declaredVersion, '0.3.0')
assert.equal(byId.uo.declaredError, 'the host is down')
// Declared and nowhere: listed anyway, with no version invented for it.
assert.equal(byId.market.declared, true)
assert.equal(byId.market.version, null)
assert.equal(byId.market.state, null)
assert.equal(byId.market.declaredError, 'not found')
})
test('a successfully resolved module is marked declared, with no error', async () => {
modules.list = async () => []
loader.list = () => [{ id: 'uo', name: 'UO', version: '0.3.0', state: 'started', stage: null, reason: null, capabilities: [] }]
install.isInstalled = () => true
install.purgeFile = () => null
declared.state = () => [{ id: 'uo', version: '0.3.0', url: 'https://x/uo.json', action: 'noop', message: null }]
const res = mockRes()
await ctrl.list(req(), res)
assert.equal(res.body.modules.length, 1, 'declared and present is ONE module, not two')
assert.equal(res.body.modules[0].declared, true)
assert.equal(res.body.modules[0].declaredError, null)
})
test('list survives a process where the loader never scanned', async () => {
loader.isLoaded = () => false
loader.list = () => { throw new Error('modules.list() before modules.load()') }
modules.list = async () => [{ id: 'uo', name: 'UO', version: '1', state: 'disabled' }]
install.isInstalled = () => false
install.purgeFile = () => null
const res = mockRes()
await ctrl.list(req(), res)
assert.equal(res.statusCode, 200)
assert.equal(res.body.modules[0].liveState, null)
})
// ── install ────────────────────────────────────────────────────────────────
test('install records provenance and says a restart is needed', async () => {
const calls = []
install.install = async ({ url, hosts }) => {
calls.push({ url, hosts })
return { id: 'uo', name: 'UO', version: '1.0.0', sha256: 'b'.repeat(64), source: url, replaced: false }
}
modules.recordInstalled = async (row) => { calls.push(row); return { ...row, state: 'installed' } }
const res = mockRes()
await ctrl.create(req({ body: { url: 'https://gitea.whitlocktech.com/x/uo.json' } }), res)
assert.equal(res.statusCode, 201)
assert.equal(res.body.restartRequired, true)
assert.deepEqual(calls[0].hosts, ['gitea.whitlocktech.com'], 'the allowlist comes from the setting')
// Provenance is written HERE and nowhere else — the boot reconcile records a
// module with null source/sha256 and leaves what it is not given.
assert.equal(calls[1].source, 'https://gitea.whitlocktech.com/x/uo.json')
assert.equal(calls[1].sha256, 'b'.repeat(64))
assert.equal(logged[0].action, 'module.install')
})
test('an install refusal is reported to the operator, with its own status', async () => {
install.install = async () => {
const err = new Error('"evil.net" is not an allowed module source host')
err.name = 'InstallError'
err.status = 400
throw err
}
const res = mockRes()
await ctrl.create(req({ body: { url: 'https://evil.net/x.json' } }), res)
assert.equal(res.statusCode, 400)
// The message is the useful part: the operator pasted a URL and needs to know
// what was wrong with what came back.
assert.match(res.body.message, /not an allowed module source host/)
assert.equal(logged.length, 0, 'a refused install is not an audit-log entry')
})
test('an unreachable host is a 502, not a 400', async () => {
install.install = async () => {
const err = new Error('could not reach x: timeout')
err.name = 'InstallError'
err.status = 502
throw err
}
const res = mockRes()
await ctrl.create(req({ body: { url: 'https://gitea.whitlocktech.com/x.json' } }), res)
assert.equal(res.statusCode, 502)
})
test('an unexpected failure is a 500 and does not leak its message', async () => {
install.install = async () => { throw new Error('ENOENT /some/internal/path') }
const res = mockRes()
await ctrl.create(req({ body: { url: 'https://gitea.whitlocktech.com/x.json' } }), res)
assert.equal(res.statusCode, 500)
assert.equal(res.body.message, 'Internal Server Error')
})
// ── enable / disable ───────────────────────────────────────────────────────
test('enable moves the row and does NOT touch the loader', async () => {
// The decision-3 invariant. Re-enabling cannot restart a module: its
// onShutdown has run, and MODULE_API.md has never promised onBoot is safe to
// run twice. Flipping the record would put it back on the nav with a
// torn-down world behind it.
let setStateCalled = false
loader.setState = () => { setStateCalled = true }
modules.enable = async (id) => ({ id, state: 'enabled' })
const res = mockRes()
await ctrl.enable(req({ params: { id: 'uo' } }), res)
assert.equal(res.body.module.state, 'enabled')
assert.equal(res.body.restartRequired, true)
assert.equal(setStateCalled, false, 'enable must not move the in-memory record')
assert.equal(logged[0].action, 'module.enable')
loader.setState = originals.loader.setState
})
test('enabling a module with no row is a 404', async () => {
modules.enable = async () => null
const res = mockRes()
await ctrl.enable(req({ params: { id: 'ghost' } }), res)
assert.equal(res.statusCode, 404)
})
test('an illegal transition is a 409, not a 500', async () => {
modules.enable = async () => {
const err = new Error("module 'uo': cannot move from 'x' to 'enabled'")
err.name = 'ModuleStateError'
throw err
}
const res = mockRes()
await ctrl.enable(req({ params: { id: 'uo' } }), res)
assert.equal(res.statusCode, 409)
})
test('disable stops the module and reports whether the hook ran', async () => {
const calls = []
modules.get = async (id) => ({ id, state: 'started' })
lifecycle.stop = async (id) => { calls.push(id); return { stopped: true, error: null } }
const res = mockRes()
await ctrl.disable(req({ params: { id: 'uo' } }), res)
assert.deepEqual(calls, ['uo'])
assert.equal(res.body.stopped, true)
// No restart: this is the one action that takes effect immediately, and it is
// the one an operator reaches for when something is going wrong.
assert.equal(res.body.restartRequired, undefined)
assert.equal(logged[0].action, 'module.disable')
})
test('a shutdown hook that failed is reported rather than swallowed', async () => {
modules.get = async (id) => ({ id, state: 'started' })
lifecycle.stop = async () => ({ stopped: false, error: 'socket would not close' })
const res = mockRes()
await ctrl.disable(req({ params: { id: 'uo' } }), res)
// It IS disabled either way; the operator should be told it did not close
// cleanly while they still have the logs in front of them.
assert.equal(res.statusCode, 200)
assert.match(res.body.shutdownError, /socket would not close/)
})
// ── uninstall and purge ────────────────────────────────────────────────────
test('uninstall purges BEFORE it removes the directory', async () => {
// The ordering that makes decision 5 work at all: purge.sql is a file inside
// the directory being deleted. Swap these two and the endpoint still answers
// 200 and deletes nothing.
const order = []
modules.get = async (id) => ({ id, state: 'started' })
install.isInstalled = () => true
install.purgeFile = () => '/modules/uo/server/db/purge.sql'
schema.runPurge = async () => { order.push('purge'); return 12 }
lifecycle.stop = async () => { order.push('stop'); return { stopped: true, error: null } }
install.removeDir = async () => { order.push('removeDir'); return true }
modules.remove = async () => { order.push('removeRow') }
const res = mockRes()
await ctrl.remove(req({ params: { id: 'uo' }, query: { purge: 'true' } }), res)
assert.deepEqual(order, ['purge', 'stop', 'removeDir', 'removeRow'])
assert.equal(res.body.purged, 12)
assert.equal(res.body.restartRequired, true)
assert.equal(logged[0].action, 'module.purge')
})
test('a plain uninstall keeps the row and does not purge', async () => {
const order = []
modules.get = async (id) => ({ id, state: 'started' })
install.isInstalled = () => true
schema.runPurge = async () => { order.push('purge'); return 1 }
lifecycle.stop = async () => { order.push('stop'); return { stopped: true, error: null } }
install.removeDir = async () => { order.push('removeDir'); return true }
modules.remove = async () => { order.push('removeRow') }
const res = mockRes()
await ctrl.remove(req({ params: { id: 'uo' } }), res)
// §2.5's default: the directory goes, the data stays, and the disabled row is
// what keeps the retained data visible and the module reinstallable.
assert.deepEqual(order, ['stop', 'removeDir'])
assert.equal(res.body.purged, null)
assert.equal(logged[0].action, 'module.uninstall')
})
test('asking to purge a module that ships no purge.sql refuses instead of pretending', async () => {
modules.get = async (id) => ({ id, state: 'started' })
install.isInstalled = () => true
install.purgeFile = () => null
let removed = false
install.removeDir = async () => { removed = true; return true }
const res = mockRes()
await ctrl.remove(req({ params: { id: 'uo' }, query: { purge: 'true' } }), res)
assert.equal(res.statusCode, 400)
assert.match(res.body.message, /ships no purge.sql/)
// Nothing happened. The operator asked for the module AND its data to go; the
// data cannot go, so doing half of it silently would be the worst answer.
assert.equal(removed, false)
})
test('uninstalling something that is neither on the volume nor in a row is a 404', async () => {
modules.get = async () => null
install.isInstalled = () => false
const res = mockRes()
await ctrl.remove(req({ params: { id: 'ghost' } }), res)
assert.equal(res.statusCode, 404)
})
test('standalone purge refuses while the module is still running', async () => {
// Dropping the tables under a module that is still serving leaves it answering
// out of a world that no longer exists. Disabling first is one click.
modules.get = async (id) => ({ id, state: 'started' })
let ran = false
schema.runPurge = async () => { ran = true; return 1 }
const res = mockRes()
await ctrl.purge(req({ params: { id: 'uo' } }), res)
assert.equal(res.statusCode, 409)
assert.match(res.body.message, /Disable this module before purging/)
assert.equal(ran, false)
})
test('standalone purge runs on a disabled module', async () => {
modules.get = async (id) => ({ id, state: 'disabled' })
install.purgeFile = () => '/modules/uo/server/db/purge.sql'
schema.runPurge = async () => 7
const res = mockRes()
await ctrl.purge(req({ params: { id: 'uo' } }), res)
assert.equal(res.body.purged, 7)
assert.equal(logged[0].action, 'module.purge')
})
// ── the allowlist ──────────────────────────────────────────────────────────
test('setSources stores a normalised list and audits the change', async () => {
let stored = null
settings.set = async (key, value) => { stored = { key, value } }
const res = mockRes()
await ctrl.setSources(req({ body: { hosts: 'Gitea.Example.com, releases.example.org' } }), res)
assert.deepEqual(res.body.sourceHosts, ['gitea.example.com', 'releases.example.org'])
assert.equal(stored.key, ctrl.HOSTS_KEY)
assert.equal(stored.value, 'gitea.example.com,releases.example.org')
// Before AND after: this setting decides what code the site will execute, so
// the audit entry has to say what it used to be.
assert.equal(logged[0].action, 'module.sources')
assert.deepEqual(logged[0].detail.before, ['gitea.whitlocktech.com'])
})
test('setSources refuses anything that is not a bare hostname', async () => {
let stored = false
settings.set = async () => { stored = true }
for (const bad of ['https://x.com', 'x.com/path', 'x.com:8443', '*.x.com', 'x_y.com']) {
const res = mockRes()
// eslint-disable-next-line no-await-in-loop
await ctrl.setSources(req({ body: { hosts: bad } }), res)
assert.equal(res.statusCode, 400, `${bad} should be refused`)
}
assert.equal(stored, false)
})
test('an empty allowlist is storable, and means no installs', async () => {
// Not a wildcard, and not an error: "nothing may be installed" is a position
// an operator is entitled to take.
let stored = null
settings.set = async (key, value) => { stored = value }
const res = mockRes()
await ctrl.setSources(req({ body: { hosts: '' } }), res)
assert.equal(res.statusCode, 200)
assert.deepEqual(res.body.sourceHosts, [])
assert.equal(stored, '')
})
// ── restart ────────────────────────────────────────────────────────────────
// These listen for the SIGTERM EVENT rather than stubbing `process.kill`, and
// that is the whole point of them now.
//
// The first version of this route called `process.kill(process.pid, 'SIGTERM')`
// and the first version of these tests stubbed `process.kill` and asserted it
// had been called with SIGTERM. Both passed. Both were wrong: Windows has no
// POSIX signals, and Node documents SIGTERM there as unconditional termination —
// so on a Windows host the route killed the server outright, with no module
// `onShutdown`, no pool close and no log flush. A stub of `process.kill` cannot
// see that, because what it asserts is precisely the call whose MEANING differs
// by platform.
//
// Asserting on the event closes the gap: it is what server.js's handler is
// actually subscribed to, so a test passing here means the handler would run.
function onceSigterm() {
return new Promise((resolve, reject) => {
const timer = setTimeout(() => {
process.removeListener('SIGTERM', handler)
reject(new Error('no SIGTERM was emitted within 1s'))
}, 1000)
function handler() {
clearTimeout(timer)
process.removeListener('SIGTERM', handler)
resolve(true)
}
process.on('SIGTERM', handler)
})
}
test('restart answers first, then triggers the one graceful-shutdown path', async () => {
const fired = onceSigterm()
const res = mockRes()
ctrl.restart(req(), res)
// Answered synchronously: once the shutdown starts there is no listener left
// to flush a response through, so the operator would be told nothing.
assert.equal(res.statusCode, 202)
assert.equal(res.body.restarting, true)
assert.equal(await fired, true)
assert.equal(logged[0].action, 'module.restart')
})
test('a failure to write the audit entry does not cancel the restart', async () => {
activity.log = async () => { throw new Error('database is gone') }
const fired = onceSigterm()
ctrl.restart(req(), mockRes())
assert.equal(await fired, true)
})

View File

@@ -0,0 +1,82 @@
// server.js's boot ORDER, which is a contract and not a style choice.
//
// Phase 4, slice 3 of MODULE_SYSTEM.md §2.7.2. Five steps have to happen in one
// order, and each arrow is a dependency that is invisible at the call site:
//
// core schema → the declared set (§2.7.2 decision 4) needs the settings row
// its seed writes, to know which hosts it may install from
// declared set → requiring app.js SCANS the volume (§1.12), so anything put
// there afterwards is not in this process
// require app → module schema fragments (§2.6) can only be replayed once the
// loader knows which modules there are
// fragments → onBoot runs against tables that exist
//
// This is a source-structure test, and it is worth being plain about what that
// does and does not prove: it cannot tell you the server boots, only that nobody
// has quietly moved one of these five lines past another. It exists because the
// defect it guards against has already happened once and was invisible to every
// other kind of test here. Deferring the app require — which slice 3 had to do —
// moved core's schema ahead of the scan, and `ensureSchema` then skipped the
// module fragments entirely. On this machine's dev database the tables already
// existed, so the module started; on a FRESH database it would have started
// against no tables at all. Every suite in this directory stubs either the
// loader or the pool, so none of them could see it. The browser smoke did, from
// one log line.
process.env.DB_HOST = '127.0.0.1'
process.env.DB_PORT = '59999'
const fs = require('fs')
const path = require('path')
const { test } = require('node:test')
const assert = require('node:assert/strict')
const SERVER = fs.readFileSync(path.join(__dirname, '..', 'src', 'server.js'), 'utf8')
/** Where a marker appears, asserted to appear exactly once. */
function at(marker) {
const first = SERVER.indexOf(marker)
assert.notEqual(first, -1, `server.js no longer contains ${JSON.stringify(marker)}`)
assert.equal(
SERVER.indexOf(marker, first + 1),
-1,
`${JSON.stringify(marker)} appears more than once in server.js — this test cannot tell which is the boot step`,
)
return first
}
test('boot runs core schema, then the declared set, then the scan, then fragments, then onBoot', () => {
const steps = [
['core schema', at('await ensureSchema({ replayModules: false })')],
['declared module set', at('await declaredModules.resolve(')],
['the volume scan', at("require('./app')")],
['module schema fragments', at('replayFragments()')],
['module onBoot', at('await moduleLifecycle.boot()')],
]
for (let i = 1; i < steps.length; i += 1) {
assert.ok(
steps[i][1] > steps[i - 1][1],
`${steps[i][0]} must come after ${steps[i - 1][0]} in server.js`,
)
}
})
test('app.js is required inside start(), never at the top of the file', () => {
// The whole mechanism depends on this. A top-level require runs when server.js
// is loaded — before a single line of start() — and the scan would then happen
// before the declared set had a chance to put anything on the volume, silently
// and with no error anywhere.
assert.ok(
at("require('./app')") > at('async function start()'),
'requiring ./app at the top of server.js scans the volume before the declared set is resolved',
)
})
test('ensureSchema is asked NOT to replay fragments, and something else does', () => {
// Both halves matter. Passing the flag without replaying elsewhere is the
// original defect with an explicit spelling; replaying without the flag runs
// the fragments twice, the second time being the one that matters.
assert.match(SERVER, /ensureSchema\(\{ replayModules: false \}\)/)
assert.match(SERVER, /modules\/schema'\)\.replayFragments\(\)/)
})

View File

@@ -60,7 +60,13 @@ test('backoffGuard returns a generic 429 while locked out', async () => {
a.post('/login', lp.backoffGuard, (req, res) => res.json({ ok: true }))
})
try {
lp.recordFailure('203.0.113.40') // lock the test client IP
// Lock the test client IP. FIVE failures, not one: the lock is
// `BASE_MS * 2 ** (count - 1)`, so a single failure locks for exactly one
// second and this test then races the round trip. It lost that race on CI
// (200 instead of 429, request arriving 1,456 ms after the lock). Five
// failures lock for sixteen seconds, which is not a race. What is under
// test is the guard's ANSWER while locked out, and that is unchanged.
for (let i = 0; i < 5; i += 1) lp.recordFailure('203.0.113.40')
const res = await fetch(`${app.url}/login`, {
method: 'POST',
headers: { 'X-Forwarded-For': '203.0.113.40' },

View File

@@ -0,0 +1,252 @@
// modules/archive.js — the hardened bundle extractor (MODULE_SYSTEM.md §2.7.2).
//
// Every rejection case is exercised against a REAL archive rather than a mocked
// tar parser, because the thing being tested is what the parser reports for a
// given sequence of bytes. A fake that returns `{type: 'SymbolicLink'}` proves
// only that the `if` is spelled correctly.
//
// The hostile archives are written here as raw ustar headers instead of being
// produced with `tar`, for two reasons that both bit during this slice:
//
// 1. `ln -s` needs a privilege Windows does not hand out by default, so a
// symlink fixture built with the shell is a fixture that silently is not
// one, and the test passes for the wrong reason on the machine most of this
// work happens on.
// 2. GNU tar will not emit `../escape` or `/etc/passwd` as a member name — it
// strips them and tells you so. The archives worth defending against are
// exactly the ones a cooperative archiver refuses to produce.
//
// A ustar header is 512 bytes of fixed-offset fields, so writing one is less
// code than persuading a tool to misbehave.
const test = require('node:test')
const assert = require('node:assert/strict')
const fs = require('fs')
const os = require('os')
const path = require('path')
const zlib = require('zlib')
const archive = require('../src/modules/archive')
// ── A minimal ustar writer ─────────────────────────────────────────────────
const BLOCK = 512
function octal(value, width) {
// ustar numeric fields are NUL-terminated octal, right-aligned with zeros.
return Number(value).toString(8).padStart(width - 1, '0') + '\0'
}
/**
* One 512-byte header plus its padded data.
*
* @param {object} entry
* @param {string} entry.name member path
* @param {string} [entry.type] '0' file · '5' dir · '2' symlink · '1' hardlink
* · '3' char dev · '6' FIFO
* @param {string} [entry.linkname] target, for the link types
* @param {string} [entry.body] file contents
* @param {number} [entry.size] declared size — defaults to the body's, and
* may be set independently to build a header
* that lies about its payload
*/
function member({ name, type = '0', linkname = '', body = '', size = null }) {
const header = Buffer.alloc(BLOCK, 0)
const data = Buffer.from(body, 'utf8')
const declared = size === null ? data.length : size
header.write(name, 0, 100, 'utf8')
header.write(octal(0o644, 8), 100, 8, 'ascii') // mode
header.write(octal(0, 8), 108, 8, 'ascii') // uid
header.write(octal(0, 8), 116, 8, 'ascii') // gid
header.write(octal(declared, 12), 124, 12, 'ascii')
header.write(octal(0, 12), 136, 12, 'ascii') // mtime
header.write(' ', 148, 8, 'ascii') // checksum field is spaces while summing
header.write(type, 156, 1, 'ascii')
header.write(linkname, 157, 100, 'utf8')
header.write('ustar\0', 257, 6, 'ascii')
header.write('00', 263, 2, 'ascii')
let sum = 0
for (const byte of header) sum += byte
header.write(`${sum.toString(8).padStart(6, '0')}\0 `, 148, 8, 'ascii')
const padding = Buffer.alloc((BLOCK - (data.length % BLOCK)) % BLOCK, 0)
return Buffer.concat([header, data, padding])
}
/** Gzip a set of members into a .tar.gz on disk, and return its path. */
function writeArchive(dir, filename, members) {
const tarball = Buffer.concat([
...members.map(member),
Buffer.alloc(BLOCK * 2, 0), // two zero blocks end the archive
])
const file = path.join(dir, filename)
fs.writeFileSync(file, zlib.gzipSync(tarball))
return file
}
/** A well-formed bundle: one top-level directory, ordinary files inside it. */
function goodMembers(root = 'module-uo-1.0.0') {
return [
{ name: `${root}/`, type: '5' },
{ name: `${root}/module.json`, body: '{"id":"uo","name":"UO","version":"1.0.0"}' },
{ name: `${root}/server/`, type: '5' },
{ name: `${root}/server/index.js`, body: 'module.exports = () => {}\n' },
]
}
// One scratch directory for the whole file, removed at the end.
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'rg-archive-'))
test.after(() => fs.rmSync(tmp, { recursive: true, force: true }))
/** Assert that inspecting `members` fails, and that the message says why. */
async function rejects(name, members, matcher) {
const file = writeArchive(tmp, `${name}.tar.gz`, members)
await assert.rejects(
() => archive.inspect(file),
(err) => {
assert.equal(err.name, 'ArchiveError', `expected an ArchiveError, got ${err.name}: ${err.message}`)
assert.match(err.message, matcher)
return true
},
)
}
// ── What a good bundle does ────────────────────────────────────────────────
test('inspect accepts a well-formed bundle and reports its single root', async () => {
const file = writeArchive(tmp, 'good.tar.gz', goodMembers())
const stats = await archive.inspect(file)
assert.equal(stats.root, 'module-uo-1.0.0')
assert.equal(stats.entries, 4)
assert.ok(stats.bytes > 0)
})
test('extract strips the top level, so the bundle lands as the module id', async () => {
// The wrapper directory is the publisher's naming (module-uo's release
// workflow packs `module-uo-<version>/`); the directory it lands in is core's,
// and has to be the id the loader scans for.
const file = writeArchive(tmp, 'strip.tar.gz', goodMembers())
const dest = path.join(tmp, 'unpacked-uo')
await archive.unpack(file, dest)
assert.ok(fs.existsSync(path.join(dest, 'module.json')), 'module.json should be at the root of the destination')
assert.ok(fs.existsSync(path.join(dest, 'server', 'index.js')))
assert.ok(!fs.existsSync(path.join(dest, 'module-uo-1.0.0')), 'the wrapper directory should not survive')
})
test('extract refuses a destination that already exists', async () => {
const file = writeArchive(tmp, 'exists.tar.gz', goodMembers())
const dest = path.join(tmp, 'already-there')
fs.mkdirSync(dest)
await assert.rejects(() => archive.extract(file, dest), /already exists/)
})
// ── What a hostile bundle does ─────────────────────────────────────────────
test('an absolute member path is refused', async () => {
await rejects('absolute', [
{ name: 'mod/', type: '5' },
{ name: '/etc/cron.d/pwned', body: '* * * * * root sh\n' },
], /absolute/)
})
test('a drive-absolute member path is refused', async () => {
// Its own node-tar advisory, and invisible to a leading-slash check.
await rejects('drive', [
{ name: 'mod/', type: '5' },
{ name: 'C:\\Windows\\Temp\\pwned', body: 'x' },
], /backslash|drive-absolute/)
})
test('an upward-escaping member path is refused', async () => {
await rejects('escape', [
{ name: 'mod/', type: '5' },
{ name: 'mod/../../../etc/passwd', body: 'root::0:0\n' },
], /escapes upward/)
})
test('a symlink member is refused, and the message names the type', async () => {
await rejects('symlink', [
{ name: 'mod/', type: '5' },
{ name: 'mod/passwd', type: '2', linkname: '/etc/passwd' },
], /SymbolicLink/)
})
test('a hardlink member is refused', async () => {
// The single most-published node-tar escape primitive. Refusing the type
// outright is what keeps this file from depending on the library getting
// hardlink containment right.
await rejects('hardlink', [
{ name: 'mod/', type: '5' },
{ name: 'mod/shadow', type: '1', linkname: '../../../etc/shadow' },
], /Link/)
})
test('a device node is refused', async () => {
await rejects('device', [
{ name: 'mod/', type: '5' },
{ name: 'mod/zero', type: '3', linkname: '' },
], /may only contain files and directories/)
})
test('a FIFO is refused', async () => {
await rejects('fifo', [
{ name: 'mod/', type: '5' },
{ name: 'mod/pipe', type: '6' },
], /may only contain files and directories/)
})
test('two top-level directories are refused', async () => {
await rejects('two-roots', [
{ name: 'mod-a/', type: '5' },
{ name: 'mod-a/module.json', body: '{}' },
{ name: 'mod-b/', type: '5' },
{ name: 'mod-b/module.json', body: '{}' },
], /exactly one top-level directory, found 2/)
})
test('a bundle whose declared sizes exceed the cap is refused before it is read to the end', async () => {
// The header lies: it declares a gigabyte and carries nothing. That is the
// decompression-bomb shape, and the point is that inspect() decides on the
// DECLARED size without ever materialising the payload.
await rejects('bomb', [
{ name: 'mod/', type: '5' },
{ name: 'mod/big', size: archive.MAX_BYTES + 1, body: '' },
], /unpacks to more than/)
})
test('an empty archive is refused', async () => {
await rejects('empty', [], /empty/)
})
// ── The path check on its own ──────────────────────────────────────────────
//
// pathProblem is exported so the cases that are awkward to express as archive
// bytes can still be asserted directly.
test('pathProblem accepts ordinary bundle paths', () => {
for (const ok of ['mod/module.json', 'mod/server/router/x.js', 'mod/a.b-c_d/e.js']) {
assert.equal(archive.pathProblem(ok), null, `${ok} should be accepted`)
}
})
test('pathProblem rejects the escape shapes', () => {
assert.match(archive.pathProblem('/etc/passwd'), /absolute/)
assert.match(archive.pathProblem('C:/Windows/x'), /drive-absolute/)
assert.match(archive.pathProblem('mod/../../x'), /escapes upward/)
assert.match(archive.pathProblem('..'), /escapes upward/)
assert.match(archive.pathProblem('mod\\x'), /backslash/)
assert.match(archive.pathProblem('mod/\0/x'), /NUL byte/)
})
test('pathProblem does not reject a filename that merely contains two dots', () => {
// `..` is a SEGMENT, not a substring — a file called `version..js` is fine,
// and a check written with `includes('..')` would refuse it.
assert.equal(archive.pathProblem('mod/version..js'), null)
assert.equal(archive.pathProblem('mod/..hidden'), null)
})

View File

@@ -0,0 +1,391 @@
// modules/declared.js — resolving the module set an environment declares.
//
// Phase 4, slice 3 of MODULE_SYSTEM.md §2.7.2, decision 4. Two claims are worth
// more than the rest and both are about what does NOT happen:
//
// - a module already unpacked at the declared version does not touch the
// network, because that is what makes a restart with the internet down come
// up unchanged;
// - a module that cannot be resolved does not fail the boot, and does not stop
// the next declaration from resolving.
//
// The last test in the file runs the whole path through the real install.js
// against a fake transport — a real gzipped tar, really hashed, really unpacked
// — because everything above it stubs `installImpl` and would keep passing if
// the two files stopped agreeing about what an install returns.
process.env.DB_HOST = '127.0.0.1'
process.env.DB_PORT = '59999'
const crypto = require('crypto')
const fs = require('fs')
const os = require('os')
const path = require('path')
const zlib = require('zlib')
const { test, beforeEach, after } = require('node:test')
const assert = require('node:assert/strict')
const db = require('../src/utils/db')
after(() => db.close())
const HOSTS = ['releases.example.com']
const URL_030 = 'https://releases.example.com/mod/uo-0.3.0.json'
let tmpRoot
let declared
/**
* A fresh declared.js bound to a fresh modules directory.
*
* loader.js resolves MODULES_DIR once at require time and install.js reads
* `loader.dir()`, so all three have to come back together — the same dance
* moduleInstall.test.js and moduleLifecycle.test.js do.
*/
function fresh(dir) {
process.env.MODULES_DIR = dir
for (const m of ['loader', 'install', 'declared']) {
delete require.cache[require.resolve(`../src/modules/${m}`)]
}
// eslint-disable-next-line global-require
return require('../src/modules/declared')
}
/** Put a module directory on the volume, as an unpack would leave it. */
function place(id, version) {
const dir = path.join(tmpRoot, id)
fs.mkdirSync(dir, { recursive: true })
fs.writeFileSync(
path.join(dir, 'module.json'),
JSON.stringify({ id, name: 'Ultima Online', version, coreApi: '^1.0.0' }),
)
return dir
}
/**
* A stub install service.
*
* Records every call, so "did this reach the network at all" is a question the
* tests can ask directly rather than inferring from an outcome.
*/
function fakeInstall({ version = '0.3.0', throws = null } = {}) {
const calls = []
return {
calls,
InstallError: Error,
async install(args) {
calls.push(args)
if (throws) throw new Error(throws)
return {
id: 'uo',
name: 'Ultima Online',
version,
sha256: 'a'.repeat(64),
source: args.url,
bytes: 1024,
replaced: false,
}
},
}
}
/** A modules.model stand-in that remembers what provenance it was given. */
function fakeModel() {
const recorded = []
return { recorded, async recordInstalled(row) { recorded.push(row); return row } }
}
beforeEach(() => {
tmpRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'rg-declared-'))
declared = fresh(tmpRoot)
})
// ── Parsing ────────────────────────────────────────────────────────────────
test('parses id@version=url entries separated by whitespace or commas', () => {
const { entries, errors } = declared.parse(
` uo@0.3.0=https://a.example/uo.json,\n market@1.2.3=https://a.example/market.json `,
)
assert.deepEqual(errors, [])
assert.deepEqual(entries, [
{ id: 'uo', version: '0.3.0', url: 'https://a.example/uo.json' },
{ id: 'market', version: '1.2.3', url: 'https://a.example/market.json' },
])
})
test('an empty or unset declaration parses to nothing, without complaint', () => {
for (const value of [undefined, '', ' ']) {
const { entries, errors } = declared.parse(value)
assert.deepEqual(entries, [])
assert.deepEqual(errors, [])
}
})
test('a malformed entry is refused on its own, leaving the others', () => {
// A bare URL, a missing version, and an id that is not one — each of which an
// operator can plausibly type, and none of which should cost the module that
// was written correctly.
const { entries, errors } = declared.parse(
'https://a.example/uo.json uo@=https://a.example/uo.json Uo@1=https://a.example/uo.json'
+ ' good@1.0.0=https://a.example/good.json',
)
assert.equal(entries.length, 1)
assert.equal(entries[0].id, 'good')
assert.equal(errors.length, 3)
})
test('a duplicate id keeps the first and says so', () => {
const { entries, errors } = declared.parse(
'uo@1.0.0=https://a.example/one.json uo@2.0.0=https://a.example/two.json',
)
assert.equal(entries.length, 1)
assert.equal(entries[0].version, '1.0.0')
assert.match(errors[0], /declared more than once/)
})
// ── Resolution ─────────────────────────────────────────────────────────────
test('a module already at the declared version is a no-op that never fetches', async () => {
place('uo', '0.3.0')
const installer = fakeInstall()
const model = fakeModel()
const results = await declared.resolve({
value: `uo@0.3.0=${URL_030}`,
hosts: HOSTS,
model,
installImpl: installer,
})
assert.deepEqual(results.map((r) => r.action), ['noop'])
// The offline guarantee, stated as an assertion: nothing was fetched and
// nothing was written. A restart with no route to the internet is this case.
assert.equal(installer.calls.length, 0)
assert.equal(model.recorded.length, 0)
})
test('a missing module is installed, with its provenance recorded', async () => {
const installer = fakeInstall()
const model = fakeModel()
const results = await declared.resolve({
value: `uo@0.3.0=${URL_030}`,
hosts: HOSTS,
model,
installImpl: installer,
})
assert.deepEqual(results.map((r) => r.action), ['installed'])
assert.equal(installer.calls[0].url, URL_030)
assert.deepEqual(installer.calls[0].hosts, HOSTS)
// The declaration is handed down so install.js can refuse a URL that turns out
// to be another module or another version BEFORE it downloads it.
assert.deepEqual(installer.calls[0].expect, { id: 'uo', version: '0.3.0' })
// Written exactly as the admin route writes it — the reason this runs in the
// server process rather than in a script that cannot reach the database.
assert.deepEqual(model.recorded, [
{ id: 'uo', name: 'Ultima Online', version: '0.3.0', source: URL_030, sha256: 'a'.repeat(64) },
])
})
test('a module at a different version is re-resolved', async () => {
place('uo', '0.2.0')
const installer = fakeInstall()
const results = await declared.resolve({
value: `uo@0.3.0=${URL_030}`,
hosts: HOSTS,
model: fakeModel(),
installImpl: installer,
})
assert.deepEqual(results.map((r) => r.action), ['installed'])
assert.equal(installer.calls.length, 1)
})
test('a directory with no readable module.json counts as absent', async () => {
fs.mkdirSync(path.join(tmpRoot, 'uo'), { recursive: true })
fs.writeFileSync(path.join(tmpRoot, 'uo', 'module.json'), '{ this is not json')
const installer = fakeInstall()
await declared.resolve({
value: `uo@0.3.0=${URL_030}`,
hosts: HOSTS,
model: fakeModel(),
installImpl: installer,
})
// Whatever is in that directory, it is not the declared module — so the
// declared module is fetched rather than assumed to be there.
assert.equal(installer.calls.length, 1)
})
test('a failure is carried, not thrown, and does not stop the next module', async () => {
const model = fakeModel()
const installer = {
calls: [],
async install(args) {
installer.calls.push(args)
if (args.expect.id === 'uo') throw new Error('could not reach releases.example.com')
return {
id: args.expect.id,
name: 'Market',
version: args.expect.version,
sha256: 'b'.repeat(64),
source: args.url,
bytes: 10,
replaced: false,
}
},
}
const results = await declared.resolve({
value: `uo@0.3.0=${URL_030} market@1.0.0=https://releases.example.com/market.json`,
hosts: HOSTS,
model,
installImpl: installer,
})
assert.deepEqual(results.map((r) => r.action), ['failed', 'installed'])
assert.match(results[0].message, /could not reach/)
// The second module still installed: one unreachable release host costs that
// module, not the deployment.
assert.equal(model.recorded.length, 1)
assert.equal(model.recorded[0].id, 'market')
})
test('a failed resolution leaves whatever was already on the volume', async () => {
place('uo', '0.2.0')
const installer = fakeInstall({ throws: 'the host is down' })
const results = await declared.resolve({
value: `uo@0.3.0=${URL_030}`,
hosts: HOSTS,
model: fakeModel(),
installImpl: installer,
})
assert.equal(results[0].action, 'failed')
// The site comes up serving the version it already had rather than not at all.
assert.equal(declared.installedVersion('uo'), '0.2.0')
})
test('resolution without a model records nothing and still installs', async () => {
// `npm run seed` and the test harness both reach code paths with no model to
// hand; the volume half must not depend on the database half.
const installer = fakeInstall()
const results = await declared.resolve({ value: `uo@0.3.0=${URL_030}`, hosts: HOSTS, installImpl: installer })
assert.deepEqual(results.map((r) => r.action), ['installed'])
})
test('state() reports the last resolution and is replaced by the next', async () => {
const installer = fakeInstall()
await declared.resolve({ value: `uo@0.3.0=${URL_030}`, hosts: HOSTS, installImpl: installer })
assert.equal(declared.state().length, 1)
assert.equal(declared.state()[0].id, 'uo')
await declared.resolve({ value: '', hosts: HOSTS, installImpl: installer })
assert.deepEqual(declared.state(), [])
})
// ── The whole path, once, for real ─────────────────────────────────────────
const BLOCK = 512
function octal(value, width) {
return Number(value).toString(8).padStart(width - 1, '0') + '\0'
}
function member({ name, type = '0', body = '' }) {
const header = Buffer.alloc(BLOCK, 0)
const data = Buffer.from(body, 'utf8')
header.write(name, 0, 100, 'utf8')
header.write(octal(0o644, 8), 100, 8, 'ascii')
header.write(octal(0, 8), 108, 8, 'ascii')
header.write(octal(0, 8), 116, 8, 'ascii')
header.write(octal(data.length, 12), 124, 12, 'ascii')
header.write(octal(0, 12), 136, 12, 'ascii')
header.write(' ', 148, 8, 'ascii')
header.write(type, 156, 1, 'ascii')
header.write('ustar\0', 257, 6, 'ascii')
header.write('00', 263, 2, 'ascii')
let sum = 0
for (const byte of header) sum += byte
header.write(`${sum.toString(8).padStart(6, '0')}\0 `, 148, 8, 'ascii')
const padding = Buffer.alloc((BLOCK - (data.length % BLOCK)) % BLOCK, 0)
return Buffer.concat([header, data, padding])
}
/** A bundle tarball, named the way module-uo's release workflow names one. */
function bundle(version) {
const root = `module-uo-${version}`
return zlib.gzipSync(Buffer.concat([
member({ name: `${root}/`, type: '5' }),
member({
name: `${root}/module.json`,
body: JSON.stringify({ id: 'uo', name: 'Ultima Online', version, coreApi: '^1.0.0', server: 'server/index.js' }),
}),
member({ name: `${root}/server/`, type: '5' }),
member({ name: `${root}/server/index.js`, body: 'module.exports = () => {}\n' }),
Buffer.alloc(BLOCK * 2, 0),
]))
}
test('end to end: a declaration installs a real bundle through the real install path', async (t) => {
const tarball = bundle('0.3.0')
const artifactUrl = 'https://releases.example.com/mod/uo-0.3.0.tar.gz'
const manifest = JSON.stringify({
schema: 1,
id: 'uo',
name: 'Ultima Online',
version: '0.3.0',
coreApi: '^1.0.0',
artifact: 'uo-0.3.0.tar.gz',
url: artifactUrl,
sha256: crypto.createHash('sha256').update(tarball).digest('hex'),
size: tarball.length,
})
const routes = { [URL_030]: manifest, [artifactUrl]: tarball }
const fetched = []
const fetchImpl = async (url) => {
fetched.push(String(url))
const body = routes[String(url)]
return body === undefined ? new Response('nope', { status: 404 }) : new Response(body, { status: 200 })
}
// The real install.js, with only the transport replaced — same seam the
// install tests use, and the same reason: what is under test is the decisions,
// not Node's TLS.
// eslint-disable-next-line global-require
const install = require('../src/modules/install')
const installImpl = { install: (args) => install.install({ ...args, fetchImpl }) }
const model = fakeModel()
const first = await declared.resolve({
value: `uo@0.3.0=${URL_030}`,
hosts: HOSTS,
model,
installImpl,
})
assert.deepEqual(first.map((r) => r.action), ['installed'])
// Unpacked, with the release's top-level directory stripped, under the id.
assert.equal(
JSON.parse(fs.readFileSync(path.join(tmpRoot, 'uo', 'module.json'), 'utf8')).version,
'0.3.0',
)
assert.equal(model.recorded[0].sha256, JSON.parse(manifest).sha256)
// And again, which is what every restart after the first one is.
const before = fetched.length
const second = await declared.resolve({ value: `uo@0.3.0=${URL_030}`, hosts: HOSTS, model, installImpl })
assert.deepEqual(second.map((r) => r.action), ['noop'])
assert.equal(fetched.length, before, 'the second resolution must not fetch anything')
// A declaration whose URL resolves to a different version is refused BEFORE
// the artifact is downloaded — the module on the volume is left alone.
const wrong = await declared.resolve({ value: `uo@0.4.0=${URL_030}`, hosts: HOSTS, model, installImpl })
assert.equal(wrong[0].action, 'failed')
assert.match(wrong[0].message, /v0\.4\.0 was asked for/)
assert.equal(declared.installedVersion('uo'), '0.3.0')
t.diagnostic(`fetched ${fetched.length} URL(s) across three resolutions`)
})

View File

@@ -0,0 +1,448 @@
// modules/install.js — fetching, verifying and placing a module bundle.
//
// Phase 4, slice 1 of MODULE_SYSTEM.md §2.7.2. The rules under test are all
// refusals, and each one is a step an attacker would otherwise walk through:
// a non-https URL, a host that is not allowed, a REDIRECT to a host that is not
// allowed, a body larger than declared, a hash that does not match, an archive
// that does not agree with the manifest about what it is.
//
// `fetchImpl` is injected rather than a TLS server being stood up, for the same
// reason `replayFragments` takes a `query`: what is being tested is what this
// file decides about a response, and a real server would mostly test Node's
// certificate handling. The responses below are real `Response` objects with
// real bodies, so the streaming, the hashing and the byte cap are exercised for
// real — only the transport is stubbed.
process.env.DB_HOST = '127.0.0.1'
process.env.DB_PORT = '59999'
const crypto = require('crypto')
const fs = require('fs')
const os = require('os')
const path = require('path')
const zlib = require('zlib')
const { test, beforeEach, after } = require('node:test')
const assert = require('node:assert/strict')
const db = require('../src/utils/db')
after(() => db.close())
const HOSTS = ['releases.example.com']
const MANIFEST_URL = 'https://releases.example.com/mod/uo-1.0.0.json'
let tmpRoot
let install
/**
* A fresh install.js bound to a fresh modules directory.
*
* install.js reads `loader.dir()`, which is resolved once at require time from
* MODULES_DIR — so both have to be re-required per test, exactly as
* moduleLifecycle.test.js does.
*/
function freshInstall(dir) {
process.env.MODULES_DIR = dir
delete require.cache[require.resolve('../src/modules/loader')]
delete require.cache[require.resolve('../src/modules/install')]
// eslint-disable-next-line global-require
return require('../src/modules/install')
}
// ── Building a bundle to serve ─────────────────────────────────────────────
const BLOCK = 512
function octal(value, width) {
return Number(value).toString(8).padStart(width - 1, '0') + '\0'
}
function member({ name, type = '0', body = '' }) {
const header = Buffer.alloc(BLOCK, 0)
const data = Buffer.from(body, 'utf8')
header.write(name, 0, 100, 'utf8')
header.write(octal(0o644, 8), 100, 8, 'ascii')
header.write(octal(0, 8), 108, 8, 'ascii')
header.write(octal(0, 8), 116, 8, 'ascii')
header.write(octal(data.length, 12), 124, 12, 'ascii')
header.write(octal(0, 12), 136, 12, 'ascii')
header.write(' ', 148, 8, 'ascii')
header.write(type, 156, 1, 'ascii')
header.write('ustar\0', 257, 6, 'ascii')
header.write('00', 263, 2, 'ascii')
let sum = 0
for (const byte of header) sum += byte
header.write(`${sum.toString(8).padStart(6, '0')}\0 `, 148, 8, 'ascii')
const padding = Buffer.alloc((BLOCK - (data.length % BLOCK)) % BLOCK, 0)
return Buffer.concat([header, data, padding])
}
/** A bundle tarball whose root directory is named the way a release names it. */
function bundle({ id = 'uo', version = '1.0.0', extra = [] } = {}) {
const root = `module-${id}-${version}`
return zlib.gzipSync(Buffer.concat([
member({ name: `${root}/`, type: '5' }),
member({
name: `${root}/module.json`,
body: JSON.stringify({ id, name: 'Ultima Online', version, coreApi: '^1.0.0', server: 'server/index.js' }),
}),
member({ name: `${root}/server/`, type: '5' }),
member({ name: `${root}/server/index.js`, body: 'module.exports = () => {}\n' }),
...extra.map(member),
Buffer.alloc(BLOCK * 2, 0),
]))
}
const sha256 = (buf) => crypto.createHash('sha256').update(buf).digest('hex')
/**
* A fake transport serving one manifest and one artifact.
*
* `routes` maps an absolute URL to either a Buffer/string body or
* `{ status, location }` for a redirect, so a test can describe exactly what the
* remote host does without describing how it does it.
*/
function fakeFetch(routes) {
const seen = []
const impl = async (url) => {
const href = String(url)
seen.push(href)
const route = routes[href]
if (route === undefined) return new Response('not found', { status: 404 })
if (route && route.status) {
return new Response(null, {
status: route.status,
headers: route.location ? { location: route.location } : {},
})
}
return new Response(route, { status: 200 })
}
impl.seen = seen
return impl
}
/** The manifest a release publishes, with whatever a test wants to change. */
function manifestFor(tarball, overrides = {}) {
return JSON.stringify({
schema: 1,
id: 'uo',
name: 'Ultima Online',
version: '1.0.0',
coreApi: '^1.0.0',
artifact: 'uo-1.0.0.tar.gz',
url: 'https://releases.example.com/mod/uo-1.0.0.tar.gz',
sha256: sha256(tarball),
size: tarball.length,
...overrides,
})
}
/** The happy-path pair: a valid manifest and the artifact it describes. */
function goodRoutes(options = {}) {
const tarball = bundle(options)
return {
tarball,
routes: {
[MANIFEST_URL]: manifestFor(tarball, options.manifest || {}),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
},
}
}
beforeEach(() => {
tmpRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'rg-install-'))
install = freshInstall(tmpRoot)
})
// ── The allowlist ──────────────────────────────────────────────────────────
test('parseHosts accepts comma and whitespace separation, and lower-cases', () => {
assert.deepEqual(install.parseHosts('a.com, B.com\nc.com'), ['a.com', 'b.com', 'c.com'])
assert.deepEqual(install.parseHosts(''), [])
assert.deepEqual(install.parseHosts(null), [])
})
test('an empty allowlist forbids everything rather than allowing everything', () => {
// The direction matters: a setting someone blanks by accident must stop
// installs, not open the door to every host on the internet.
assert.throws(() => install.checkUrl('https://releases.example.com/x.json', []), /no module source hosts are allowed/)
})
test('only https may be installed from', () => {
// The sha256 is no help against a plaintext fetch: whoever can rewrite the
// artifact in flight can rewrite the manifest that declares its hash.
assert.throws(() => install.checkUrl('http://releases.example.com/x.json', HOSTS), /only https/)
assert.throws(() => install.checkUrl('file:///etc/passwd', HOSTS), /only https/)
})
test('a host that is not on the allowlist is refused, and the message says which are', () => {
assert.throws(
() => install.checkUrl('https://evil.example.net/x.json', HOSTS),
/"evil.example.net" is not an allowed module source host \(allowed: releases.example.com\)/,
)
})
test('the allowlist is matched on the host, not on a substring of the URL', () => {
// `https://evil.com/?x=releases.example.com` must not pass, and neither must a
// subdomain nobody listed.
assert.throws(() => install.checkUrl('https://evil.com/?releases.example.com', HOSTS), /not an allowed/)
assert.throws(() => install.checkUrl('https://sub.releases.example.com/x', HOSTS), /not an allowed/)
assert.throws(() => install.checkUrl('https://releases.example.com.evil.net/x', HOSTS), /not an allowed/)
})
// ── Redirects ──────────────────────────────────────────────────────────────
test('a redirect is followed, and re-checked against the allowlist', async () => {
const { tarball } = goodRoutes()
const impl = fakeFetch({
[MANIFEST_URL]: { status: 302, location: 'https://releases.example.com/cdn/uo-1.0.0.json' },
'https://releases.example.com/cdn/uo-1.0.0.json': manifestFor(tarball),
})
const manifest = await install.fetchManifest(MANIFEST_URL, HOSTS, impl)
assert.equal(manifest.id, 'uo')
assert.deepEqual(impl.seen, [MANIFEST_URL, 'https://releases.example.com/cdn/uo-1.0.0.json'])
})
test('a redirect to a host that is not allowed is refused', async () => {
// The hole the allowlist exists to close. `fetch`'s own redirect following
// would check the first hop and then go wherever it was pointed — which is
// why get() follows them by hand.
const impl = fakeFetch({
[MANIFEST_URL]: { status: 302, location: 'http://169.254.169.254/latest/meta-data/' },
})
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /only https/)
})
test('a redirect loop is bounded rather than followed forever', async () => {
const impl = fakeFetch({
[MANIFEST_URL]: { status: 302, location: MANIFEST_URL },
})
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /too many redirects/)
})
// ── The install manifest ───────────────────────────────────────────────────
test('a manifest that is not JSON is refused with a readable reason', async () => {
const impl = fakeFetch({ [MANIFEST_URL]: 'this is not json' })
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /not valid JSON/)
})
test('a manifest with an invalid module id is refused', async () => {
// The loader would refuse to scan a directory called `../etc`, but the point
// is that one is never created: the id becomes a path segment.
for (const id of ['../etc', 'UO', '', 'x', 'a'.repeat(40)]) {
const impl = fakeFetch({ [MANIFEST_URL]: JSON.stringify({ id, name: 'n', version: '1', sha256: 'a'.repeat(64) }) })
// eslint-disable-next-line no-await-in-loop
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /invalid module id/, `id ${JSON.stringify(id)}`)
}
})
test('a manifest with no usable sha256 is refused', async () => {
const impl = fakeFetch({
[MANIFEST_URL]: JSON.stringify({ id: 'uo', name: 'n', version: '1.0.0', sha256: 'not-a-hash' }),
})
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /no valid sha256/)
})
test('the artifact URL is resolved relative to the manifest when it carries no absolute one', async () => {
const { tarball } = goodRoutes()
const impl = fakeFetch({ [MANIFEST_URL]: manifestFor(tarball, { url: undefined }) })
const manifest = await install.fetchManifest(MANIFEST_URL, HOSTS, impl)
// Release assets sit beside their manifest, so a manifest that travelled
// without its absolute URL still resolves to the right place.
assert.equal(manifest.artifactUrl, 'https://releases.example.com/mod/uo-1.0.0.tar.gz')
})
// ── The install ────────────────────────────────────────────────────────────
test('a good bundle installs into modules/<id>, stripped of its wrapper', async () => {
const { routes, tarball } = goodRoutes()
const result = await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) })
assert.equal(result.id, 'uo')
assert.equal(result.version, '1.0.0')
assert.equal(result.sha256, sha256(tarball))
assert.equal(result.source, MANIFEST_URL)
assert.equal(result.replaced, false)
// The directory is named for the module id, not for the archive's root — the
// loader scans for the former and the publisher chose the latter.
const dir = path.join(tmpRoot, 'uo')
assert.ok(fs.existsSync(path.join(dir, 'module.json')))
assert.ok(fs.existsSync(path.join(dir, 'server', 'index.js')))
assert.ok(!fs.existsSync(path.join(tmpRoot, 'module-uo-1.0.0')))
})
test('a hash that does not match refuses the install and writes nothing', async () => {
const { routes } = goodRoutes()
// The bytes are fine; the manifest lies about them. Which is the same thing an
// artifact swapped after publication looks like.
routes[MANIFEST_URL] = manifestFor(Buffer.from('different'), {})
await assert.rejects(
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
/does not match the sha256/,
)
assert.deepEqual(fs.readdirSync(tmpRoot), [], 'nothing may be left on the volume')
})
test('a bundle whose module.json disagrees with the manifest is refused', async () => {
// A manifest promising `uo` and delivering something else would otherwise be
// installed into `modules/uo/` under a name it is not.
const tarball = bundle({ id: 'rust', version: '1.0.0' })
const routes = {
[MANIFEST_URL]: manifestFor(tarball),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
}
await assert.rejects(
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
/declares module id "rust" but the install manifest promised "uo"/,
)
assert.deepEqual(fs.readdirSync(tmpRoot), [])
})
test('a bundle whose version disagrees with the manifest is refused', async () => {
const tarball = bundle({ id: 'uo', version: '9.9.9' })
const routes = {
[MANIFEST_URL]: manifestFor(tarball),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
}
await assert.rejects(
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
/declares version "9.9.9"/,
)
})
test('an artifact whose length differs from the declared size is refused', async () => {
const tarball = bundle()
const routes = {
[MANIFEST_URL]: manifestFor(tarball, { size: tarball.length + 10 }),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
}
await assert.rejects(
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
/but the manifest declares/,
)
})
test('a hostile archive is refused, and nothing of it reaches the volume', async () => {
// The case node-tar alone does NOT cover: it throws on the escaping member,
// but only once it reaches it — the members before it are already on disk.
// archive.inspect() decides before extract() runs, and the unpack happens in a
// scratch directory that is removed either way.
const tarball = bundle({
extra: [
{ name: 'module-uo-1.0.0/../../ESCAPED.txt', body: 'escaped' },
],
})
const routes = {
[MANIFEST_URL]: manifestFor(tarball),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
}
await assert.rejects(
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
/escapes upward/,
)
assert.deepEqual(fs.readdirSync(tmpRoot), [], 'no scratch directory, no partial module, no escape')
})
test('an upgrade replaces the directory and reports that it did', async () => {
const first = goodRoutes()
await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(first.routes) })
fs.writeFileSync(path.join(tmpRoot, 'uo', 'STALE.txt'), 'from the old version')
const second = goodRoutes({ version: '2.0.0', manifest: { version: '2.0.0' } })
second.routes[MANIFEST_URL] = manifestFor(second.tarball, { version: '2.0.0' })
const result = await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(second.routes) })
assert.equal(result.replaced, true)
assert.equal(result.version, '2.0.0')
// A replace, not a merge: a file the previous version left behind must not
// survive into the new one, or an upgrade quietly keeps dead code loadable.
assert.ok(!fs.existsSync(path.join(tmpRoot, 'uo', 'STALE.txt')))
assert.deepEqual(fs.readdirSync(tmpRoot), ['uo'], 'the aside copy is cleaned up')
})
test('a failed upgrade leaves the previous version in place', async () => {
const first = goodRoutes()
await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(first.routes) })
const badTarball = bundle({ id: 'uo', version: '2.0.0', extra: [{ name: 'evil/', type: '5' }] })
await assert.rejects(() => install.install({
url: MANIFEST_URL,
hosts: HOSTS,
fetchImpl: fakeFetch({
[MANIFEST_URL]: manifestFor(badTarball, { version: '2.0.0' }),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': badTarball,
}),
}))
// The installed module is untouched: the swap is the last step, so a bundle
// rejected before it never got near the live directory.
assert.ok(fs.existsSync(path.join(tmpRoot, 'uo', 'module.json')))
assert.equal(JSON.parse(fs.readFileSync(path.join(tmpRoot, 'uo', 'module.json'), 'utf8')).version, '1.0.0')
assert.deepEqual(fs.readdirSync(tmpRoot), ['uo'])
})
// ── The volume ─────────────────────────────────────────────────────────────
test('moduleDir refuses an id that is not one', () => {
for (const id of ['../escape', 'a/b', '', 'UO', '.']) {
assert.throws(() => install.moduleDir(id), /invalid module id/, `id ${JSON.stringify(id)}`)
}
assert.equal(install.moduleDir('uo'), path.join(tmpRoot, 'uo'))
})
test('isInstalled and removeDir report what they did', async () => {
const { routes } = goodRoutes()
assert.equal(install.isInstalled('uo'), false)
await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) })
assert.equal(install.isInstalled('uo'), true)
assert.equal(await install.removeDir('uo'), true)
assert.equal(install.isInstalled('uo'), false)
// Removing what is not there is not an error — an uninstall of a module whose
// directory was already deleted by hand should still tidy up the row.
assert.equal(await install.removeDir('uo'), false)
})
test('purgeFile resolves the manifest declaration, and refuses one that escapes', async () => {
const dir = path.join(tmpRoot, 'uo')
fs.mkdirSync(path.join(dir, 'server', 'db'), { recursive: true })
fs.writeFileSync(path.join(dir, 'server', 'db', 'purge.sql'), 'DROP TABLE IF EXISTS x;')
const write = (purge) => fs.writeFileSync(
path.join(dir, 'module.json'),
JSON.stringify({ id: 'uo', name: 'UO', version: '1.0.0', purge }),
)
write('server/db/purge.sql')
assert.equal(install.purgeFile('uo'), path.join(dir, 'server', 'db', 'purge.sql'))
// The same containment rule the loader applies to client.entry: a manifest may
// not point core at a file outside the module it belongs to.
write('../../../../etc/passwd')
assert.equal(install.purgeFile('uo'), null)
// Declared but absent, and not declared at all, are both "nothing to run".
write('server/db/missing.sql')
assert.equal(install.purgeFile('uo'), null)
write(undefined)
assert.equal(install.purgeFile('uo'), null)
})
test('purgeFile is null for a module that is not on the volume', () => {
assert.equal(install.purgeFile('nothere'), null)
})

View File

@@ -125,6 +125,11 @@ function fakeModel(seed = []) {
if (!row || row.state === 'disabled') return
Object.assign(row, { state: 'startup_failed', failureStage: stage, failureReason: reason })
},
async disable(id) {
calls.push(`disable:${id}`)
const row = rows.get(id)
if (row) Object.assign(row, { state: 'disabled', failureStage: null, failureReason: null })
},
}
return model
}
@@ -374,3 +379,151 @@ test('a hook that throws does not stop the ones behind it', async () => {
await assert.doesNotReject(() => lifecycle.shutdown({ modules: loader }))
assert.deepEqual(noted(file), ['shutdown:zzz', 'shutdown:aaa'])
})
// ── stop(): the admin panel's Disable ──────────────────────────────────────
//
// Phase 4, §2.7.2 decision 3. Phase 2's disable moved a record and left the
// module running; the whole point of these tests is that it no longer does.
test('stop runs that one module\'s onShutdown and leaves the others alone', async () => {
const file = path.join(tmpRoot, 'log.txt')
writeModule('aaa', { boot: '', shutdown: '', log: file })
writeModule('bbb', { boot: '', shutdown: '', log: file })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
fs.writeFileSync(file, '')
const result = await lifecycle.stop('aaa', { modules: loader, model })
assert.deepEqual(result, { stopped: true, error: null })
// Only aaa. This is the difference from shutdown(), which runs everything.
assert.deepEqual(noted(file), ['shutdown:aaa'])
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
assert.equal(stateOf(loader, 'bbb').state, 'started', 'the other module keeps running')
assert.equal(model.rows.get('aaa').state, 'disabled')
})
test('stop moves the record only after the hook has run', async () => {
// While onShutdown runs, the module is still `started` — the only state in
// which its routes and the world it is tearing down agree with each other. The
// module reports its own view of itself, so a record moved too early shows up
// here as `disabled` instead of `started`.
const file = path.join(tmpRoot, 'log.txt')
const dir = path.join(tmpRoot, 'aaa')
fs.mkdirSync(dir, { recursive: true })
fs.writeFileSync(path.join(dir, 'module.json'), JSON.stringify({
id: 'aaa', name: 'A', version: '1.0.0', coreApi: '^1.0.0', server: 'index.js',
}))
fs.writeFileSync(path.join(dir, 'index.js'), `
module.exports = (ctx, api) => {
api.onBoot(async () => {})
api.onShutdown(async () => {
const loader = require(${JSON.stringify(require.resolve('../src/modules/loader'))})
const me = loader.list().find((m) => m.id === 'aaa')
require('fs').appendFileSync(${JSON.stringify(file)}, 'state-during-hook:' + me.state + ${JSON.stringify('\n')})
})
}`)
const loader = freshLoader(tmpRoot)
await lifecycle.boot({ modules: loader, model: fakeModel() })
await lifecycle.stop('aaa', { modules: loader, model: fakeModel([{ id: 'aaa', state: 'started' }]) })
assert.deepEqual(noted(file), ['state-during-hook:started'])
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
})
test('a hook that throws does not prevent the disable', async () => {
// The opposite of the boot path's rule, deliberately. There, a failure means
// the module never became safe to use; here, the operator has asked for it to
// stop answering and a module that could not close cleanly is a reason to log
// loudly, not a reason to leave it serving.
writeModule('aaa', { boot: '', shutdown: 'throw new Error("socket stuck")' })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
const result = await lifecycle.stop('aaa', { modules: loader, model })
assert.equal(result.stopped, false)
assert.match(result.error, /socket stuck/)
assert.equal(stateOf(loader, 'aaa').state, 'disabled', 'disabled anyway')
assert.equal(model.rows.get('aaa').state, 'disabled')
})
test('a hook that hangs costs its budget, and the module is still disabled', async () => {
writeModule('aaa', { boot: '', shutdown: 'await new Promise(() => {})' })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
const started = Date.now()
const result = await lifecycle.stop('aaa', { modules: loader, model, budgetMs: 50 })
assert.equal(result.stopped, false)
assert.match(result.error, /budget/)
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
assert.ok(Date.now() - started < 2000)
})
test('stopping a module with no onShutdown still disables it', async () => {
// A hookless module has nothing to run and must still stop answering, or the
// guard and the row disagree about what is serving.
writeModule('aaa', { boot: '' })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
const result = await lifecycle.stop('aaa', { modules: loader, model })
assert.deepEqual(result, { stopped: false, error: null })
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
assert.equal(model.rows.get('aaa').state, 'disabled')
})
test('stopping a module whose onBoot failed does not run its onShutdown', async () => {
// Same rule shutdownHooks() applies: a module that never finished warming up
// has a half-built world its onShutdown was not written for. It is still
// disabled — it just is not asked to tear anything down.
const file = path.join(tmpRoot, 'log.txt')
writeModule('aaa', { boot: 'throw new Error("no")', shutdown: '', log: file })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
const before = noted(file)
const result = await lifecycle.stop('aaa', { modules: loader, model })
assert.equal(result.stopped, false)
// Compared against what was already there rather than against an empty file:
// `noted` splits, so a blanked file reads as [''] and an emptiness assertion
// would pass for the wrong reason.
assert.deepEqual(noted(file), before, 'no new hook output')
assert.ok(!noted(file).includes('shutdown:aaa'))
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
})
test('stopping an unknown id writes the row and does not throw', async () => {
// A row can exist for a module that is not on the volume, and disabling it is
// exactly what an operator would do about that.
writeModule('aaa', { boot: '' })
const loader = freshLoader(tmpRoot)
const model = fakeModel([{ id: 'ghost', state: 'startup_failed' }])
await assert.doesNotReject(() => lifecycle.stop('ghost', { modules: loader, model }))
assert.equal(model.rows.get('ghost').state, 'disabled')
})
test('a row that will not write does not stop the module from being disabled', async () => {
// The same rule the boot path follows: bookkeeping failure is not the
// operation failing. The guard is what stops traffic, and it has already moved.
writeModule('aaa', { boot: '', shutdown: '' })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
model.disable = async () => { throw new Error('database is gone') }
await assert.doesNotReject(() => lifecycle.stop('aaa', { modules: loader, model }))
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
})

View File

@@ -25,7 +25,7 @@ const assert = require('node:assert/strict')
const express = require('express')
const db = require('../src/utils/db')
const { replayFragments } = require('../src/modules/schema')
const { replayFragments, runPurge } = require('../src/modules/schema')
const { splitStatements } = require('../src/utils/sqlStatements')
const { startApp } = require('./_helper')
@@ -245,3 +245,66 @@ test('replay is skipped, not thrown, when no scan happened in this process', asy
assert.deepEqual(rec.ran, [])
assert.equal(loader.isLoaded(), false)
})
// ── runPurge: the destructive twin ─────────────────────────────────────────
//
// Phase 4, slice 1. Same splitter, same pool, same serial execution as the
// replay above — pointed the other way. The two differences are deliberate and
// are what these tests are for.
test('purge runs every statement in the file, in order', async () => {
const file = path.join(tmpRoot, 'purge.sql')
fs.writeFileSync(file, [
'-- drop everything this module owns',
'DROP TABLE IF EXISTS mod_b;',
'DROP TABLE IF EXISTS mod_a;',
"DELETE FROM settings WHERE `key` = 'mod_thing';",
].join('\n'))
const rec = recorder()
const ran = await runPurge(file, { query: rec.query })
assert.equal(ran, 3)
assert.match(rec.ran[0], /DROP TABLE IF EXISTS mod_b/)
assert.match(rec.ran[2], /DELETE FROM settings/)
})
test('purge is not held to the fragment allowlist — DROP is the point of it', async () => {
// §2.6's leading-verb allowlist exists because a fragment replays on every
// boot. purge.sql never does, which is exactly why it is the one file a module
// may put a DROP in.
const file = path.join(tmpRoot, 'purge.sql')
fs.writeFileSync(file, 'DROP TABLE IF EXISTS mod_a;\nTRUNCATE TABLE mod_b;')
const rec = recorder()
assert.equal(await runPurge(file, { query: rec.query }), 2)
})
test('purge THROWS on failure, unlike the replay', async () => {
// A replay failure is one module failing to start, which the site survives by
// 503ing it. A purge failure is an operator's explicit destructive request not
// having happened — reporting success would leave them believing data is gone
// when it is not.
const file = path.join(tmpRoot, 'purge.sql')
fs.writeFileSync(file, 'DROP TABLE IF EXISTS mod_a;\nDROP TABLE mod_missing;')
const query = async (sql) => {
if (sql.includes('mod_missing')) throw new Error("Unknown table 'mod_missing'")
}
await assert.rejects(
() => runPurge(file, { query }),
// The message names WHICH statement stopped it, because a purge is not
// transactional — the ones before it have already committed and the operator
// needs to know where it got to.
/purge failed at statement 2 of 2: Unknown table 'mod_missing'/,
)
})
test('an empty purge file runs nothing and does not throw', async () => {
const file = path.join(tmpRoot, 'purge.sql')
fs.writeFileSync(file, '-- nothing to drop yet\n')
const rec = recorder()
assert.equal(await runPurge(file, { query: rec.query }), 0)
assert.deepEqual(rec.ran, [])
})

View File

@@ -38,7 +38,17 @@ modulesDb.getOne = async (id) => (rows.has(id) ? [rows.get(id)] : [])
modulesDb.upsert = async ({ id, name, version, source, sha256 }) => {
const existing = rows.get(id)
if (existing) {
Object.assign(existing, { name, version, source: source ?? null, sha256: sha256 ?? null })
// COALESCE, matching the real statement: a value overwrites, a NULL leaves
// what is there. This fake used to assign unconditionally — faithfully
// reproducing the defect it was supposed to be able to catch, which is why
// the boot refresh nulling an install's provenance survived until a browser
// showed it.
Object.assign(existing, {
name,
version,
source: source ?? existing.source ?? null,
sha256: sha256 ?? existing.sha256 ?? null,
})
return { affectedRows: 1 }
}
rows.set(id, {
@@ -116,6 +126,46 @@ test('a hand-placed module records with no provenance', async () => {
assert.equal(mod.sha256, null)
})
test('the boot refresh does not wipe an install\'s provenance', async () => {
// The defect a browser found in Phase 4, and the reason the upsert COALESCEs.
//
// lifecycle.boot() re-records every scanned module with NO source and NO
// sha256, because a hand-placed directory genuinely has neither. That refresh
// used to write both columns as NULL, so an admin-panel install's provenance
// survived exactly until the restart the install asked for — and the screen
// then described a module installed from a URL as "placed on the volume by
// hand". Nothing could see it until Phase 4: no caller had ever passed a
// non-null value.
await modules.recordInstalled({
id: 'uo',
name: 'Ultima Online',
version: '1.0.0',
source: 'https://gitea.example/x/uo-1.0.0.json',
sha256: 'b'.repeat(64),
})
// What the next boot does.
const after = await modules.recordInstalled({ id: 'uo', name: 'Ultima Online', version: '1.0.0' })
assert.equal(after.source, 'https://gitea.example/x/uo-1.0.0.json')
assert.equal(after.sha256, 'b'.repeat(64))
})
test('a re-install from a new URL does replace the provenance', async () => {
// The other half: COALESCE must not make the columns write-once, or an upgrade
// would for ever show where the FIRST version came from.
await modules.recordInstalled({
id: 'uo', name: 'Ultima Online', version: '1.0.0', source: 'https://old/x.json', sha256: 'c'.repeat(64),
})
const after = await modules.recordInstalled({
id: 'uo', name: 'Ultima Online', version: '2.0.0', source: 'https://new/y.json', sha256: 'd'.repeat(64),
})
assert.equal(after.source, 'https://new/y.json')
assert.equal(after.sha256, 'd'.repeat(64))
assert.equal(after.version, '2.0.0')
})
test('recordInstalled refuses a manifest missing id, name or version', async () => {
await assert.rejects(
() => modules.recordInstalled({ id: 'uo', version: '1.0.0' }),