Compare commits
7 Commits
2cb549e9e5
...
feature/mo
| Author | SHA1 | Date | |
|---|---|---|---|
| 92c6972344 | |||
| 9b16f39a52 | |||
| 75f4d29e93 | |||
| 9083e4135a | |||
| 732927a6bb | |||
| 50d719cf46 | |||
| b30e82cde2 |
14
.env.example
14
.env.example
@@ -113,6 +113,20 @@ BOT_INTERNAL_KEY=change-me-to-a-long-random-string
|
||||
# knows nothing about any of them. A module may read its own env vars, and they
|
||||
# belong here because Compose passes this file to the container.
|
||||
#
|
||||
# MODULES declares the set this deployment runs, and the container arrives at it
|
||||
# on its own — no admin panel, no `tar -xf` on the host. One entry per module,
|
||||
# `<id>@<version>=<install manifest URL>`, whitespace- or comma-separated:
|
||||
#
|
||||
# MODULES=uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json
|
||||
#
|
||||
# A module already unpacked at the declared version is a no-op that never touches
|
||||
# the network, so a restart with the internet down brings the site up exactly as
|
||||
# it was; only a missing or different version is fetched, verified against the
|
||||
# sha256 its manifest declares, and unpacked. A failure is logged and shown in
|
||||
# Admin → Modules, and the site starts anyway. The variable owns what is on the
|
||||
# volume, not what runs — a module disabled from the admin panel stays disabled.
|
||||
# Leave it unset to install from the admin panel instead.
|
||||
#
|
||||
# RunicGateway/Module-uo, for example, reads UOLINK_BASE_URL / UOLINK_WS_URL /
|
||||
# UOLINK_PROTOCOL as the defaults for its connection to a uo-link sidecar, and
|
||||
# TOWNCRIER_DURATION_SEC for its news leg. Its README documents them; they are
|
||||
|
||||
28
README.md
28
README.md
@@ -489,6 +489,32 @@ modules/
|
||||
supported install. The directory is tracked in git (via its README) on purpose: Docker recreates a
|
||||
*missing* bind-mount source as `root:root`, and the container is uid 1000.
|
||||
|
||||
### Three ways in, and none of them is a build
|
||||
|
||||
| | How | Where it fits |
|
||||
|---|---|---|
|
||||
| **Admin panel** | Admin → Modules, paste the URL of a release's install manifest | The click path. Installs, upgrades, disables, uninstalls and purges, with a restart button — no shell on the box |
|
||||
| **`MODULES`** | Declare the set in the environment; the container resolves it at every start | The compose-managed host. The running set is a line in a file you version-control, not the residue of past clicks |
|
||||
| **By hand** | `tar -xf module-uo-0.3.0.tar.gz -C ./modules && mv modules/module-uo-0.3.0 modules/uo`, then restart | Development, and any host where the other two do not fit |
|
||||
|
||||
`MODULES` takes one entry per module, whitespace- or comma-separated:
|
||||
|
||||
```
|
||||
MODULES=uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json
|
||||
```
|
||||
|
||||
The id and the version are written out rather than discovered inside the manifest so that **the
|
||||
no-op case needs no network**: a module already unpacked at the declared version is answered by
|
||||
reading its own `module.json`, so a restart with the internet down brings the site up exactly as it
|
||||
was. Only a missing or different version is fetched, and it goes through the same
|
||||
verify-and-unpack path — allowlisted `https` host, sha256 from the manifest, whole-archive
|
||||
inspection before anything is written — that the admin panel uses. A version that cannot be
|
||||
resolved is logged and shown on the admin screen; **it never stops the site from starting**.
|
||||
|
||||
The declaration owns what is *on the volume*, never what runs. A module disabled from the admin
|
||||
panel gets its files back at the next start and stays disabled, because the row and the variable are
|
||||
answering different questions.
|
||||
|
||||
### What a module gets, and what it may not do
|
||||
|
||||
At boot, `app.js` scans the volume synchronously, validates each `module.json`, and calls the
|
||||
@@ -534,6 +560,8 @@ Copy `.env.example` (Compose) or `server/.env.example` (local) and fill in. **`.
|
||||
| `PORT` | `3000` | server listens on `0.0.0.0:PORT` |
|
||||
| `UPLOAD_DIR` | `<server>/uploads` | where post images are written (`/app/uploads`, volume-mounted, in Compose) |
|
||||
| `MODULES_DIR` | `<repo>/modules` | where installed modules are scanned from (`/app/modules`, bind-mounted, in Compose) |
|
||||
| `MODULES` | — | the module set this deployment runs, resolved at every start: `<id>@<version>=<install manifest URL>`, whitespace/comma separated. Already at the declared version = no network. A failure is logged and shown in Admin → Modules, never fatal. See [Modules](#modules) |
|
||||
| `MODULE_SOURCE_HOSTS` | `gitea.whitlocktech.com` | **bootstrap only** — seeds the `module_source_hosts` setting on first boot; after that the setting is authoritative and is edited in Admin → Modules |
|
||||
| `DB_HOST` / `DB_PORT` | `db` / `3306` | `db` in Compose; `127.0.0.1` for local dev |
|
||||
| `DB_NAME` / `DB_USER` / `DB_PASSWORD` | `runic_gateway` / `runic` / — | app database credentials |
|
||||
| `DB_ROOT_PASSWORD` | — | MariaDB root (Compose only) |
|
||||
|
||||
@@ -41,6 +41,7 @@ import AuthProvidersAdmin from './routes/admin/views/AuthProvidersAdmin.jsx'
|
||||
import UsersAdmin from './routes/admin/views/UsersAdmin.jsx'
|
||||
import UserDetail from './routes/admin/views/UserDetail.jsx'
|
||||
import InvitesAdmin from './routes/admin/views/InvitesAdmin.jsx'
|
||||
import ModulesAdmin from './routes/admin/views/ModulesAdmin.jsx'
|
||||
import AccountAdmin from './routes/admin/views/AccountAdmin.jsx'
|
||||
import Moderation from './routes/admin/views/Moderation.jsx'
|
||||
import ModerationUser from './routes/admin/views/ModerationUser.jsx'
|
||||
@@ -169,6 +170,10 @@ export default function App() {
|
||||
<Route path="users" element={<UsersAdmin />} />
|
||||
<Route path="users/:id" element={<UserDetail />} />
|
||||
<Route path="invites" element={<InvitesAdmin />} />
|
||||
{/* Core's own screen, and it has to be: it is how a module reaches
|
||||
the volume in the first place. Declared here with the rest of
|
||||
core's routes, above the module-supplied ones below. */}
|
||||
<Route path="modules" element={<ModulesAdmin />} />
|
||||
<Route path="account" element={<AccountAdmin />} />
|
||||
{/* Installed modules' admin pages, at /admin/<id>/…, already inside
|
||||
RequireAuth + AdminLayout. A module cannot supply its own auth
|
||||
|
||||
@@ -233,6 +233,20 @@ export const api = {
|
||||
req('/admin/invites', { method: 'POST', body: { email, role, sendEmail } }),
|
||||
revokeInvite: (id) => req(`/admin/invites/${id}`, { method: 'DELETE' }),
|
||||
|
||||
// Installed modules (MODULE_SYSTEM.md §2.7.2). `uninstallModule`'s purge flag
|
||||
// is a query parameter rather than a body because it hangs off a DELETE, and
|
||||
// it is spelled out at the call site rather than defaulted, so the
|
||||
// destructive branch is never the one you get by forgetting an argument.
|
||||
listModules: () => req('/admin/modules'),
|
||||
installModule: (url) => req('/admin/modules', { method: 'POST', body: { url } }),
|
||||
enableModule: (id) => req(`/admin/modules/${encodeURIComponent(id)}/enable`, { method: 'POST' }),
|
||||
disableModule: (id) => req(`/admin/modules/${encodeURIComponent(id)}/disable`, { method: 'POST' }),
|
||||
uninstallModule: (id, { purge } = {}) =>
|
||||
req(`/admin/modules/${encodeURIComponent(id)}${purge ? '?purge=true' : ''}`, { method: 'DELETE' }),
|
||||
purgeModule: (id) => req(`/admin/modules/${encodeURIComponent(id)}/purge`, { method: 'POST' }),
|
||||
setModuleSources: (hosts) => req('/admin/modules/sources', { method: 'PUT', body: { hosts } }),
|
||||
restartServer: () => req('/admin/modules/restart', { method: 'POST' }),
|
||||
|
||||
// ----- moderation dashboard (admin + moderator) -----
|
||||
modSummary: () => req('/admin/moderation/stats/summary'),
|
||||
modRecent: (params = {}) => {
|
||||
|
||||
252
client/src/lib/moduleAdmin.js
Normal file
252
client/src/lib/moduleAdmin.js
Normal file
@@ -0,0 +1,252 @@
|
||||
// What an admin should be told about one installed module, and what they may do
|
||||
// to it — derived, not spelled out at each button.
|
||||
//
|
||||
// Phase 4, slice 2 of docs/website/MODULE_SYSTEM.md §2.7.2. Plain JS rather than
|
||||
// a hook or a chunk of JSX, for the same reason `lib/adminNav.js` is: the test
|
||||
// runner here has no DOM, and this is the part of the Modules screen that is
|
||||
// actually worth testing.
|
||||
//
|
||||
// **The screen has four sources of truth and they are allowed to disagree**
|
||||
// (MODULE_SYSTEM.md §2.4, and slice 3 for the fourth):
|
||||
//
|
||||
// state what the DATABASE row records — what the operator decided, and
|
||||
// what the last boot ended up doing
|
||||
// liveState what the LOADER has mounted in this process and is answering with
|
||||
// onVolume whether there is still a directory there at all
|
||||
// declared what this container's MODULES variable asks for — the only one of
|
||||
// the four that no button on this screen can change
|
||||
//
|
||||
// Picking one and rendering it would be simpler and would lie. The case that
|
||||
// makes this concrete is the one decision 3 creates on purpose: an operator
|
||||
// disables a module (its onShutdown runs, its routes 404) and then enables it
|
||||
// again. The row says `enabled`; the loader still says `disabled`, because
|
||||
// there is no `onBoot` re-dispatch and nothing can start it before a restart.
|
||||
// It is neither running nor off, and the honest thing to show is "enabled —
|
||||
// restart to start it".
|
||||
|
||||
/**
|
||||
* The one-line status of a module, and whether that status is waiting on a
|
||||
* restart.
|
||||
*
|
||||
* Ordering matters here. The checks run most-alarming first, so a module whose
|
||||
* directory has been deleted is described that way rather than by whatever its
|
||||
* row happens to still say.
|
||||
*
|
||||
* @param {object} m a row from GET /admin/modules
|
||||
* @returns {{ label: string, tone: 'ok'|'warn'|'bad'|'idle', pending: boolean, detail: string }}
|
||||
*/
|
||||
export function statusOf(m) {
|
||||
// Declared by the environment and not there at all: no row, no directory,
|
||||
// nothing mounted. Every other branch below reads one of those three, so
|
||||
// without this the screen would describe a module it has never had as though
|
||||
// a row had gone stale — and the one thing the operator needs, the reason
|
||||
// resolution failed, would be nowhere.
|
||||
if (m.declared && !m.onVolume && m.state === null) {
|
||||
return {
|
||||
label: 'Declared, not installed',
|
||||
tone: 'bad',
|
||||
pending: false,
|
||||
detail: m.declaredError
|
||||
? `MODULES asks for v${m.declaredVersion}; the last start could not install it: ${m.declaredError}`
|
||||
: `MODULES asks for v${m.declaredVersion}. It will be installed when the server next starts.`,
|
||||
}
|
||||
}
|
||||
|
||||
// Gone from the volume, but still known. Either a hand-deleted directory (the
|
||||
// boot reconcile marks that `startup_failed`) or an uninstall waiting for its
|
||||
// restart. Both are "there is nothing to run here".
|
||||
if (!m.onVolume) {
|
||||
return {
|
||||
label: m.state === 'disabled' ? 'Uninstalled' : 'Missing from the volume',
|
||||
tone: m.state === 'disabled' ? 'idle' : 'bad',
|
||||
pending: m.liveState !== null,
|
||||
detail: m.state === 'disabled'
|
||||
? 'The files are gone. Its data was kept, and reinstalling brings it back.'
|
||||
: 'A row exists but there is no module directory. Reinstall it, or uninstall to clear the row.',
|
||||
}
|
||||
}
|
||||
|
||||
// **Installed since this process booted**, and this check has to come before
|
||||
// the failure one. `liveState` is the loader's record, and the loader scans
|
||||
// the volume once at require time — so a module that is on the volume NOW and
|
||||
// has no live record was put there after the scan. Anything the row still says
|
||||
// about it therefore predates the install and is stale by definition.
|
||||
//
|
||||
// Found by the §7.7 browser smoke, and no unit test here had modelled it:
|
||||
// installing over a row left `startup_failed` by the previous boot rendered
|
||||
// "Failed at the require stage: module directory not present on the volume"
|
||||
// one second after the file had been written to the volume — and, because that
|
||||
// branch is not pending, suppressed the restart banner the install had just
|
||||
// told the operator to use.
|
||||
if (m.liveState === null) {
|
||||
return {
|
||||
label: 'Restart to start',
|
||||
tone: 'warn',
|
||||
pending: true,
|
||||
detail: 'Installed. It mounts when the server next starts.',
|
||||
}
|
||||
}
|
||||
|
||||
if (m.state === 'startup_failed' || m.liveState === 'startup_failed') {
|
||||
return {
|
||||
label: 'Failed to start',
|
||||
tone: 'bad',
|
||||
pending: false,
|
||||
detail: m.failureReason
|
||||
? `Failed at the ${m.failureStage || 'unknown'} stage: ${m.failureReason}`
|
||||
: 'It failed to start and recorded no reason.',
|
||||
}
|
||||
}
|
||||
|
||||
if (m.state === 'disabled') {
|
||||
return {
|
||||
label: 'Disabled',
|
||||
tone: 'idle',
|
||||
pending: false,
|
||||
detail: 'Stopped and switched off. Its routes answer 404 and it stays off across restarts.',
|
||||
}
|
||||
}
|
||||
|
||||
// The row has been switched on but the loader has not started it — the
|
||||
// decision-3 case: disable ran its onShutdown, and nothing can start it again
|
||||
// before a restart.
|
||||
if (m.liveState !== 'started') {
|
||||
return {
|
||||
label: 'Restart to start',
|
||||
tone: 'warn',
|
||||
pending: true,
|
||||
detail: m.liveState === 'disabled'
|
||||
? 'Enabled, but still stopped in the running server — it cannot be restarted in place.'
|
||||
: 'Enabled. It mounts when the server next starts.',
|
||||
}
|
||||
}
|
||||
|
||||
// Running, but not the version that is installed. An upgrade writes new files
|
||||
// and a new row while the old code stays loaded, so the row's `version` is a
|
||||
// promise about the next boot rather than a description of this one — and
|
||||
// "Running v2.0.0" beside a process serving v1.0.0 is the same lie as the
|
||||
// stale-failure one above, in a different place.
|
||||
if (m.liveVersion && m.liveVersion !== m.version) {
|
||||
return {
|
||||
label: 'Restart to finish upgrading',
|
||||
tone: 'warn',
|
||||
pending: true,
|
||||
detail: `v${m.version} is installed; v${m.liveVersion} is still running.`,
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
label: 'Running',
|
||||
tone: 'ok',
|
||||
pending: false,
|
||||
detail: 'Mounted and serving.',
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* What the environment's declaration means for this module, as one sentence — or
|
||||
* null if nothing declares it.
|
||||
*
|
||||
* Kept out of `statusOf` on purpose. A module can be running perfectly while its
|
||||
* declared upgrade is failing, and collapsing both into one label would have to
|
||||
* pick which of the two is "the" status. This is a second line, beside the first.
|
||||
*
|
||||
* The sentence an operator most needs is the uninstall one: MODULES owns what is
|
||||
* on the volume and the row owns whether it runs, so uninstalling a declared
|
||||
* module puts its files back at the next start and leaves it switched off. Files
|
||||
* reappearing unexplained is exactly the kind of thing that gets debugged for an
|
||||
* afternoon.
|
||||
*
|
||||
* @param {object} m a row from GET /admin/modules
|
||||
* @returns {{ text: string, tone: 'warn'|'idle' }|null}
|
||||
*/
|
||||
export function declarationNoteFor(m) {
|
||||
if (!m.declared) return null
|
||||
|
||||
if (m.declaredError) {
|
||||
return {
|
||||
text: `MODULES asks for v${m.declaredVersion} and the last start could not install it: ${m.declaredError}`,
|
||||
tone: 'warn',
|
||||
}
|
||||
}
|
||||
if (!m.onVolume) {
|
||||
return {
|
||||
text:
|
||||
`MODULES declares v${m.declaredVersion}, so its files come back when the server next starts`
|
||||
+ (m.state === 'disabled' ? ' — switched off, until you enable it.' : '.'),
|
||||
tone: 'warn',
|
||||
}
|
||||
}
|
||||
return { text: `Declared by this deployment's MODULES variable at v${m.declaredVersion}.`, tone: 'idle' }
|
||||
}
|
||||
|
||||
/**
|
||||
* Which actions are offered for a module, and why the others are not.
|
||||
*
|
||||
* Returned as a map of `{ shown, reason }` rather than a list of shown actions,
|
||||
* so a disabled button can say what would make it available. Every rule here
|
||||
* mirrors one the server enforces — this is presentation, never the boundary.
|
||||
*
|
||||
* @param {object} m a row from GET /admin/modules
|
||||
*/
|
||||
export function actionsFor(m) {
|
||||
const running = m.liveState === 'started'
|
||||
const disabled = m.state === 'disabled'
|
||||
|
||||
return {
|
||||
// Only offered while something is actually running: disabling a module that
|
||||
// is already stopped has nothing to stop and no guard to flip.
|
||||
disable: {
|
||||
shown: !disabled && m.onVolume,
|
||||
reason: disabled ? 'Already disabled.' : 'Nothing is running to stop.',
|
||||
},
|
||||
enable: {
|
||||
shown: disabled && m.onVolume,
|
||||
reason: 'Only a disabled module can be enabled.',
|
||||
},
|
||||
uninstall: {
|
||||
shown: m.onVolume,
|
||||
reason: 'There are no files left to remove.',
|
||||
},
|
||||
// The server refuses a standalone purge unless the module is disabled, so
|
||||
// the button says so rather than offering a click that 409s.
|
||||
purge: {
|
||||
shown: m.onVolume && m.canPurge,
|
||||
enabled: disabled,
|
||||
reason: !m.canPurge
|
||||
? 'This module ships no purge.sql, so its data cannot be deleted.'
|
||||
: 'Disable it first, so nothing is serving out of the tables being dropped.',
|
||||
},
|
||||
// A row with no directory is the one thing an uninstall cannot tidy through
|
||||
// the normal path — offer clearing it instead.
|
||||
forget: {
|
||||
shown: !m.onVolume && m.state !== null,
|
||||
reason: 'The module is still installed.',
|
||||
},
|
||||
running,
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Does anything on this list need a restart before it matches what is running?
|
||||
*
|
||||
* Drives the one banner at the top of the screen rather than a badge per row:
|
||||
* the restart is a property of the SERVER, not of a module, and offering it
|
||||
* five times would suggest otherwise.
|
||||
*/
|
||||
export const needsRestart = (modules) => modules.some((m) => statusOf(m).pending)
|
||||
|
||||
/**
|
||||
* Split a hosts string the way the server will.
|
||||
*
|
||||
* Duplicated from `install.parseHosts` deliberately — it is four lines, and the
|
||||
* alternative is an API round trip to preview what the field is going to mean.
|
||||
* The server remains the one that decides; this only shows the operator how
|
||||
* their typing will be read.
|
||||
*/
|
||||
export function parseHosts(value) {
|
||||
return String(value || '')
|
||||
.split(/[,\s]+/)
|
||||
.map((h) => h.trim().toLowerCase())
|
||||
.filter(Boolean)
|
||||
}
|
||||
@@ -45,6 +45,7 @@ const IconPulse = () => <Icon><path d="M3 12h3l2 6 4-14 2 8h7" /></Icon>
|
||||
const IconUser = () => <Icon><circle cx="12" cy="8" r="4" /><path d="M4 21a8 8 0 0 1 16 0" /></Icon>
|
||||
const IconNav = () => <Icon><path d="M4 6h16M4 12h16M4 18h10" /><circle cx="18" cy="18" r="2.5" /></Icon>
|
||||
const IconPalette = () => <Icon><path d="M12 3a9 9 0 1 0 0 18 2 2 0 0 0 1.6-3.2 2 2 0 0 1 1.6-3.2H18a3 3 0 0 0 3-3 9 9 0 0 0-9-8.6z" /><circle cx="7.5" cy="11.5" r="1" /><circle cx="10.5" cy="7.5" r="1" /><circle cx="15" cy="8.5" r="1" /></Icon>
|
||||
const IconModules = () => <Icon><path d="M12 3l8 4.5-8 4.5-8-4.5z" /><path d="M4 12l8 4.5 8-4.5" /><path d="M4 16.5L12 21l8-4.5" /></Icon>
|
||||
|
||||
// Nav is grouped into collapsible categories. A group with no `title` renders
|
||||
// its items ungrouped (Dashboard at top, Account at bottom). Each item's `roles`
|
||||
@@ -83,6 +84,10 @@ export const NAV = [
|
||||
{ to: '/admin/users', label: 'Users', icon: IconUsers, roles: ['admin'] },
|
||||
{ to: '/admin/invites', label: 'Invites', icon: IconUsers, roles: ['admin'] },
|
||||
{ to: '/admin/settings', label: 'Settings', icon: IconGear, roles: ['admin'] },
|
||||
// Admin-only, matching the server: every route under /admin/modules
|
||||
// re-gates to `admin` on top of the group's staff gate, because installing
|
||||
// a module runs its code in this process.
|
||||
{ to: '/admin/modules', label: 'Modules', icon: IconModules, roles: ['admin'] },
|
||||
{ to: '/admin/appearance', label: 'Appearance', icon: IconPalette, roles: ['admin'] },
|
||||
{ to: '/admin/navigation', label: 'Navigation', icon: IconNav, roles: ['admin'] },
|
||||
{ to: '/admin/hero', label: 'Hero Editor', icon: IconHero, roles: ['admin'] },
|
||||
|
||||
424
client/src/routes/admin/views/ModulesAdmin.jsx
Normal file
424
client/src/routes/admin/views/ModulesAdmin.jsx
Normal file
@@ -0,0 +1,424 @@
|
||||
import { useCallback, useEffect, useState } from 'react'
|
||||
import { Loading, ErrorState } from '../../../components/PageState.jsx'
|
||||
import { dateTime } from '../../../lib/format.js'
|
||||
import { statusOf, actionsFor, declarationNoteFor, needsRestart, parseHosts } from '../../../lib/moduleAdmin.js'
|
||||
import { api } from '../../../api/client.js'
|
||||
|
||||
// Installed modules: install from a release URL, enable, disable, uninstall,
|
||||
// purge, and restart the server so the changes take effect.
|
||||
//
|
||||
// Phase 4, slice 2 of docs/website/MODULE_SYSTEM.md §2.7.2. Everything that
|
||||
// decides what a row SAYS and which buttons it offers lives in
|
||||
// lib/moduleAdmin.js, which is plain JS and has tests; this file renders it.
|
||||
//
|
||||
// Two things about this screen are unlike the rest of the admin panel and are
|
||||
// deliberate:
|
||||
//
|
||||
// 1. **Restart is a banner, not a per-row button.** A restart is a property of
|
||||
// the server, not of a module. Offering it on five rows would suggest
|
||||
// otherwise, and an operator who installed three modules should restart
|
||||
// once.
|
||||
// 2. **Disable is the only action that takes effect immediately.** Everything
|
||||
// else is "true after the next boot", because the loader reads the volume
|
||||
// at require time (§1.12). The buttons say which they are.
|
||||
|
||||
const TONE = {
|
||||
ok: '#7fd0a4',
|
||||
warn: 'var(--accent)',
|
||||
bad: '#d98b84',
|
||||
idle: 'var(--muted)',
|
||||
}
|
||||
|
||||
const DANGER = { color: '#d98b84', borderColor: '#5b2020' }
|
||||
|
||||
function Pill({ tone, children }) {
|
||||
return (
|
||||
<span
|
||||
className="badge"
|
||||
style={{ color: TONE[tone] || 'var(--muted)', borderColor: 'var(--line)', background: 'var(--panel-flat)' }}
|
||||
>
|
||||
{children}
|
||||
</span>
|
||||
)
|
||||
}
|
||||
|
||||
// ── Install ────────────────────────────────────────────────────────────────
|
||||
|
||||
function InstallForm({ sourceHosts, onInstalled }) {
|
||||
const [url, setUrl] = useState('')
|
||||
const [busy, setBusy] = useState(false)
|
||||
const [error, setError] = useState('')
|
||||
const [result, setResult] = useState(null)
|
||||
|
||||
async function submit(e) {
|
||||
e.preventDefault()
|
||||
setError('')
|
||||
setResult(null)
|
||||
if (!url.trim()) return setError('Paste the URL of a release install manifest.')
|
||||
setBusy(true)
|
||||
try {
|
||||
const res = await api.admin.installModule(url.trim())
|
||||
setResult(res)
|
||||
setUrl('')
|
||||
await onInstalled()
|
||||
} catch (err) {
|
||||
// The server's message is written to be read by whoever pasted the URL —
|
||||
// which host was refused, which hash did not match, what the archive
|
||||
// contained. Replacing it with something friendlier would throw away the
|
||||
// only part that helps.
|
||||
setError(err.message || 'Could not install that module.')
|
||||
} finally {
|
||||
setBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="panel" style={{ padding: 22, marginBottom: 22 }}>
|
||||
<div className="field-label" style={{ marginBottom: 10 }}>Install a module</div>
|
||||
<form onSubmit={submit} style={{ display: 'flex', gap: 12, alignItems: 'flex-end', flexWrap: 'wrap' }}>
|
||||
<label style={{ flex: '1 1 380px' }}>
|
||||
<span className="field-label">Release install-manifest URL</span>
|
||||
<input
|
||||
type="url"
|
||||
value={url}
|
||||
onChange={(e) => setUrl(e.target.value)}
|
||||
className="input"
|
||||
placeholder="https://gitea.example.com/org/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json"
|
||||
/>
|
||||
</label>
|
||||
<button type="submit" disabled={busy} className="btn btn-primary btn-sq">
|
||||
{busy ? 'Installing…' : 'Install'}
|
||||
</button>
|
||||
</form>
|
||||
|
||||
<p className="sans" style={{ margin: '12px 0 0', fontSize: '0.8rem', color: 'var(--muted)' }}>
|
||||
The bundle is downloaded, checked against the <code>sha256</code> its release published, and
|
||||
unpacked onto the modules volume. It starts serving after a restart.{' '}
|
||||
{sourceHosts.length === 0
|
||||
? 'No source hosts are allowed yet — add one below before installing.'
|
||||
: `Allowed hosts: ${sourceHosts.join(', ')}.`}
|
||||
</p>
|
||||
|
||||
{error && <p className="sans" style={{ margin: '12px 0 0', color: TONE.bad, fontSize: '0.85rem' }}>{error}</p>}
|
||||
{result && (
|
||||
<p className="sans" style={{ margin: '12px 0 0', color: TONE.ok, fontSize: '0.85rem' }}>
|
||||
{result.replaced ? 'Upgraded' : 'Installed'} {result.module?.name} v{result.module?.version}. Restart to load it.
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// ── The restart banner ─────────────────────────────────────────────────────
|
||||
|
||||
function RestartBanner({ onDone }) {
|
||||
const [busy, setBusy] = useState(false)
|
||||
const [sent, setSent] = useState(false)
|
||||
|
||||
async function restart() {
|
||||
// Said plainly, because it is true and because the failure mode is bad: a
|
||||
// deployment with no supervisor does not come back on its own.
|
||||
const ok = window.confirm(
|
||||
'Restart the server now?\n\n'
|
||||
+ 'The site will be briefly unavailable. It comes back on its own only if something is '
|
||||
+ 'supervising the process — the shipped Docker Compose file does. If you are running '
|
||||
+ '`npm start` by hand, you will have to start it again yourself.',
|
||||
)
|
||||
if (!ok) return
|
||||
setBusy(true)
|
||||
try {
|
||||
await api.admin.restartServer()
|
||||
setSent(true)
|
||||
// Nothing is coming back on this connection: the process is exiting. Give
|
||||
// the supervisor a moment and then reload, which is what the operator was
|
||||
// about to do anyway.
|
||||
setTimeout(() => { if (onDone) onDone() }, 6000)
|
||||
} catch {
|
||||
// A failed request here is expected as often as not — the process can win
|
||||
// the race and drop the socket before the response lands.
|
||||
setSent(true)
|
||||
setTimeout(() => { if (onDone) onDone() }, 6000)
|
||||
} finally {
|
||||
setBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="panel" style={{ padding: 18, marginBottom: 22, borderColor: 'var(--accent)' }}>
|
||||
<div style={{ display: 'flex', gap: 14, alignItems: 'center', flexWrap: 'wrap' }}>
|
||||
<div style={{ flex: '1 1 320px' }}>
|
||||
<div className="field-label" style={{ marginBottom: 4 }}>Restart needed</div>
|
||||
<p className="sans" style={{ margin: 0, fontSize: '0.84rem', color: 'var(--muted)' }}>
|
||||
{sent
|
||||
? 'Restarting. This page will reload once the server is back.'
|
||||
: 'Modules are read from disk when the server starts, so an install, an uninstall or a re-enable only takes effect after a restart.'}
|
||||
</p>
|
||||
</div>
|
||||
<button type="button" className="btn btn-primary btn-sq" disabled={busy || sent} onClick={restart}>
|
||||
{sent ? 'Restarting…' : 'Restart the server'}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// ── The source allowlist ───────────────────────────────────────────────────
|
||||
|
||||
function SourceHosts({ hosts, onSaved }) {
|
||||
const [value, setValue] = useState(hosts.join(', '))
|
||||
const [busy, setBusy] = useState(false)
|
||||
const [error, setError] = useState('')
|
||||
const [saved, setSaved] = useState(false)
|
||||
|
||||
useEffect(() => { setValue(hosts.join(', ')) }, [hosts])
|
||||
|
||||
async function save(e) {
|
||||
e.preventDefault()
|
||||
setError('')
|
||||
setSaved(false)
|
||||
setBusy(true)
|
||||
try {
|
||||
await api.admin.setModuleSources(value)
|
||||
setSaved(true)
|
||||
await onSaved()
|
||||
} catch (err) {
|
||||
setError(err.message || 'Could not save the allowlist.')
|
||||
} finally {
|
||||
setBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
const parsed = parseHosts(value)
|
||||
|
||||
return (
|
||||
<div className="panel" style={{ padding: 22, marginTop: 22 }}>
|
||||
<div className="field-label" style={{ marginBottom: 10 }}>Where modules may be installed from</div>
|
||||
<form onSubmit={save} style={{ display: 'flex', gap: 12, alignItems: 'flex-end', flexWrap: 'wrap' }}>
|
||||
<label style={{ flex: '1 1 380px' }}>
|
||||
<span className="field-label">Allowed hosts</span>
|
||||
<input
|
||||
type="text"
|
||||
value={value}
|
||||
onChange={(e) => setValue(e.target.value)}
|
||||
className="input"
|
||||
placeholder="gitea.example.com, releases.example.org"
|
||||
/>
|
||||
</label>
|
||||
<button type="submit" disabled={busy} className="btn btn-sq">{busy ? 'Saving…' : 'Save'}</button>
|
||||
</form>
|
||||
|
||||
<p className="sans" style={{ margin: '12px 0 0', fontSize: '0.8rem', color: 'var(--muted)' }}>
|
||||
Installing a module runs its code inside this server, so only hosts listed here may be
|
||||
installed from — over HTTPS, and re-checked on every redirect. An empty list blocks all
|
||||
installs.{' '}
|
||||
{parsed.length > 0 && <>Will be saved as: <code>{parsed.join(', ')}</code>.</>}
|
||||
</p>
|
||||
|
||||
{error && <p className="sans" style={{ margin: '10px 0 0', color: TONE.bad, fontSize: '0.85rem' }}>{error}</p>}
|
||||
{saved && !error && <p className="sans" style={{ margin: '10px 0 0', color: TONE.ok, fontSize: '0.85rem' }}>Saved.</p>}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// ── One module ─────────────────────────────────────────────────────────────
|
||||
|
||||
function ModuleRow({ m, onChanged, onError }) {
|
||||
const [busy, setBusy] = useState('')
|
||||
const status = statusOf(m)
|
||||
const actions = actionsFor(m)
|
||||
const note = declarationNoteFor(m)
|
||||
|
||||
async function run(name, fn) {
|
||||
setBusy(name)
|
||||
try {
|
||||
await fn()
|
||||
await onChanged()
|
||||
} catch (err) {
|
||||
onError(err.message || `Could not ${name} ${m.id}.`)
|
||||
} finally {
|
||||
setBusy('')
|
||||
}
|
||||
}
|
||||
|
||||
const disable = () => run('disable', () => api.admin.disableModule(m.id))
|
||||
const enable = () => run('enable', () => api.admin.enableModule(m.id))
|
||||
|
||||
function uninstall() {
|
||||
// The purge choice is made HERE and only here, because purge.sql lives
|
||||
// inside the directory the uninstall is about to delete — there is no
|
||||
// "purge it later" (§2.7.2 decision 5). Two prompts rather than one, so
|
||||
// "delete the data too" is never something you agree to by reflex.
|
||||
if (!window.confirm(`Uninstall ${m.name}?\n\nIts files are removed. Its data is kept unless you ask otherwise next.`)) return
|
||||
let purge = false
|
||||
if (m.canPurge) {
|
||||
purge = window.confirm(
|
||||
`Also permanently delete ${m.name}'s data?\n\n`
|
||||
+ 'This drops its tables and cannot be undone. This is the only moment it can be offered — '
|
||||
+ 'the script that does it is part of the files being removed.\n\n'
|
||||
+ 'OK deletes the data. Cancel keeps it.',
|
||||
)
|
||||
}
|
||||
return run('uninstall', () => api.admin.uninstallModule(m.id, { purge }))
|
||||
}
|
||||
|
||||
function purge() {
|
||||
if (!window.confirm(`Permanently delete ${m.name}'s data?\n\nThis drops its tables and cannot be undone.`)) return
|
||||
return run('purge', () => api.admin.purgeModule(m.id))
|
||||
}
|
||||
|
||||
const forget = () => run('forget', () => api.admin.uninstallModule(m.id))
|
||||
|
||||
return (
|
||||
<tr>
|
||||
<td className="adm-td" style={{ color: 'var(--text)' }}>
|
||||
<div style={{ fontWeight: 600 }}>{m.name}</div>
|
||||
<div className="dim" style={{ fontSize: '0.76rem' }}>
|
||||
{/* A declared module that has never installed has no version to show —
|
||||
only the one MODULES asks for, which the status column carries. */}
|
||||
{m.id}{m.version ? ` · v${m.version}` : ''}
|
||||
</div>
|
||||
{m.capabilities?.length > 0 && (
|
||||
<div className="dim" style={{ fontSize: '0.72rem', marginTop: 2 }}>{m.capabilities.join(' · ')}</div>
|
||||
)}
|
||||
</td>
|
||||
|
||||
<td className="adm-td">
|
||||
<Pill tone={status.tone}>{status.label}</Pill>
|
||||
<div className="dim" style={{ fontSize: '0.74rem', marginTop: 4, maxWidth: 380 }}>{status.detail}</div>
|
||||
{/* The environment's declaration, on its own line: a module can be
|
||||
running fine while its declared upgrade is failing, and the status
|
||||
above can only be one of those two things. */}
|
||||
{note && (
|
||||
<div
|
||||
style={{
|
||||
fontSize: '0.74rem',
|
||||
marginTop: 4,
|
||||
maxWidth: 380,
|
||||
color: note.tone === 'warn' ? TONE.warn : 'var(--muted)',
|
||||
}}
|
||||
>
|
||||
{note.text}
|
||||
</div>
|
||||
)}
|
||||
</td>
|
||||
|
||||
<td className="adm-td dim" style={{ fontSize: '0.74rem' }}>
|
||||
{/* Provenance. Null for a directory placed on the volume by hand, which
|
||||
stays a supported install — so it is shown as that, not as missing.
|
||||
A declared module can also reach a boot with no provenance: the
|
||||
no-op path never fetches, so it has no sha256 to record and no
|
||||
reason to write a row. Saying "by hand" there would be the one
|
||||
wrong answer. */}
|
||||
{m.source ? (
|
||||
<>
|
||||
<div style={{ wordBreak: 'break-all', maxWidth: 260 }}>{m.source}</div>
|
||||
{m.sha256 && <div style={{ marginTop: 2 }}>sha256 {m.sha256.slice(0, 12)}…</div>}
|
||||
</>
|
||||
) : (
|
||||
<span>{m.declared ? 'From the declared module set' : 'Placed on the volume by hand'}</span>
|
||||
)}
|
||||
{m.installedAt && <div style={{ marginTop: 2 }}>{dateTime(m.installedAt)}</div>}
|
||||
</td>
|
||||
|
||||
<td className="adm-td" style={{ textAlign: 'right', whiteSpace: 'nowrap' }}>
|
||||
<div style={{ display: 'inline-flex', gap: 6, flexWrap: 'wrap', justifyContent: 'flex-end' }}>
|
||||
{actions.disable.shown && (
|
||||
<button type="button" className="pill" style={{ fontSize: '0.72rem' }} disabled={Boolean(busy)} onClick={disable}>
|
||||
{busy === 'disable' ? 'Stopping…' : 'Disable'}
|
||||
</button>
|
||||
)}
|
||||
{actions.enable.shown && (
|
||||
<button type="button" className="pill" style={{ fontSize: '0.72rem' }} disabled={Boolean(busy)} onClick={enable}>
|
||||
{busy === 'enable' ? 'Enabling…' : 'Enable'}
|
||||
</button>
|
||||
)}
|
||||
{actions.purge.shown && (
|
||||
<button
|
||||
type="button"
|
||||
className="pill"
|
||||
style={{ fontSize: '0.72rem', ...DANGER, opacity: actions.purge.enabled ? 1 : 0.45 }}
|
||||
disabled={Boolean(busy) || !actions.purge.enabled}
|
||||
title={actions.purge.enabled ? undefined : actions.purge.reason}
|
||||
onClick={purge}
|
||||
>
|
||||
{busy === 'purge' ? 'Purging…' : 'Purge data'}
|
||||
</button>
|
||||
)}
|
||||
{actions.uninstall.shown && (
|
||||
<button type="button" className="pill" style={{ fontSize: '0.72rem', ...DANGER }} disabled={Boolean(busy)} onClick={uninstall}>
|
||||
{busy === 'uninstall' ? 'Removing…' : 'Uninstall'}
|
||||
</button>
|
||||
)}
|
||||
{actions.forget.shown && (
|
||||
<button type="button" className="pill" style={{ fontSize: '0.72rem' }} disabled={Boolean(busy)} onClick={forget}>
|
||||
{busy === 'forget' ? 'Clearing…' : 'Clear the row'}
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
</td>
|
||||
</tr>
|
||||
)
|
||||
}
|
||||
|
||||
// ── The screen ─────────────────────────────────────────────────────────────
|
||||
|
||||
export default function ModulesAdmin() {
|
||||
const [data, setData] = useState(null)
|
||||
const [error, setError] = useState('')
|
||||
const [actionError, setActionError] = useState('')
|
||||
|
||||
const load = useCallback(async () => {
|
||||
setError('')
|
||||
try {
|
||||
setData(await api.admin.listModules())
|
||||
} catch {
|
||||
setError('Could not load installed modules.')
|
||||
}
|
||||
}, [])
|
||||
useEffect(() => { load() }, [load])
|
||||
|
||||
if (error) return <ErrorState message={error} />
|
||||
if (!data) return <Loading />
|
||||
|
||||
const modules = data.modules || []
|
||||
const sourceHosts = data.sourceHosts || []
|
||||
|
||||
return (
|
||||
<section>
|
||||
{needsRestart(modules) && <RestartBanner onDone={() => window.location.reload()} />}
|
||||
|
||||
<InstallForm sourceHosts={sourceHosts} onInstalled={load} />
|
||||
|
||||
{actionError && (
|
||||
<p className="sans" style={{ margin: '0 0 14px', color: TONE.bad, fontSize: '0.85rem' }}>{actionError}</p>
|
||||
)}
|
||||
|
||||
<div className="panel-flat">
|
||||
<table className="adm-table">
|
||||
<thead>
|
||||
<tr>
|
||||
<th className="adm-th">Module</th>
|
||||
<th className="adm-th">Status</th>
|
||||
<th className="adm-th">Installed from</th>
|
||||
<th className="adm-th" />
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{modules.length === 0 && (
|
||||
<tr>
|
||||
<td className="adm-td" colSpan={4} style={{ color: 'var(--muted)' }}>
|
||||
No modules installed. Paste a release install-manifest URL above to add one.
|
||||
</td>
|
||||
</tr>
|
||||
)}
|
||||
{modules.map((m) => (
|
||||
<ModuleRow key={m.id} m={m} onChanged={load} onError={setActionError} />
|
||||
))}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<SourceHosts hosts={sourceHosts} onSaved={load} />
|
||||
</section>
|
||||
)
|
||||
}
|
||||
@@ -140,3 +140,48 @@ test('DELETE self-service session revoke encodes the id and uses the DELETE meth
|
||||
assert.equal(calls[0].opts.method, 'DELETE')
|
||||
assert.match(calls[0].url, /\/auth\/me\/sessions\/a%20b%2Fc$/)
|
||||
})
|
||||
|
||||
// ── admin: installed modules (MODULE_SYSTEM.md §2.7.2) ──────────────────
|
||||
//
|
||||
// These pin the URLs, because the destructive one differs from the harmless one
|
||||
// by a query parameter and nothing else.
|
||||
|
||||
test('module actions hit the right paths and methods', async () => {
|
||||
const cases = [
|
||||
[() => api.admin.listModules(), 'GET', '/api/v1/admin/modules'],
|
||||
[() => api.admin.installModule('https://x/y.json'), 'POST', '/api/v1/admin/modules'],
|
||||
[() => api.admin.enableModule('uo'), 'POST', '/api/v1/admin/modules/uo/enable'],
|
||||
[() => api.admin.disableModule('uo'), 'POST', '/api/v1/admin/modules/uo/disable'],
|
||||
[() => api.admin.purgeModule('uo'), 'POST', '/api/v1/admin/modules/uo/purge'],
|
||||
[() => api.admin.setModuleSources('a.com'), 'PUT', '/api/v1/admin/modules/sources'],
|
||||
[() => api.admin.restartServer(), 'POST', '/api/v1/admin/modules/restart'],
|
||||
]
|
||||
for (const [call, method, url] of cases) {
|
||||
calls = []
|
||||
willReply({ body: {} })
|
||||
await call()
|
||||
assert.equal(calls[0].url, url)
|
||||
assert.equal(calls[0].opts.method || 'GET', method)
|
||||
}
|
||||
})
|
||||
|
||||
test('uninstall only asks for a purge when it is told to', async () => {
|
||||
// The difference between "remove the module" and "remove the module and drop
|
||||
// every table it owns" is this query parameter, so a default that leaned the
|
||||
// wrong way would be irreversible.
|
||||
willReply({ body: {} })
|
||||
await api.admin.uninstallModule('uo')
|
||||
assert.equal(calls[0].url, '/api/v1/admin/modules/uo')
|
||||
assert.equal(calls[0].opts.method, 'DELETE')
|
||||
|
||||
calls = []
|
||||
willReply({ body: {} })
|
||||
await api.admin.uninstallModule('uo', { purge: true })
|
||||
assert.equal(calls[0].url, '/api/v1/admin/modules/uo?purge=true')
|
||||
})
|
||||
|
||||
test('a module id is URL-encoded on the way into the path', async () => {
|
||||
willReply({ body: {} })
|
||||
await api.admin.disableModule('a b/c')
|
||||
assert.equal(calls[0].url, '/api/v1/admin/modules/a%20b%2Fc/disable')
|
||||
})
|
||||
|
||||
281
client/test/moduleAdmin.test.js
Normal file
281
client/test/moduleAdmin.test.js
Normal file
@@ -0,0 +1,281 @@
|
||||
import { test } from 'node:test'
|
||||
import assert from 'node:assert/strict'
|
||||
import { statusOf, actionsFor, declarationNoteFor, needsRestart, parseHosts } from '../src/lib/moduleAdmin.js'
|
||||
|
||||
// lib/moduleAdmin.js — what the Modules screen says about a module and what it
|
||||
// lets you do to it. Phase 4, slice 2 of MODULE_SYSTEM.md §2.7.2.
|
||||
//
|
||||
// This is the part of the screen worth testing, and it is plain JS so this
|
||||
// runner can reach it (there is no DOM here). What it encodes is §2.4's rule
|
||||
// that the row, the loader and the volume are three sources of truth which are
|
||||
// ALLOWED to disagree — so most of these cases are combinations that a screen
|
||||
// picking one source would render as a lie.
|
||||
|
||||
/** A module as GET /admin/modules returns it, with the running case as default. */
|
||||
const mod = (over = {}) => ({
|
||||
id: 'uo',
|
||||
name: 'Ultima Online',
|
||||
version: '1.0.0',
|
||||
state: 'started',
|
||||
failureStage: null,
|
||||
failureReason: null,
|
||||
source: 'https://gitea.example.com/x/uo.json',
|
||||
sha256: 'a'.repeat(64),
|
||||
installedAt: null,
|
||||
startedAt: null,
|
||||
liveState: 'started',
|
||||
liveVersion: '1.0.0',
|
||||
capabilities: [],
|
||||
onVolume: true,
|
||||
canPurge: true,
|
||||
declared: false,
|
||||
declaredVersion: null,
|
||||
declaredError: null,
|
||||
...over,
|
||||
})
|
||||
|
||||
// ── statusOf ───────────────────────────────────────────────────────────────
|
||||
|
||||
test('a mounted, started module is Running and needs nothing', () => {
|
||||
const s = statusOf(mod())
|
||||
assert.equal(s.label, 'Running')
|
||||
assert.equal(s.tone, 'ok')
|
||||
assert.equal(s.pending, false)
|
||||
})
|
||||
|
||||
test('enabled in the row but disabled in the loader is "Restart to start"', () => {
|
||||
// THE case decision 3 creates on purpose: disable ran the module's onShutdown,
|
||||
// then the operator enabled it again. The row says enabled; nothing can start
|
||||
// it before a restart. Showing either "Running" or "Disabled" would be false.
|
||||
const s = statusOf(mod({ state: 'enabled', liveState: 'disabled' }))
|
||||
assert.equal(s.label, 'Restart to start')
|
||||
assert.equal(s.tone, 'warn')
|
||||
assert.equal(s.pending, true)
|
||||
assert.match(s.detail, /cannot be restarted in place/)
|
||||
})
|
||||
|
||||
test('freshly installed and never booted into is also "Restart to start"', () => {
|
||||
const s = statusOf(mod({ state: 'installed', liveState: null }))
|
||||
assert.equal(s.label, 'Restart to start')
|
||||
assert.equal(s.pending, true)
|
||||
assert.match(s.detail, /mounts when the server next starts/)
|
||||
})
|
||||
|
||||
test('a disabled module is Disabled, and that is not pending anything', () => {
|
||||
// Disable takes effect immediately — it is the one action that does — so there
|
||||
// is nothing for a restart banner to be about.
|
||||
const s = statusOf(mod({ state: 'disabled', liveState: 'disabled' }))
|
||||
assert.equal(s.label, 'Disabled')
|
||||
assert.equal(s.pending, false)
|
||||
})
|
||||
|
||||
test('a fresh install over a failed row is pending, not failed', () => {
|
||||
// THE defect the §7.7 browser smoke found, and one no test here had modelled.
|
||||
// Installing over a row the previous boot left `startup_failed` rendered
|
||||
// "Failed at the require stage: module directory not present on the volume" a
|
||||
// second after the files had been written — and suppressed the restart banner
|
||||
// the install had just told the operator to use.
|
||||
//
|
||||
// `liveState === null` with the module on the volume means the loader's scan
|
||||
// never saw it, so it arrived after boot and everything the row says predates
|
||||
// it.
|
||||
const s = statusOf(mod({
|
||||
state: 'startup_failed',
|
||||
liveState: null,
|
||||
failureStage: 'require',
|
||||
failureReason: 'module directory not present on the volume',
|
||||
}))
|
||||
assert.equal(s.label, 'Restart to start')
|
||||
assert.equal(s.pending, true)
|
||||
assert.doesNotMatch(s.detail, /not present on the volume/, 'the stale reason must not survive the install')
|
||||
})
|
||||
|
||||
test('the restart banner appears for that install', () => {
|
||||
// The second half of the same defect: the banner is driven by `pending`, so a
|
||||
// row wrongly classified as failed silently removed the only way to act on it.
|
||||
assert.equal(needsRestart([mod({ state: 'startup_failed', liveState: null })]), true)
|
||||
})
|
||||
|
||||
test('an upgrade that has not been restarted into says so', () => {
|
||||
// Same class as the stale-failure defect: the row is a promise about the next
|
||||
// boot, not a description of this one. Reporting "Running v2.0.0" while the
|
||||
// process is serving v1.0.0 would hide the only action that fixes it.
|
||||
const s = statusOf(mod({ version: '2.0.0', liveVersion: '1.0.0' }))
|
||||
assert.equal(s.label, 'Restart to finish upgrading')
|
||||
assert.equal(s.pending, true)
|
||||
assert.match(s.detail, /v2\.0\.0 is installed; v1\.0\.0 is still running/)
|
||||
})
|
||||
|
||||
test('reinstalling the SAME version is not an upgrade in progress', () => {
|
||||
assert.equal(statusOf(mod({ version: '1.0.0', liveVersion: '1.0.0' })).label, 'Running')
|
||||
})
|
||||
|
||||
test('a failed module reports the stage and the reason it recorded', () => {
|
||||
const s = statusOf(mod({
|
||||
state: 'startup_failed',
|
||||
liveState: 'startup_failed',
|
||||
failureStage: 'schema',
|
||||
failureReason: "Unknown column 'x' in 'field list'",
|
||||
}))
|
||||
assert.equal(s.label, 'Failed to start')
|
||||
assert.equal(s.tone, 'bad')
|
||||
assert.match(s.detail, /schema stage/)
|
||||
assert.match(s.detail, /Unknown column/)
|
||||
})
|
||||
|
||||
test('a failure with no recorded reason says so rather than showing a blank', () => {
|
||||
const s = statusOf(mod({ state: 'startup_failed', liveState: 'startup_failed' }))
|
||||
assert.match(s.detail, /recorded no reason/)
|
||||
})
|
||||
|
||||
test('a row whose directory is gone by hand is bad, not merely disabled', () => {
|
||||
// The boot reconcile marks this `startup_failed` because a row claiming to be
|
||||
// enabled for a module that is not on the volume is simply untrue.
|
||||
const s = statusOf(mod({ state: 'startup_failed', liveState: null, onVolume: false, failureStage: 'require', failureReason: 'module directory not present on the volume' }))
|
||||
assert.equal(s.label, 'Missing from the volume')
|
||||
assert.equal(s.tone, 'bad')
|
||||
})
|
||||
|
||||
test('an uninstalled module reads as uninstalled, and says the data was kept', () => {
|
||||
// Uninstall leaves the row `disabled` and the data alone — which is the whole
|
||||
// point of keeping the row, so the screen has to say it.
|
||||
const s = statusOf(mod({ state: 'disabled', liveState: 'disabled', onVolume: false }))
|
||||
assert.equal(s.label, 'Uninstalled')
|
||||
assert.equal(s.tone, 'idle')
|
||||
assert.match(s.detail, /data was kept/i)
|
||||
})
|
||||
|
||||
test('missing-from-the-volume beats every other status', () => {
|
||||
// Ordering: a module with no files is described that way whatever its row
|
||||
// still claims, because there is nothing there to be running.
|
||||
for (const state of ['started', 'enabled', 'installed', 'startup_failed']) {
|
||||
assert.match(statusOf(mod({ state, onVolume: false })).label, /Missing from the volume/)
|
||||
}
|
||||
})
|
||||
|
||||
// ── actionsFor ─────────────────────────────────────────────────────────────
|
||||
|
||||
test('a running module offers disable, uninstall and a blocked purge', () => {
|
||||
const a = actionsFor(mod())
|
||||
assert.equal(a.disable.shown, true)
|
||||
assert.equal(a.enable.shown, false)
|
||||
assert.equal(a.uninstall.shown, true)
|
||||
assert.equal(a.purge.shown, true)
|
||||
// Shown but not clickable: the server refuses a standalone purge on anything
|
||||
// that is not disabled, so offering the click would only produce a 409.
|
||||
assert.equal(a.purge.enabled, false)
|
||||
assert.match(a.purge.reason, /Disable it first/)
|
||||
})
|
||||
|
||||
test('a disabled module offers enable, and purge is now live', () => {
|
||||
const a = actionsFor(mod({ state: 'disabled', liveState: 'disabled' }))
|
||||
assert.equal(a.enable.shown, true)
|
||||
assert.equal(a.disable.shown, false)
|
||||
assert.equal(a.purge.enabled, true)
|
||||
})
|
||||
|
||||
test('a module with no purge.sql never offers purge, and says why', () => {
|
||||
const a = actionsFor(mod({ state: 'disabled', liveState: 'disabled', canPurge: false }))
|
||||
assert.equal(a.purge.shown, false)
|
||||
assert.match(a.purge.reason, /ships no purge.sql/)
|
||||
})
|
||||
|
||||
test('a module with no files offers only clearing the row', () => {
|
||||
const a = actionsFor(mod({ state: 'disabled', liveState: null, onVolume: false }))
|
||||
assert.equal(a.uninstall.shown, false)
|
||||
assert.equal(a.disable.shown, false)
|
||||
assert.equal(a.enable.shown, false)
|
||||
assert.equal(a.purge.shown, false, 'there is no purge.sql left to run')
|
||||
assert.equal(a.forget.shown, true)
|
||||
})
|
||||
|
||||
test('a directory with no row yet is actionable, and offers nothing to forget', () => {
|
||||
// A hand-placed install before its first boot: it has no row, so `state` is
|
||||
// null. Its routes are already being served, so it must be disableable.
|
||||
const a = actionsFor(mod({ state: null, liveState: 'started' }))
|
||||
assert.equal(a.disable.shown, true)
|
||||
assert.equal(a.uninstall.shown, true)
|
||||
assert.equal(a.forget.shown, false)
|
||||
})
|
||||
|
||||
// ── needsRestart ───────────────────────────────────────────────────────────
|
||||
|
||||
test('the restart banner is driven by the list, not by any one module', () => {
|
||||
// A restart is a property of the SERVER. One pending module is enough, and
|
||||
// three do not mean three restarts.
|
||||
assert.equal(needsRestart([mod(), mod({ id: 'b' })]), false)
|
||||
assert.equal(needsRestart([mod(), mod({ id: 'b', state: 'installed', liveState: null })]), true)
|
||||
assert.equal(needsRestart([]), false)
|
||||
})
|
||||
|
||||
test('a disabled module does not ask for a restart', () => {
|
||||
// Disable is immediate; a banner here would be asking for a restart that
|
||||
// would change nothing.
|
||||
assert.equal(needsRestart([mod({ state: 'disabled', liveState: 'disabled' })]), false)
|
||||
})
|
||||
|
||||
test('a failed module does not ask for a restart either', () => {
|
||||
// It is retried on every boot anyway, and the operator has to fix the cause
|
||||
// first — a banner would suggest restarting is the remedy.
|
||||
assert.equal(needsRestart([mod({ state: 'startup_failed', liveState: 'startup_failed' })]), false)
|
||||
})
|
||||
|
||||
// ── parseHosts ─────────────────────────────────────────────────────────────
|
||||
|
||||
test('parseHosts previews exactly what the server will store', () => {
|
||||
assert.deepEqual(parseHosts('A.com, b.com\n c.com'), ['a.com', 'b.com', 'c.com'])
|
||||
assert.deepEqual(parseHosts(' '), [])
|
||||
assert.deepEqual(parseHosts(undefined), [])
|
||||
})
|
||||
|
||||
// ── the declaration (slice 3) ──────────────────────────────────────────────
|
||||
|
||||
test('a declared module that has never installed says so, with the reason', () => {
|
||||
// No row, no directory, nothing mounted — invisible to the other three
|
||||
// sources, so without this branch the screen would describe a module it has
|
||||
// never had as a row gone stale.
|
||||
const s = statusOf(mod({
|
||||
state: null,
|
||||
liveState: null,
|
||||
liveVersion: null,
|
||||
version: null,
|
||||
onVolume: false,
|
||||
declared: true,
|
||||
declaredVersion: '0.3.0',
|
||||
declaredError: 'could not reach releases.example.com',
|
||||
}))
|
||||
assert.equal(s.label, 'Declared, not installed')
|
||||
assert.equal(s.tone, 'bad')
|
||||
assert.equal(s.pending, false, 'a restart will not fix an unreachable host')
|
||||
assert.match(s.detail, /could not reach releases.example.com/)
|
||||
})
|
||||
|
||||
test('a declared module waiting for its first resolution is not reported as failed', () => {
|
||||
const s = statusOf(mod({ state: null, liveState: null, version: null, onVolume: false, declared: true, declaredVersion: '0.3.0' }))
|
||||
assert.match(s.detail, /installed when the server next starts/)
|
||||
})
|
||||
|
||||
test('a running module whose declared upgrade is failing is still Running', () => {
|
||||
// Both facts are true at once. The status is one label, so the declaration
|
||||
// gets its own line rather than overwriting it.
|
||||
const m = mod({ declared: true, declaredVersion: '2.0.0', declaredError: 'sha256 did not match' })
|
||||
assert.equal(statusOf(m).label, 'Running')
|
||||
const note = declarationNoteFor(m)
|
||||
assert.equal(note.tone, 'warn')
|
||||
assert.match(note.text, /sha256 did not match/)
|
||||
})
|
||||
|
||||
test('uninstalling a declared module is told that its files come back', () => {
|
||||
// The sentence that saves an afternoon: MODULES owns what is on the volume,
|
||||
// the row owns whether it runs.
|
||||
const note = declarationNoteFor(mod({ state: 'disabled', liveState: null, onVolume: false, declared: true, declaredVersion: '1.0.0' }))
|
||||
assert.match(note.text, /come back when the server next starts/)
|
||||
assert.match(note.text, /switched off/)
|
||||
})
|
||||
|
||||
test('an ordinary declared module gets a quiet note, and an undeclared one none', () => {
|
||||
assert.equal(declarationNoteFor(mod()), null)
|
||||
const note = declarationNoteFor(mod({ declared: true, declaredVersion: '1.0.0' }))
|
||||
assert.equal(note.tone, 'idle')
|
||||
assert.match(note.text, /MODULES/)
|
||||
})
|
||||
@@ -39,6 +39,24 @@ services:
|
||||
# defaults to (<repo>/modules, and the repo is /app in the image), set
|
||||
# explicitly because the bind mount below is what makes it meaningful.
|
||||
MODULES_DIR: /app/modules
|
||||
# WHICH modules this deployment runs (MODULE_SYSTEM.md §2.7.2 decision 4).
|
||||
# One entry per module, `<id>@<version>=<install manifest URL>`, whitespace-
|
||||
# or comma-separated. The container resolves this set for itself at every
|
||||
# start: a module already unpacked at the declared version is left alone
|
||||
# without a single network call — so a restart with the internet down comes
|
||||
# up unchanged — and only a missing or different version is fetched,
|
||||
# verified against the sha256 its release manifest declares, and unpacked.
|
||||
# A failure is logged and surfaced in Admin → Modules; it never stops the
|
||||
# site from starting.
|
||||
#
|
||||
# Uncomment to declare a set here, in the file this host version-controls,
|
||||
# or leave it out and set MODULES in .env (env_file above) — or leave it
|
||||
# unset entirely and install from the admin panel. What it declares is what
|
||||
# is ON the volume, never whether a module runs: a module disabled from the
|
||||
# admin panel gets its files back and stays disabled.
|
||||
#
|
||||
# MODULES: >-
|
||||
# uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
@@ -56,10 +74,12 @@ services:
|
||||
# pull-only deployment without building anything. A bind mount rather than
|
||||
# a named volume because placing a module directory by hand is a supported
|
||||
# install — `tar -xf uo-1.0.0.tgz -C ./modules` then restart — and that has
|
||||
# to be doable from the host, not through `docker cp`.
|
||||
# to be doable from the host, not through `docker cp`. It is no longer the
|
||||
# usual way in: declare MODULES above, or install from Admin → Modules.
|
||||
#
|
||||
# Read-WRITE: the admin panel's install/uninstall unpacks and removes
|
||||
# directories here from inside the container.
|
||||
# Read-WRITE: MODULES resolution at start, and the admin panel's
|
||||
# install/uninstall, both unpack and remove directories here from inside
|
||||
# the container.
|
||||
#
|
||||
# `modules/` is tracked (it ships a README) so the directory exists in the
|
||||
# checkout with the operator's own ownership. Do not delete it — Docker
|
||||
|
||||
@@ -129,3 +129,38 @@ ANNOUNCE_POLL_MS=15000
|
||||
# NTFY_PUBLIC_URL=https://ntfy.example.com
|
||||
# NTFY_ALLOWED_ORIGINS=https://ntfy.example.com
|
||||
# NTFY_PUBLISH_TOKEN=
|
||||
|
||||
# Modules (MODULE_SYSTEM.md §2.5) — where installable modules live, and where
|
||||
# they may be installed from.
|
||||
# MODULES_DIR Directory the loader scans at require time. Defaults to
|
||||
# <repo>/modules; docker-compose.yml sets it to /app/modules,
|
||||
# which is the bind mount that makes it meaningful.
|
||||
# MODULE_SOURCE_HOSTS BOOTSTRAP ONLY. Comma-separated hostnames the admin panel
|
||||
# may install a module from, seeded into the `module_source_hosts`
|
||||
# setting the first time the site boots without one. From then
|
||||
# on the SETTING is authoritative and is edited in
|
||||
# Admin → Modules — changing this variable on an existing
|
||||
# deployment does nothing, deliberately, so a redeploy cannot
|
||||
# silently undo an operator's choice. Installs are https-only
|
||||
# and an empty list forbids all of them.
|
||||
# MODULES The module set this deployment RUNS, resolved at every
|
||||
# start (§2.7.2 decision 4). One entry per module, separated
|
||||
# by whitespace or commas:
|
||||
#
|
||||
# <id>@<version>=<install manifest URL>
|
||||
#
|
||||
# A module already unpacked at the declared version is left
|
||||
# alone WITHOUT touching the network, so a restart with no
|
||||
# route to the internet comes up unchanged; only a missing or
|
||||
# different version is fetched, through the same verify-and-
|
||||
# unpack path (and the same host allowlist) the admin panel
|
||||
# uses. A version that cannot be fetched is logged and shown
|
||||
# in Admin → Modules — it never stops the site from starting.
|
||||
#
|
||||
# This variable owns what is ON the volume, not what runs: a
|
||||
# module disabled from the admin panel stays disabled even
|
||||
# though its files are put back. Leave it unset to manage
|
||||
# modules entirely from the admin panel.
|
||||
# MODULES_DIR=/app/modules
|
||||
# MODULE_SOURCE_HOSTS=gitea.whitlocktech.com
|
||||
# MODULES=uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json
|
||||
|
||||
@@ -23,6 +23,17 @@ const DEFAULT_SETTINGS = {
|
||||
site_title: brand.name,
|
||||
// Android App Links opt-in — off until an admin enables it (docs/android/APP_LINKS.md).
|
||||
mobile_app_links_enabled: 'false',
|
||||
// Hosts a module may be installed from (MODULE_SYSTEM.md §2.7.2 decision 6).
|
||||
//
|
||||
// The environment BOOTSTRAPS this and does not own it: seedDefault is an
|
||||
// INSERT IGNORE, so the variable supplies a sane default on a fresh install
|
||||
// and never reaches back in to overwrite what an admin later chose in
|
||||
// Admin → Modules. Changing MODULE_SOURCE_HOSTS on an existing deployment is
|
||||
// therefore a no-op, which is the intended behaviour and not an oversight.
|
||||
//
|
||||
// An empty stored value forbids every install rather than allowing every host
|
||||
// — the safe direction for a setting someone might blank by accident.
|
||||
module_source_hosts: process.env.MODULE_SOURCE_HOSTS || 'gitea.whitlocktech.com',
|
||||
}
|
||||
|
||||
// Starter wiki sections (editable later via the admin panel).
|
||||
|
||||
68
server/package-lock.json
generated
68
server/package-lock.json
generated
@@ -27,6 +27,7 @@
|
||||
"sanitize-html": "^2.17.5",
|
||||
"speakeasy": "^2.0.0",
|
||||
"swagger-ui-express": "^5.0.1",
|
||||
"tar": "^7.5.22",
|
||||
"ws": "^8.21.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
@@ -34,6 +35,18 @@
|
||||
"swagger-autogen": "^2.23.7"
|
||||
}
|
||||
},
|
||||
"node_modules/@isaacs/fs-minipass": {
|
||||
"version": "4.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@isaacs/fs-minipass/-/fs-minipass-4.0.1.tgz",
|
||||
"integrity": "sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w==",
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"minipass": "^7.0.4"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@scarf/scarf": {
|
||||
"version": "1.4.0",
|
||||
"resolved": "https://registry.npmjs.org/@scarf/scarf/-/scarf-1.4.0.tgz",
|
||||
@@ -330,6 +343,15 @@
|
||||
"fsevents": "~2.3.2"
|
||||
}
|
||||
},
|
||||
"node_modules/chownr": {
|
||||
"version": "3.0.0",
|
||||
"resolved": "https://registry.npmjs.org/chownr/-/chownr-3.0.0.tgz",
|
||||
"integrity": "sha512-+IxzY9BZOQd/XuYPRmrvEVjF/nqj5kgT4kEq7VofrDoM1MxoRjEWkrCC3EtLi59TVawxTAn+orJwFQcrqEN1+g==",
|
||||
"license": "BlueOak-1.0.0",
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/cliui": {
|
||||
"version": "6.0.0",
|
||||
"resolved": "https://registry.npmjs.org/cliui/-/cliui-6.0.0.tgz",
|
||||
@@ -1488,6 +1510,27 @@
|
||||
"url": "https://github.com/sponsors/isaacs"
|
||||
}
|
||||
},
|
||||
"node_modules/minipass": {
|
||||
"version": "7.1.3",
|
||||
"resolved": "https://registry.npmjs.org/minipass/-/minipass-7.1.3.tgz",
|
||||
"integrity": "sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==",
|
||||
"license": "BlueOak-1.0.0",
|
||||
"engines": {
|
||||
"node": ">=16 || 14 >=14.17"
|
||||
}
|
||||
},
|
||||
"node_modules/minizlib": {
|
||||
"version": "3.1.0",
|
||||
"resolved": "https://registry.npmjs.org/minizlib/-/minizlib-3.1.0.tgz",
|
||||
"integrity": "sha512-KZxYo1BUkWD2TVFLr0MQoM8vUUigWD3LlD83a/75BqC+4qE0Hb1Vo5v1FgcfaNXvfXzr+5EhQ6ing/CaBijTlw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"minipass": "^7.1.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 18"
|
||||
}
|
||||
},
|
||||
"node_modules/morgan": {
|
||||
"version": "1.11.0",
|
||||
"resolved": "https://registry.npmjs.org/morgan/-/morgan-1.11.0.tgz",
|
||||
@@ -2254,6 +2297,22 @@
|
||||
"express": ">=4.0.0 || >=5.0.0-beta"
|
||||
}
|
||||
},
|
||||
"node_modules/tar": {
|
||||
"version": "7.5.22",
|
||||
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.22.tgz",
|
||||
"integrity": "sha512-MFO/QzvtAOmJbkhOaCTvbGcFN9L9b+JunIsDwaKljSOdcLMea3NJ1k9Usz/rjdfSXTq4dfzfeS7W4p4YOAAHeA==",
|
||||
"license": "BlueOak-1.0.0",
|
||||
"dependencies": {
|
||||
"@isaacs/fs-minipass": "^4.0.0",
|
||||
"chownr": "^3.0.0",
|
||||
"minipass": "^7.1.2",
|
||||
"minizlib": "^3.1.0",
|
||||
"yallist": "^5.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/to-regex-range": {
|
||||
"version": "5.0.1",
|
||||
"resolved": "https://registry.npmjs.org/to-regex-range/-/to-regex-range-5.0.1.tgz",
|
||||
@@ -2414,6 +2473,15 @@
|
||||
"integrity": "sha512-JKhqTOwSrqNA1NY5lSztJ1GrBiUodLMmIZuLiDaMRJ+itFd+ABVE8XBjOvIWL+rSqNDC74LCSFmlb/U4UZ4hJQ==",
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/yallist": {
|
||||
"version": "5.0.0",
|
||||
"resolved": "https://registry.npmjs.org/yallist/-/yallist-5.0.0.tgz",
|
||||
"integrity": "sha512-YgvUTfwqyc7UXVMrB+SImsVYSmTS8X/tSrtdNZMImM+n7+QTriRXyXim0mBrTXNeqzVF0KWGgHPeiyViFFrNDw==",
|
||||
"license": "BlueOak-1.0.0",
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/yargs": {
|
||||
"version": "15.4.1",
|
||||
"resolved": "https://registry.npmjs.org/yargs/-/yargs-15.4.1.tgz",
|
||||
|
||||
@@ -38,6 +38,7 @@
|
||||
"sanitize-html": "^2.17.5",
|
||||
"speakeasy": "^2.0.0",
|
||||
"swagger-ui-express": "^5.0.1",
|
||||
"tar": "^7.5.22",
|
||||
"ws": "^8.21.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
|
||||
@@ -427,6 +427,90 @@
|
||||
"validate"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/modules",
|
||||
"handlers": 2,
|
||||
"gates": [
|
||||
"noindex",
|
||||
"requireAuth"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules",
|
||||
"handlers": 4,
|
||||
"gates": [
|
||||
"noindex",
|
||||
"requireAuth",
|
||||
"middleware",
|
||||
"validate"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/modules/:id",
|
||||
"handlers": 5,
|
||||
"gates": [
|
||||
"noindex",
|
||||
"requireAuth",
|
||||
"middleware",
|
||||
"validate"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules/:id/disable",
|
||||
"handlers": 4,
|
||||
"gates": [
|
||||
"noindex",
|
||||
"requireAuth",
|
||||
"middleware",
|
||||
"validate"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules/:id/enable",
|
||||
"handlers": 4,
|
||||
"gates": [
|
||||
"noindex",
|
||||
"requireAuth",
|
||||
"middleware",
|
||||
"validate"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules/:id/purge",
|
||||
"handlers": 4,
|
||||
"gates": [
|
||||
"noindex",
|
||||
"requireAuth",
|
||||
"middleware",
|
||||
"validate"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules/restart",
|
||||
"handlers": 2,
|
||||
"gates": [
|
||||
"noindex",
|
||||
"requireAuth"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/modules/sources",
|
||||
"handlers": 4,
|
||||
"gates": [
|
||||
"noindex",
|
||||
"requireAuth",
|
||||
"middleware",
|
||||
"validate"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/pages",
|
||||
|
||||
@@ -177,6 +177,38 @@
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/moderation/user/:discordId/notes"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/modules"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/modules/:id"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules/:id/disable"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules/:id/enable"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules/:id/purge"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules/restart"
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/modules/sources"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/pages"
|
||||
|
||||
@@ -17,6 +17,23 @@ const getOne = (id) => query(`SELECT ${COLS} FROM installed_modules WHERE id = ?
|
||||
// module must not silently disable it, and re-installing a disabled one must not
|
||||
// silently switch it back on. A brand-new row lands in `installed`, the transient
|
||||
// state the next restart resolves.
|
||||
// Provenance is COALESCEd, and that is the whole difference between this
|
||||
// working and not.
|
||||
//
|
||||
// `lifecycle.boot()` re-records every module it scanned with no source and no
|
||||
// sha256 — a directory placed on the volume by hand genuinely has neither, and
|
||||
// the boot has no way to know where one came from. With a plain
|
||||
// `source = VALUES(source)` that refresh overwrote both columns with NULL on
|
||||
// EVERY boot, so an admin-panel install's provenance survived exactly until the
|
||||
// restart that install asked for. Nothing could catch it before Phase 4: until
|
||||
// then no caller ever passed a non-null value, and lifecycle.js's comment
|
||||
// asserting that "recordInstalled leaves what it is not given" was a description
|
||||
// of an intention rather than of this statement.
|
||||
//
|
||||
// COALESCE makes that comment true: a value overwrites, a NULL leaves what is
|
||||
// there. The cost is that re-placing a DIFFERENT bundle by hand over a row that
|
||||
// was once installed from a URL keeps the old provenance — which is wrong but
|
||||
// stale, and strictly better than the alternative, which was wrong and blank.
|
||||
const upsert = ({ id, name, version, source, sha256 }) =>
|
||||
query(
|
||||
`INSERT INTO installed_modules (id, name, version, source, sha256, state)
|
||||
@@ -24,8 +41,8 @@ const upsert = ({ id, name, version, source, sha256 }) =>
|
||||
ON DUPLICATE KEY UPDATE
|
||||
name = VALUES(name),
|
||||
version = VALUES(version),
|
||||
source = VALUES(source),
|
||||
sha256 = VALUES(sha256)`,
|
||||
source = COALESCE(VALUES(source), source),
|
||||
sha256 = COALESCE(VALUES(sha256), sha256)`,
|
||||
[id, name, version, source ?? null, sha256 ?? null],
|
||||
)
|
||||
|
||||
|
||||
221
server/src/modules/archive.js
Normal file
221
server/src/modules/archive.js
Normal file
@@ -0,0 +1,221 @@
|
||||
// ── The hardened bundle extractor ──────────────────────────────────────────
|
||||
//
|
||||
// Phase 4, slice 1 of docs/website/MODULE_SYSTEM.md §2.7.2. This is the part of
|
||||
// the install path that handles input an attacker chose, and it is separated
|
||||
// from install.js so that it can be tested against crafted archives without a
|
||||
// network, a database or a filesystem layout.
|
||||
//
|
||||
// **The download is not the dangerous part.** An allowlisted host, a declared
|
||||
// sha256 and a size cap between them settle where the bytes came from and that
|
||||
// they are the bytes that were published. Unpacking is different: the ARCHIVE
|
||||
// chooses the filenames, and core writes into a directory bind-mounted from the
|
||||
// host (MODULE_SYSTEM.md §2.5), so an escape is not confined to the container.
|
||||
//
|
||||
// The rule this file follows is **reject, never sanitise.** node-tar will
|
||||
// happily strip a leading `/` and drop a `..` for you, which turns a hostile
|
||||
// archive into a slightly different archive that then gets installed. An archive
|
||||
// that needs correcting is an archive that should not be trusted, so anything on
|
||||
// the list below fails the whole install and nothing is written.
|
||||
//
|
||||
// - an absolute path, POSIX (`/etc/…`) or Windows (`C:\…`, `\\server\…`)
|
||||
// - any `..` segment, anywhere
|
||||
// - anything that is not a regular file or a directory — so no symlinks, no
|
||||
// hardlinks, no devices, no FIFOs. A module bundle has no legitimate use for
|
||||
// any of them, and every one is a documented escape primitive
|
||||
// - more than one top-level entry
|
||||
// - more entries, or more unpacked bytes, than the caps below
|
||||
//
|
||||
// That third rule deserves its own note, because it is why the `tar` dependency
|
||||
// is pinned forward rather than merely present: **the majority of node-tar's
|
||||
// published advisories are hardlink or symlink path traversal**, several of them
|
||||
// through interpretation differences in PAX and GNU long-name headers rather
|
||||
// than through anything the calling code did wrong. Refusing those entry types
|
||||
// outright means this extractor is not relying on the library to get their
|
||||
// containment right. The remaining advisories are parser denial-of-service, and
|
||||
// those are what the caps and the two-pass shape address.
|
||||
//
|
||||
// Two passes, deliberately: `inspect()` reads the archive and decides, and only
|
||||
// an archive that survived that is handed to `extract()`. A `filter` callback
|
||||
// during extraction cannot reject — by the time it is asked about entry 400, the
|
||||
// first 399 are already on disk.
|
||||
|
||||
const fs = require('fs')
|
||||
const fsp = require('fs/promises')
|
||||
const path = require('path')
|
||||
const tar = require('tar')
|
||||
|
||||
// Caps. Generous against a real bundle (module-uo's is a few megabytes, a few
|
||||
// hundred files) and small enough that a decompression bomb is refused rather
|
||||
// than paged in. Both are checked DURING the inspect pass, not after it, so a
|
||||
// hostile archive stops being read at the limit instead of at its end.
|
||||
const MAX_BYTES = 128 * 1024 * 1024
|
||||
const MAX_ENTRIES = 20000
|
||||
|
||||
// tar entry types worth naming. node-tar reports these as strings; anything not
|
||||
// in this set is refused, including the ones no one has thought of, which is the
|
||||
// point of an allowlist here rather than a list of the types known to be bad.
|
||||
const ALLOWED_TYPES = new Set(['File', 'Directory'])
|
||||
|
||||
class ArchiveError extends Error {
|
||||
constructor(message) {
|
||||
super(message)
|
||||
this.name = 'ArchiveError'
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Is this entry path safe to unpack anywhere?
|
||||
*
|
||||
* Returns a reason string, or null when the path is fine. Written as a reason
|
||||
* rather than a boolean because the operator pasting a URL needs to be told what
|
||||
* was wrong with what they were served, and "the archive is invalid" is not that.
|
||||
*/
|
||||
function pathProblem(entryPath) {
|
||||
const p = String(entryPath)
|
||||
|
||||
// NUL is not a path character. node-tar has had uncaught-exception advisories
|
||||
// for NUL bytes inside PAX records, so this is checked here rather than left
|
||||
// to the parser.
|
||||
if (p.includes('\0')) return 'an entry path contains a NUL byte'
|
||||
|
||||
// A backslash is a path separator on the platform this may be unpacked on, so
|
||||
// an entry that contains one is not the single path component it looks like.
|
||||
if (p.includes('\\')) return `an entry path contains a backslash: "${p}"`
|
||||
|
||||
if (p.startsWith('/')) return `an entry path is absolute: "${p}"`
|
||||
// Drive-relative and UNC. Both are the subject of their own node-tar
|
||||
// advisories, and neither is refused by a leading-slash check.
|
||||
if (/^[a-zA-Z]:/.test(p)) return `an entry path is drive-absolute: "${p}"`
|
||||
|
||||
const segments = p.split('/')
|
||||
if (segments.includes('..')) return `an entry path escapes upward: "${p}"`
|
||||
|
||||
return null
|
||||
}
|
||||
|
||||
/**
|
||||
* Read an archive without unpacking it, and decide whether it may be unpacked.
|
||||
*
|
||||
* @param {string} file absolute path to the .tar.gz
|
||||
* @returns {Promise<{root: string, entries: number, bytes: number}>} the single
|
||||
* top-level directory name, and what is inside it.
|
||||
* @throws {ArchiveError} on anything in this file's header list.
|
||||
*/
|
||||
async function inspect(file) {
|
||||
const roots = new Set()
|
||||
let entries = 0
|
||||
let bytes = 0
|
||||
// The first problem found, kept rather than thrown from inside the callback:
|
||||
// throwing out of `onentry` escapes through the parser's stream machinery and
|
||||
// arrives as an unhelpful wrapped error, when it arrives at all.
|
||||
let problem = null
|
||||
|
||||
const note = (message) => {
|
||||
if (!problem) problem = message
|
||||
}
|
||||
|
||||
await tar.t({
|
||||
file,
|
||||
onentry(entry) {
|
||||
if (problem) return
|
||||
|
||||
entries += 1
|
||||
if (entries > MAX_ENTRIES) {
|
||||
note(`the archive has more than ${MAX_ENTRIES} entries`)
|
||||
return
|
||||
}
|
||||
|
||||
if (!ALLOWED_TYPES.has(entry.type)) {
|
||||
// The message names the type because "symbolic link" is a much more
|
||||
// useful thing for an operator to read than "invalid entry".
|
||||
note(`the archive contains a ${entry.type} ("${entry.path}") — a module bundle may only contain files and directories`)
|
||||
return
|
||||
}
|
||||
|
||||
const bad = pathProblem(entry.path)
|
||||
if (bad) {
|
||||
note(bad)
|
||||
return
|
||||
}
|
||||
|
||||
// A negative or absurd size is a parser-confusion primitive in its own
|
||||
// right; clamping at zero keeps the running total honest.
|
||||
bytes += Math.max(0, Number(entry.size) || 0)
|
||||
if (bytes > MAX_BYTES) {
|
||||
note(`the archive unpacks to more than ${Math.round(MAX_BYTES / 1024 / 1024)} MB`)
|
||||
return
|
||||
}
|
||||
|
||||
const [root] = String(entry.path).split('/')
|
||||
if (root) roots.add(root)
|
||||
},
|
||||
})
|
||||
|
||||
if (problem) throw new ArchiveError(problem)
|
||||
if (entries === 0) throw new ArchiveError('the archive is empty')
|
||||
if (roots.size !== 1) {
|
||||
// A bundle is one directory. More than one top-level entry means `strip: 1`
|
||||
// below would silently merge or discard things, and a bundle that needs
|
||||
// interpreting is not a bundle.
|
||||
throw new ArchiveError(
|
||||
`the archive must contain exactly one top-level directory, found ${roots.size}` +
|
||||
(roots.size > 1 ? ` (${[...roots].slice(0, 4).join(', ')})` : ''),
|
||||
)
|
||||
}
|
||||
|
||||
return { root: [...roots][0], entries, bytes }
|
||||
}
|
||||
|
||||
/**
|
||||
* Unpack an inspected archive into `dest`, stripping its single top level.
|
||||
*
|
||||
* The top-level directory is stripped rather than kept, because its name belongs
|
||||
* to whoever published the bundle — module-uo's release workflow packs
|
||||
* `module-uo-<version>/`, not `uo/` — while the directory it lands in is core's
|
||||
* decision and has to be the module id the loader scans for. What is inside the
|
||||
* bundle is the module; what the wrapper is called is packaging.
|
||||
*
|
||||
* `dest` must not exist. Callers unpack to a temporary directory and move it
|
||||
* into place, so a failure halfway through leaves nothing for the next boot's
|
||||
* scan to find.
|
||||
*/
|
||||
async function extract(file, dest) {
|
||||
if (fs.existsSync(dest)) throw new ArchiveError(`extraction target already exists: ${dest}`)
|
||||
await fsp.mkdir(dest, { recursive: true })
|
||||
|
||||
await tar.x({
|
||||
file,
|
||||
cwd: dest,
|
||||
strip: 1,
|
||||
// Belt and braces on top of inspect(): the library's own refusal to write
|
||||
// outside cwd, and its refusal to overwrite through a link. Neither is
|
||||
// load-bearing here — inspect() has already rejected every entry that could
|
||||
// exercise them — but a second lock on a door that is already locked costs
|
||||
// nothing, and this is the door.
|
||||
preservePaths: false,
|
||||
// Reject rather than warn on anything the parser itself objects to.
|
||||
strict: true,
|
||||
})
|
||||
|
||||
return dest
|
||||
}
|
||||
|
||||
/**
|
||||
* Inspect, then extract. The only entry point install.js uses.
|
||||
*/
|
||||
async function unpack(file, dest) {
|
||||
const stats = await inspect(file)
|
||||
await extract(file, dest)
|
||||
return stats
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
ArchiveError,
|
||||
inspect,
|
||||
extract,
|
||||
unpack,
|
||||
pathProblem,
|
||||
MAX_BYTES,
|
||||
MAX_ENTRIES,
|
||||
ALLOWED_TYPES,
|
||||
}
|
||||
220
server/src/modules/declared.js
Normal file
220
server/src/modules/declared.js
Normal file
@@ -0,0 +1,220 @@
|
||||
// ── The declared module set ────────────────────────────────────────────────
|
||||
//
|
||||
// Phase 4, slice 3 of docs/website/MODULE_SYSTEM.md §2.7.2 — decision 4. A
|
||||
// compose-managed host is not driven by clicking: it declares which modules it
|
||||
// runs, in the file it already edits and version-controls, and the container
|
||||
// arrives at that set by itself.
|
||||
//
|
||||
// MODULES: uo@0.3.0=https://<host>/…/module-uo-0.3.0.json
|
||||
//
|
||||
// One entry per module, `<id>@<version>=<install manifest URL>`, separated by
|
||||
// whitespace or commas. The id and the version are written out rather than left
|
||||
// to be discovered inside the manifest for one reason: **the no-op case must not
|
||||
// need the network.** A module already unpacked at the declared version is
|
||||
// answered by reading its own `module.json` off the volume, so a restart with
|
||||
// the network down brings the site up exactly as it was. Only a module that is
|
||||
// missing, or unpacked at some other version, reaches out — and it reaches out
|
||||
// through modules/install.js, the same fetch-verify-unpack path the admin panel
|
||||
// uses, under the same host allowlist.
|
||||
//
|
||||
// Three things this file deliberately does not do:
|
||||
//
|
||||
// - **It does not decide whether a module RUNS.** Resolution owns what is on
|
||||
// the volume; `installed_modules` owns whether a mounted module answers. An
|
||||
// admin who uninstalls a declared module gets its directory back at the next
|
||||
// boot with the row still `disabled`, so it stays off until they enable it.
|
||||
// The two never fight because they are not answering the same question.
|
||||
// - **It does not fail a boot.** A module publisher's host being unreachable
|
||||
// must not take a shard's website down with it; core is built to serve with
|
||||
// a module absent (§1.6). Every failure is logged loudly and kept for the
|
||||
// admin screen, and the site comes up.
|
||||
// - **It does not mount anything.** §1.12 makes the volume the mounting source
|
||||
// of truth, read once at require time — which is why this runs before
|
||||
// `require('./app')` in server.js and not from inside it.
|
||||
//
|
||||
// It runs on every boot, not only in Docker: a bare `npm start` with MODULES set
|
||||
// resolves the same way. The Docker path is the reason it exists, not a special
|
||||
// case in it.
|
||||
|
||||
const fs = require('fs')
|
||||
const path = require('path')
|
||||
|
||||
const install = require('./install')
|
||||
|
||||
const log = require('../utils/logger')('modules')
|
||||
|
||||
// The variable an operator sets. Named next to MODULES_DIR, which is the other
|
||||
// half of the same story: one says where modules live, the other says which.
|
||||
const VAR = 'MODULES'
|
||||
|
||||
// Same id rule the loader enforces when scanning and install.js enforces when
|
||||
// placing, restated here so a declaration cannot name something neither would
|
||||
// accept.
|
||||
const ID = /^[a-z][a-z0-9-]{1,31}$/
|
||||
|
||||
// The outcome of the last resolution, in memory, for the admin screen. Not a
|
||||
// database row: a declaration is a fact about this process's environment, and
|
||||
// writing it down would put it in front of the boot reconcile, which resets
|
||||
// every non-disabled row (§2.4). The screen merges it as a fourth source
|
||||
// alongside the row, the loader and the volume.
|
||||
let results = []
|
||||
|
||||
/**
|
||||
* Parse the declaration into entries.
|
||||
*
|
||||
* Malformed entries are collected rather than thrown: one operator typo should
|
||||
* cost that module, not every module on the host. A duplicate id keeps the
|
||||
* first — there is no sensible way to run two versions of one module, and
|
||||
* silently preferring the last would make the outcome depend on the order of a
|
||||
* list nobody reads as ordered.
|
||||
*
|
||||
* @param {string} value the raw variable
|
||||
* @returns {{entries: Array<{id,version,url}>, errors: string[]}}
|
||||
*/
|
||||
function parse(value) {
|
||||
const entries = []
|
||||
const errors = []
|
||||
const seen = new Set()
|
||||
|
||||
for (const token of String(value || '').split(/[,\s]+/).filter(Boolean)) {
|
||||
const at = token.indexOf('@')
|
||||
const eq = token.indexOf('=')
|
||||
if (at < 1 || eq < at + 2) {
|
||||
errors.push(`"${token}" is not <id>@<version>=<url>`)
|
||||
continue
|
||||
}
|
||||
const id = token.slice(0, at)
|
||||
const version = token.slice(at + 1, eq)
|
||||
const url = token.slice(eq + 1)
|
||||
|
||||
if (!ID.test(id)) {
|
||||
errors.push(`"${token}" names an invalid module id "${id}"`)
|
||||
continue
|
||||
}
|
||||
if (!url) {
|
||||
errors.push(`"${token}" has no install manifest URL`)
|
||||
continue
|
||||
}
|
||||
if (seen.has(id)) {
|
||||
errors.push(`"${id}" is declared more than once — keeping the first`)
|
||||
continue
|
||||
}
|
||||
seen.add(id)
|
||||
entries.push({ id, version, url })
|
||||
}
|
||||
|
||||
return { entries, errors }
|
||||
}
|
||||
|
||||
/**
|
||||
* The version currently unpacked on the volume, or null.
|
||||
*
|
||||
* Read straight out of the module's own `module.json`, which is the same file
|
||||
* the loader trusts for the same fact — and never from `installed_modules`,
|
||||
* because the row records what was installed and this has to answer what is
|
||||
* actually there. An unreadable manifest counts as absent: whatever is in that
|
||||
* directory, it is not a module at the declared version.
|
||||
*/
|
||||
function installedVersion(id) {
|
||||
try {
|
||||
const manifest = JSON.parse(
|
||||
fs.readFileSync(path.join(install.moduleDir(id), 'module.json'), 'utf8'),
|
||||
)
|
||||
return manifest && manifest.version ? String(manifest.version) : null
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Bring the volume in line with the declaration.
|
||||
*
|
||||
* Never throws and never rejects. Returns one outcome per declared entry, and
|
||||
* remembers them for `state()`.
|
||||
*
|
||||
* @param {object} args
|
||||
* @param {string} [args.value] the raw variable (defaults to the environment)
|
||||
* @param {string[]} args.hosts the install allowlist, already parsed
|
||||
* @param {object} args.model modules.model, for recording provenance
|
||||
* @param {object} [args.installImpl] injection seam, as everywhere else here
|
||||
* @returns {Promise<Array<{id,version,url,action,message}>>}
|
||||
*/
|
||||
async function resolve({ value = process.env[VAR], hosts = [], model, installImpl = install } = {}) {
|
||||
const { entries, errors } = parse(value)
|
||||
results = []
|
||||
|
||||
for (const message of errors) log.error(`${VAR}: ${message}`)
|
||||
|
||||
if (!entries.length) return results
|
||||
|
||||
log.info(`${VAR} declares ${entries.length} module(s)`, {
|
||||
modules: entries.map((e) => `${e.id}@${e.version}`).join(' '),
|
||||
})
|
||||
|
||||
for (const entry of entries) {
|
||||
const present = installedVersion(entry.id)
|
||||
if (present === entry.version) {
|
||||
// The offline path, and the common one: nothing is fetched, nothing is
|
||||
// written, and a host with no route to the internet boots unchanged.
|
||||
log.info(`module "${entry.id}" is already at the declared version ${entry.version}`)
|
||||
results.push({ ...entry, action: 'noop', message: null })
|
||||
continue
|
||||
}
|
||||
|
||||
try {
|
||||
// `expect` is the declaration itself, handed down so install.js can refuse
|
||||
// a URL that resolves to another module or another version while it is
|
||||
// still only a manifest — a check made after the unpack would be made with
|
||||
// the undeclared module already on the volume.
|
||||
const result = await installImpl.install({
|
||||
url: entry.url,
|
||||
hosts,
|
||||
expect: { id: entry.id, version: entry.version },
|
||||
})
|
||||
|
||||
// Provenance, written exactly as the admin route writes it — the whole
|
||||
// point of resolving in-process rather than from a script that cannot
|
||||
// reach the database. A module installed by the compose file and one
|
||||
// installed by an admin are then indistinguishable on the screen, which
|
||||
// is what makes this one feature and not two.
|
||||
if (model) {
|
||||
await model.recordInstalled({
|
||||
id: result.id,
|
||||
name: result.name,
|
||||
version: result.version,
|
||||
source: entry.url,
|
||||
sha256: result.sha256,
|
||||
})
|
||||
}
|
||||
|
||||
log.warn(`installed declared module "${entry.id}" v${entry.version}`, {
|
||||
from: present || 'nothing',
|
||||
source: entry.url,
|
||||
sha256: result.sha256,
|
||||
})
|
||||
results.push({ ...entry, action: 'installed', message: null })
|
||||
} catch (err) {
|
||||
// Loud, and then onward. The site serves; this module does not, or serves
|
||||
// the version that was already there.
|
||||
log.error(
|
||||
`could not resolve declared module "${entry.id}@${entry.version}": ${err.message}` +
|
||||
(present ? ` — leaving version ${present} in place` : ''),
|
||||
)
|
||||
results.push({ ...entry, action: 'failed', message: err.message })
|
||||
}
|
||||
}
|
||||
|
||||
return results
|
||||
}
|
||||
|
||||
/** What the last resolution decided, for the admin screen. */
|
||||
function state() {
|
||||
return results.map((r) => ({ ...r }))
|
||||
}
|
||||
|
||||
/** Test seam: forget the last resolution. */
|
||||
function reset() {
|
||||
results = []
|
||||
}
|
||||
|
||||
module.exports = { VAR, parse, installedVersion, resolve, state, reset }
|
||||
450
server/src/modules/install.js
Normal file
450
server/src/modules/install.js
Normal file
@@ -0,0 +1,450 @@
|
||||
// ── Installing and removing a module bundle ────────────────────────────────
|
||||
//
|
||||
// Phase 4, slice 1 of docs/website/MODULE_SYSTEM.md §2.7.2 — the consumer half
|
||||
// of a release the module's own CI already publishes. Nothing here touches the
|
||||
// database or the loader: this file moves bytes onto the volume and off it, and
|
||||
// the caller (router/v1/admin/modules.controller.js) writes down what happened.
|
||||
//
|
||||
// The shape of an install, and why it is this shape:
|
||||
//
|
||||
// 1. The operator pastes the URL of a release's install manifest — a small
|
||||
// JSON document naming the artifact, its sha256 and its size (decision 2).
|
||||
// There is no catalog, because a catalog would make core's release cadence
|
||||
// decide which modules can exist.
|
||||
// 2. Every URL fetched — the manifest, the artifact, and every redirect hop —
|
||||
// is checked against the admin-managed host allowlist (decision 6). That is
|
||||
// what keeps a pasted URL from also being an SSRF primitive.
|
||||
// 3. The artifact is streamed to a temporary file under a byte cap, hashed as
|
||||
// it arrives, and compared against the manifest's `sha256`. The hash is the
|
||||
// trust anchor; releases are unsigned and say so.
|
||||
// 4. modules/archive.js inspects the archive in full before a single byte is
|
||||
// unpacked, and only then unpacks it — into a TEMPORARY directory.
|
||||
// 5. The unpacked tree's own `module.json` must agree with the manifest about
|
||||
// what it is. A manifest that promises `uo` and delivers something else is
|
||||
// refused rather than installed under the name it was promised.
|
||||
// 6. Only then is anything moved into `modules/<id>/`, and the directory it
|
||||
// replaces is kept aside until the move has succeeded.
|
||||
//
|
||||
// **Nothing is written into the modules directory until every check has passed**,
|
||||
// and that is not belt-and-braces. Verified against node-tar 7.5.22 while writing
|
||||
// this: extracting an archive whose fourth member escapes upward throws — and
|
||||
// leaves the first three members on disk. A loader scans that directory at
|
||||
// require time on the next boot; a half-unpacked module is a module.
|
||||
//
|
||||
// One thing this file deliberately does NOT do: mount anything. Installing puts
|
||||
// a directory on the volume, and §1.12 means the volume is read at require time,
|
||||
// so the module appears when the process restarts. The admin screen offers that
|
||||
// restart (decision 1); it is not implied here.
|
||||
|
||||
const crypto = require('crypto')
|
||||
const fs = require('fs')
|
||||
const fsp = require('fs/promises')
|
||||
const path = require('path')
|
||||
const { Readable, Transform } = require('stream')
|
||||
const { pipeline } = require('stream/promises')
|
||||
|
||||
const archive = require('./archive')
|
||||
const loader = require('./loader')
|
||||
|
||||
const log = require('../utils/logger')('modules')
|
||||
|
||||
// An install manifest is a small JSON document. A megabyte of it is not a
|
||||
// manifest, and reading it into memory unbounded is the one place this file
|
||||
// would otherwise trust a remote length.
|
||||
const MAX_MANIFEST_BYTES = 256 * 1024
|
||||
// The artifact cap is the archive's own unpacked cap — a compressed bundle
|
||||
// larger than what it is allowed to unpack to has nothing to offer.
|
||||
const MAX_ARTIFACT_BYTES = archive.MAX_BYTES
|
||||
const FETCH_TIMEOUT_MS = 30000
|
||||
// Gitea serves a release asset through at least one redirect. Following them is
|
||||
// necessary; following them blindly is how an allowlist gets bypassed, so each
|
||||
// hop is re-checked and the chain is bounded.
|
||||
const MAX_REDIRECTS = 5
|
||||
|
||||
// Same id rule the loader enforces when scanning, restated rather than imported
|
||||
// so an install cannot put a directory on the volume that the loader would then
|
||||
// refuse to look at.
|
||||
const ID = /^[a-z][a-z0-9-]{1,31}$/
|
||||
const SHA256 = /^[0-9a-f]{64}$/
|
||||
|
||||
class InstallError extends Error {
|
||||
constructor(message, { status = 400 } = {}) {
|
||||
super(message)
|
||||
this.name = 'InstallError'
|
||||
// Carried so the controller can answer 400 for "your URL is wrong" and 502
|
||||
// for "the host you named misbehaved" without re-deriving it from the text.
|
||||
this.status = status
|
||||
}
|
||||
}
|
||||
|
||||
// ── The allowlist ──────────────────────────────────────────────────────────
|
||||
|
||||
// The settings row the allowlist lives in (decision 6): seeded from
|
||||
// MODULE_SOURCE_HOSTS on a fresh install and admin-managed from then on. The KEY
|
||||
// lives here rather than in the admin controller because it is now read from two
|
||||
// places — the controller, and the boot-time resolution of the declared module
|
||||
// set (modules/declared.js), which has no route and no request.
|
||||
const HOSTS_SETTING = 'module_source_hosts'
|
||||
|
||||
/**
|
||||
* Parse the stored allowlist setting into hostnames.
|
||||
*
|
||||
* Comma or whitespace separated, case-insensitive, empty entries dropped. An
|
||||
* empty list means nothing may be installed from anywhere — a refusal, never a
|
||||
* wildcard. That direction matters: a setting an admin accidentally blanks
|
||||
* should stop installs, not permit every host on the internet.
|
||||
*/
|
||||
function parseHosts(value) {
|
||||
return String(value || '')
|
||||
.split(/[,\s]+/)
|
||||
.map((h) => h.trim().toLowerCase())
|
||||
.filter(Boolean)
|
||||
}
|
||||
|
||||
/**
|
||||
* Check one URL against the allowlist, and return it parsed.
|
||||
*
|
||||
* `https` only. A plaintext fetch of code this process is going to execute is
|
||||
* not something an operator should be able to opt into by typing a URL, and the
|
||||
* sha256 does not help — whoever can rewrite the artifact in flight can rewrite
|
||||
* the manifest that declares its hash.
|
||||
*/
|
||||
function checkUrl(raw, hosts) {
|
||||
let url
|
||||
try {
|
||||
url = new URL(String(raw))
|
||||
} catch {
|
||||
throw new InstallError(`"${raw}" is not a valid URL`)
|
||||
}
|
||||
if (url.protocol !== 'https:') {
|
||||
throw new InstallError(`only https URLs may be installed from (got "${url.protocol}")`)
|
||||
}
|
||||
if (!hosts.length) {
|
||||
throw new InstallError(
|
||||
'no module source hosts are allowed — set one in Admin → Modules before installing',
|
||||
)
|
||||
}
|
||||
if (!hosts.includes(url.hostname.toLowerCase())) {
|
||||
throw new InstallError(
|
||||
`"${url.hostname}" is not an allowed module source host (allowed: ${hosts.join(', ')})`,
|
||||
)
|
||||
}
|
||||
return url
|
||||
}
|
||||
|
||||
// ── Fetching ───────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* GET a URL, following redirects by hand so every hop is re-checked.
|
||||
*
|
||||
* `fetch`'s own redirect following would take the first hop off the allowlist
|
||||
* and the rest wherever it was pointed, which is exactly the hole the allowlist
|
||||
* exists to close.
|
||||
*
|
||||
* `fetchImpl` is the same injection seam `replayFragments({query})` and
|
||||
* `lifecycle.boot({model})` use, and it exists for the same reason: the rules
|
||||
* this file enforces — https only, allowlisted host, allowlisted REDIRECT host,
|
||||
* bounded body, matching hash — are all decisions about a response, and testing
|
||||
* them against a real TLS server would mean testing Node's certificate handling
|
||||
* instead. Production never passes it.
|
||||
*
|
||||
* @returns {Promise<Response>} a response whose body has not been read.
|
||||
*/
|
||||
async function get(rawUrl, hosts, fetchImpl = fetch) {
|
||||
let url = checkUrl(rawUrl, hosts)
|
||||
|
||||
for (let hop = 0; hop <= MAX_REDIRECTS; hop += 1) {
|
||||
let res
|
||||
try {
|
||||
res = await fetchImpl(url, {
|
||||
redirect: 'manual',
|
||||
signal: AbortSignal.timeout(FETCH_TIMEOUT_MS),
|
||||
headers: { accept: '*/*' },
|
||||
})
|
||||
} catch (err) {
|
||||
throw new InstallError(`could not reach ${url.hostname}: ${err.message}`, { status: 502 })
|
||||
}
|
||||
|
||||
if (res.status >= 300 && res.status < 400) {
|
||||
const location = res.headers.get('location')
|
||||
if (!location) throw new InstallError(`${url.hostname} redirected without a location`, { status: 502 })
|
||||
// Resolved against the current URL, then re-checked — a relative redirect
|
||||
// is normal and a cross-host one is the thing being guarded against.
|
||||
url = checkUrl(new URL(location, url).toString(), hosts)
|
||||
continue
|
||||
}
|
||||
|
||||
if (!res.ok) {
|
||||
throw new InstallError(`${url.href} returned ${res.status}`, { status: 502 })
|
||||
}
|
||||
return res
|
||||
}
|
||||
|
||||
throw new InstallError(`too many redirects (more than ${MAX_REDIRECTS})`, { status: 502 })
|
||||
}
|
||||
|
||||
/** Read a bounded response body as text. */
|
||||
async function readText(res, cap, what) {
|
||||
const declared = Number(res.headers.get('content-length'))
|
||||
if (Number.isFinite(declared) && declared > cap) {
|
||||
throw new InstallError(`${what} is larger than ${cap} bytes`)
|
||||
}
|
||||
const buf = Buffer.from(await res.arrayBuffer())
|
||||
// Checked again after reading: content-length is the server's claim, not a
|
||||
// limit it is obliged to honour.
|
||||
if (buf.length > cap) throw new InstallError(`${what} is larger than ${cap} bytes`)
|
||||
return buf.toString('utf8')
|
||||
}
|
||||
|
||||
/**
|
||||
* Stream a response body to a file, hashing as it goes and stopping at the cap.
|
||||
*
|
||||
* The hash is computed from the bytes that were written rather than by re-reading
|
||||
* the file, so there is no window in which the file could differ from what was
|
||||
* verified.
|
||||
*/
|
||||
async function download(res, dest, cap) {
|
||||
const hash = crypto.createHash('sha256')
|
||||
let bytes = 0
|
||||
|
||||
const body = Readable.fromWeb(res.body)
|
||||
const counter = new Transform({
|
||||
transform(chunk, _enc, cb) {
|
||||
bytes += chunk.length
|
||||
if (bytes > cap) {
|
||||
cb(new InstallError(`the artifact is larger than ${Math.round(cap / 1024 / 1024)} MB`))
|
||||
return
|
||||
}
|
||||
hash.update(chunk)
|
||||
cb(null, chunk)
|
||||
},
|
||||
})
|
||||
|
||||
await pipeline(body, counter, fs.createWriteStream(dest))
|
||||
return { sha256: hash.digest('hex'), bytes }
|
||||
}
|
||||
|
||||
// ── The install manifest ───────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Fetch and validate a release's install manifest.
|
||||
*
|
||||
* The shape is the one module-uo's release workflow already writes:
|
||||
* `{schema, id, name, version, coreApi, artifact, url, sha256, size}`. Only the
|
||||
* fields this side needs are required — a module publisher may carry more.
|
||||
*/
|
||||
async function fetchManifest(manifestUrl, hosts, fetchImpl = fetch) {
|
||||
const res = await get(manifestUrl, hosts, fetchImpl)
|
||||
const text = await readText(res, MAX_MANIFEST_BYTES, 'the install manifest')
|
||||
|
||||
let manifest
|
||||
try {
|
||||
manifest = JSON.parse(text)
|
||||
} catch (err) {
|
||||
throw new InstallError(`the install manifest is not valid JSON: ${err.message}`)
|
||||
}
|
||||
if (!manifest || typeof manifest !== 'object' || Array.isArray(manifest)) {
|
||||
throw new InstallError('the install manifest is not a JSON object')
|
||||
}
|
||||
|
||||
const { id, name, version, sha256 } = manifest
|
||||
if (!ID.test(String(id || ''))) {
|
||||
throw new InstallError(`the install manifest has an invalid module id: ${JSON.stringify(id)}`)
|
||||
}
|
||||
if (!name || !version) throw new InstallError('the install manifest is missing name or version')
|
||||
if (!SHA256.test(String(sha256 || '').toLowerCase())) {
|
||||
throw new InstallError('the install manifest has no valid sha256 for its artifact')
|
||||
}
|
||||
|
||||
// `url` is the absolute artifact URL the publisher wrote; `artifact` is its
|
||||
// filename. Prefer the URL, fall back to resolving the filename beside the
|
||||
// manifest — which is where a release's assets sit — so a manifest that
|
||||
// travelled without its absolute URL still installs.
|
||||
const artifactUrl = new URL(manifest.url || manifest.artifact || '', manifestUrl).toString()
|
||||
|
||||
return {
|
||||
id: String(id),
|
||||
name: String(name),
|
||||
version: String(version),
|
||||
sha256: String(sha256).toLowerCase(),
|
||||
size: Number(manifest.size) || null,
|
||||
artifactUrl,
|
||||
coreApi: manifest.coreApi ? String(manifest.coreApi) : null,
|
||||
}
|
||||
}
|
||||
|
||||
// ── Paths on the volume ────────────────────────────────────────────────────
|
||||
|
||||
/** Where a module with this id lives. Rejects an id that is not one. */
|
||||
function moduleDir(id) {
|
||||
if (!ID.test(String(id || ''))) throw new InstallError(`invalid module id: ${JSON.stringify(id)}`)
|
||||
return path.join(loader.dir(), String(id))
|
||||
}
|
||||
|
||||
/** Is there a directory for this module on the volume right now? */
|
||||
function isInstalled(id) {
|
||||
try {
|
||||
return fs.statSync(moduleDir(id)).isDirectory()
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The absolute path of a module's `purge.sql`, or null.
|
||||
*
|
||||
* Read from the module's own `module.json` on disk rather than from the loader,
|
||||
* because purge has to work for a module that never loaded — a `startup_failed`
|
||||
* one is exactly when an operator wants its tables gone.
|
||||
*/
|
||||
function purgeFile(id) {
|
||||
try {
|
||||
const manifest = JSON.parse(fs.readFileSync(path.join(moduleDir(id), 'module.json'), 'utf8'))
|
||||
if (!manifest.purge) return null
|
||||
const file = path.resolve(moduleDir(id), manifest.purge)
|
||||
// The same containment check the loader applies to `client.entry`: a
|
||||
// manifest may not point core at a file outside the module.
|
||||
if (!file.startsWith(moduleDir(id) + path.sep)) return null
|
||||
return fs.existsSync(file) ? file : null
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
// ── Install ────────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Download, verify, unpack and install a module from an install-manifest URL.
|
||||
*
|
||||
* Never leaves a partial module on the volume: everything happens under a
|
||||
* scratch directory that is removed on any failure, and the move into place is
|
||||
* the last step.
|
||||
*
|
||||
* `expect` is what the CALLER was promised, as opposed to what the manifest
|
||||
* promises about itself — the declared module set (modules/declared.js) pins an
|
||||
* id and a version in the environment, and a URL that resolves to something else
|
||||
* has to be refused rather than installed. Checked against the manifest, before
|
||||
* a byte is downloaded: catching it after the unpack would mean the undeclared
|
||||
* module is already on the volume when the objection is raised. The admin panel
|
||||
* passes nothing, because there a URL is the whole of what was asked for.
|
||||
*
|
||||
* @param {object} args
|
||||
* @param {string} args.url the install manifest URL the admin pasted
|
||||
* @param {string[]} args.hosts the allowlist, already parsed
|
||||
* @param {{id?: string, version?: string}} [args.expect] what the caller pinned
|
||||
* @returns {Promise<{id,name,version,sha256,source,bytes,replaced}>}
|
||||
*/
|
||||
async function install({ url, hosts, expect = null, fetchImpl = fetch }) {
|
||||
const manifest = await fetchManifest(url, hosts, fetchImpl)
|
||||
|
||||
if (expect && expect.id && manifest.id !== expect.id) {
|
||||
throw new InstallError(
|
||||
`that URL installs the module "${manifest.id}", but "${expect.id}" was asked for`,
|
||||
)
|
||||
}
|
||||
if (expect && expect.version && manifest.version !== expect.version) {
|
||||
throw new InstallError(
|
||||
`that URL installs ${manifest.id} v${manifest.version}, but v${expect.version} was asked for`,
|
||||
)
|
||||
}
|
||||
|
||||
const target = moduleDir(manifest.id)
|
||||
const scratch = await fsp.mkdtemp(path.join(loader.dir(), `.install-${manifest.id}-`))
|
||||
const tarball = path.join(scratch, 'bundle.tar.gz')
|
||||
const unpacked = path.join(scratch, 'unpacked')
|
||||
|
||||
try {
|
||||
const res = await get(manifest.artifactUrl, hosts, fetchImpl)
|
||||
const { sha256, bytes } = await download(res, tarball, MAX_ARTIFACT_BYTES)
|
||||
|
||||
if (sha256 !== manifest.sha256) {
|
||||
// The whole trust model in one comparison. Deliberately does not name the
|
||||
// computed hash in a way that reads like a value to copy into the
|
||||
// manifest — a mismatch means stop, not reconcile.
|
||||
throw new InstallError(
|
||||
`the downloaded artifact does not match the sha256 in the install manifest — refusing to install`,
|
||||
)
|
||||
}
|
||||
if (manifest.size && bytes !== manifest.size) {
|
||||
throw new InstallError(`the artifact is ${bytes} bytes but the manifest declares ${manifest.size}`)
|
||||
}
|
||||
|
||||
await archive.unpack(tarball, unpacked)
|
||||
|
||||
// What the bundle says it is, checked against what the manifest promised.
|
||||
let inner
|
||||
try {
|
||||
inner = JSON.parse(await fsp.readFile(path.join(unpacked, 'module.json'), 'utf8'))
|
||||
} catch {
|
||||
throw new InstallError('the bundle has no readable module.json at its root')
|
||||
}
|
||||
if (inner.id !== manifest.id) {
|
||||
throw new InstallError(
|
||||
`the bundle declares module id "${inner.id}" but the install manifest promised "${manifest.id}"`,
|
||||
)
|
||||
}
|
||||
if (inner.version !== manifest.version) {
|
||||
throw new InstallError(
|
||||
`the bundle declares version "${inner.version}" but the install manifest promised "${manifest.version}"`,
|
||||
)
|
||||
}
|
||||
|
||||
// The swap. The outgoing directory is moved aside rather than deleted first,
|
||||
// so a failed rename leaves the previous version recoverable instead of
|
||||
// leaving no module at all — the same reasoning the installer repo applies
|
||||
// to a ServUO tree.
|
||||
const replaced = fs.existsSync(target)
|
||||
const aside = `${target}.replaced-${Date.now()}`
|
||||
if (replaced) await fsp.rename(target, aside)
|
||||
try {
|
||||
await fsp.rename(unpacked, target)
|
||||
} catch (err) {
|
||||
if (replaced) await fsp.rename(aside, target).catch(() => {})
|
||||
throw err
|
||||
}
|
||||
if (replaced) await fsp.rm(aside, { recursive: true, force: true })
|
||||
|
||||
log.info(`installed module "${manifest.id}" v${manifest.version}`, {
|
||||
source: url,
|
||||
sha256: manifest.sha256,
|
||||
bytes,
|
||||
replaced,
|
||||
})
|
||||
|
||||
return { ...manifest, source: url, bytes, replaced }
|
||||
} finally {
|
||||
await fsp.rm(scratch, { recursive: true, force: true }).catch(() => {})
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove a module's directory from the volume.
|
||||
*
|
||||
* Returns whether there was one. Does not touch the database, does not run
|
||||
* purge.sql, and does not stop the running module — the controller sequences
|
||||
* all three, because the order matters and only it knows what the operator
|
||||
* asked for.
|
||||
*/
|
||||
async function removeDir(id) {
|
||||
const dir = moduleDir(id)
|
||||
if (!fs.existsSync(dir)) return false
|
||||
await fsp.rm(dir, { recursive: true, force: true })
|
||||
log.info(`removed module directory for "${id}"`)
|
||||
return true
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
InstallError,
|
||||
HOSTS_SETTING,
|
||||
parseHosts,
|
||||
checkUrl,
|
||||
get,
|
||||
fetchManifest,
|
||||
install,
|
||||
removeDir,
|
||||
moduleDir,
|
||||
isInstalled,
|
||||
purgeFile,
|
||||
MAX_MANIFEST_BYTES,
|
||||
MAX_ARTIFACT_BYTES,
|
||||
}
|
||||
@@ -101,10 +101,16 @@ async function boot({ modules, model } = {}) {
|
||||
id: m.id,
|
||||
name: m.name,
|
||||
version: m.version,
|
||||
// Null provenance is what a hand-placed directory looks like. An install
|
||||
// performed through the admin panel (§2.5, a later phase) writes the row
|
||||
// with its source and hash first; this refresh deliberately does not
|
||||
// overwrite either, because recordInstalled leaves what it is not given.
|
||||
// Null provenance is what a hand-placed directory looks like, and it is
|
||||
// all this step can honestly say: a scan finds a directory, never where it
|
||||
// came from. An install through the admin panel writes source and sha256
|
||||
// first, and this refresh must not undo that.
|
||||
//
|
||||
// It used to. `upsert` assigned both columns unconditionally, so every
|
||||
// boot nulled them and an install's provenance survived only until the
|
||||
// restart it asked for. The statement now COALESCEs — see the note on
|
||||
// modules.db.js's upsert. Nothing could have caught it before Phase 4:
|
||||
// this was the only caller, and it has never had a value to pass.
|
||||
}))
|
||||
}
|
||||
|
||||
@@ -205,4 +211,67 @@ async function shutdown({ modules, budgetMs = SHUTDOWN_BUDGET_MS } = {}) {
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { boot, shutdown, SHUTDOWN_BUDGET_MS }
|
||||
/**
|
||||
* Stop ONE module and mark it disabled — the admin panel's Disable (§2.7.2
|
||||
* decision 3).
|
||||
*
|
||||
* Phase 2 built disable as a pure state flip: the record moved to `disabled` and
|
||||
* the dispatch guard started answering 404. That makes a module invisible, not
|
||||
* stopped. Everything a module does that is not a response to a request — the
|
||||
* sockets and timers its `onBoot` armed — carried on running, so an operator
|
||||
* disabling a module *because* it was misbehaving got nothing until the next
|
||||
* restart, which is the one thing the button was supposed to save them.
|
||||
*
|
||||
* So the hook runs first, and the state moves after it: while `onShutdown` is
|
||||
* running the module is still `started`, which is the only state in which its
|
||||
* own routes and the things it is tearing down are consistent with each other.
|
||||
* The hook gets the same budget the exit path gives it.
|
||||
*
|
||||
* A hook that throws does NOT stop the disable. The operator asked for this
|
||||
* module to stop answering; a module that could not close cleanly is a reason to
|
||||
* log loudly, not a reason to leave it serving. That is the opposite of the boot
|
||||
* path's rule, and deliberately: there, a failure means the module never became
|
||||
* safe to use.
|
||||
*
|
||||
* **Enable is not the mirror of this, and there is no `start(id)` beside it.**
|
||||
* There is no `onBoot` re-dispatch, and MODULE_API.md has never promised the
|
||||
* hooks are re-entrant — no module author has written `onBoot` to be safe to run
|
||||
* twice in one process. Re-enabling therefore moves the row and waits for a
|
||||
* restart, which the admin screen offers.
|
||||
*
|
||||
* @returns {Promise<{stopped: boolean, error: string|null}>} whether a hook ran.
|
||||
*/
|
||||
async function stop(id, { modules, model, budgetMs = SHUTDOWN_BUDGET_MS } = {}) {
|
||||
/* eslint-disable global-require */
|
||||
const loader = modules || require('./loader')
|
||||
const rows = model || require('../model/modules/modules.model')
|
||||
/* eslint-enable global-require */
|
||||
|
||||
let error = null
|
||||
let stopped = false
|
||||
|
||||
if (loader.isLoaded()) {
|
||||
const target = loader.stopHook(id)
|
||||
if (target) {
|
||||
try {
|
||||
await withBudget(target.hook, budgetMs)
|
||||
stopped = true
|
||||
log.info(`module "${id}" stopped by an operator`)
|
||||
} catch (err) {
|
||||
error = err.message
|
||||
log.warn(`module "${id}" onShutdown failed or timed out while being disabled — disabling anyway`, {
|
||||
error: err.message,
|
||||
})
|
||||
}
|
||||
}
|
||||
// Unconditional, and after the hook: this is what makes its routes and its
|
||||
// client chunk answer 404 (§4.5). A module that is not loaded in this
|
||||
// process has no record to move, and setState ignores an unknown id.
|
||||
loader.setState(id, 'disabled')
|
||||
}
|
||||
|
||||
await safe(`disabling module "${id}"`, () => rows.disable(id))
|
||||
return { stopped, error }
|
||||
}
|
||||
|
||||
module.exports = { boot, shutdown, stop, SHUTDOWN_BUDGET_MS }
|
||||
|
||||
@@ -796,6 +796,31 @@ function shutdownHooks() {
|
||||
.reverse()
|
||||
}
|
||||
|
||||
/**
|
||||
* One module's `onShutdown`, for stopping it on its own rather than at exit.
|
||||
*
|
||||
* Phase 4 (§2.7.2 decision 3) gave the admin panel's Disable a real meaning.
|
||||
* Until then, disabling flipped this record's state and the dispatch guard began
|
||||
* answering 404 — which made the module invisible without making it stop. A
|
||||
* module's `onBoot` is where it opens its sockets and arms its timers, and none
|
||||
* of that is reachable through a URL, so an operator disabling a misbehaving
|
||||
* module got no relief from it at all until the next restart.
|
||||
*
|
||||
* `started` only, the same rule shutdownHooks() applies and for the same reason:
|
||||
* a module whose `onBoot` threw has a half-built world its `onShutdown` was
|
||||
* never written to tear down. Returns null when there is nothing to run — which
|
||||
* covers "not started", "no hook", and "no such module", none of which is an
|
||||
* error the caller can act on differently.
|
||||
*
|
||||
* @returns {{id: string, hook: Function}|null}
|
||||
*/
|
||||
function stopHook(id) {
|
||||
assertLoaded('stopHook')
|
||||
const record = modules.get(id)
|
||||
if (!record || record.state !== 'started' || !record.hooks.onShutdown) return null
|
||||
return { id: record.id, hook: record.hooks.onShutdown }
|
||||
}
|
||||
|
||||
/**
|
||||
* Every schema fragment waiting to be replayed, in scan order.
|
||||
*
|
||||
@@ -896,6 +921,7 @@ module.exports = {
|
||||
fragments,
|
||||
bootable,
|
||||
shutdownHooks,
|
||||
stopHook,
|
||||
clientChunks,
|
||||
clientEntryUrls,
|
||||
specFragments,
|
||||
|
||||
@@ -81,4 +81,48 @@ async function replayFragments({ query, modules } = {}) {
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { replayFragments }
|
||||
/**
|
||||
* Run one module's `purge.sql` — the destructive twin of the replay above
|
||||
* (MODULE_SYSTEM.md §2.5, and §2.7.2 decision 5 for when it is offered).
|
||||
*
|
||||
* It lives beside replayFragments because they are the same operation pointed in
|
||||
* opposite directions: a file of statements the module ships, split by the same
|
||||
* splitter, run serially on the same pool. Keeping them together is what makes
|
||||
* it obvious that the fragment's leading-verb allowlist does NOT apply here —
|
||||
* `purge.sql` is the one file a module may put a DROP in, precisely because it
|
||||
* is the one file that never runs on a boot.
|
||||
*
|
||||
* Unlike the replay, this **throws**. A replay failure is one module failing to
|
||||
* start, which the site survives by 503ing that module; a purge failure is an
|
||||
* operator's explicit destructive request not having happened, and reporting
|
||||
* success for that would leave them believing data is gone when it is not.
|
||||
*
|
||||
* Statements run serially and are not wrapped in a transaction, for the reason
|
||||
* the replay's header already gives: MariaDB commits DDL implicitly, so there is
|
||||
* no rollback to have. A purge that fails halfway has dropped some tables — the
|
||||
* error names the statement that stopped it, and running it again is safe
|
||||
* because a purge script is required to be idempotent in the same way a fragment
|
||||
* is (`DROP TABLE IF EXISTS`).
|
||||
*
|
||||
* @param {string} file absolute path to the module's purge.sql
|
||||
* @param {object} [deps] injection seam for tests
|
||||
* @returns {Promise<number>} how many statements ran
|
||||
*/
|
||||
async function runPurge(file, { query } = {}) {
|
||||
// eslint-disable-next-line global-require
|
||||
const run = query || require('../utils/db').query
|
||||
|
||||
const statements = splitStatements(fs.readFileSync(file, 'utf8'))
|
||||
let ran = 0
|
||||
for (const statement of statements) {
|
||||
try {
|
||||
await run(statement)
|
||||
ran += 1
|
||||
} catch (err) {
|
||||
throw new Error(`purge failed at statement ${ran + 1} of ${statements.length}: ${err.message}`)
|
||||
}
|
||||
}
|
||||
return ran
|
||||
}
|
||||
|
||||
module.exports = { replayFragments, runPurge }
|
||||
|
||||
@@ -30,6 +30,7 @@ const pagesRouter = require('./pages.router')
|
||||
const emailRouter = require('./email.router')
|
||||
const discordBotRouter = require('./discordBot.router')
|
||||
const settingsRouter = require('./settings.router')
|
||||
const modulesRouter = require('./modules.router')
|
||||
const dashboardRouter = require('./dashboard.router')
|
||||
|
||||
const adminRouter = express.Router()
|
||||
@@ -73,6 +74,11 @@ adminRouter.use('/pages', pagesRouter)
|
||||
adminRouter.use('/email', emailRouter)
|
||||
adminRouter.use('/discord-bot', discordBotRouter)
|
||||
adminRouter.use('/settings', settingsRouter)
|
||||
// The Modules screen (Phase 4). Core's, not a module's — and it has to be
|
||||
// core's: it is how a module gets onto the volume in the first place. Mounted
|
||||
// here alongside the other configuration capabilities, and admin-only per route
|
||||
// rather than at this line, so the gate sits next to what it is guarding.
|
||||
adminRouter.use('/modules', modulesRouter)
|
||||
|
||||
// The two singletons that own no path segment of their own: GET /dashboard and
|
||||
// PUT /site-mode. Mounted at the group root, last, exactly where the residual
|
||||
|
||||
426
server/src/router/v1/admin/modules.controller.js
Normal file
426
server/src/router/v1/admin/modules.controller.js
Normal file
@@ -0,0 +1,426 @@
|
||||
// ── Admin: installed modules ───────────────────────────────────────────────
|
||||
//
|
||||
// Phase 4, slice 1 of docs/website/MODULE_SYSTEM.md §2.7.2. Admin-only, and more
|
||||
// so than anything else in this directory: installing a module puts JavaScript on
|
||||
// the volume that core will `require` into its own process on the next boot. That
|
||||
// is the feature — it is what "an operator never builds anything" means (§1.14) —
|
||||
// but it is worth being plain that this controller is remote code execution with
|
||||
// an audit trail, not a settings screen.
|
||||
//
|
||||
// What guards it, in the order an attacker would meet them:
|
||||
//
|
||||
// 1. `requireRole('admin')` on every route, on top of the group's staff gate.
|
||||
// 2. An `https`-only host allowlist, re-checked on every redirect hop, so a
|
||||
// pasted URL cannot be pointed at the compose network or a metadata service
|
||||
// (install.js).
|
||||
// 3. The sha256 the release published, compared against the bytes that arrived.
|
||||
// 4. A full inspection of the archive before a byte of it is unpacked, and an
|
||||
// unpack into a scratch directory that is only moved into place once the
|
||||
// bundle has agreed with the manifest about what it is (archive.js).
|
||||
// 5. Every action here writes to core's one audit log.
|
||||
//
|
||||
// The one thing this file cannot do is mount anything. §1.12 makes the volume the
|
||||
// mounting source of truth, read once at require time, so install and uninstall
|
||||
// take effect on the next boot — which is why `restart` is a route here rather
|
||||
// than a sentence in a tooltip (decision 1).
|
||||
|
||||
const modules = require('../../../model/modules/modules.model')
|
||||
const activity = require('../../../model/activity/activity.model')
|
||||
const settings = require('../../../model/settings/settings.model')
|
||||
const loader = require('../../../modules/loader')
|
||||
const lifecycle = require('../../../modules/lifecycle')
|
||||
const install = require('../../../modules/install')
|
||||
const declared = require('../../../modules/declared')
|
||||
// A namespace import, like every other require in this file, and not
|
||||
// `const { runPurge } = …`: destructuring at require time captures the function
|
||||
// rather than the module, which makes it the one dependency here that cannot be
|
||||
// substituted. That matters because the two tests worth having about purge are
|
||||
// about the ORDER it runs in relative to the directory being removed.
|
||||
const schema = require('../../../modules/schema')
|
||||
|
||||
const log = require('../../../utils/logger')('admin-modules')
|
||||
|
||||
// The allowlist setting. Seeded from MODULE_SOURCE_HOSTS on first boot and
|
||||
// admin-managed from then on (decision 6) — db/seed.js writes it once and never
|
||||
// overwrites it, so changing the variable later does not silently reach in and
|
||||
// undo an operator's choice. The key itself lives in install.js, which is now
|
||||
// read by boot-time declared-set resolution as well as by this controller.
|
||||
const HOSTS_KEY = install.HOSTS_SETTING
|
||||
|
||||
// A hostname, not a URL: no scheme, no path, no port, no wildcard. Deliberately
|
||||
// strict — every character allowed here is a character that can appear in the
|
||||
// host of a URL this server will fetch and execute the contents of.
|
||||
const HOSTNAME = /^[a-z0-9]([a-z0-9-]*[a-z0-9])?(\.[a-z0-9]([a-z0-9-]*[a-z0-9])?)*$/
|
||||
|
||||
async function allowedHosts() {
|
||||
return install.parseHosts(await settings.get(HOSTS_KEY))
|
||||
}
|
||||
|
||||
/**
|
||||
* One module, as the admin screen needs it.
|
||||
*
|
||||
* Four sources have to be reconciled, and which one answers which question is
|
||||
* the whole of §2.4:
|
||||
*
|
||||
* - the ROW says what the operator decided and what the last boot recorded;
|
||||
* - the LOADER says what is mounted and answering right now;
|
||||
* - the VOLUME says whether there is still a directory there at all;
|
||||
* - the DECLARATION (slice 3) says what this container's environment asks for,
|
||||
* which is the only one of the four an admin cannot change from this screen.
|
||||
*
|
||||
* They can legitimately disagree, and the screen has to show that rather than
|
||||
* pick a winner. A row `enabled` with a loader state of `disabled` is a module
|
||||
* the operator has just switched back on and which is waiting for a restart —
|
||||
* exactly the case decision 3 creates, and it would be a lie to render it as
|
||||
* either "running" or "off". A declared module with no row and no directory is
|
||||
* the newest of those disagreements: MODULES asked for it and resolution could
|
||||
* not get it, so the screen carries the reason rather than showing nothing.
|
||||
*/
|
||||
function present(row, live, onVolume, declaration = null) {
|
||||
const id = row ? row.id : live ? live.id : declaration.id
|
||||
return {
|
||||
id,
|
||||
name: row ? row.name : live ? live.name : id,
|
||||
version: row ? row.version : live ? live.version : null,
|
||||
// What the database records.
|
||||
state: row ? row.state : null,
|
||||
failureStage: row ? row.failureStage : (live && live.stage) || null,
|
||||
failureReason: row ? row.failureReason : (live && live.reason) || null,
|
||||
source: row ? row.source : null,
|
||||
sha256: row ? row.sha256 : null,
|
||||
installedAt: row ? row.installedAt : null,
|
||||
startedAt: row ? row.startedAt : null,
|
||||
// What is actually mounted in this process, and what it is answering.
|
||||
liveState: live ? live.state : null,
|
||||
// The version RUNNING, which is not always the version installed: an upgrade
|
||||
// writes new files and a new row while the old code stays loaded until the
|
||||
// restart. Without this the screen would report the new version as
|
||||
// "Running", which is the same lie in a different place.
|
||||
liveVersion: live ? live.version : null,
|
||||
capabilities: live ? live.capabilities : [],
|
||||
// What is on the volume.
|
||||
onVolume,
|
||||
canPurge: onVolume && Boolean(install.purgeFile(id)),
|
||||
// What the environment declares. `declaredVersion` is what MODULES pins, not
|
||||
// what is installed — they differ exactly while a resolution is failing, and
|
||||
// `declaredError` says why. Uninstalling a declared module from this screen
|
||||
// removes its directory and disables its row; the next boot puts the
|
||||
// directory back and leaves the row disabled, so the screen says so rather
|
||||
// than letting the files reappear unexplained.
|
||||
declared: Boolean(declaration),
|
||||
declaredVersion: declaration ? declaration.version : null,
|
||||
declaredError: declaration && declaration.action === 'failed' ? declaration.message : null,
|
||||
}
|
||||
}
|
||||
|
||||
// GET /admin/modules — every module core knows about, from all three sources,
|
||||
// plus the source allowlist the install form needs.
|
||||
async function list(req, res) {
|
||||
try {
|
||||
const rows = await modules.list()
|
||||
// The loader throws rather than returning [] before load() has run (§7.6),
|
||||
// and this controller is reachable from a process where that is true —
|
||||
// `npm run seed` never gets here, but a test harness might.
|
||||
const live = loader.isLoaded() ? loader.list() : []
|
||||
const byId = new Map(live.map((m) => [m.id, m]))
|
||||
const declaredById = new Map(declared.state().map((d) => [d.id, d]))
|
||||
|
||||
const seen = new Set()
|
||||
const out = []
|
||||
for (const row of rows) {
|
||||
seen.add(row.id)
|
||||
out.push(
|
||||
present(row, byId.get(row.id) || null, install.isInstalled(row.id), declaredById.get(row.id)),
|
||||
)
|
||||
}
|
||||
// A directory on the volume that has no row yet — a hand-placed install
|
||||
// before its first boot. It has to be listed, or the screen would show
|
||||
// nothing for a module whose routes are already being served.
|
||||
for (const m of live) {
|
||||
if (!seen.has(m.id)) {
|
||||
seen.add(m.id)
|
||||
out.push(present(null, m, true, declaredById.get(m.id)))
|
||||
}
|
||||
}
|
||||
// A module MODULES declares that has neither. Resolution failed and left
|
||||
// nothing behind — the case an operator most needs told, because from the
|
||||
// screen's other three sources it is indistinguishable from never having
|
||||
// asked for it.
|
||||
for (const d of declaredById.values()) {
|
||||
if (!seen.has(d.id)) out.push(present(null, null, install.isInstalled(d.id), d))
|
||||
}
|
||||
|
||||
return res.json({ modules: out, sourceHosts: await allowedHosts() })
|
||||
} catch (err) {
|
||||
log.error('list modules', err)
|
||||
return res.status(500).json({ message: 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
// POST /admin/modules — install (or upgrade) from an install-manifest URL.
|
||||
async function create(req, res) {
|
||||
const url = String(req.body.url || '').trim()
|
||||
try {
|
||||
const hosts = await allowedHosts()
|
||||
const result = await install.install({ url, hosts })
|
||||
|
||||
// Provenance is written here and nowhere else: the boot reconcile records a
|
||||
// module with NULL source/sha256 and leaves what it is not given, precisely
|
||||
// so that a refresh cannot overwrite what an install knew (lifecycle.js).
|
||||
const row = await modules.recordInstalled({
|
||||
id: result.id,
|
||||
name: result.name,
|
||||
version: result.version,
|
||||
source: result.source,
|
||||
sha256: result.sha256,
|
||||
})
|
||||
|
||||
await activity.log({
|
||||
req,
|
||||
userId: req.user.id,
|
||||
action: 'module.install',
|
||||
detail: { id: result.id, version: result.version, source: url, sha256: result.sha256, replaced: result.replaced },
|
||||
})
|
||||
log.warn('module installed — it will mount on the next restart', {
|
||||
id: result.id,
|
||||
version: result.version,
|
||||
by: req.user.username,
|
||||
})
|
||||
|
||||
return res.status(201).json({ module: row, restartRequired: true, replaced: result.replaced })
|
||||
} catch (err) {
|
||||
if (err.name === 'InstallError' || err.name === 'ArchiveError') {
|
||||
// The operator pasted a URL and something about what came back was wrong.
|
||||
// The message is the useful part and is written to be read by them.
|
||||
log.warn('module install refused', { url, reason: err.message })
|
||||
return res.status(err.status || 400).json({ message: err.message })
|
||||
}
|
||||
log.error('install module', err)
|
||||
return res.status(500).json({ message: 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
// POST /admin/modules/:id/enable — switch a module back on, for the next boot.
|
||||
//
|
||||
// Deliberately does NOT touch the loader's record. Disable ran the module's
|
||||
// onShutdown (decision 3), and there is no onBoot re-dispatch to undo that: a
|
||||
// module whose sockets were closed and timers cleared cannot be made to serve
|
||||
// again by flipping a flag, and pretending otherwise would put it back on the
|
||||
// nav with a torn-down world behind it. The row moves; the restart starts it.
|
||||
async function enable(req, res) {
|
||||
const { id } = req.params
|
||||
try {
|
||||
const row = await modules.enable(id)
|
||||
if (!row) return res.status(404).json({ message: 'No such module.' })
|
||||
|
||||
await activity.log({ req, userId: req.user.id, action: 'module.enable', detail: { id } })
|
||||
return res.json({ module: row, restartRequired: true })
|
||||
} catch (err) {
|
||||
if (err.name === 'ModuleStateError') return res.status(409).json({ message: err.message })
|
||||
log.error('enable module', err)
|
||||
return res.status(500).json({ message: 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
// POST /admin/modules/:id/disable — stop it now.
|
||||
//
|
||||
// The one action on this screen that takes effect without a restart, and the
|
||||
// reason it does is that it is the one an operator reaches for when something is
|
||||
// going wrong. Its routes answer 404 from the moment this returns, and its
|
||||
// onShutdown has already run.
|
||||
async function disable(req, res) {
|
||||
const { id } = req.params
|
||||
try {
|
||||
const current = await modules.get(id)
|
||||
if (!current) return res.status(404).json({ message: 'No such module.' })
|
||||
|
||||
const { stopped, error } = await lifecycle.stop(id)
|
||||
const row = await modules.get(id)
|
||||
|
||||
await activity.log({
|
||||
req,
|
||||
userId: req.user.id,
|
||||
action: 'module.disable',
|
||||
detail: { id, hookRan: stopped, hookError: error },
|
||||
})
|
||||
log.warn('module disabled by an operator', { id, hookRan: stopped, by: req.user.username })
|
||||
|
||||
// `shutdownError` is reported rather than swallowed: the module IS disabled
|
||||
// either way, and an operator whose module could not close cleanly should be
|
||||
// told so while they still have the logs to look at.
|
||||
return res.json({ module: row, stopped, shutdownError: error })
|
||||
} catch (err) {
|
||||
log.error('disable module', err)
|
||||
return res.status(500).json({ message: 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
// DELETE /admin/modules/:id[?purge=true] — uninstall.
|
||||
//
|
||||
// Non-destructive by default (§2.5): the directory goes, the row stays
|
||||
// `disabled`, and the module's tables and data are left alone.
|
||||
//
|
||||
// The purge option is here rather than as a follow-up action because it cannot
|
||||
// be a follow-up action (decision 5): `purge.sql` is a file inside the directory
|
||||
// this is about to delete, so after an uninstall there is nothing left to purge
|
||||
// with. Ticking the box is the last moment the file exists.
|
||||
//
|
||||
// The order below is the whole of it, and each step depends on the one above:
|
||||
// purge while the SQL is still readable, stop while the code is still loaded,
|
||||
// then delete.
|
||||
async function remove(req, res) {
|
||||
const { id } = req.params
|
||||
const purge = req.query.purge === 'true' || req.query.purge === '1'
|
||||
try {
|
||||
const current = await modules.get(id)
|
||||
const onVolume = install.isInstalled(id)
|
||||
if (!current && !onVolume) return res.status(404).json({ message: 'No such module.' })
|
||||
|
||||
let purged = null
|
||||
if (purge) {
|
||||
const file = install.purgeFile(id)
|
||||
if (!file) {
|
||||
return res.status(400).json({
|
||||
message: 'This module ships no purge.sql, so its data cannot be deleted. Uninstall without purging instead.',
|
||||
})
|
||||
}
|
||||
purged = await schema.runPurge(file)
|
||||
}
|
||||
|
||||
// Stop it before its files vanish. A module whose directory is deleted out
|
||||
// from under a running onShutdown is being asked to tear down a world whose
|
||||
// code may already be half-unreadable — and its sockets would otherwise stay
|
||||
// open until the restart, holding a connection on behalf of a module that no
|
||||
// longer exists on disk.
|
||||
await lifecycle.stop(id)
|
||||
const removed = await install.removeDir(id)
|
||||
|
||||
// A purge leaves nothing: no directory, no tables, no data. Keeping a
|
||||
// `disabled` row for that is a tombstone with nothing to offer and a Purge
|
||||
// button that would fail. A plain uninstall keeps its row, which is what
|
||||
// makes the retained data visible and reinstallable.
|
||||
if (purge) await modules.remove(id)
|
||||
|
||||
await activity.log({
|
||||
req,
|
||||
userId: req.user.id,
|
||||
action: purge ? 'module.purge' : 'module.uninstall',
|
||||
detail: { id, purged, removed },
|
||||
})
|
||||
log.warn(`module ${purge ? 'uninstalled and purged' : 'uninstalled'}`, {
|
||||
id,
|
||||
statements: purged,
|
||||
by: req.user.username,
|
||||
})
|
||||
|
||||
return res.json({ id, removed, purged, restartRequired: true })
|
||||
} catch (err) {
|
||||
log.error('uninstall module', err)
|
||||
return res.status(500).json({ message: err.message || 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
// POST /admin/modules/:id/purge — drop a still-installed module's data.
|
||||
//
|
||||
// Refuses unless the module is already disabled, and that guard is the point:
|
||||
// dropping the tables under a module that is still serving requests leaves it
|
||||
// answering out of a world that no longer exists. Disabling first is one click
|
||||
// and makes the destructive step happen against something that has stopped.
|
||||
async function purge(req, res) {
|
||||
const { id } = req.params
|
||||
try {
|
||||
const current = await modules.get(id)
|
||||
if (!current) return res.status(404).json({ message: 'No such module.' })
|
||||
if (current.state !== 'disabled') {
|
||||
return res.status(409).json({
|
||||
message: 'Disable this module before purging its data, so nothing is serving out of the tables being dropped.',
|
||||
})
|
||||
}
|
||||
|
||||
const file = install.purgeFile(id)
|
||||
if (!file) {
|
||||
return res.status(400).json({ message: 'This module ships no purge.sql, so its data cannot be deleted.' })
|
||||
}
|
||||
|
||||
const statements = await schema.runPurge(file)
|
||||
await activity.log({ req, userId: req.user.id, action: 'module.purge', detail: { id, statements } })
|
||||
log.warn('module data purged', { id, statements, by: req.user.username })
|
||||
|
||||
return res.json({ id, purged: statements })
|
||||
} catch (err) {
|
||||
log.error('purge module', err)
|
||||
return res.status(500).json({ message: err.message || 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
// PUT /admin/modules/sources — the host allowlist.
|
||||
async function setSources(req, res) {
|
||||
const hosts = install.parseHosts(req.body.hosts)
|
||||
const bad = hosts.find((h) => !HOSTNAME.test(h))
|
||||
if (bad) return res.status(400).json({ message: `"${bad}" is not a valid hostname.` })
|
||||
|
||||
try {
|
||||
const before = await allowedHosts()
|
||||
await settings.set(HOSTS_KEY, hosts.join(','), req.user.id)
|
||||
await activity.log({
|
||||
req,
|
||||
userId: req.user.id,
|
||||
action: 'module.sources',
|
||||
detail: { before, after: hosts },
|
||||
})
|
||||
log.warn('module source allowlist changed', { before, after: hosts, by: req.user.username })
|
||||
return res.json({ sourceHosts: hosts })
|
||||
} catch (err) {
|
||||
log.error('set module sources', err)
|
||||
return res.status(500).json({ message: 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
// POST /admin/modules/restart — restart the server process.
|
||||
//
|
||||
// Decision 1. Install, uninstall and re-enable all only take effect at boot
|
||||
// because §1.12 reads the volume at require time, and §2.4 promises recovery
|
||||
// "with no shell access to the box" — which a banner saying "please restart your
|
||||
// container" does not deliver.
|
||||
//
|
||||
// It reaches server.js's existing SIGTERM handler rather than doing the work
|
||||
// itself: that handler stops the modules, the workers and the listeners in the
|
||||
// right order and closes the pool and the log file before exiting 0, and going
|
||||
// through it means there is exactly one graceful-shutdown path that this route
|
||||
// cannot drift from.
|
||||
//
|
||||
// It gets there by EMITTING the event, not by signalling the process, and that
|
||||
// is not a detail. `process.kill(process.pid, 'SIGTERM')` is what this did
|
||||
// first, and it works on Linux — but **Windows has no POSIX signals, and Node
|
||||
// documents SIGTERM there as unconditional termination of the target process**.
|
||||
// So on a Windows host the restart killed the server outright: no module
|
||||
// `onShutdown`, no pool close, no log flush. Verified by running it — the
|
||||
// process was gone and the shutdown handler had logged nothing.
|
||||
//
|
||||
// `process.on('SIGTERM', …)` is an ordinary EventEmitter listener, so
|
||||
// `process.emit('SIGTERM')` invokes exactly the same handler on every platform
|
||||
// without involving the OS at all. Deployment is Linux containers and would
|
||||
// never have shown this; development is not.
|
||||
//
|
||||
// What brings the process BACK is the supervisor, not this. The shipped
|
||||
// docker-compose.yml declares `restart: unless-stopped` on `app`, which restarts
|
||||
// on a clean exit as well as a crash. A bare `npm start` does not come back, and
|
||||
// the screen says so before it asks.
|
||||
function restart(req, res) {
|
||||
log.warn('restart requested from the admin panel', { by: req.user.username })
|
||||
|
||||
// Logged and answered first. Once the signal is raised the response has no
|
||||
// listener left to flush through, so the operator would be told nothing.
|
||||
res.status(202).json({ restarting: true })
|
||||
|
||||
activity
|
||||
.log({ req, userId: req.user.id, action: 'module.restart', detail: {} })
|
||||
.catch((err) => log.error('failed to record the restart in the audit log', err))
|
||||
.finally(() => {
|
||||
// A beat, so the 202 is on the wire. `unref` so this timer is not itself
|
||||
// something keeping the process alive.
|
||||
setTimeout(() => process.emit('SIGTERM'), 250).unref()
|
||||
})
|
||||
}
|
||||
|
||||
module.exports = { list, create, enable, disable, remove, purge, setSources, restart, HOSTS_KEY }
|
||||
143
server/src/router/v1/admin/modules.router.js
Normal file
143
server/src/router/v1/admin/modules.router.js
Normal file
@@ -0,0 +1,143 @@
|
||||
// Admin · Modules — install, enable, disable, uninstall, purge and restart.
|
||||
//
|
||||
// Mounted at /api/v1/admin/modules by admin/index.js, which has already applied
|
||||
// `noindex, isLoggedIn, staffOnly`. Every route here re-gates to `admin`: an
|
||||
// editor or moderator has no business installing code into the server process,
|
||||
// and the group gate alone would let them.
|
||||
//
|
||||
// Route order matters in one place. `/restart` and `/sources` are declared
|
||||
// BEFORE the `/:id/...` routes, because express matches in declaration order and
|
||||
// a module whose id was `restart` would otherwise shadow — or be shadowed by —
|
||||
// the literal path. The id pattern below makes that unreachable in practice; the
|
||||
// ordering makes it unreachable by construction.
|
||||
|
||||
const express = require('express')
|
||||
const { body, param, query } = require('express-validator')
|
||||
|
||||
const controller = require('./modules.controller')
|
||||
const { requireRole } = require('../../../utils/auth')
|
||||
const validate = require('../../../middleware/validate')
|
||||
|
||||
const modulesRouter = express.Router()
|
||||
const adminOnly = requireRole('admin')
|
||||
|
||||
// The loader's own id rule (MODULE_API.md §2.1). Applied at the edge so a
|
||||
// traversal-shaped id never reaches a path join, even though install.js checks
|
||||
// it again — this one produces a 400 with a readable message, that one is the
|
||||
// guarantee.
|
||||
const ID = /^[a-z][a-z0-9-]{1,31}$/
|
||||
|
||||
modulesRouter.get(
|
||||
'/',
|
||||
// #swagger.tags = ['Admin · Modules']
|
||||
// #swagger.summary = 'List installed modules, their live state, and the source allowlist'
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
/* #swagger.responses[200] = { description: 'Modules and the install source allowlist', content: { "application/json": { schema: { type: "object", properties: { modules: { type: "array", items: { type: "object", additionalProperties: true } }, sourceHosts: { type: "array", items: { type: "string" } } } } } } } */
|
||||
adminOnly,
|
||||
controller.list,
|
||||
)
|
||||
|
||||
modulesRouter.post(
|
||||
'/',
|
||||
// #swagger.tags = ['Admin · Modules']
|
||||
// #swagger.summary = 'Install or upgrade a module from a release install-manifest URL'
|
||||
// #swagger.description = 'Downloads the artifact the manifest names, verifies its sha256, inspects the archive in full and unpacks it onto the modules volume. The module mounts on the next restart.'
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["url"], properties: { url: { type: "string", description: "https URL of the release install manifest, on an allowed host" } } } } } } */
|
||||
/* #swagger.responses[201] = { description: 'Installed — restart to mount it', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
|
||||
/* #swagger.responses[400] = { description: 'The URL, the manifest, the hash or the archive was refused', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
/* #swagger.responses[502] = { description: 'The source host could not be reached or answered badly', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
adminOnly,
|
||||
body('url').isString().trim().isLength({ min: 1, max: 2048 }),
|
||||
validate,
|
||||
controller.create,
|
||||
)
|
||||
|
||||
modulesRouter.put(
|
||||
'/sources',
|
||||
// #swagger.tags = ['Admin · Modules']
|
||||
// #swagger.summary = 'Replace the allowlist of hosts modules may be installed from'
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["hosts"], properties: { hosts: { type: "string", description: "Comma- or space-separated hostnames. An empty list forbids all installs." } } } } } } */
|
||||
/* #swagger.responses[200] = { description: 'The new allowlist', content: { "application/json": { schema: { type: "object", properties: { sourceHosts: { type: "array", items: { type: "string" } } } } } } } */
|
||||
/* #swagger.responses[400] = { description: 'One of the entries is not a hostname', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
adminOnly,
|
||||
body('hosts').isString().isLength({ max: 2048 }),
|
||||
validate,
|
||||
controller.setSources,
|
||||
)
|
||||
|
||||
modulesRouter.post(
|
||||
'/restart',
|
||||
// #swagger.tags = ['Admin · Modules']
|
||||
// #swagger.summary = 'Restart the server process so module changes take effect'
|
||||
// #swagger.description = 'Runs the same graceful shutdown a SIGTERM does. The process is brought back by the supervisor, which the shipped docker-compose.yml provides; a bare `npm start` will not come back.'
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
/* #swagger.responses[202] = { description: 'Shutting down', content: { "application/json": { schema: { type: "object", properties: { restarting: { type: "boolean" } } } } } } */
|
||||
adminOnly,
|
||||
controller.restart,
|
||||
)
|
||||
|
||||
modulesRouter.post(
|
||||
'/:id/enable',
|
||||
// #swagger.tags = ['Admin · Modules']
|
||||
// #swagger.summary = 'Enable a module (takes effect on the next restart)'
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Module id.' }
|
||||
/* #swagger.responses[200] = { description: 'Enabled — restart to start it', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
|
||||
/* #swagger.responses[404] = { description: 'No such module', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
adminOnly,
|
||||
param('id').matches(ID),
|
||||
validate,
|
||||
controller.enable,
|
||||
)
|
||||
|
||||
modulesRouter.post(
|
||||
'/:id/disable',
|
||||
// #swagger.tags = ['Admin · Modules']
|
||||
// #swagger.summary = 'Stop a module now — runs its onShutdown, then its routes answer 404'
|
||||
// #swagger.description = 'The only module action that takes effect without a restart. Re-enabling needs one, because there is no onBoot re-dispatch.'
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Module id.' }
|
||||
/* #swagger.responses[200] = { description: 'Disabled', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
|
||||
/* #swagger.responses[404] = { description: 'No such module', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
adminOnly,
|
||||
param('id').matches(ID),
|
||||
validate,
|
||||
controller.disable,
|
||||
)
|
||||
|
||||
modulesRouter.post(
|
||||
'/:id/purge',
|
||||
// #swagger.tags = ['Admin · Modules']
|
||||
// #swagger.summary = "Run a disabled module’s purge.sql, dropping its tables and data"
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Module id.' }
|
||||
/* #swagger.responses[200] = { description: 'Purged', content: { "application/json": { schema: { type: "object", properties: { id: { type: "string" }, purged: { type: "integer" } } } } } } */
|
||||
/* #swagger.responses[400] = { description: 'The module ships no purge.sql', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
/* #swagger.responses[409] = { description: 'The module must be disabled first', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
adminOnly,
|
||||
param('id').matches(ID),
|
||||
validate,
|
||||
controller.purge,
|
||||
)
|
||||
|
||||
modulesRouter.delete(
|
||||
'/:id',
|
||||
// #swagger.tags = ['Admin · Modules']
|
||||
// #swagger.summary = 'Uninstall a module, optionally deleting its data too'
|
||||
// #swagger.description = "Removes the module directory and leaves its row disabled. With purge=true it also runs purge.sql first — which is the only moment it can, since purge.sql lives inside the directory being deleted."
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Module id.' }
|
||||
// #swagger.parameters['purge'] = { in: 'query', required: false, schema: { type: 'boolean' }, description: "Also run the module’s purge.sql and drop its row. Destructive and irreversible." }
|
||||
/* #swagger.responses[200] = { description: 'Uninstalled — restart to unmount it', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
|
||||
/* #swagger.responses[400] = { description: 'Purge was asked for and the module ships no purge.sql', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
/* #swagger.responses[404] = { description: 'No such module', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
adminOnly,
|
||||
param('id').matches(ID),
|
||||
query('purge').optional().isIn(['true', 'false', '1', '0']),
|
||||
validate,
|
||||
controller.remove,
|
||||
)
|
||||
|
||||
module.exports = modulesRouter
|
||||
@@ -1,8 +1,12 @@
|
||||
require('dotenv').config()
|
||||
const http = require('http')
|
||||
|
||||
const app = require('./app')
|
||||
const internalApp = require('./internalApp')
|
||||
// NOTE: `./app` and `./internalApp` are deliberately NOT required here. Requiring
|
||||
// app.js runs `modules.load()`, which scans the volume and mounts whatever is on
|
||||
// it (MODULE_API.md §4.1) — so the declared module set has to be resolved before
|
||||
// that require, not before the listener. They are required inside start(), after
|
||||
// resolveDeclaredModules(); everything else this file needs is safe to pull in
|
||||
// now because none of it reaches the loader's scan.
|
||||
const botScore = require('./middleware/botScore')
|
||||
const announceWorker = require('./utils/announceWorker')
|
||||
const { ensureSchema, close } = require('./utils/db')
|
||||
@@ -11,6 +15,9 @@ const settings = require('./model/settings/settings.model')
|
||||
const revokedSessions = require('./model/revokedSessions/revokedSessions.model')
|
||||
const mobileAuthBridge = require('./model/mobileAuthBridge/mobileAuthBridge.model')
|
||||
const moduleLifecycle = require('./modules/lifecycle')
|
||||
const declaredModules = require('./modules/declared')
|
||||
const moduleInstall = require('./modules/install')
|
||||
const moduleModel = require('./model/modules/modules.model')
|
||||
const createLogger = require('./utils/logger')
|
||||
const { evaluateBotInternalKey } = require('./utils/botInternalKey')
|
||||
const brand = require('./config/brand')
|
||||
@@ -51,7 +58,9 @@ async function start() {
|
||||
}
|
||||
|
||||
log.info('ensuring database schema...')
|
||||
await ensureSchema()
|
||||
// Core's schema only. Each installed module's fragment is replayed further
|
||||
// down, after the volume has been scanned — see the require of ./app below.
|
||||
await ensureSchema({ replayModules: false })
|
||||
log.info('seeding defaults...')
|
||||
await seedDefaults()
|
||||
await createInitialAdminFromEnv()
|
||||
@@ -77,6 +86,36 @@ async function start() {
|
||||
const mode = await settings.get('site_mode')
|
||||
log.info(`site mode: ${String(mode || 'live').toUpperCase()}`)
|
||||
|
||||
// Bring the modules volume in line with what MODULES declares (§2.7.2
|
||||
// decision 4), and only then require the app — the loader scans and mounts at
|
||||
// require time, so this is the last moment at which a module can be put on the
|
||||
// volume and still be part of this process.
|
||||
//
|
||||
// After the schema and the seed, because the host allowlist it installs under
|
||||
// is a settings row that the seed creates on a fresh instance. Never throws:
|
||||
// an unreachable release host leaves the site serving without that module
|
||||
// rather than taking the site down with it.
|
||||
await declaredModules.resolve({
|
||||
hosts: moduleInstall.parseHosts(await settings.get(moduleInstall.HOSTS_SETTING)),
|
||||
model: moduleModel,
|
||||
})
|
||||
|
||||
// Requiring app.js is what scans the volume and mounts what is on it. Every
|
||||
// line above this one runs against a core that has no modules in it yet.
|
||||
// eslint-disable-next-line global-require
|
||||
const app = require('./app')
|
||||
// eslint-disable-next-line global-require
|
||||
const internalApp = require('./internalApp')
|
||||
|
||||
// Now that the scan has happened, replay each module's schema fragment
|
||||
// (MODULE_API.md §2.6). This used to ride inside ensureSchema() and could,
|
||||
// because app.js was required at the top of this file; resolving the declared
|
||||
// set first moved the scan after it, and a booting server quietly getting no
|
||||
// module tables is precisely what §7.6 warns about. Caught by the browser
|
||||
// smoke rather than by a test: every suite here stubs one side or the other.
|
||||
// eslint-disable-next-line global-require
|
||||
await require('./modules/schema').replayFragments()
|
||||
|
||||
// Reconcile installed_modules with what the loader found on the volume at
|
||||
// require time, then run each module's onBoot (MODULE_API.md §2.5).
|
||||
//
|
||||
|
||||
@@ -55,11 +55,18 @@ const SCHEMA_PATH = path.join(__dirname, '..', '..', 'db', 'schema.sql')
|
||||
* below — the discovery, splitting and per-module failure handling all live in
|
||||
* modules/schema.js, required lazily so that requiring the pool never drags the
|
||||
* loader in with it.
|
||||
*
|
||||
* `replayModules: false` is for a caller that has not scanned the volume YET and
|
||||
* intends to. server.js is the one: since slice 3 it resolves the declared
|
||||
* module set before requiring app.js, which puts core's schema *before* the scan
|
||||
* — so it replays the fragments itself, in the one place that knows the scan has
|
||||
* happened. Left true everywhere else, so the ordinary caller cannot get module
|
||||
* tables by accident and lose them by refactor.
|
||||
*/
|
||||
async function ensureSchema({ retries = 10, delayMs = 2000 } = {}) {
|
||||
async function ensureSchema({ retries = 10, delayMs = 2000, replayModules = true } = {}) {
|
||||
await ensureCoreSchema({ retries, delayMs })
|
||||
// eslint-disable-next-line global-require
|
||||
await require('../modules/schema').replayFragments()
|
||||
if (replayModules) await require('../modules/schema').replayFragments()
|
||||
}
|
||||
|
||||
/** Core's own schema.sql, with the wait-for-the-database retry. */
|
||||
|
||||
@@ -2353,6 +2353,490 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/admin/modules": {
|
||||
"get": {
|
||||
"tags": [
|
||||
"Admin · Modules"
|
||||
],
|
||||
"summary": "List installed modules, their live state, and the source allowlist",
|
||||
"description": "",
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Modules and the install source allowlist",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"modules": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"additionalProperties": true
|
||||
}
|
||||
},
|
||||
"sourceHosts": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"500": {
|
||||
"description": "Internal Server Error"
|
||||
}
|
||||
},
|
||||
"security": [
|
||||
{
|
||||
"cookieAuth": []
|
||||
},
|
||||
{
|
||||
"bearerAuth": []
|
||||
}
|
||||
]
|
||||
},
|
||||
"post": {
|
||||
"tags": [
|
||||
"Admin · Modules"
|
||||
],
|
||||
"summary": "Install or upgrade a module from a release install-manifest URL",
|
||||
"description": "Downloads the artifact the manifest names, verifies its sha256, inspects the archive in full and unpacks it onto the modules volume. The module mounts on the next restart.",
|
||||
"responses": {
|
||||
"201": {
|
||||
"description": "Installed — restart to mount it",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"400": {
|
||||
"description": "The URL, the manifest, the hash or the archive was refused",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/Error"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"500": {
|
||||
"description": "Internal Server Error"
|
||||
},
|
||||
"502": {
|
||||
"description": "The source host could not be reached or answered badly",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/Error"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"security": [
|
||||
{
|
||||
"cookieAuth": []
|
||||
},
|
||||
{
|
||||
"bearerAuth": []
|
||||
}
|
||||
],
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"url"
|
||||
],
|
||||
"properties": {
|
||||
"url": {
|
||||
"type": "string",
|
||||
"description": "https URL of the release install manifest, on an allowed host"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/admin/modules/restart": {
|
||||
"post": {
|
||||
"tags": [
|
||||
"Admin · Modules"
|
||||
],
|
||||
"summary": "Restart the server process so module changes take effect",
|
||||
"description": "Runs the same graceful shutdown a SIGTERM does. The process is brought back by the supervisor, which the shipped docker-compose.yml provides; a bare `npm start` will not come back.",
|
||||
"responses": {
|
||||
"202": {
|
||||
"description": "Shutting down",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"restarting": {
|
||||
"type": "boolean"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"security": [
|
||||
{
|
||||
"cookieAuth": []
|
||||
},
|
||||
{
|
||||
"bearerAuth": []
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"/api/v1/admin/modules/sources": {
|
||||
"put": {
|
||||
"tags": [
|
||||
"Admin · Modules"
|
||||
],
|
||||
"summary": "Replace the allowlist of hosts modules may be installed from",
|
||||
"description": "",
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "The new allowlist",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"sourceHosts": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"400": {
|
||||
"description": "One of the entries is not a hostname",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/Error"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"500": {
|
||||
"description": "Internal Server Error"
|
||||
}
|
||||
},
|
||||
"security": [
|
||||
{
|
||||
"cookieAuth": []
|
||||
},
|
||||
{
|
||||
"bearerAuth": []
|
||||
}
|
||||
],
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"hosts"
|
||||
],
|
||||
"properties": {
|
||||
"hosts": {
|
||||
"type": "string",
|
||||
"description": "Comma- or space-separated hostnames. An empty list forbids all installs."
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/admin/modules/{id}": {
|
||||
"delete": {
|
||||
"tags": [
|
||||
"Admin · Modules"
|
||||
],
|
||||
"summary": "Uninstall a module, optionally deleting its data too",
|
||||
"description": "Removes the module directory and leaves its row disabled. With purge=true it also runs purge.sql first — which is the only moment it can, since purge.sql lives inside the directory being deleted.",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "id",
|
||||
"in": "path",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "string"
|
||||
},
|
||||
"description": "Module id."
|
||||
},
|
||||
{
|
||||
"name": "purge",
|
||||
"in": "query",
|
||||
"required": false,
|
||||
"schema": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"description": "Also run the module’s purge.sql and drop its row. Destructive and irreversible."
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Uninstalled — restart to unmount it",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"400": {
|
||||
"description": "Purge was asked for and the module ships no purge.sql",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/Error"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"404": {
|
||||
"description": "No such module",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/Error"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"500": {
|
||||
"description": "Internal Server Error"
|
||||
}
|
||||
},
|
||||
"security": [
|
||||
{
|
||||
"cookieAuth": []
|
||||
},
|
||||
{
|
||||
"bearerAuth": []
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"/api/v1/admin/modules/{id}/disable": {
|
||||
"post": {
|
||||
"tags": [
|
||||
"Admin · Modules"
|
||||
],
|
||||
"summary": "Stop a module now — runs its onShutdown, then its routes answer 404",
|
||||
"description": "The only module action that takes effect without a restart. Re-enabling needs one, because there is no onBoot re-dispatch.",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "id",
|
||||
"in": "path",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "string"
|
||||
},
|
||||
"description": "Module id."
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Disabled",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"400": {
|
||||
"description": "Bad Request"
|
||||
},
|
||||
"404": {
|
||||
"description": "No such module",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/Error"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"500": {
|
||||
"description": "Internal Server Error"
|
||||
}
|
||||
},
|
||||
"security": [
|
||||
{
|
||||
"cookieAuth": []
|
||||
},
|
||||
{
|
||||
"bearerAuth": []
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"/api/v1/admin/modules/{id}/enable": {
|
||||
"post": {
|
||||
"tags": [
|
||||
"Admin · Modules"
|
||||
],
|
||||
"summary": "Enable a module (takes effect on the next restart)",
|
||||
"description": "",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "id",
|
||||
"in": "path",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "string"
|
||||
},
|
||||
"description": "Module id."
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Enabled — restart to start it",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"400": {
|
||||
"description": "Bad Request"
|
||||
},
|
||||
"404": {
|
||||
"description": "No such module",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/Error"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"409": {
|
||||
"description": "Conflict"
|
||||
},
|
||||
"500": {
|
||||
"description": "Internal Server Error"
|
||||
}
|
||||
},
|
||||
"security": [
|
||||
{
|
||||
"cookieAuth": []
|
||||
},
|
||||
{
|
||||
"bearerAuth": []
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"/api/v1/admin/modules/{id}/purge": {
|
||||
"post": {
|
||||
"tags": [
|
||||
"Admin · Modules"
|
||||
],
|
||||
"summary": "Run a disabled module’s purge.sql, dropping its tables and data",
|
||||
"description": "",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "id",
|
||||
"in": "path",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "string"
|
||||
},
|
||||
"description": "Module id."
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Purged",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string"
|
||||
},
|
||||
"purged": {
|
||||
"type": "integer"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"400": {
|
||||
"description": "The module ships no purge.sql",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/Error"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"404": {
|
||||
"description": "Not Found"
|
||||
},
|
||||
"409": {
|
||||
"description": "The module must be disabled first",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/Error"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"500": {
|
||||
"description": "Internal Server Error"
|
||||
}
|
||||
},
|
||||
"security": [
|
||||
{
|
||||
"cookieAuth": []
|
||||
},
|
||||
{
|
||||
"bearerAuth": []
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"/api/v1/admin/pages": {
|
||||
"get": {
|
||||
"tags": [
|
||||
|
||||
528
server/test/adminModules.test.js
Normal file
528
server/test/adminModules.test.js
Normal file
@@ -0,0 +1,528 @@
|
||||
// ── Admin · Modules: the delivery surface ──────────────────────────────────
|
||||
//
|
||||
// Phase 4, slice 1 of MODULE_SYSTEM.md §2.7.2. The controller is tested directly
|
||||
// with a mock `res` and stubbed models — the same shape adminUsers.test.js uses —
|
||||
// because what is interesting here is not the HTTP plumbing but the ORDER of
|
||||
// operations and which of the three sources of truth answers which question.
|
||||
//
|
||||
// Two of these tests exist to pin decisions that are easy to "fix" back into
|
||||
// being wrong:
|
||||
//
|
||||
// - **enable must not touch the loader.** Disable ran the module's onShutdown;
|
||||
// there is no onBoot re-dispatch, so flipping the record back would put a
|
||||
// module with closed sockets and cleared timers back on the nav.
|
||||
// - **purge must run before the directory is removed.** purge.sql lives inside
|
||||
// that directory. Reorder those two lines and the feature silently stops
|
||||
// working, with a 200 and no data deleted.
|
||||
//
|
||||
// Point the DB at a closed port BEFORE requiring anything that builds the pool.
|
||||
process.env.DB_HOST = '127.0.0.1'
|
||||
process.env.DB_PORT = '59999'
|
||||
|
||||
const { test, beforeEach, after } = require('node:test')
|
||||
const assert = require('node:assert/strict')
|
||||
|
||||
const ctrl = require('../src/router/v1/admin/modules.controller')
|
||||
const modules = require('../src/model/modules/modules.model')
|
||||
const activity = require('../src/model/activity/activity.model')
|
||||
const settings = require('../src/model/settings/settings.model')
|
||||
const loader = require('../src/modules/loader')
|
||||
const lifecycle = require('../src/modules/lifecycle')
|
||||
const install = require('../src/modules/install')
|
||||
const declared = require('../src/modules/declared')
|
||||
const schema = require('../src/modules/schema')
|
||||
const db = require('../src/utils/db')
|
||||
|
||||
after(() => db.close())
|
||||
|
||||
function mockRes() {
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: null,
|
||||
status(c) { this.statusCode = c; return this },
|
||||
json(b) { this.body = b; return this },
|
||||
}
|
||||
}
|
||||
|
||||
const req = (extra = {}) => ({
|
||||
user: { id: 1, username: 'admin' },
|
||||
params: {},
|
||||
query: {},
|
||||
body: {},
|
||||
...extra,
|
||||
})
|
||||
|
||||
// Everything the controller reaches for, replaced wholesale per test. Restored
|
||||
// from these originals rather than from a snapshot taken mid-run, so one test
|
||||
// leaking a stub cannot quietly become another test's fixture.
|
||||
const originals = {
|
||||
modules: { ...modules },
|
||||
activity: { log: activity.log },
|
||||
settings: { get: settings.get, set: settings.set },
|
||||
loader: { isLoaded: loader.isLoaded, list: loader.list },
|
||||
lifecycle: { stop: lifecycle.stop },
|
||||
install: {
|
||||
install: install.install,
|
||||
isInstalled: install.isInstalled,
|
||||
purgeFile: install.purgeFile,
|
||||
removeDir: install.removeDir,
|
||||
},
|
||||
schema: { runPurge: schema.runPurge },
|
||||
declared: { state: declared.state },
|
||||
}
|
||||
|
||||
let logged
|
||||
|
||||
beforeEach(() => {
|
||||
Object.assign(modules, originals.modules)
|
||||
Object.assign(activity, originals.activity)
|
||||
Object.assign(settings, originals.settings)
|
||||
Object.assign(loader, originals.loader)
|
||||
Object.assign(lifecycle, originals.lifecycle)
|
||||
Object.assign(install, originals.install)
|
||||
Object.assign(schema, originals.schema)
|
||||
Object.assign(declared, originals.declared)
|
||||
|
||||
logged = []
|
||||
activity.log = async (entry) => { logged.push(entry) }
|
||||
settings.get = async () => 'gitea.whitlocktech.com'
|
||||
loader.isLoaded = () => true
|
||||
loader.list = () => []
|
||||
})
|
||||
|
||||
// ── list ───────────────────────────────────────────────────────────────────
|
||||
|
||||
test('list reconciles the row, the loader and the volume without picking a winner', async () => {
|
||||
// The case §2.4 creates and decision 3 makes routine: the row says `enabled`
|
||||
// because the operator just switched it back on, the loader still says
|
||||
// `disabled` because its onShutdown has run and there is no way back without a
|
||||
// restart. Rendering either one alone would be a lie.
|
||||
modules.list = async () => [{
|
||||
id: 'uo', name: 'UO', version: '1.0.0', state: 'enabled',
|
||||
failureStage: null, failureReason: null, source: 'https://x/y.json', sha256: 'a'.repeat(64),
|
||||
installedAt: null, startedAt: null,
|
||||
}]
|
||||
loader.list = () => [{ id: 'uo', name: 'UO', version: '1.0.0', state: 'disabled', stage: null, reason: null, capabilities: ['shard'] }]
|
||||
install.isInstalled = () => true
|
||||
install.purgeFile = () => '/modules/uo/server/db/purge.sql'
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.list(req(), res)
|
||||
|
||||
const [m] = res.body.modules
|
||||
assert.equal(m.state, 'enabled', 'what the operator decided')
|
||||
assert.equal(m.liveState, 'disabled', 'what is actually answering')
|
||||
assert.equal(m.onVolume, true)
|
||||
assert.equal(m.canPurge, true)
|
||||
assert.deepEqual(m.capabilities, ['shard'])
|
||||
assert.deepEqual(res.body.sourceHosts, ['gitea.whitlocktech.com'])
|
||||
})
|
||||
|
||||
test('list includes a module on the volume that has no row yet', async () => {
|
||||
// A hand-placed directory before its first boot. §2.5 keeps that a supported
|
||||
// install, and its routes are already being served — a screen showing nothing
|
||||
// for it would be showing the wrong thing.
|
||||
modules.list = async () => []
|
||||
loader.list = () => [{ id: 'byhand', name: 'By Hand', version: '0.1.0', state: 'started', stage: null, reason: null, capabilities: [] }]
|
||||
install.isInstalled = () => true
|
||||
install.purgeFile = () => null
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.list(req(), res)
|
||||
|
||||
assert.equal(res.body.modules.length, 1)
|
||||
assert.equal(res.body.modules[0].id, 'byhand')
|
||||
assert.equal(res.body.modules[0].state, null, 'no row means no recorded state, not a guessed one')
|
||||
assert.equal(res.body.modules[0].liveState, 'started')
|
||||
})
|
||||
|
||||
test('list carries what MODULES declares, including a module it could not install', async () => {
|
||||
// Slice 3's fourth source. A declared module that failed to resolve has no
|
||||
// row, no directory and nothing mounted — so it is invisible to the other
|
||||
// three, and the reason it is missing is the one thing the operator needs.
|
||||
modules.list = async () => [{
|
||||
id: 'uo', name: 'UO', version: '0.2.0', state: 'enabled',
|
||||
failureStage: null, failureReason: null, source: null, sha256: null,
|
||||
installedAt: null, startedAt: null,
|
||||
}]
|
||||
loader.list = () => [{ id: 'uo', name: 'UO', version: '0.2.0', state: 'started', stage: null, reason: null, capabilities: [] }]
|
||||
install.isInstalled = (id) => id === 'uo'
|
||||
install.purgeFile = () => null
|
||||
declared.state = () => [
|
||||
{ id: 'uo', version: '0.3.0', url: 'https://x/uo.json', action: 'failed', message: 'the host is down' },
|
||||
{ id: 'market', version: '1.0.0', url: 'https://x/market.json', action: 'failed', message: 'not found' },
|
||||
]
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.list(req(), res)
|
||||
|
||||
const byId = Object.fromEntries(res.body.modules.map((m) => [m.id, m]))
|
||||
// Running fine at 0.2.0 while the declared upgrade to 0.3.0 is failing: both
|
||||
// facts survive, because collapsing them would have to discard one.
|
||||
assert.equal(byId.uo.liveState, 'started')
|
||||
assert.equal(byId.uo.declared, true)
|
||||
assert.equal(byId.uo.declaredVersion, '0.3.0')
|
||||
assert.equal(byId.uo.declaredError, 'the host is down')
|
||||
// Declared and nowhere: listed anyway, with no version invented for it.
|
||||
assert.equal(byId.market.declared, true)
|
||||
assert.equal(byId.market.version, null)
|
||||
assert.equal(byId.market.state, null)
|
||||
assert.equal(byId.market.declaredError, 'not found')
|
||||
})
|
||||
|
||||
test('a successfully resolved module is marked declared, with no error', async () => {
|
||||
modules.list = async () => []
|
||||
loader.list = () => [{ id: 'uo', name: 'UO', version: '0.3.0', state: 'started', stage: null, reason: null, capabilities: [] }]
|
||||
install.isInstalled = () => true
|
||||
install.purgeFile = () => null
|
||||
declared.state = () => [{ id: 'uo', version: '0.3.0', url: 'https://x/uo.json', action: 'noop', message: null }]
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.list(req(), res)
|
||||
|
||||
assert.equal(res.body.modules.length, 1, 'declared and present is ONE module, not two')
|
||||
assert.equal(res.body.modules[0].declared, true)
|
||||
assert.equal(res.body.modules[0].declaredError, null)
|
||||
})
|
||||
|
||||
test('list survives a process where the loader never scanned', async () => {
|
||||
loader.isLoaded = () => false
|
||||
loader.list = () => { throw new Error('modules.list() before modules.load()') }
|
||||
modules.list = async () => [{ id: 'uo', name: 'UO', version: '1', state: 'disabled' }]
|
||||
install.isInstalled = () => false
|
||||
install.purgeFile = () => null
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.list(req(), res)
|
||||
|
||||
assert.equal(res.statusCode, 200)
|
||||
assert.equal(res.body.modules[0].liveState, null)
|
||||
})
|
||||
|
||||
// ── install ────────────────────────────────────────────────────────────────
|
||||
|
||||
test('install records provenance and says a restart is needed', async () => {
|
||||
const calls = []
|
||||
install.install = async ({ url, hosts }) => {
|
||||
calls.push({ url, hosts })
|
||||
return { id: 'uo', name: 'UO', version: '1.0.0', sha256: 'b'.repeat(64), source: url, replaced: false }
|
||||
}
|
||||
modules.recordInstalled = async (row) => { calls.push(row); return { ...row, state: 'installed' } }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.create(req({ body: { url: 'https://gitea.whitlocktech.com/x/uo.json' } }), res)
|
||||
|
||||
assert.equal(res.statusCode, 201)
|
||||
assert.equal(res.body.restartRequired, true)
|
||||
assert.deepEqual(calls[0].hosts, ['gitea.whitlocktech.com'], 'the allowlist comes from the setting')
|
||||
// Provenance is written HERE and nowhere else — the boot reconcile records a
|
||||
// module with null source/sha256 and leaves what it is not given.
|
||||
assert.equal(calls[1].source, 'https://gitea.whitlocktech.com/x/uo.json')
|
||||
assert.equal(calls[1].sha256, 'b'.repeat(64))
|
||||
assert.equal(logged[0].action, 'module.install')
|
||||
})
|
||||
|
||||
test('an install refusal is reported to the operator, with its own status', async () => {
|
||||
install.install = async () => {
|
||||
const err = new Error('"evil.net" is not an allowed module source host')
|
||||
err.name = 'InstallError'
|
||||
err.status = 400
|
||||
throw err
|
||||
}
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.create(req({ body: { url: 'https://evil.net/x.json' } }), res)
|
||||
|
||||
assert.equal(res.statusCode, 400)
|
||||
// The message is the useful part: the operator pasted a URL and needs to know
|
||||
// what was wrong with what came back.
|
||||
assert.match(res.body.message, /not an allowed module source host/)
|
||||
assert.equal(logged.length, 0, 'a refused install is not an audit-log entry')
|
||||
})
|
||||
|
||||
test('an unreachable host is a 502, not a 400', async () => {
|
||||
install.install = async () => {
|
||||
const err = new Error('could not reach x: timeout')
|
||||
err.name = 'InstallError'
|
||||
err.status = 502
|
||||
throw err
|
||||
}
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.create(req({ body: { url: 'https://gitea.whitlocktech.com/x.json' } }), res)
|
||||
assert.equal(res.statusCode, 502)
|
||||
})
|
||||
|
||||
test('an unexpected failure is a 500 and does not leak its message', async () => {
|
||||
install.install = async () => { throw new Error('ENOENT /some/internal/path') }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.create(req({ body: { url: 'https://gitea.whitlocktech.com/x.json' } }), res)
|
||||
|
||||
assert.equal(res.statusCode, 500)
|
||||
assert.equal(res.body.message, 'Internal Server Error')
|
||||
})
|
||||
|
||||
// ── enable / disable ───────────────────────────────────────────────────────
|
||||
|
||||
test('enable moves the row and does NOT touch the loader', async () => {
|
||||
// The decision-3 invariant. Re-enabling cannot restart a module: its
|
||||
// onShutdown has run, and MODULE_API.md has never promised onBoot is safe to
|
||||
// run twice. Flipping the record would put it back on the nav with a
|
||||
// torn-down world behind it.
|
||||
let setStateCalled = false
|
||||
loader.setState = () => { setStateCalled = true }
|
||||
modules.enable = async (id) => ({ id, state: 'enabled' })
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.enable(req({ params: { id: 'uo' } }), res)
|
||||
|
||||
assert.equal(res.body.module.state, 'enabled')
|
||||
assert.equal(res.body.restartRequired, true)
|
||||
assert.equal(setStateCalled, false, 'enable must not move the in-memory record')
|
||||
assert.equal(logged[0].action, 'module.enable')
|
||||
loader.setState = originals.loader.setState
|
||||
})
|
||||
|
||||
test('enabling a module with no row is a 404', async () => {
|
||||
modules.enable = async () => null
|
||||
const res = mockRes()
|
||||
await ctrl.enable(req({ params: { id: 'ghost' } }), res)
|
||||
assert.equal(res.statusCode, 404)
|
||||
})
|
||||
|
||||
test('an illegal transition is a 409, not a 500', async () => {
|
||||
modules.enable = async () => {
|
||||
const err = new Error("module 'uo': cannot move from 'x' to 'enabled'")
|
||||
err.name = 'ModuleStateError'
|
||||
throw err
|
||||
}
|
||||
const res = mockRes()
|
||||
await ctrl.enable(req({ params: { id: 'uo' } }), res)
|
||||
assert.equal(res.statusCode, 409)
|
||||
})
|
||||
|
||||
test('disable stops the module and reports whether the hook ran', async () => {
|
||||
const calls = []
|
||||
modules.get = async (id) => ({ id, state: 'started' })
|
||||
lifecycle.stop = async (id) => { calls.push(id); return { stopped: true, error: null } }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.disable(req({ params: { id: 'uo' } }), res)
|
||||
|
||||
assert.deepEqual(calls, ['uo'])
|
||||
assert.equal(res.body.stopped, true)
|
||||
// No restart: this is the one action that takes effect immediately, and it is
|
||||
// the one an operator reaches for when something is going wrong.
|
||||
assert.equal(res.body.restartRequired, undefined)
|
||||
assert.equal(logged[0].action, 'module.disable')
|
||||
})
|
||||
|
||||
test('a shutdown hook that failed is reported rather than swallowed', async () => {
|
||||
modules.get = async (id) => ({ id, state: 'started' })
|
||||
lifecycle.stop = async () => ({ stopped: false, error: 'socket would not close' })
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.disable(req({ params: { id: 'uo' } }), res)
|
||||
|
||||
// It IS disabled either way; the operator should be told it did not close
|
||||
// cleanly while they still have the logs in front of them.
|
||||
assert.equal(res.statusCode, 200)
|
||||
assert.match(res.body.shutdownError, /socket would not close/)
|
||||
})
|
||||
|
||||
// ── uninstall and purge ────────────────────────────────────────────────────
|
||||
|
||||
test('uninstall purges BEFORE it removes the directory', async () => {
|
||||
// The ordering that makes decision 5 work at all: purge.sql is a file inside
|
||||
// the directory being deleted. Swap these two and the endpoint still answers
|
||||
// 200 and deletes nothing.
|
||||
const order = []
|
||||
modules.get = async (id) => ({ id, state: 'started' })
|
||||
install.isInstalled = () => true
|
||||
install.purgeFile = () => '/modules/uo/server/db/purge.sql'
|
||||
schema.runPurge = async () => { order.push('purge'); return 12 }
|
||||
lifecycle.stop = async () => { order.push('stop'); return { stopped: true, error: null } }
|
||||
install.removeDir = async () => { order.push('removeDir'); return true }
|
||||
modules.remove = async () => { order.push('removeRow') }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.remove(req({ params: { id: 'uo' }, query: { purge: 'true' } }), res)
|
||||
|
||||
assert.deepEqual(order, ['purge', 'stop', 'removeDir', 'removeRow'])
|
||||
assert.equal(res.body.purged, 12)
|
||||
assert.equal(res.body.restartRequired, true)
|
||||
assert.equal(logged[0].action, 'module.purge')
|
||||
})
|
||||
|
||||
test('a plain uninstall keeps the row and does not purge', async () => {
|
||||
const order = []
|
||||
modules.get = async (id) => ({ id, state: 'started' })
|
||||
install.isInstalled = () => true
|
||||
schema.runPurge = async () => { order.push('purge'); return 1 }
|
||||
lifecycle.stop = async () => { order.push('stop'); return { stopped: true, error: null } }
|
||||
install.removeDir = async () => { order.push('removeDir'); return true }
|
||||
modules.remove = async () => { order.push('removeRow') }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.remove(req({ params: { id: 'uo' } }), res)
|
||||
|
||||
// §2.5's default: the directory goes, the data stays, and the disabled row is
|
||||
// what keeps the retained data visible and the module reinstallable.
|
||||
assert.deepEqual(order, ['stop', 'removeDir'])
|
||||
assert.equal(res.body.purged, null)
|
||||
assert.equal(logged[0].action, 'module.uninstall')
|
||||
})
|
||||
|
||||
test('asking to purge a module that ships no purge.sql refuses instead of pretending', async () => {
|
||||
modules.get = async (id) => ({ id, state: 'started' })
|
||||
install.isInstalled = () => true
|
||||
install.purgeFile = () => null
|
||||
let removed = false
|
||||
install.removeDir = async () => { removed = true; return true }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.remove(req({ params: { id: 'uo' }, query: { purge: 'true' } }), res)
|
||||
|
||||
assert.equal(res.statusCode, 400)
|
||||
assert.match(res.body.message, /ships no purge.sql/)
|
||||
// Nothing happened. The operator asked for the module AND its data to go; the
|
||||
// data cannot go, so doing half of it silently would be the worst answer.
|
||||
assert.equal(removed, false)
|
||||
})
|
||||
|
||||
test('uninstalling something that is neither on the volume nor in a row is a 404', async () => {
|
||||
modules.get = async () => null
|
||||
install.isInstalled = () => false
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.remove(req({ params: { id: 'ghost' } }), res)
|
||||
assert.equal(res.statusCode, 404)
|
||||
})
|
||||
|
||||
test('standalone purge refuses while the module is still running', async () => {
|
||||
// Dropping the tables under a module that is still serving leaves it answering
|
||||
// out of a world that no longer exists. Disabling first is one click.
|
||||
modules.get = async (id) => ({ id, state: 'started' })
|
||||
let ran = false
|
||||
schema.runPurge = async () => { ran = true; return 1 }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.purge(req({ params: { id: 'uo' } }), res)
|
||||
|
||||
assert.equal(res.statusCode, 409)
|
||||
assert.match(res.body.message, /Disable this module before purging/)
|
||||
assert.equal(ran, false)
|
||||
})
|
||||
|
||||
test('standalone purge runs on a disabled module', async () => {
|
||||
modules.get = async (id) => ({ id, state: 'disabled' })
|
||||
install.purgeFile = () => '/modules/uo/server/db/purge.sql'
|
||||
schema.runPurge = async () => 7
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.purge(req({ params: { id: 'uo' } }), res)
|
||||
|
||||
assert.equal(res.body.purged, 7)
|
||||
assert.equal(logged[0].action, 'module.purge')
|
||||
})
|
||||
|
||||
// ── the allowlist ──────────────────────────────────────────────────────────
|
||||
|
||||
test('setSources stores a normalised list and audits the change', async () => {
|
||||
let stored = null
|
||||
settings.set = async (key, value) => { stored = { key, value } }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.setSources(req({ body: { hosts: 'Gitea.Example.com, releases.example.org' } }), res)
|
||||
|
||||
assert.deepEqual(res.body.sourceHosts, ['gitea.example.com', 'releases.example.org'])
|
||||
assert.equal(stored.key, ctrl.HOSTS_KEY)
|
||||
assert.equal(stored.value, 'gitea.example.com,releases.example.org')
|
||||
// Before AND after: this setting decides what code the site will execute, so
|
||||
// the audit entry has to say what it used to be.
|
||||
assert.equal(logged[0].action, 'module.sources')
|
||||
assert.deepEqual(logged[0].detail.before, ['gitea.whitlocktech.com'])
|
||||
})
|
||||
|
||||
test('setSources refuses anything that is not a bare hostname', async () => {
|
||||
let stored = false
|
||||
settings.set = async () => { stored = true }
|
||||
|
||||
for (const bad of ['https://x.com', 'x.com/path', 'x.com:8443', '*.x.com', 'x_y.com']) {
|
||||
const res = mockRes()
|
||||
// eslint-disable-next-line no-await-in-loop
|
||||
await ctrl.setSources(req({ body: { hosts: bad } }), res)
|
||||
assert.equal(res.statusCode, 400, `${bad} should be refused`)
|
||||
}
|
||||
assert.equal(stored, false)
|
||||
})
|
||||
|
||||
test('an empty allowlist is storable, and means no installs', async () => {
|
||||
// Not a wildcard, and not an error: "nothing may be installed" is a position
|
||||
// an operator is entitled to take.
|
||||
let stored = null
|
||||
settings.set = async (key, value) => { stored = value }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.setSources(req({ body: { hosts: '' } }), res)
|
||||
|
||||
assert.equal(res.statusCode, 200)
|
||||
assert.deepEqual(res.body.sourceHosts, [])
|
||||
assert.equal(stored, '')
|
||||
})
|
||||
|
||||
// ── restart ────────────────────────────────────────────────────────────────
|
||||
|
||||
// These listen for the SIGTERM EVENT rather than stubbing `process.kill`, and
|
||||
// that is the whole point of them now.
|
||||
//
|
||||
// The first version of this route called `process.kill(process.pid, 'SIGTERM')`
|
||||
// and the first version of these tests stubbed `process.kill` and asserted it
|
||||
// had been called with SIGTERM. Both passed. Both were wrong: Windows has no
|
||||
// POSIX signals, and Node documents SIGTERM there as unconditional termination —
|
||||
// so on a Windows host the route killed the server outright, with no module
|
||||
// `onShutdown`, no pool close and no log flush. A stub of `process.kill` cannot
|
||||
// see that, because what it asserts is precisely the call whose MEANING differs
|
||||
// by platform.
|
||||
//
|
||||
// Asserting on the event closes the gap: it is what server.js's handler is
|
||||
// actually subscribed to, so a test passing here means the handler would run.
|
||||
function onceSigterm() {
|
||||
return new Promise((resolve, reject) => {
|
||||
const timer = setTimeout(() => {
|
||||
process.removeListener('SIGTERM', handler)
|
||||
reject(new Error('no SIGTERM was emitted within 1s'))
|
||||
}, 1000)
|
||||
function handler() {
|
||||
clearTimeout(timer)
|
||||
process.removeListener('SIGTERM', handler)
|
||||
resolve(true)
|
||||
}
|
||||
process.on('SIGTERM', handler)
|
||||
})
|
||||
}
|
||||
|
||||
test('restart answers first, then triggers the one graceful-shutdown path', async () => {
|
||||
const fired = onceSigterm()
|
||||
const res = mockRes()
|
||||
|
||||
ctrl.restart(req(), res)
|
||||
|
||||
// Answered synchronously: once the shutdown starts there is no listener left
|
||||
// to flush a response through, so the operator would be told nothing.
|
||||
assert.equal(res.statusCode, 202)
|
||||
assert.equal(res.body.restarting, true)
|
||||
|
||||
assert.equal(await fired, true)
|
||||
assert.equal(logged[0].action, 'module.restart')
|
||||
})
|
||||
|
||||
test('a failure to write the audit entry does not cancel the restart', async () => {
|
||||
activity.log = async () => { throw new Error('database is gone') }
|
||||
const fired = onceSigterm()
|
||||
|
||||
ctrl.restart(req(), mockRes())
|
||||
|
||||
assert.equal(await fired, true)
|
||||
})
|
||||
82
server/test/bootOrder.test.js
Normal file
82
server/test/bootOrder.test.js
Normal file
@@ -0,0 +1,82 @@
|
||||
// server.js's boot ORDER, which is a contract and not a style choice.
|
||||
//
|
||||
// Phase 4, slice 3 of MODULE_SYSTEM.md §2.7.2. Five steps have to happen in one
|
||||
// order, and each arrow is a dependency that is invisible at the call site:
|
||||
//
|
||||
// core schema → the declared set (§2.7.2 decision 4) needs the settings row
|
||||
// its seed writes, to know which hosts it may install from
|
||||
// declared set → requiring app.js SCANS the volume (§1.12), so anything put
|
||||
// there afterwards is not in this process
|
||||
// require app → module schema fragments (§2.6) can only be replayed once the
|
||||
// loader knows which modules there are
|
||||
// fragments → onBoot runs against tables that exist
|
||||
//
|
||||
// This is a source-structure test, and it is worth being plain about what that
|
||||
// does and does not prove: it cannot tell you the server boots, only that nobody
|
||||
// has quietly moved one of these five lines past another. It exists because the
|
||||
// defect it guards against has already happened once and was invisible to every
|
||||
// other kind of test here. Deferring the app require — which slice 3 had to do —
|
||||
// moved core's schema ahead of the scan, and `ensureSchema` then skipped the
|
||||
// module fragments entirely. On this machine's dev database the tables already
|
||||
// existed, so the module started; on a FRESH database it would have started
|
||||
// against no tables at all. Every suite in this directory stubs either the
|
||||
// loader or the pool, so none of them could see it. The browser smoke did, from
|
||||
// one log line.
|
||||
process.env.DB_HOST = '127.0.0.1'
|
||||
process.env.DB_PORT = '59999'
|
||||
|
||||
const fs = require('fs')
|
||||
const path = require('path')
|
||||
|
||||
const { test } = require('node:test')
|
||||
const assert = require('node:assert/strict')
|
||||
|
||||
const SERVER = fs.readFileSync(path.join(__dirname, '..', 'src', 'server.js'), 'utf8')
|
||||
|
||||
/** Where a marker appears, asserted to appear exactly once. */
|
||||
function at(marker) {
|
||||
const first = SERVER.indexOf(marker)
|
||||
assert.notEqual(first, -1, `server.js no longer contains ${JSON.stringify(marker)}`)
|
||||
assert.equal(
|
||||
SERVER.indexOf(marker, first + 1),
|
||||
-1,
|
||||
`${JSON.stringify(marker)} appears more than once in server.js — this test cannot tell which is the boot step`,
|
||||
)
|
||||
return first
|
||||
}
|
||||
|
||||
test('boot runs core schema, then the declared set, then the scan, then fragments, then onBoot', () => {
|
||||
const steps = [
|
||||
['core schema', at('await ensureSchema({ replayModules: false })')],
|
||||
['declared module set', at('await declaredModules.resolve(')],
|
||||
['the volume scan', at("require('./app')")],
|
||||
['module schema fragments', at('replayFragments()')],
|
||||
['module onBoot', at('await moduleLifecycle.boot()')],
|
||||
]
|
||||
|
||||
for (let i = 1; i < steps.length; i += 1) {
|
||||
assert.ok(
|
||||
steps[i][1] > steps[i - 1][1],
|
||||
`${steps[i][0]} must come after ${steps[i - 1][0]} in server.js`,
|
||||
)
|
||||
}
|
||||
})
|
||||
|
||||
test('app.js is required inside start(), never at the top of the file', () => {
|
||||
// The whole mechanism depends on this. A top-level require runs when server.js
|
||||
// is loaded — before a single line of start() — and the scan would then happen
|
||||
// before the declared set had a chance to put anything on the volume, silently
|
||||
// and with no error anywhere.
|
||||
assert.ok(
|
||||
at("require('./app')") > at('async function start()'),
|
||||
'requiring ./app at the top of server.js scans the volume before the declared set is resolved',
|
||||
)
|
||||
})
|
||||
|
||||
test('ensureSchema is asked NOT to replay fragments, and something else does', () => {
|
||||
// Both halves matter. Passing the flag without replaying elsewhere is the
|
||||
// original defect with an explicit spelling; replaying without the flag runs
|
||||
// the fragments twice, the second time being the one that matters.
|
||||
assert.match(SERVER, /ensureSchema\(\{ replayModules: false \}\)/)
|
||||
assert.match(SERVER, /modules\/schema'\)\.replayFragments\(\)/)
|
||||
})
|
||||
@@ -60,7 +60,13 @@ test('backoffGuard returns a generic 429 while locked out', async () => {
|
||||
a.post('/login', lp.backoffGuard, (req, res) => res.json({ ok: true }))
|
||||
})
|
||||
try {
|
||||
lp.recordFailure('203.0.113.40') // lock the test client IP
|
||||
// Lock the test client IP. FIVE failures, not one: the lock is
|
||||
// `BASE_MS * 2 ** (count - 1)`, so a single failure locks for exactly one
|
||||
// second and this test then races the round trip. It lost that race on CI
|
||||
// (200 instead of 429, request arriving 1,456 ms after the lock). Five
|
||||
// failures lock for sixteen seconds, which is not a race. What is under
|
||||
// test is the guard's ANSWER while locked out, and that is unchanged.
|
||||
for (let i = 0; i < 5; i += 1) lp.recordFailure('203.0.113.40')
|
||||
const res = await fetch(`${app.url}/login`, {
|
||||
method: 'POST',
|
||||
headers: { 'X-Forwarded-For': '203.0.113.40' },
|
||||
|
||||
252
server/test/moduleArchive.test.js
Normal file
252
server/test/moduleArchive.test.js
Normal file
@@ -0,0 +1,252 @@
|
||||
// modules/archive.js — the hardened bundle extractor (MODULE_SYSTEM.md §2.7.2).
|
||||
//
|
||||
// Every rejection case is exercised against a REAL archive rather than a mocked
|
||||
// tar parser, because the thing being tested is what the parser reports for a
|
||||
// given sequence of bytes. A fake that returns `{type: 'SymbolicLink'}` proves
|
||||
// only that the `if` is spelled correctly.
|
||||
//
|
||||
// The hostile archives are written here as raw ustar headers instead of being
|
||||
// produced with `tar`, for two reasons that both bit during this slice:
|
||||
//
|
||||
// 1. `ln -s` needs a privilege Windows does not hand out by default, so a
|
||||
// symlink fixture built with the shell is a fixture that silently is not
|
||||
// one, and the test passes for the wrong reason on the machine most of this
|
||||
// work happens on.
|
||||
// 2. GNU tar will not emit `../escape` or `/etc/passwd` as a member name — it
|
||||
// strips them and tells you so. The archives worth defending against are
|
||||
// exactly the ones a cooperative archiver refuses to produce.
|
||||
//
|
||||
// A ustar header is 512 bytes of fixed-offset fields, so writing one is less
|
||||
// code than persuading a tool to misbehave.
|
||||
|
||||
const test = require('node:test')
|
||||
const assert = require('node:assert/strict')
|
||||
const fs = require('fs')
|
||||
const os = require('os')
|
||||
const path = require('path')
|
||||
const zlib = require('zlib')
|
||||
|
||||
const archive = require('../src/modules/archive')
|
||||
|
||||
// ── A minimal ustar writer ─────────────────────────────────────────────────
|
||||
|
||||
const BLOCK = 512
|
||||
|
||||
function octal(value, width) {
|
||||
// ustar numeric fields are NUL-terminated octal, right-aligned with zeros.
|
||||
return Number(value).toString(8).padStart(width - 1, '0') + '\0'
|
||||
}
|
||||
|
||||
/**
|
||||
* One 512-byte header plus its padded data.
|
||||
*
|
||||
* @param {object} entry
|
||||
* @param {string} entry.name member path
|
||||
* @param {string} [entry.type] '0' file · '5' dir · '2' symlink · '1' hardlink
|
||||
* · '3' char dev · '6' FIFO
|
||||
* @param {string} [entry.linkname] target, for the link types
|
||||
* @param {string} [entry.body] file contents
|
||||
* @param {number} [entry.size] declared size — defaults to the body's, and
|
||||
* may be set independently to build a header
|
||||
* that lies about its payload
|
||||
*/
|
||||
function member({ name, type = '0', linkname = '', body = '', size = null }) {
|
||||
const header = Buffer.alloc(BLOCK, 0)
|
||||
const data = Buffer.from(body, 'utf8')
|
||||
const declared = size === null ? data.length : size
|
||||
|
||||
header.write(name, 0, 100, 'utf8')
|
||||
header.write(octal(0o644, 8), 100, 8, 'ascii') // mode
|
||||
header.write(octal(0, 8), 108, 8, 'ascii') // uid
|
||||
header.write(octal(0, 8), 116, 8, 'ascii') // gid
|
||||
header.write(octal(declared, 12), 124, 12, 'ascii')
|
||||
header.write(octal(0, 12), 136, 12, 'ascii') // mtime
|
||||
header.write(' ', 148, 8, 'ascii') // checksum field is spaces while summing
|
||||
header.write(type, 156, 1, 'ascii')
|
||||
header.write(linkname, 157, 100, 'utf8')
|
||||
header.write('ustar\0', 257, 6, 'ascii')
|
||||
header.write('00', 263, 2, 'ascii')
|
||||
|
||||
let sum = 0
|
||||
for (const byte of header) sum += byte
|
||||
header.write(`${sum.toString(8).padStart(6, '0')}\0 `, 148, 8, 'ascii')
|
||||
|
||||
const padding = Buffer.alloc((BLOCK - (data.length % BLOCK)) % BLOCK, 0)
|
||||
return Buffer.concat([header, data, padding])
|
||||
}
|
||||
|
||||
/** Gzip a set of members into a .tar.gz on disk, and return its path. */
|
||||
function writeArchive(dir, filename, members) {
|
||||
const tarball = Buffer.concat([
|
||||
...members.map(member),
|
||||
Buffer.alloc(BLOCK * 2, 0), // two zero blocks end the archive
|
||||
])
|
||||
const file = path.join(dir, filename)
|
||||
fs.writeFileSync(file, zlib.gzipSync(tarball))
|
||||
return file
|
||||
}
|
||||
|
||||
/** A well-formed bundle: one top-level directory, ordinary files inside it. */
|
||||
function goodMembers(root = 'module-uo-1.0.0') {
|
||||
return [
|
||||
{ name: `${root}/`, type: '5' },
|
||||
{ name: `${root}/module.json`, body: '{"id":"uo","name":"UO","version":"1.0.0"}' },
|
||||
{ name: `${root}/server/`, type: '5' },
|
||||
{ name: `${root}/server/index.js`, body: 'module.exports = () => {}\n' },
|
||||
]
|
||||
}
|
||||
|
||||
// One scratch directory for the whole file, removed at the end.
|
||||
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'rg-archive-'))
|
||||
test.after(() => fs.rmSync(tmp, { recursive: true, force: true }))
|
||||
|
||||
/** Assert that inspecting `members` fails, and that the message says why. */
|
||||
async function rejects(name, members, matcher) {
|
||||
const file = writeArchive(tmp, `${name}.tar.gz`, members)
|
||||
await assert.rejects(
|
||||
() => archive.inspect(file),
|
||||
(err) => {
|
||||
assert.equal(err.name, 'ArchiveError', `expected an ArchiveError, got ${err.name}: ${err.message}`)
|
||||
assert.match(err.message, matcher)
|
||||
return true
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
// ── What a good bundle does ────────────────────────────────────────────────
|
||||
|
||||
test('inspect accepts a well-formed bundle and reports its single root', async () => {
|
||||
const file = writeArchive(tmp, 'good.tar.gz', goodMembers())
|
||||
const stats = await archive.inspect(file)
|
||||
|
||||
assert.equal(stats.root, 'module-uo-1.0.0')
|
||||
assert.equal(stats.entries, 4)
|
||||
assert.ok(stats.bytes > 0)
|
||||
})
|
||||
|
||||
test('extract strips the top level, so the bundle lands as the module id', async () => {
|
||||
// The wrapper directory is the publisher's naming (module-uo's release
|
||||
// workflow packs `module-uo-<version>/`); the directory it lands in is core's,
|
||||
// and has to be the id the loader scans for.
|
||||
const file = writeArchive(tmp, 'strip.tar.gz', goodMembers())
|
||||
const dest = path.join(tmp, 'unpacked-uo')
|
||||
|
||||
await archive.unpack(file, dest)
|
||||
|
||||
assert.ok(fs.existsSync(path.join(dest, 'module.json')), 'module.json should be at the root of the destination')
|
||||
assert.ok(fs.existsSync(path.join(dest, 'server', 'index.js')))
|
||||
assert.ok(!fs.existsSync(path.join(dest, 'module-uo-1.0.0')), 'the wrapper directory should not survive')
|
||||
})
|
||||
|
||||
test('extract refuses a destination that already exists', async () => {
|
||||
const file = writeArchive(tmp, 'exists.tar.gz', goodMembers())
|
||||
const dest = path.join(tmp, 'already-there')
|
||||
fs.mkdirSync(dest)
|
||||
|
||||
await assert.rejects(() => archive.extract(file, dest), /already exists/)
|
||||
})
|
||||
|
||||
// ── What a hostile bundle does ─────────────────────────────────────────────
|
||||
|
||||
test('an absolute member path is refused', async () => {
|
||||
await rejects('absolute', [
|
||||
{ name: 'mod/', type: '5' },
|
||||
{ name: '/etc/cron.d/pwned', body: '* * * * * root sh\n' },
|
||||
], /absolute/)
|
||||
})
|
||||
|
||||
test('a drive-absolute member path is refused', async () => {
|
||||
// Its own node-tar advisory, and invisible to a leading-slash check.
|
||||
await rejects('drive', [
|
||||
{ name: 'mod/', type: '5' },
|
||||
{ name: 'C:\\Windows\\Temp\\pwned', body: 'x' },
|
||||
], /backslash|drive-absolute/)
|
||||
})
|
||||
|
||||
test('an upward-escaping member path is refused', async () => {
|
||||
await rejects('escape', [
|
||||
{ name: 'mod/', type: '5' },
|
||||
{ name: 'mod/../../../etc/passwd', body: 'root::0:0\n' },
|
||||
], /escapes upward/)
|
||||
})
|
||||
|
||||
test('a symlink member is refused, and the message names the type', async () => {
|
||||
await rejects('symlink', [
|
||||
{ name: 'mod/', type: '5' },
|
||||
{ name: 'mod/passwd', type: '2', linkname: '/etc/passwd' },
|
||||
], /SymbolicLink/)
|
||||
})
|
||||
|
||||
test('a hardlink member is refused', async () => {
|
||||
// The single most-published node-tar escape primitive. Refusing the type
|
||||
// outright is what keeps this file from depending on the library getting
|
||||
// hardlink containment right.
|
||||
await rejects('hardlink', [
|
||||
{ name: 'mod/', type: '5' },
|
||||
{ name: 'mod/shadow', type: '1', linkname: '../../../etc/shadow' },
|
||||
], /Link/)
|
||||
})
|
||||
|
||||
test('a device node is refused', async () => {
|
||||
await rejects('device', [
|
||||
{ name: 'mod/', type: '5' },
|
||||
{ name: 'mod/zero', type: '3', linkname: '' },
|
||||
], /may only contain files and directories/)
|
||||
})
|
||||
|
||||
test('a FIFO is refused', async () => {
|
||||
await rejects('fifo', [
|
||||
{ name: 'mod/', type: '5' },
|
||||
{ name: 'mod/pipe', type: '6' },
|
||||
], /may only contain files and directories/)
|
||||
})
|
||||
|
||||
test('two top-level directories are refused', async () => {
|
||||
await rejects('two-roots', [
|
||||
{ name: 'mod-a/', type: '5' },
|
||||
{ name: 'mod-a/module.json', body: '{}' },
|
||||
{ name: 'mod-b/', type: '5' },
|
||||
{ name: 'mod-b/module.json', body: '{}' },
|
||||
], /exactly one top-level directory, found 2/)
|
||||
})
|
||||
|
||||
test('a bundle whose declared sizes exceed the cap is refused before it is read to the end', async () => {
|
||||
// The header lies: it declares a gigabyte and carries nothing. That is the
|
||||
// decompression-bomb shape, and the point is that inspect() decides on the
|
||||
// DECLARED size without ever materialising the payload.
|
||||
await rejects('bomb', [
|
||||
{ name: 'mod/', type: '5' },
|
||||
{ name: 'mod/big', size: archive.MAX_BYTES + 1, body: '' },
|
||||
], /unpacks to more than/)
|
||||
})
|
||||
|
||||
test('an empty archive is refused', async () => {
|
||||
await rejects('empty', [], /empty/)
|
||||
})
|
||||
|
||||
// ── The path check on its own ──────────────────────────────────────────────
|
||||
//
|
||||
// pathProblem is exported so the cases that are awkward to express as archive
|
||||
// bytes can still be asserted directly.
|
||||
|
||||
test('pathProblem accepts ordinary bundle paths', () => {
|
||||
for (const ok of ['mod/module.json', 'mod/server/router/x.js', 'mod/a.b-c_d/e.js']) {
|
||||
assert.equal(archive.pathProblem(ok), null, `${ok} should be accepted`)
|
||||
}
|
||||
})
|
||||
|
||||
test('pathProblem rejects the escape shapes', () => {
|
||||
assert.match(archive.pathProblem('/etc/passwd'), /absolute/)
|
||||
assert.match(archive.pathProblem('C:/Windows/x'), /drive-absolute/)
|
||||
assert.match(archive.pathProblem('mod/../../x'), /escapes upward/)
|
||||
assert.match(archive.pathProblem('..'), /escapes upward/)
|
||||
assert.match(archive.pathProblem('mod\\x'), /backslash/)
|
||||
assert.match(archive.pathProblem('mod/\0/x'), /NUL byte/)
|
||||
})
|
||||
|
||||
test('pathProblem does not reject a filename that merely contains two dots', () => {
|
||||
// `..` is a SEGMENT, not a substring — a file called `version..js` is fine,
|
||||
// and a check written with `includes('..')` would refuse it.
|
||||
assert.equal(archive.pathProblem('mod/version..js'), null)
|
||||
assert.equal(archive.pathProblem('mod/..hidden'), null)
|
||||
})
|
||||
391
server/test/moduleDeclared.test.js
Normal file
391
server/test/moduleDeclared.test.js
Normal file
@@ -0,0 +1,391 @@
|
||||
// modules/declared.js — resolving the module set an environment declares.
|
||||
//
|
||||
// Phase 4, slice 3 of MODULE_SYSTEM.md §2.7.2, decision 4. Two claims are worth
|
||||
// more than the rest and both are about what does NOT happen:
|
||||
//
|
||||
// - a module already unpacked at the declared version does not touch the
|
||||
// network, because that is what makes a restart with the internet down come
|
||||
// up unchanged;
|
||||
// - a module that cannot be resolved does not fail the boot, and does not stop
|
||||
// the next declaration from resolving.
|
||||
//
|
||||
// The last test in the file runs the whole path through the real install.js
|
||||
// against a fake transport — a real gzipped tar, really hashed, really unpacked
|
||||
// — because everything above it stubs `installImpl` and would keep passing if
|
||||
// the two files stopped agreeing about what an install returns.
|
||||
process.env.DB_HOST = '127.0.0.1'
|
||||
process.env.DB_PORT = '59999'
|
||||
|
||||
const crypto = require('crypto')
|
||||
const fs = require('fs')
|
||||
const os = require('os')
|
||||
const path = require('path')
|
||||
const zlib = require('zlib')
|
||||
|
||||
const { test, beforeEach, after } = require('node:test')
|
||||
const assert = require('node:assert/strict')
|
||||
|
||||
const db = require('../src/utils/db')
|
||||
|
||||
after(() => db.close())
|
||||
|
||||
const HOSTS = ['releases.example.com']
|
||||
const URL_030 = 'https://releases.example.com/mod/uo-0.3.0.json'
|
||||
|
||||
let tmpRoot
|
||||
let declared
|
||||
|
||||
/**
|
||||
* A fresh declared.js bound to a fresh modules directory.
|
||||
*
|
||||
* loader.js resolves MODULES_DIR once at require time and install.js reads
|
||||
* `loader.dir()`, so all three have to come back together — the same dance
|
||||
* moduleInstall.test.js and moduleLifecycle.test.js do.
|
||||
*/
|
||||
function fresh(dir) {
|
||||
process.env.MODULES_DIR = dir
|
||||
for (const m of ['loader', 'install', 'declared']) {
|
||||
delete require.cache[require.resolve(`../src/modules/${m}`)]
|
||||
}
|
||||
// eslint-disable-next-line global-require
|
||||
return require('../src/modules/declared')
|
||||
}
|
||||
|
||||
/** Put a module directory on the volume, as an unpack would leave it. */
|
||||
function place(id, version) {
|
||||
const dir = path.join(tmpRoot, id)
|
||||
fs.mkdirSync(dir, { recursive: true })
|
||||
fs.writeFileSync(
|
||||
path.join(dir, 'module.json'),
|
||||
JSON.stringify({ id, name: 'Ultima Online', version, coreApi: '^1.0.0' }),
|
||||
)
|
||||
return dir
|
||||
}
|
||||
|
||||
/**
|
||||
* A stub install service.
|
||||
*
|
||||
* Records every call, so "did this reach the network at all" is a question the
|
||||
* tests can ask directly rather than inferring from an outcome.
|
||||
*/
|
||||
function fakeInstall({ version = '0.3.0', throws = null } = {}) {
|
||||
const calls = []
|
||||
return {
|
||||
calls,
|
||||
InstallError: Error,
|
||||
async install(args) {
|
||||
calls.push(args)
|
||||
if (throws) throw new Error(throws)
|
||||
return {
|
||||
id: 'uo',
|
||||
name: 'Ultima Online',
|
||||
version,
|
||||
sha256: 'a'.repeat(64),
|
||||
source: args.url,
|
||||
bytes: 1024,
|
||||
replaced: false,
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/** A modules.model stand-in that remembers what provenance it was given. */
|
||||
function fakeModel() {
|
||||
const recorded = []
|
||||
return { recorded, async recordInstalled(row) { recorded.push(row); return row } }
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
tmpRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'rg-declared-'))
|
||||
declared = fresh(tmpRoot)
|
||||
})
|
||||
|
||||
// ── Parsing ────────────────────────────────────────────────────────────────
|
||||
|
||||
test('parses id@version=url entries separated by whitespace or commas', () => {
|
||||
const { entries, errors } = declared.parse(
|
||||
` uo@0.3.0=https://a.example/uo.json,\n market@1.2.3=https://a.example/market.json `,
|
||||
)
|
||||
assert.deepEqual(errors, [])
|
||||
assert.deepEqual(entries, [
|
||||
{ id: 'uo', version: '0.3.0', url: 'https://a.example/uo.json' },
|
||||
{ id: 'market', version: '1.2.3', url: 'https://a.example/market.json' },
|
||||
])
|
||||
})
|
||||
|
||||
test('an empty or unset declaration parses to nothing, without complaint', () => {
|
||||
for (const value of [undefined, '', ' ']) {
|
||||
const { entries, errors } = declared.parse(value)
|
||||
assert.deepEqual(entries, [])
|
||||
assert.deepEqual(errors, [])
|
||||
}
|
||||
})
|
||||
|
||||
test('a malformed entry is refused on its own, leaving the others', () => {
|
||||
// A bare URL, a missing version, and an id that is not one — each of which an
|
||||
// operator can plausibly type, and none of which should cost the module that
|
||||
// was written correctly.
|
||||
const { entries, errors } = declared.parse(
|
||||
'https://a.example/uo.json uo@=https://a.example/uo.json Uo@1=https://a.example/uo.json'
|
||||
+ ' good@1.0.0=https://a.example/good.json',
|
||||
)
|
||||
assert.equal(entries.length, 1)
|
||||
assert.equal(entries[0].id, 'good')
|
||||
assert.equal(errors.length, 3)
|
||||
})
|
||||
|
||||
test('a duplicate id keeps the first and says so', () => {
|
||||
const { entries, errors } = declared.parse(
|
||||
'uo@1.0.0=https://a.example/one.json uo@2.0.0=https://a.example/two.json',
|
||||
)
|
||||
assert.equal(entries.length, 1)
|
||||
assert.equal(entries[0].version, '1.0.0')
|
||||
assert.match(errors[0], /declared more than once/)
|
||||
})
|
||||
|
||||
// ── Resolution ─────────────────────────────────────────────────────────────
|
||||
|
||||
test('a module already at the declared version is a no-op that never fetches', async () => {
|
||||
place('uo', '0.3.0')
|
||||
const installer = fakeInstall()
|
||||
const model = fakeModel()
|
||||
|
||||
const results = await declared.resolve({
|
||||
value: `uo@0.3.0=${URL_030}`,
|
||||
hosts: HOSTS,
|
||||
model,
|
||||
installImpl: installer,
|
||||
})
|
||||
|
||||
assert.deepEqual(results.map((r) => r.action), ['noop'])
|
||||
// The offline guarantee, stated as an assertion: nothing was fetched and
|
||||
// nothing was written. A restart with no route to the internet is this case.
|
||||
assert.equal(installer.calls.length, 0)
|
||||
assert.equal(model.recorded.length, 0)
|
||||
})
|
||||
|
||||
test('a missing module is installed, with its provenance recorded', async () => {
|
||||
const installer = fakeInstall()
|
||||
const model = fakeModel()
|
||||
|
||||
const results = await declared.resolve({
|
||||
value: `uo@0.3.0=${URL_030}`,
|
||||
hosts: HOSTS,
|
||||
model,
|
||||
installImpl: installer,
|
||||
})
|
||||
|
||||
assert.deepEqual(results.map((r) => r.action), ['installed'])
|
||||
assert.equal(installer.calls[0].url, URL_030)
|
||||
assert.deepEqual(installer.calls[0].hosts, HOSTS)
|
||||
// The declaration is handed down so install.js can refuse a URL that turns out
|
||||
// to be another module or another version BEFORE it downloads it.
|
||||
assert.deepEqual(installer.calls[0].expect, { id: 'uo', version: '0.3.0' })
|
||||
// Written exactly as the admin route writes it — the reason this runs in the
|
||||
// server process rather than in a script that cannot reach the database.
|
||||
assert.deepEqual(model.recorded, [
|
||||
{ id: 'uo', name: 'Ultima Online', version: '0.3.0', source: URL_030, sha256: 'a'.repeat(64) },
|
||||
])
|
||||
})
|
||||
|
||||
test('a module at a different version is re-resolved', async () => {
|
||||
place('uo', '0.2.0')
|
||||
const installer = fakeInstall()
|
||||
|
||||
const results = await declared.resolve({
|
||||
value: `uo@0.3.0=${URL_030}`,
|
||||
hosts: HOSTS,
|
||||
model: fakeModel(),
|
||||
installImpl: installer,
|
||||
})
|
||||
|
||||
assert.deepEqual(results.map((r) => r.action), ['installed'])
|
||||
assert.equal(installer.calls.length, 1)
|
||||
})
|
||||
|
||||
test('a directory with no readable module.json counts as absent', async () => {
|
||||
fs.mkdirSync(path.join(tmpRoot, 'uo'), { recursive: true })
|
||||
fs.writeFileSync(path.join(tmpRoot, 'uo', 'module.json'), '{ this is not json')
|
||||
const installer = fakeInstall()
|
||||
|
||||
await declared.resolve({
|
||||
value: `uo@0.3.0=${URL_030}`,
|
||||
hosts: HOSTS,
|
||||
model: fakeModel(),
|
||||
installImpl: installer,
|
||||
})
|
||||
|
||||
// Whatever is in that directory, it is not the declared module — so the
|
||||
// declared module is fetched rather than assumed to be there.
|
||||
assert.equal(installer.calls.length, 1)
|
||||
})
|
||||
|
||||
test('a failure is carried, not thrown, and does not stop the next module', async () => {
|
||||
const model = fakeModel()
|
||||
const installer = {
|
||||
calls: [],
|
||||
async install(args) {
|
||||
installer.calls.push(args)
|
||||
if (args.expect.id === 'uo') throw new Error('could not reach releases.example.com')
|
||||
return {
|
||||
id: args.expect.id,
|
||||
name: 'Market',
|
||||
version: args.expect.version,
|
||||
sha256: 'b'.repeat(64),
|
||||
source: args.url,
|
||||
bytes: 10,
|
||||
replaced: false,
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
const results = await declared.resolve({
|
||||
value: `uo@0.3.0=${URL_030} market@1.0.0=https://releases.example.com/market.json`,
|
||||
hosts: HOSTS,
|
||||
model,
|
||||
installImpl: installer,
|
||||
})
|
||||
|
||||
assert.deepEqual(results.map((r) => r.action), ['failed', 'installed'])
|
||||
assert.match(results[0].message, /could not reach/)
|
||||
// The second module still installed: one unreachable release host costs that
|
||||
// module, not the deployment.
|
||||
assert.equal(model.recorded.length, 1)
|
||||
assert.equal(model.recorded[0].id, 'market')
|
||||
})
|
||||
|
||||
test('a failed resolution leaves whatever was already on the volume', async () => {
|
||||
place('uo', '0.2.0')
|
||||
const installer = fakeInstall({ throws: 'the host is down' })
|
||||
|
||||
const results = await declared.resolve({
|
||||
value: `uo@0.3.0=${URL_030}`,
|
||||
hosts: HOSTS,
|
||||
model: fakeModel(),
|
||||
installImpl: installer,
|
||||
})
|
||||
|
||||
assert.equal(results[0].action, 'failed')
|
||||
// The site comes up serving the version it already had rather than not at all.
|
||||
assert.equal(declared.installedVersion('uo'), '0.2.0')
|
||||
})
|
||||
|
||||
test('resolution without a model records nothing and still installs', async () => {
|
||||
// `npm run seed` and the test harness both reach code paths with no model to
|
||||
// hand; the volume half must not depend on the database half.
|
||||
const installer = fakeInstall()
|
||||
const results = await declared.resolve({ value: `uo@0.3.0=${URL_030}`, hosts: HOSTS, installImpl: installer })
|
||||
assert.deepEqual(results.map((r) => r.action), ['installed'])
|
||||
})
|
||||
|
||||
test('state() reports the last resolution and is replaced by the next', async () => {
|
||||
const installer = fakeInstall()
|
||||
await declared.resolve({ value: `uo@0.3.0=${URL_030}`, hosts: HOSTS, installImpl: installer })
|
||||
assert.equal(declared.state().length, 1)
|
||||
assert.equal(declared.state()[0].id, 'uo')
|
||||
|
||||
await declared.resolve({ value: '', hosts: HOSTS, installImpl: installer })
|
||||
assert.deepEqual(declared.state(), [])
|
||||
})
|
||||
|
||||
// ── The whole path, once, for real ─────────────────────────────────────────
|
||||
|
||||
const BLOCK = 512
|
||||
|
||||
function octal(value, width) {
|
||||
return Number(value).toString(8).padStart(width - 1, '0') + '\0'
|
||||
}
|
||||
|
||||
function member({ name, type = '0', body = '' }) {
|
||||
const header = Buffer.alloc(BLOCK, 0)
|
||||
const data = Buffer.from(body, 'utf8')
|
||||
header.write(name, 0, 100, 'utf8')
|
||||
header.write(octal(0o644, 8), 100, 8, 'ascii')
|
||||
header.write(octal(0, 8), 108, 8, 'ascii')
|
||||
header.write(octal(0, 8), 116, 8, 'ascii')
|
||||
header.write(octal(data.length, 12), 124, 12, 'ascii')
|
||||
header.write(octal(0, 12), 136, 12, 'ascii')
|
||||
header.write(' ', 148, 8, 'ascii')
|
||||
header.write(type, 156, 1, 'ascii')
|
||||
header.write('ustar\0', 257, 6, 'ascii')
|
||||
header.write('00', 263, 2, 'ascii')
|
||||
let sum = 0
|
||||
for (const byte of header) sum += byte
|
||||
header.write(`${sum.toString(8).padStart(6, '0')}\0 `, 148, 8, 'ascii')
|
||||
const padding = Buffer.alloc((BLOCK - (data.length % BLOCK)) % BLOCK, 0)
|
||||
return Buffer.concat([header, data, padding])
|
||||
}
|
||||
|
||||
/** A bundle tarball, named the way module-uo's release workflow names one. */
|
||||
function bundle(version) {
|
||||
const root = `module-uo-${version}`
|
||||
return zlib.gzipSync(Buffer.concat([
|
||||
member({ name: `${root}/`, type: '5' }),
|
||||
member({
|
||||
name: `${root}/module.json`,
|
||||
body: JSON.stringify({ id: 'uo', name: 'Ultima Online', version, coreApi: '^1.0.0', server: 'server/index.js' }),
|
||||
}),
|
||||
member({ name: `${root}/server/`, type: '5' }),
|
||||
member({ name: `${root}/server/index.js`, body: 'module.exports = () => {}\n' }),
|
||||
Buffer.alloc(BLOCK * 2, 0),
|
||||
]))
|
||||
}
|
||||
|
||||
test('end to end: a declaration installs a real bundle through the real install path', async (t) => {
|
||||
const tarball = bundle('0.3.0')
|
||||
const artifactUrl = 'https://releases.example.com/mod/uo-0.3.0.tar.gz'
|
||||
const manifest = JSON.stringify({
|
||||
schema: 1,
|
||||
id: 'uo',
|
||||
name: 'Ultima Online',
|
||||
version: '0.3.0',
|
||||
coreApi: '^1.0.0',
|
||||
artifact: 'uo-0.3.0.tar.gz',
|
||||
url: artifactUrl,
|
||||
sha256: crypto.createHash('sha256').update(tarball).digest('hex'),
|
||||
size: tarball.length,
|
||||
})
|
||||
const routes = { [URL_030]: manifest, [artifactUrl]: tarball }
|
||||
const fetched = []
|
||||
const fetchImpl = async (url) => {
|
||||
fetched.push(String(url))
|
||||
const body = routes[String(url)]
|
||||
return body === undefined ? new Response('nope', { status: 404 }) : new Response(body, { status: 200 })
|
||||
}
|
||||
|
||||
// The real install.js, with only the transport replaced — same seam the
|
||||
// install tests use, and the same reason: what is under test is the decisions,
|
||||
// not Node's TLS.
|
||||
// eslint-disable-next-line global-require
|
||||
const install = require('../src/modules/install')
|
||||
const installImpl = { install: (args) => install.install({ ...args, fetchImpl }) }
|
||||
const model = fakeModel()
|
||||
|
||||
const first = await declared.resolve({
|
||||
value: `uo@0.3.0=${URL_030}`,
|
||||
hosts: HOSTS,
|
||||
model,
|
||||
installImpl,
|
||||
})
|
||||
assert.deepEqual(first.map((r) => r.action), ['installed'])
|
||||
// Unpacked, with the release's top-level directory stripped, under the id.
|
||||
assert.equal(
|
||||
JSON.parse(fs.readFileSync(path.join(tmpRoot, 'uo', 'module.json'), 'utf8')).version,
|
||||
'0.3.0',
|
||||
)
|
||||
assert.equal(model.recorded[0].sha256, JSON.parse(manifest).sha256)
|
||||
|
||||
// And again, which is what every restart after the first one is.
|
||||
const before = fetched.length
|
||||
const second = await declared.resolve({ value: `uo@0.3.0=${URL_030}`, hosts: HOSTS, model, installImpl })
|
||||
assert.deepEqual(second.map((r) => r.action), ['noop'])
|
||||
assert.equal(fetched.length, before, 'the second resolution must not fetch anything')
|
||||
|
||||
// A declaration whose URL resolves to a different version is refused BEFORE
|
||||
// the artifact is downloaded — the module on the volume is left alone.
|
||||
const wrong = await declared.resolve({ value: `uo@0.4.0=${URL_030}`, hosts: HOSTS, model, installImpl })
|
||||
assert.equal(wrong[0].action, 'failed')
|
||||
assert.match(wrong[0].message, /v0\.4\.0 was asked for/)
|
||||
assert.equal(declared.installedVersion('uo'), '0.3.0')
|
||||
t.diagnostic(`fetched ${fetched.length} URL(s) across three resolutions`)
|
||||
})
|
||||
448
server/test/moduleInstall.test.js
Normal file
448
server/test/moduleInstall.test.js
Normal file
@@ -0,0 +1,448 @@
|
||||
// modules/install.js — fetching, verifying and placing a module bundle.
|
||||
//
|
||||
// Phase 4, slice 1 of MODULE_SYSTEM.md §2.7.2. The rules under test are all
|
||||
// refusals, and each one is a step an attacker would otherwise walk through:
|
||||
// a non-https URL, a host that is not allowed, a REDIRECT to a host that is not
|
||||
// allowed, a body larger than declared, a hash that does not match, an archive
|
||||
// that does not agree with the manifest about what it is.
|
||||
//
|
||||
// `fetchImpl` is injected rather than a TLS server being stood up, for the same
|
||||
// reason `replayFragments` takes a `query`: what is being tested is what this
|
||||
// file decides about a response, and a real server would mostly test Node's
|
||||
// certificate handling. The responses below are real `Response` objects with
|
||||
// real bodies, so the streaming, the hashing and the byte cap are exercised for
|
||||
// real — only the transport is stubbed.
|
||||
process.env.DB_HOST = '127.0.0.1'
|
||||
process.env.DB_PORT = '59999'
|
||||
|
||||
const crypto = require('crypto')
|
||||
const fs = require('fs')
|
||||
const os = require('os')
|
||||
const path = require('path')
|
||||
const zlib = require('zlib')
|
||||
|
||||
const { test, beforeEach, after } = require('node:test')
|
||||
const assert = require('node:assert/strict')
|
||||
|
||||
const db = require('../src/utils/db')
|
||||
|
||||
after(() => db.close())
|
||||
|
||||
const HOSTS = ['releases.example.com']
|
||||
const MANIFEST_URL = 'https://releases.example.com/mod/uo-1.0.0.json'
|
||||
|
||||
let tmpRoot
|
||||
let install
|
||||
|
||||
/**
|
||||
* A fresh install.js bound to a fresh modules directory.
|
||||
*
|
||||
* install.js reads `loader.dir()`, which is resolved once at require time from
|
||||
* MODULES_DIR — so both have to be re-required per test, exactly as
|
||||
* moduleLifecycle.test.js does.
|
||||
*/
|
||||
function freshInstall(dir) {
|
||||
process.env.MODULES_DIR = dir
|
||||
delete require.cache[require.resolve('../src/modules/loader')]
|
||||
delete require.cache[require.resolve('../src/modules/install')]
|
||||
// eslint-disable-next-line global-require
|
||||
return require('../src/modules/install')
|
||||
}
|
||||
|
||||
// ── Building a bundle to serve ─────────────────────────────────────────────
|
||||
|
||||
const BLOCK = 512
|
||||
|
||||
function octal(value, width) {
|
||||
return Number(value).toString(8).padStart(width - 1, '0') + '\0'
|
||||
}
|
||||
|
||||
function member({ name, type = '0', body = '' }) {
|
||||
const header = Buffer.alloc(BLOCK, 0)
|
||||
const data = Buffer.from(body, 'utf8')
|
||||
header.write(name, 0, 100, 'utf8')
|
||||
header.write(octal(0o644, 8), 100, 8, 'ascii')
|
||||
header.write(octal(0, 8), 108, 8, 'ascii')
|
||||
header.write(octal(0, 8), 116, 8, 'ascii')
|
||||
header.write(octal(data.length, 12), 124, 12, 'ascii')
|
||||
header.write(octal(0, 12), 136, 12, 'ascii')
|
||||
header.write(' ', 148, 8, 'ascii')
|
||||
header.write(type, 156, 1, 'ascii')
|
||||
header.write('ustar\0', 257, 6, 'ascii')
|
||||
header.write('00', 263, 2, 'ascii')
|
||||
let sum = 0
|
||||
for (const byte of header) sum += byte
|
||||
header.write(`${sum.toString(8).padStart(6, '0')}\0 `, 148, 8, 'ascii')
|
||||
const padding = Buffer.alloc((BLOCK - (data.length % BLOCK)) % BLOCK, 0)
|
||||
return Buffer.concat([header, data, padding])
|
||||
}
|
||||
|
||||
/** A bundle tarball whose root directory is named the way a release names it. */
|
||||
function bundle({ id = 'uo', version = '1.0.0', extra = [] } = {}) {
|
||||
const root = `module-${id}-${version}`
|
||||
return zlib.gzipSync(Buffer.concat([
|
||||
member({ name: `${root}/`, type: '5' }),
|
||||
member({
|
||||
name: `${root}/module.json`,
|
||||
body: JSON.stringify({ id, name: 'Ultima Online', version, coreApi: '^1.0.0', server: 'server/index.js' }),
|
||||
}),
|
||||
member({ name: `${root}/server/`, type: '5' }),
|
||||
member({ name: `${root}/server/index.js`, body: 'module.exports = () => {}\n' }),
|
||||
...extra.map(member),
|
||||
Buffer.alloc(BLOCK * 2, 0),
|
||||
]))
|
||||
}
|
||||
|
||||
const sha256 = (buf) => crypto.createHash('sha256').update(buf).digest('hex')
|
||||
|
||||
/**
|
||||
* A fake transport serving one manifest and one artifact.
|
||||
*
|
||||
* `routes` maps an absolute URL to either a Buffer/string body or
|
||||
* `{ status, location }` for a redirect, so a test can describe exactly what the
|
||||
* remote host does without describing how it does it.
|
||||
*/
|
||||
function fakeFetch(routes) {
|
||||
const seen = []
|
||||
const impl = async (url) => {
|
||||
const href = String(url)
|
||||
seen.push(href)
|
||||
const route = routes[href]
|
||||
if (route === undefined) return new Response('not found', { status: 404 })
|
||||
if (route && route.status) {
|
||||
return new Response(null, {
|
||||
status: route.status,
|
||||
headers: route.location ? { location: route.location } : {},
|
||||
})
|
||||
}
|
||||
return new Response(route, { status: 200 })
|
||||
}
|
||||
impl.seen = seen
|
||||
return impl
|
||||
}
|
||||
|
||||
/** The manifest a release publishes, with whatever a test wants to change. */
|
||||
function manifestFor(tarball, overrides = {}) {
|
||||
return JSON.stringify({
|
||||
schema: 1,
|
||||
id: 'uo',
|
||||
name: 'Ultima Online',
|
||||
version: '1.0.0',
|
||||
coreApi: '^1.0.0',
|
||||
artifact: 'uo-1.0.0.tar.gz',
|
||||
url: 'https://releases.example.com/mod/uo-1.0.0.tar.gz',
|
||||
sha256: sha256(tarball),
|
||||
size: tarball.length,
|
||||
...overrides,
|
||||
})
|
||||
}
|
||||
|
||||
/** The happy-path pair: a valid manifest and the artifact it describes. */
|
||||
function goodRoutes(options = {}) {
|
||||
const tarball = bundle(options)
|
||||
return {
|
||||
tarball,
|
||||
routes: {
|
||||
[MANIFEST_URL]: manifestFor(tarball, options.manifest || {}),
|
||||
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
tmpRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'rg-install-'))
|
||||
install = freshInstall(tmpRoot)
|
||||
})
|
||||
|
||||
// ── The allowlist ──────────────────────────────────────────────────────────
|
||||
|
||||
test('parseHosts accepts comma and whitespace separation, and lower-cases', () => {
|
||||
assert.deepEqual(install.parseHosts('a.com, B.com\nc.com'), ['a.com', 'b.com', 'c.com'])
|
||||
assert.deepEqual(install.parseHosts(''), [])
|
||||
assert.deepEqual(install.parseHosts(null), [])
|
||||
})
|
||||
|
||||
test('an empty allowlist forbids everything rather than allowing everything', () => {
|
||||
// The direction matters: a setting someone blanks by accident must stop
|
||||
// installs, not open the door to every host on the internet.
|
||||
assert.throws(() => install.checkUrl('https://releases.example.com/x.json', []), /no module source hosts are allowed/)
|
||||
})
|
||||
|
||||
test('only https may be installed from', () => {
|
||||
// The sha256 is no help against a plaintext fetch: whoever can rewrite the
|
||||
// artifact in flight can rewrite the manifest that declares its hash.
|
||||
assert.throws(() => install.checkUrl('http://releases.example.com/x.json', HOSTS), /only https/)
|
||||
assert.throws(() => install.checkUrl('file:///etc/passwd', HOSTS), /only https/)
|
||||
})
|
||||
|
||||
test('a host that is not on the allowlist is refused, and the message says which are', () => {
|
||||
assert.throws(
|
||||
() => install.checkUrl('https://evil.example.net/x.json', HOSTS),
|
||||
/"evil.example.net" is not an allowed module source host \(allowed: releases.example.com\)/,
|
||||
)
|
||||
})
|
||||
|
||||
test('the allowlist is matched on the host, not on a substring of the URL', () => {
|
||||
// `https://evil.com/?x=releases.example.com` must not pass, and neither must a
|
||||
// subdomain nobody listed.
|
||||
assert.throws(() => install.checkUrl('https://evil.com/?releases.example.com', HOSTS), /not an allowed/)
|
||||
assert.throws(() => install.checkUrl('https://sub.releases.example.com/x', HOSTS), /not an allowed/)
|
||||
assert.throws(() => install.checkUrl('https://releases.example.com.evil.net/x', HOSTS), /not an allowed/)
|
||||
})
|
||||
|
||||
// ── Redirects ──────────────────────────────────────────────────────────────
|
||||
|
||||
test('a redirect is followed, and re-checked against the allowlist', async () => {
|
||||
const { tarball } = goodRoutes()
|
||||
const impl = fakeFetch({
|
||||
[MANIFEST_URL]: { status: 302, location: 'https://releases.example.com/cdn/uo-1.0.0.json' },
|
||||
'https://releases.example.com/cdn/uo-1.0.0.json': manifestFor(tarball),
|
||||
})
|
||||
|
||||
const manifest = await install.fetchManifest(MANIFEST_URL, HOSTS, impl)
|
||||
|
||||
assert.equal(manifest.id, 'uo')
|
||||
assert.deepEqual(impl.seen, [MANIFEST_URL, 'https://releases.example.com/cdn/uo-1.0.0.json'])
|
||||
})
|
||||
|
||||
test('a redirect to a host that is not allowed is refused', async () => {
|
||||
// The hole the allowlist exists to close. `fetch`'s own redirect following
|
||||
// would check the first hop and then go wherever it was pointed — which is
|
||||
// why get() follows them by hand.
|
||||
const impl = fakeFetch({
|
||||
[MANIFEST_URL]: { status: 302, location: 'http://169.254.169.254/latest/meta-data/' },
|
||||
})
|
||||
|
||||
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /only https/)
|
||||
})
|
||||
|
||||
test('a redirect loop is bounded rather than followed forever', async () => {
|
||||
const impl = fakeFetch({
|
||||
[MANIFEST_URL]: { status: 302, location: MANIFEST_URL },
|
||||
})
|
||||
|
||||
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /too many redirects/)
|
||||
})
|
||||
|
||||
// ── The install manifest ───────────────────────────────────────────────────
|
||||
|
||||
test('a manifest that is not JSON is refused with a readable reason', async () => {
|
||||
const impl = fakeFetch({ [MANIFEST_URL]: 'this is not json' })
|
||||
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /not valid JSON/)
|
||||
})
|
||||
|
||||
test('a manifest with an invalid module id is refused', async () => {
|
||||
// The loader would refuse to scan a directory called `../etc`, but the point
|
||||
// is that one is never created: the id becomes a path segment.
|
||||
for (const id of ['../etc', 'UO', '', 'x', 'a'.repeat(40)]) {
|
||||
const impl = fakeFetch({ [MANIFEST_URL]: JSON.stringify({ id, name: 'n', version: '1', sha256: 'a'.repeat(64) }) })
|
||||
// eslint-disable-next-line no-await-in-loop
|
||||
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /invalid module id/, `id ${JSON.stringify(id)}`)
|
||||
}
|
||||
})
|
||||
|
||||
test('a manifest with no usable sha256 is refused', async () => {
|
||||
const impl = fakeFetch({
|
||||
[MANIFEST_URL]: JSON.stringify({ id: 'uo', name: 'n', version: '1.0.0', sha256: 'not-a-hash' }),
|
||||
})
|
||||
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /no valid sha256/)
|
||||
})
|
||||
|
||||
test('the artifact URL is resolved relative to the manifest when it carries no absolute one', async () => {
|
||||
const { tarball } = goodRoutes()
|
||||
const impl = fakeFetch({ [MANIFEST_URL]: manifestFor(tarball, { url: undefined }) })
|
||||
|
||||
const manifest = await install.fetchManifest(MANIFEST_URL, HOSTS, impl)
|
||||
|
||||
// Release assets sit beside their manifest, so a manifest that travelled
|
||||
// without its absolute URL still resolves to the right place.
|
||||
assert.equal(manifest.artifactUrl, 'https://releases.example.com/mod/uo-1.0.0.tar.gz')
|
||||
})
|
||||
|
||||
// ── The install ────────────────────────────────────────────────────────────
|
||||
|
||||
test('a good bundle installs into modules/<id>, stripped of its wrapper', async () => {
|
||||
const { routes, tarball } = goodRoutes()
|
||||
|
||||
const result = await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) })
|
||||
|
||||
assert.equal(result.id, 'uo')
|
||||
assert.equal(result.version, '1.0.0')
|
||||
assert.equal(result.sha256, sha256(tarball))
|
||||
assert.equal(result.source, MANIFEST_URL)
|
||||
assert.equal(result.replaced, false)
|
||||
|
||||
// The directory is named for the module id, not for the archive's root — the
|
||||
// loader scans for the former and the publisher chose the latter.
|
||||
const dir = path.join(tmpRoot, 'uo')
|
||||
assert.ok(fs.existsSync(path.join(dir, 'module.json')))
|
||||
assert.ok(fs.existsSync(path.join(dir, 'server', 'index.js')))
|
||||
assert.ok(!fs.existsSync(path.join(tmpRoot, 'module-uo-1.0.0')))
|
||||
})
|
||||
|
||||
test('a hash that does not match refuses the install and writes nothing', async () => {
|
||||
const { routes } = goodRoutes()
|
||||
// The bytes are fine; the manifest lies about them. Which is the same thing an
|
||||
// artifact swapped after publication looks like.
|
||||
routes[MANIFEST_URL] = manifestFor(Buffer.from('different'), {})
|
||||
|
||||
await assert.rejects(
|
||||
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
|
||||
/does not match the sha256/,
|
||||
)
|
||||
assert.deepEqual(fs.readdirSync(tmpRoot), [], 'nothing may be left on the volume')
|
||||
})
|
||||
|
||||
test('a bundle whose module.json disagrees with the manifest is refused', async () => {
|
||||
// A manifest promising `uo` and delivering something else would otherwise be
|
||||
// installed into `modules/uo/` under a name it is not.
|
||||
const tarball = bundle({ id: 'rust', version: '1.0.0' })
|
||||
const routes = {
|
||||
[MANIFEST_URL]: manifestFor(tarball),
|
||||
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
|
||||
}
|
||||
|
||||
await assert.rejects(
|
||||
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
|
||||
/declares module id "rust" but the install manifest promised "uo"/,
|
||||
)
|
||||
assert.deepEqual(fs.readdirSync(tmpRoot), [])
|
||||
})
|
||||
|
||||
test('a bundle whose version disagrees with the manifest is refused', async () => {
|
||||
const tarball = bundle({ id: 'uo', version: '9.9.9' })
|
||||
const routes = {
|
||||
[MANIFEST_URL]: manifestFor(tarball),
|
||||
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
|
||||
}
|
||||
|
||||
await assert.rejects(
|
||||
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
|
||||
/declares version "9.9.9"/,
|
||||
)
|
||||
})
|
||||
|
||||
test('an artifact whose length differs from the declared size is refused', async () => {
|
||||
const tarball = bundle()
|
||||
const routes = {
|
||||
[MANIFEST_URL]: manifestFor(tarball, { size: tarball.length + 10 }),
|
||||
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
|
||||
}
|
||||
|
||||
await assert.rejects(
|
||||
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
|
||||
/but the manifest declares/,
|
||||
)
|
||||
})
|
||||
|
||||
test('a hostile archive is refused, and nothing of it reaches the volume', async () => {
|
||||
// The case node-tar alone does NOT cover: it throws on the escaping member,
|
||||
// but only once it reaches it — the members before it are already on disk.
|
||||
// archive.inspect() decides before extract() runs, and the unpack happens in a
|
||||
// scratch directory that is removed either way.
|
||||
const tarball = bundle({
|
||||
extra: [
|
||||
{ name: 'module-uo-1.0.0/../../ESCAPED.txt', body: 'escaped' },
|
||||
],
|
||||
})
|
||||
const routes = {
|
||||
[MANIFEST_URL]: manifestFor(tarball),
|
||||
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
|
||||
}
|
||||
|
||||
await assert.rejects(
|
||||
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
|
||||
/escapes upward/,
|
||||
)
|
||||
assert.deepEqual(fs.readdirSync(tmpRoot), [], 'no scratch directory, no partial module, no escape')
|
||||
})
|
||||
|
||||
test('an upgrade replaces the directory and reports that it did', async () => {
|
||||
const first = goodRoutes()
|
||||
await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(first.routes) })
|
||||
fs.writeFileSync(path.join(tmpRoot, 'uo', 'STALE.txt'), 'from the old version')
|
||||
|
||||
const second = goodRoutes({ version: '2.0.0', manifest: { version: '2.0.0' } })
|
||||
second.routes[MANIFEST_URL] = manifestFor(second.tarball, { version: '2.0.0' })
|
||||
const result = await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(second.routes) })
|
||||
|
||||
assert.equal(result.replaced, true)
|
||||
assert.equal(result.version, '2.0.0')
|
||||
// A replace, not a merge: a file the previous version left behind must not
|
||||
// survive into the new one, or an upgrade quietly keeps dead code loadable.
|
||||
assert.ok(!fs.existsSync(path.join(tmpRoot, 'uo', 'STALE.txt')))
|
||||
assert.deepEqual(fs.readdirSync(tmpRoot), ['uo'], 'the aside copy is cleaned up')
|
||||
})
|
||||
|
||||
test('a failed upgrade leaves the previous version in place', async () => {
|
||||
const first = goodRoutes()
|
||||
await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(first.routes) })
|
||||
|
||||
const badTarball = bundle({ id: 'uo', version: '2.0.0', extra: [{ name: 'evil/', type: '5' }] })
|
||||
await assert.rejects(() => install.install({
|
||||
url: MANIFEST_URL,
|
||||
hosts: HOSTS,
|
||||
fetchImpl: fakeFetch({
|
||||
[MANIFEST_URL]: manifestFor(badTarball, { version: '2.0.0' }),
|
||||
'https://releases.example.com/mod/uo-1.0.0.tar.gz': badTarball,
|
||||
}),
|
||||
}))
|
||||
|
||||
// The installed module is untouched: the swap is the last step, so a bundle
|
||||
// rejected before it never got near the live directory.
|
||||
assert.ok(fs.existsSync(path.join(tmpRoot, 'uo', 'module.json')))
|
||||
assert.equal(JSON.parse(fs.readFileSync(path.join(tmpRoot, 'uo', 'module.json'), 'utf8')).version, '1.0.0')
|
||||
assert.deepEqual(fs.readdirSync(tmpRoot), ['uo'])
|
||||
})
|
||||
|
||||
// ── The volume ─────────────────────────────────────────────────────────────
|
||||
|
||||
test('moduleDir refuses an id that is not one', () => {
|
||||
for (const id of ['../escape', 'a/b', '', 'UO', '.']) {
|
||||
assert.throws(() => install.moduleDir(id), /invalid module id/, `id ${JSON.stringify(id)}`)
|
||||
}
|
||||
assert.equal(install.moduleDir('uo'), path.join(tmpRoot, 'uo'))
|
||||
})
|
||||
|
||||
test('isInstalled and removeDir report what they did', async () => {
|
||||
const { routes } = goodRoutes()
|
||||
assert.equal(install.isInstalled('uo'), false)
|
||||
|
||||
await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) })
|
||||
assert.equal(install.isInstalled('uo'), true)
|
||||
|
||||
assert.equal(await install.removeDir('uo'), true)
|
||||
assert.equal(install.isInstalled('uo'), false)
|
||||
// Removing what is not there is not an error — an uninstall of a module whose
|
||||
// directory was already deleted by hand should still tidy up the row.
|
||||
assert.equal(await install.removeDir('uo'), false)
|
||||
})
|
||||
|
||||
test('purgeFile resolves the manifest declaration, and refuses one that escapes', async () => {
|
||||
const dir = path.join(tmpRoot, 'uo')
|
||||
fs.mkdirSync(path.join(dir, 'server', 'db'), { recursive: true })
|
||||
fs.writeFileSync(path.join(dir, 'server', 'db', 'purge.sql'), 'DROP TABLE IF EXISTS x;')
|
||||
|
||||
const write = (purge) => fs.writeFileSync(
|
||||
path.join(dir, 'module.json'),
|
||||
JSON.stringify({ id: 'uo', name: 'UO', version: '1.0.0', purge }),
|
||||
)
|
||||
|
||||
write('server/db/purge.sql')
|
||||
assert.equal(install.purgeFile('uo'), path.join(dir, 'server', 'db', 'purge.sql'))
|
||||
|
||||
// The same containment rule the loader applies to client.entry: a manifest may
|
||||
// not point core at a file outside the module it belongs to.
|
||||
write('../../../../etc/passwd')
|
||||
assert.equal(install.purgeFile('uo'), null)
|
||||
|
||||
// Declared but absent, and not declared at all, are both "nothing to run".
|
||||
write('server/db/missing.sql')
|
||||
assert.equal(install.purgeFile('uo'), null)
|
||||
write(undefined)
|
||||
assert.equal(install.purgeFile('uo'), null)
|
||||
})
|
||||
|
||||
test('purgeFile is null for a module that is not on the volume', () => {
|
||||
assert.equal(install.purgeFile('nothere'), null)
|
||||
})
|
||||
@@ -125,6 +125,11 @@ function fakeModel(seed = []) {
|
||||
if (!row || row.state === 'disabled') return
|
||||
Object.assign(row, { state: 'startup_failed', failureStage: stage, failureReason: reason })
|
||||
},
|
||||
async disable(id) {
|
||||
calls.push(`disable:${id}`)
|
||||
const row = rows.get(id)
|
||||
if (row) Object.assign(row, { state: 'disabled', failureStage: null, failureReason: null })
|
||||
},
|
||||
}
|
||||
return model
|
||||
}
|
||||
@@ -374,3 +379,151 @@ test('a hook that throws does not stop the ones behind it', async () => {
|
||||
await assert.doesNotReject(() => lifecycle.shutdown({ modules: loader }))
|
||||
assert.deepEqual(noted(file), ['shutdown:zzz', 'shutdown:aaa'])
|
||||
})
|
||||
|
||||
// ── stop(): the admin panel's Disable ──────────────────────────────────────
|
||||
//
|
||||
// Phase 4, §2.7.2 decision 3. Phase 2's disable moved a record and left the
|
||||
// module running; the whole point of these tests is that it no longer does.
|
||||
|
||||
test('stop runs that one module\'s onShutdown and leaves the others alone', async () => {
|
||||
const file = path.join(tmpRoot, 'log.txt')
|
||||
writeModule('aaa', { boot: '', shutdown: '', log: file })
|
||||
writeModule('bbb', { boot: '', shutdown: '', log: file })
|
||||
const loader = freshLoader(tmpRoot)
|
||||
const model = fakeModel()
|
||||
|
||||
await lifecycle.boot({ modules: loader, model })
|
||||
fs.writeFileSync(file, '')
|
||||
|
||||
const result = await lifecycle.stop('aaa', { modules: loader, model })
|
||||
|
||||
assert.deepEqual(result, { stopped: true, error: null })
|
||||
// Only aaa. This is the difference from shutdown(), which runs everything.
|
||||
assert.deepEqual(noted(file), ['shutdown:aaa'])
|
||||
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
|
||||
assert.equal(stateOf(loader, 'bbb').state, 'started', 'the other module keeps running')
|
||||
assert.equal(model.rows.get('aaa').state, 'disabled')
|
||||
})
|
||||
|
||||
test('stop moves the record only after the hook has run', async () => {
|
||||
// While onShutdown runs, the module is still `started` — the only state in
|
||||
// which its routes and the world it is tearing down agree with each other. The
|
||||
// module reports its own view of itself, so a record moved too early shows up
|
||||
// here as `disabled` instead of `started`.
|
||||
const file = path.join(tmpRoot, 'log.txt')
|
||||
const dir = path.join(tmpRoot, 'aaa')
|
||||
fs.mkdirSync(dir, { recursive: true })
|
||||
fs.writeFileSync(path.join(dir, 'module.json'), JSON.stringify({
|
||||
id: 'aaa', name: 'A', version: '1.0.0', coreApi: '^1.0.0', server: 'index.js',
|
||||
}))
|
||||
fs.writeFileSync(path.join(dir, 'index.js'), `
|
||||
module.exports = (ctx, api) => {
|
||||
api.onBoot(async () => {})
|
||||
api.onShutdown(async () => {
|
||||
const loader = require(${JSON.stringify(require.resolve('../src/modules/loader'))})
|
||||
const me = loader.list().find((m) => m.id === 'aaa')
|
||||
require('fs').appendFileSync(${JSON.stringify(file)}, 'state-during-hook:' + me.state + ${JSON.stringify('\n')})
|
||||
})
|
||||
}`)
|
||||
|
||||
const loader = freshLoader(tmpRoot)
|
||||
await lifecycle.boot({ modules: loader, model: fakeModel() })
|
||||
await lifecycle.stop('aaa', { modules: loader, model: fakeModel([{ id: 'aaa', state: 'started' }]) })
|
||||
|
||||
assert.deepEqual(noted(file), ['state-during-hook:started'])
|
||||
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
|
||||
})
|
||||
|
||||
test('a hook that throws does not prevent the disable', async () => {
|
||||
// The opposite of the boot path's rule, deliberately. There, a failure means
|
||||
// the module never became safe to use; here, the operator has asked for it to
|
||||
// stop answering and a module that could not close cleanly is a reason to log
|
||||
// loudly, not a reason to leave it serving.
|
||||
writeModule('aaa', { boot: '', shutdown: 'throw new Error("socket stuck")' })
|
||||
const loader = freshLoader(tmpRoot)
|
||||
const model = fakeModel()
|
||||
|
||||
await lifecycle.boot({ modules: loader, model })
|
||||
const result = await lifecycle.stop('aaa', { modules: loader, model })
|
||||
|
||||
assert.equal(result.stopped, false)
|
||||
assert.match(result.error, /socket stuck/)
|
||||
assert.equal(stateOf(loader, 'aaa').state, 'disabled', 'disabled anyway')
|
||||
assert.equal(model.rows.get('aaa').state, 'disabled')
|
||||
})
|
||||
|
||||
test('a hook that hangs costs its budget, and the module is still disabled', async () => {
|
||||
writeModule('aaa', { boot: '', shutdown: 'await new Promise(() => {})' })
|
||||
const loader = freshLoader(tmpRoot)
|
||||
const model = fakeModel()
|
||||
|
||||
await lifecycle.boot({ modules: loader, model })
|
||||
const started = Date.now()
|
||||
const result = await lifecycle.stop('aaa', { modules: loader, model, budgetMs: 50 })
|
||||
|
||||
assert.equal(result.stopped, false)
|
||||
assert.match(result.error, /budget/)
|
||||
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
|
||||
assert.ok(Date.now() - started < 2000)
|
||||
})
|
||||
|
||||
test('stopping a module with no onShutdown still disables it', async () => {
|
||||
// A hookless module has nothing to run and must still stop answering, or the
|
||||
// guard and the row disagree about what is serving.
|
||||
writeModule('aaa', { boot: '' })
|
||||
const loader = freshLoader(tmpRoot)
|
||||
const model = fakeModel()
|
||||
|
||||
await lifecycle.boot({ modules: loader, model })
|
||||
const result = await lifecycle.stop('aaa', { modules: loader, model })
|
||||
|
||||
assert.deepEqual(result, { stopped: false, error: null })
|
||||
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
|
||||
assert.equal(model.rows.get('aaa').state, 'disabled')
|
||||
})
|
||||
|
||||
test('stopping a module whose onBoot failed does not run its onShutdown', async () => {
|
||||
// Same rule shutdownHooks() applies: a module that never finished warming up
|
||||
// has a half-built world its onShutdown was not written for. It is still
|
||||
// disabled — it just is not asked to tear anything down.
|
||||
const file = path.join(tmpRoot, 'log.txt')
|
||||
writeModule('aaa', { boot: 'throw new Error("no")', shutdown: '', log: file })
|
||||
const loader = freshLoader(tmpRoot)
|
||||
const model = fakeModel()
|
||||
|
||||
await lifecycle.boot({ modules: loader, model })
|
||||
const before = noted(file)
|
||||
const result = await lifecycle.stop('aaa', { modules: loader, model })
|
||||
|
||||
assert.equal(result.stopped, false)
|
||||
// Compared against what was already there rather than against an empty file:
|
||||
// `noted` splits, so a blanked file reads as [''] and an emptiness assertion
|
||||
// would pass for the wrong reason.
|
||||
assert.deepEqual(noted(file), before, 'no new hook output')
|
||||
assert.ok(!noted(file).includes('shutdown:aaa'))
|
||||
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
|
||||
})
|
||||
|
||||
test('stopping an unknown id writes the row and does not throw', async () => {
|
||||
// A row can exist for a module that is not on the volume, and disabling it is
|
||||
// exactly what an operator would do about that.
|
||||
writeModule('aaa', { boot: '' })
|
||||
const loader = freshLoader(tmpRoot)
|
||||
const model = fakeModel([{ id: 'ghost', state: 'startup_failed' }])
|
||||
|
||||
await assert.doesNotReject(() => lifecycle.stop('ghost', { modules: loader, model }))
|
||||
assert.equal(model.rows.get('ghost').state, 'disabled')
|
||||
})
|
||||
|
||||
test('a row that will not write does not stop the module from being disabled', async () => {
|
||||
// The same rule the boot path follows: bookkeeping failure is not the
|
||||
// operation failing. The guard is what stops traffic, and it has already moved.
|
||||
writeModule('aaa', { boot: '', shutdown: '' })
|
||||
const loader = freshLoader(tmpRoot)
|
||||
const model = fakeModel()
|
||||
await lifecycle.boot({ modules: loader, model })
|
||||
model.disable = async () => { throw new Error('database is gone') }
|
||||
|
||||
await assert.doesNotReject(() => lifecycle.stop('aaa', { modules: loader, model }))
|
||||
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
|
||||
})
|
||||
|
||||
@@ -25,7 +25,7 @@ const assert = require('node:assert/strict')
|
||||
const express = require('express')
|
||||
|
||||
const db = require('../src/utils/db')
|
||||
const { replayFragments } = require('../src/modules/schema')
|
||||
const { replayFragments, runPurge } = require('../src/modules/schema')
|
||||
const { splitStatements } = require('../src/utils/sqlStatements')
|
||||
const { startApp } = require('./_helper')
|
||||
|
||||
@@ -245,3 +245,66 @@ test('replay is skipped, not thrown, when no scan happened in this process', asy
|
||||
assert.deepEqual(rec.ran, [])
|
||||
assert.equal(loader.isLoaded(), false)
|
||||
})
|
||||
|
||||
// ── runPurge: the destructive twin ─────────────────────────────────────────
|
||||
//
|
||||
// Phase 4, slice 1. Same splitter, same pool, same serial execution as the
|
||||
// replay above — pointed the other way. The two differences are deliberate and
|
||||
// are what these tests are for.
|
||||
|
||||
test('purge runs every statement in the file, in order', async () => {
|
||||
const file = path.join(tmpRoot, 'purge.sql')
|
||||
fs.writeFileSync(file, [
|
||||
'-- drop everything this module owns',
|
||||
'DROP TABLE IF EXISTS mod_b;',
|
||||
'DROP TABLE IF EXISTS mod_a;',
|
||||
"DELETE FROM settings WHERE `key` = 'mod_thing';",
|
||||
].join('\n'))
|
||||
const rec = recorder()
|
||||
|
||||
const ran = await runPurge(file, { query: rec.query })
|
||||
|
||||
assert.equal(ran, 3)
|
||||
assert.match(rec.ran[0], /DROP TABLE IF EXISTS mod_b/)
|
||||
assert.match(rec.ran[2], /DELETE FROM settings/)
|
||||
})
|
||||
|
||||
test('purge is not held to the fragment allowlist — DROP is the point of it', async () => {
|
||||
// §2.6's leading-verb allowlist exists because a fragment replays on every
|
||||
// boot. purge.sql never does, which is exactly why it is the one file a module
|
||||
// may put a DROP in.
|
||||
const file = path.join(tmpRoot, 'purge.sql')
|
||||
fs.writeFileSync(file, 'DROP TABLE IF EXISTS mod_a;\nTRUNCATE TABLE mod_b;')
|
||||
const rec = recorder()
|
||||
|
||||
assert.equal(await runPurge(file, { query: rec.query }), 2)
|
||||
})
|
||||
|
||||
test('purge THROWS on failure, unlike the replay', async () => {
|
||||
// A replay failure is one module failing to start, which the site survives by
|
||||
// 503ing it. A purge failure is an operator's explicit destructive request not
|
||||
// having happened — reporting success would leave them believing data is gone
|
||||
// when it is not.
|
||||
const file = path.join(tmpRoot, 'purge.sql')
|
||||
fs.writeFileSync(file, 'DROP TABLE IF EXISTS mod_a;\nDROP TABLE mod_missing;')
|
||||
const query = async (sql) => {
|
||||
if (sql.includes('mod_missing')) throw new Error("Unknown table 'mod_missing'")
|
||||
}
|
||||
|
||||
await assert.rejects(
|
||||
() => runPurge(file, { query }),
|
||||
// The message names WHICH statement stopped it, because a purge is not
|
||||
// transactional — the ones before it have already committed and the operator
|
||||
// needs to know where it got to.
|
||||
/purge failed at statement 2 of 2: Unknown table 'mod_missing'/,
|
||||
)
|
||||
})
|
||||
|
||||
test('an empty purge file runs nothing and does not throw', async () => {
|
||||
const file = path.join(tmpRoot, 'purge.sql')
|
||||
fs.writeFileSync(file, '-- nothing to drop yet\n')
|
||||
const rec = recorder()
|
||||
|
||||
assert.equal(await runPurge(file, { query: rec.query }), 0)
|
||||
assert.deepEqual(rec.ran, [])
|
||||
})
|
||||
|
||||
@@ -38,7 +38,17 @@ modulesDb.getOne = async (id) => (rows.has(id) ? [rows.get(id)] : [])
|
||||
modulesDb.upsert = async ({ id, name, version, source, sha256 }) => {
|
||||
const existing = rows.get(id)
|
||||
if (existing) {
|
||||
Object.assign(existing, { name, version, source: source ?? null, sha256: sha256 ?? null })
|
||||
// COALESCE, matching the real statement: a value overwrites, a NULL leaves
|
||||
// what is there. This fake used to assign unconditionally — faithfully
|
||||
// reproducing the defect it was supposed to be able to catch, which is why
|
||||
// the boot refresh nulling an install's provenance survived until a browser
|
||||
// showed it.
|
||||
Object.assign(existing, {
|
||||
name,
|
||||
version,
|
||||
source: source ?? existing.source ?? null,
|
||||
sha256: sha256 ?? existing.sha256 ?? null,
|
||||
})
|
||||
return { affectedRows: 1 }
|
||||
}
|
||||
rows.set(id, {
|
||||
@@ -116,6 +126,46 @@ test('a hand-placed module records with no provenance', async () => {
|
||||
assert.equal(mod.sha256, null)
|
||||
})
|
||||
|
||||
test('the boot refresh does not wipe an install\'s provenance', async () => {
|
||||
// The defect a browser found in Phase 4, and the reason the upsert COALESCEs.
|
||||
//
|
||||
// lifecycle.boot() re-records every scanned module with NO source and NO
|
||||
// sha256, because a hand-placed directory genuinely has neither. That refresh
|
||||
// used to write both columns as NULL, so an admin-panel install's provenance
|
||||
// survived exactly until the restart the install asked for — and the screen
|
||||
// then described a module installed from a URL as "placed on the volume by
|
||||
// hand". Nothing could see it until Phase 4: no caller had ever passed a
|
||||
// non-null value.
|
||||
await modules.recordInstalled({
|
||||
id: 'uo',
|
||||
name: 'Ultima Online',
|
||||
version: '1.0.0',
|
||||
source: 'https://gitea.example/x/uo-1.0.0.json',
|
||||
sha256: 'b'.repeat(64),
|
||||
})
|
||||
|
||||
// What the next boot does.
|
||||
const after = await modules.recordInstalled({ id: 'uo', name: 'Ultima Online', version: '1.0.0' })
|
||||
|
||||
assert.equal(after.source, 'https://gitea.example/x/uo-1.0.0.json')
|
||||
assert.equal(after.sha256, 'b'.repeat(64))
|
||||
})
|
||||
|
||||
test('a re-install from a new URL does replace the provenance', async () => {
|
||||
// The other half: COALESCE must not make the columns write-once, or an upgrade
|
||||
// would for ever show where the FIRST version came from.
|
||||
await modules.recordInstalled({
|
||||
id: 'uo', name: 'Ultima Online', version: '1.0.0', source: 'https://old/x.json', sha256: 'c'.repeat(64),
|
||||
})
|
||||
const after = await modules.recordInstalled({
|
||||
id: 'uo', name: 'Ultima Online', version: '2.0.0', source: 'https://new/y.json', sha256: 'd'.repeat(64),
|
||||
})
|
||||
|
||||
assert.equal(after.source, 'https://new/y.json')
|
||||
assert.equal(after.sha256, 'd'.repeat(64))
|
||||
assert.equal(after.version, '2.0.0')
|
||||
})
|
||||
|
||||
test('recordInstalled refuses a manifest missing id, name or version', async () => {
|
||||
await assert.rejects(
|
||||
() => modules.recordInstalled({ id: 'uo', version: '1.0.0' }),
|
||||
|
||||
Reference in New Issue
Block a user