RunicGateway/website
3
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Activity

Compare commits

base: RunicGateway:a99ead4ee4df3c0e1754c6e5300a84799c807f84
Branches Tags
RunicGateway:main RunicGateway:edge RunicGateway:feat/asset-bridge-p2 RunicGateway:feature/engagement-uo-triggers RunicGateway:feature/module-docker-path RunicGateway:feature/module-install-service RunicGateway:spike/module-atlas RunicGateway:ci/gitea-actions-image-build RunicGateway:feature/moderation-dashboard
...
compare: RunicGateway:feature/module-docker-path
Branches Tags
RunicGateway:main RunicGateway:edge RunicGateway:feat/asset-bridge-p2 RunicGateway:feature/engagement-uo-triggers RunicGateway:feature/module-docker-path RunicGateway:feature/module-install-service RunicGateway:spike/module-atlas RunicGateway:ci/gitea-actions-image-build RunicGateway:feature/moderation-dashboard

11 Commits
a99ead4ee4 ... feature/mo

Author SHA1 Message Date
wtclaude
92c6972344 test(login): stop the backoff-guard test racing its own one-second lock 
A single recordFailure() locks for BASE_MS * 2 ** 0 — exactly one second — and
the test then does a real HTTP round trip against it. On CI that round trip took
1,456 ms and the guard correctly answered 200, failing the run for a reason that
has nothing to do with what the test is about.

Five failures lock for sixteen seconds. The subject is the guard's answer while
locked out, which is unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-12 08:04:56 -05:00
wtclaude
9b16f39a52 feat(modules): the declarative Docker path (phase 4, slice 3)
Some checks failed
PR Checks / bot-install (pull_request) Successful in 23s
Details
PR Checks / client-build (pull_request) Successful in 30s
Details
PR Checks / server-tests (pull_request) Failing after 4m23s
Details 
MODULES declares the module set a deployment runs, one entry per module as
`<id>@<version>=<install manifest URL>`, and the container arrives at it by
itself (MODULE_SYSTEM.md §2.7.2 decision 4). A module already unpacked at the
declared version is a no-op that makes NO network call, so a restart with the
network down comes up unchanged; anything else goes through install.js — same
allowlist, same sha256, same inspect-then-extract — and install() now takes an
`expect: {id, version}` so a URL resolving to another module or version is
refused while it is still only a manifest.

Resolution runs inside start(), between the seed and the require of app.js: the
seed is where the host allowlist setting comes from, and the require is what
scans the volume. That buys it the database, so a compose-installed module gets
the same provenance columns an admin install writes.

A failure is logged and carried, never fatal — an unreachable release host must
not take the site down. The declaration owns what is on the volume; the row owns
whether a module runs, so uninstalling a declared module returns its files at
the next start and leaves it disabled. The admin list gains that as a fourth
source (declared / declaredVersion / declaredError), because a declared module
that failed to resolve has no row, no directory and nothing mounted.

Deferring the app require moved core's schema ahead of the volume scan, and the
module schema-fragment replay was wired to core's schema — so every installed
module silently got no tables. Invisible to the suite (each one stubs the loader
or the pool) and to a smoke on a database that already had the tables; found by
booting against an empty one. ensureSchema() now takes `replayModules: false`
for the one caller that scans later, server.js replays them itself after the
require, and a bootOrder test pins the five steps in the only order they work in.

741 server tests (+18), 187 client (+5); manifest unchanged at 166 public + 2
internal, OpenAPI byte-identical.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-12 07:57:08 -05:00
Colby Whitlock
75f4d29e93 Merge pull request 'feat(admin): the Modules screen (phase 4, slice 2)' (#143) from feature/module-admin-screen into edge 
Reviewed-on: #143
 2026-08-12 08:52:23 +00:00
wtclaude
9083e4135a feat(admin): the Modules screen (phase 4, slice 2) 
The screen slice 1's API was written for: install from a release URL, enable,
disable, uninstall, purge, and restart. Admin-only, matching the server, and
core's own screen because it is how a module reaches the volume at all.

182 client tests (+21), manifest and OpenAPI unchanged.

Everything that decides what a row SAYS and which buttons it offers is in
`lib/moduleAdmin.js` -- plain JS, so the DOM-less runner can reach it, the
same reason `lib/adminNav.js` is. The JSX renders what it returns.

Three sources of truth, and they are allowed to disagree
--------------------------------------------------------
The row records what the operator decided and what the last boot did; the
loader says what is mounted and answering; the volume says whether there is a
directory at all. Picking one and rendering it is simpler and lies. The case
that makes it concrete is the one decision 3 creates on purpose: disable a
module (its onShutdown runs) and enable it again, and the row says `enabled`
while the loader still says `disabled` because nothing can start it before a
restart. Neither "Running" nor "Disabled" is true; "Restart to start" is.

Two shapes that are deliberately unlike the rest of the panel: the restart is
a BANNER, because a restart is a property of the server rather than of a
module and an operator who installed three modules should restart once; and
purge is offered inside the uninstall flow as a second confirm, because
purge.sql lives inside the directory being deleted and there is no later.

What the browser found that no test could
-----------------------------------------
Installing over a row the previous boot had left `startup_failed` rendered
"Failed at the require stage: module directory not present on the volume" one
second after the files had been written to the volume -- and, because that
branch is not pending, it suppressed the restart banner the install had just
told the operator to use. Every unit test passed, because none of them had
modelled a stale row plus a fresh install.

The fix is a derivation rather than a special case: the loader scans the
volume once at require time, so a module that is on the volume now and has no
live record arrived after that scan, and everything the row says about it
predates the install. That check runs before the failure one.

The same class, one place further on: an upgrade leaves the old code loaded,
so the row's version is a promise about the next boot. `liveVersion` (slice 1)
lets the screen say "Restart to finish upgrading" instead of reporting the new
version as running.

Verified against a live server and the real published release: pasted the
v0.3.0 install-manifest URL, restarted, watched the module register its five
mounts and seven streams and its own nav rows appear in the sidebar. Disable
ran its onShutdown for real -- the uo-link WebSocket closed, its routes went
to 404, and it left /public/modules -- and enable then showed the decision-3
state with the banner. The restart button itself was exercised through its
endpoint rather than clicked, because a window.confirm wedges the browser
automation.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-12 03:49:02 -05:00
wtclaude
732927a6bb fix(modules): three defects a real install exposed (phase 4, slice 1) 
Standing the slice-2 screen up against a live server and installing the
published module-uo v0.3.0 through it found three things, none of which any
unit test in this repo could have caught. Two of them are older than this
phase.

1. The boot refresh nulled every install's provenance
--------------------------------------------------------
`installed_modules.source` and `.sha256` exist so the admin panel can say
where a module came from. They never survived a restart.

`lifecycle.boot()` re-records every scanned module with no source and no
sha256 -- correctly, because a scan finds a directory and never where it came
from -- and `upsert` assigned both columns unconditionally. So an install's
provenance lasted exactly until the restart that install asked for, and the
screen then described a module installed from a URL as "placed on the volume
by hand". Verified live: install, restart, provenance gone.

Nothing could have caught it before now. Phase 4 wrote the first non-null
value these columns had ever had, so lifecycle.js's comment asserting that
"recordInstalled leaves what it is not given" described an intention rather
than the statement below it -- and modules.model.test.js's fake reproduced
the defect faithfully, assigning unconditionally just like the SQL.

Fixed with COALESCE(VALUES(col), col): a value overwrites, a NULL leaves what
is there. The fake now matches, and two tests pin both directions -- a boot
refresh must not wipe it, and a re-install from a new URL must still replace
it, or the column would become write-once and an upgrade would for ever show
where the first version came from.

2. The restart killed the server on Windows instead of stopping it
------------------------------------------------------------------
The route called `process.kill(process.pid, 'SIGTERM')` to reach server.js's
graceful-shutdown handler. That works on Linux. **Windows has no POSIX
signals, and Node documents SIGTERM there as unconditional termination of the
target process** -- so on a Windows host the restart killed the server
outright: no module onShutdown, no listener close, no pool close, no log
flush. Observed exactly that: the process was gone and the shutdown handler
had logged nothing at all.

`process.on('SIGTERM', ...)` is an ordinary EventEmitter listener, so
`process.emit('SIGTERM')` reaches the same handler on every platform without
involving the OS. One shutdown path, still; it just gets there by an event.

Deployment is Linux containers and would never have shown this. Development
is not, and neither is the smoke that found it.

The test was worse than useless: it stubbed `process.kill` and asserted it
had been called with SIGTERM, which is precisely the call whose MEANING
differs by platform. It now waits for the SIGTERM EVENT -- what server.js is
actually subscribed to -- so a pass here means the handler would run.

3. `present()` did not publish the running version
--------------------------------------------------
An upgrade writes new files and a new row while the old code stays loaded, so
the row's version is a promise about the next boot rather than a description
of this one. Adds `liveVersion` from the loader beside `liveState`, so the
screen can tell the two apart instead of reporting the new version as running.

723 server tests (+2), manifest and OpenAPI both unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-12 03:48:27 -05:00
Colby Whitlock
50d719cf46 Merge pull request 'feat(modules): install, uninstall, purge and restart (phase 4, slice 1)' (#142) from feature/module-install-service into edge 
Reviewed-on: #142
 2026-08-12 08:31:38 +00:00
wtclaude
b30e82cde2 feat(modules): install, uninstall, purge and restart (phase 4, slice 1)
All checks were successful
PR Checks / bot-install (pull_request) Successful in 17s
Details
PR Checks / client-build (pull_request) Successful in 26s
Details
PR Checks / server-tests (pull_request) Successful in 33s
Details 
The consumer half of a release module-uo's CI has been publishing since
phase 3 closed. Before this, core had the installed_modules provenance
columns and no code that could ever fill them: nothing fetched, verified,
unpacked, removed or purged anything, and there was no admin route at all.

Adds modules/archive.js, modules/install.js, schema.runPurge(),
lifecycle.stop(), loader.stopHook(), and /api/v1/admin/modules with eight
routes. 797 server tests (+76), manifest 158 -> 166 + 2 internal, OpenAPI
gains 8 operations and loses nothing.

Reject, never sanitise
----------------------
The download is the easy part: an https-only allowlist re-checked on every
redirect hop, a declared sha256 compared against the bytes that arrived, and
a byte cap. Unpacking is where the archive chooses the filenames, and core
writes into a directory bind-mounted from the host, so an escape is not
confined to the container.

archive.js inspects the whole archive before a byte is unpacked and refuses
absolute and drive-absolute paths, `..` segments, NUL bytes, backslashes,
anything that is not a regular file or a directory, more than one top-level
entry, and anything over the entry or byte caps. Refusing symlinks and
hardlinks outright is what keeps this off the majority of node-tar's
published advisories rather than depending on the library to contain them.

That two-pass shape is load-bearing, and it was measured rather than assumed:
extracting an archive whose fourth member escapes upward throws under
node-tar 7.5.22 -- and leaves the first three members on disk. The loader
scans that directory at require time on the next boot, so a half-unpacked
module is a module. Everything therefore happens in a scratch directory that
is removed on any failure, and the move into place is the last step.

`tar` is pinned to ^7.5.22 rather than the ^6 that installs by default: 6.x
is flagged critical, and reading the advisory list is what the file's header
now says out loud -- almost all of it is hardlink or symlink traversal and
PAX header interpretation differentials, which is exactly this feature's
threat model.

Two things the plan had wrong
-----------------------------
The bundle's top-level directory is `module-uo-<version>`, not the module id
-- so "the top-level name must equal the id" was checked against nothing real.
The extractor strips that level instead, because its name belongs to whoever
published the bundle and the directory it lands in is core's. What is checked
instead is the unpacked module.json: a manifest promising `uo` and delivering
something else is refused rather than installed under the name it promised.

And purge cannot be a follow-up action (decision 5): purge.sql lives inside
the directory uninstall deletes. It is offered in the uninstall flow and as a
standalone action on a still-installed module, and the standalone one refuses
unless the module is already disabled -- dropping tables under something that
is still serving leaves it answering out of a world that no longer exists.

Disable now means stopped
-------------------------
lifecycle.stop() dispatches that one module's onShutdown before flipping the
guard, so a module an operator switches off actually releases its sockets and
closes its streams instead of merely becoming unreachable. The hook runs
first and the state moves after it, because while onShutdown runs the module
is still `started` and that is the only state in which its routes and the
world it is tearing down agree. A hook that throws does not stop the disable
-- the opposite of the boot path's rule, and deliberately.

Enable is not its mirror and there is no start(id) beside it. There is no
onBoot re-dispatch and the hooks were never promised re-entrant, so enable
moves the row and the restart route starts it. A test pins that enable does
not touch the loader, because "fixing" it is a one-line change that would put
a module with closed sockets back on the nav.

Restart raises SIGTERM against its own process rather than calling the
shutdown path directly, so server.js's handler stays the one graceful-shutdown
path and this route cannot drift from it.

The allowlist bootstraps from MODULE_SOURCE_HOSTS into a settings row and is
admin-managed after that (decision 6); seedDefault is INSERT IGNORE, so
changing the variable on an existing deployment is a no-op by design. An empty
list forbids every install rather than allowing every host -- the safe
direction for a value someone might blank by accident.

Verified against the real v0.3.0 release
----------------------------------------
Not a fixture: fetched the published install manifest over the real Gitea
host and its redirect chain, verified the sha256, inspected and unpacked the
252,517-byte artifact to 82 files, and then booted core against the result --
the module registered its five mounts, seven streams and eight capabilities
and resolved its client chunk, with no scratch directory left behind.

Two defects this slice's own tooling caught, both of which had already been
written down as classes:
  - the controller destructured runPurge at require time, capturing the
    function rather than the module, which made the one dependency whose
    ORDER matters the one that could not be substituted;
  - two swagger annotations carried an apostrophe inside a quoted string,
    dropped silently by swagger-autogen before slice 5 taught it to fail loudly.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-12 03:09:45 -05:00
Colby Whitlock
2cb549e9e5 Merge pull request 'feat(modules): merge module OpenAPI fragments into /api/docs.json (phase 3, slice 5)' (#141) from feature/module-openapi-merge into edge 
Reviewed-on: #141
 2026-08-12 04:12:07 +00:00
wtclaude
adff20be7b feat(modules): merge module OpenAPI fragments into /api/docs.json (phase 3, slice 5)
All checks were successful
PR Checks / bot-install (pull_request) Successful in 18s
Details
PR Checks / client-build (pull_request) Successful in 26s
Details
PR Checks / server-tests (pull_request) Successful in 31s
Details 
Core's half of the slice that closes phase 3. Two things: the request-time
fragment merge core has owed since phase 1, and the last of core's UO copy.

**The merge (MODULE_API.md §6.1a).** `swagger-output.json` is core's own routes
and cannot be anything else — it is generated on a developer's machine and
committed, so it must come out the same regardless of what they had checked out,
and a module arrives on a volume long after the image was built. Module routes
therefore reach the document at request time, from the `swagger-fragment.json`
each module ships: `swagger/docsSpec.js` merges the fragments of STARTED modules
over the committed spec, cached on a new loader state version and rebuilt when a
module's state moves.

Until now neither half existed. `swagger/mergeSpec.js` named the request-time
caller in its header and that caller was never written, so the 72 routes
module-uo serves were in no OpenAPI spec at all — core's standing rule ("never
ship a route that isn't in the spec") broken by the extraction rather than by a
route.

Core wins every key collision, `swagger-output.json` is never mutated (it is a
require()d JSON module — one in-place merge would be permanent AND cumulative),
and a fragment that is missing or unreadable costs that module its paths and
nothing else. The Swagger UI is now built per request for the same reason the
JSON is: bound once at require time it would show core's routes for the life of
the process while /api/docs.json showed the merged set.

**The last of core's UO copy** (slice 4 deferred it; §5.2's check reads code, not
prose, so none of this was caught):

- 31 UO schemas and 4 UO tags in `swagger/swagger.js`, describing routes core has
  not served since slice 1 — 578 lines. They moved to module-uo, namespaced
  `Uo…`, and arrive back through the merge on an instance that installs it.
- `info.description` said "a private Ultima Online shard".
- README.md's 48 UO mentions, including the architecture diagram and the whole
  `## Shard integration (uo-link)` section, now `## Modules`.
- `TOWNCRIER_DURATION_SEC` and `UOLINK_*` in the two `.env.example`s: read by the
  module, not by core, and documented in the module's README instead.

**Two dropped annotations, and the reason nobody knew.** swagger-autogen reports
an annotation it cannot parse and then prints Success in green, having skipped
it. `npm run swagger` now captures its diagnostics and fails — which immediately
found `POST /api/v1/admin/invites` and `POST /api/v1/auth/invite/:token/accept`
documented with an EMPTY request body, both since the day they were written.

Fixing the tag list also cleared five tags used by routes but never declared
(`Admin · Email`, `Admin · Invites`, `Admin · Moderation`, `Admin · Pages`,
`Auth · Me`) — the same defect class, in the other direction.

- 646 server tests (+9), 157 client tests unchanged
- routes.manifest.json unchanged (158 public + 2 internal); check:modules clean
- swagger-output.json: 128 paths, 69 schemas, 0 orphan tags, 0 orphan schemas
- verified against a real boot with module-uo installed: 197 merged paths
  (128 core + 69 module), all four module tags, 31 Uo schemas, no dangling $refs,
  /api/docs renders the module's operations with zero console errors

Refs: docs/website/MODULE_API.md §2.8, §6.1a; MODULE_SYSTEM.md §2.7.1

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-11 22:57:37 -05:00
Colby Whitlock
87230c879a Merge pull request 'refactor(modules)!: de-UO core's copy, and enforce it (phase 3, slice 4)' (#140) from feature/module-de-uo-core into edge 
Reviewed-on: #140
 2026-08-12 03:02:37 +00:00
wtclaude
0c4eacfa4a refactor(modules)!: de-UO core's copy, and enforce it (phase 3, slice 4)
All checks were successful
PR Checks / bot-install (pull_request) Successful in 18s
Details
PR Checks / client-build (pull_request) Successful in 27s
Details
PR Checks / server-tests (pull_request) Successful in 31s
Details 
Phase 3's acceptance criterion 1, made real. Three things, one review:

**The dead bindings.** `client/src/api/client.js` still carried ~190 lines of UO
namespaces — `shard`, `atlas`, the two SSE URLs, `admin.shard/shardOps/atlas/
userShard`, the uo-link and town-crier calls, `player.shard` — with zero core
consumers since slice 3 deleted the views. module-uo vendors its own bindings.
The five assertions core's `apiClient.test.js` made about those URLs moved with
them (Module-uo#5); the encoding test that used `governorHistory` now uses a
core route.

**The copy.** Core is the platform, not one game's site, so its words are
game-neutral now: `About`, `Screenshots`, `Website`'s cards, `Status` (which was
never about a game server at all — it reports site mode), `Wiki`, `SiteFooter`,
the default hero, `brand.js`'s tagline and description, the seeded wiki
categories, and two user-visible NavEditor strings that named a module's admin
screen by its proper name. Which game an instance is for is the operator's to
say — BRAND_* vars, the hero editor, CMS pages — and every real instance already
does: `.env.uomysticmoon.example` sets both brand strings explicitly, so nothing
live changes wording. Wiki page SLUGS are untouched: `seedDefault*` only inserts
what is absent, so renaming one adds a duplicate page to every install.

Also gone: an orphan comment block in `schema.sql` describing the spawn-atlas
tables slice 1 took away, and the two settings rows core seeded for a module
(`game_account_signup`, `uo_link_protocol_3_migrated`). The second was a live
defect — see Module-uo#5, which takes ownership of both and repairs the
one-shot migration core's ordering had disabled.

**The check.** `scripts/checkModuleIdentifiers.js` + `npm run check:modules`,
first step of the server-tests job because it needs no dependencies. It reads
CODE, not prose — file names, import specifiers, route path literals, declared
identifiers and property names — per §5.2, so core's English may still say
"shard" where saying it is worth more than the word costs.

Two things it gets right only because getting them wrong was tried first: it
matches WHOLE WORDS (a substring pass flags `defaultImage`, which contains
"ultIma", four times in this repo), and it strips comments and string bodies in
one character walk (a comment contains quotes, a string contains `//`) — the
`checkImports.js` lesson. It has its own 17-test suite, because a boundary check
that silently stops checking is worse than none. The three §6.5 grandfathering
allowlists are exempt by name, and an exemption that stops matching fails the
build rather than lingering.

BREAKING CHANGE: core no longer seeds `game_account_signup` or
`uo_link_protocol_3_migrated`; module-uo's schema fragment does. An install
running core without module-uo keeps whatever rows it already has and gains no
new ones — nothing in core reads either key.

Deferred to slice 5, deliberately: README.md's 48 UO mentions, including a
`## Shard integration (uo-link)` section and the architecture diagram. That is
documentation, which §5.2 does not cover, and it belongs with the phase-closing
docs pass rather than half-done here.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-11 21:41:18 -05:00
61 changed files with 6642 additions and 5520 deletions
Expand all files Collapse all files

44
.env.example
View File

@@ -33,8 +33,8 @@ LOG_FILE=app.log
# BRAND_NAME / BRAND_CONTACT_EMAIL.
BRAND_NAME=Runic Gateway
BRAND_SHORT_NAME=Runic Gateway
BRAND_TAGLINE=an independent private Ultima Online shard
BRAND_DESCRIPTION=Runic Gateway — an independent private Ultima Online shard. News, screenshots, guides, and community notes.
BRAND_TAGLINE=an independent game community
BRAND_DESCRIPTION=Runic Gateway — an independent game community. News, screenshots, guides, and community notes.
BRAND_CONTACT_EMAIL=
BRAND_URL=
# Accent color — drives the web theme's --accent and the Discord embed color.
@@ -107,20 +107,32 @@ CLIENT_ORIGIN=http://localhost:5173
BOT_INTERNAL_URL=http://bot:4100
BOT_INTERNAL_KEY=change-me-to-a-long-random-string
# uo-link sidecar — the HTTP + WebSocket bridge to the ServUO game server. The
# website ingests its live event feed and proxies its read queries/commands
# (shard status, online players, player-vendor sales, IDOC houses, character
# sheets, account linking, town-crier). In production the sidecar + shard run on
# a DIFFERENT host from the website, so both URLs are configurable. The
# shared-secret auth token is NOT an env var — it is entered in the admin panel
# (Shard page) and stored encrypted in the DB (same pattern as the Discord bot
# token). These URLs are just defaults; the admin can override them at runtime.
UOLINK_BASE_URL=http://127.0.0.1:8080
UOLINK_WS_URL=ws://127.0.0.1:8080/ws
# Wire protocol this build speaks (3 = Protocol 3.0). Only a fallback for a site
# with nothing saved yet — the admin panel's pinned value wins — but set it lower
# if you deliberately run an older sidecar.
UOLINK_PROTOCOL=3
# ─── Installed modules ───
# A module is a directory on the modules volume (see MODULES_DIR in
# server/.env.example); everything about a specific game lives in one, and core
# knows nothing about any of them. A module may read its own env vars, and they
# belong here because Compose passes this file to the container.
#
# MODULES declares the set this deployment runs, and the container arrives at it
# on its own — no admin panel, no `tar -xf` on the host. One entry per module,
# `<id>@<version>=<install manifest URL>`, whitespace- or comma-separated:
#
# MODULES=uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json
#
# A module already unpacked at the declared version is a no-op that never touches
# the network, so a restart with the internet down brings the site up exactly as
# it was; only a missing or different version is fetched, verified against the
# sha256 its manifest declares, and unpacked. A failure is logged and shown in
# Admin → Modules, and the site starts anyway. The variable owns what is on the
# volume, not what runs — a module disabled from the admin panel stays disabled.
# Leave it unset to install from the admin panel instead.
#
# RunicGateway/Module-uo, for example, reads UOLINK_BASE_URL / UOLINK_WS_URL /
# UOLINK_PROTOCOL as the defaults for its connection to a uo-link sidecar, and
# TOWNCRIER_DURATION_SEC for its news leg. Its README documents them; they are
# left out here rather than half-copied, because a copy of another repo's
# settings is a copy that goes stale silently. With no module installed, none of
# this applies and the site runs as core.
# ─── Push notifications (M7) — self-hosted ntfy UnifiedPush relay ───
# The `ntfy` compose service and the backend's push fan-out (opt-in notifications

10
.gitea/workflows/pr-checks.yml
View File

@@ -47,10 +47,20 @@ jobs:
node-version: 20
cache: npm
cache-dependency-path: server/package-lock.json
- name: Check core names no module identifier
# Phase 3's acceptance criterion 1 (MODULE_API.md §5.2): core must not
# name a module's files, import them, route to them, or declare its
# symbols. Before `npm ci`, deliberately — it is plain Node over
# server/ and client/ source with no dependency of its own, so putting it
# first makes a boundary break the first thing a reviewer sees instead of
# something found under a pile of unrelated failures, and it costs
# nothing when it passes.
run: npm run check:modules
- name: Install server deps
run: npm ci --prefix server
- name: Run server tests
run: npm test --prefix server
- name: Check the route manifest is current
# The URL surface is frozen while the routers are carved up by capability
# (docs/website/API_V2_PLAN.md § Phase 2). Regenerating from the live Express

274
README.md
View File

@@ -8,16 +8,19 @@
[![Security Rating](https://sonar.whitlocktech.com/api/project_badges/measure?project=runic-gateway-website&metric=security_rating&token=sqb_d3593f26ac5663cd3e666039b7038f3248e8df50)](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
[![Vulnerabilities](https://sonar.whitlocktech.com/api/project_badges/measure?project=runic-gateway-website&metric=vulnerabilities&token=sqb_d3593f26ac5663cd3e666039b7038f3248e8df50)](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
Public site, wiki, and protected admin panel for a private Ultima Online shard — a
full-stack app in one repo. Branding is instance-configurable via `BRAND_*` (see
[Branding](#branding)); **UOMysticmoon** is the first instance.
Public site, wiki, and protected admin panel for a game community — a full-stack app
in one repo. Everything specific to a *particular* game lives in an installable
module, not here. Branding is instance-configurable via `BRAND_*` (see
[Branding](#branding)); **UOMysticmoon**, an Ultima Online shard, is the first
instance, and its game half is
[RunicGateway/Module-uo](https://gitea.whitlocktech.com/RunicGateway/Module-uo).
A full-stack app in one repo:
- **Backend** — Node.js + Express REST API (layered `router → controller → model → db`), MariaDB, a provider-agnostic session layer (JWT cookie for web, bearer tokens for mobile, pluggable SSO).
- **Frontend** — React + Vite single-page app (public site, wiki, and the admin panel), dark "gothic" theme (Cinzel + Georgia).
- **Deploy** — Docker Compose (app + MariaDB) behind a reverse proxy (Pangolin, Nginx, Caddy, Traefik, …). Express serves the built SPA in production.
- **Shard link** — a live bridge to the in-game ServUO shard through the **uo-link** sidecar ([RunicGateway/link](https://gitea.whitlocktech.com/RunicGateway/link)): the site ingests a live event feed and makes server-side REST calls to show shard status, economy, staff presence, IDOCs, live activity, and per-character sheets. See [Shard integration (uo-link)](#shard-integration-uo-link).
- **Modules** — the game-specific half of a site is a module dropped onto a volume: it adds routes, database tables, nav entries and whole SPA pages without this repo knowing anything about the game. See [Modules](#modules).
The design reference is [BACKEND_DESIGN.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/BACKEND_DESIGN.md) (API contract, schema, security), in the [**RunicGateway/docs**](https://gitea.whitlocktech.com/RunicGateway/docs) repo — where all project documentation now lives.
@@ -37,7 +40,7 @@ The design reference is [BACKEND_DESIGN.md](https://gitea.whitlocktech.com/Runic
- [Pages & routes](#pages--routes)
- [API endpoints](#api-endpoints)
- [API documentation (Swagger)](#api-documentation-swagger)
- [Shard integration (uo-link)](#shard-integration-uo-link)
- [Modules](#modules)
- [Environment variables](#environment-variables)
- [Security](#security)
- [Logging](#logging)
@@ -48,8 +51,8 @@ The design reference is [BACKEND_DESIGN.md](https://gitea.whitlocktech.com/Runic
## Architecture
How the pieces fit together — the React SPA and native app talk to one Express backend
(`router → controller → model → db`), which persists to MariaDB and bridges to the live
game world only through the **uo-link** sidecar. The shard itself is never internet-facing.
(`router → controller → model → db`), which persists to MariaDB. Anything that knows
what game this site is about lives in an installed module, on the right of the diagram.
```mermaid
flowchart TB
@@ -69,32 +72,29 @@ flowchart TB
subgraph backend["server/ — Express backend"]
direction TB
mw["Middleware<br/>helmet · siteMode · noindex<br/>rateLimit · loginProtection · botScore · validate"]
router["Router /api/v1<br/>auth (web · mobile · sso) · public · admin"]
router["Router /api/v1<br/>auth (web · mobile · sso) · public · admin · player"]
ctrl["Controllers"]
auth["Session layer (auth/)<br/>sessionService · JWT/cookie · bearer · SSO+PKCE"]
model["Models (.model + .db)<br/>raw parameterized SQL — no ORM"]
sse["SSE fan-out<br/>public stream (allowlist) · admin stream (sensitive)"]
subgraph shardutil["Shard integration (utils/)"]
ingest["shardIngest.js<br/>WS ingest dispatcher"]
restcli["uoLinkClient.js<br/>REST client (never throws)"]
end
loader["modules/loader.js<br/>scans the volume · mounts · registries · lifecycle"]
secret["secretBox.js<br/>AES-256-GCM secrets at rest"]
end
bot["bot/<br/>Discord bot"]
end
db[("MariaDB<br/>users · posts · wiki · settings · activity<br/>mobileSessions · authProviders · userIdentities<br/>uoLinkConfig · shard_online/economy/houses/events")]
db[("MariaDB<br/>users · posts · wiki · settings · activity<br/>mobileSessions · authProviders · userIdentities<br/>installed_modules · &lt;module&gt;_*")]
%% ---------- Shard side ----------
subgraph shardside["Game shard (never internet-facing)"]
%% ---------- Module side ----------
subgraph modside["modules/&lt;id&gt;/ &nbsp;— installed, not built (e.g. Module-uo)"]
direction TB
sidecar["uo-link sidecar<br/>(Rust) — the only bridge exposed"]
servuo["ServUO shard<br/>(C# plugin)"]
modsrv["server/ — routers, models, schema fragment<br/>reaches core only through ctx"]
modcli["client/dist/entry.js — prebuilt ESM chunk<br/>React shared via window.__rg"]
end
game["The game<br/>whatever the module talks to<br/>(for Module-uo: a ServUO shard,<br/>via the uo-link sidecar)"]
%% ---------- Edges ----------
browser <-->|"same-origin JSON + SSE (cookie)"| mw
mobile -->|"REST (bearer access/refresh)"| mw
@@ -104,40 +104,42 @@ flowchart TB
mw --> router --> ctrl
ctrl --> auth
ctrl --> model
ctrl --> restcli
ctrl --> sse
auth --> model
model <--> db
auth -. reads/writes secrets .-> secret
restcli -. reads config/token .-> secret
ingest --> model
ingest --> sse
sse -->|"live events"| browser
bot -->|"messages"| discord
bot <--> db
restcli -->|"REST: /char /roster /economy /history · /link/confirm · /towncrier"| sidecar
sidecar -->|"WebSocket live event feed (bearer + X-UOLink-Version)"| ingest
servuo -->|"loopback TCP 127.0.0.1:7788<br/>newline-delimited JSON (shard dials out)"| sidecar
loader -->|"mounts under /api/v1/&lt;tier&gt;/&lt;prefix&gt;"| router
loader -->|"require() + register(ctx, api)"| modsrv
modsrv -->|"ctx.db · ctx.push · ctx.activity …"| model
modsrv <--> game
browser -->|"&lt;script type=module&gt; injected by htmlShell"| modcli
%% ---------- Styling ----------
classDef ext fill:#2d2233,stroke:#7a5c94,color:#e8dff0;
classDef store fill:#1f2d2a,stroke:#4c8c7d,color:#dff0ea;
classDef bridge fill:#2d2620,stroke:#94764c,color:#f0e6d8;
class idp,discord ext;
classDef mod fill:#2d2620,stroke:#94764c,color:#f0e6d8;
class idp,discord,game ext;
class db store;
class sidecar,servuo bridge;
class modsrv,modcli mod;
```
- **One backend, layered.** Every request flows `middleware → router → controller → model → db`.
Web browsers authenticate with an httpOnly JWT cookie; the native app uses short-lived bearer
access tokens plus rotated refresh tokens; SSO (Google/Discord/OIDC) is link-only and PKCE-guarded.
All three surfaces produce the *same* session via the session layer.
- **The shard is never reachable.** The ServUO shard *dials out* over loopback TCP to the uo-link
sidecar; only the sidecar is exposed, and only the backend talks to it. The REST client
(`uoLinkClient.js`) never throws, so the site degrades gracefully when the shard is down.
- **Sensitive events stay private.** Ingested game events fan out to browsers over two SSE channels —
a public allowlist stream and an admin-only stream that adds staff audit / cheat / login events.
- **Core knows nothing about any game.** Routes, tables, nav entries, SPA pages and push streams for
a specific game arrive from a module the operator installed. Core provides the seams; the module
fills them. See [Modules](#modules).
- **A module that fails must never take the site down.** The loader catches failures across a
module's whole lifecycle and marks that one module `startup_failed`; the site comes up with its
routes and nav absent, and the admin panel says why.
- **Sensitive events stay private.** Events fan out to browsers over two SSE channels — a public
allowlist stream and an admin-only stream that adds staff audit / cheat / login events. Which
event kinds are public is decided by the module that publishes them, and core enforces the split.
---
@@ -164,12 +166,13 @@ website/
│ │ ├─ server.js bootstrap: ensure schema → seed → listen (0.0.0.0)
│ │ ├─ app.js middleware + static SPA + routes
│ │ ├─ auth/ session layer: session.service · token (JWT/cookies) · session.middleware · ssoState (PKCE/CSRF) · providers/ (base · oauth2 · google · discord · genericOidc · registry)
│ │ ├─ router/v1/ auth (web · mobile · sso) / public / admin route groups
│ │ ├─ model/ users · posts · wiki · settings · activity · mobileSessions · authProviders · userIdentities (.model + .db)
│ │ ├─ router/v1/ auth (web · mobile · sso) / public / admin / player route groups
│ │ ├─ model/ users · posts · wiki · settings · activity · mobileSessions · authProviders · userIdentities · modules (.model + .db)
│ │ ├─ modules/ loader (scan · validate · mount) · registries (the seams) · lifecycle (boot/shutdown + reconcile)
│ │ ├─ middleware/ siteMode · noindex · rateLimit · loginProtection · botScore · validate
│ │ └─ utils/ auth (compat facade) · totp (2FA) · secretBox (AES-GCM secrets) · db (pool) · mailer · logger
│ │ └─ utils/ auth (compat facade) · totp (2FA) · secretBox (AES-GCM secrets) · db (pool) · mailer · logger · htmlShell
│ ├─ db/ schema.sql + seed.js
│ ├─ swagger/ swagger.js (OpenAPI generator config) + swagger-output.json (generated spec)
│ ├─ swagger/ swagger.js (generator config) · swagger-output.json (generated, core only) · docsSpec.js (merges module fragments at request time)
│ └─ .env.example
├─ client/ React + Vite SPA
│ ├─ src/
@@ -178,9 +181,11 @@ website/
│ │ ├─ routes/admin/ AdminLogin (password + TOTP + SSO buttons), AdminLayout, views/ (Dashboard, Posts, Wiki, Settings, Activity, Bot Activity, Authentication, Users, Account) + editors
│ │ ├─ components/ SiteHeader, SiteFooter, layout, guards, Modal, ProviderIcon (inline SSO SVGs), …
│ │ ├─ contexts/ AuthContext, SiteContext
│ │ ├─ modules/ the client registry: routes · nav · slots · feature gates · window.__rg
│ │ ├─ api/client.js fetch wrapper (sends cookies)
│ │ └─ styles/theme.css design tokens
│ └─ public/assets/img/ hero image
├─ modules/ installed modules, one directory each — a Docker bind mount; empty here
├─ Dockerfile builds client → serves via Express
├─ docker-compose.yml app + MariaDB
├─ .env.example root env (used by Compose)
@@ -306,7 +311,7 @@ npm start # node server → serves API + SPA at http://localhost:3
| `/site/screenshots` | Screenshot gallery |
| `/site/five-on-friday` | Five on Friday |
| `/site/newsletter` · `/site/newsletter/:id` | Newsletter list + issue |
| `/site/about` · `/site/status` | About · Shard status |
| `/site/about` · `/site/status` | About · Site status |
| `/wiki` · `/wiki/:slug` | Wiki landing + article (auto table-of-contents) |
**Admin** (cookie auth, `noindex`):
@@ -324,6 +329,13 @@ npm start # node server → serves API + SPA at http://localhost:3
| `/admin/users` | User management |
| `/admin/account` | Account security (self-service TOTP two-factor + linked SSO accounts) |
**Player** (any signed-in account, `noindex`): `/player` and its self-service views. Staff are a
superset of players and reach these too.
An installed module adds its own pages under `/<id>/*`, `/admin/<id>/*` and `/player/<id>/*` — for
Module-uo that is `/uo/shard`, `/admin/uo/link`, `/player/uo/characters` and the rest. Core does not
know their names; they arrive with the module and are interleaved into the nav.
---
## API endpoints
@@ -335,9 +347,15 @@ npm start # node server → serves API + SPA at http://localhost:3
| SSO | `/api/v1/auth` (`providers` — public discovery; `sso/:provider/start`, `sso/:provider/link`, `sso/:provider/callback`) | redirect flow |
| Public | `/api/v1/public` (`settings`, `status`, `posts/:category`, `posts/:category/:idOrSlug`, `wiki`, `wiki/:slug`, `contact`) | none |
| Admin | `/api/v1/admin` (`dashboard`, `site-mode`, `posts`, `posts/upload`, `wiki`, `settings`, `activity`, `bot-activity`, `bot-activity/unban`, `auth/providers` (CRUD), `users`, `account`, `account/totp/*`, `account/identities`) | cookie (admin) |
| Public · Shard | `/api/v1/public/shard` (`status`, `feed`, `economy`, `online`, `idoc`, `stream`) | none |
| Player · Shard | `/api/v1/player/shard` (`link`, `accounts`, `roster/:account`, `vendors/:account`, `char/:serial`, `sales`) | cookie/bearer (player) |
| Admin · Shard | `/api/v1/admin/shard` (self linking, same as player) · `/api/v1/admin/uo-link` (`config`, `towncrier`, `stream`) | cookie (staff / admin) |
| Player | `/api/v1/player` (`me`, credentials, 2FA, identities, appeals) | cookie/bearer (any signed-in account) |
| Modules | `/api/v1/public/modules` — id, name, version and capabilities of the modules currently serving | none |
**Module routes are not in this table**, because they are not core's. An installed module mounts
under `/api/v1/public/<prefix>`, `/api/v1/admin/<prefix>` and `/api/v1/player/<prefix>`; which
prefixes exist depends on what is installed. Module-uo, for instance, serves 72 routes under
`/shard`, `/atlas` and `/uo-link` — see its own
[`routes.manifest.json`](https://gitea.whitlocktech.com/RunicGateway/Module-uo/src/branch/main/routes.manifest.json).
On a running instance, `/api/docs` lists everything, core and modules together.
Post categories (URL form): `news`, `five-on-friday`, `newsletter`, `screenshots`.
`authMethod` on a session ∈ `local · totp · mobile · google · discord · oidc`.
@@ -380,6 +398,28 @@ npm run swagger # → server/swagger/swagger-output.json
If the generated spec is missing, the server logs a warning and simply disables `/api/docs` (it does
not crash).
**The committed spec is core only, and the served one is not.** swagger-autogen is *static
analysis* — it parses `src/app.js` as text and follows the literal `app.use(…)` chain — so it can
see neither an installed module (which arrives on a volume long after the image was built, and
mounts through a call no parser can follow) nor an extension slot (whose router is created empty and
filled later). Both are handled by merging a **fragment**:
- **Extension slots** contribute at generation time, from `server/swagger/slotSpecs.js`, so they are
in the committed file.
- **Modules** contribute at request time, from the `swagger-fragment.json` each one ships, merged by
`server/swagger/docsSpec.js`. So `/api/docs.json` on a running instance describes more than
`npm run swagger` produces here, and `swagger-output.json` stays reproducible on any machine
regardless of what is installed.
**Core wins every key collision** — a module cannot redefine a core path, tag or schema by shipping
one with the same name; the collision is logged and the module's version dropped.
One thing worth knowing if you edit an annotation: swagger-autogen **reports a broken one and then
succeeds anyway**, dropping it. `npm run swagger` now captures those diagnostics and fails, which is
how two annotations that had been silently documenting an empty request body were found. If it
rejects yours, the usual causes are an object literal a brace short, or a `"` or backtick inside a
single-quoted description (it re-quotes both to `'` before evaluating).
### The route manifest (frozen URL surface)
`server/routes.manifest.json` is a generated, sorted `{ method, path }` list of every route the two
@@ -412,94 +452,101 @@ annotated routes appear), the manifest records reality.
---
## Shard integration (uo-link)
## Modules
The site is wired to the live in-game world through **uo-link**, a standalone sidecar service that
runs next to the ServUO shard. Its source lives in a separate repo:
**[RunicGateway/link](https://gitea.whitlocktech.com/RunicGateway/link)**. uo-link speaks the shard's internals and
exposes a small, authenticated HTTP + WebSocket API; this website is a *client* of it. The shard
itself is never exposed to the internet — only the sidecar is, and only the website's backend talks
to it.
**Everything specific to a game is a module.** Core has no idea what an "account", a "character" or
a "shard" is; it provides seams, and a module fills them. That is what makes one image able to run a
site for any game rather than for Ultima Online in particular.
### Setting up the shard side
The design of record is
[MODULE_SYSTEM.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_SYSTEM.md);
the normative contract — the one to read before writing a module — is
[MODULE_API.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_API.md).
The worked example is [RunicGateway/Module-uo](https://gitea.whitlocktech.com/RunicGateway/Module-uo),
which is where everything this README used to describe under *Shard integration (uo-link)* now
lives: the sidecar client, the ingest dispatcher, account linking, the town crier, the spawn atlas,
and every page that renders them.
You do not build or place any of it by hand. The
**[Runic Gateway installer](https://gitea.whitlocktech.com/RunicGateway/installer)** runs on the
shard host, deploys the ServUO plugin and the uo-link sidecar as a matched, protocol-checked pair,
registers the sidecar as a service, and ends by printing the four values this site needs:
### An operator never builds anything
That constraint shapes the whole design. Installing a module is the WordPress-plugin experience — an
admin-panel action, or a directory dropped onto the `modules/` volume — because production runs a
prebuilt, pull-only image with no toolchain in it. So a module ships **assembled**: its client half
is a prebuilt ESM chunk that resolves React from a `window.__rg` global core owns (an import map
would have to be inline, and the CSP is `script-src 'self'`), and its one runtime dependency travels
inside the tarball.
```
Base URL http://<shard-host>:8080
WebSocket URL ws://<shard-host>:8080/ws
Protocol version 3
Auth token 4f9c…
modules/
└─ uo/ one directory per module; the id is the directory name
├─ module.json id, version, coreApi range, mounts, extensions, capabilities
├─ swagger-fragment.json merged into /api/docs.json while the module is running
├─ server/ routers, models, and an idempotent schema.sql fragment
└─ client/dist/entry.js the prebuilt chunk, injected by utils/htmlShell.js
```
Paste them into **Admin → Shard** here and the bridge is live. The operator guide is
[installer/INSTALL.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/installer/INSTALL.md);
its [Appendix A](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/installer/INSTALL.md#appendix-a--installing-by-hand)
is the same deployment done by hand, still supported, for a host that cannot run the binary or a
developer working from a source tree.
`modules/` is a bind mount in `docker-compose.yml`, so placing a directory there by hand is a
supported install. The directory is tracked in git (via its README) on purpose: Docker recreates a
*missing* bind-mount source as `root:root`, and the container is uid 1000.
Nothing here needs the shard to exist: with no sidecar configured the site renders normally and
shows the shard offline.
### Three ways in, and none of them is a build
### How it works
| | How | Where it fits |
|---|---|---|
| **Admin panel** | Admin → Modules, paste the URL of a release's install manifest | The click path. Installs, upgrades, disables, uninstalls and purges, with a restart button — no shell on the box |
| **`MODULES`** | Declare the set in the environment; the container resolves it at every start | The compose-managed host. The running set is a line in a file you version-control, not the residue of past clicks |
| **By hand** | `tar -xf module-uo-0.3.0.tar.gz -C ./modules && mv modules/module-uo-0.3.0 modules/uo`, then restart | Development, and any host where the other two do not fit |
`MODULES` takes one entry per module, whitespace- or comma-separated:
```
ServUO shard ──▶ uo-link sidecar (RunicGateway/link) ──▶ website backend ──▶ browser
REST + WebSocket, bearer-auth ingest + REST same-origin JSON/SSE
MODULES=uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json
```
- **Connection is admin-managed, not env.** The sidecar's base URL, WebSocket URL, shared-secret
token, and protocol version are stored in the database (`uoLinkConfig`), edited from the
**Admin → Shard** panel. The token is **encrypted at rest** (AES-256-GCM) and is **write-only** in
the API — it is never returned to any client and never sent to the browser. Every call the backend
makes carries `Authorization: Bearer <token>` and an `X-UOLink-Version` header (a protocol
mismatch fails fast with `409` instead of being mis-parsed).
- **Live ingest (WebSocket).** When enabled, the backend opens an outbound WebSocket to the sidecar
and receives a stream of game events — `mob.login`/`logout`, `char.vitals`, `economy.supply`,
`vendor.sale`, `player.death`/`murdered`, `house.decay` (IDOC), staff `audit.*`/`cheat.*`,
`link.request`, and `server.hello`/`shutdown`. A single dispatcher (`utils/shardIngest.js`) routes
each event: state-changing kinds update `shard_online` / `shard_economy` / `shard_houses`; notable
kinds are appended to an append-only `shard_events` log; high-frequency kinds (vitals, supply
ticks) only update state and are not logged. A changed boot id on `server.hello` is detected as a
restart and stale "online" rows are cleared. On reconnect the backend backfills missed events via
the sidecar's `/history`.
- **Live round-trips (REST).** For point-in-time reads the backend calls the sidecar directly —
`/char/serial/:serial`, `/roster/:account`, `/vendors/:account`, `/economy`, `/history` — plus
commands `/link/confirm` and `/towncrier`. The REST client (`utils/uoLinkClient.js`) **never
throws**: every call returns `{ ok, data, status }`, so a shard that is down or mid-restart
degrades to a `503`/retry banner instead of a 500.
- **Fan-out to the browser.** Ingested events are pushed to browsers over **Server-Sent Events**.
Two channels exist: a **public** stream carrying only a safe allowlist of kinds, and an
**admin-only** stream that also includes sensitive kinds (staff audit, cheat detection, login
attempts, IPs). Sensitive kinds can never leak onto the public channel.
The id and the version are written out rather than discovered inside the manifest so that **the
no-op case needs no network**: a module already unpacked at the declared version is answered by
reading its own `module.json`, so a restart with the internet down brings the site up exactly as it
was. Only a missing or different version is fetched, and it goes through the same
verify-and-unpack path — allowlisted `https` host, sha256 from the manifest, whole-archive
inspection before anything is written — that the admin panel uses. A version that cannot be
resolved is logged and shown on the admin screen; **it never stops the site from starting**.
### Account linking
The declaration owns what is *on the volume*, never what runs. A module disabled from the admin
panel gets its files back at the next start and stays disabled, because the row and the variable are
answering different questions.
A player (or staff member) proves ownership of a game account without sharing any game credentials:
### What a module gets, and what it may not do
1. In game, the player runs **`[link`** and receives a one-time code.
2. On the website (Player portal, or Admin → Account for staff) they enter the code.
3. The backend confirms the code with the sidecar (`POST /link/confirm`), which permanently tags the
game account with the website user id, and mirrors the link locally in `shard_account_links`.
At boot, `app.js` scans the volume synchronously, validates each `module.json`, and calls the
module's `register(ctx, api)`:
That mirror is the authorization basis for character reads: roster/vendor/character-sheet endpoints
are **ownership-checked** so a user only sees accounts they linked. **Admins may view any
character**; players and editor/moderator staff are limited to their own linked accounts.
- **`ctx` is everything core hands over** — the database, the logger, settings, the session reader,
push, the secret box, the middleware, the rate-limit factory, the activity log, and **express
itself**. A module lives outside `server/`, so Node's resolver never reaches core's
`node_modules`; anything it must share has to be handed to it, or there would be two Expresses and
two Reacts in one process.
- **`api` is everything it may register** — routes (one prefix per tier), an extension slot fill,
notification streams, a news-announce leg, a post hook, and `onBoot`/`onShutdown`.
- **It may not reach into core's tree**, mount outside its declared prefixes, or create tables
outside its `<id>_` prefix. Each of those is checked, in the module's CI and again by the loader.
### What each audience sees
Two things are guaranteed regardless of what a module does. **A failure never takes the site down**:
the loader catches everything from `require` to `onBoot`, marks that module `startup_failed`, and
the site comes up with its routes and nav absent and the reason on the admin screen. And **no URL of
core's may move** — a module that displaced one is caught by the frozen route manifest, which is
generated from a real core with the module loaded.
| Surface | Endpoints | Who | Data |
|---|---|---|---|
| **Public** | `/api/v1/public/shard/*` (`status`, `feed`, `economy`, `online`, `idoc`, `stream`) | anyone | Shard up/down, gold-supply series, IDOC houses, a curated live feed, and **"Staff online"** — only players whose account is linked to a **staff** user (admin/editor/moderator), shown with name + map location. Linked *players* are never listed publicly; no vitals or account are exposed. |
| **Player** | `/api/v1/player/shard/*` (`link`, `accounts`, `roster/:account`, `vendors/:account`, `char/:serial`, `sales`) | logged-in player | Their own linked accounts: character rosters, character sheets, player-vendor snapshots, and recent vendor sales. |
| **Admin** | `/api/v1/admin/shard/*` (self-linking, same as player) · `/api/v1/admin/uo-link/*` (`config`, `towncrier`, `stream`) | staff / admin | Staff link their own accounts like players; **admins** additionally read *any* character's data, edit the sidecar connection config, publish/remove **town-crier** messages, and subscribe to the full event stream (incl. audit/cheat). |
### What is running right now
The sidecar URL and token are set once in **Admin → Shard**; if uo-link is not configured (or the
shard is offline), every shard surface degrades gracefully — the public page still renders, showing
the shard as offline.
```
GET /api/v1/public/modules
{ "modules": [ { "id": "uo", "name": "Ultima Online", "version": "0.3.0",
"capabilities": ["shard", "atlas", "market", …] } ] }
```
Anonymous, database-free, never site-mode gated, and **`started` modules only** — a module that is
disabled or failed is absent, exactly as its routes and its nav already are. Clients feature-detect
against it; they do not use it to decide what to load (the HTML shell injects each chunk's tag).
---
@@ -513,6 +560,8 @@ Copy `.env.example` (Compose) or `server/.env.example` (local) and fill in. **`.
| `PORT` | `3000` | server listens on `0.0.0.0:PORT` |
| `UPLOAD_DIR` | `<server>/uploads` | where post images are written (`/app/uploads`, volume-mounted, in Compose) |
| `MODULES_DIR` | `<repo>/modules` | where installed modules are scanned from (`/app/modules`, bind-mounted, in Compose) |
| `MODULES` | — | the module set this deployment runs, resolved at every start: `<id>@<version>=<install manifest URL>`, whitespace/comma separated. Already at the declared version = no network. A failure is logged and shown in Admin → Modules, never fatal. See [Modules](#modules) |
| `MODULE_SOURCE_HOSTS` | `gitea.whitlocktech.com` | **bootstrap only** — seeds the `module_source_hosts` setting on first boot; after that the setting is authoritative and is edited in Admin → Modules |
| `DB_HOST` / `DB_PORT` | `db` / `3306` | `db` in Compose; `127.0.0.1` for local dev |
| `DB_NAME` / `DB_USER` / `DB_PASSWORD` | `runic_gateway` / `runic` / — | app database credentials |
| `DB_ROOT_PASSWORD` | — | MariaDB root (Compose only) |
@@ -534,15 +583,14 @@ Copy `.env.example` (Compose) or `server/.env.example` (local) and fill in. **`.
| `CLIENT_ORIGIN` | `http://localhost:5173` | enables CORS in dev only |
| `LOG_LEVEL` / `FILE_LOG_LEVEL` | `info` / `debug` | console / file verbosity |
| `LOG_TO_FILE` / `LOG_DIR` / `LOG_FILE` | `true` / `<server>/logs` / `app.log` | log file (bind-mounted to `./logs` in Docker) |
| `ANNOUNCE_POLL_MS` | `15000` | how often the news-announcement dispatcher sweeps `announce_jobs` for due/retry legs (town crier + Discord) |
| `TOWNCRIER_DURATION_SEC` | `3600` | how long a news post's in-game town-crier message stays up (≤ `86400`) |
| `ANNOUNCE_POLL_MS` | `15000` | how often the news-announcement dispatcher sweeps `announce_jobs` for legs that are due or retrying. Which legs exist is up to what has registered one — Discord is core's; a module may add its own |
---
## Branding
Instance identity is data, not code — set via `BRAND_*` env vars, so one prebuilt
image can run as any shard. With none set, everything renders as **Runic Gateway**.
image can run as any community. With none set, everything renders as **Runic Gateway**.
| Var | What |
|---|---|

9
client/src/App.jsx
View File

@@ -41,6 +41,7 @@ import AuthProvidersAdmin from './routes/admin/views/AuthProvidersAdmin.jsx'
import UsersAdmin from './routes/admin/views/UsersAdmin.jsx'
import UserDetail from './routes/admin/views/UserDetail.jsx'
import InvitesAdmin from './routes/admin/views/InvitesAdmin.jsx'
import ModulesAdmin from './routes/admin/views/ModulesAdmin.jsx'
import AccountAdmin from './routes/admin/views/AccountAdmin.jsx'
import Moderation from './routes/admin/views/Moderation.jsx'
import ModerationUser from './routes/admin/views/ModerationUser.jsx'
@@ -61,8 +62,8 @@ export default function App() {
<AuthProvider>
<SiteProvider>
{/* Inside the auth and site contexts, because a feature provider is a
hook that may well read either — the shard one does, indirectly, by
asking an endpoint whose answer depends on the session. Outside the
hook that may well read either — a live-status one does, indirectly,
by asking an endpoint whose answer depends on the session. Outside the
routes, so the nav in every layout is filtered by the same gate and
the provider hooks are called once for the whole app rather than
once per screen. */}
@@ -169,6 +170,10 @@ export default function App() {
<Route path="users" element={<UsersAdmin />} />
<Route path="users/:id" element={<UserDetail />} />
<Route path="invites" element={<InvitesAdmin />} />
{/* Core's own screen, and it has to be: it is how a module reaches
the volume in the first place. Declared here with the rest of
core's routes, above the module-supplied ones below. */}
<Route path="modules" element={<ModulesAdmin />} />
<Route path="account" element={<AccountAdmin />} />
{/* Installed modules' admin pages, at /admin/<id>/…, already inside
RequireAuth + AdminLayout. A module cannot supply its own auth

204
client/src/api/client.js
View File

@@ -48,9 +48,9 @@ function safeParse(text) {
// on a non-2xx — and nothing above them: a module owns the paths it calls,
// because it owns the routes at the other end.
//
// The `api` object below stays core's own binding surface. Its `atlas` and
// `shard` namespaces are module bindings that only still live here because
// Phase 3 has not moved them yet.
// The `api` object below is core's own binding surface and nothing else: every
// namespace in it belongs to a route core still serves. A module binds its own
// paths in its own chunk, against this primitive.
// `BASE` goes with it: a module that needs an EventSource URL cannot go through
// `req` (fetch-only) and must not hardcode `/api/v1`, which is core's choice of
// mount point and not a promise it has made.
@@ -141,110 +141,6 @@ export const api = {
pagePreview: (id, token) => req(`/public/pages/${id}/preview/${token}`),
contact: (payload) => req('/public/contact', { method: 'POST', body: payload }),
// ----- shard live data (uo-link) -----
// Token-free, same-origin reads backed by the ingested feed + a cached live
// character round-trip. shardStreamUrl is the SSE endpoint for useShardFeed.
shard: {
status: () => req('/public/shard/status'),
feed: (opts = {}) => {
const qs = new URLSearchParams()
if (opts.kind) qs.set('kind', opts.kind)
if (opts.limit) qs.set('limit', opts.limit)
const s = qs.toString()
return req(`/public/shard/feed${withQs(s)}`)
},
economy: (limit) => {
const q = limit ? `limit=${limit}` : ''
return req(`/public/shard/economy${withQs(q)}`)
},
online: () => req('/public/shard/online'),
idoc: () => req('/public/shard/idoc'),
champs: () => req('/public/shard/champs'),
// Protocol 2.0 boards.
guilds: () => req('/public/shard/guilds'),
governors: () => req('/public/shard/governors'),
governorHistory: (city, limit) => {
const q = limit ? `limit=${limit}` : ''
return req(`/public/shard/governors/${encodeURIComponent(city)}/history${withQs(q)}`)
},
presence: () => req('/public/shard/presence'),
houses: () => req('/public/shard/houses'),
// Protocol 3.0: the shard's published ruleset. Resolves to null when the
// shard has never published one — a real answer, not an error.
ruleset: () => req('/public/shard/ruleset'),
// Protocol 3.0: points/loyalty leaderboards, one board per point system.
// `board` 404s for a system the shard has never published.
points: () => req('/public/shard/points'),
pointsBoard: (system) => req(`/public/shard/points/${encodeURIComponent(system)}`),
// Protocol 3.0: the player-vendor marketplace. Rate-limited server-side, so
// the page debounces its search box rather than firing per keystroke.
market: (opts = {}) => {
const qs = new URLSearchParams()
if (opts.q) qs.set('q', opts.q)
if (opts.minPrice != null && opts.minPrice !== '') qs.set('minPrice', opts.minPrice)
if (opts.maxPrice != null && opts.maxPrice !== '') qs.set('maxPrice', opts.maxPrice)
if (opts.itemId != null && opts.itemId !== '') qs.set('itemId', opts.itemId)
if (opts.map) qs.set('map', opts.map)
if (opts.region) qs.set('region', opts.region)
if (opts.sort) qs.set('sort', opts.sort)
if (opts.limit) qs.set('limit', opts.limit)
if (opts.offset) qs.set('offset', opts.offset)
return req(`/public/shard/market${withQs(qs.toString())}`)
},
marketMeta: () => req('/public/shard/market/meta'),
marketVendor: (serial, opts = {}) => {
const qs = new URLSearchParams()
if (opts.limit) qs.set('limit', opts.limit)
if (opts.offset) qs.set('offset', opts.offset)
return req(`/public/shard/market/vendors/${encodeURIComponent(serial)}${withQs(qs.toString())}`)
},
// Which shard surfaces this caller may reach, plus the audience rung they
// resolved to. Drives nav so we never render a link that would 403.
features: () => req('/public/shard/features'),
},
// ----- spawn atlas (Protocol 3.0 Part C) -----
// Static shard CONTENT, parsed from the shard's own ServUO tree — deliberately
// not under /shard, because nothing here depends on the sidecar and the pages
// stay populated while the shard is offline.
atlas: {
creatures: (opts = {}) => {
const qs = new URLSearchParams()
if (opts.q) qs.set('q', opts.q)
if (opts.facet) qs.set('facet', opts.facet)
if (opts.limit) qs.set('limit', opts.limit)
if (opts.offset) qs.set('offset', opts.offset)
return req(`/public/atlas/creatures${withQs(qs.toString())}`)
},
creature: (slug, opts = {}) => {
const qs = new URLSearchParams()
if (opts.facet) qs.set('facet', opts.facet)
if (opts.points) qs.set('points', opts.points)
return req(`/public/atlas/creatures/${encodeURIComponent(slug)}${withQs(qs.toString())}`)
},
regions: (opts = {}) => {
const qs = new URLSearchParams()
if (opts.facet) qs.set('facet', opts.facet)
if (opts.q) qs.set('q', opts.q)
return req(`/public/atlas/regions${withQs(qs.toString())}`)
},
landmarks: (opts = {}) => {
const qs = new URLSearchParams()
if (opts.facet) qs.set('facet', opts.facet)
if (opts.q) qs.set('q', opts.q)
return req(`/public/atlas/landmarks${withQs(qs.toString())}`)
},
// The CONFIGURED altar roster, not the live board — see shard.champs() for
// "which spawn is on level 3 right now".
champions: (facet) => req(`/public/atlas/champions${withQs(facet ? `facet=${encodeURIComponent(facet)}` : '')}`),
meta: () => req('/public/atlas/meta'),
},
// Full paths (incl. /api/v1) for the browser EventSource — the req() wrapper is
// fetch-only, so SSE subscribers build the URL from here. The admin stream
// carries every kind (incl. audit/cheat) and needs the staff session cookie.
shardStreamUrl: `${BASE}/public/shard/stream`,
adminShardStreamUrl: `${BASE}/admin/uo-link/stream`,
// ----- admin -----
admin: {
dashboard: () => req('/admin/dashboard'),
@@ -336,21 +232,20 @@ export const api = {
createInvite: (email, role, sendEmail = true) =>
req('/admin/invites', { method: 'POST', body: { email, role, sendEmail } }),
revokeInvite: (id) => req(`/admin/invites/${id}`, { method: 'DELETE' }),
// A single user's shard (uo-link) footprint, scoped to their linked accounts.
// accounts/sales/houses/online are user-scoped endpoints; roster/vendors/char
// reuse the admin-bypass /admin/shard/* endpoints (which already read any
// account) so the shared GameAccounts component works unchanged.
userShard: (id) => ({
accounts: () => req(`/admin/users/${id}/shard/accounts`),
roster: (account) => req(`/admin/shard/roster/${encodeURIComponent(account)}`),
vendors: (account) => req(`/admin/shard/vendors/${encodeURIComponent(account)}`),
char: (serial) => req(`/admin/shard/char/${encodeURIComponent(serial)}`),
sales: () => req(`/admin/users/${id}/shard/sales`),
houses: () => req(`/admin/users/${id}/shard/houses`),
online: () => req(`/admin/users/${id}/shard/online`),
standing: () => req(`/admin/users/${id}/shard/standing`),
unlink: (account) => req(`/admin/users/${id}/shard/link/${encodeURIComponent(account)}`, { method: 'DELETE' }),
}),
// Installed modules (MODULE_SYSTEM.md §2.7.2). `uninstallModule`'s purge flag
// is a query parameter rather than a body because it hangs off a DELETE, and
// it is spelled out at the call site rather than defaulted, so the
// destructive branch is never the one you get by forgetting an argument.
listModules: () => req('/admin/modules'),
installModule: (url) => req('/admin/modules', { method: 'POST', body: { url } }),
enableModule: (id) => req(`/admin/modules/${encodeURIComponent(id)}/enable`, { method: 'POST' }),
disableModule: (id) => req(`/admin/modules/${encodeURIComponent(id)}/disable`, { method: 'POST' }),
uninstallModule: (id, { purge } = {}) =>
req(`/admin/modules/${encodeURIComponent(id)}${purge ? '?purge=true' : ''}`, { method: 'DELETE' }),
purgeModule: (id) => req(`/admin/modules/${encodeURIComponent(id)}/purge`, { method: 'POST' }),
setModuleSources: (hosts) => req('/admin/modules/sources', { method: 'PUT', body: { hosts } }),
restartServer: () => req('/admin/modules/restart', { method: 'POST' }),
// ----- moderation dashboard (admin + moderator) -----
modSummary: () => req('/admin/moderation/stats/summary'),
@@ -423,19 +318,6 @@ export const api = {
linkedIdentities: () => req('/admin/account/identities'),
unlinkIdentity: (provider) => req(`/admin/account/identities/${provider}`, { method: 'DELETE' }),
// ----- game account linking (self-service, staff) -----
shard: {
link: (code) => req('/admin/shard/link', { method: 'POST', body: { code } }),
accounts: () => req('/admin/shard/accounts'),
roster: (account) => req(`/admin/shard/roster/${encodeURIComponent(account)}`),
vendors: (account) => req(`/admin/shard/vendors/${encodeURIComponent(account)}`),
char: (serial) => req(`/admin/shard/char/${encodeURIComponent(serial)}`),
sales: () => req('/admin/shard/sales'),
houses: () => req('/admin/shard/houses'), // full registry (admin/moderator)
createAccount: (account, password) =>
req('/admin/shard/account', { method: 'POST', body: { account, password } }),
},
// ----- auth providers / SSO config (admin only) -----
listAuthProviders: () => req('/admin/auth/providers'),
createAuthProvider: (data) => req('/admin/auth/providers', { method: 'POST', body: data }),
@@ -446,45 +328,6 @@ export const api = {
getDiscordBotConfig: () => req('/admin/discord-bot/config'),
saveDiscordBotConfig: (data) => req('/admin/discord-bot/config', { method: 'PUT', body: data }),
// ----- uo-link sidecar control (admin only) -----
getUoLinkConfig: () => req('/admin/uo-link/config'),
saveUoLinkConfig: (data) => req('/admin/uo-link/config', { method: 'PUT', body: data }),
postTownCrier: (data) => req('/admin/uo-link/towncrier', { method: 'POST', body: data }),
deleteTownCrier: (id) => req(`/admin/uo-link/towncrier/${encodeURIComponent(id)}`, { method: 'DELETE' }),
// Per-feature shard visibility: who may see which shard surface, and which
// sensitive fields within it. Admin only — it decides what ANONYMOUS
// visitors get. acct/webId are admin-only always and the API rejects any
// attempt to configure them.
getShardVisibility: () => req('/admin/shard/visibility'),
saveShardVisibility: (features) =>
req('/admin/shard/visibility', { method: 'PUT', body: { features } }),
// ----- spawn atlas operation (admin only) -----
// The atlas re-derives itself from the ServUO tree on every boot; these are
// for applying a map change without a restart, and for the approve/reject
// decision on a refresh that would remove a facet.
atlas: {
status: () => req('/admin/shard/atlas'),
import: (force = false) => req('/admin/shard/atlas/import', { method: 'POST', body: { force } }),
approve: () => req('/admin/shard/atlas/approve', { method: 'POST', body: {} }),
reject: () => req('/admin/shard/atlas/reject', { method: 'POST', body: {} }),
setPath: (path) => req('/admin/shard/atlas/path', { method: 'PUT', body: { path } }),
},
// ----- in-game staff operations: write plane + support queue (admin/moderator) -----
// `actor` is stamped server-side from the session — never sent from here.
shardOps: {
kick: (data) => req('/admin/shard/kick', { method: 'POST', body: data }),
ban: (data) => req('/admin/shard/ban', { method: 'POST', body: data }),
unban: (account) => req('/admin/shard/unban', { method: 'POST', body: { account } }),
broadcast: (data) => req('/admin/shard/broadcast', { method: 'POST', body: data }),
pages: () => req('/admin/shard/pages'),
respondPage: (id, data) =>
req(`/admin/shard/pages/${encodeURIComponent(id)}/respond`, { method: 'POST', body: data }),
closePage: (id) => req(`/admin/shard/pages/${encodeURIComponent(id)}/close`, { method: 'POST' }),
audit: (limit) => req(`/admin/shard/audit${limit ? `?limit=${limit}` : ''}`),
},
// ----- Email delivery / Gmail OAuth2 (admin only) -----
getEmailConfig: () => req('/admin/email/config'),
saveEmailConfig: (data) => req('/admin/email/config', { method: 'PUT', body: data }),
@@ -508,19 +351,6 @@ export const api = {
linkedIdentities: () => req('/player/account/identities'),
unlinkIdentity: (provider) => req(`/player/account/identities/${provider}`, { method: 'DELETE' }),
// ----- game account linking (uo-link) -----
shard: {
link: (code) => req('/player/shard/link', { method: 'POST', body: { code } }),
accounts: () => req('/player/shard/accounts'),
roster: (account) => req(`/player/shard/roster/${encodeURIComponent(account)}`),
vendors: (account) => req(`/player/shard/vendors/${encodeURIComponent(account)}`),
char: (serial) => req(`/player/shard/char/${encodeURIComponent(serial)}`),
sales: () => req('/player/shard/sales'),
houses: () => req('/player/shard/houses'), // the caller's own houses
createAccount: (account, password) =>
req('/player/shard/account', { method: 'POST', body: { account, password } }),
},
// ----- moderation appeals (self-service) -----
getMyAppeals: () => req('/player/appeals'),
getEligibleAppeals: () => req('/player/appeals/eligible'),

2
client/src/components/MaintenanceGate.jsx
View File

@@ -2,7 +2,7 @@ import { useAuth } from '../contexts/AuthContext.jsx'
import { useSite } from '../contexts/SiteContext.jsx'
import Maintenance from '../routes/public/Maintenance.jsx'
// Wraps the public site. When the shard is in maintenance, visitors see the
// Wraps the public site. When the site is in maintenance, visitors see the
// coming-soon page; a logged-in admin sees the real site (live preview).
export default function MaintenanceGate({ children }) {
const { mode, loading } = useSite()

2
client/src/components/SiteFooter.jsx
View File

@@ -40,7 +40,7 @@ export default function SiteFooter() {
</span>
</div>
<div className="site-footer-info">
<span>{siteTitle} is an independent private shard project.</span>
<span>{siteTitle} is an independent, privately-run game server.</span>
<span style={{ color: 'var(--dim)', fontSize: '0.84rem' }}>
<a href={`mailto:${contactEmail}`} style={{ color: 'var(--accent)', textDecoration: 'none' }}>
{contactEmail}

4
client/src/components/SiteHeader.jsx
View File

@@ -56,8 +56,8 @@ export default function SiteHeader() {
// their own (THEMING_AND_NAV.md §7). Two things about the order here:
//
// • the override merge runs FIRST and the feature filter after it, so the
// filter stays the boundary — an override cannot un-hide a shard surface
// this viewer may not see, whatever it says. `pruneNav` applies the same
// filter stays the boundary — an override cannot un-hide a surface this
// viewer may not see, whatever it says. `pruneNav` applies the same
// check inside a section and drops one it leaves empty, so a dropdown
// never opens onto nothing;
// • with no stored row this is the coded NAV, in code order, so an

9
client/src/lib/heroLayout.js
View File

@@ -67,6 +67,11 @@ export function parseLayout(str) {
// The current hardcoded hero as a HeroLayout, so the page is unchanged until
// staff publish their own. Font sizes use the existing clamp() strings so the
// default stays responsive (editor-created text uses px).
//
// The copy is deliberately game-neutral, and deliberately still copy: this is
// also the starting point the hero editor loads, so an instance that wants to
// name its game says so there, once, and the result is stored — rather than core
// shipping one game's words for every instance to overwrite in source.
export function defaultLayout(teaser, name = 'Runic Gateway') {
return {
version: 1,
@@ -84,9 +89,9 @@ export function defaultLayout(teaser, name = 'Runic Gateway') {
align: 'center',
width: 760,
lines: [
{ text: 'Private shard project', tag: 'span', fontSize: '0.74rem', color: '#c2d2e6', weight: 700, letterSpacing: '0.22em', transform: 'uppercase', font: 'sans' },
{ text: 'Private game server', tag: 'span', fontSize: '0.74rem', color: '#c2d2e6', weight: 700, letterSpacing: '0.22em', transform: 'uppercase', font: 'sans' },
{ text: name, tag: 'h1', fontSize: 'clamp(3rem,8.5vw,5.75rem)', color: 'var(--head)', weight: 600, letterSpacing: '0.02em', lineHeight: 1, font: 'display', marginTop: 14 },
{ text: 'A private Ultima Online world in progress', tag: 'p', fontSize: '1.32rem', color: '#dbe2ea', italic: true, marginTop: 22 },
{ text: 'A private world in progress', tag: 'p', fontSize: '1.32rem', color: '#dbe2ea', italic: true, marginTop: 22 },
{ text: teaser, tag: 'div', html: true, fontSize: '1.06rem', color: '#c4cdd8', maxWidth: 600, marginTop: 22 },
],
},

252
client/src/lib/moduleAdmin.js Normal file
View File

@@ -0,0 +1,252 @@
// What an admin should be told about one installed module, and what they may do
// to it — derived, not spelled out at each button.
//
// Phase 4, slice 2 of docs/website/MODULE_SYSTEM.md §2.7.2. Plain JS rather than
// a hook or a chunk of JSX, for the same reason `lib/adminNav.js` is: the test
// runner here has no DOM, and this is the part of the Modules screen that is
// actually worth testing.
//
// **The screen has four sources of truth and they are allowed to disagree**
// (MODULE_SYSTEM.md §2.4, and slice 3 for the fourth):
//
// state what the DATABASE row records — what the operator decided, and
// what the last boot ended up doing
// liveState what the LOADER has mounted in this process and is answering with
// onVolume whether there is still a directory there at all
// declared what this container's MODULES variable asks for — the only one of
// the four that no button on this screen can change
//
// Picking one and rendering it would be simpler and would lie. The case that
// makes this concrete is the one decision 3 creates on purpose: an operator
// disables a module (its onShutdown runs, its routes 404) and then enables it
// again. The row says `enabled`; the loader still says `disabled`, because
// there is no `onBoot` re-dispatch and nothing can start it before a restart.
// It is neither running nor off, and the honest thing to show is "enabled —
// restart to start it".
/**
* The one-line status of a module, and whether that status is waiting on a
* restart.
*
* Ordering matters here. The checks run most-alarming first, so a module whose
* directory has been deleted is described that way rather than by whatever its
* row happens to still say.
*
* @param {object} m a row from GET /admin/modules
* @returns {{ label: string, tone: 'ok'|'warn'|'bad'|'idle', pending: boolean, detail: string }}
*/
export function statusOf(m) {
// Declared by the environment and not there at all: no row, no directory,
// nothing mounted. Every other branch below reads one of those three, so
// without this the screen would describe a module it has never had as though
// a row had gone stale — and the one thing the operator needs, the reason
// resolution failed, would be nowhere.
if (m.declared && !m.onVolume && m.state === null) {
return {
label: 'Declared, not installed',
tone: 'bad',
pending: false,
detail: m.declaredError
? `MODULES asks for v${m.declaredVersion}; the last start could not install it: ${m.declaredError}`
: `MODULES asks for v${m.declaredVersion}. It will be installed when the server next starts.`,
}
}
// Gone from the volume, but still known. Either a hand-deleted directory (the
// boot reconcile marks that `startup_failed`) or an uninstall waiting for its
// restart. Both are "there is nothing to run here".
if (!m.onVolume) {
return {
label: m.state === 'disabled' ? 'Uninstalled' : 'Missing from the volume',
tone: m.state === 'disabled' ? 'idle' : 'bad',
pending: m.liveState !== null,
detail: m.state === 'disabled'
? 'The files are gone. Its data was kept, and reinstalling brings it back.'
: 'A row exists but there is no module directory. Reinstall it, or uninstall to clear the row.',
}
}
// **Installed since this process booted**, and this check has to come before
// the failure one. `liveState` is the loader's record, and the loader scans
// the volume once at require time — so a module that is on the volume NOW and
// has no live record was put there after the scan. Anything the row still says
// about it therefore predates the install and is stale by definition.
//
// Found by the §7.7 browser smoke, and no unit test here had modelled it:
// installing over a row left `startup_failed` by the previous boot rendered
// "Failed at the require stage: module directory not present on the volume"
// one second after the file had been written to the volume — and, because that
// branch is not pending, suppressed the restart banner the install had just
// told the operator to use.
if (m.liveState === null) {
return {
label: 'Restart to start',
tone: 'warn',
pending: true,
detail: 'Installed. It mounts when the server next starts.',
}
}
if (m.state === 'startup_failed' || m.liveState === 'startup_failed') {
return {
label: 'Failed to start',
tone: 'bad',
pending: false,
detail: m.failureReason
? `Failed at the ${m.failureStage || 'unknown'} stage: ${m.failureReason}`
: 'It failed to start and recorded no reason.',
}
}
if (m.state === 'disabled') {
return {
label: 'Disabled',
tone: 'idle',
pending: false,
detail: 'Stopped and switched off. Its routes answer 404 and it stays off across restarts.',
}
}
// The row has been switched on but the loader has not started it — the
// decision-3 case: disable ran its onShutdown, and nothing can start it again
// before a restart.
if (m.liveState !== 'started') {
return {
label: 'Restart to start',
tone: 'warn',
pending: true,
detail: m.liveState === 'disabled'
? 'Enabled, but still stopped in the running server — it cannot be restarted in place.'
: 'Enabled. It mounts when the server next starts.',
}
}
// Running, but not the version that is installed. An upgrade writes new files
// and a new row while the old code stays loaded, so the row's `version` is a
// promise about the next boot rather than a description of this one — and
// "Running v2.0.0" beside a process serving v1.0.0 is the same lie as the
// stale-failure one above, in a different place.
if (m.liveVersion && m.liveVersion !== m.version) {
return {
label: 'Restart to finish upgrading',
tone: 'warn',
pending: true,
detail: `v${m.version} is installed; v${m.liveVersion} is still running.`,
}
}
return {
label: 'Running',
tone: 'ok',
pending: false,
detail: 'Mounted and serving.',
}
}
/**
* What the environment's declaration means for this module, as one sentence — or
* null if nothing declares it.
*
* Kept out of `statusOf` on purpose. A module can be running perfectly while its
* declared upgrade is failing, and collapsing both into one label would have to
* pick which of the two is "the" status. This is a second line, beside the first.
*
* The sentence an operator most needs is the uninstall one: MODULES owns what is
* on the volume and the row owns whether it runs, so uninstalling a declared
* module puts its files back at the next start and leaves it switched off. Files
* reappearing unexplained is exactly the kind of thing that gets debugged for an
* afternoon.
*
* @param {object} m a row from GET /admin/modules
* @returns {{ text: string, tone: 'warn'|'idle' }|null}
*/
export function declarationNoteFor(m) {
if (!m.declared) return null
if (m.declaredError) {
return {
text: `MODULES asks for v${m.declaredVersion} and the last start could not install it: ${m.declaredError}`,
tone: 'warn',
}
}
if (!m.onVolume) {
return {
text:
`MODULES declares v${m.declaredVersion}, so its files come back when the server next starts`
+ (m.state === 'disabled' ? ' — switched off, until you enable it.' : '.'),
tone: 'warn',
}
}
return { text: `Declared by this deployment's MODULES variable at v${m.declaredVersion}.`, tone: 'idle' }
}
/**
* Which actions are offered for a module, and why the others are not.
*
* Returned as a map of `{ shown, reason }` rather than a list of shown actions,
* so a disabled button can say what would make it available. Every rule here
* mirrors one the server enforces — this is presentation, never the boundary.
*
* @param {object} m a row from GET /admin/modules
*/
export function actionsFor(m) {
const running = m.liveState === 'started'
const disabled = m.state === 'disabled'
return {
// Only offered while something is actually running: disabling a module that
// is already stopped has nothing to stop and no guard to flip.
disable: {
shown: !disabled && m.onVolume,
reason: disabled ? 'Already disabled.' : 'Nothing is running to stop.',
},
enable: {
shown: disabled && m.onVolume,
reason: 'Only a disabled module can be enabled.',
},
uninstall: {
shown: m.onVolume,
reason: 'There are no files left to remove.',
},
// The server refuses a standalone purge unless the module is disabled, so
// the button says so rather than offering a click that 409s.
purge: {
shown: m.onVolume && m.canPurge,
enabled: disabled,
reason: !m.canPurge
? 'This module ships no purge.sql, so its data cannot be deleted.'
: 'Disable it first, so nothing is serving out of the tables being dropped.',
},
// A row with no directory is the one thing an uninstall cannot tidy through
// the normal path — offer clearing it instead.
forget: {
shown: !m.onVolume && m.state !== null,
reason: 'The module is still installed.',
},
running,
}
}
/**
* Does anything on this list need a restart before it matches what is running?
*
* Drives the one banner at the top of the screen rather than a badge per row:
* the restart is a property of the SERVER, not of a module, and offering it
* five times would suggest otherwise.
*/
export const needsRestart = (modules) => modules.some((m) => statusOf(m).pending)
/**
* Split a hosts string the way the server will.
*
* Duplicated from `install.parseHosts` deliberately — it is four lines, and the
* alternative is an API round trip to preview what the field is going to mean.
* The server remains the one that decides; this only shows the operator how
* their typing will be read.
*/
export function parseHosts(value) {
return String(value || '')
.split(/[,\s]+/)
.map((h) => h.trim().toLowerCase())
.filter(Boolean)
}

11
client/src/lib/navOverrides.js
View File

@@ -204,7 +204,7 @@ export function buildNavRows(baseNav, overrides) {
//
// The base side is restricted to the rows the editor is actually holding: §8.1
// filters the palette to what this admin can themselves see, and an item that
// their role or a shard feature kept off the screen is not a reorder.
// their role or a module's feature gate kept off the screen is not a reorder.
function orderMatchesBase(groups, baseNav) {
const flatten = (gs) => gs.flatMap((g) => g.items.map((i) => `${g.title ?? ''}::${i.to}`))
const base = isGrouped(baseNav)
@@ -408,9 +408,10 @@ export function buildPublicNav(baseNav, overrides, { keepHidden = false } = {})
* Apply the caller's visibility gate — and drop a section it leaves empty.
*
* Kept here rather than in SiteHeader because the empty-dropdown case is the one
* with real correctness risk: a section whose every entry is hidden by shard
* visibility must not render as a menu that opens onto nothing. The predicate
* stays the caller's, so this module still knows nothing about shard features.
* with real correctness risk: a section whose every entry is hidden by a
* module's visibility rules must not render as a menu that opens onto nothing.
* The predicate stays the caller's, so this module still knows nothing about
* what any module gates on.
*
* Added links carry no gate, so they are always visible — see the note above.
*
@@ -489,7 +490,7 @@ export function buildPublicNavOverrides(tree, baseNav, stored = null) {
}
// Carry through an entry for a coded item this admin's palette never showed
// them (shard-feature gated), so their save does not silently reset it.
// them (feature-gated by its module), so their save does not silently reset it.
const { items: storedItems } = unwrapPublic(stored)
for (const [to, entry] of Object.entries(storedItems)) {
if (!shown.has(to) && baseLabels.has(to) && entry && typeof entry === 'object') items[to] = entry

5
client/src/routes/admin/AdminLayout.jsx
View File

@@ -45,6 +45,7 @@ const IconPulse = () => <Icon><path d="M3 12h3l2 6 4-14 2 8h7" /></Icon>
const IconUser = () => <Icon><circle cx="12" cy="8" r="4" /><path d="M4 21a8 8 0 0 1 16 0" /></Icon>
const IconNav = () => <Icon><path d="M4 6h16M4 12h16M4 18h10" /><circle cx="18" cy="18" r="2.5" /></Icon>
const IconPalette = () => <Icon><path d="M12 3a9 9 0 1 0 0 18 2 2 0 0 0 1.6-3.2 2 2 0 0 1 1.6-3.2H18a3 3 0 0 0 3-3 9 9 0 0 0-9-8.6z" /><circle cx="7.5" cy="11.5" r="1" /><circle cx="10.5" cy="7.5" r="1" /><circle cx="15" cy="8.5" r="1" /></Icon>
const IconModules = () => <Icon><path d="M12 3l8 4.5-8 4.5-8-4.5z" /><path d="M4 12l8 4.5 8-4.5" /><path d="M4 16.5L12 21l8-4.5" /></Icon>
// Nav is grouped into collapsible categories. A group with no `title` renders
// its items ungrouped (Dashboard at top, Account at bottom). Each item's `roles`
@@ -83,6 +84,10 @@ export const NAV = [
{ to: '/admin/users', label: 'Users', icon: IconUsers, roles: ['admin'] },
{ to: '/admin/invites', label: 'Invites', icon: IconUsers, roles: ['admin'] },
{ to: '/admin/settings', label: 'Settings', icon: IconGear, roles: ['admin'] },
// Admin-only, matching the server: every route under /admin/modules
// re-gates to `admin` on top of the group's staff gate, because installing
// a module runs its code in this process.
{ to: '/admin/modules', label: 'Modules', icon: IconModules, roles: ['admin'] },
{ to: '/admin/appearance', label: 'Appearance', icon: IconPalette, roles: ['admin'] },
{ to: '/admin/navigation', label: 'Navigation', icon: IconNav, roles: ['admin'] },
{ to: '/admin/hero', label: 'Hero Editor', icon: IconHero, roles: ['admin'] },

424
client/src/routes/admin/views/ModulesAdmin.jsx Normal file
View File

@@ -0,0 +1,424 @@
import { useCallback, useEffect, useState } from 'react'
import { Loading, ErrorState } from '../../../components/PageState.jsx'
import { dateTime } from '../../../lib/format.js'
import { statusOf, actionsFor, declarationNoteFor, needsRestart, parseHosts } from '../../../lib/moduleAdmin.js'
import { api } from '../../../api/client.js'
// Installed modules: install from a release URL, enable, disable, uninstall,
// purge, and restart the server so the changes take effect.
//
// Phase 4, slice 2 of docs/website/MODULE_SYSTEM.md §2.7.2. Everything that
// decides what a row SAYS and which buttons it offers lives in
// lib/moduleAdmin.js, which is plain JS and has tests; this file renders it.
//
// Two things about this screen are unlike the rest of the admin panel and are
// deliberate:
//
// 1. **Restart is a banner, not a per-row button.** A restart is a property of
// the server, not of a module. Offering it on five rows would suggest
// otherwise, and an operator who installed three modules should restart
// once.
// 2. **Disable is the only action that takes effect immediately.** Everything
// else is "true after the next boot", because the loader reads the volume
// at require time (§1.12). The buttons say which they are.
const TONE = {
ok: '#7fd0a4',
warn: 'var(--accent)',
bad: '#d98b84',
idle: 'var(--muted)',
}
const DANGER = { color: '#d98b84', borderColor: '#5b2020' }
function Pill({ tone, children }) {
return (
<span
className="badge"
style={{ color: TONE[tone] || 'var(--muted)', borderColor: 'var(--line)', background: 'var(--panel-flat)' }}
>
{children}
</span>
)
}
// ── Install ────────────────────────────────────────────────────────────────
function InstallForm({ sourceHosts, onInstalled }) {
const [url, setUrl] = useState('')
const [busy, setBusy] = useState(false)
const [error, setError] = useState('')
const [result, setResult] = useState(null)
async function submit(e) {
e.preventDefault()
setError('')
setResult(null)
if (!url.trim()) return setError('Paste the URL of a release install manifest.')
setBusy(true)
try {
const res = await api.admin.installModule(url.trim())
setResult(res)
setUrl('')
await onInstalled()
} catch (err) {
// The server's message is written to be read by whoever pasted the URL —
// which host was refused, which hash did not match, what the archive
// contained. Replacing it with something friendlier would throw away the
// only part that helps.
setError(err.message || 'Could not install that module.')
} finally {
setBusy(false)
}
}
return (
<div className="panel" style={{ padding: 22, marginBottom: 22 }}>
<div className="field-label" style={{ marginBottom: 10 }}>Install a module</div>
<form onSubmit={submit} style={{ display: 'flex', gap: 12, alignItems: 'flex-end', flexWrap: 'wrap' }}>
<label style={{ flex: '1 1 380px' }}>
<span className="field-label">Release install-manifest URL</span>
<input
type="url"
value={url}
onChange={(e) => setUrl(e.target.value)}
className="input"
placeholder="https://gitea.example.com/org/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json"
/>
</label>
<button type="submit" disabled={busy} className="btn btn-primary btn-sq">
{busy ? 'Installing…' : 'Install'}
</button>
</form>
<p className="sans" style={{ margin: '12px 0 0', fontSize: '0.8rem', color: 'var(--muted)' }}>
The bundle is downloaded, checked against the <code>sha256</code> its release published, and
unpacked onto the modules volume. It starts serving after a restart.{' '}
{sourceHosts.length === 0
? 'No source hosts are allowed yet — add one below before installing.'
: `Allowed hosts: ${sourceHosts.join(', ')}.`}
</p>
{error && <p className="sans" style={{ margin: '12px 0 0', color: TONE.bad, fontSize: '0.85rem' }}>{error}</p>}
{result && (
<p className="sans" style={{ margin: '12px 0 0', color: TONE.ok, fontSize: '0.85rem' }}>
{result.replaced ? 'Upgraded' : 'Installed'} {result.module?.name} v{result.module?.version}. Restart to load it.
</p>
)}
</div>
)
}
// ── The restart banner ─────────────────────────────────────────────────────
function RestartBanner({ onDone }) {
const [busy, setBusy] = useState(false)
const [sent, setSent] = useState(false)
async function restart() {
// Said plainly, because it is true and because the failure mode is bad: a
// deployment with no supervisor does not come back on its own.
const ok = window.confirm(
'Restart the server now?\n\n'
+ 'The site will be briefly unavailable. It comes back on its own only if something is '
+ 'supervising the process — the shipped Docker Compose file does. If you are running '
+ '`npm start` by hand, you will have to start it again yourself.',
)
if (!ok) return
setBusy(true)
try {
await api.admin.restartServer()
setSent(true)
// Nothing is coming back on this connection: the process is exiting. Give
// the supervisor a moment and then reload, which is what the operator was
// about to do anyway.
setTimeout(() => { if (onDone) onDone() }, 6000)
} catch {
// A failed request here is expected as often as not — the process can win
// the race and drop the socket before the response lands.
setSent(true)
setTimeout(() => { if (onDone) onDone() }, 6000)
} finally {
setBusy(false)
}
}
return (
<div className="panel" style={{ padding: 18, marginBottom: 22, borderColor: 'var(--accent)' }}>
<div style={{ display: 'flex', gap: 14, alignItems: 'center', flexWrap: 'wrap' }}>
<div style={{ flex: '1 1 320px' }}>
<div className="field-label" style={{ marginBottom: 4 }}>Restart needed</div>
<p className="sans" style={{ margin: 0, fontSize: '0.84rem', color: 'var(--muted)' }}>
{sent
? 'Restarting. This page will reload once the server is back.'
: 'Modules are read from disk when the server starts, so an install, an uninstall or a re-enable only takes effect after a restart.'}
</p>
</div>
<button type="button" className="btn btn-primary btn-sq" disabled={busy || sent} onClick={restart}>
{sent ? 'Restarting…' : 'Restart the server'}
</button>
</div>
</div>
)
}
// ── The source allowlist ───────────────────────────────────────────────────
function SourceHosts({ hosts, onSaved }) {
const [value, setValue] = useState(hosts.join(', '))
const [busy, setBusy] = useState(false)
const [error, setError] = useState('')
const [saved, setSaved] = useState(false)
useEffect(() => { setValue(hosts.join(', ')) }, [hosts])
async function save(e) {
e.preventDefault()
setError('')
setSaved(false)
setBusy(true)
try {
await api.admin.setModuleSources(value)
setSaved(true)
await onSaved()
} catch (err) {
setError(err.message || 'Could not save the allowlist.')
} finally {
setBusy(false)
}
}
const parsed = parseHosts(value)
return (
<div className="panel" style={{ padding: 22, marginTop: 22 }}>
<div className="field-label" style={{ marginBottom: 10 }}>Where modules may be installed from</div>
<form onSubmit={save} style={{ display: 'flex', gap: 12, alignItems: 'flex-end', flexWrap: 'wrap' }}>
<label style={{ flex: '1 1 380px' }}>
<span className="field-label">Allowed hosts</span>
<input
type="text"
value={value}
onChange={(e) => setValue(e.target.value)}
className="input"
placeholder="gitea.example.com, releases.example.org"
/>
</label>
<button type="submit" disabled={busy} className="btn btn-sq">{busy ? 'Saving…' : 'Save'}</button>
</form>
<p className="sans" style={{ margin: '12px 0 0', fontSize: '0.8rem', color: 'var(--muted)' }}>
Installing a module runs its code inside this server, so only hosts listed here may be
installed from — over HTTPS, and re-checked on every redirect. An empty list blocks all
installs.{' '}
{parsed.length > 0 && <>Will be saved as: <code>{parsed.join(', ')}</code>.</>}
</p>
{error && <p className="sans" style={{ margin: '10px 0 0', color: TONE.bad, fontSize: '0.85rem' }}>{error}</p>}
{saved && !error && <p className="sans" style={{ margin: '10px 0 0', color: TONE.ok, fontSize: '0.85rem' }}>Saved.</p>}
</div>
)
}
// ── One module ─────────────────────────────────────────────────────────────
function ModuleRow({ m, onChanged, onError }) {
const [busy, setBusy] = useState('')
const status = statusOf(m)
const actions = actionsFor(m)
const note = declarationNoteFor(m)
async function run(name, fn) {
setBusy(name)
try {
await fn()
await onChanged()
} catch (err) {
onError(err.message || `Could not ${name} ${m.id}.`)
} finally {
setBusy('')
}
}
const disable = () => run('disable', () => api.admin.disableModule(m.id))
const enable = () => run('enable', () => api.admin.enableModule(m.id))
function uninstall() {
// The purge choice is made HERE and only here, because purge.sql lives
// inside the directory the uninstall is about to delete — there is no
// "purge it later" (§2.7.2 decision 5). Two prompts rather than one, so
// "delete the data too" is never something you agree to by reflex.
if (!window.confirm(`Uninstall ${m.name}?\n\nIts files are removed. Its data is kept unless you ask otherwise next.`)) return
let purge = false
if (m.canPurge) {
purge = window.confirm(
`Also permanently delete ${m.name}'s data?\n\n`
+ 'This drops its tables and cannot be undone. This is the only moment it can be offered — '
+ 'the script that does it is part of the files being removed.\n\n'
+ 'OK deletes the data. Cancel keeps it.',
)
}
return run('uninstall', () => api.admin.uninstallModule(m.id, { purge }))
}
function purge() {
if (!window.confirm(`Permanently delete ${m.name}'s data?\n\nThis drops its tables and cannot be undone.`)) return
return run('purge', () => api.admin.purgeModule(m.id))
}
const forget = () => run('forget', () => api.admin.uninstallModule(m.id))
return (
<tr>
<td className="adm-td" style={{ color: 'var(--text)' }}>
<div style={{ fontWeight: 600 }}>{m.name}</div>
<div className="dim" style={{ fontSize: '0.76rem' }}>
{/* A declared module that has never installed has no version to show —
only the one MODULES asks for, which the status column carries. */}
{m.id}{m.version ? ` · v${m.version}` : ''}
</div>
{m.capabilities?.length > 0 && (
<div className="dim" style={{ fontSize: '0.72rem', marginTop: 2 }}>{m.capabilities.join(' · ')}</div>
)}
</td>
<td className="adm-td">
<Pill tone={status.tone}>{status.label}</Pill>
<div className="dim" style={{ fontSize: '0.74rem', marginTop: 4, maxWidth: 380 }}>{status.detail}</div>
{/* The environment's declaration, on its own line: a module can be
running fine while its declared upgrade is failing, and the status
above can only be one of those two things. */}
{note && (
<div
style={{
fontSize: '0.74rem',
marginTop: 4,
maxWidth: 380,
color: note.tone === 'warn' ? TONE.warn : 'var(--muted)',
}}
>
{note.text}
</div>
)}
</td>
<td className="adm-td dim" style={{ fontSize: '0.74rem' }}>
{/* Provenance. Null for a directory placed on the volume by hand, which
stays a supported install — so it is shown as that, not as missing.
A declared module can also reach a boot with no provenance: the
no-op path never fetches, so it has no sha256 to record and no
reason to write a row. Saying "by hand" there would be the one
wrong answer. */}
{m.source ? (
<>
<div style={{ wordBreak: 'break-all', maxWidth: 260 }}>{m.source}</div>
{m.sha256 && <div style={{ marginTop: 2 }}>sha256 {m.sha256.slice(0, 12)}…</div>}
</>
) : (
<span>{m.declared ? 'From the declared module set' : 'Placed on the volume by hand'}</span>
)}
{m.installedAt && <div style={{ marginTop: 2 }}>{dateTime(m.installedAt)}</div>}
</td>
<td className="adm-td" style={{ textAlign: 'right', whiteSpace: 'nowrap' }}>
<div style={{ display: 'inline-flex', gap: 6, flexWrap: 'wrap', justifyContent: 'flex-end' }}>
{actions.disable.shown && (
<button type="button" className="pill" style={{ fontSize: '0.72rem' }} disabled={Boolean(busy)} onClick={disable}>
{busy === 'disable' ? 'Stopping…' : 'Disable'}
</button>
)}
{actions.enable.shown && (
<button type="button" className="pill" style={{ fontSize: '0.72rem' }} disabled={Boolean(busy)} onClick={enable}>
{busy === 'enable' ? 'Enabling…' : 'Enable'}
</button>
)}
{actions.purge.shown && (
<button
type="button"
className="pill"
style={{ fontSize: '0.72rem', ...DANGER, opacity: actions.purge.enabled ? 1 : 0.45 }}
disabled={Boolean(busy) || !actions.purge.enabled}
title={actions.purge.enabled ? undefined : actions.purge.reason}
onClick={purge}
>
{busy === 'purge' ? 'Purging…' : 'Purge data'}
</button>
)}
{actions.uninstall.shown && (
<button type="button" className="pill" style={{ fontSize: '0.72rem', ...DANGER }} disabled={Boolean(busy)} onClick={uninstall}>
{busy === 'uninstall' ? 'Removing…' : 'Uninstall'}
</button>
)}
{actions.forget.shown && (
<button type="button" className="pill" style={{ fontSize: '0.72rem' }} disabled={Boolean(busy)} onClick={forget}>
{busy === 'forget' ? 'Clearing…' : 'Clear the row'}
</button>
)}
</div>
</td>
</tr>
)
}
// ── The screen ─────────────────────────────────────────────────────────────
export default function ModulesAdmin() {
const [data, setData] = useState(null)
const [error, setError] = useState('')
const [actionError, setActionError] = useState('')
const load = useCallback(async () => {
setError('')
try {
setData(await api.admin.listModules())
} catch {
setError('Could not load installed modules.')
}
}, [])
useEffect(() => { load() }, [load])
if (error) return <ErrorState message={error} />
if (!data) return <Loading />
const modules = data.modules || []
const sourceHosts = data.sourceHosts || []
return (
<section>
{needsRestart(modules) && <RestartBanner onDone={() => window.location.reload()} />}
<InstallForm sourceHosts={sourceHosts} onInstalled={load} />
{actionError && (
<p className="sans" style={{ margin: '0 0 14px', color: TONE.bad, fontSize: '0.85rem' }}>{actionError}</p>
)}
<div className="panel-flat">
<table className="adm-table">
<thead>
<tr>
<th className="adm-th">Module</th>
<th className="adm-th">Status</th>
<th className="adm-th">Installed from</th>
<th className="adm-th" />
</tr>
</thead>
<tbody>
{modules.length === 0 && (
<tr>
<td className="adm-td" colSpan={4} style={{ color: 'var(--muted)' }}>
No modules installed. Paste a release install-manifest URL above to add one.
</td>
</tr>
)}
{modules.map((m) => (
<ModuleRow key={m.id} m={m} onChanged={load} onError={setActionError} />
))}
</tbody>
</table>
</div>
<SourceHosts hosts={sourceHosts} onSaved={load} />
</section>
)
}

11
client/src/routes/admin/views/NavEditor.jsx
View File

@@ -35,7 +35,8 @@ import { NAV as PLAYER_NAV } from '../../player/PlayerPortalLayout.jsx'
// Three things shape the screen:
//
// • The palette is filtered to the editing admin's OWN visible rows (§8.1) —
// the base array run through their role and this shard's feature gates. An
// the base array run through their role and the feature gates of whichever
// module registered each row (client/src/modules/featureGate.js). An
// admin cannot drag in, and so can never accidentally advertise, something
// they cannot see themselves. An override on a row they cannot see is
// carried through their save untouched rather than quietly reset.
@@ -458,8 +459,8 @@ export default function NavEditor() {
<section style={{ maxWidth: 860, display: 'flex', flexDirection: 'column', gap: 22 }}>
<p className="sans dim" style={{ margin: 0, fontSize: '0.82rem', lineHeight: 1.7 }}>
Rename, reorder and hide the entries in each navigation. The pages themselves are unchanged — this
only decides what is advertised, and it can never show anyone a link their role or this shard&rsquo;s
visibility settings would hide.
only decides what is advertised, and it can never show anyone a link their role, or the visibility
settings of an installed module, would hide.
</p>
{/* ── Tabs ───────────────────────────────────────────────── */}
@@ -552,8 +553,8 @@ export default function NavEditor() {
</div>
<p className="sans dim" style={{ margin: 0, fontSize: '0.76rem', lineHeight: 1.7 }}>
Only entries you can see yourself are listed. Anything hidden from you by your role or by Shard
Visibility keeps whatever it was already set to.
Only entries you can see yourself are listed. Anything hidden from you by your role, or by a
module&rsquo;s visibility settings, keeps whatever it was already set to.
</p>
</section>
)

14
client/src/routes/public/About.jsx
View File

@@ -10,19 +10,19 @@ export default function About() {
<PageHeader eyebrow="About" title={`About ${siteShortName}`} />
<div className="prose">
<p>
{siteShortName} is an independent, privately-run Ultima Online shard built by a small group of long-time players.
It is not affiliated with or endorsed by the owners of Ultima Online — it is a labor of love for the old
worlds and the friendships made in them.
{siteShortName} is an independent, privately-run game server built by a small group of long-time
players. It is not affiliated with or endorsed by the owners of the game it runs — it is a labor of
love for the old worlds and the friendships made in them.
</p>
<p>
Our aim is a calm, hand-tended world: a contested wilderness worth exploring, safe towns worth living in,
and systems that reward curiosity over grind. We are building slowly and in the open, sharing news,
Our aim is a calm, hand-tended world: somewhere worth exploring, somewhere worth living in, and
systems that reward curiosity over grind. We are building slowly and in the open, sharing news,
screenshots, and guides as the world comes online.
</p>
<h2>What to expect</h2>
<ul>
<li>A hybrid ruleset — safe towns, a dangerous wild.</li>
<li>Custom crafting, housing, and exploration content.</li>
<li>A world that is hand-tended rather than left to run itself.</li>
<li>Custom content, and changes explained before they land.</li>
<li>A small, friendly population and an active wiki.</li>
</ul>
</div>

2
client/src/routes/public/Screenshots.jsx
View File

@@ -16,7 +16,7 @@ export default function Screenshots() {
<PageHeader
eyebrow="Gallery"
title="Gameplay Pictures"
lead="Glimpses of towns, dungeons, events, and daily life on the shard."
lead="Glimpses of the world, its events, and daily life on the server."
/>
{loading && <Loading />}
{error && <ErrorState message="Could not load the gallery right now." />}

4
client/src/routes/public/Status.jsx
View File

@@ -19,7 +19,7 @@ export default function Status() {
return (
<PublicLayout section="website">
<div className="shell-narrow page-body">
<PageHeader eyebrow="Live" title="Shard Status" />
<PageHeader eyebrow="Live" title="Site Status" />
{loading && <Loading />}
{error && <ErrorState message="Could not load status right now." />}
@@ -58,7 +58,7 @@ export default function Status() {
{isLive ? 'Live — the gates are open' : 'Maintenance — building in progress'}
</strong>
<span style={{ color: isLive ? '#a9cdb8' : '#cdbf9a', fontSize: '0.98rem' }}>
{statusMessage || (isLive ? 'The shard is online.' : 'The gates are closed while we shape the world. Public login is not open yet.')}
{statusMessage || (isLive ? 'The site is open.' : 'The gates are closed while we shape the world. Public login is not open yet.')}
</span>
</div>
</section>

6
client/src/routes/public/Website.jsx
View File

@@ -4,12 +4,12 @@ import PageHeader from '../../components/PageHeader.jsx'
import { useSite } from '../../contexts/SiteContext.jsx'
const CARDS = [
{ kicker: 'Gallery', title: 'Gameplay Pictures', body: 'Screenshots from towns, dungeons, events, and daily life on the shard.', to: '/site/screenshots' },
{ kicker: 'Updates', title: 'Development News', body: 'Progress notes, shard milestones, and public announcements.', to: '/site/news' },
{ kicker: 'Gallery', title: 'Gameplay Pictures', body: 'Screenshots of the world, its events, and daily life on the server.', to: '/site/screenshots' },
{ kicker: 'Updates', title: 'Development News', body: 'Progress notes, project milestones, and public announcements.', to: '/site/news' },
{ kicker: 'Community', title: 'Five on Friday', body: 'Weekly questions, small previews, and notes from the team.', to: '/site/five-on-friday' },
{ kicker: 'Long-form', title: 'Monthly Newsletter', body: 'Fuller summaries for players who want the whole picture.', to: '/site/newsletter' },
{ kicker: 'Reference', title: 'Wiki', body: 'Guides and reference pages for the game world.', to: '/wiki' },
{ kicker: 'Live', title: 'Shard Status', body: 'Launch state, test windows, and known issues.', to: '/site/status' },
{ kicker: 'Live', title: 'Site Status', body: 'Launch state, test windows, and known issues.', to: '/site/status' },
]
export default function Website() {

2
client/src/routes/wiki/Wiki.jsx
View File

@@ -97,7 +97,7 @@ export default function Wiki() {
center
eyebrow="Knowledge base"
title={`${siteShortName} Wiki`}
lead="A calm starting point for shard guides, the world and its lore, gameplay systems, and community rules."
lead="A calm starting point for guides, the world and its lore, gameplay systems, and community rules."
/>
<SearchBox initial={activeQ || ''} onSubmit={runSearch} />

74
client/test/apiClient.test.js
View File

@@ -128,10 +128,10 @@ test('wiki() with no options sends no query string at all', async () => {
assert.equal(calls[0].url, '/api/v1/public/wiki')
})
test('path params are URL-encoded (a token/city with unsafe characters is escaped)', async () => {
test('path params are URL-encoded (a token with unsafe characters is escaped)', async () => {
willReply({ body: {} })
await api.shard.governorHistory('Serpent’s Hold', 5)
assert.match(calls[0].url, /\/governors\/Serpent%E2%80%99s%20Hold\/history\?limit=5/)
await api.getInvite('a b/c?d')
assert.equal(calls[0].url, '/api/v1/auth/invite/a%20b%2Fc%3Fd')
})
test('DELETE self-service session revoke encodes the id and uses the DELETE method', async () => {
@@ -141,43 +141,47 @@ test('DELETE self-service session revoke encodes the id and uses the DELETE meth
assert.match(calls[0].url, /\/auth\/me\/sessions\/a%20b%2Fc$/)
})
// ── spawn atlas (Protocol 3.0 Part C) ───────────────────────────────────
// The atlas lives at /public/atlas, NOT under /public/shard: it is static shard
// content parsed from the shard's own files, so it must not look sidecar-backed.
// Asserted here because the split is a design decision, not an accident of
// spelling.
test('atlas reads hit /public/atlas, not /public/shard', async () => {
willReply({ body: { creatures: [] } })
await api.atlas.creatures()
assert.equal(calls[0].url, '/api/v1/public/atlas/creatures')
})
// ── admin: installed modules (MODULE_SYSTEM.md §2.7.2) ──────────────────
//
// These pin the URLs, because the destructive one differs from the harmless one
// by a query parameter and nothing else.
test('atlas.creatures() sends only the filters that are set', async () => {
willReply({ body: { creatures: [] } })
await api.atlas.creatures({ q: 'lizard man', facet: 'Ter Mur', limit: 25 })
const url = new URL(calls[0].url, 'http://x')
assert.equal(url.pathname, '/api/v1/public/atlas/creatures')
assert.equal(url.searchParams.get('q'), 'lizard man')
assert.equal(url.searchParams.get('facet'), 'Ter Mur')
assert.equal(url.searchParams.get('limit'), '25')
assert.equal(url.searchParams.get('offset'), null) // 0 is not sent
})
test('atlas.creature() encodes the slug and carries the facet filter through', async () => {
test('module actions hit the right paths and methods', async () => {
const cases = [
[() => api.admin.listModules(), 'GET', '/api/v1/admin/modules'],
[() => api.admin.installModule('https://x/y.json'), 'POST', '/api/v1/admin/modules'],
[() => api.admin.enableModule('uo'), 'POST', '/api/v1/admin/modules/uo/enable'],
[() => api.admin.disableModule('uo'), 'POST', '/api/v1/admin/modules/uo/disable'],
[() => api.admin.purgeModule('uo'), 'POST', '/api/v1/admin/modules/uo/purge'],
[() => api.admin.setModuleSources('a.com'), 'PUT', '/api/v1/admin/modules/sources'],
[() => api.admin.restartServer(), 'POST', '/api/v1/admin/modules/restart'],
]
for (const [call, method, url] of cases) {
calls = []
willReply({ body: {} })
await api.atlas.creature('lizardman/rare', { facet: 'Felucca' })
assert.match(calls[0].url, /\/public\/atlas\/creatures\/lizardman%2Frare\?facet=Felucca$/)
await call()
assert.equal(calls[0].url, url)
assert.equal(calls[0].opts.method || 'GET', method)
}
})
test('admin atlas actions use the right methods and bodies', async () => {
test('uninstall only asks for a purge when it is told to', async () => {
// The difference between "remove the module" and "remove the module and drop
// every table it owns" is this query parameter, so a default that leaned the
// wrong way would be irreversible.
willReply({ body: {} })
await api.admin.atlas.import(true)
assert.equal(calls[0].url, '/api/v1/admin/shard/atlas/import')
assert.equal(calls[0].opts.method, 'POST')
assert.equal(calls[0].opts.body, JSON.stringify({ force: true }))
await api.admin.uninstallModule('uo')
assert.equal(calls[0].url, '/api/v1/admin/modules/uo')
assert.equal(calls[0].opts.method, 'DELETE')
calls = []
willReply({ body: {} })
await api.admin.atlas.setPath('/srv/servuo')
assert.equal(calls[1].opts.method, 'PUT')
assert.equal(calls[1].opts.body, JSON.stringify({ path: '/srv/servuo' }))
await api.admin.uninstallModule('uo', { purge: true })
assert.equal(calls[0].url, '/api/v1/admin/modules/uo?purge=true')
})
test('a module id is URL-encoded on the way into the path', async () => {
willReply({ body: {} })
await api.admin.disableModule('a b/c')
assert.equal(calls[0].url, '/api/v1/admin/modules/a%20b%2Fc/disable')
})

281
client/test/moduleAdmin.test.js Normal file
View File

@@ -0,0 +1,281 @@
import { test } from 'node:test'
import assert from 'node:assert/strict'
import { statusOf, actionsFor, declarationNoteFor, needsRestart, parseHosts } from '../src/lib/moduleAdmin.js'
// lib/moduleAdmin.js — what the Modules screen says about a module and what it
// lets you do to it. Phase 4, slice 2 of MODULE_SYSTEM.md §2.7.2.
//
// This is the part of the screen worth testing, and it is plain JS so this
// runner can reach it (there is no DOM here). What it encodes is §2.4's rule
// that the row, the loader and the volume are three sources of truth which are
// ALLOWED to disagree — so most of these cases are combinations that a screen
// picking one source would render as a lie.
/** A module as GET /admin/modules returns it, with the running case as default. */
const mod = (over = {}) => ({
id: 'uo',
name: 'Ultima Online',
version: '1.0.0',
state: 'started',
failureStage: null,
failureReason: null,
source: 'https://gitea.example.com/x/uo.json',
sha256: 'a'.repeat(64),
installedAt: null,
startedAt: null,
liveState: 'started',
liveVersion: '1.0.0',
capabilities: [],
onVolume: true,
canPurge: true,
declared: false,
declaredVersion: null,
declaredError: null,
...over,
})
// ── statusOf ───────────────────────────────────────────────────────────────
test('a mounted, started module is Running and needs nothing', () => {
const s = statusOf(mod())
assert.equal(s.label, 'Running')
assert.equal(s.tone, 'ok')
assert.equal(s.pending, false)
})
test('enabled in the row but disabled in the loader is "Restart to start"', () => {
// THE case decision 3 creates on purpose: disable ran the module's onShutdown,
// then the operator enabled it again. The row says enabled; nothing can start
// it before a restart. Showing either "Running" or "Disabled" would be false.
const s = statusOf(mod({ state: 'enabled', liveState: 'disabled' }))
assert.equal(s.label, 'Restart to start')
assert.equal(s.tone, 'warn')
assert.equal(s.pending, true)
assert.match(s.detail, /cannot be restarted in place/)
})
test('freshly installed and never booted into is also "Restart to start"', () => {
const s = statusOf(mod({ state: 'installed', liveState: null }))
assert.equal(s.label, 'Restart to start')
assert.equal(s.pending, true)
assert.match(s.detail, /mounts when the server next starts/)
})
test('a disabled module is Disabled, and that is not pending anything', () => {
// Disable takes effect immediately — it is the one action that does — so there
// is nothing for a restart banner to be about.
const s = statusOf(mod({ state: 'disabled', liveState: 'disabled' }))
assert.equal(s.label, 'Disabled')
assert.equal(s.pending, false)
})
test('a fresh install over a failed row is pending, not failed', () => {
// THE defect the §7.7 browser smoke found, and one no test here had modelled.
// Installing over a row the previous boot left `startup_failed` rendered
// "Failed at the require stage: module directory not present on the volume" a
// second after the files had been written — and suppressed the restart banner
// the install had just told the operator to use.
//
// `liveState === null` with the module on the volume means the loader's scan
// never saw it, so it arrived after boot and everything the row says predates
// it.
const s = statusOf(mod({
state: 'startup_failed',
liveState: null,
failureStage: 'require',
failureReason: 'module directory not present on the volume',
}))
assert.equal(s.label, 'Restart to start')
assert.equal(s.pending, true)
assert.doesNotMatch(s.detail, /not present on the volume/, 'the stale reason must not survive the install')
})
test('the restart banner appears for that install', () => {
// The second half of the same defect: the banner is driven by `pending`, so a
// row wrongly classified as failed silently removed the only way to act on it.
assert.equal(needsRestart([mod({ state: 'startup_failed', liveState: null })]), true)
})
test('an upgrade that has not been restarted into says so', () => {
// Same class as the stale-failure defect: the row is a promise about the next
// boot, not a description of this one. Reporting "Running v2.0.0" while the
// process is serving v1.0.0 would hide the only action that fixes it.
const s = statusOf(mod({ version: '2.0.0', liveVersion: '1.0.0' }))
assert.equal(s.label, 'Restart to finish upgrading')
assert.equal(s.pending, true)
assert.match(s.detail, /v2\.0\.0 is installed; v1\.0\.0 is still running/)
})
test('reinstalling the SAME version is not an upgrade in progress', () => {
assert.equal(statusOf(mod({ version: '1.0.0', liveVersion: '1.0.0' })).label, 'Running')
})
test('a failed module reports the stage and the reason it recorded', () => {
const s = statusOf(mod({
state: 'startup_failed',
liveState: 'startup_failed',
failureStage: 'schema',
failureReason: "Unknown column 'x' in 'field list'",
}))
assert.equal(s.label, 'Failed to start')
assert.equal(s.tone, 'bad')
assert.match(s.detail, /schema stage/)
assert.match(s.detail, /Unknown column/)
})
test('a failure with no recorded reason says so rather than showing a blank', () => {
const s = statusOf(mod({ state: 'startup_failed', liveState: 'startup_failed' }))
assert.match(s.detail, /recorded no reason/)
})
test('a row whose directory is gone by hand is bad, not merely disabled', () => {
// The boot reconcile marks this `startup_failed` because a row claiming to be
// enabled for a module that is not on the volume is simply untrue.
const s = statusOf(mod({ state: 'startup_failed', liveState: null, onVolume: false, failureStage: 'require', failureReason: 'module directory not present on the volume' }))
assert.equal(s.label, 'Missing from the volume')
assert.equal(s.tone, 'bad')
})
test('an uninstalled module reads as uninstalled, and says the data was kept', () => {
// Uninstall leaves the row `disabled` and the data alone — which is the whole
// point of keeping the row, so the screen has to say it.
const s = statusOf(mod({ state: 'disabled', liveState: 'disabled', onVolume: false }))
assert.equal(s.label, 'Uninstalled')
assert.equal(s.tone, 'idle')
assert.match(s.detail, /data was kept/i)
})
test('missing-from-the-volume beats every other status', () => {
// Ordering: a module with no files is described that way whatever its row
// still claims, because there is nothing there to be running.
for (const state of ['started', 'enabled', 'installed', 'startup_failed']) {
assert.match(statusOf(mod({ state, onVolume: false })).label, /Missing from the volume/)
}
})
// ── actionsFor ─────────────────────────────────────────────────────────────
test('a running module offers disable, uninstall and a blocked purge', () => {
const a = actionsFor(mod())
assert.equal(a.disable.shown, true)
assert.equal(a.enable.shown, false)
assert.equal(a.uninstall.shown, true)
assert.equal(a.purge.shown, true)
// Shown but not clickable: the server refuses a standalone purge on anything
// that is not disabled, so offering the click would only produce a 409.
assert.equal(a.purge.enabled, false)
assert.match(a.purge.reason, /Disable it first/)
})
test('a disabled module offers enable, and purge is now live', () => {
const a = actionsFor(mod({ state: 'disabled', liveState: 'disabled' }))
assert.equal(a.enable.shown, true)
assert.equal(a.disable.shown, false)
assert.equal(a.purge.enabled, true)
})
test('a module with no purge.sql never offers purge, and says why', () => {
const a = actionsFor(mod({ state: 'disabled', liveState: 'disabled', canPurge: false }))
assert.equal(a.purge.shown, false)
assert.match(a.purge.reason, /ships no purge.sql/)
})
test('a module with no files offers only clearing the row', () => {
const a = actionsFor(mod({ state: 'disabled', liveState: null, onVolume: false }))
assert.equal(a.uninstall.shown, false)
assert.equal(a.disable.shown, false)
assert.equal(a.enable.shown, false)
assert.equal(a.purge.shown, false, 'there is no purge.sql left to run')
assert.equal(a.forget.shown, true)
})
test('a directory with no row yet is actionable, and offers nothing to forget', () => {
// A hand-placed install before its first boot: it has no row, so `state` is
// null. Its routes are already being served, so it must be disableable.
const a = actionsFor(mod({ state: null, liveState: 'started' }))
assert.equal(a.disable.shown, true)
assert.equal(a.uninstall.shown, true)
assert.equal(a.forget.shown, false)
})
// ── needsRestart ───────────────────────────────────────────────────────────
test('the restart banner is driven by the list, not by any one module', () => {
// A restart is a property of the SERVER. One pending module is enough, and
// three do not mean three restarts.
assert.equal(needsRestart([mod(), mod({ id: 'b' })]), false)
assert.equal(needsRestart([mod(), mod({ id: 'b', state: 'installed', liveState: null })]), true)
assert.equal(needsRestart([]), false)
})
test('a disabled module does not ask for a restart', () => {
// Disable is immediate; a banner here would be asking for a restart that
// would change nothing.
assert.equal(needsRestart([mod({ state: 'disabled', liveState: 'disabled' })]), false)
})
test('a failed module does not ask for a restart either', () => {
// It is retried on every boot anyway, and the operator has to fix the cause
// first — a banner would suggest restarting is the remedy.
assert.equal(needsRestart([mod({ state: 'startup_failed', liveState: 'startup_failed' })]), false)
})
// ── parseHosts ─────────────────────────────────────────────────────────────
test('parseHosts previews exactly what the server will store', () => {
assert.deepEqual(parseHosts('A.com, b.com\n c.com'), ['a.com', 'b.com', 'c.com'])
assert.deepEqual(parseHosts(' '), [])
assert.deepEqual(parseHosts(undefined), [])
})
// ── the declaration (slice 3) ──────────────────────────────────────────────
test('a declared module that has never installed says so, with the reason', () => {
// No row, no directory, nothing mounted — invisible to the other three
// sources, so without this branch the screen would describe a module it has
// never had as a row gone stale.
const s = statusOf(mod({
state: null,
liveState: null,
liveVersion: null,
version: null,
onVolume: false,
declared: true,
declaredVersion: '0.3.0',
declaredError: 'could not reach releases.example.com',
}))
assert.equal(s.label, 'Declared, not installed')
assert.equal(s.tone, 'bad')
assert.equal(s.pending, false, 'a restart will not fix an unreachable host')
assert.match(s.detail, /could not reach releases.example.com/)
})
test('a declared module waiting for its first resolution is not reported as failed', () => {
const s = statusOf(mod({ state: null, liveState: null, version: null, onVolume: false, declared: true, declaredVersion: '0.3.0' }))
assert.match(s.detail, /installed when the server next starts/)
})
test('a running module whose declared upgrade is failing is still Running', () => {
// Both facts are true at once. The status is one label, so the declaration
// gets its own line rather than overwriting it.
const m = mod({ declared: true, declaredVersion: '2.0.0', declaredError: 'sha256 did not match' })
assert.equal(statusOf(m).label, 'Running')
const note = declarationNoteFor(m)
assert.equal(note.tone, 'warn')
assert.match(note.text, /sha256 did not match/)
})
test('uninstalling a declared module is told that its files come back', () => {
// The sentence that saves an afternoon: MODULES owns what is on the volume,
// the row owns whether it runs.
const note = declarationNoteFor(mod({ state: 'disabled', liveState: null, onVolume: false, declared: true, declaredVersion: '1.0.0' }))
assert.match(note.text, /come back when the server next starts/)
assert.match(note.text, /switched off/)
})
test('an ordinary declared module gets a quiet note, and an undeclared one none', () => {
assert.equal(declarationNoteFor(mod()), null)
const note = declarationNoteFor(mod({ declared: true, declaredVersion: '1.0.0' }))
assert.equal(note.tone, 'idle')
assert.match(note.text, /MODULES/)
})

26
docker-compose.yml
View File

@@ -39,6 +39,24 @@ services:
# defaults to (<repo>/modules, and the repo is /app in the image), set
# explicitly because the bind mount below is what makes it meaningful.
MODULES_DIR: /app/modules
# WHICH modules this deployment runs (MODULE_SYSTEM.md §2.7.2 decision 4).
# One entry per module, `<id>@<version>=<install manifest URL>`, whitespace-
# or comma-separated. The container resolves this set for itself at every
# start: a module already unpacked at the declared version is left alone
# without a single network call — so a restart with the internet down comes
# up unchanged — and only a missing or different version is fetched,
# verified against the sha256 its release manifest declares, and unpacked.
# A failure is logged and surfaced in Admin → Modules; it never stops the
# site from starting.
#
# Uncomment to declare a set here, in the file this host version-controls,
# or leave it out and set MODULES in .env (env_file above) — or leave it
# unset entirely and install from the admin panel. What it declares is what
# is ON the volume, never whether a module runs: a module disabled from the
# admin panel gets its files back and stays disabled.
#
# MODULES: >-
# uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json
depends_on:
db:
condition: service_healthy
@@ -56,10 +74,12 @@ services:
# pull-only deployment without building anything. A bind mount rather than
# a named volume because placing a module directory by hand is a supported
# install — `tar -xf uo-1.0.0.tgz -C ./modules` then restart — and that has
# to be doable from the host, not through `docker cp`.
# to be doable from the host, not through `docker cp`. It is no longer the
# usual way in: declare MODULES above, or install from Admin → Modules.
#
# Read-WRITE: the admin panel's install/uninstall unpacks and removes
# directories here from inside the container.
# Read-WRITE: MODULES resolution at start, and the admin panel's
# install/uninstall, both unpack and remove directories here from inside
# the container.
#
# `modules/` is tracked (it ships a README) so the directory exists in the
# checkout with the operator's own ownership. Do not delete it — Docker

5
package.json
View File

@@ -1,7 +1,7 @@
{
"name": "runic-gateway-website",
"version": "1.0.0",
"description": "Runic Gateway — public site, wiki, and admin panel for a private Ultima Online shard",
"description": "Runic Gateway — public site, wiki, and admin panel for a private game server",
"private": true,
"scripts": {
"install-server": "npm install --prefix server",
@@ -13,7 +13,8 @@
"bot": "npm run dev --prefix bot",
"seed": "npm run seed --prefix server",
"build": "npm run build --prefix client",
"start": "npm start --prefix server"
"start": "npm start --prefix server",
"check:modules": "node scripts/checkModuleIdentifiers.js"
},
"keywords": ["express", "mariadb", "react", "vite", "jwt"],
"author": "whitlocktech",

328
scripts/checkModuleIdentifiers.js Normal file
View File

@@ -0,0 +1,328 @@
#!/usr/bin/env node
// ── §5.2 — zero module identifiers in core ─────────────────────────────────
//
// Phase 3's acceptance criterion 1, as a check rather than a review promise: no
// `shard`, `uoLink`, `cliloc`, `atlas` or `towncrier` anywhere in core's source
// (MODULE_API.md §5.2, MODULE_SYSTEM.md §2.7.1 slice 4). The extraction is only
// worth what this is worth — a boundary nothing enforces grows a hole the first
// time someone is in a hurry, and the hole looks exactly like the code that was
// there before.
//
// **It reads code, not prose, and that is the whole design.** Four things are
// checked, and each is a thing a module owns:
//
// 1. file and directory names
// 2. import and require SPECIFIERS — the path, not the file's contents
// 3. route path literals — the string handed to .get/.post/.put/.patch/
// .delete/.use
// 4. declared identifiers — function, const, class, and object property names
//
// Comments and string content in general are NOT read. Core's own English may
// legitimately say "shard": `About.jsx` did until slice 4 rewrote it, and a
// comment explaining *what moved and why* — `AcceptInvite.jsx` has one — is
// worth more than the word costs. A literal word grep would fail on both, prove
// nothing about the boundary, and teach people to phrase around it. The
// boundary this defends is structural: core must not NAME a module's files,
// import them, route to them, or declare their symbols. It may talk about them.
//
// Two things learned the hard way, both of which this file would have got wrong:
//
// • **Match on word boundaries, not substrings.** `defaultImage` contains
// "ultIma"; `atlas` is inside "atlasSomething" legitimately only when it is
// the same word. The tokeniser below splits identifiers on camelCase and
// separators and compares WHOLE words, so `shardStatus` is a hit and
// `defaultImage` is not. A substring pass flagged four innocent lines in
// this repo on its first run.
// • **Strip comments and strings with a character walk, not a regexp.** The
// module's own `checkImports.js` flagged the comments that explain what it
// catches. A comment contains quotes (`-- '' when randomised`), a string
// contains `//` (any URL), and a regexp literal contains both. Doing it in
// one pass, in order, is the only way that comes out right — and this file
// has its own test suite (`server/test/checkModuleIdentifiers.test.js`)
// because a check that silently stops checking is worse than no check.
const fs = require('fs')
const path = require('path')
const { execFileSync } = require('child_process')
const ROOT = path.resolve(__dirname, '..')
// The trees core owns. `modules/` is deliberately absent — that is where a
// module's own code lives, and it is the one place these words belong.
const TREES = [
path.join(ROOT, 'server', 'src'),
path.join(ROOT, 'server', 'scripts'),
path.join(ROOT, 'server', 'db'),
path.join(ROOT, 'client', 'src'),
]
const SKIP_DIRS = new Set(['node_modules', 'coverage', 'dist', '.git'])
const CODE = new Set(['.js', '.jsx', '.mjs', '.cjs', '.ts', '.tsx'])
// The words a module owns. Lower-cased whole words, compared against the
// tokeniser's output — so `uoLink`, `uo_link` and `uo-link` all reduce to the
// two tokens `uo` and `link`, and the pair is what is matched.
const RESERVED = new Set(['shard', 'shards', 'cliloc', 'clilocs', 'atlas', 'towncrier'])
// Sequences of tokens that are reserved together but innocent apart: "uo" and
// "link" each appear in ordinary core code ("link" especially), and only the
// pair names the sidecar.
const RESERVED_PAIRS = [['uo', 'link'], ['town', 'crier'], ['spawn', 'atlas'], ['serv', 'uo']]
// Standalone `uo` is reserved too: it is the module id, and a core file called
// `uo.js` or a route `/uo` is the boundary being crossed in the plainest way.
const RESERVED_ALONE = new Set(['uo', 'uolink', 'servuo', 'ultima'])
// ── The grandfathering exemptions ───────────────────────────────────────────
//
// Exactly three, and all three are the SAME mechanism: core's per-module legacy
// allowlists (MODULE_API.md §6.5). A table prefix, a set of stream ids and an
// announce leg all predate the module system, are stored in live rows, and are
// read by a shipped Android client — so `uo` keeps them, and keeping them means
// core holds a map whose KEY is the module id. There is no way to write that
// down without naming the module; that is what grandfathering is.
//
// Nothing else may be added here without the same kind of reason. In particular
// this is not an escape hatch for "core still needs this for now" — that is the
// state slice 4 exists to end.
//
// Each entry must MATCH something. An exemption that no longer fires is deleted
// by the check itself (`unused exemption` below), because a stale one is how an
// allowlist quietly becomes permission for whatever drifts into it later.
const EXEMPT = [
{
file: 'server/src/modules/loader.js',
name: 'uo',
kind: 'property name',
why: 'LEGACY_TABLE_PREFIXES — the grandfathered shard_/uo_link_ table prefixes (API §6.5)',
},
{
file: 'server/src/modules/registries.js',
name: 'uo',
kind: 'property name',
why: 'LEGACY_STREAM_IDS and LEGACY_LEGS — grandfathered stream ids and the towncrier leg (API §6.5)',
},
]
const isExempt = (hit) =>
EXEMPT.some((e) => e.file === hit.file && e.name === hit.name && e.kind === hit.kind)
/**
* Split a name into lower-case words: camelCase humps, and runs separated by
* `-`, `_`, `.`, `/` or digits.
*
* `shardStatus` → [shard, status]; `uo_link_config` → [uo, link, config];
* `defaultImage` → [default, image] — which is the point: the substring
* "ultIma" inside it is not a word and never appears here.
*/
function tokenize(name) {
return String(name)
.replace(/([a-z0-9])([A-Z])/g, '$1 $2')
.replace(/([A-Z]+)([A-Z][a-z])/g, '$1 $2')
.split(/[^A-Za-z]+/)
.filter(Boolean)
.map((w) => w.toLowerCase())
}
/** Does this name contain a reserved word, as a word? */
function reservedWordIn(name) {
const words = tokenize(name)
for (const w of words) {
if (RESERVED.has(w) || RESERVED_ALONE.has(w)) return w
}
for (const [a, b] of RESERVED_PAIRS) {
for (let i = 0; i < words.length - 1; i++) {
if (words[i] === a && words[i + 1] === b) return `${a}-${b}`
}
}
return null
}
/**
* Blank out comments, and MASK string/template/regexp contents, in one
* left-to-right pass.
*
* Masking rather than deleting: the checks that run afterwards need to know
* WHERE a string was (a route path literal is a string) while not reading what
* is in an arbitrary one. So a string's delimiters and length survive and its
* body becomes spaces, except that the string-literal check below re-reads the
* original text at the same offsets. Comments are replaced by spaces so every
* offset in the returned text still lines up with the input — line numbers stay
* honest without a second pass.
*/
function maskCode(src) {
const out = Array.from(src)
const blank = (from, to) => {
for (let i = from; i < to && i < out.length; i++) if (out[i] !== '\n') out[i] = ' '
}
let i = 0
while (i < src.length) {
const c = src[i]
const next = src[i + 1]
if (c === '/' && next === '/') {
let j = i
while (j < src.length && src[j] !== '\n') j++
blank(i, j)
i = j
continue
}
if (c === '/' && next === '*') {
const end = src.indexOf('*/', i + 2)
const j = end === -1 ? src.length : end + 2
blank(i, j)
i = j
continue
}
if (c === '-' && next === '-' && src[i + 2] === ' ') {
// SQL line comment; harmless in JS, where `-- ` cannot start an expression.
let j = i
while (j < src.length && src[j] !== '\n') j++
blank(i, j)
i = j
continue
}
if (c === '"' || c === "'" || c === '`') {
let j = i + 1
while (j < src.length) {
if (src[j] === '\\') { j += 2; continue }
if (src[j] === c) break
j++
}
blank(i + 1, j) // keep the quotes, blank the body
i = j + 1
continue
}
i++
}
return out.join('')
}
// ── the four checks ─────────────────────────────────────────────────────────
const SPECIFIER = /(?:require\(\s*|from\s+|import\(\s*)(['"])([^'"]+)\1/g
const ROUTE = /\.(?:get|post|put|patch|delete|use|all)\(\s*(['"`])([^'"`]*)\1/g
const DECLARED = /\b(?:function|const|let|var|class)\s+([A-Za-z_$][\w$]*)/g
const PROPERTY = /(?:^|[{,]\s*)([A-Za-z_$][\w$]*)\s*:/gm
function lineOf(src, index) {
return src.slice(0, index).split('\n').length
}
/**
* Check one file. `src` is the raw text; `masked` has comments blanked and
* string bodies blanked at the same offsets, so a regexp run over `masked`
* finds only real code — and the captured offsets index back into `src` when a
* check legitimately needs the string's content (specifiers and route paths).
*/
function checkFile(rel, src) {
const hits = []
const masked = maskCode(src)
const add = (kind, name, index) => {
const word = reservedWordIn(name)
if (word) hits.push({ file: rel, line: lineOf(src, index), kind, name, word })
}
for (const m of masked.matchAll(SPECIFIER)) {
// Read the specifier out of the ORIGINAL text: its body was masked, and a
// path is the one string whose content is structural.
const start = m.index + m[0].indexOf(m[1]) + 1
add('import specifier', src.slice(start, start + m[2].length), m.index)
}
for (const m of masked.matchAll(ROUTE)) {
const start = m.index + m[0].indexOf(m[1]) + 1
add('route path', src.slice(start, start + m[2].length), m.index)
}
for (const m of masked.matchAll(DECLARED)) add('declared identifier', m[1], m.index)
for (const m of masked.matchAll(PROPERTY)) add('property name', m[1], m.index)
return hits
}
function walk(dir, out = []) {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
if (SKIP_DIRS.has(entry.name)) continue
const full = path.join(dir, entry.name)
if (entry.isDirectory()) walk(full, out)
else out.push(full)
}
return out
}
/**
* The files core SHIPS, which is what the boundary is about — not whatever
* happens to be in a working tree.
*
* `git ls-files` rather than a walk, because an untracked local artifact is not
* core's source and must not fail anyone's build. This is not hypothetical: an
* operator-supplied `server/db/data/spawnAtlas.art.json` is gitignored, sits in
* the tree of anyone who has run the atlas import, and would otherwise report a
* file-name violation that no commit could fix. The walk stays as the fallback
* for an export with no git in it, where over-reporting is the safer failure.
*/
function sourceFiles() {
try {
const out = execFileSync('git', ['ls-files', '-z', '--cached', '--', ...TREES.map((t) => path.relative(ROOT, t))], {
cwd: ROOT,
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'ignore'],
})
const files = out.split('\0').filter(Boolean).map((f) => path.join(ROOT, f))
if (files.length) return files
} catch {
// no git, or not a checkout — fall through
}
return TREES.filter((t) => fs.existsSync(t)).flatMap((t) => walk(t))
}
function run() {
const hits = []
for (const file of sourceFiles()) {
if (!fs.existsSync(file)) continue
const rel = path.relative(ROOT, file).split(path.sep).join('/')
// 1. the name itself
const word = reservedWordIn(path.basename(file))
if (word) hits.push({ file: rel, line: 0, kind: 'file name', name: path.basename(file), word })
// 2-4. the contents, for code files only
if (!CODE.has(path.extname(file))) continue
hits.push(...checkFile(rel, fs.readFileSync(file, 'utf8')))
}
const live = hits.filter((h) => !isExempt(h))
// A grandfathering entry that matches nothing is deleted, loudly. Reported as
// a failure rather than a warning: the exemption list is the one part of this
// check that can only get weaker, so it is the part that needs the noise.
const unused = EXEMPT.filter((e) => !hits.some((h) => h.file === e.file && h.name === e.name && h.kind === e.kind))
return { hits: live, unused }
}
module.exports = { run, checkFile, maskCode, tokenize, reservedWordIn, EXEMPT }
if (require.main === module) {
const { hits, unused } = run()
if (hits.length === 0 && unused.length === 0) {
console.log('OK — core names no module identifier (MODULE_API.md §5.2).')
process.exit(0)
}
for (const e of unused) {
console.error(
`\nUnused exemption: ${e.file} "${e.name}" (${e.kind}) matches nothing any more.\n` +
` ${e.why}\n` +
' Delete it from EXEMPT in this file. A grandfathering entry that has outlived what it ' +
'grandfathered is permission with nothing attached to it.',
)
}
if (hits.length === 0) process.exit(1)
console.error(
`\nCore names ${hits.length} module identifier${hits.length === 1 ? '' : 's'} ` +
'(MODULE_API.md §5.2). Each of these belongs to an installed module:\n',
)
for (const h of hits) {
console.error(` ${h.file}:${h.line} ${h.kind} "${h.name}" — reserved word "${h.word}"`)
}
console.error(
'\nCore may TALK about a module in English; it may not name its files, import them, ' +
'route to them, or declare its symbols. If one of these is core\'s own and the word is ' +
'a coincidence, the fix is to rename it — the reserved list is short and deliberate.\n',
)
process.exit(1)
}

49
server/.env.example
View File

@@ -99,13 +99,15 @@ CLIENT_ORIGIN=http://localhost:5173
BOT_INTERNAL_URL=http://localhost:4100
BOT_INTERNAL_KEY=dev-only-change-me-bot-key
# News announcement pipeline (published news post -> in-game town crier + Discord
# #news). The dispatcher is an in-process poller; these tune it. Links in the
# News announcement pipeline (published news post -> every registered delivery
# leg). The dispatcher is an in-process poller; this tunes it. Links in the
# announcements use APP_BASE_URL (set above), so set that in production too.
# ANNOUNCE_POLL_MS how often the dispatcher sweeps for due/retry legs
# TOWNCRIER_DURATION_SEC how long the in-game town-crier message stays up (<= 86400)
#
# Which legs exist depends on what has registered one: Discord (#news) is core's,
# and an installed module may add its own. A module's leg brings its own settings
# with it -- module-uo's in-game town crier reads TOWNCRIER_DURATION_SEC, which is
# documented in that module rather than here, because core has no town crier.
ANNOUNCE_POLL_MS=15000
TOWNCRIER_DURATION_SEC=3600
# Push notifications (M7) — opt-in fan-out to the Android app via a self-hosted
# ntfy UnifiedPush relay (docs/android/PLAN.md §11). The publisher POSTs
@@ -122,8 +124,43 @@ TOWNCRIER_DURATION_SEC=3600
# NTFY_PUBLISH_TOKEN Optional bearer token for backend->ntfy publishes (off by default).
# Leave NTFY_BASE_URL unset in local dev to allow any public HTTPS endpoint
# (private/loopback hosts are always rejected). Without NTFY_PUBLIC_URL /
# NTFY_ALLOWED_ORIGINS the app shows push as unavailable for the shard.
# NTFY_ALLOWED_ORIGINS the app shows push as unavailable for this instance.
# NTFY_BASE_URL=https://ntfy.example.com
# NTFY_PUBLIC_URL=https://ntfy.example.com
# NTFY_ALLOWED_ORIGINS=https://ntfy.example.com
# NTFY_PUBLISH_TOKEN=
# Modules (MODULE_SYSTEM.md §2.5) — where installable modules live, and where
# they may be installed from.
# MODULES_DIR Directory the loader scans at require time. Defaults to
# <repo>/modules; docker-compose.yml sets it to /app/modules,
# which is the bind mount that makes it meaningful.
# MODULE_SOURCE_HOSTS BOOTSTRAP ONLY. Comma-separated hostnames the admin panel
# may install a module from, seeded into the `module_source_hosts`
# setting the first time the site boots without one. From then
# on the SETTING is authoritative and is edited in
# Admin → Modules — changing this variable on an existing
# deployment does nothing, deliberately, so a redeploy cannot
# silently undo an operator's choice. Installs are https-only
# and an empty list forbids all of them.
# MODULES The module set this deployment RUNS, resolved at every
# start (§2.7.2 decision 4). One entry per module, separated
# by whitespace or commas:
#
# <id>@<version>=<install manifest URL>
#
# A module already unpacked at the declared version is left
# alone WITHOUT touching the network, so a restart with no
# route to the internet comes up unchanged; only a missing or
# different version is fetched, through the same verify-and-
# unpack path (and the same host allowlist) the admin panel
# uses. A version that cannot be fetched is logged and shown
# in Admin → Modules — it never stops the site from starting.
#
# This variable owns what is ON the volume, not what runs: a
# module disabled from the admin panel stays disabled even
# though its files are put back. Leave it unset to manage
# modules entirely from the admin panel.
# MODULES_DIR=/app/modules
# MODULE_SOURCE_HOSTS=gitea.whitlocktech.com
# MODULES=uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json

19
server/db/schema.sql
View File

@@ -802,20 +802,6 @@ ALTER TABLE announce_jobs
DROP INDEX IF EXISTS idx_announce_due,
DROP INDEX IF EXISTS idx_announce_due_discord;
-- ── Spawn atlas (Protocol 3.0 Part C) ───────────────────────────────────────
-- Static shard CONTENT, not live shard state: what spawns where, which regions
-- and landmarks exist, and which champion altars are configured. Nothing here
-- comes from the sidecar — it is imported from a committed artifact built off a
-- ServUO tree by `npm run atlas:build` (see docs/website/SPAWN_ATLAS.md), so
-- these tables stay populated whether the shard is up or not.
--
-- Every table is import-owned: `npm run atlas:import` TRUNCATEs and reloads them
-- in one transaction. Nothing else may write here, and nothing else may hold a
-- foreign key to them. No FKs at all, consistent with every other shard_* table.
-- One row per spawnable type, aggregated across the world. `total` is the sum of
-- each type's own MX across every point that spawns it (how many exist at once);
-- Installed modules (module system, docs/website/MODULE_SYSTEM.md §2.4). One row
-- per module the operator has installed onto the modules volume, keyed by the
-- module id from its module.json — the same id that names the directory, the URL
@@ -888,10 +874,6 @@ ALTER TABLE users ADD COLUMN IF NOT EXISTS last_login_ip VARCHAR(45) NULL;
-- Player self-registration mode: disabled | password | sso | both. Default off,
-- so the system behaves exactly as today until an admin opts in.
INSERT IGNORE INTO settings (`key`, value) VALUES ('player_registration', 'disabled');
-- Game-account signup (Protocol 2.0 hybrid mode): whether a signed-in website user
-- may provision a linked game account from the site. Default off; the shard's own
-- signup mode still has the final say (a 'game'-mode shard refuses regardless).
INSERT IGNORE INTO settings (`key`, value) VALUES ('game_account_signup', 'disabled');
ALTER TABLE wiki_pages ADD COLUMN IF NOT EXISTS excerpt VARCHAR(400) NULL;
ALTER TABLE wiki_pages ADD COLUMN IF NOT EXISTS category_id INT NULL;
@@ -921,4 +903,3 @@ ALTER TABLE mobile_refresh_tokens ADD COLUMN IF NOT EXISTS last_used_at DATETIME
-- trust token. A boolean only — the token is returned over that app→server call
-- and never persisted here (only its sha256 lands in trusted_devices).
ALTER TABLE mobile_auth_sessions ADD COLUMN IF NOT EXISTS trust_device TINYINT(1) NOT NULL DEFAULT 0;
INSERT IGNORE INTO settings (`key`, value) VALUES ('uo_link_protocol_3_migrated', '1');

21
server/db/seed.js
View File

@@ -23,15 +23,30 @@ const DEFAULT_SETTINGS = {
site_title: brand.name,
// Android App Links opt-in — off until an admin enables it (docs/android/APP_LINKS.md).
mobile_app_links_enabled: 'false',
// Hosts a module may be installed from (MODULE_SYSTEM.md §2.7.2 decision 6).
//
// The environment BOOTSTRAPS this and does not own it: seedDefault is an
// INSERT IGNORE, so the variable supplies a sane default on a fresh install
// and never reaches back in to overwrite what an admin later chose in
// Admin → Modules. Changing MODULE_SOURCE_HOSTS on an existing deployment is
// therefore a no-op, which is the intended behaviour and not an oversight.
//
// An empty stored value forbids every install rather than allowing every host
// — the safe direction for a setting someone might blank by accident.
module_source_hosts: process.env.MODULE_SOURCE_HOSTS || 'gitea.whitlocktech.com',
}
// Starter wiki sections (editable later via the admin panel).
//
// The SLUGS are deliberately untouched by the de-UO pass: `seedDefault*` only
// inserts a row that is not already there, so renaming one adds a duplicate page
// to every existing install rather than renaming anything.
// [slug, title, description, sort_order]
const WIKI_CATEGORIES = [
['guides', 'Guides', 'Getting started and how-to guides.', 10],
['world', 'World & Lore', `Regions, maps, and the story of ${brand.shortName}.`, 20],
['gameplay', 'Systems & Gameplay', 'Mechanics, items, monsters, and crafting.', 30],
['community', 'Community & Rules', 'Player conduct and shard policies.', 40],
['community', 'Community & Rules', 'Player conduct and server policies.', 40],
]
// The 8 starter pages, each mapped to a section. [slug, title, body, categorySlug]
@@ -39,11 +54,11 @@ const WIKI_PAGES = [
['new-player-guide', 'New Player Guide', 'First steps, basic survival, and early goals.', 'guides'],
['maps-atlas', 'Maps & Atlas', 'Regions, towns, routes, and travel notes.', 'world'],
['lore', 'Lore', 'Stories, places, factions, and mysteries.', 'world'],
['systems', 'Server Systems', 'Shard mechanics and custom features.', 'gameplay'],
['systems', 'Server Systems', 'Server mechanics and custom features.', 'gameplay'],
['items', 'Items & Rewards', 'Equipment, treasures, rewards, and curiosities.', 'gameplay'],
['monsters', 'Monsters & Encounters', 'Creatures, bosses, spawns, and dangers.', 'gameplay'],
['crafting', 'Crafting', 'Professions, materials, recipes, and tools.', 'gameplay'],
['rules', 'Rules', 'Player conduct, shard expectations, and policies.', 'community'],
['rules', 'Rules', 'Player conduct, server expectations, and policies.', 'community'],
]
async function seedDefaults() {

68
server/package-lock.json generated
View File

@@ -27,6 +27,7 @@
"sanitize-html": "^2.17.5",
"speakeasy": "^2.0.0",
"swagger-ui-express": "^5.0.1",
"tar": "^7.5.22",
"ws": "^8.21.0"
},
"devDependencies": {
@@ -34,6 +35,18 @@
"swagger-autogen": "^2.23.7"
}
},
"node_modules/@isaacs/fs-minipass": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/@isaacs/fs-minipass/-/fs-minipass-4.0.1.tgz",
"integrity": "sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w==",
"license": "ISC",
"dependencies": {
"minipass": "^7.0.4"
},
"engines": {
"node": ">=18.0.0"
}
},
"node_modules/@scarf/scarf": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/@scarf/scarf/-/scarf-1.4.0.tgz",
@@ -330,6 +343,15 @@
"fsevents": "~2.3.2"
}
},
"node_modules/chownr": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/chownr/-/chownr-3.0.0.tgz",
"integrity": "sha512-+IxzY9BZOQd/XuYPRmrvEVjF/nqj5kgT4kEq7VofrDoM1MxoRjEWkrCC3EtLi59TVawxTAn+orJwFQcrqEN1+g==",
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=18"
}
},
"node_modules/cliui": {
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/cliui/-/cliui-6.0.0.tgz",
@@ -1488,6 +1510,27 @@
"url": "https://github.com/sponsors/isaacs"
}
},
"node_modules/minipass": {
"version": "7.1.3",
"resolved": "https://registry.npmjs.org/minipass/-/minipass-7.1.3.tgz",
"integrity": "sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==",
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=16 || 14 >=14.17"
}
},
"node_modules/minizlib": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/minizlib/-/minizlib-3.1.0.tgz",
"integrity": "sha512-KZxYo1BUkWD2TVFLr0MQoM8vUUigWD3LlD83a/75BqC+4qE0Hb1Vo5v1FgcfaNXvfXzr+5EhQ6ing/CaBijTlw==",
"license": "MIT",
"dependencies": {
"minipass": "^7.1.2"
},
"engines": {
"node": ">= 18"
}
},
"node_modules/morgan": {
"version": "1.11.0",
"resolved": "https://registry.npmjs.org/morgan/-/morgan-1.11.0.tgz",
@@ -2254,6 +2297,22 @@
"express": ">=4.0.0 || >=5.0.0-beta"
}
},
"node_modules/tar": {
"version": "7.5.22",
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.22.tgz",
"integrity": "sha512-MFO/QzvtAOmJbkhOaCTvbGcFN9L9b+JunIsDwaKljSOdcLMea3NJ1k9Usz/rjdfSXTq4dfzfeS7W4p4YOAAHeA==",
"license": "BlueOak-1.0.0",
"dependencies": {
"@isaacs/fs-minipass": "^4.0.0",
"chownr": "^3.0.0",
"minipass": "^7.1.2",
"minizlib": "^3.1.0",
"yallist": "^5.0.0"
},
"engines": {
"node": ">=18"
}
},
"node_modules/to-regex-range": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/to-regex-range/-/to-regex-range-5.0.1.tgz",
@@ -2414,6 +2473,15 @@
"integrity": "sha512-JKhqTOwSrqNA1NY5lSztJ1GrBiUodLMmIZuLiDaMRJ+itFd+ABVE8XBjOvIWL+rSqNDC74LCSFmlb/U4UZ4hJQ==",
"license": "ISC"
},
"node_modules/yallist": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/yallist/-/yallist-5.0.0.tgz",
"integrity": "sha512-YgvUTfwqyc7UXVMrB+SImsVYSmTS8X/tSrtdNZMImM+n7+QTriRXyXim0mBrTXNeqzVF0KWGgHPeiyViFFrNDw==",
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=18"
}
},
"node_modules/yargs": {
"version": "15.4.1",
"resolved": "https://registry.npmjs.org/yargs/-/yargs-15.4.1.tgz",

1
server/package.json
View File

@@ -38,6 +38,7 @@
"sanitize-html": "^2.17.5",
"speakeasy": "^2.0.0",
"swagger-ui-express": "^5.0.1",
"tar": "^7.5.22",
"ws": "^8.21.0"
},
"devDependencies": {

84
server/routes.guards.json
View File

@@ -427,6 +427,90 @@
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/modules",
"handlers": 2,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "POST",
"path": "/api/v1/admin/modules",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "DELETE",
"path": "/api/v1/admin/modules/:id",
"handlers": 5,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "POST",
"path": "/api/v1/admin/modules/:id/disable",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "POST",
"path": "/api/v1/admin/modules/:id/enable",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "POST",
"path": "/api/v1/admin/modules/:id/purge",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "POST",
"path": "/api/v1/admin/modules/restart",
"handlers": 2,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "PUT",
"path": "/api/v1/admin/modules/sources",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/pages",

32
server/routes.manifest.json
View File

@@ -177,6 +177,38 @@
"method": "POST",
"path": "/api/v1/admin/moderation/user/:discordId/notes"
},
{
"method": "GET",
"path": "/api/v1/admin/modules"
},
{
"method": "POST",
"path": "/api/v1/admin/modules"
},
{
"method": "DELETE",
"path": "/api/v1/admin/modules/:id"
},
{
"method": "POST",
"path": "/api/v1/admin/modules/:id/disable"
},
{
"method": "POST",
"path": "/api/v1/admin/modules/:id/enable"
},
{
"method": "POST",
"path": "/api/v1/admin/modules/:id/purge"
},
{
"method": "POST",
"path": "/api/v1/admin/modules/restart"
},
{
"method": "PUT",
"path": "/api/v1/admin/modules/sources"
},
{
"method": "GET",
"path": "/api/v1/admin/pages"

27
server/src/app.js
View File

@@ -111,15 +111,22 @@ app.use(
)
// ── API docs (Swagger UI) ─────────────────────────────────────────────
// Interactive OpenAPI docs at /api/docs, raw spec at /api/docs.json. The spec
// is generated from route annotations by `npm run swagger` (server/swagger/).
// Interactive OpenAPI docs at /api/docs, raw spec at /api/docs.json. Core's own
// routes are generated from their annotations by `npm run swagger`
// (server/swagger/) and committed; an installed module's routes cannot be —
// swagger-autogen is static analysis and a module arrives on the volume after the
// image was built — so each module ships its own fragment and they are merged
// HERE, per request, over core's committed spec (docs/website/MODULE_API.md §6.1a).
// Loaded lazily and guarded so a missing spec never crashes the server.
try {
// eslint-disable-next-line global-require
/* eslint-disable global-require */
const swaggerSpec = require('../swagger/swagger-output.json')
const { docsSpec } = require('../swagger/docsSpec')
/* eslint-enable global-require */
app.get('/api/docs.json', (req, res) => {
// #swagger.ignore = true
res.json(swaggerSpec)
res.json(docsSpec(swaggerSpec))
})
// swagger-ui-express injects an inline bootstrap script and inline styles, which
// the global 'self'-only script-src would block — relax CSP for this route only.
@@ -132,10 +139,18 @@ try {
'upgrade-insecure-requests': null,
},
})
app.use('/api/docs', swaggerCsp, swaggerUi.serve, swaggerUi.setup(swaggerSpec, {
// `setup()` is called PER REQUEST rather than once here, because the document it
// renders is not fixed at boot: a module reaching `started` (or failing to) adds
// or removes paths, and a UI bound to the spec as it looked while app.js was
// still being required would show core's routes for the life of the process
// while /api/docs.json showed the merged set. `docsSpec` is cached on the
// loader's state version, so the repeated call costs a comparison.
const swaggerOpts = {
customSiteTitle: `${brand.name} API docs`,
swaggerOptions: { persistAuthorization: true },
}))
}
app.use('/api/docs', swaggerCsp, swaggerUi.serve, (req, res, next) =>
swaggerUi.setup(docsSpec(swaggerSpec), swaggerOpts)(req, res, next))
} catch (err) {
errLog.error('Swagger spec not found — run `npm run swagger` to generate it. API docs disabled.', {
message: err.message,

9
server/src/config/brand.js
View File

@@ -22,10 +22,15 @@ const name = process.env.BRAND_NAME || 'Runic Gateway'
const brand = {
name,
shortName: process.env.BRAND_SHORT_NAME || name,
tagline: process.env.BRAND_TAGLINE || 'an independent private Ultima Online shard',
// Game-neutral defaults. Core is the platform, not one game's site: which game
// this instance is for is the operator's to say, through these two vars or an
// installed module (MODULE_SYSTEM.md §2.7.1, slice 4). Every real instance
// overrides both — `.env.uomysticmoon.example` sets its own wording — so these
// are what an unconfigured instance shows, not what anyone ships.
tagline: process.env.BRAND_TAGLINE || 'an independent, privately-run game server',
description:
process.env.BRAND_DESCRIPTION ||
`${name} — an independent private Ultima Online shard. News, screenshots, guides, and community notes.`,
`${name} — an independent, privately-run game server. News, screenshots, guides, and community notes.`,
contactEmail: process.env.BRAND_CONTACT_EMAIL || process.env.CONTACT_TO || '',
url: process.env.BRAND_URL || '',
// Visual

21
server/src/model/modules/modules.db.js
View File

@@ -17,6 +17,23 @@ const getOne = (id) => query(`SELECT ${COLS} FROM installed_modules WHERE id = ?
// module must not silently disable it, and re-installing a disabled one must not
// silently switch it back on. A brand-new row lands in `installed`, the transient
// state the next restart resolves.
// Provenance is COALESCEd, and that is the whole difference between this
// working and not.
//
// `lifecycle.boot()` re-records every module it scanned with no source and no
// sha256 — a directory placed on the volume by hand genuinely has neither, and
// the boot has no way to know where one came from. With a plain
// `source = VALUES(source)` that refresh overwrote both columns with NULL on
// EVERY boot, so an admin-panel install's provenance survived exactly until the
// restart that install asked for. Nothing could catch it before Phase 4: until
// then no caller ever passed a non-null value, and lifecycle.js's comment
// asserting that "recordInstalled leaves what it is not given" was a description
// of an intention rather than of this statement.
//
// COALESCE makes that comment true: a value overwrites, a NULL leaves what is
// there. The cost is that re-placing a DIFFERENT bundle by hand over a row that
// was once installed from a URL keeps the old provenance — which is wrong but
// stale, and strictly better than the alternative, which was wrong and blank.
const upsert = ({ id, name, version, source, sha256 }) =>
query(
`INSERT INTO installed_modules (id, name, version, source, sha256, state)
@@ -24,8 +41,8 @@ const upsert = ({ id, name, version, source, sha256 }) =>
ON DUPLICATE KEY UPDATE
name = VALUES(name),
version = VALUES(version),
source = VALUES(source),
sha256 = VALUES(sha256)`,
source = COALESCE(VALUES(source), source),
sha256 = COALESCE(VALUES(sha256), sha256)`,
[id, name, version, source ?? null, sha256 ?? null],
)

221
server/src/modules/archive.js Normal file
View File

@@ -0,0 +1,221 @@
// ── The hardened bundle extractor ──────────────────────────────────────────
//
// Phase 4, slice 1 of docs/website/MODULE_SYSTEM.md §2.7.2. This is the part of
// the install path that handles input an attacker chose, and it is separated
// from install.js so that it can be tested against crafted archives without a
// network, a database or a filesystem layout.
//
// **The download is not the dangerous part.** An allowlisted host, a declared
// sha256 and a size cap between them settle where the bytes came from and that
// they are the bytes that were published. Unpacking is different: the ARCHIVE
// chooses the filenames, and core writes into a directory bind-mounted from the
// host (MODULE_SYSTEM.md §2.5), so an escape is not confined to the container.
//
// The rule this file follows is **reject, never sanitise.** node-tar will
// happily strip a leading `/` and drop a `..` for you, which turns a hostile
// archive into a slightly different archive that then gets installed. An archive
// that needs correcting is an archive that should not be trusted, so anything on
// the list below fails the whole install and nothing is written.
//
// - an absolute path, POSIX (`/etc/…`) or Windows (`C:\…`, `\\server\…`)
// - any `..` segment, anywhere
// - anything that is not a regular file or a directory — so no symlinks, no
// hardlinks, no devices, no FIFOs. A module bundle has no legitimate use for
// any of them, and every one is a documented escape primitive
// - more than one top-level entry
// - more entries, or more unpacked bytes, than the caps below
//
// That third rule deserves its own note, because it is why the `tar` dependency
// is pinned forward rather than merely present: **the majority of node-tar's
// published advisories are hardlink or symlink path traversal**, several of them
// through interpretation differences in PAX and GNU long-name headers rather
// than through anything the calling code did wrong. Refusing those entry types
// outright means this extractor is not relying on the library to get their
// containment right. The remaining advisories are parser denial-of-service, and
// those are what the caps and the two-pass shape address.
//
// Two passes, deliberately: `inspect()` reads the archive and decides, and only
// an archive that survived that is handed to `extract()`. A `filter` callback
// during extraction cannot reject — by the time it is asked about entry 400, the
// first 399 are already on disk.
const fs = require('fs')
const fsp = require('fs/promises')
const path = require('path')
const tar = require('tar')
// Caps. Generous against a real bundle (module-uo's is a few megabytes, a few
// hundred files) and small enough that a decompression bomb is refused rather
// than paged in. Both are checked DURING the inspect pass, not after it, so a
// hostile archive stops being read at the limit instead of at its end.
const MAX_BYTES = 128 * 1024 * 1024
const MAX_ENTRIES = 20000
// tar entry types worth naming. node-tar reports these as strings; anything not
// in this set is refused, including the ones no one has thought of, which is the
// point of an allowlist here rather than a list of the types known to be bad.
const ALLOWED_TYPES = new Set(['File', 'Directory'])
class ArchiveError extends Error {
constructor(message) {
super(message)
this.name = 'ArchiveError'
}
}
/**
* Is this entry path safe to unpack anywhere?
*
* Returns a reason string, or null when the path is fine. Written as a reason
* rather than a boolean because the operator pasting a URL needs to be told what
* was wrong with what they were served, and "the archive is invalid" is not that.
*/
function pathProblem(entryPath) {
const p = String(entryPath)
// NUL is not a path character. node-tar has had uncaught-exception advisories
// for NUL bytes inside PAX records, so this is checked here rather than left
// to the parser.
if (p.includes('\0')) return 'an entry path contains a NUL byte'
// A backslash is a path separator on the platform this may be unpacked on, so
// an entry that contains one is not the single path component it looks like.
if (p.includes('\\')) return `an entry path contains a backslash: "${p}"`
if (p.startsWith('/')) return `an entry path is absolute: "${p}"`
// Drive-relative and UNC. Both are the subject of their own node-tar
// advisories, and neither is refused by a leading-slash check.
if (/^[a-zA-Z]:/.test(p)) return `an entry path is drive-absolute: "${p}"`
const segments = p.split('/')
if (segments.includes('..')) return `an entry path escapes upward: "${p}"`
return null
}
/**
* Read an archive without unpacking it, and decide whether it may be unpacked.
*
* @param {string} file absolute path to the .tar.gz
* @returns {Promise<{root: string, entries: number, bytes: number}>} the single
* top-level directory name, and what is inside it.
* @throws {ArchiveError} on anything in this file's header list.
*/
async function inspect(file) {
const roots = new Set()
let entries = 0
let bytes = 0
// The first problem found, kept rather than thrown from inside the callback:
// throwing out of `onentry` escapes through the parser's stream machinery and
// arrives as an unhelpful wrapped error, when it arrives at all.
let problem = null
const note = (message) => {
if (!problem) problem = message
}
await tar.t({
file,
onentry(entry) {
if (problem) return
entries += 1
if (entries > MAX_ENTRIES) {
note(`the archive has more than ${MAX_ENTRIES} entries`)
return
}
if (!ALLOWED_TYPES.has(entry.type)) {
// The message names the type because "symbolic link" is a much more
// useful thing for an operator to read than "invalid entry".
note(`the archive contains a ${entry.type} ("${entry.path}") — a module bundle may only contain files and directories`)
return
}
const bad = pathProblem(entry.path)
if (bad) {
note(bad)
return
}
// A negative or absurd size is a parser-confusion primitive in its own
// right; clamping at zero keeps the running total honest.
bytes += Math.max(0, Number(entry.size) || 0)
if (bytes > MAX_BYTES) {
note(`the archive unpacks to more than ${Math.round(MAX_BYTES / 1024 / 1024)} MB`)
return
}
const [root] = String(entry.path).split('/')
if (root) roots.add(root)
},
})
if (problem) throw new ArchiveError(problem)
if (entries === 0) throw new ArchiveError('the archive is empty')
if (roots.size !== 1) {
// A bundle is one directory. More than one top-level entry means `strip: 1`
// below would silently merge or discard things, and a bundle that needs
// interpreting is not a bundle.
throw new ArchiveError(
`the archive must contain exactly one top-level directory, found ${roots.size}` +
(roots.size > 1 ? ` (${[...roots].slice(0, 4).join(', ')})` : ''),
)
}
return { root: [...roots][0], entries, bytes }
}
/**
* Unpack an inspected archive into `dest`, stripping its single top level.
*
* The top-level directory is stripped rather than kept, because its name belongs
* to whoever published the bundle — module-uo's release workflow packs
* `module-uo-<version>/`, not `uo/` — while the directory it lands in is core's
* decision and has to be the module id the loader scans for. What is inside the
* bundle is the module; what the wrapper is called is packaging.
*
* `dest` must not exist. Callers unpack to a temporary directory and move it
* into place, so a failure halfway through leaves nothing for the next boot's
* scan to find.
*/
async function extract(file, dest) {
if (fs.existsSync(dest)) throw new ArchiveError(`extraction target already exists: ${dest}`)
await fsp.mkdir(dest, { recursive: true })
await tar.x({
file,
cwd: dest,
strip: 1,
// Belt and braces on top of inspect(): the library's own refusal to write
// outside cwd, and its refusal to overwrite through a link. Neither is
// load-bearing here — inspect() has already rejected every entry that could
// exercise them — but a second lock on a door that is already locked costs
// nothing, and this is the door.
preservePaths: false,
// Reject rather than warn on anything the parser itself objects to.
strict: true,
})
return dest
}
/**
* Inspect, then extract. The only entry point install.js uses.
*/
async function unpack(file, dest) {
const stats = await inspect(file)
await extract(file, dest)
return stats
}
module.exports = {
ArchiveError,
inspect,
extract,
unpack,
pathProblem,
MAX_BYTES,
MAX_ENTRIES,
ALLOWED_TYPES,
}

220
server/src/modules/declared.js Normal file
View File

@@ -0,0 +1,220 @@
// ── The declared module set ────────────────────────────────────────────────
//
// Phase 4, slice 3 of docs/website/MODULE_SYSTEM.md §2.7.2 — decision 4. A
// compose-managed host is not driven by clicking: it declares which modules it
// runs, in the file it already edits and version-controls, and the container
// arrives at that set by itself.
//
// MODULES: uo@0.3.0=https://<host>/…/module-uo-0.3.0.json
//
// One entry per module, `<id>@<version>=<install manifest URL>`, separated by
// whitespace or commas. The id and the version are written out rather than left
// to be discovered inside the manifest for one reason: **the no-op case must not
// need the network.** A module already unpacked at the declared version is
// answered by reading its own `module.json` off the volume, so a restart with
// the network down brings the site up exactly as it was. Only a module that is
// missing, or unpacked at some other version, reaches out — and it reaches out
// through modules/install.js, the same fetch-verify-unpack path the admin panel
// uses, under the same host allowlist.
//
// Three things this file deliberately does not do:
//
// - **It does not decide whether a module RUNS.** Resolution owns what is on
// the volume; `installed_modules` owns whether a mounted module answers. An
// admin who uninstalls a declared module gets its directory back at the next
// boot with the row still `disabled`, so it stays off until they enable it.
// The two never fight because they are not answering the same question.
// - **It does not fail a boot.** A module publisher's host being unreachable
// must not take a shard's website down with it; core is built to serve with
// a module absent (§1.6). Every failure is logged loudly and kept for the
// admin screen, and the site comes up.
// - **It does not mount anything.** §1.12 makes the volume the mounting source
// of truth, read once at require time — which is why this runs before
// `require('./app')` in server.js and not from inside it.
//
// It runs on every boot, not only in Docker: a bare `npm start` with MODULES set
// resolves the same way. The Docker path is the reason it exists, not a special
// case in it.
const fs = require('fs')
const path = require('path')
const install = require('./install')
const log = require('../utils/logger')('modules')
// The variable an operator sets. Named next to MODULES_DIR, which is the other
// half of the same story: one says where modules live, the other says which.
const VAR = 'MODULES'
// Same id rule the loader enforces when scanning and install.js enforces when
// placing, restated here so a declaration cannot name something neither would
// accept.
const ID = /^[a-z][a-z0-9-]{1,31}$/
// The outcome of the last resolution, in memory, for the admin screen. Not a
// database row: a declaration is a fact about this process's environment, and
// writing it down would put it in front of the boot reconcile, which resets
// every non-disabled row (§2.4). The screen merges it as a fourth source
// alongside the row, the loader and the volume.
let results = []
/**
* Parse the declaration into entries.
*
* Malformed entries are collected rather than thrown: one operator typo should
* cost that module, not every module on the host. A duplicate id keeps the
* first — there is no sensible way to run two versions of one module, and
* silently preferring the last would make the outcome depend on the order of a
* list nobody reads as ordered.
*
* @param {string} value the raw variable
* @returns {{entries: Array<{id,version,url}>, errors: string[]}}
*/
function parse(value) {
const entries = []
const errors = []
const seen = new Set()
for (const token of String(value || '').split(/[,\s]+/).filter(Boolean)) {
const at = token.indexOf('@')
const eq = token.indexOf('=')
if (at < 1 || eq < at + 2) {
errors.push(`"${token}" is not <id>@<version>=<url>`)
continue
}
const id = token.slice(0, at)
const version = token.slice(at + 1, eq)
const url = token.slice(eq + 1)
if (!ID.test(id)) {
errors.push(`"${token}" names an invalid module id "${id}"`)
continue
}
if (!url) {
errors.push(`"${token}" has no install manifest URL`)
continue
}
if (seen.has(id)) {
errors.push(`"${id}" is declared more than once — keeping the first`)
continue
}
seen.add(id)
entries.push({ id, version, url })
}
return { entries, errors }
}
/**
* The version currently unpacked on the volume, or null.
*
* Read straight out of the module's own `module.json`, which is the same file
* the loader trusts for the same fact — and never from `installed_modules`,
* because the row records what was installed and this has to answer what is
* actually there. An unreadable manifest counts as absent: whatever is in that
* directory, it is not a module at the declared version.
*/
function installedVersion(id) {
try {
const manifest = JSON.parse(
fs.readFileSync(path.join(install.moduleDir(id), 'module.json'), 'utf8'),
)
return manifest && manifest.version ? String(manifest.version) : null
} catch {
return null
}
}
/**
* Bring the volume in line with the declaration.
*
* Never throws and never rejects. Returns one outcome per declared entry, and
* remembers them for `state()`.
*
* @param {object} args
* @param {string} [args.value] the raw variable (defaults to the environment)
* @param {string[]} args.hosts the install allowlist, already parsed
* @param {object} args.model modules.model, for recording provenance
* @param {object} [args.installImpl] injection seam, as everywhere else here
* @returns {Promise<Array<{id,version,url,action,message}>>}
*/
async function resolve({ value = process.env[VAR], hosts = [], model, installImpl = install } = {}) {
const { entries, errors } = parse(value)
results = []
for (const message of errors) log.error(`${VAR}: ${message}`)
if (!entries.length) return results
log.info(`${VAR} declares ${entries.length} module(s)`, {
modules: entries.map((e) => `${e.id}@${e.version}`).join(' '),
})
for (const entry of entries) {
const present = installedVersion(entry.id)
if (present === entry.version) {
// The offline path, and the common one: nothing is fetched, nothing is
// written, and a host with no route to the internet boots unchanged.
log.info(`module "${entry.id}" is already at the declared version ${entry.version}`)
results.push({ ...entry, action: 'noop', message: null })
continue
}
try {
// `expect` is the declaration itself, handed down so install.js can refuse
// a URL that resolves to another module or another version while it is
// still only a manifest — a check made after the unpack would be made with
// the undeclared module already on the volume.
const result = await installImpl.install({
url: entry.url,
hosts,
expect: { id: entry.id, version: entry.version },
})
// Provenance, written exactly as the admin route writes it — the whole
// point of resolving in-process rather than from a script that cannot
// reach the database. A module installed by the compose file and one
// installed by an admin are then indistinguishable on the screen, which
// is what makes this one feature and not two.
if (model) {
await model.recordInstalled({
id: result.id,
name: result.name,
version: result.version,
source: entry.url,
sha256: result.sha256,
})
}
log.warn(`installed declared module "${entry.id}" v${entry.version}`, {
from: present || 'nothing',
source: entry.url,
sha256: result.sha256,
})
results.push({ ...entry, action: 'installed', message: null })
} catch (err) {
// Loud, and then onward. The site serves; this module does not, or serves
// the version that was already there.
log.error(
`could not resolve declared module "${entry.id}@${entry.version}": ${err.message}` +
(present ? ` — leaving version ${present} in place` : ''),
)
results.push({ ...entry, action: 'failed', message: err.message })
}
}
return results
}
/** What the last resolution decided, for the admin screen. */
function state() {
return results.map((r) => ({ ...r }))
}
/** Test seam: forget the last resolution. */
function reset() {
results = []
}
module.exports = { VAR, parse, installedVersion, resolve, state, reset }

450
server/src/modules/install.js Normal file
View File

@@ -0,0 +1,450 @@
// ── Installing and removing a module bundle ────────────────────────────────
//
// Phase 4, slice 1 of docs/website/MODULE_SYSTEM.md §2.7.2 — the consumer half
// of a release the module's own CI already publishes. Nothing here touches the
// database or the loader: this file moves bytes onto the volume and off it, and
// the caller (router/v1/admin/modules.controller.js) writes down what happened.
//
// The shape of an install, and why it is this shape:
//
// 1. The operator pastes the URL of a release's install manifest — a small
// JSON document naming the artifact, its sha256 and its size (decision 2).
// There is no catalog, because a catalog would make core's release cadence
// decide which modules can exist.
// 2. Every URL fetched — the manifest, the artifact, and every redirect hop —
// is checked against the admin-managed host allowlist (decision 6). That is
// what keeps a pasted URL from also being an SSRF primitive.
// 3. The artifact is streamed to a temporary file under a byte cap, hashed as
// it arrives, and compared against the manifest's `sha256`. The hash is the
// trust anchor; releases are unsigned and say so.
// 4. modules/archive.js inspects the archive in full before a single byte is
// unpacked, and only then unpacks it — into a TEMPORARY directory.
// 5. The unpacked tree's own `module.json` must agree with the manifest about
// what it is. A manifest that promises `uo` and delivers something else is
// refused rather than installed under the name it was promised.
// 6. Only then is anything moved into `modules/<id>/`, and the directory it
// replaces is kept aside until the move has succeeded.
//
// **Nothing is written into the modules directory until every check has passed**,
// and that is not belt-and-braces. Verified against node-tar 7.5.22 while writing
// this: extracting an archive whose fourth member escapes upward throws — and
// leaves the first three members on disk. A loader scans that directory at
// require time on the next boot; a half-unpacked module is a module.
//
// One thing this file deliberately does NOT do: mount anything. Installing puts
// a directory on the volume, and §1.12 means the volume is read at require time,
// so the module appears when the process restarts. The admin screen offers that
// restart (decision 1); it is not implied here.
const crypto = require('crypto')
const fs = require('fs')
const fsp = require('fs/promises')
const path = require('path')
const { Readable, Transform } = require('stream')
const { pipeline } = require('stream/promises')
const archive = require('./archive')
const loader = require('./loader')
const log = require('../utils/logger')('modules')
// An install manifest is a small JSON document. A megabyte of it is not a
// manifest, and reading it into memory unbounded is the one place this file
// would otherwise trust a remote length.
const MAX_MANIFEST_BYTES = 256 * 1024
// The artifact cap is the archive's own unpacked cap — a compressed bundle
// larger than what it is allowed to unpack to has nothing to offer.
const MAX_ARTIFACT_BYTES = archive.MAX_BYTES
const FETCH_TIMEOUT_MS = 30000
// Gitea serves a release asset through at least one redirect. Following them is
// necessary; following them blindly is how an allowlist gets bypassed, so each
// hop is re-checked and the chain is bounded.
const MAX_REDIRECTS = 5
// Same id rule the loader enforces when scanning, restated rather than imported
// so an install cannot put a directory on the volume that the loader would then
// refuse to look at.
const ID = /^[a-z][a-z0-9-]{1,31}$/
const SHA256 = /^[0-9a-f]{64}$/
class InstallError extends Error {
constructor(message, { status = 400 } = {}) {
super(message)
this.name = 'InstallError'
// Carried so the controller can answer 400 for "your URL is wrong" and 502
// for "the host you named misbehaved" without re-deriving it from the text.
this.status = status
}
}
// ── The allowlist ──────────────────────────────────────────────────────────
// The settings row the allowlist lives in (decision 6): seeded from
// MODULE_SOURCE_HOSTS on a fresh install and admin-managed from then on. The KEY
// lives here rather than in the admin controller because it is now read from two
// places — the controller, and the boot-time resolution of the declared module
// set (modules/declared.js), which has no route and no request.
const HOSTS_SETTING = 'module_source_hosts'
/**
* Parse the stored allowlist setting into hostnames.
*
* Comma or whitespace separated, case-insensitive, empty entries dropped. An
* empty list means nothing may be installed from anywhere — a refusal, never a
* wildcard. That direction matters: a setting an admin accidentally blanks
* should stop installs, not permit every host on the internet.
*/
function parseHosts(value) {
return String(value || '')
.split(/[,\s]+/)
.map((h) => h.trim().toLowerCase())
.filter(Boolean)
}
/**
* Check one URL against the allowlist, and return it parsed.
*
* `https` only. A plaintext fetch of code this process is going to execute is
* not something an operator should be able to opt into by typing a URL, and the
* sha256 does not help — whoever can rewrite the artifact in flight can rewrite
* the manifest that declares its hash.
*/
function checkUrl(raw, hosts) {
let url
try {
url = new URL(String(raw))
} catch {
throw new InstallError(`"${raw}" is not a valid URL`)
}
if (url.protocol !== 'https:') {
throw new InstallError(`only https URLs may be installed from (got "${url.protocol}")`)
}
if (!hosts.length) {
throw new InstallError(
'no module source hosts are allowed — set one in Admin → Modules before installing',
)
}
if (!hosts.includes(url.hostname.toLowerCase())) {
throw new InstallError(
`"${url.hostname}" is not an allowed module source host (allowed: ${hosts.join(', ')})`,
)
}
return url
}
// ── Fetching ───────────────────────────────────────────────────────────────
/**
* GET a URL, following redirects by hand so every hop is re-checked.
*
* `fetch`'s own redirect following would take the first hop off the allowlist
* and the rest wherever it was pointed, which is exactly the hole the allowlist
* exists to close.
*
* `fetchImpl` is the same injection seam `replayFragments({query})` and
* `lifecycle.boot({model})` use, and it exists for the same reason: the rules
* this file enforces — https only, allowlisted host, allowlisted REDIRECT host,
* bounded body, matching hash — are all decisions about a response, and testing
* them against a real TLS server would mean testing Node's certificate handling
* instead. Production never passes it.
*
* @returns {Promise<Response>} a response whose body has not been read.
*/
async function get(rawUrl, hosts, fetchImpl = fetch) {
let url = checkUrl(rawUrl, hosts)
for (let hop = 0; hop <= MAX_REDIRECTS; hop += 1) {
let res
try {
res = await fetchImpl(url, {
redirect: 'manual',
signal: AbortSignal.timeout(FETCH_TIMEOUT_MS),
headers: { accept: '*/*' },
})
} catch (err) {
throw new InstallError(`could not reach ${url.hostname}: ${err.message}`, { status: 502 })
}
if (res.status >= 300 && res.status < 400) {
const location = res.headers.get('location')
if (!location) throw new InstallError(`${url.hostname} redirected without a location`, { status: 502 })
// Resolved against the current URL, then re-checked — a relative redirect
// is normal and a cross-host one is the thing being guarded against.
url = checkUrl(new URL(location, url).toString(), hosts)
continue
}
if (!res.ok) {
throw new InstallError(`${url.href} returned ${res.status}`, { status: 502 })
}
return res
}
throw new InstallError(`too many redirects (more than ${MAX_REDIRECTS})`, { status: 502 })
}
/** Read a bounded response body as text. */
async function readText(res, cap, what) {
const declared = Number(res.headers.get('content-length'))
if (Number.isFinite(declared) && declared > cap) {
throw new InstallError(`${what} is larger than ${cap} bytes`)
}
const buf = Buffer.from(await res.arrayBuffer())
// Checked again after reading: content-length is the server's claim, not a
// limit it is obliged to honour.
if (buf.length > cap) throw new InstallError(`${what} is larger than ${cap} bytes`)
return buf.toString('utf8')
}
/**
* Stream a response body to a file, hashing as it goes and stopping at the cap.
*
* The hash is computed from the bytes that were written rather than by re-reading
* the file, so there is no window in which the file could differ from what was
* verified.
*/
async function download(res, dest, cap) {
const hash = crypto.createHash('sha256')
let bytes = 0
const body = Readable.fromWeb(res.body)
const counter = new Transform({
transform(chunk, _enc, cb) {
bytes += chunk.length
if (bytes > cap) {
cb(new InstallError(`the artifact is larger than ${Math.round(cap / 1024 / 1024)} MB`))
return
}
hash.update(chunk)
cb(null, chunk)
},
})
await pipeline(body, counter, fs.createWriteStream(dest))
return { sha256: hash.digest('hex'), bytes }
}
// ── The install manifest ───────────────────────────────────────────────────
/**
* Fetch and validate a release's install manifest.
*
* The shape is the one module-uo's release workflow already writes:
* `{schema, id, name, version, coreApi, artifact, url, sha256, size}`. Only the
* fields this side needs are required — a module publisher may carry more.
*/
async function fetchManifest(manifestUrl, hosts, fetchImpl = fetch) {
const res = await get(manifestUrl, hosts, fetchImpl)
const text = await readText(res, MAX_MANIFEST_BYTES, 'the install manifest')
let manifest
try {
manifest = JSON.parse(text)
} catch (err) {
throw new InstallError(`the install manifest is not valid JSON: ${err.message}`)
}
if (!manifest || typeof manifest !== 'object' || Array.isArray(manifest)) {
throw new InstallError('the install manifest is not a JSON object')
}
const { id, name, version, sha256 } = manifest
if (!ID.test(String(id || ''))) {
throw new InstallError(`the install manifest has an invalid module id: ${JSON.stringify(id)}`)
}
if (!name || !version) throw new InstallError('the install manifest is missing name or version')
if (!SHA256.test(String(sha256 || '').toLowerCase())) {
throw new InstallError('the install manifest has no valid sha256 for its artifact')
}
// `url` is the absolute artifact URL the publisher wrote; `artifact` is its
// filename. Prefer the URL, fall back to resolving the filename beside the
// manifest — which is where a release's assets sit — so a manifest that
// travelled without its absolute URL still installs.
const artifactUrl = new URL(manifest.url || manifest.artifact || '', manifestUrl).toString()
return {
id: String(id),
name: String(name),
version: String(version),
sha256: String(sha256).toLowerCase(),
size: Number(manifest.size) || null,
artifactUrl,
coreApi: manifest.coreApi ? String(manifest.coreApi) : null,
}
}
// ── Paths on the volume ────────────────────────────────────────────────────
/** Where a module with this id lives. Rejects an id that is not one. */
function moduleDir(id) {
if (!ID.test(String(id || ''))) throw new InstallError(`invalid module id: ${JSON.stringify(id)}`)
return path.join(loader.dir(), String(id))
}
/** Is there a directory for this module on the volume right now? */
function isInstalled(id) {
try {
return fs.statSync(moduleDir(id)).isDirectory()
} catch {
return false
}
}
/**
* The absolute path of a module's `purge.sql`, or null.
*
* Read from the module's own `module.json` on disk rather than from the loader,
* because purge has to work for a module that never loaded — a `startup_failed`
* one is exactly when an operator wants its tables gone.
*/
function purgeFile(id) {
try {
const manifest = JSON.parse(fs.readFileSync(path.join(moduleDir(id), 'module.json'), 'utf8'))
if (!manifest.purge) return null
const file = path.resolve(moduleDir(id), manifest.purge)
// The same containment check the loader applies to `client.entry`: a
// manifest may not point core at a file outside the module.
if (!file.startsWith(moduleDir(id) + path.sep)) return null
return fs.existsSync(file) ? file : null
} catch {
return null
}
}
// ── Install ────────────────────────────────────────────────────────────────
/**
* Download, verify, unpack and install a module from an install-manifest URL.
*
* Never leaves a partial module on the volume: everything happens under a
* scratch directory that is removed on any failure, and the move into place is
* the last step.
*
* `expect` is what the CALLER was promised, as opposed to what the manifest
* promises about itself — the declared module set (modules/declared.js) pins an
* id and a version in the environment, and a URL that resolves to something else
* has to be refused rather than installed. Checked against the manifest, before
* a byte is downloaded: catching it after the unpack would mean the undeclared
* module is already on the volume when the objection is raised. The admin panel
* passes nothing, because there a URL is the whole of what was asked for.
*
* @param {object} args
* @param {string} args.url the install manifest URL the admin pasted
* @param {string[]} args.hosts the allowlist, already parsed
* @param {{id?: string, version?: string}} [args.expect] what the caller pinned
* @returns {Promise<{id,name,version,sha256,source,bytes,replaced}>}
*/
async function install({ url, hosts, expect = null, fetchImpl = fetch }) {
const manifest = await fetchManifest(url, hosts, fetchImpl)
if (expect && expect.id && manifest.id !== expect.id) {
throw new InstallError(
`that URL installs the module "${manifest.id}", but "${expect.id}" was asked for`,
)
}
if (expect && expect.version && manifest.version !== expect.version) {
throw new InstallError(
`that URL installs ${manifest.id} v${manifest.version}, but v${expect.version} was asked for`,
)
}
const target = moduleDir(manifest.id)
const scratch = await fsp.mkdtemp(path.join(loader.dir(), `.install-${manifest.id}-`))
const tarball = path.join(scratch, 'bundle.tar.gz')
const unpacked = path.join(scratch, 'unpacked')
try {
const res = await get(manifest.artifactUrl, hosts, fetchImpl)
const { sha256, bytes } = await download(res, tarball, MAX_ARTIFACT_BYTES)
if (sha256 !== manifest.sha256) {
// The whole trust model in one comparison. Deliberately does not name the
// computed hash in a way that reads like a value to copy into the
// manifest — a mismatch means stop, not reconcile.
throw new InstallError(
`the downloaded artifact does not match the sha256 in the install manifest — refusing to install`,
)
}
if (manifest.size && bytes !== manifest.size) {
throw new InstallError(`the artifact is ${bytes} bytes but the manifest declares ${manifest.size}`)
}
await archive.unpack(tarball, unpacked)
// What the bundle says it is, checked against what the manifest promised.
let inner
try {
inner = JSON.parse(await fsp.readFile(path.join(unpacked, 'module.json'), 'utf8'))
} catch {
throw new InstallError('the bundle has no readable module.json at its root')
}
if (inner.id !== manifest.id) {
throw new InstallError(
`the bundle declares module id "${inner.id}" but the install manifest promised "${manifest.id}"`,
)
}
if (inner.version !== manifest.version) {
throw new InstallError(
`the bundle declares version "${inner.version}" but the install manifest promised "${manifest.version}"`,
)
}
// The swap. The outgoing directory is moved aside rather than deleted first,
// so a failed rename leaves the previous version recoverable instead of
// leaving no module at all — the same reasoning the installer repo applies
// to a ServUO tree.
const replaced = fs.existsSync(target)
const aside = `${target}.replaced-${Date.now()}`
if (replaced) await fsp.rename(target, aside)
try {
await fsp.rename(unpacked, target)
} catch (err) {
if (replaced) await fsp.rename(aside, target).catch(() => {})
throw err
}
if (replaced) await fsp.rm(aside, { recursive: true, force: true })
log.info(`installed module "${manifest.id}" v${manifest.version}`, {
source: url,
sha256: manifest.sha256,
bytes,
replaced,
})
return { ...manifest, source: url, bytes, replaced }
} finally {
await fsp.rm(scratch, { recursive: true, force: true }).catch(() => {})
}
}
/**
* Remove a module's directory from the volume.
*
* Returns whether there was one. Does not touch the database, does not run
* purge.sql, and does not stop the running module — the controller sequences
* all three, because the order matters and only it knows what the operator
* asked for.
*/
async function removeDir(id) {
const dir = moduleDir(id)
if (!fs.existsSync(dir)) return false
await fsp.rm(dir, { recursive: true, force: true })
log.info(`removed module directory for "${id}"`)
return true
}
module.exports = {
InstallError,
HOSTS_SETTING,
parseHosts,
checkUrl,
get,
fetchManifest,
install,
removeDir,
moduleDir,
isInstalled,
purgeFile,
MAX_MANIFEST_BYTES,
MAX_ARTIFACT_BYTES,
}

79
server/src/modules/lifecycle.js
View File

@@ -101,10 +101,16 @@ async function boot({ modules, model } = {}) {
id: m.id,
name: m.name,
version: m.version,
// Null provenance is what a hand-placed directory looks like. An install
// performed through the admin panel (§2.5, a later phase) writes the row
// with its source and hash first; this refresh deliberately does not
// overwrite either, because recordInstalled leaves what it is not given.
// Null provenance is what a hand-placed directory looks like, and it is
// all this step can honestly say: a scan finds a directory, never where it
// came from. An install through the admin panel writes source and sha256
// first, and this refresh must not undo that.
//
// It used to. `upsert` assigned both columns unconditionally, so every
// boot nulled them and an install's provenance survived only until the
// restart it asked for. The statement now COALESCEs — see the note on
// modules.db.js's upsert. Nothing could have caught it before Phase 4:
// this was the only caller, and it has never had a value to pass.
}))
}
@@ -205,4 +211,67 @@ async function shutdown({ modules, budgetMs = SHUTDOWN_BUDGET_MS } = {}) {
}
}
module.exports = { boot, shutdown, SHUTDOWN_BUDGET_MS }
/**
* Stop ONE module and mark it disabled — the admin panel's Disable (§2.7.2
* decision 3).
*
* Phase 2 built disable as a pure state flip: the record moved to `disabled` and
* the dispatch guard started answering 404. That makes a module invisible, not
* stopped. Everything a module does that is not a response to a request — the
* sockets and timers its `onBoot` armed — carried on running, so an operator
* disabling a module *because* it was misbehaving got nothing until the next
* restart, which is the one thing the button was supposed to save them.
*
* So the hook runs first, and the state moves after it: while `onShutdown` is
* running the module is still `started`, which is the only state in which its
* own routes and the things it is tearing down are consistent with each other.
* The hook gets the same budget the exit path gives it.
*
* A hook that throws does NOT stop the disable. The operator asked for this
* module to stop answering; a module that could not close cleanly is a reason to
* log loudly, not a reason to leave it serving. That is the opposite of the boot
* path's rule, and deliberately: there, a failure means the module never became
* safe to use.
*
* **Enable is not the mirror of this, and there is no `start(id)` beside it.**
* There is no `onBoot` re-dispatch, and MODULE_API.md has never promised the
* hooks are re-entrant — no module author has written `onBoot` to be safe to run
* twice in one process. Re-enabling therefore moves the row and waits for a
* restart, which the admin screen offers.
*
* @returns {Promise<{stopped: boolean, error: string|null}>} whether a hook ran.
*/
async function stop(id, { modules, model, budgetMs = SHUTDOWN_BUDGET_MS } = {}) {
/* eslint-disable global-require */
const loader = modules || require('./loader')
const rows = model || require('../model/modules/modules.model')
/* eslint-enable global-require */
let error = null
let stopped = false
if (loader.isLoaded()) {
const target = loader.stopHook(id)
if (target) {
try {
await withBudget(target.hook, budgetMs)
stopped = true
log.info(`module "${id}" stopped by an operator`)
} catch (err) {
error = err.message
log.warn(`module "${id}" onShutdown failed or timed out while being disabled — disabling anyway`, {
error: err.message,
})
}
}
// Unconditional, and after the hook: this is what makes its routes and its
// client chunk answer 404 (§4.5). A module that is not loaded in this
// process has no record to move, and setState ignores an unknown id.
loader.setState(id, 'disabled')
}
await safe(`disabling module "${id}"`, () => rows.disable(id))
return { stopped, error }
}
module.exports = { boot, shutdown, stop, SHUTDOWN_BUDGET_MS }

69
server/src/modules/loader.js
View File

@@ -74,6 +74,13 @@ const MANIFEST_KEYS = new Set([
const modules = new Map()
let loaded = false
// Bumped by every state CHANGE. One consumer today: the merged OpenAPI document
// at /api/docs.json, which is built from the fragments of `started` modules and
// so has to be rebuilt when that set moves (§6.1a). A counter rather than an
// event, because the question a cache asks is "is what I have still current",
// and a number answers it without anyone having to remember to subscribe.
let stateVersion = 0
// ── ctx ────────────────────────────────────────────────────────────────────
// Everything a module may reach in core, and nothing else (§2.3). Required
@@ -709,6 +716,7 @@ function setState(id, state, { stage = null, reason = null } = {}) {
if (!RECORD_STATES.has(state)) throw new Error(`unknown module state "${state}"`)
const record = modules.get(id)
if (!record) return
if (record.state !== state) stateVersion += 1
record.state = state
record.stage = state === 'startup_failed' ? stage : null
record.reason = state === 'startup_failed' ? reason : null
@@ -788,6 +796,31 @@ function shutdownHooks() {
.reverse()
}
/**
* One module's `onShutdown`, for stopping it on its own rather than at exit.
*
* Phase 4 (§2.7.2 decision 3) gave the admin panel's Disable a real meaning.
* Until then, disabling flipped this record's state and the dispatch guard began
* answering 404 — which made the module invisible without making it stop. A
* module's `onBoot` is where it opens its sockets and arms its timers, and none
* of that is reachable through a URL, so an operator disabling a misbehaving
* module got no relief from it at all until the next restart.
*
* `started` only, the same rule shutdownHooks() applies and for the same reason:
* a module whose `onBoot` threw has a half-built world its `onShutdown` was
* never written to tear down. Returns null when there is nothing to run — which
* covers "not started", "no hook", and "no such module", none of which is an
* error the caller can act on differently.
*
* @returns {{id: string, hook: Function}|null}
*/
function stopHook(id) {
assertLoaded('stopHook')
const record = modules.get(id)
if (!record || record.state !== 'started' || !record.hooks.onShutdown) return null
return { id: record.id, hook: record.hooks.onShutdown }
}
/**
* Every schema fragment waiting to be replayed, in scan order.
*
@@ -845,6 +878,39 @@ function clientEntryUrls() {
.map((r) => r.client.entryUrl)
}
/**
* Every started module's OpenAPI fragment, in scan order.
*
* `started` only, matching clientEntryUrls() rather than clientChunks(): the
* merged document is built when it is asked for, at which point the state is
* known, and documenting a module that is 503ing every one of those paths would
* send a client somewhere it cannot go.
*
* The filename is fixed by §2.8 — `swagger-fragment.json` in the bundle root —
* rather than declared in `module.json`, so a module cannot point core at
* something else. A module that ships none is simply absent: registering routes
* without documenting them is checked in the module's OWN CI (§2.8), where the
* routes are known; core has no way to tell the difference here between a module
* with no routes and one that forgot.
*
* @returns {{id: string, file: string}[]}
*/
function specFragments() {
assertLoaded('specFragments')
return [...modules.values()]
.filter((r) => r.state === 'started')
.map((r) => ({ id: r.id, file: path.join(r.dir, 'swagger-fragment.json') }))
.filter((f) => fs.existsSync(f.file))
}
/**
* How many times a module's state has CHANGED in this process.
*
* A cache key, and nothing more: hold the value you built with, compare, rebuild
* when it differs. It says nothing about which module moved or where to.
*/
const version = () => stateVersion
/** Absolute path of the modules directory. */
const dir = () => MODULES_DIR
@@ -855,8 +921,11 @@ module.exports = {
fragments,
bootable,
shutdownHooks,
stopHook,
clientChunks,
clientEntryUrls,
specFragments,
version,
isLoaded,
dir,
}

46
server/src/modules/schema.js
View File

@@ -81,4 +81,48 @@ async function replayFragments({ query, modules } = {}) {
}
}
module.exports = { replayFragments }
/**
* Run one module's `purge.sql` — the destructive twin of the replay above
* (MODULE_SYSTEM.md §2.5, and §2.7.2 decision 5 for when it is offered).
*
* It lives beside replayFragments because they are the same operation pointed in
* opposite directions: a file of statements the module ships, split by the same
* splitter, run serially on the same pool. Keeping them together is what makes
* it obvious that the fragment's leading-verb allowlist does NOT apply here —
* `purge.sql` is the one file a module may put a DROP in, precisely because it
* is the one file that never runs on a boot.
*
* Unlike the replay, this **throws**. A replay failure is one module failing to
* start, which the site survives by 503ing that module; a purge failure is an
* operator's explicit destructive request not having happened, and reporting
* success for that would leave them believing data is gone when it is not.
*
* Statements run serially and are not wrapped in a transaction, for the reason
* the replay's header already gives: MariaDB commits DDL implicitly, so there is
* no rollback to have. A purge that fails halfway has dropped some tables — the
* error names the statement that stopped it, and running it again is safe
* because a purge script is required to be idempotent in the same way a fragment
* is (`DROP TABLE IF EXISTS`).
*
* @param {string} file absolute path to the module's purge.sql
* @param {object} [deps] injection seam for tests
* @returns {Promise<number>} how many statements ran
*/
async function runPurge(file, { query } = {}) {
// eslint-disable-next-line global-require
const run = query || require('../utils/db').query
const statements = splitStatements(fs.readFileSync(file, 'utf8'))
let ran = 0
for (const statement of statements) {
try {
await run(statement)
ran += 1
} catch (err) {
throw new Error(`purge failed at statement ${ran + 1} of ${statements.length}: ${err.message}`)
}
}
return ran
}
module.exports = { replayFragments, runPurge }

6
server/src/router/v1/admin/index.js
View File

@@ -30,6 +30,7 @@ const pagesRouter = require('./pages.router')
const emailRouter = require('./email.router')
const discordBotRouter = require('./discordBot.router')
const settingsRouter = require('./settings.router')
const modulesRouter = require('./modules.router')
const dashboardRouter = require('./dashboard.router')
const adminRouter = express.Router()
@@ -73,6 +74,11 @@ adminRouter.use('/pages', pagesRouter)
adminRouter.use('/email', emailRouter)
adminRouter.use('/discord-bot', discordBotRouter)
adminRouter.use('/settings', settingsRouter)
// The Modules screen (Phase 4). Core's, not a module's — and it has to be
// core's: it is how a module gets onto the volume in the first place. Mounted
// here alongside the other configuration capabilities, and admin-only per route
// rather than at this line, so the gate sits next to what it is guarding.
adminRouter.use('/modules', modulesRouter)
// The two singletons that own no path segment of their own: GET /dashboard and
// PUT /site-mode. Mounted at the group root, last, exactly where the residual

2
server/src/router/v1/admin/invites.router.js
View File

@@ -19,7 +19,7 @@ invitesRouter.post(
// #swagger.tags = ['Admin · Invites']
// #swagger.summary = 'Create and email an account invite at a chosen access level'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["email","role"], properties: { email: { type: "string" }, role: { type: "string" } } } } } */
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["email","role"], properties: { email: { type: "string" }, role: { type: "string" } } } } } } */
/* #swagger.responses[201] = { description: 'Invite created', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[400] = { description: 'Validation error', content: { "application/json": { schema: { $ref: "#/components/schemas/ValidationError" } } } } */
adminOnly,

426
server/src/router/v1/admin/modules.controller.js Normal file
View File

@@ -0,0 +1,426 @@
// ── Admin: installed modules ───────────────────────────────────────────────
//
// Phase 4, slice 1 of docs/website/MODULE_SYSTEM.md §2.7.2. Admin-only, and more
// so than anything else in this directory: installing a module puts JavaScript on
// the volume that core will `require` into its own process on the next boot. That
// is the feature — it is what "an operator never builds anything" means (§1.14) —
// but it is worth being plain that this controller is remote code execution with
// an audit trail, not a settings screen.
//
// What guards it, in the order an attacker would meet them:
//
// 1. `requireRole('admin')` on every route, on top of the group's staff gate.
// 2. An `https`-only host allowlist, re-checked on every redirect hop, so a
// pasted URL cannot be pointed at the compose network or a metadata service
// (install.js).
// 3. The sha256 the release published, compared against the bytes that arrived.
// 4. A full inspection of the archive before a byte of it is unpacked, and an
// unpack into a scratch directory that is only moved into place once the
// bundle has agreed with the manifest about what it is (archive.js).
// 5. Every action here writes to core's one audit log.
//
// The one thing this file cannot do is mount anything. §1.12 makes the volume the
// mounting source of truth, read once at require time, so install and uninstall
// take effect on the next boot — which is why `restart` is a route here rather
// than a sentence in a tooltip (decision 1).
const modules = require('../../../model/modules/modules.model')
const activity = require('../../../model/activity/activity.model')
const settings = require('../../../model/settings/settings.model')
const loader = require('../../../modules/loader')
const lifecycle = require('../../../modules/lifecycle')
const install = require('../../../modules/install')
const declared = require('../../../modules/declared')
// A namespace import, like every other require in this file, and not
// `const { runPurge } = …`: destructuring at require time captures the function
// rather than the module, which makes it the one dependency here that cannot be
// substituted. That matters because the two tests worth having about purge are
// about the ORDER it runs in relative to the directory being removed.
const schema = require('../../../modules/schema')
const log = require('../../../utils/logger')('admin-modules')
// The allowlist setting. Seeded from MODULE_SOURCE_HOSTS on first boot and
// admin-managed from then on (decision 6) — db/seed.js writes it once and never
// overwrites it, so changing the variable later does not silently reach in and
// undo an operator's choice. The key itself lives in install.js, which is now
// read by boot-time declared-set resolution as well as by this controller.
const HOSTS_KEY = install.HOSTS_SETTING
// A hostname, not a URL: no scheme, no path, no port, no wildcard. Deliberately
// strict — every character allowed here is a character that can appear in the
// host of a URL this server will fetch and execute the contents of.
const HOSTNAME = /^[a-z0-9]([a-z0-9-]*[a-z0-9])?(\.[a-z0-9]([a-z0-9-]*[a-z0-9])?)*$/
async function allowedHosts() {
return install.parseHosts(await settings.get(HOSTS_KEY))
}
/**
* One module, as the admin screen needs it.
*
* Four sources have to be reconciled, and which one answers which question is
* the whole of §2.4:
*
* - the ROW says what the operator decided and what the last boot recorded;
* - the LOADER says what is mounted and answering right now;
* - the VOLUME says whether there is still a directory there at all;
* - the DECLARATION (slice 3) says what this container's environment asks for,
* which is the only one of the four an admin cannot change from this screen.
*
* They can legitimately disagree, and the screen has to show that rather than
* pick a winner. A row `enabled` with a loader state of `disabled` is a module
* the operator has just switched back on and which is waiting for a restart —
* exactly the case decision 3 creates, and it would be a lie to render it as
* either "running" or "off". A declared module with no row and no directory is
* the newest of those disagreements: MODULES asked for it and resolution could
* not get it, so the screen carries the reason rather than showing nothing.
*/
function present(row, live, onVolume, declaration = null) {
const id = row ? row.id : live ? live.id : declaration.id
return {
id,
name: row ? row.name : live ? live.name : id,
version: row ? row.version : live ? live.version : null,
// What the database records.
state: row ? row.state : null,
failureStage: row ? row.failureStage : (live && live.stage) || null,
failureReason: row ? row.failureReason : (live && live.reason) || null,
source: row ? row.source : null,
sha256: row ? row.sha256 : null,
installedAt: row ? row.installedAt : null,
startedAt: row ? row.startedAt : null,
// What is actually mounted in this process, and what it is answering.
liveState: live ? live.state : null,
// The version RUNNING, which is not always the version installed: an upgrade
// writes new files and a new row while the old code stays loaded until the
// restart. Without this the screen would report the new version as
// "Running", which is the same lie in a different place.
liveVersion: live ? live.version : null,
capabilities: live ? live.capabilities : [],
// What is on the volume.
onVolume,
canPurge: onVolume && Boolean(install.purgeFile(id)),
// What the environment declares. `declaredVersion` is what MODULES pins, not
// what is installed — they differ exactly while a resolution is failing, and
// `declaredError` says why. Uninstalling a declared module from this screen
// removes its directory and disables its row; the next boot puts the
// directory back and leaves the row disabled, so the screen says so rather
// than letting the files reappear unexplained.
declared: Boolean(declaration),
declaredVersion: declaration ? declaration.version : null,
declaredError: declaration && declaration.action === 'failed' ? declaration.message : null,
}
}
// GET /admin/modules — every module core knows about, from all three sources,
// plus the source allowlist the install form needs.
async function list(req, res) {
try {
const rows = await modules.list()
// The loader throws rather than returning [] before load() has run (§7.6),
// and this controller is reachable from a process where that is true —
// `npm run seed` never gets here, but a test harness might.
const live = loader.isLoaded() ? loader.list() : []
const byId = new Map(live.map((m) => [m.id, m]))
const declaredById = new Map(declared.state().map((d) => [d.id, d]))
const seen = new Set()
const out = []
for (const row of rows) {
seen.add(row.id)
out.push(
present(row, byId.get(row.id) || null, install.isInstalled(row.id), declaredById.get(row.id)),
)
}
// A directory on the volume that has no row yet — a hand-placed install
// before its first boot. It has to be listed, or the screen would show
// nothing for a module whose routes are already being served.
for (const m of live) {
if (!seen.has(m.id)) {
seen.add(m.id)
out.push(present(null, m, true, declaredById.get(m.id)))
}
}
// A module MODULES declares that has neither. Resolution failed and left
// nothing behind — the case an operator most needs told, because from the
// screen's other three sources it is indistinguishable from never having
// asked for it.
for (const d of declaredById.values()) {
if (!seen.has(d.id)) out.push(present(null, null, install.isInstalled(d.id), d))
}
return res.json({ modules: out, sourceHosts: await allowedHosts() })
} catch (err) {
log.error('list modules', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/modules — install (or upgrade) from an install-manifest URL.
async function create(req, res) {
const url = String(req.body.url || '').trim()
try {
const hosts = await allowedHosts()
const result = await install.install({ url, hosts })
// Provenance is written here and nowhere else: the boot reconcile records a
// module with NULL source/sha256 and leaves what it is not given, precisely
// so that a refresh cannot overwrite what an install knew (lifecycle.js).
const row = await modules.recordInstalled({
id: result.id,
name: result.name,
version: result.version,
source: result.source,
sha256: result.sha256,
})
await activity.log({
req,
userId: req.user.id,
action: 'module.install',
detail: { id: result.id, version: result.version, source: url, sha256: result.sha256, replaced: result.replaced },
})
log.warn('module installed — it will mount on the next restart', {
id: result.id,
version: result.version,
by: req.user.username,
})
return res.status(201).json({ module: row, restartRequired: true, replaced: result.replaced })
} catch (err) {
if (err.name === 'InstallError' || err.name === 'ArchiveError') {
// The operator pasted a URL and something about what came back was wrong.
// The message is the useful part and is written to be read by them.
log.warn('module install refused', { url, reason: err.message })
return res.status(err.status || 400).json({ message: err.message })
}
log.error('install module', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/modules/:id/enable — switch a module back on, for the next boot.
//
// Deliberately does NOT touch the loader's record. Disable ran the module's
// onShutdown (decision 3), and there is no onBoot re-dispatch to undo that: a
// module whose sockets were closed and timers cleared cannot be made to serve
// again by flipping a flag, and pretending otherwise would put it back on the
// nav with a torn-down world behind it. The row moves; the restart starts it.
async function enable(req, res) {
const { id } = req.params
try {
const row = await modules.enable(id)
if (!row) return res.status(404).json({ message: 'No such module.' })
await activity.log({ req, userId: req.user.id, action: 'module.enable', detail: { id } })
return res.json({ module: row, restartRequired: true })
} catch (err) {
if (err.name === 'ModuleStateError') return res.status(409).json({ message: err.message })
log.error('enable module', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/modules/:id/disable — stop it now.
//
// The one action on this screen that takes effect without a restart, and the
// reason it does is that it is the one an operator reaches for when something is
// going wrong. Its routes answer 404 from the moment this returns, and its
// onShutdown has already run.
async function disable(req, res) {
const { id } = req.params
try {
const current = await modules.get(id)
if (!current) return res.status(404).json({ message: 'No such module.' })
const { stopped, error } = await lifecycle.stop(id)
const row = await modules.get(id)
await activity.log({
req,
userId: req.user.id,
action: 'module.disable',
detail: { id, hookRan: stopped, hookError: error },
})
log.warn('module disabled by an operator', { id, hookRan: stopped, by: req.user.username })
// `shutdownError` is reported rather than swallowed: the module IS disabled
// either way, and an operator whose module could not close cleanly should be
// told so while they still have the logs to look at.
return res.json({ module: row, stopped, shutdownError: error })
} catch (err) {
log.error('disable module', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// DELETE /admin/modules/:id[?purge=true] — uninstall.
//
// Non-destructive by default (§2.5): the directory goes, the row stays
// `disabled`, and the module's tables and data are left alone.
//
// The purge option is here rather than as a follow-up action because it cannot
// be a follow-up action (decision 5): `purge.sql` is a file inside the directory
// this is about to delete, so after an uninstall there is nothing left to purge
// with. Ticking the box is the last moment the file exists.
//
// The order below is the whole of it, and each step depends on the one above:
// purge while the SQL is still readable, stop while the code is still loaded,
// then delete.
async function remove(req, res) {
const { id } = req.params
const purge = req.query.purge === 'true' || req.query.purge === '1'
try {
const current = await modules.get(id)
const onVolume = install.isInstalled(id)
if (!current && !onVolume) return res.status(404).json({ message: 'No such module.' })
let purged = null
if (purge) {
const file = install.purgeFile(id)
if (!file) {
return res.status(400).json({
message: 'This module ships no purge.sql, so its data cannot be deleted. Uninstall without purging instead.',
})
}
purged = await schema.runPurge(file)
}
// Stop it before its files vanish. A module whose directory is deleted out
// from under a running onShutdown is being asked to tear down a world whose
// code may already be half-unreadable — and its sockets would otherwise stay
// open until the restart, holding a connection on behalf of a module that no
// longer exists on disk.
await lifecycle.stop(id)
const removed = await install.removeDir(id)
// A purge leaves nothing: no directory, no tables, no data. Keeping a
// `disabled` row for that is a tombstone with nothing to offer and a Purge
// button that would fail. A plain uninstall keeps its row, which is what
// makes the retained data visible and reinstallable.
if (purge) await modules.remove(id)
await activity.log({
req,
userId: req.user.id,
action: purge ? 'module.purge' : 'module.uninstall',
detail: { id, purged, removed },
})
log.warn(`module ${purge ? 'uninstalled and purged' : 'uninstalled'}`, {
id,
statements: purged,
by: req.user.username,
})
return res.json({ id, removed, purged, restartRequired: true })
} catch (err) {
log.error('uninstall module', err)
return res.status(500).json({ message: err.message || 'Internal Server Error' })
}
}
// POST /admin/modules/:id/purge — drop a still-installed module's data.
//
// Refuses unless the module is already disabled, and that guard is the point:
// dropping the tables under a module that is still serving requests leaves it
// answering out of a world that no longer exists. Disabling first is one click
// and makes the destructive step happen against something that has stopped.
async function purge(req, res) {
const { id } = req.params
try {
const current = await modules.get(id)
if (!current) return res.status(404).json({ message: 'No such module.' })
if (current.state !== 'disabled') {
return res.status(409).json({
message: 'Disable this module before purging its data, so nothing is serving out of the tables being dropped.',
})
}
const file = install.purgeFile(id)
if (!file) {
return res.status(400).json({ message: 'This module ships no purge.sql, so its data cannot be deleted.' })
}
const statements = await schema.runPurge(file)
await activity.log({ req, userId: req.user.id, action: 'module.purge', detail: { id, statements } })
log.warn('module data purged', { id, statements, by: req.user.username })
return res.json({ id, purged: statements })
} catch (err) {
log.error('purge module', err)
return res.status(500).json({ message: err.message || 'Internal Server Error' })
}
}
// PUT /admin/modules/sources — the host allowlist.
async function setSources(req, res) {
const hosts = install.parseHosts(req.body.hosts)
const bad = hosts.find((h) => !HOSTNAME.test(h))
if (bad) return res.status(400).json({ message: `"${bad}" is not a valid hostname.` })
try {
const before = await allowedHosts()
await settings.set(HOSTS_KEY, hosts.join(','), req.user.id)
await activity.log({
req,
userId: req.user.id,
action: 'module.sources',
detail: { before, after: hosts },
})
log.warn('module source allowlist changed', { before, after: hosts, by: req.user.username })
return res.json({ sourceHosts: hosts })
} catch (err) {
log.error('set module sources', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/modules/restart — restart the server process.
//
// Decision 1. Install, uninstall and re-enable all only take effect at boot
// because §1.12 reads the volume at require time, and §2.4 promises recovery
// "with no shell access to the box" — which a banner saying "please restart your
// container" does not deliver.
//
// It reaches server.js's existing SIGTERM handler rather than doing the work
// itself: that handler stops the modules, the workers and the listeners in the
// right order and closes the pool and the log file before exiting 0, and going
// through it means there is exactly one graceful-shutdown path that this route
// cannot drift from.
//
// It gets there by EMITTING the event, not by signalling the process, and that
// is not a detail. `process.kill(process.pid, 'SIGTERM')` is what this did
// first, and it works on Linux — but **Windows has no POSIX signals, and Node
// documents SIGTERM there as unconditional termination of the target process**.
// So on a Windows host the restart killed the server outright: no module
// `onShutdown`, no pool close, no log flush. Verified by running it — the
// process was gone and the shutdown handler had logged nothing.
//
// `process.on('SIGTERM', …)` is an ordinary EventEmitter listener, so
// `process.emit('SIGTERM')` invokes exactly the same handler on every platform
// without involving the OS at all. Deployment is Linux containers and would
// never have shown this; development is not.
//
// What brings the process BACK is the supervisor, not this. The shipped
// docker-compose.yml declares `restart: unless-stopped` on `app`, which restarts
// on a clean exit as well as a crash. A bare `npm start` does not come back, and
// the screen says so before it asks.
function restart(req, res) {
log.warn('restart requested from the admin panel', { by: req.user.username })
// Logged and answered first. Once the signal is raised the response has no
// listener left to flush through, so the operator would be told nothing.
res.status(202).json({ restarting: true })
activity
.log({ req, userId: req.user.id, action: 'module.restart', detail: {} })
.catch((err) => log.error('failed to record the restart in the audit log', err))
.finally(() => {
// A beat, so the 202 is on the wire. `unref` so this timer is not itself
// something keeping the process alive.
setTimeout(() => process.emit('SIGTERM'), 250).unref()
})
}
module.exports = { list, create, enable, disable, remove, purge, setSources, restart, HOSTS_KEY }

143
server/src/router/v1/admin/modules.router.js Normal file
View File

@@ -0,0 +1,143 @@
// Admin · Modules — install, enable, disable, uninstall, purge and restart.
//
// Mounted at /api/v1/admin/modules by admin/index.js, which has already applied
// `noindex, isLoggedIn, staffOnly`. Every route here re-gates to `admin`: an
// editor or moderator has no business installing code into the server process,
// and the group gate alone would let them.
//
// Route order matters in one place. `/restart` and `/sources` are declared
// BEFORE the `/:id/...` routes, because express matches in declaration order and
// a module whose id was `restart` would otherwise shadow — or be shadowed by —
// the literal path. The id pattern below makes that unreachable in practice; the
// ordering makes it unreachable by construction.
const express = require('express')
const { body, param, query } = require('express-validator')
const controller = require('./modules.controller')
const { requireRole } = require('../../../utils/auth')
const validate = require('../../../middleware/validate')
const modulesRouter = express.Router()
const adminOnly = requireRole('admin')
// The loader's own id rule (MODULE_API.md §2.1). Applied at the edge so a
// traversal-shaped id never reaches a path join, even though install.js checks
// it again — this one produces a 400 with a readable message, that one is the
// guarantee.
const ID = /^[a-z][a-z0-9-]{1,31}$/
modulesRouter.get(
'/',
// #swagger.tags = ['Admin · Modules']
// #swagger.summary = 'List installed modules, their live state, and the source allowlist'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Modules and the install source allowlist', content: { "application/json": { schema: { type: "object", properties: { modules: { type: "array", items: { type: "object", additionalProperties: true } }, sourceHosts: { type: "array", items: { type: "string" } } } } } } } */
adminOnly,
controller.list,
)
modulesRouter.post(
'/',
// #swagger.tags = ['Admin · Modules']
// #swagger.summary = 'Install or upgrade a module from a release install-manifest URL'
// #swagger.description = 'Downloads the artifact the manifest names, verifies its sha256, inspects the archive in full and unpacks it onto the modules volume. The module mounts on the next restart.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["url"], properties: { url: { type: "string", description: "https URL of the release install manifest, on an allowed host" } } } } } } */
/* #swagger.responses[201] = { description: 'Installed — restart to mount it', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[400] = { description: 'The URL, the manifest, the hash or the archive was refused', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[502] = { description: 'The source host could not be reached or answered badly', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
body('url').isString().trim().isLength({ min: 1, max: 2048 }),
validate,
controller.create,
)
modulesRouter.put(
'/sources',
// #swagger.tags = ['Admin · Modules']
// #swagger.summary = 'Replace the allowlist of hosts modules may be installed from'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["hosts"], properties: { hosts: { type: "string", description: "Comma- or space-separated hostnames. An empty list forbids all installs." } } } } } } */
/* #swagger.responses[200] = { description: 'The new allowlist', content: { "application/json": { schema: { type: "object", properties: { sourceHosts: { type: "array", items: { type: "string" } } } } } } } */
/* #swagger.responses[400] = { description: 'One of the entries is not a hostname', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
body('hosts').isString().isLength({ max: 2048 }),
validate,
controller.setSources,
)
modulesRouter.post(
'/restart',
// #swagger.tags = ['Admin · Modules']
// #swagger.summary = 'Restart the server process so module changes take effect'
// #swagger.description = 'Runs the same graceful shutdown a SIGTERM does. The process is brought back by the supervisor, which the shipped docker-compose.yml provides; a bare `npm start` will not come back.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[202] = { description: 'Shutting down', content: { "application/json": { schema: { type: "object", properties: { restarting: { type: "boolean" } } } } } } */
adminOnly,
controller.restart,
)
modulesRouter.post(
'/:id/enable',
// #swagger.tags = ['Admin · Modules']
// #swagger.summary = 'Enable a module (takes effect on the next restart)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Module id.' }
/* #swagger.responses[200] = { description: 'Enabled — restart to start it', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[404] = { description: 'No such module', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
param('id').matches(ID),
validate,
controller.enable,
)
modulesRouter.post(
'/:id/disable',
// #swagger.tags = ['Admin · Modules']
// #swagger.summary = 'Stop a module now — runs its onShutdown, then its routes answer 404'
// #swagger.description = 'The only module action that takes effect without a restart. Re-enabling needs one, because there is no onBoot re-dispatch.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Module id.' }
/* #swagger.responses[200] = { description: 'Disabled', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[404] = { description: 'No such module', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
param('id').matches(ID),
validate,
controller.disable,
)
modulesRouter.post(
'/:id/purge',
// #swagger.tags = ['Admin · Modules']
// #swagger.summary = "Run a disabled module’s purge.sql, dropping its tables and data"
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Module id.' }
/* #swagger.responses[200] = { description: 'Purged', content: { "application/json": { schema: { type: "object", properties: { id: { type: "string" }, purged: { type: "integer" } } } } } } */
/* #swagger.responses[400] = { description: 'The module ships no purge.sql', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[409] = { description: 'The module must be disabled first', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
param('id').matches(ID),
validate,
controller.purge,
)
modulesRouter.delete(
'/:id',
// #swagger.tags = ['Admin · Modules']
// #swagger.summary = 'Uninstall a module, optionally deleting its data too'
// #swagger.description = "Removes the module directory and leaves its row disabled. With purge=true it also runs purge.sql first — which is the only moment it can, since purge.sql lives inside the directory being deleted."
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Module id.' }
// #swagger.parameters['purge'] = { in: 'query', required: false, schema: { type: 'boolean' }, description: "Also run the module’s purge.sql and drop its row. Destructive and irreversible." }
/* #swagger.responses[200] = { description: 'Uninstalled — restart to unmount it', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[400] = { description: 'Purge was asked for and the module ships no purge.sql', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[404] = { description: 'No such module', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
param('id').matches(ID),
query('purge').optional().isIn(['true', 'false', '1', '0']),
validate,
controller.remove,
)
module.exports = modulesRouter

2
server/src/router/v1/auth/invite.router.js
View File

@@ -33,7 +33,7 @@ inviteRouter.post(
// #swagger.tags = ['Auth']
// #swagger.summary = 'Accept an email invite (creates the account at the invited role)'
// #swagger.description = 'Creates the website user at the invite’s pre-assigned role and logs them in (sets the session cookie). Bypasses the player_registration gate — the invite is its own authority. Rate limited + honeypot-guarded like registration.'
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["username","password"], properties: { username: { type: "string" }, password: { type: "string" } } } } } */
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["username","password"], properties: { username: { type: "string" }, password: { type: "string" } } } } } } */
/* #swagger.responses[200] = { description: 'Account created and session issued', content: { "application/json": { schema: { $ref: "#/components/schemas/LoginResponse" } } } } */
/* #swagger.responses[400] = { description: 'Validation error', content: { "application/json": { schema: { $ref: "#/components/schemas/ValidationError" } } } } */
/* #swagger.responses[404] = { description: 'Invalid or expired invite', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */

45
server/src/server.js
View File

@@ -1,8 +1,12 @@
require('dotenv').config()
const http = require('http')
const app = require('./app')
const internalApp = require('./internalApp')
// NOTE: `./app` and `./internalApp` are deliberately NOT required here. Requiring
// app.js runs `modules.load()`, which scans the volume and mounts whatever is on
// it (MODULE_API.md §4.1) — so the declared module set has to be resolved before
// that require, not before the listener. They are required inside start(), after
// resolveDeclaredModules(); everything else this file needs is safe to pull in
// now because none of it reaches the loader's scan.
const botScore = require('./middleware/botScore')
const announceWorker = require('./utils/announceWorker')
const { ensureSchema, close } = require('./utils/db')
@@ -11,6 +15,9 @@ const settings = require('./model/settings/settings.model')
const revokedSessions = require('./model/revokedSessions/revokedSessions.model')
const mobileAuthBridge = require('./model/mobileAuthBridge/mobileAuthBridge.model')
const moduleLifecycle = require('./modules/lifecycle')
const declaredModules = require('./modules/declared')
const moduleInstall = require('./modules/install')
const moduleModel = require('./model/modules/modules.model')
const createLogger = require('./utils/logger')
const { evaluateBotInternalKey } = require('./utils/botInternalKey')
const brand = require('./config/brand')
@@ -51,7 +58,9 @@ async function start() {
}
log.info('ensuring database schema...')
await ensureSchema()
// Core's schema only. Each installed module's fragment is replayed further
// down, after the volume has been scanned — see the require of ./app below.
await ensureSchema({ replayModules: false })
log.info('seeding defaults...')
await seedDefaults()
await createInitialAdminFromEnv()
@@ -77,6 +86,36 @@ async function start() {
const mode = await settings.get('site_mode')
log.info(`site mode: ${String(mode || 'live').toUpperCase()}`)
// Bring the modules volume in line with what MODULES declares (§2.7.2
// decision 4), and only then require the app — the loader scans and mounts at
// require time, so this is the last moment at which a module can be put on the
// volume and still be part of this process.
//
// After the schema and the seed, because the host allowlist it installs under
// is a settings row that the seed creates on a fresh instance. Never throws:
// an unreachable release host leaves the site serving without that module
// rather than taking the site down with it.
await declaredModules.resolve({
hosts: moduleInstall.parseHosts(await settings.get(moduleInstall.HOSTS_SETTING)),
model: moduleModel,
})
// Requiring app.js is what scans the volume and mounts what is on it. Every
// line above this one runs against a core that has no modules in it yet.
// eslint-disable-next-line global-require
const app = require('./app')
// eslint-disable-next-line global-require
const internalApp = require('./internalApp')
// Now that the scan has happened, replay each module's schema fragment
// (MODULE_API.md §2.6). This used to ride inside ensureSchema() and could,
// because app.js was required at the top of this file; resolving the declared
// set first moved the scan after it, and a booting server quietly getting no
// module tables is precisely what §7.6 warns about. Caught by the browser
// smoke rather than by a test: every suite here stubs one side or the other.
// eslint-disable-next-line global-require
await require('./modules/schema').replayFragments()
// Reconcile installed_modules with what the loader found on the volume at
// require time, then run each module's onBoot (MODULE_API.md §2.5).
//

11
server/src/utils/db.js
View File

@@ -55,11 +55,18 @@ const SCHEMA_PATH = path.join(__dirname, '..', '..', 'db', 'schema.sql')
* below — the discovery, splitting and per-module failure handling all live in
* modules/schema.js, required lazily so that requiring the pool never drags the
* loader in with it.
*
* `replayModules: false` is for a caller that has not scanned the volume YET and
* intends to. server.js is the one: since slice 3 it resolves the declared
* module set before requiring app.js, which puts core's schema *before* the scan
* — so it replays the fragments itself, in the one place that knows the scan has
* happened. Left true everywhere else, so the ordinary caller cannot get module
* tables by accident and lose them by refactor.
*/
async function ensureSchema({ retries = 10, delayMs = 2000 } = {}) {
async function ensureSchema({ retries = 10, delayMs = 2000, replayModules = true } = {}) {
await ensureCoreSchema({ retries, delayMs })
// eslint-disable-next-line global-require
await require('../modules/schema').replayFragments()
if (replayModules) await require('../modules/schema').replayFragments()
}
/** Core's own schema.sql, with the wait-for-the-database retry. */

100
server/swagger/docsSpec.js Normal file
View File

@@ -0,0 +1,100 @@
// ── The OpenAPI document core actually serves ──────────────────────────────
//
// `swagger-output.json` is core's own routes and only core's own routes: it is
// generated by `npm run swagger` on a developer's machine and committed, so it
// must come out the same regardless of which modules that developer happened to
// have checked out. A module's routes cannot be in it, and not merely because
// nobody put them there — a module arrives on a volume long after the image was
// built, and core never has its sources to analyse.
//
// So the document served at `/api/docs.json` is assembled at REQUEST time: core's
// committed spec, plus the `swagger-fragment.json` of every started module
// (docs/website/MODULE_API.md §2.8 and §6.1a). This file is that assembly.
//
// **Core always wins a key collision.** `mergeFragment` enforces it and reports
// what it dropped. A module cannot redefine a core path, tag or schema by shipping
// one with the same name — which is why §6.1a tells modules to namespace the
// schemas they define (`UoShardStatus`) while referencing core's shared ones
// (`Error`) by core's name: the first would collide and lose, the second resolves
// here, in the merged document, which is the only place both exist.
//
// **Cached, keyed on the loader's state version.** Building the document reads a
// file per module and deep-copies a 5,000-line spec; `/api/docs` is a page an
// operator opens occasionally and a crawler may hit repeatedly. The cache is
// invalidated by any module state CHANGE — which is what "started modules only"
// depends on, and the only input here that can move without a restart.
const fs = require('fs')
const modules = require('../src/modules/loader')
const createLogger = require('../src/utils/logger')
const { mergeFragment } = require('./mergeSpec')
const log = createLogger('swagger')
let cached = null
let cachedVersion = -1
/**
* Core's spec with every started module's fragment merged over it.
*
* Never throws: `/api/docs.json` answering with core's routes alone is a worse
* document than the full one, but it is a document. A fragment that is missing,
* unreadable or not JSON costs that module its paths and nothing else — the same
* bargain §4.4 makes everywhere else, where one module's failure is never the
* site's.
*
* @param {object} coreSpec the committed swagger-output.json — never mutated
* @returns {object}
*/
function docsSpec(coreSpec) {
// Before app.js has called modules.load(), asking is a mis-ordered boot rather
// than a core with nothing installed (§7.6) — but this is a request handler, and
// 500ing the docs page over it would be the wrong trade. Core's own spec is the
// honest answer to "what is documented" at that point anyway.
if (!modules.isLoaded()) return coreSpec
const version = modules.version()
if (cached && cachedVersion === version) return cached
// A structural copy, because mergeFragment writes into what it is given and
// `coreSpec` is a require()d JSON module: mutating it would make the merge
// cumulative across rebuilds and permanent for the life of the process.
const spec = JSON.parse(JSON.stringify(coreSpec))
spec.paths = spec.paths || {}
spec.tags = spec.tags || []
spec.components = spec.components || {}
spec.components.schemas = spec.components.schemas || {}
for (const { id, file } of modules.specFragments()) {
let fragment
try {
fragment = JSON.parse(fs.readFileSync(file, 'utf8'))
} catch (err) {
log.warn('module OpenAPI fragment could not be read — its routes will be undocumented', {
module: id,
file,
error: err.message,
})
continue
}
const before = Object.keys(spec.paths).length
mergeFragment(spec, fragment, `module ${id}`)
log.debug('merged module OpenAPI fragment', {
module: id,
paths: Object.keys(spec.paths).length - before,
})
}
cached = spec
cachedVersion = version
return spec
}
/** Test seam: forget the cached document. */
function reset() {
cached = null
cachedVersion = -1
}
module.exports = { docsSpec, reset }

5042
server/swagger/swagger-output.json
View File

File diff suppressed because it is too large Load Diff

629
server/swagger/swagger.js
View File

@@ -33,8 +33,11 @@ const doc = {
title: `${brand.name} API`,
version: pkg.version,
description:
`REST API for the ${brand.name} website, wiki and admin panel — a private ` +
'Ultima Online shard.\n\n' +
`REST API for the ${brand.name} website, wiki and admin panel.\n\n` +
'This document is core. Installed modules add their own paths, tags and ' +
'schemas to it at request time from the fragment each one ships, so ' +
'`/api/docs.json` on a running instance describes more than `npm run swagger` ' +
'generates here (docs/website/MODULE_API.md §6.1a).\n\n' +
'### Authentication\n' +
`- **Web / admin panel** uses an httpOnly session cookie (\`${COOKIE_NAME}\`) issued by ` +
'`POST /api/v1/auth/login` (plus `/login/totp` when 2FA is enabled).\n' +
@@ -47,27 +50,33 @@ const doc = {
{ url: '/', description: 'Same-origin (current host)' },
{ url: 'http://localhost:3000', description: 'Local development' },
],
// Core's tags only. A module contributes its own in its fragment, and they are
// merged in beside these — the four game-specific ones that used to sit here
// (`Public · Shard`, `Public · Atlas`, `Player · Shard`, `Admin · Shard`) went
// with the routes they group, and arrive back from module-uo on any instance
// that has it installed.
tags: [
{ name: 'Health', description: 'Liveness probe' },
{ name: 'Auth', description: 'Web session login/logout (cookie + TOTP)' },
{ name: 'Auth · Me', description: 'The signed-in account: profile, notification streams and devices' },
{ name: 'Auth · Mobile', description: 'Native bearer-token login, refresh and logout' },
{ name: 'Auth · SSO', description: 'OAuth2 / OIDC provider discovery and redirect flow' },
{ name: 'Public', description: 'Unauthenticated site content (settings, posts, wiki, contact)' },
{ name: 'Public · Shard', description: 'Live shard data ingested from the uo-link sidecar (status, feed, economy, IDOC, characters)' },
{ name: 'Public · Atlas', description: 'Spawn atlas / bestiary — static shard content parsed from the shard\'s own ServUO tree, independent of the sidecar' },
{ name: 'Admin · Account', description: 'Self-service account security (2FA, linked identities)' },
{ name: 'Player', description: 'Self-service player accounts (register, credentials, 2FA, linked identities)' },
{ name: 'Player · Shard', description: 'Link an in-game account and read its roster / vendors (uo-link)' },
{ name: 'Player · Appeals', description: 'Player-submitted moderation appeals' },
{ name: 'Settings', description: 'Site-wide settings any authenticated account may read (nav overrides)' },
{ name: 'Admin · Dashboard', description: 'Dashboard summary and site mode' },
{ name: 'Admin · Posts', description: 'News / five-on-friday / newsletter / screenshots + uploads' },
{ name: 'Admin · Pages', description: 'Editable static site pages' },
{ name: 'Admin · Wiki', description: 'Wiki pages, categories, tags and revisions' },
{ name: 'Admin · Settings', description: 'Site settings (admin only)' },
{ name: 'Admin · Email', description: 'Outbound email configuration and delivery test (admin only)' },
{ name: 'Admin · Invites', description: 'Registration invites — issue, list and revoke' },
{ name: 'Admin · Moderation', description: 'Player reports, appeals and moderator actions' },
{ name: 'Admin · Activity', description: 'Admin activity log' },
{ name: 'Admin · Bot Activity', description: 'Bot-scoring/ban state and emergency unban (admin only)' },
{ name: 'Admin · Discord Bot', description: 'Discord bot token/config and live status (admin only)' },
{ name: 'Admin · Shard', description: 'uo-link sidecar connection config, live status and town crier (admin only)' },
{ name: 'Admin · Auth Providers', description: 'SSO provider configuration (admin only)' },
{ name: 'Admin · Users', description: 'User management (admin only)' },
],
@@ -918,584 +927,6 @@ const doc = {
removed: { type: 'boolean', description: 'Whether the IP had an entry that was cleared.', example: true },
},
},
// ── uo-link shard data ──────────────────────────────────────────────
ShardStatus: {
type: 'object',
description: 'Public shard status (GET /public/shard/status).',
properties: {
enabled: { type: 'boolean', example: true },
status: { type: 'string', example: 'connected', description: 'connected | reconnecting | disconnected | error' },
pluginConnected: { type: 'boolean', description: 'Is the shard link up right now?', example: true },
lastEventAt: { type: 'string', format: 'date-time', nullable: true },
onlineCount: { type: 'integer', example: 12 },
economy: { $ref: '#/components/schemas/ShardEconomyPoint' },
},
},
ShardEvent: {
type: 'object',
description: 'A logged shard event.',
properties: {
id: { type: 'integer', example: 4821 },
kind: { type: 'string', example: 'vendor.sale' },
t: { type: 'integer', description: 'Event time, epoch ms.', example: 1783720195626 },
bootId: { type: 'string', nullable: true, example: 'boot-abc123' },
payload: { type: 'object', additionalProperties: true, description: 'The full event object.' },
createdAt: { type: 'string', format: 'date-time' },
},
},
ShardEconomyPoint: {
type: 'object',
nullable: true,
description: 'One gold-supply sample.',
properties: {
accounts: { type: 'integer', nullable: true, example: 240 },
gold: { type: 'integer', nullable: true, example: 1028983421 },
t: { type: 'integer', description: 'Sample time, epoch ms.', example: 1783720000000 },
},
},
ShardOnlinePlayer: {
type: 'object',
description: 'A LINKED player online now (only accounts linked to a website user are listed).',
properties: {
serial: { type: 'string', example: '0x24C' },
name: { type: 'string', example: 'Darrow' },
map: { type: 'string', nullable: true, example: 'Trammel' },
x: { type: 'integer', nullable: true, example: 1402 },
y: { type: 'integer', nullable: true, example: 1604 },
z: { type: 'integer', nullable: true, example: 0 },
},
},
ShardVendorSale: {
type: 'object',
description: 'A player-vendor sale (visible only to the linked owner).',
properties: {
t: { type: 'integer', description: 'Sale time, epoch ms.', example: 1783720195626 },
itemType: { type: 'string', example: 'Longsword' },
amount: { type: 'integer', example: 1 },
price: { type: 'integer', example: 100 },
commission: { type: 'integer', nullable: true, example: 5 },
ownerAcct: { type: 'string', example: 'whitlocktech' },
},
},
ShardHouse: {
type: 'object',
description: 'A house at its current decay stage.',
properties: {
serial: { type: 'string', example: '0x4004705F' },
stage: { type: 'string', example: 'IDOC' },
map: { type: 'string', nullable: true, example: 'Trammel' },
x: { type: 'integer', nullable: true },
y: { type: 'integer', nullable: true },
z: { type: 'integer', nullable: true },
region: { type: 'string', nullable: true },
name: { type: 'string', nullable: true, example: 'An Unnamed House' },
ownerSerial: { type: 'string', nullable: true },
ownerAcct: { type: 'string', nullable: true },
builtOn: { type: 'string', format: 'date-time', nullable: true },
lastRefreshed: { type: 'string', format: 'date-time', nullable: true },
isIdoc: { type: 'boolean', example: true },
updatedAt: { type: 'string', format: 'date-time' },
},
},
ShardPointsBoard: {
type: 'object',
description:
"One point system's leaderboard (Protocol 3.0 points.board). The shard carries ~25 separate point currencies; each publishes its own board. The display name may arrive as a literal string, a cliloc id, or both — resolve clilocs client-side.",
properties: {
system: { type: 'string', example: 'QueensLoyalty', description: "The shard's PointsType name; the board's stable key." },
nameString: { type: 'string', nullable: true, example: "Queen's Loyalty" },
nameNumber: { type: 'integer', nullable: true, example: 1114938, description: 'Cliloc id, 0 when the name is a literal.' },
maxPoints: { type: 'integer', nullable: true, example: 30000 },
players: { type: 'integer', nullable: true, example: 842, description: 'Players actually holding points in this system.' },
showOnGump: { type: 'boolean', example: true, description: "The shard's own 'is this player-facing?' flag." },
top: {
type: 'array',
description: 'The ranked players, best first. Capped by the shard (10 by default). Empty when nobody has scored yet.',
items: {
type: 'object',
properties: {
rank: { type: 'integer', example: 1 },
serial: { type: 'string', example: '0x1A2B' },
name: { type: 'string', example: 'Darrow', description: 'Omitted when the leaderboards `name` field is gated above the caller.' },
points: { type: 'integer', example: 29500 },
},
},
},
t: { type: 'integer', nullable: true, description: 'Frame time, epoch ms.' },
updatedAt: { type: 'string', format: 'date-time' },
},
},
ShardMarketLocation: {
type: 'object',
nullable: true,
description:
"Where a vendor is standing. ONE nested object rather than flat map/x/y/region because it is one admin-configurable field (`market.location`) — the whole object is omitted when that field is gated above the caller.",
properties: {
map: { type: 'string', nullable: true, example: 'Trammel' },
x: { type: 'integer', nullable: true, example: 1421 },
y: { type: 'integer', nullable: true, example: 1699 },
z: { type: 'integer', nullable: true, example: 0 },
region: { type: 'string', nullable: true, example: 'Britain' },
house: { type: 'string', nullable: true, example: "Darrow's Villa", description: "The house SIGN's name, not the house type. Null for a vendor standing outside one." },
},
},
ShardMarketListing: {
type: 'object',
description:
'One priced listing on a player vendor, carrying enough of its shop to be actionable without a second request.',
properties: {
serial: { type: 'string', example: '0x40012ABC' },
itemId: { type: 'integer', example: 3922, description: 'ItemID (the art/graphic id).' },
hue: { type: 'integer', example: 0 },
amount: { type: 'integer', example: 1 },
price: { type: 'integer', example: 25000 },
name: { type: 'string', nullable: true, description: "The item's own literal name, set by a player. Null for most items." },
cliloc: { type: 'integer', nullable: true, example: 1023721, description: "The item's LabelNumber." },
displayName: {
type: 'string',
nullable: true,
example: 'quarter staff',
description: 'Resolved server-side from `name` (preferred, being player-set and more specific) else `cliloc`. Null on a shard with no cliloc table configured — render the item id.',
},
child: { type: 'boolean', example: false, description: 'Priced by an enclosing container rather than itself, exactly as the in-game Vendor Search reports it.' },
vendor: {
type: 'object',
properties: {
serial: { type: 'string', example: '0x40001234' },
shopName: { type: 'string', nullable: true, example: "Darrow's Bargains" },
ownerSerial: { type: 'string', nullable: true, example: '0x1A2B', description: 'Omitted when the market `ownerSerial` field is gated above the caller.' },
ownerName: { type: 'string', nullable: true, example: 'Darrow', description: 'Omitted when the market `ownerName` field is gated above the caller.' },
location: { $ref: '#/components/schemas/ShardMarketLocation' },
updatedAt: { type: 'string', format: 'date-time', description: 'When the shard last published this shop.' },
},
},
},
},
ShardMarketPage: {
type: 'object',
description: 'A page of marketplace listings plus the unpaginated total and the staleness stamp.',
properties: {
listings: { type: 'array', items: { $ref: '#/components/schemas/ShardMarketListing' } },
total: { type: 'integer', example: 1284, description: 'Matching listings, ignoring paging.' },
limit: { type: 'integer', example: 50 },
offset: { type: 'integer', example: 0 },
vendors: { type: 'integer', example: 137, description: 'Vendors in the whole index.' },
staleAt: {
type: 'string',
format: 'date-time',
nullable: true,
description: 'The OLDEST vendor row. The shard sweeps vendors round-robin, so the index can be a full cycle behind and a client must say so rather than implying live prices.',
},
},
},
ShardMarketVendor: {
type: 'object',
description: 'One player vendor and its listings.',
properties: {
serial: { type: 'string', example: '0x40001234' },
shopName: { type: 'string', nullable: true, example: "Darrow's Bargains" },
ownerSerial: { type: 'string', nullable: true },
ownerName: { type: 'string', nullable: true, example: 'Darrow' },
location: { $ref: '#/components/schemas/ShardMarketLocation' },
count: { type: 'integer', example: 250, description: 'Listings the shard published for this shop.' },
total: { type: 'integer', example: 3104, description: 'Listings the shop actually holds.' },
truncated: { type: 'boolean', example: true, description: '`total` exceeds `count` — the shop holds more than the shard publishes per frame.' },
updatedAt: { type: 'string', format: 'date-time' },
items: { type: 'array', items: { $ref: '#/components/schemas/ShardMarketListing' } },
},
},
ShardMarketMeta: {
type: 'object',
description: 'Marketplace size, staleness and the filter options a client needs to build its UI.',
properties: {
vendors: { type: 'integer', example: 137 },
items: { type: 'integer', example: 18422 },
staleAt: { type: 'string', format: 'date-time', nullable: true },
freshAt: { type: 'string', format: 'date-time', nullable: true },
maps: { type: 'array', items: { type: 'string' }, example: ['Felucca', 'Trammel'], description: "Facets that actually hold vendors. From the shard's own data — never a hardcoded list." },
regions: { type: 'array', items: { type: 'string' }, example: ['Britain', 'Luna'] },
},
},
ShardFeatures: {
type: 'object',
description:
"The shard features the caller may reach, plus the audience rung they resolved to. Drives client nav so it never renders a link that would 403.",
properties: {
level: {
type: 'string',
enum: ['anonymous', 'logged_in', 'player', 'staff', 'admin'],
example: 'anonymous',
},
features: {
type: 'array',
items: { type: 'string' },
example: ['status', 'activity', 'champs', 'guilds', 'governors', 'houses', 'presence'],
},
},
},
ShardFeatureVisibility: {
type: 'object',
description: 'Visibility settings for one shard feature.',
properties: {
enabled: { type: 'boolean', example: true },
audience: {
type: 'string',
enum: ['anonymous', 'logged_in', 'player', 'staff', 'admin'],
description: 'Minimum rung that may reach this feature. Each rung implies the ones below it.',
example: 'anonymous',
},
stream: {
type: 'boolean',
description: "Whether this feature's event kinds fan out over SSE at all.",
example: true,
},
fieldRules: {
type: 'object',
additionalProperties: { type: 'string' },
description:
'Per-field rung overrides for the sensitive fields this feature exposes. acct / webId are admin-only always and are rejected here.',
example: { location: 'staff' },
},
},
},
ShardVisibilityConfig: {
type: 'object',
properties: {
ladder: {
type: 'array',
items: { type: 'string' },
example: ['anonymous', 'logged_in', 'player', 'staff', 'admin'],
},
lockedFields: { type: 'array', items: { type: 'string' }, example: ['acct', 'webId'] },
defaults: {
type: 'object',
additionalProperties: { $ref: '#/components/schemas/ShardFeatureVisibility' },
},
features: {
type: 'object',
additionalProperties: { $ref: '#/components/schemas/ShardFeatureVisibility' },
},
},
},
ShardVisibilityUpdate: {
type: 'object',
required: ['features'],
properties: {
features: {
type: 'object',
additionalProperties: { $ref: '#/components/schemas/ShardFeatureVisibility' },
example: { market: { enabled: true, audience: 'player', stream: false, fieldRules: { ownerName: 'player' } } },
},
},
},
// ── Spawn atlas (Protocol 3.0 Part C) ────────────────────────────────
// Static shard content, parsed from the shard's own ServUO tree. Nothing
// here comes from the sidecar, so it stays populated while the shard is
// down. Facet names are whatever the shard's files declare — the examples
// below are stock ServUO, not a fixed list.
AtlasCreature: {
type: 'object',
description: 'A creature in the bestiary. `places`/`points`/`alsoHere` are present only on the single-creature route.',
properties: {
slug: { type: 'string', example: 'lizardman' },
name: { type: 'string', example: 'Lizardman' },
total: { type: 'integer', description: 'How many can be alive at once, summed across every spawner.', example: 214 },
points: { type: 'integer', description: 'How many spawners mention this creature.', example: 62 },
facets: {
type: 'object',
additionalProperties: { type: 'integer' },
description: "This creature's share per facet.",
example: { Felucca: 96, Trammel: 88, Tokuno: 30 },
},
art: { type: 'string', nullable: true, description: 'Operator-supplied art under uploads/atlas/. NULL on a fresh import — the repo ships no creature art.' },
places: {
type: 'array',
description: 'Where it spawns, aggregated by resolved place. The answer the atlas exists to give.',
items: {
type: 'object',
properties: {
facet: { type: 'string', example: 'Trammel' },
label: { type: 'string', description: 'Resolved region, else nearest landmark group, else "Wilderness".', example: 'Shrines' },
spawners: { type: 'integer', example: 7 },
maxAlive: { type: 'integer', example: 21 },
},
},
},
spawners: {
type: 'array',
description: 'The individual spawners. Named separately from `points` (the count) so one key never means two things.',
items: { $ref: '#/components/schemas/AtlasSpawner' },
},
spawnersTruncated: { type: 'boolean', description: 'True when the spawner list was cut at the requested bound.', example: false },
alsoHere: {
type: 'array',
description: 'Creatures sharing a spawner with this one.',
items: {
type: 'object',
properties: {
slug: { type: 'string', example: 'lizardman-warrior' },
name: { type: 'string', example: 'Lizardman Warrior' },
shared: { type: 'integer', example: 12 },
},
},
},
},
},
AtlasSpawner: {
type: 'object',
description: 'One ServUO spawner, with the place its coordinates resolved to.',
properties: {
id: { type: 'integer' },
facet: { type: 'string', example: 'Felucca' },
name: { type: 'string', nullable: true, description: "The spawner's own name in the ServUO file." },
x: { type: 'integer', example: 5411 },
y: { type: 'integer', example: 1234 },
width: { type: 'integer' },
height: { type: 'integer' },
range: { type: 'integer', description: 'Spawn radius.' },
maxCount: { type: 'integer', description: 'How many of THIS creature this spawner keeps alive.', example: 3 },
minDelay: { type: 'integer', description: 'Respawn window, in SECONDS. Normalised at parse time — the source stores minutes or seconds per record, decided by its own DelayInSec flag.', example: 300 },
maxDelay: { type: 'integer', example: 600 },
todStart: { type: 'integer', description: 'Meaningless unless todMode is non-zero.' },
todEnd: { type: 'integer' },
todMode: { type: 'integer' },
region: { type: 'string', nullable: true, example: 'Despise' },
landmark: { type: 'string', nullable: true, example: 'Covetous' },
label: { type: 'string', description: 'Region, else landmark group, else "Wilderness".', example: 'Despise' },
},
},
AtlasCreaturePage: {
type: 'object',
properties: {
total: { type: 'integer', description: 'Matching creatures before pagination.', example: 800 },
limit: { type: 'integer', example: 50 },
offset: { type: 'integer', example: 0 },
creatures: { type: 'array', items: { $ref: '#/components/schemas/AtlasCreature' } },
},
},
AtlasRegion: {
type: 'object',
description: 'A named region, flattened out of the shard\'s nested Regions.xml.',
properties: {
facet: { type: 'string', example: 'Felucca' },
name: { type: 'string', example: 'Despise' },
type: { type: 'string', nullable: true, description: 'ServUO region class.', example: 'DungeonRegion' },
priority: { type: 'integer', example: 50 },
parent: { type: 'string', nullable: true, example: 'Britain' },
rects: {
type: 'array',
description: 'The rectangles that placed each spawn point.',
items: { type: 'object', additionalProperties: true },
},
},
},
AtlasLandmark: {
type: 'object',
properties: {
facet: { type: 'string', example: 'Trammel' },
name: { type: 'string', example: 'Level 1' },
group: { type: 'string', nullable: true, description: 'Innermost enclosing parent — the label worth showing.', example: 'Covetous' },
x: { type: 'integer', example: 5411 },
y: { type: 'integer', example: 1234 },
z: { type: 'integer', example: 0 },
},
},
AtlasChampion: {
type: 'object',
description: 'A CONFIGURED champion altar. Not the live board — see GET /public/shard/champs for that.',
properties: {
slug: { type: 'string', example: 'felucca-deceit' },
name: { type: 'string', example: 'Deceit' },
group: { type: 'string', nullable: true, description: 'Spawn group; one altar active per group.', example: 'Dungeons' },
type: { type: 'string', nullable: true, description: 'NULL when the champion is drawn at activation.', example: 'UnholyTerror' },
randomType: { type: 'boolean', example: false },
facet: { type: 'string', example: 'Felucca' },
x: { type: 'integer' },
y: { type: 'integer' },
z: { type: 'integer' },
radius: { type: 'integer', example: 60 },
label: { type: 'string', nullable: true, example: 'Deceit' },
},
},
AtlasMeta: {
type: 'object',
description: 'What atlas is loaded. Game-world facts only: the ServUO path, source hashes and any pending refresh are operator detail and live on the admin status route.',
properties: {
importedAt: { type: 'string', format: 'date-time', nullable: true },
generatedAt: { type: 'string', format: 'date-time', nullable: true },
counts: {
type: 'object',
nullable: true,
additionalProperties: true,
example: { facets: 6, points: 6455, creatures: 800, regions: 387, landmarks: 558, champions: 25, unresolvedPoints: 1086 },
},
facets: { type: 'array', items: { type: 'string' }, example: ['Felucca', 'Ilshenar', 'Malas', 'TerMur', 'Tokuno', 'Trammel'] },
},
},
AtlasStatus: {
type: 'object',
description: 'Admin view of atlas state: where the tree is, whether it is readable, whether it has drifted from what is loaded, and any refresh staged for review.',
properties: {
configured: { type: 'boolean', example: true },
path: { type: 'string', example: '/srv/servuo' },
treeReadable: { type: 'boolean', example: true },
drift: { type: 'boolean', nullable: true, description: 'True when the tree\'s source hashes differ from the loaded atlas. NULL when the tree could not be read.', example: false },
facets: { type: 'array', items: { type: 'string' } },
importedAt: { type: 'string', format: 'date-time', nullable: true },
counts: { type: 'object', nullable: true, additionalProperties: true },
pending: {
type: 'object',
nullable: true,
description: 'A refresh that was parsed but NOT applied because it would remove a facet. `status` is pending or rejected.',
additionalProperties: true,
},
},
},
AtlasRefreshResult: {
type: 'object',
description: 'Outcome of a refresh. Reported rather than thrown, so an unreadable tree is an answer and not a 500.',
properties: {
status: {
type: 'string',
enum: ['skipped', 'unavailable', 'unchanged', 'imported', 'needsReview', 'failed', 'rejected', 'none'],
example: 'imported',
},
reason: { type: 'string', nullable: true },
path: { type: 'string', nullable: true },
counts: { type: 'object', nullable: true, additionalProperties: true },
addedFacets: { type: 'array', items: { type: 'string' } },
removedFacets: { type: 'array', items: { type: 'string' } },
},
},
ClilocStatus: {
type: 'object',
description:
'Admin view of cliloc state: where the converted file is, whether it is readable, how many entries are loaded, and whether the file has drifted from them. `configured: false` is a supported state — item names then render as ids.',
properties: {
configured: { type: 'boolean', example: true },
path: { type: 'string', example: '/srv/uo-client' },
file: { type: 'string', nullable: true, description: 'The file actually resolved, when the path is a directory.', example: '/srv/uo-client/clilocs.tsv' },
fileReadable: { type: 'boolean', example: true },
problem: { type: 'string', nullable: true, description: 'Why the file cannot be used, when it cannot. Set (with code COMPRESSED) for a readable-but-unconverted client file.', example: null },
code: { type: 'string', nullable: true, description: 'Machine-readable cause of `problem`.', enum: ['NO_PATH', 'NOT_FOUND', 'NO_FILE', 'UNREADABLE', 'COMPRESSED'] },
drift: { type: 'boolean', nullable: true, description: 'True when any source hash differs from the loaded table. NULL when the sources could not be read or are not usable.', example: false },
count: { type: 'integer', description: 'Entries currently loaded.', example: 67496 },
sources: {
type: 'array',
items: { type: 'string' },
description: 'Every source found now, root-relative, base first then overlays in merge order.',
example: ['clilocs.plain', 'custom/uomysticmoon.tsv'],
},
loadedSources: {
type: 'array',
nullable: true,
description: 'What each source contributed at the last import.',
items: {
type: 'object',
properties: {
label: { type: 'string', example: 'custom/uomysticmoon.tsv' },
kind: { type: 'string', enum: ['base', 'custom'], example: 'custom' },
entries: { type: 'integer', example: 37 },
added: { type: 'integer', description: 'Ids this source introduced.', example: 25 },
overrode: { type: 'integer', description: 'Ids it replaced from an earlier source.', example: 12 },
},
},
},
missingSources: {
type: 'array',
items: { type: 'string' },
description: 'Sources loaded previously and now absent. An import refuses these without `approve`.',
example: [],
},
importedAt: { type: 'string', format: 'date-time', nullable: true },
sourceBytes: { type: 'integer', nullable: true, example: 4973525 },
},
},
ClilocRefreshResult: {
type: 'object',
description:
'Outcome of a cliloc refresh. Reported rather than thrown, so a missing or compressed file is an answer and not a 500.',
properties: {
status: {
type: 'string',
enum: ['skipped', 'unavailable', 'unchanged', 'imported', 'needsReview', 'failed'],
description: '`needsReview` means a previously-loaded source has vanished and nothing was applied; re-run with `approve` to accept it.',
example: 'imported',
},
reason: { type: 'string', nullable: true },
code: {
type: 'string',
nullable: true,
description: 'Machine-readable cause. `COMPRESSED` means the client\'s own Cliloc.enu was supplied instead of a converted one.',
enum: ['NO_PATH', 'NOT_FOUND', 'NO_FILE', 'UNREADABLE', 'COMPRESSED', 'TRUNCATED', 'EMPTY', 'NOT_BUFFER'],
},
path: { type: 'string', nullable: true },
file: { type: 'string', nullable: true },
count: { type: 'integer', nullable: true, description: 'Entries stored (blank strings are dropped).', example: 67496 },
parsed: { type: 'integer', nullable: true, description: 'Entries read across every source before blanks were dropped.', example: 123527 },
blank: { type: 'integer', nullable: true, example: 55994 },
sources: {
type: 'array',
nullable: true,
description: 'Per-source breakdown: what each file contributed and how much of it overrode an earlier source.',
items: {
type: 'object',
properties: {
label: { type: 'string' },
kind: { type: 'string', enum: ['base', 'custom'] },
entries: { type: 'integer' },
added: { type: 'integer' },
overrode: { type: 'integer' },
},
},
},
missingSources: {
type: 'array',
nullable: true,
items: { type: 'string' },
description: 'On `needsReview`: the sources that vanished. Nothing was applied.',
},
acceptedMissing: {
type: 'array',
nullable: true,
items: { type: 'string' },
description: 'On `imported` with `approve`: the vanished sources the admin accepted.',
},
},
},
ShardLinkRequest: {
type: 'object',
required: ['code'],
properties: {
code: { type: 'string', description: 'The one-time code shown by [link in game.', example: 'AB12CD' },
},
},
ShardLinkResult: {
type: 'object',
properties: {
linked: { type: 'boolean', example: true },
account: { type: 'string', example: 'whitlocktech' },
},
},
ShardLink: {
type: 'object',
description: 'A linked in-game account (GET /player/shard/accounts).',
properties: {
account: { type: 'string', example: 'whitlocktech' },
userId: { type: 'integer', example: 42 },
charName: { type: 'string', nullable: true, example: 'Darrow' },
linkedAt: { type: 'string', format: 'date-time' },
},
},
TownCrierRequest: {
type: 'object',
required: ['id', 'lines'],
properties: {
id: { type: 'string', maxLength: 64, description: 'Re-posting the same id replaces the prior entry.', example: 'news-42' },
lines: { type: 'array', items: { type: 'string', maxLength: 200 }, example: ['Hear ye!', 'Market tax is now 5%.'] },
durationSec: { type: 'integer', minimum: 1, maximum: 86400, example: 3600 },
},
},
},
},
}
@@ -1549,8 +980,37 @@ function normalizePaths(spec) {
process.env.DB_HOST = process.env.DB_HOST || '127.0.0.1'
process.env.DB_PORT = process.env.DB_PORT || '59999'
// ── An annotation swagger-autogen cannot parse is DROPPED, not failed ──────
//
// It `console.error`s "Syntax error" or "out of structure", skips that one
// annotation, and prints `Success` in green. Nothing was listening, so the tree
// had been carrying a broken one — `POST /api/v1/admin/invites` documented with an
// EMPTY request body — for as long as it had existed. The same class turned up
// four more times in module-uo, whose annotations came from here.
//
// Two ways one breaks, both of them invisible in review: an object literal a
// brace short, and a `"` or a backtick inside a single-quoted description
// (swagger-autogen re-quotes both to `'` before evaluating, which ends the string
// early). Capturing the diagnostics is the only way to be told.
const swaggerComplaints = []
const realConsoleError = console.error
console.error = (...args) => {
const line = args.map(String).join(' ')
if (/syntax error|out of structure/i.test(line)) swaggerComplaints.push(line.trim())
else realConsoleError(...args)
}
/* eslint-disable global-require */
swaggerAutogen(outputFile, routes, doc)
.then(() => {
console.error = realConsoleError
if (swaggerComplaints.length > 0) {
throw new Error(
`swagger: ${swaggerComplaints.length} annotation(s) could not be parsed and were DROPPED ` +
`— the spec would be missing what they described:\n ${swaggerComplaints.join('\n ')}`,
)
}
})
.then(() => require('./slotSpecs').mergeSlotSpecs(outputFile))
.then(() => {
const written = JSON.parse(fs.readFileSync(outputFile, 'utf8'))
@@ -1561,6 +1021,7 @@ swaggerAutogen(outputFile, routes, doc)
return require('../src/utils/db').close()
})
.catch((err) => {
console.error = realConsoleError
process.stderr.write(`${err.stack || err.message}\n`)
process.exit(1)
})

528
server/test/adminModules.test.js Normal file
View File

@@ -0,0 +1,528 @@
// ── Admin · Modules: the delivery surface ──────────────────────────────────
//
// Phase 4, slice 1 of MODULE_SYSTEM.md §2.7.2. The controller is tested directly
// with a mock `res` and stubbed models — the same shape adminUsers.test.js uses —
// because what is interesting here is not the HTTP plumbing but the ORDER of
// operations and which of the three sources of truth answers which question.
//
// Two of these tests exist to pin decisions that are easy to "fix" back into
// being wrong:
//
// - **enable must not touch the loader.** Disable ran the module's onShutdown;
// there is no onBoot re-dispatch, so flipping the record back would put a
// module with closed sockets and cleared timers back on the nav.
// - **purge must run before the directory is removed.** purge.sql lives inside
// that directory. Reorder those two lines and the feature silently stops
// working, with a 200 and no data deleted.
//
// Point the DB at a closed port BEFORE requiring anything that builds the pool.
process.env.DB_HOST = '127.0.0.1'
process.env.DB_PORT = '59999'
const { test, beforeEach, after } = require('node:test')
const assert = require('node:assert/strict')
const ctrl = require('../src/router/v1/admin/modules.controller')
const modules = require('../src/model/modules/modules.model')
const activity = require('../src/model/activity/activity.model')
const settings = require('../src/model/settings/settings.model')
const loader = require('../src/modules/loader')
const lifecycle = require('../src/modules/lifecycle')
const install = require('../src/modules/install')
const declared = require('../src/modules/declared')
const schema = require('../src/modules/schema')
const db = require('../src/utils/db')
after(() => db.close())
function mockRes() {
return {
statusCode: 200,
body: null,
status(c) { this.statusCode = c; return this },
json(b) { this.body = b; return this },
}
}
const req = (extra = {}) => ({
user: { id: 1, username: 'admin' },
params: {},
query: {},
body: {},
...extra,
})
// Everything the controller reaches for, replaced wholesale per test. Restored
// from these originals rather than from a snapshot taken mid-run, so one test
// leaking a stub cannot quietly become another test's fixture.
const originals = {
modules: { ...modules },
activity: { log: activity.log },
settings: { get: settings.get, set: settings.set },
loader: { isLoaded: loader.isLoaded, list: loader.list },
lifecycle: { stop: lifecycle.stop },
install: {
install: install.install,
isInstalled: install.isInstalled,
purgeFile: install.purgeFile,
removeDir: install.removeDir,
},
schema: { runPurge: schema.runPurge },
declared: { state: declared.state },
}
let logged
beforeEach(() => {
Object.assign(modules, originals.modules)
Object.assign(activity, originals.activity)
Object.assign(settings, originals.settings)
Object.assign(loader, originals.loader)
Object.assign(lifecycle, originals.lifecycle)
Object.assign(install, originals.install)
Object.assign(schema, originals.schema)
Object.assign(declared, originals.declared)
logged = []
activity.log = async (entry) => { logged.push(entry) }
settings.get = async () => 'gitea.whitlocktech.com'
loader.isLoaded = () => true
loader.list = () => []
})
// ── list ───────────────────────────────────────────────────────────────────
test('list reconciles the row, the loader and the volume without picking a winner', async () => {
// The case §2.4 creates and decision 3 makes routine: the row says `enabled`
// because the operator just switched it back on, the loader still says
// `disabled` because its onShutdown has run and there is no way back without a
// restart. Rendering either one alone would be a lie.
modules.list = async () => [{
id: 'uo', name: 'UO', version: '1.0.0', state: 'enabled',
failureStage: null, failureReason: null, source: 'https://x/y.json', sha256: 'a'.repeat(64),
installedAt: null, startedAt: null,
}]
loader.list = () => [{ id: 'uo', name: 'UO', version: '1.0.0', state: 'disabled', stage: null, reason: null, capabilities: ['shard'] }]
install.isInstalled = () => true
install.purgeFile = () => '/modules/uo/server/db/purge.sql'
const res = mockRes()
await ctrl.list(req(), res)
const [m] = res.body.modules
assert.equal(m.state, 'enabled', 'what the operator decided')
assert.equal(m.liveState, 'disabled', 'what is actually answering')
assert.equal(m.onVolume, true)
assert.equal(m.canPurge, true)
assert.deepEqual(m.capabilities, ['shard'])
assert.deepEqual(res.body.sourceHosts, ['gitea.whitlocktech.com'])
})
test('list includes a module on the volume that has no row yet', async () => {
// A hand-placed directory before its first boot. §2.5 keeps that a supported
// install, and its routes are already being served — a screen showing nothing
// for it would be showing the wrong thing.
modules.list = async () => []
loader.list = () => [{ id: 'byhand', name: 'By Hand', version: '0.1.0', state: 'started', stage: null, reason: null, capabilities: [] }]
install.isInstalled = () => true
install.purgeFile = () => null
const res = mockRes()
await ctrl.list(req(), res)
assert.equal(res.body.modules.length, 1)
assert.equal(res.body.modules[0].id, 'byhand')
assert.equal(res.body.modules[0].state, null, 'no row means no recorded state, not a guessed one')
assert.equal(res.body.modules[0].liveState, 'started')
})
test('list carries what MODULES declares, including a module it could not install', async () => {
// Slice 3's fourth source. A declared module that failed to resolve has no
// row, no directory and nothing mounted — so it is invisible to the other
// three, and the reason it is missing is the one thing the operator needs.
modules.list = async () => [{
id: 'uo', name: 'UO', version: '0.2.0', state: 'enabled',
failureStage: null, failureReason: null, source: null, sha256: null,
installedAt: null, startedAt: null,
}]
loader.list = () => [{ id: 'uo', name: 'UO', version: '0.2.0', state: 'started', stage: null, reason: null, capabilities: [] }]
install.isInstalled = (id) => id === 'uo'
install.purgeFile = () => null
declared.state = () => [
{ id: 'uo', version: '0.3.0', url: 'https://x/uo.json', action: 'failed', message: 'the host is down' },
{ id: 'market', version: '1.0.0', url: 'https://x/market.json', action: 'failed', message: 'not found' },
]
const res = mockRes()
await ctrl.list(req(), res)
const byId = Object.fromEntries(res.body.modules.map((m) => [m.id, m]))
// Running fine at 0.2.0 while the declared upgrade to 0.3.0 is failing: both
// facts survive, because collapsing them would have to discard one.
assert.equal(byId.uo.liveState, 'started')
assert.equal(byId.uo.declared, true)
assert.equal(byId.uo.declaredVersion, '0.3.0')
assert.equal(byId.uo.declaredError, 'the host is down')
// Declared and nowhere: listed anyway, with no version invented for it.
assert.equal(byId.market.declared, true)
assert.equal(byId.market.version, null)
assert.equal(byId.market.state, null)
assert.equal(byId.market.declaredError, 'not found')
})
test('a successfully resolved module is marked declared, with no error', async () => {
modules.list = async () => []
loader.list = () => [{ id: 'uo', name: 'UO', version: '0.3.0', state: 'started', stage: null, reason: null, capabilities: [] }]
install.isInstalled = () => true
install.purgeFile = () => null
declared.state = () => [{ id: 'uo', version: '0.3.0', url: 'https://x/uo.json', action: 'noop', message: null }]
const res = mockRes()
await ctrl.list(req(), res)
assert.equal(res.body.modules.length, 1, 'declared and present is ONE module, not two')
assert.equal(res.body.modules[0].declared, true)
assert.equal(res.body.modules[0].declaredError, null)
})
test('list survives a process where the loader never scanned', async () => {
loader.isLoaded = () => false
loader.list = () => { throw new Error('modules.list() before modules.load()') }
modules.list = async () => [{ id: 'uo', name: 'UO', version: '1', state: 'disabled' }]
install.isInstalled = () => false
install.purgeFile = () => null
const res = mockRes()
await ctrl.list(req(), res)
assert.equal(res.statusCode, 200)
assert.equal(res.body.modules[0].liveState, null)
})
// ── install ────────────────────────────────────────────────────────────────
test('install records provenance and says a restart is needed', async () => {
const calls = []
install.install = async ({ url, hosts }) => {
calls.push({ url, hosts })
return { id: 'uo', name: 'UO', version: '1.0.0', sha256: 'b'.repeat(64), source: url, replaced: false }
}
modules.recordInstalled = async (row) => { calls.push(row); return { ...row, state: 'installed' } }
const res = mockRes()
await ctrl.create(req({ body: { url: 'https://gitea.whitlocktech.com/x/uo.json' } }), res)
assert.equal(res.statusCode, 201)
assert.equal(res.body.restartRequired, true)
assert.deepEqual(calls[0].hosts, ['gitea.whitlocktech.com'], 'the allowlist comes from the setting')
// Provenance is written HERE and nowhere else — the boot reconcile records a
// module with null source/sha256 and leaves what it is not given.
assert.equal(calls[1].source, 'https://gitea.whitlocktech.com/x/uo.json')
assert.equal(calls[1].sha256, 'b'.repeat(64))
assert.equal(logged[0].action, 'module.install')
})
test('an install refusal is reported to the operator, with its own status', async () => {
install.install = async () => {
const err = new Error('"evil.net" is not an allowed module source host')
err.name = 'InstallError'
err.status = 400
throw err
}
const res = mockRes()
await ctrl.create(req({ body: { url: 'https://evil.net/x.json' } }), res)
assert.equal(res.statusCode, 400)
// The message is the useful part: the operator pasted a URL and needs to know
// what was wrong with what came back.
assert.match(res.body.message, /not an allowed module source host/)
assert.equal(logged.length, 0, 'a refused install is not an audit-log entry')
})
test('an unreachable host is a 502, not a 400', async () => {
install.install = async () => {
const err = new Error('could not reach x: timeout')
err.name = 'InstallError'
err.status = 502
throw err
}
const res = mockRes()
await ctrl.create(req({ body: { url: 'https://gitea.whitlocktech.com/x.json' } }), res)
assert.equal(res.statusCode, 502)
})
test('an unexpected failure is a 500 and does not leak its message', async () => {
install.install = async () => { throw new Error('ENOENT /some/internal/path') }
const res = mockRes()
await ctrl.create(req({ body: { url: 'https://gitea.whitlocktech.com/x.json' } }), res)
assert.equal(res.statusCode, 500)
assert.equal(res.body.message, 'Internal Server Error')
})
// ── enable / disable ───────────────────────────────────────────────────────
test('enable moves the row and does NOT touch the loader', async () => {
// The decision-3 invariant. Re-enabling cannot restart a module: its
// onShutdown has run, and MODULE_API.md has never promised onBoot is safe to
// run twice. Flipping the record would put it back on the nav with a
// torn-down world behind it.
let setStateCalled = false
loader.setState = () => { setStateCalled = true }
modules.enable = async (id) => ({ id, state: 'enabled' })
const res = mockRes()
await ctrl.enable(req({ params: { id: 'uo' } }), res)
assert.equal(res.body.module.state, 'enabled')
assert.equal(res.body.restartRequired, true)
assert.equal(setStateCalled, false, 'enable must not move the in-memory record')
assert.equal(logged[0].action, 'module.enable')
loader.setState = originals.loader.setState
})
test('enabling a module with no row is a 404', async () => {
modules.enable = async () => null
const res = mockRes()
await ctrl.enable(req({ params: { id: 'ghost' } }), res)
assert.equal(res.statusCode, 404)
})
test('an illegal transition is a 409, not a 500', async () => {
modules.enable = async () => {
const err = new Error("module 'uo': cannot move from 'x' to 'enabled'")
err.name = 'ModuleStateError'
throw err
}
const res = mockRes()
await ctrl.enable(req({ params: { id: 'uo' } }), res)
assert.equal(res.statusCode, 409)
})
test('disable stops the module and reports whether the hook ran', async () => {
const calls = []
modules.get = async (id) => ({ id, state: 'started' })
lifecycle.stop = async (id) => { calls.push(id); return { stopped: true, error: null } }
const res = mockRes()
await ctrl.disable(req({ params: { id: 'uo' } }), res)
assert.deepEqual(calls, ['uo'])
assert.equal(res.body.stopped, true)
// No restart: this is the one action that takes effect immediately, and it is
// the one an operator reaches for when something is going wrong.
assert.equal(res.body.restartRequired, undefined)
assert.equal(logged[0].action, 'module.disable')
})
test('a shutdown hook that failed is reported rather than swallowed', async () => {
modules.get = async (id) => ({ id, state: 'started' })
lifecycle.stop = async () => ({ stopped: false, error: 'socket would not close' })
const res = mockRes()
await ctrl.disable(req({ params: { id: 'uo' } }), res)
// It IS disabled either way; the operator should be told it did not close
// cleanly while they still have the logs in front of them.
assert.equal(res.statusCode, 200)
assert.match(res.body.shutdownError, /socket would not close/)
})
// ── uninstall and purge ────────────────────────────────────────────────────
test('uninstall purges BEFORE it removes the directory', async () => {
// The ordering that makes decision 5 work at all: purge.sql is a file inside
// the directory being deleted. Swap these two and the endpoint still answers
// 200 and deletes nothing.
const order = []
modules.get = async (id) => ({ id, state: 'started' })
install.isInstalled = () => true
install.purgeFile = () => '/modules/uo/server/db/purge.sql'
schema.runPurge = async () => { order.push('purge'); return 12 }
lifecycle.stop = async () => { order.push('stop'); return { stopped: true, error: null } }
install.removeDir = async () => { order.push('removeDir'); return true }
modules.remove = async () => { order.push('removeRow') }
const res = mockRes()
await ctrl.remove(req({ params: { id: 'uo' }, query: { purge: 'true' } }), res)
assert.deepEqual(order, ['purge', 'stop', 'removeDir', 'removeRow'])
assert.equal(res.body.purged, 12)
assert.equal(res.body.restartRequired, true)
assert.equal(logged[0].action, 'module.purge')
})
test('a plain uninstall keeps the row and does not purge', async () => {
const order = []
modules.get = async (id) => ({ id, state: 'started' })
install.isInstalled = () => true
schema.runPurge = async () => { order.push('purge'); return 1 }
lifecycle.stop = async () => { order.push('stop'); return { stopped: true, error: null } }
install.removeDir = async () => { order.push('removeDir'); return true }
modules.remove = async () => { order.push('removeRow') }
const res = mockRes()
await ctrl.remove(req({ params: { id: 'uo' } }), res)
// §2.5's default: the directory goes, the data stays, and the disabled row is
// what keeps the retained data visible and the module reinstallable.
assert.deepEqual(order, ['stop', 'removeDir'])
assert.equal(res.body.purged, null)
assert.equal(logged[0].action, 'module.uninstall')
})
test('asking to purge a module that ships no purge.sql refuses instead of pretending', async () => {
modules.get = async (id) => ({ id, state: 'started' })
install.isInstalled = () => true
install.purgeFile = () => null
let removed = false
install.removeDir = async () => { removed = true; return true }
const res = mockRes()
await ctrl.remove(req({ params: { id: 'uo' }, query: { purge: 'true' } }), res)
assert.equal(res.statusCode, 400)
assert.match(res.body.message, /ships no purge.sql/)
// Nothing happened. The operator asked for the module AND its data to go; the
// data cannot go, so doing half of it silently would be the worst answer.
assert.equal(removed, false)
})
test('uninstalling something that is neither on the volume nor in a row is a 404', async () => {
modules.get = async () => null
install.isInstalled = () => false
const res = mockRes()
await ctrl.remove(req({ params: { id: 'ghost' } }), res)
assert.equal(res.statusCode, 404)
})
test('standalone purge refuses while the module is still running', async () => {
// Dropping the tables under a module that is still serving leaves it answering
// out of a world that no longer exists. Disabling first is one click.
modules.get = async (id) => ({ id, state: 'started' })
let ran = false
schema.runPurge = async () => { ran = true; return 1 }
const res = mockRes()
await ctrl.purge(req({ params: { id: 'uo' } }), res)
assert.equal(res.statusCode, 409)
assert.match(res.body.message, /Disable this module before purging/)
assert.equal(ran, false)
})
test('standalone purge runs on a disabled module', async () => {
modules.get = async (id) => ({ id, state: 'disabled' })
install.purgeFile = () => '/modules/uo/server/db/purge.sql'
schema.runPurge = async () => 7
const res = mockRes()
await ctrl.purge(req({ params: { id: 'uo' } }), res)
assert.equal(res.body.purged, 7)
assert.equal(logged[0].action, 'module.purge')
})
// ── the allowlist ──────────────────────────────────────────────────────────
test('setSources stores a normalised list and audits the change', async () => {
let stored = null
settings.set = async (key, value) => { stored = { key, value } }
const res = mockRes()
await ctrl.setSources(req({ body: { hosts: 'Gitea.Example.com, releases.example.org' } }), res)
assert.deepEqual(res.body.sourceHosts, ['gitea.example.com', 'releases.example.org'])
assert.equal(stored.key, ctrl.HOSTS_KEY)
assert.equal(stored.value, 'gitea.example.com,releases.example.org')
// Before AND after: this setting decides what code the site will execute, so
// the audit entry has to say what it used to be.
assert.equal(logged[0].action, 'module.sources')
assert.deepEqual(logged[0].detail.before, ['gitea.whitlocktech.com'])
})
test('setSources refuses anything that is not a bare hostname', async () => {
let stored = false
settings.set = async () => { stored = true }
for (const bad of ['https://x.com', 'x.com/path', 'x.com:8443', '*.x.com', 'x_y.com']) {
const res = mockRes()
// eslint-disable-next-line no-await-in-loop
await ctrl.setSources(req({ body: { hosts: bad } }), res)
assert.equal(res.statusCode, 400, `${bad} should be refused`)
}
assert.equal(stored, false)
})
test('an empty allowlist is storable, and means no installs', async () => {
// Not a wildcard, and not an error: "nothing may be installed" is a position
// an operator is entitled to take.
let stored = null
settings.set = async (key, value) => { stored = value }
const res = mockRes()
await ctrl.setSources(req({ body: { hosts: '' } }), res)
assert.equal(res.statusCode, 200)
assert.deepEqual(res.body.sourceHosts, [])
assert.equal(stored, '')
})
// ── restart ────────────────────────────────────────────────────────────────
// These listen for the SIGTERM EVENT rather than stubbing `process.kill`, and
// that is the whole point of them now.
//
// The first version of this route called `process.kill(process.pid, 'SIGTERM')`
// and the first version of these tests stubbed `process.kill` and asserted it
// had been called with SIGTERM. Both passed. Both were wrong: Windows has no
// POSIX signals, and Node documents SIGTERM there as unconditional termination —
// so on a Windows host the route killed the server outright, with no module
// `onShutdown`, no pool close and no log flush. A stub of `process.kill` cannot
// see that, because what it asserts is precisely the call whose MEANING differs
// by platform.
//
// Asserting on the event closes the gap: it is what server.js's handler is
// actually subscribed to, so a test passing here means the handler would run.
function onceSigterm() {
return new Promise((resolve, reject) => {
const timer = setTimeout(() => {
process.removeListener('SIGTERM', handler)
reject(new Error('no SIGTERM was emitted within 1s'))
}, 1000)
function handler() {
clearTimeout(timer)
process.removeListener('SIGTERM', handler)
resolve(true)
}
process.on('SIGTERM', handler)
})
}
test('restart answers first, then triggers the one graceful-shutdown path', async () => {
const fired = onceSigterm()
const res = mockRes()
ctrl.restart(req(), res)
// Answered synchronously: once the shutdown starts there is no listener left
// to flush a response through, so the operator would be told nothing.
assert.equal(res.statusCode, 202)
assert.equal(res.body.restarting, true)
assert.equal(await fired, true)
assert.equal(logged[0].action, 'module.restart')
})
test('a failure to write the audit entry does not cancel the restart', async () => {
activity.log = async () => { throw new Error('database is gone') }
const fired = onceSigterm()
ctrl.restart(req(), mockRes())
assert.equal(await fired, true)
})

82
server/test/bootOrder.test.js Normal file
View File

@@ -0,0 +1,82 @@
// server.js's boot ORDER, which is a contract and not a style choice.
//
// Phase 4, slice 3 of MODULE_SYSTEM.md §2.7.2. Five steps have to happen in one
// order, and each arrow is a dependency that is invisible at the call site:
//
// core schema → the declared set (§2.7.2 decision 4) needs the settings row
// its seed writes, to know which hosts it may install from
// declared set → requiring app.js SCANS the volume (§1.12), so anything put
// there afterwards is not in this process
// require app → module schema fragments (§2.6) can only be replayed once the
// loader knows which modules there are
// fragments → onBoot runs against tables that exist
//
// This is a source-structure test, and it is worth being plain about what that
// does and does not prove: it cannot tell you the server boots, only that nobody
// has quietly moved one of these five lines past another. It exists because the
// defect it guards against has already happened once and was invisible to every
// other kind of test here. Deferring the app require — which slice 3 had to do —
// moved core's schema ahead of the scan, and `ensureSchema` then skipped the
// module fragments entirely. On this machine's dev database the tables already
// existed, so the module started; on a FRESH database it would have started
// against no tables at all. Every suite in this directory stubs either the
// loader or the pool, so none of them could see it. The browser smoke did, from
// one log line.
process.env.DB_HOST = '127.0.0.1'
process.env.DB_PORT = '59999'
const fs = require('fs')
const path = require('path')
const { test } = require('node:test')
const assert = require('node:assert/strict')
const SERVER = fs.readFileSync(path.join(__dirname, '..', 'src', 'server.js'), 'utf8')
/** Where a marker appears, asserted to appear exactly once. */
function at(marker) {
const first = SERVER.indexOf(marker)
assert.notEqual(first, -1, `server.js no longer contains ${JSON.stringify(marker)}`)
assert.equal(
SERVER.indexOf(marker, first + 1),
-1,
`${JSON.stringify(marker)} appears more than once in server.js — this test cannot tell which is the boot step`,
)
return first
}
test('boot runs core schema, then the declared set, then the scan, then fragments, then onBoot', () => {
const steps = [
['core schema', at('await ensureSchema({ replayModules: false })')],
['declared module set', at('await declaredModules.resolve(')],
['the volume scan', at("require('./app')")],
['module schema fragments', at('replayFragments()')],
['module onBoot', at('await moduleLifecycle.boot()')],
]
for (let i = 1; i < steps.length; i += 1) {
assert.ok(
steps[i][1] > steps[i - 1][1],
`${steps[i][0]} must come after ${steps[i - 1][0]} in server.js`,
)
}
})
test('app.js is required inside start(), never at the top of the file', () => {
// The whole mechanism depends on this. A top-level require runs when server.js
// is loaded — before a single line of start() — and the scan would then happen
// before the declared set had a chance to put anything on the volume, silently
// and with no error anywhere.
assert.ok(
at("require('./app')") > at('async function start()'),
'requiring ./app at the top of server.js scans the volume before the declared set is resolved',
)
})
test('ensureSchema is asked NOT to replay fragments, and something else does', () => {
// Both halves matter. Passing the flag without replaying elsewhere is the
// original defect with an explicit spelling; replaying without the flag runs
// the fragments twice, the second time being the one that matters.
assert.match(SERVER, /ensureSchema\(\{ replayModules: false \}\)/)
assert.match(SERVER, /modules\/schema'\)\.replayFragments\(\)/)
})

151
server/test/checkModuleIdentifiers.test.js Normal file
View File

@@ -0,0 +1,151 @@
// ── The §5.2 check has to be provably still checking ───────────────────────
//
// `scripts/checkModuleIdentifiers.js` passes today. So does a check that reads
// nothing, and the two are indistinguishable from CI's green tick — which is the
// whole failure mode of a boundary test: it is written when the boundary is
// clean, it never fires again, and nobody finds out it stopped working until the
// thing it guards has been broken for months.
//
// So this file feeds it code it MUST reject and code it MUST accept. The
// accept cases are not filler: every one of them is a false positive that a
// simpler implementation actually produced against this repo, and each would
// have made the check something people route around rather than obey.
const { test } = require('node:test')
const assert = require('node:assert/strict')
const {
run,
checkFile,
maskCode,
tokenize,
reservedWordIn,
EXEMPT,
} = require('../../scripts/checkModuleIdentifiers')
const words = (src) => checkFile('x.js', src).map((h) => `${h.kind}:${h.name}`)
// ── the repo itself ─────────────────────────────────────────────────────────
test('core, as it stands, names no module identifier', () => {
const { hits, unused } = run()
assert.deepEqual(
hits.map((h) => `${h.file}:${h.line} ${h.kind} "${h.name}"`),
[],
)
assert.deepEqual(unused.map((e) => `${e.file} "${e.name}"`), [], 'a grandfathering exemption matches nothing')
})
test('every exemption is one of the §6.5 grandfathering allowlists', () => {
// Not a count assertion — a reason assertion. The exemption list is the only
// part of this check that can weaken it, so what it may contain is pinned.
for (const e of EXEMPT) {
assert.match(e.file, /^server\/src\/modules\//, `${e.file} is not core's module machinery`)
assert.equal(e.name, 'uo')
assert.match(e.why, /§6\.5/)
}
})
// ── what it must catch ──────────────────────────────────────────────────────
test('an import specifier reaching into a module is a hit', () => {
assert.deepEqual(words("const s = require('../shardState/shardState.model')"), [
'import specifier:../shardState/shardState.model',
])
assert.deepEqual(words("import { feed } from './lib/useShardFeed.js'"), [
'import specifier:./lib/useShardFeed.js',
])
})
test('a route path literal for a module surface is a hit', () => {
assert.deepEqual(words("router.get('/shard/status', handler)"), ['route path:/shard/status'])
assert.deepEqual(words("app.use('/uo-link', r)"), ['route path:/uo-link'])
assert.deepEqual(words("router.post('/atlas/import', h)"), ['route path:/atlas/import'])
})
test('a declared identifier carrying a module word is a hit', () => {
assert.deepEqual(words('function shardStatus() {}'), ['declared identifier:shardStatus'])
assert.deepEqual(words('const uoLinkClient = 1'), ['declared identifier:uoLinkClient'])
assert.deepEqual(words('class TownCrier {}'), ['declared identifier:TownCrier'])
})
test('a property name carrying a module word is a hit', () => {
assert.deepEqual(words('const api = {\n shard: {},\n}'), ['property name:shard'])
})
test('the module id on its own is a hit', () => {
assert.deepEqual(words("router.use('/uo', r)"), ['route path:/uo'])
})
// ── what it must NOT catch ──────────────────────────────────────────────────
test('a substring that is not a word is not a hit — `defaultImage` contains "ultIma"', () => {
// The real false positive: a case-insensitive substring pass flagged
// heroLayout.js and Portal.jsx four times over on its first run, for a
// parameter called `defaultImage`.
assert.deepEqual(words('function heroBackground(layout, { defaultImage } = {}) {}'), [])
assert.deepEqual(words('const fallback = defaultImage || DEFAULT_HERO_IMAGE'), [])
})
test('core prose may say the words — comments are not read', () => {
assert.deepEqual(words('// the shard one does, indirectly, by asking an endpoint'), [])
assert.deepEqual(words('/* Wraps the public site. When the shard is in maintenance … */'), [])
assert.deepEqual(words('/**\n * hidden by a module\'s shard visibility rules\n */'), [])
})
test('core copy may say the words — string CONTENT is not read', () => {
assert.deepEqual(words("const lead = 'daily life on the shard'"), [])
assert.deepEqual(words('const t = `an independent Ultima Online shard`'), [])
})
test('a comment containing quotes does not swallow the file', () => {
// The `checkImports.js` lesson, restated: strip comments BEFORE quotes, in one
// pass. A comment with an apostrophe used to open a string that ran to EOF,
// and everything after it silently stopped being checked.
const src = "// it's the operator's own file\nfunction shardStatus() {}"
assert.deepEqual(words(src), ['declared identifier:shardStatus'])
})
test('a string containing // does not hide the code after it', () => {
const src = "const u = 'https://example.com/x'\nfunction shardFeed() {}"
assert.deepEqual(words(src), ['declared identifier:shardFeed'])
})
test('an escaped quote inside a string does not end it early', () => {
const src = "const s = 'it\\'s fine, shard'\nconst clilocMap = {}"
assert.deepEqual(words(src), ['declared identifier:clilocMap'])
})
test('ordinary core words that merely resemble a reserved one pass', () => {
assert.deepEqual(words('const link = 1'), []) // "link" alone is not "uo-link"
assert.deepEqual(words('const unlinkIdentity = () => {}'), [])
assert.deepEqual(words('const townName = 1'), []) // "town" alone is not "towncrier"
assert.deepEqual(words('const uploadUrl = 1'), [])
})
// ── the internals the above depends on ──────────────────────────────────────
test('tokenize splits camelCase, snake_case and kebab-case the same way', () => {
assert.deepEqual(tokenize('shardStatus'), ['shard', 'status'])
assert.deepEqual(tokenize('uo_link_config'), ['uo', 'link', 'config'])
assert.deepEqual(tokenize('uo-link'), ['uo', 'link'])
assert.deepEqual(tokenize('UOLinkClient'), ['uo', 'link', 'client'])
assert.deepEqual(tokenize('defaultImage'), ['default', 'image'])
})
test('reservedWordIn matches whole words and adjacent pairs only', () => {
assert.equal(reservedWordIn('shardFeed'), 'shard')
assert.equal(reservedWordIn('uoLinkConfig'), 'uo')
assert.equal(reservedWordIn('townCrier'), 'town-crier')
assert.equal(reservedWordIn('defaultImage'), null)
assert.equal(reservedWordIn('townHall'), null)
})
test('maskCode keeps every offset, so reported line numbers are the real ones', () => {
const src = "// shard\nconst a = 'shard'\nfunction shardX() {}"
const masked = maskCode(src)
assert.equal(masked.length, src.length)
assert.equal(masked.split('\n').length, src.split('\n').length)
const [hit] = checkFile('x.js', src)
assert.equal(hit.line, 3)
})

8
server/test/loginProtection.test.js
View File

@@ -60,7 +60,13 @@ test('backoffGuard returns a generic 429 while locked out', async () => {
a.post('/login', lp.backoffGuard, (req, res) => res.json({ ok: true }))
})
try {
lp.recordFailure('203.0.113.40') // lock the test client IP
// Lock the test client IP. FIVE failures, not one: the lock is
// `BASE_MS * 2 ** (count - 1)`, so a single failure locks for exactly one
// second and this test then races the round trip. It lost that race on CI
// (200 instead of 429, request arriving 1,456 ms after the lock). Five
// failures lock for sixteen seconds, which is not a race. What is under
// test is the guard's ANSWER while locked out, and that is unchanged.
for (let i = 0; i < 5; i += 1) lp.recordFailure('203.0.113.40')
const res = await fetch(`${app.url}/login`, {
method: 'POST',
headers: { 'X-Forwarded-For': '203.0.113.40' },

252
server/test/moduleArchive.test.js Normal file
View File

@@ -0,0 +1,252 @@
// modules/archive.js — the hardened bundle extractor (MODULE_SYSTEM.md §2.7.2).
//
// Every rejection case is exercised against a REAL archive rather than a mocked
// tar parser, because the thing being tested is what the parser reports for a
// given sequence of bytes. A fake that returns `{type: 'SymbolicLink'}` proves
// only that the `if` is spelled correctly.
//
// The hostile archives are written here as raw ustar headers instead of being
// produced with `tar`, for two reasons that both bit during this slice:
//
// 1. `ln -s` needs a privilege Windows does not hand out by default, so a
// symlink fixture built with the shell is a fixture that silently is not
// one, and the test passes for the wrong reason on the machine most of this
// work happens on.
// 2. GNU tar will not emit `../escape` or `/etc/passwd` as a member name — it
// strips them and tells you so. The archives worth defending against are
// exactly the ones a cooperative archiver refuses to produce.
//
// A ustar header is 512 bytes of fixed-offset fields, so writing one is less
// code than persuading a tool to misbehave.
const test = require('node:test')
const assert = require('node:assert/strict')
const fs = require('fs')
const os = require('os')
const path = require('path')
const zlib = require('zlib')
const archive = require('../src/modules/archive')
// ── A minimal ustar writer ─────────────────────────────────────────────────
const BLOCK = 512
function octal(value, width) {
// ustar numeric fields are NUL-terminated octal, right-aligned with zeros.
return Number(value).toString(8).padStart(width - 1, '0') + '\0'
}
/**
* One 512-byte header plus its padded data.
*
* @param {object} entry
* @param {string} entry.name member path
* @param {string} [entry.type] '0' file · '5' dir · '2' symlink · '1' hardlink
* · '3' char dev · '6' FIFO
* @param {string} [entry.linkname] target, for the link types
* @param {string} [entry.body] file contents
* @param {number} [entry.size] declared size — defaults to the body's, and
* may be set independently to build a header
* that lies about its payload
*/
function member({ name, type = '0', linkname = '', body = '', size = null }) {
const header = Buffer.alloc(BLOCK, 0)
const data = Buffer.from(body, 'utf8')
const declared = size === null ? data.length : size
header.write(name, 0, 100, 'utf8')
header.write(octal(0o644, 8), 100, 8, 'ascii') // mode
header.write(octal(0, 8), 108, 8, 'ascii') // uid
header.write(octal(0, 8), 116, 8, 'ascii') // gid
header.write(octal(declared, 12), 124, 12, 'ascii')
header.write(octal(0, 12), 136, 12, 'ascii') // mtime
header.write(' ', 148, 8, 'ascii') // checksum field is spaces while summing
header.write(type, 156, 1, 'ascii')
header.write(linkname, 157, 100, 'utf8')
header.write('ustar\0', 257, 6, 'ascii')
header.write('00', 263, 2, 'ascii')
let sum = 0
for (const byte of header) sum += byte
header.write(`${sum.toString(8).padStart(6, '0')}\0 `, 148, 8, 'ascii')
const padding = Buffer.alloc((BLOCK - (data.length % BLOCK)) % BLOCK, 0)
return Buffer.concat([header, data, padding])
}
/** Gzip a set of members into a .tar.gz on disk, and return its path. */
function writeArchive(dir, filename, members) {
const tarball = Buffer.concat([
...members.map(member),
Buffer.alloc(BLOCK * 2, 0), // two zero blocks end the archive
])
const file = path.join(dir, filename)
fs.writeFileSync(file, zlib.gzipSync(tarball))
return file
}
/** A well-formed bundle: one top-level directory, ordinary files inside it. */
function goodMembers(root = 'module-uo-1.0.0') {
return [
{ name: `${root}/`, type: '5' },
{ name: `${root}/module.json`, body: '{"id":"uo","name":"UO","version":"1.0.0"}' },
{ name: `${root}/server/`, type: '5' },
{ name: `${root}/server/index.js`, body: 'module.exports = () => {}\n' },
]
}
// One scratch directory for the whole file, removed at the end.
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'rg-archive-'))
test.after(() => fs.rmSync(tmp, { recursive: true, force: true }))
/** Assert that inspecting `members` fails, and that the message says why. */
async function rejects(name, members, matcher) {
const file = writeArchive(tmp, `${name}.tar.gz`, members)
await assert.rejects(
() => archive.inspect(file),
(err) => {
assert.equal(err.name, 'ArchiveError', `expected an ArchiveError, got ${err.name}: ${err.message}`)
assert.match(err.message, matcher)
return true
},
)
}
// ── What a good bundle does ────────────────────────────────────────────────
test('inspect accepts a well-formed bundle and reports its single root', async () => {
const file = writeArchive(tmp, 'good.tar.gz', goodMembers())
const stats = await archive.inspect(file)
assert.equal(stats.root, 'module-uo-1.0.0')
assert.equal(stats.entries, 4)
assert.ok(stats.bytes > 0)
})
test('extract strips the top level, so the bundle lands as the module id', async () => {
// The wrapper directory is the publisher's naming (module-uo's release
// workflow packs `module-uo-<version>/`); the directory it lands in is core's,
// and has to be the id the loader scans for.
const file = writeArchive(tmp, 'strip.tar.gz', goodMembers())
const dest = path.join(tmp, 'unpacked-uo')
await archive.unpack(file, dest)
assert.ok(fs.existsSync(path.join(dest, 'module.json')), 'module.json should be at the root of the destination')
assert.ok(fs.existsSync(path.join(dest, 'server', 'index.js')))
assert.ok(!fs.existsSync(path.join(dest, 'module-uo-1.0.0')), 'the wrapper directory should not survive')
})
test('extract refuses a destination that already exists', async () => {
const file = writeArchive(tmp, 'exists.tar.gz', goodMembers())
const dest = path.join(tmp, 'already-there')
fs.mkdirSync(dest)
await assert.rejects(() => archive.extract(file, dest), /already exists/)
})
// ── What a hostile bundle does ─────────────────────────────────────────────
test('an absolute member path is refused', async () => {
await rejects('absolute', [
{ name: 'mod/', type: '5' },
{ name: '/etc/cron.d/pwned', body: '* * * * * root sh\n' },
], /absolute/)
})
test('a drive-absolute member path is refused', async () => {
// Its own node-tar advisory, and invisible to a leading-slash check.
await rejects('drive', [
{ name: 'mod/', type: '5' },
{ name: 'C:\\Windows\\Temp\\pwned', body: 'x' },
], /backslash|drive-absolute/)
})
test('an upward-escaping member path is refused', async () => {
await rejects('escape', [
{ name: 'mod/', type: '5' },
{ name: 'mod/../../../etc/passwd', body: 'root::0:0\n' },
], /escapes upward/)
})
test('a symlink member is refused, and the message names the type', async () => {
await rejects('symlink', [
{ name: 'mod/', type: '5' },
{ name: 'mod/passwd', type: '2', linkname: '/etc/passwd' },
], /SymbolicLink/)
})
test('a hardlink member is refused', async () => {
// The single most-published node-tar escape primitive. Refusing the type
// outright is what keeps this file from depending on the library getting
// hardlink containment right.
await rejects('hardlink', [
{ name: 'mod/', type: '5' },
{ name: 'mod/shadow', type: '1', linkname: '../../../etc/shadow' },
], /Link/)
})
test('a device node is refused', async () => {
await rejects('device', [
{ name: 'mod/', type: '5' },
{ name: 'mod/zero', type: '3', linkname: '' },
], /may only contain files and directories/)
})
test('a FIFO is refused', async () => {
await rejects('fifo', [
{ name: 'mod/', type: '5' },
{ name: 'mod/pipe', type: '6' },
], /may only contain files and directories/)
})
test('two top-level directories are refused', async () => {
await rejects('two-roots', [
{ name: 'mod-a/', type: '5' },
{ name: 'mod-a/module.json', body: '{}' },
{ name: 'mod-b/', type: '5' },
{ name: 'mod-b/module.json', body: '{}' },
], /exactly one top-level directory, found 2/)
})
test('a bundle whose declared sizes exceed the cap is refused before it is read to the end', async () => {
// The header lies: it declares a gigabyte and carries nothing. That is the
// decompression-bomb shape, and the point is that inspect() decides on the
// DECLARED size without ever materialising the payload.
await rejects('bomb', [
{ name: 'mod/', type: '5' },
{ name: 'mod/big', size: archive.MAX_BYTES + 1, body: '' },
], /unpacks to more than/)
})
test('an empty archive is refused', async () => {
await rejects('empty', [], /empty/)
})
// ── The path check on its own ──────────────────────────────────────────────
//
// pathProblem is exported so the cases that are awkward to express as archive
// bytes can still be asserted directly.
test('pathProblem accepts ordinary bundle paths', () => {
for (const ok of ['mod/module.json', 'mod/server/router/x.js', 'mod/a.b-c_d/e.js']) {
assert.equal(archive.pathProblem(ok), null, `${ok} should be accepted`)
}
})
test('pathProblem rejects the escape shapes', () => {
assert.match(archive.pathProblem('/etc/passwd'), /absolute/)
assert.match(archive.pathProblem('C:/Windows/x'), /drive-absolute/)
assert.match(archive.pathProblem('mod/../../x'), /escapes upward/)
assert.match(archive.pathProblem('..'), /escapes upward/)
assert.match(archive.pathProblem('mod\\x'), /backslash/)
assert.match(archive.pathProblem('mod/\0/x'), /NUL byte/)
})
test('pathProblem does not reject a filename that merely contains two dots', () => {
// `..` is a SEGMENT, not a substring — a file called `version..js` is fine,
// and a check written with `includes('..')` would refuse it.
assert.equal(archive.pathProblem('mod/version..js'), null)
assert.equal(archive.pathProblem('mod/..hidden'), null)
})

391
server/test/moduleDeclared.test.js Normal file
View File

@@ -0,0 +1,391 @@
// modules/declared.js — resolving the module set an environment declares.
//
// Phase 4, slice 3 of MODULE_SYSTEM.md §2.7.2, decision 4. Two claims are worth
// more than the rest and both are about what does NOT happen:
//
// - a module already unpacked at the declared version does not touch the
// network, because that is what makes a restart with the internet down come
// up unchanged;
// - a module that cannot be resolved does not fail the boot, and does not stop
// the next declaration from resolving.
//
// The last test in the file runs the whole path through the real install.js
// against a fake transport — a real gzipped tar, really hashed, really unpacked
// — because everything above it stubs `installImpl` and would keep passing if
// the two files stopped agreeing about what an install returns.
process.env.DB_HOST = '127.0.0.1'
process.env.DB_PORT = '59999'
const crypto = require('crypto')
const fs = require('fs')
const os = require('os')
const path = require('path')
const zlib = require('zlib')
const { test, beforeEach, after } = require('node:test')
const assert = require('node:assert/strict')
const db = require('../src/utils/db')
after(() => db.close())
const HOSTS = ['releases.example.com']
const URL_030 = 'https://releases.example.com/mod/uo-0.3.0.json'
let tmpRoot
let declared
/**
* A fresh declared.js bound to a fresh modules directory.
*
* loader.js resolves MODULES_DIR once at require time and install.js reads
* `loader.dir()`, so all three have to come back together — the same dance
* moduleInstall.test.js and moduleLifecycle.test.js do.
*/
function fresh(dir) {
process.env.MODULES_DIR = dir
for (const m of ['loader', 'install', 'declared']) {
delete require.cache[require.resolve(`../src/modules/${m}`)]
}
// eslint-disable-next-line global-require
return require('../src/modules/declared')
}
/** Put a module directory on the volume, as an unpack would leave it. */
function place(id, version) {
const dir = path.join(tmpRoot, id)
fs.mkdirSync(dir, { recursive: true })
fs.writeFileSync(
path.join(dir, 'module.json'),
JSON.stringify({ id, name: 'Ultima Online', version, coreApi: '^1.0.0' }),
)
return dir
}
/**
* A stub install service.
*
* Records every call, so "did this reach the network at all" is a question the
* tests can ask directly rather than inferring from an outcome.
*/
function fakeInstall({ version = '0.3.0', throws = null } = {}) {
const calls = []
return {
calls,
InstallError: Error,
async install(args) {
calls.push(args)
if (throws) throw new Error(throws)
return {
id: 'uo',
name: 'Ultima Online',
version,
sha256: 'a'.repeat(64),
source: args.url,
bytes: 1024,
replaced: false,
}
},
}
}
/** A modules.model stand-in that remembers what provenance it was given. */
function fakeModel() {
const recorded = []
return { recorded, async recordInstalled(row) { recorded.push(row); return row } }
}
beforeEach(() => {
tmpRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'rg-declared-'))
declared = fresh(tmpRoot)
})
// ── Parsing ────────────────────────────────────────────────────────────────
test('parses id@version=url entries separated by whitespace or commas', () => {
const { entries, errors } = declared.parse(
` uo@0.3.0=https://a.example/uo.json,\n market@1.2.3=https://a.example/market.json `,
)
assert.deepEqual(errors, [])
assert.deepEqual(entries, [
{ id: 'uo', version: '0.3.0', url: 'https://a.example/uo.json' },
{ id: 'market', version: '1.2.3', url: 'https://a.example/market.json' },
])
})
test('an empty or unset declaration parses to nothing, without complaint', () => {
for (const value of [undefined, '', ' ']) {
const { entries, errors } = declared.parse(value)
assert.deepEqual(entries, [])
assert.deepEqual(errors, [])
}
})
test('a malformed entry is refused on its own, leaving the others', () => {
// A bare URL, a missing version, and an id that is not one — each of which an
// operator can plausibly type, and none of which should cost the module that
// was written correctly.
const { entries, errors } = declared.parse(
'https://a.example/uo.json uo@=https://a.example/uo.json Uo@1=https://a.example/uo.json'
+ ' good@1.0.0=https://a.example/good.json',
)
assert.equal(entries.length, 1)
assert.equal(entries[0].id, 'good')
assert.equal(errors.length, 3)
})
test('a duplicate id keeps the first and says so', () => {
const { entries, errors } = declared.parse(
'uo@1.0.0=https://a.example/one.json uo@2.0.0=https://a.example/two.json',
)
assert.equal(entries.length, 1)
assert.equal(entries[0].version, '1.0.0')
assert.match(errors[0], /declared more than once/)
})
// ── Resolution ─────────────────────────────────────────────────────────────
test('a module already at the declared version is a no-op that never fetches', async () => {
place('uo', '0.3.0')
const installer = fakeInstall()
const model = fakeModel()
const results = await declared.resolve({
value: `uo@0.3.0=${URL_030}`,
hosts: HOSTS,
model,
installImpl: installer,
})
assert.deepEqual(results.map((r) => r.action), ['noop'])
// The offline guarantee, stated as an assertion: nothing was fetched and
// nothing was written. A restart with no route to the internet is this case.
assert.equal(installer.calls.length, 0)
assert.equal(model.recorded.length, 0)
})
test('a missing module is installed, with its provenance recorded', async () => {
const installer = fakeInstall()
const model = fakeModel()
const results = await declared.resolve({
value: `uo@0.3.0=${URL_030}`,
hosts: HOSTS,
model,
installImpl: installer,
})
assert.deepEqual(results.map((r) => r.action), ['installed'])
assert.equal(installer.calls[0].url, URL_030)
assert.deepEqual(installer.calls[0].hosts, HOSTS)
// The declaration is handed down so install.js can refuse a URL that turns out
// to be another module or another version BEFORE it downloads it.
assert.deepEqual(installer.calls[0].expect, { id: 'uo', version: '0.3.0' })
// Written exactly as the admin route writes it — the reason this runs in the
// server process rather than in a script that cannot reach the database.
assert.deepEqual(model.recorded, [
{ id: 'uo', name: 'Ultima Online', version: '0.3.0', source: URL_030, sha256: 'a'.repeat(64) },
])
})
test('a module at a different version is re-resolved', async () => {
place('uo', '0.2.0')
const installer = fakeInstall()
const results = await declared.resolve({
value: `uo@0.3.0=${URL_030}`,
hosts: HOSTS,
model: fakeModel(),
installImpl: installer,
})
assert.deepEqual(results.map((r) => r.action), ['installed'])
assert.equal(installer.calls.length, 1)
})
test('a directory with no readable module.json counts as absent', async () => {
fs.mkdirSync(path.join(tmpRoot, 'uo'), { recursive: true })
fs.writeFileSync(path.join(tmpRoot, 'uo', 'module.json'), '{ this is not json')
const installer = fakeInstall()
await declared.resolve({
value: `uo@0.3.0=${URL_030}`,
hosts: HOSTS,
model: fakeModel(),
installImpl: installer,
})
// Whatever is in that directory, it is not the declared module — so the
// declared module is fetched rather than assumed to be there.
assert.equal(installer.calls.length, 1)
})
test('a failure is carried, not thrown, and does not stop the next module', async () => {
const model = fakeModel()
const installer = {
calls: [],
async install(args) {
installer.calls.push(args)
if (args.expect.id === 'uo') throw new Error('could not reach releases.example.com')
return {
id: args.expect.id,
name: 'Market',
version: args.expect.version,
sha256: 'b'.repeat(64),
source: args.url,
bytes: 10,
replaced: false,
}
},
}
const results = await declared.resolve({
value: `uo@0.3.0=${URL_030} market@1.0.0=https://releases.example.com/market.json`,
hosts: HOSTS,
model,
installImpl: installer,
})
assert.deepEqual(results.map((r) => r.action), ['failed', 'installed'])
assert.match(results[0].message, /could not reach/)
// The second module still installed: one unreachable release host costs that
// module, not the deployment.
assert.equal(model.recorded.length, 1)
assert.equal(model.recorded[0].id, 'market')
})
test('a failed resolution leaves whatever was already on the volume', async () => {
place('uo', '0.2.0')
const installer = fakeInstall({ throws: 'the host is down' })
const results = await declared.resolve({
value: `uo@0.3.0=${URL_030}`,
hosts: HOSTS,
model: fakeModel(),
installImpl: installer,
})
assert.equal(results[0].action, 'failed')
// The site comes up serving the version it already had rather than not at all.
assert.equal(declared.installedVersion('uo'), '0.2.0')
})
test('resolution without a model records nothing and still installs', async () => {
// `npm run seed` and the test harness both reach code paths with no model to
// hand; the volume half must not depend on the database half.
const installer = fakeInstall()
const results = await declared.resolve({ value: `uo@0.3.0=${URL_030}`, hosts: HOSTS, installImpl: installer })
assert.deepEqual(results.map((r) => r.action), ['installed'])
})
test('state() reports the last resolution and is replaced by the next', async () => {
const installer = fakeInstall()
await declared.resolve({ value: `uo@0.3.0=${URL_030}`, hosts: HOSTS, installImpl: installer })
assert.equal(declared.state().length, 1)
assert.equal(declared.state()[0].id, 'uo')
await declared.resolve({ value: '', hosts: HOSTS, installImpl: installer })
assert.deepEqual(declared.state(), [])
})
// ── The whole path, once, for real ─────────────────────────────────────────
const BLOCK = 512
function octal(value, width) {
return Number(value).toString(8).padStart(width - 1, '0') + '\0'
}
function member({ name, type = '0', body = '' }) {
const header = Buffer.alloc(BLOCK, 0)
const data = Buffer.from(body, 'utf8')
header.write(name, 0, 100, 'utf8')
header.write(octal(0o644, 8), 100, 8, 'ascii')
header.write(octal(0, 8), 108, 8, 'ascii')
header.write(octal(0, 8), 116, 8, 'ascii')
header.write(octal(data.length, 12), 124, 12, 'ascii')
header.write(octal(0, 12), 136, 12, 'ascii')
header.write(' ', 148, 8, 'ascii')
header.write(type, 156, 1, 'ascii')
header.write('ustar\0', 257, 6, 'ascii')
header.write('00', 263, 2, 'ascii')
let sum = 0
for (const byte of header) sum += byte
header.write(`${sum.toString(8).padStart(6, '0')}\0 `, 148, 8, 'ascii')
const padding = Buffer.alloc((BLOCK - (data.length % BLOCK)) % BLOCK, 0)
return Buffer.concat([header, data, padding])
}
/** A bundle tarball, named the way module-uo's release workflow names one. */
function bundle(version) {
const root = `module-uo-${version}`
return zlib.gzipSync(Buffer.concat([
member({ name: `${root}/`, type: '5' }),
member({
name: `${root}/module.json`,
body: JSON.stringify({ id: 'uo', name: 'Ultima Online', version, coreApi: '^1.0.0', server: 'server/index.js' }),
}),
member({ name: `${root}/server/`, type: '5' }),
member({ name: `${root}/server/index.js`, body: 'module.exports = () => {}\n' }),
Buffer.alloc(BLOCK * 2, 0),
]))
}
test('end to end: a declaration installs a real bundle through the real install path', async (t) => {
const tarball = bundle('0.3.0')
const artifactUrl = 'https://releases.example.com/mod/uo-0.3.0.tar.gz'
const manifest = JSON.stringify({
schema: 1,
id: 'uo',
name: 'Ultima Online',
version: '0.3.0',
coreApi: '^1.0.0',
artifact: 'uo-0.3.0.tar.gz',
url: artifactUrl,
sha256: crypto.createHash('sha256').update(tarball).digest('hex'),
size: tarball.length,
})
const routes = { [URL_030]: manifest, [artifactUrl]: tarball }
const fetched = []
const fetchImpl = async (url) => {
fetched.push(String(url))
const body = routes[String(url)]
return body === undefined ? new Response('nope', { status: 404 }) : new Response(body, { status: 200 })
}
// The real install.js, with only the transport replaced — same seam the
// install tests use, and the same reason: what is under test is the decisions,
// not Node's TLS.
// eslint-disable-next-line global-require
const install = require('../src/modules/install')
const installImpl = { install: (args) => install.install({ ...args, fetchImpl }) }
const model = fakeModel()
const first = await declared.resolve({
value: `uo@0.3.0=${URL_030}`,
hosts: HOSTS,
model,
installImpl,
})
assert.deepEqual(first.map((r) => r.action), ['installed'])
// Unpacked, with the release's top-level directory stripped, under the id.
assert.equal(
JSON.parse(fs.readFileSync(path.join(tmpRoot, 'uo', 'module.json'), 'utf8')).version,
'0.3.0',
)
assert.equal(model.recorded[0].sha256, JSON.parse(manifest).sha256)
// And again, which is what every restart after the first one is.
const before = fetched.length
const second = await declared.resolve({ value: `uo@0.3.0=${URL_030}`, hosts: HOSTS, model, installImpl })
assert.deepEqual(second.map((r) => r.action), ['noop'])
assert.equal(fetched.length, before, 'the second resolution must not fetch anything')
// A declaration whose URL resolves to a different version is refused BEFORE
// the artifact is downloaded — the module on the volume is left alone.
const wrong = await declared.resolve({ value: `uo@0.4.0=${URL_030}`, hosts: HOSTS, model, installImpl })
assert.equal(wrong[0].action, 'failed')
assert.match(wrong[0].message, /v0\.4\.0 was asked for/)
assert.equal(declared.installedVersion('uo'), '0.3.0')
t.diagnostic(`fetched ${fetched.length} URL(s) across three resolutions`)
})

205
server/test/moduleDocsSpec.test.js Normal file
View File

@@ -0,0 +1,205 @@
// ── /api/docs.json, with a module installed ────────────────────────────────
//
// The request-time half of docs/website/MODULE_API.md §6.1's settled decision.
// `swagger-output.json` is core's own routes and cannot be anything else: it is
// generated on a developer's machine and committed, so it has to come out the
// same regardless of what they had checked out, and a module arrives on the
// volume long after the image was built. The module's routes therefore reach the
// document only here, from the fragment it ships (§2.8, §6.1a).
//
// This boots the REAL app against a throwaway module directory, because the two
// things worth locking are properties of the served document rather than of the
// merge helper: that a started module's paths are IN it, and that core wins.
//
// The modules directory is written and MODULES_DIR set BEFORE app.js is required
// — the scan is synchronous and happens during that require.
process.env.DB_HOST = '127.0.0.1'
process.env.DB_PORT = '59999'
const fs = require('fs')
const os = require('os')
const path = require('path')
const { test, before, after } = require('node:test')
const assert = require('node:assert/strict')
const tmpRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'rg-module-docs-'))
const moduleDir = path.join(tmpRoot, 'atlas')
fs.mkdirSync(path.join(moduleDir, 'client', 'dist'), { recursive: true })
fs.writeFileSync(path.join(moduleDir, 'client', 'dist', 'entry.js'), 'export const hello = 1\n')
fs.writeFileSync(
path.join(moduleDir, 'module.json'),
JSON.stringify({
id: 'atlas',
name: 'Atlas',
version: '1.0.0',
coreApi: '^1.0.0',
client: { entry: 'client/dist/entry.js' },
}),
)
fs.writeFileSync(
path.join(moduleDir, 'swagger-fragment.json'),
JSON.stringify({
paths: {
'/api/v1/public/atlas/creatures': { get: { tags: ['Public · Atlas'], summary: 'List creatures' } },
// The collision case, and the one that matters: a module trying to
// redefine a path core already declares. Core wins and the module's
// version is dropped (§6.1a) — a module cannot rewrite core's docs.
'/api/v1/public/settings': { get: { summary: 'MODULE OVERRIDE' } },
},
tags: [{ name: 'Public · Atlas', description: 'from the module' }],
components: {
schemas: {
AtlasCreature: { type: 'object' },
// Same shape of collision, one section down.
Error: { type: 'string', description: 'MODULE OVERRIDE' },
},
},
}),
)
process.env.MODULES_DIR = tmpRoot
/* eslint-disable global-require */
const app = require('../src/app')
const loader = require('../src/modules/loader')
const db = require('../src/utils/db')
const coreSpec = require('../swagger/swagger-output.json')
const { docsSpec, reset } = require('../swagger/docsSpec')
/* eslint-enable global-require */
let server
let base
before(async () => {
server = await new Promise((resolve) => {
const s = app.listen(0, '127.0.0.1', () => resolve(s))
})
base = `http://127.0.0.1:${server.address().port}`
})
after(async () => {
server.closeAllConnections()
await new Promise((resolve) => server.close(resolve))
await db.close()
fs.rmSync(tmpRoot, { recursive: true, force: true })
})
const fetchSpec = async () => {
const res = await fetch(`${base}/api/docs.json`)
assert.equal(res.status, 200)
return res.json()
}
test('a module that is not started contributes nothing', async () => {
// It is `registered` here: loaded cleanly, onBoot not yet dispatched. Its
// routes answer 503 in that state, so documenting them would send a client
// somewhere it cannot go — the same reason the client chunk's <script> tag is
// withheld until `started`.
reset()
const spec = await fetchSpec()
assert.equal(spec.paths['/api/v1/public/atlas/creatures'], undefined)
})
test('a started module\'s paths, tags and schemas are in the served document', async () => {
loader.setState('atlas', 'started')
const spec = await fetchSpec()
assert.equal(spec.paths['/api/v1/public/atlas/creatures'].get.summary, 'List creatures')
assert.ok(spec.tags.some((t) => t.name === 'Public · Atlas'))
assert.deepEqual(spec.components.schemas.AtlasCreature, { type: 'object' })
})
test('core wins every collision, in every section', async () => {
loader.setState('atlas', 'started')
const spec = await fetchSpec()
assert.notEqual(spec.paths['/api/v1/public/settings'].get.summary, 'MODULE OVERRIDE')
assert.notEqual(spec.components.schemas.Error.description, 'MODULE OVERRIDE')
assert.deepEqual(spec.components.schemas.Error, coreSpec.components.schemas.Error)
})
test('the committed spec is never mutated by the merge', async () => {
// `swagger-output.json` is a require()d JSON module, so one in-place merge
// would be permanent for the life of the process AND cumulative across
// rebuilds — a module's paths surviving its own uninstall.
loader.setState('atlas', 'started')
await fetchSpec()
assert.equal(coreSpec.paths['/api/v1/public/atlas/creatures'], undefined)
assert.equal(coreSpec.components.schemas.AtlasCreature, undefined)
})
test('a state change rebuilds the document rather than serving the cached one', async () => {
loader.setState('atlas', 'started')
assert.ok((await fetchSpec()).paths['/api/v1/public/atlas/creatures'])
loader.setState('atlas', 'disabled')
assert.equal((await fetchSpec()).paths['/api/v1/public/atlas/creatures'], undefined)
loader.setState('atlas', 'started')
assert.ok((await fetchSpec()).paths['/api/v1/public/atlas/creatures'])
})
test('an unreadable fragment costs that module its paths and nothing else', async () => {
// §4.4's bargain, applied here: one module's failure is never the site's. A
// docs page that 500s is strictly worse than one missing a module's routes.
loader.setState('atlas', 'started')
const file = path.join(moduleDir, 'swagger-fragment.json')
const good = fs.readFileSync(file, 'utf8')
fs.writeFileSync(file, 'not json {')
try {
reset()
const spec = await fetchSpec()
assert.equal(spec.paths['/api/v1/public/atlas/creatures'], undefined)
assert.ok(spec.paths['/api/v1/public/settings'], 'core\'s own paths must survive')
} finally {
fs.writeFileSync(file, good)
reset()
}
})
test('a module with no fragment at all is simply absent', async () => {
// Registering routes without documenting them is checked in the MODULE's CI,
// where the routes are known. Core cannot tell a module with no routes from
// one that forgot, so it does not guess.
loader.setState('atlas', 'started')
const file = path.join(moduleDir, 'swagger-fragment.json')
const good = fs.readFileSync(file, 'utf8')
fs.rmSync(file)
try {
reset()
const spec = await fetchSpec()
assert.equal(spec.paths['/api/v1/public/atlas/creatures'], undefined)
assert.ok(Object.keys(spec.paths).length > 100)
} finally {
fs.writeFileSync(file, good)
reset()
}
})
test('before the loader has scanned, core\'s own spec is the answer', () => {
// §7.6 makes every other accessor THROW when asked before modules.load(), so a
// mis-ordered boot cannot be mistaken for an empty install. A request handler
// is the exception: 500ing the docs page over it would be the wrong trade, and
// core's routes are the honest answer to "what is documented" at that point.
reset()
const stub = { isLoaded: () => false }
const original = Object.getOwnPropertyDescriptor(loader, 'isLoaded')
Object.defineProperty(loader, 'isLoaded', { value: stub.isLoaded, configurable: true })
try {
assert.equal(docsSpec(coreSpec), coreSpec)
} finally {
Object.defineProperty(loader, 'isLoaded', original)
reset()
}
})
test('the interactive UI is rebuilt per request, not bound to boot\'s document', async () => {
// Bound once at require time, /api/docs would show core's routes for the life
// of the process while /api/docs.json showed the merged set.
loader.setState('atlas', 'started')
reset()
const withModule = await fetch(`${base}/api/docs/`)
assert.equal(withModule.status, 200)
assert.match(await withModule.text(), /swagger/i)
})

448
server/test/moduleInstall.test.js Normal file
View File

@@ -0,0 +1,448 @@
// modules/install.js — fetching, verifying and placing a module bundle.
//
// Phase 4, slice 1 of MODULE_SYSTEM.md §2.7.2. The rules under test are all
// refusals, and each one is a step an attacker would otherwise walk through:
// a non-https URL, a host that is not allowed, a REDIRECT to a host that is not
// allowed, a body larger than declared, a hash that does not match, an archive
// that does not agree with the manifest about what it is.
//
// `fetchImpl` is injected rather than a TLS server being stood up, for the same
// reason `replayFragments` takes a `query`: what is being tested is what this
// file decides about a response, and a real server would mostly test Node's
// certificate handling. The responses below are real `Response` objects with
// real bodies, so the streaming, the hashing and the byte cap are exercised for
// real — only the transport is stubbed.
process.env.DB_HOST = '127.0.0.1'
process.env.DB_PORT = '59999'
const crypto = require('crypto')
const fs = require('fs')
const os = require('os')
const path = require('path')
const zlib = require('zlib')
const { test, beforeEach, after } = require('node:test')
const assert = require('node:assert/strict')
const db = require('../src/utils/db')
after(() => db.close())
const HOSTS = ['releases.example.com']
const MANIFEST_URL = 'https://releases.example.com/mod/uo-1.0.0.json'
let tmpRoot
let install
/**
* A fresh install.js bound to a fresh modules directory.
*
* install.js reads `loader.dir()`, which is resolved once at require time from
* MODULES_DIR — so both have to be re-required per test, exactly as
* moduleLifecycle.test.js does.
*/
function freshInstall(dir) {
process.env.MODULES_DIR = dir
delete require.cache[require.resolve('../src/modules/loader')]
delete require.cache[require.resolve('../src/modules/install')]
// eslint-disable-next-line global-require
return require('../src/modules/install')
}
// ── Building a bundle to serve ─────────────────────────────────────────────
const BLOCK = 512
function octal(value, width) {
return Number(value).toString(8).padStart(width - 1, '0') + '\0'
}
function member({ name, type = '0', body = '' }) {
const header = Buffer.alloc(BLOCK, 0)
const data = Buffer.from(body, 'utf8')
header.write(name, 0, 100, 'utf8')
header.write(octal(0o644, 8), 100, 8, 'ascii')
header.write(octal(0, 8), 108, 8, 'ascii')
header.write(octal(0, 8), 116, 8, 'ascii')
header.write(octal(data.length, 12), 124, 12, 'ascii')
header.write(octal(0, 12), 136, 12, 'ascii')
header.write(' ', 148, 8, 'ascii')
header.write(type, 156, 1, 'ascii')
header.write('ustar\0', 257, 6, 'ascii')
header.write('00', 263, 2, 'ascii')
let sum = 0
for (const byte of header) sum += byte
header.write(`${sum.toString(8).padStart(6, '0')}\0 `, 148, 8, 'ascii')
const padding = Buffer.alloc((BLOCK - (data.length % BLOCK)) % BLOCK, 0)
return Buffer.concat([header, data, padding])
}
/** A bundle tarball whose root directory is named the way a release names it. */
function bundle({ id = 'uo', version = '1.0.0', extra = [] } = {}) {
const root = `module-${id}-${version}`
return zlib.gzipSync(Buffer.concat([
member({ name: `${root}/`, type: '5' }),
member({
name: `${root}/module.json`,
body: JSON.stringify({ id, name: 'Ultima Online', version, coreApi: '^1.0.0', server: 'server/index.js' }),
}),
member({ name: `${root}/server/`, type: '5' }),
member({ name: `${root}/server/index.js`, body: 'module.exports = () => {}\n' }),
...extra.map(member),
Buffer.alloc(BLOCK * 2, 0),
]))
}
const sha256 = (buf) => crypto.createHash('sha256').update(buf).digest('hex')
/**
* A fake transport serving one manifest and one artifact.
*
* `routes` maps an absolute URL to either a Buffer/string body or
* `{ status, location }` for a redirect, so a test can describe exactly what the
* remote host does without describing how it does it.
*/
function fakeFetch(routes) {
const seen = []
const impl = async (url) => {
const href = String(url)
seen.push(href)
const route = routes[href]
if (route === undefined) return new Response('not found', { status: 404 })
if (route && route.status) {
return new Response(null, {
status: route.status,
headers: route.location ? { location: route.location } : {},
})
}
return new Response(route, { status: 200 })
}
impl.seen = seen
return impl
}
/** The manifest a release publishes, with whatever a test wants to change. */
function manifestFor(tarball, overrides = {}) {
return JSON.stringify({
schema: 1,
id: 'uo',
name: 'Ultima Online',
version: '1.0.0',
coreApi: '^1.0.0',
artifact: 'uo-1.0.0.tar.gz',
url: 'https://releases.example.com/mod/uo-1.0.0.tar.gz',
sha256: sha256(tarball),
size: tarball.length,
...overrides,
})
}
/** The happy-path pair: a valid manifest and the artifact it describes. */
function goodRoutes(options = {}) {
const tarball = bundle(options)
return {
tarball,
routes: {
[MANIFEST_URL]: manifestFor(tarball, options.manifest || {}),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
},
}
}
beforeEach(() => {
tmpRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'rg-install-'))
install = freshInstall(tmpRoot)
})
// ── The allowlist ──────────────────────────────────────────────────────────
test('parseHosts accepts comma and whitespace separation, and lower-cases', () => {
assert.deepEqual(install.parseHosts('a.com, B.com\nc.com'), ['a.com', 'b.com', 'c.com'])
assert.deepEqual(install.parseHosts(''), [])
assert.deepEqual(install.parseHosts(null), [])
})
test('an empty allowlist forbids everything rather than allowing everything', () => {
// The direction matters: a setting someone blanks by accident must stop
// installs, not open the door to every host on the internet.
assert.throws(() => install.checkUrl('https://releases.example.com/x.json', []), /no module source hosts are allowed/)
})
test('only https may be installed from', () => {
// The sha256 is no help against a plaintext fetch: whoever can rewrite the
// artifact in flight can rewrite the manifest that declares its hash.
assert.throws(() => install.checkUrl('http://releases.example.com/x.json', HOSTS), /only https/)
assert.throws(() => install.checkUrl('file:///etc/passwd', HOSTS), /only https/)
})
test('a host that is not on the allowlist is refused, and the message says which are', () => {
assert.throws(
() => install.checkUrl('https://evil.example.net/x.json', HOSTS),
/"evil.example.net" is not an allowed module source host \(allowed: releases.example.com\)/,
)
})
test('the allowlist is matched on the host, not on a substring of the URL', () => {
// `https://evil.com/?x=releases.example.com` must not pass, and neither must a
// subdomain nobody listed.
assert.throws(() => install.checkUrl('https://evil.com/?releases.example.com', HOSTS), /not an allowed/)
assert.throws(() => install.checkUrl('https://sub.releases.example.com/x', HOSTS), /not an allowed/)
assert.throws(() => install.checkUrl('https://releases.example.com.evil.net/x', HOSTS), /not an allowed/)
})
// ── Redirects ──────────────────────────────────────────────────────────────
test('a redirect is followed, and re-checked against the allowlist', async () => {
const { tarball } = goodRoutes()
const impl = fakeFetch({
[MANIFEST_URL]: { status: 302, location: 'https://releases.example.com/cdn/uo-1.0.0.json' },
'https://releases.example.com/cdn/uo-1.0.0.json': manifestFor(tarball),
})
const manifest = await install.fetchManifest(MANIFEST_URL, HOSTS, impl)
assert.equal(manifest.id, 'uo')
assert.deepEqual(impl.seen, [MANIFEST_URL, 'https://releases.example.com/cdn/uo-1.0.0.json'])
})
test('a redirect to a host that is not allowed is refused', async () => {
// The hole the allowlist exists to close. `fetch`'s own redirect following
// would check the first hop and then go wherever it was pointed — which is
// why get() follows them by hand.
const impl = fakeFetch({
[MANIFEST_URL]: { status: 302, location: 'http://169.254.169.254/latest/meta-data/' },
})
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /only https/)
})
test('a redirect loop is bounded rather than followed forever', async () => {
const impl = fakeFetch({
[MANIFEST_URL]: { status: 302, location: MANIFEST_URL },
})
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /too many redirects/)
})
// ── The install manifest ───────────────────────────────────────────────────
test('a manifest that is not JSON is refused with a readable reason', async () => {
const impl = fakeFetch({ [MANIFEST_URL]: 'this is not json' })
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /not valid JSON/)
})
test('a manifest with an invalid module id is refused', async () => {
// The loader would refuse to scan a directory called `../etc`, but the point
// is that one is never created: the id becomes a path segment.
for (const id of ['../etc', 'UO', '', 'x', 'a'.repeat(40)]) {
const impl = fakeFetch({ [MANIFEST_URL]: JSON.stringify({ id, name: 'n', version: '1', sha256: 'a'.repeat(64) }) })
// eslint-disable-next-line no-await-in-loop
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /invalid module id/, `id ${JSON.stringify(id)}`)
}
})
test('a manifest with no usable sha256 is refused', async () => {
const impl = fakeFetch({
[MANIFEST_URL]: JSON.stringify({ id: 'uo', name: 'n', version: '1.0.0', sha256: 'not-a-hash' }),
})
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /no valid sha256/)
})
test('the artifact URL is resolved relative to the manifest when it carries no absolute one', async () => {
const { tarball } = goodRoutes()
const impl = fakeFetch({ [MANIFEST_URL]: manifestFor(tarball, { url: undefined }) })
const manifest = await install.fetchManifest(MANIFEST_URL, HOSTS, impl)
// Release assets sit beside their manifest, so a manifest that travelled
// without its absolute URL still resolves to the right place.
assert.equal(manifest.artifactUrl, 'https://releases.example.com/mod/uo-1.0.0.tar.gz')
})
// ── The install ────────────────────────────────────────────────────────────
test('a good bundle installs into modules/<id>, stripped of its wrapper', async () => {
const { routes, tarball } = goodRoutes()
const result = await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) })
assert.equal(result.id, 'uo')
assert.equal(result.version, '1.0.0')
assert.equal(result.sha256, sha256(tarball))
assert.equal(result.source, MANIFEST_URL)
assert.equal(result.replaced, false)
// The directory is named for the module id, not for the archive's root — the
// loader scans for the former and the publisher chose the latter.
const dir = path.join(tmpRoot, 'uo')
assert.ok(fs.existsSync(path.join(dir, 'module.json')))
assert.ok(fs.existsSync(path.join(dir, 'server', 'index.js')))
assert.ok(!fs.existsSync(path.join(tmpRoot, 'module-uo-1.0.0')))
})
test('a hash that does not match refuses the install and writes nothing', async () => {
const { routes } = goodRoutes()
// The bytes are fine; the manifest lies about them. Which is the same thing an
// artifact swapped after publication looks like.
routes[MANIFEST_URL] = manifestFor(Buffer.from('different'), {})
await assert.rejects(
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
/does not match the sha256/,
)
assert.deepEqual(fs.readdirSync(tmpRoot), [], 'nothing may be left on the volume')
})
test('a bundle whose module.json disagrees with the manifest is refused', async () => {
// A manifest promising `uo` and delivering something else would otherwise be
// installed into `modules/uo/` under a name it is not.
const tarball = bundle({ id: 'rust', version: '1.0.0' })
const routes = {
[MANIFEST_URL]: manifestFor(tarball),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
}
await assert.rejects(
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
/declares module id "rust" but the install manifest promised "uo"/,
)
assert.deepEqual(fs.readdirSync(tmpRoot), [])
})
test('a bundle whose version disagrees with the manifest is refused', async () => {
const tarball = bundle({ id: 'uo', version: '9.9.9' })
const routes = {
[MANIFEST_URL]: manifestFor(tarball),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
}
await assert.rejects(
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
/declares version "9.9.9"/,
)
})
test('an artifact whose length differs from the declared size is refused', async () => {
const tarball = bundle()
const routes = {
[MANIFEST_URL]: manifestFor(tarball, { size: tarball.length + 10 }),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
}
await assert.rejects(
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
/but the manifest declares/,
)
})
test('a hostile archive is refused, and nothing of it reaches the volume', async () => {
// The case node-tar alone does NOT cover: it throws on the escaping member,
// but only once it reaches it — the members before it are already on disk.
// archive.inspect() decides before extract() runs, and the unpack happens in a
// scratch directory that is removed either way.
const tarball = bundle({
extra: [
{ name: 'module-uo-1.0.0/../../ESCAPED.txt', body: 'escaped' },
],
})
const routes = {
[MANIFEST_URL]: manifestFor(tarball),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
}
await assert.rejects(
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
/escapes upward/,
)
assert.deepEqual(fs.readdirSync(tmpRoot), [], 'no scratch directory, no partial module, no escape')
})
test('an upgrade replaces the directory and reports that it did', async () => {
const first = goodRoutes()
await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(first.routes) })
fs.writeFileSync(path.join(tmpRoot, 'uo', 'STALE.txt'), 'from the old version')
const second = goodRoutes({ version: '2.0.0', manifest: { version: '2.0.0' } })
second.routes[MANIFEST_URL] = manifestFor(second.tarball, { version: '2.0.0' })
const result = await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(second.routes) })
assert.equal(result.replaced, true)
assert.equal(result.version, '2.0.0')
// A replace, not a merge: a file the previous version left behind must not
// survive into the new one, or an upgrade quietly keeps dead code loadable.
assert.ok(!fs.existsSync(path.join(tmpRoot, 'uo', 'STALE.txt')))
assert.deepEqual(fs.readdirSync(tmpRoot), ['uo'], 'the aside copy is cleaned up')
})
test('a failed upgrade leaves the previous version in place', async () => {
const first = goodRoutes()
await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(first.routes) })
const badTarball = bundle({ id: 'uo', version: '2.0.0', extra: [{ name: 'evil/', type: '5' }] })
await assert.rejects(() => install.install({
url: MANIFEST_URL,
hosts: HOSTS,
fetchImpl: fakeFetch({
[MANIFEST_URL]: manifestFor(badTarball, { version: '2.0.0' }),
'https://releases.example.com/mod/uo-1.0.0.tar.gz': badTarball,
}),
}))
// The installed module is untouched: the swap is the last step, so a bundle
// rejected before it never got near the live directory.
assert.ok(fs.existsSync(path.join(tmpRoot, 'uo', 'module.json')))
assert.equal(JSON.parse(fs.readFileSync(path.join(tmpRoot, 'uo', 'module.json'), 'utf8')).version, '1.0.0')
assert.deepEqual(fs.readdirSync(tmpRoot), ['uo'])
})
// ── The volume ─────────────────────────────────────────────────────────────
test('moduleDir refuses an id that is not one', () => {
for (const id of ['../escape', 'a/b', '', 'UO', '.']) {
assert.throws(() => install.moduleDir(id), /invalid module id/, `id ${JSON.stringify(id)}`)
}
assert.equal(install.moduleDir('uo'), path.join(tmpRoot, 'uo'))
})
test('isInstalled and removeDir report what they did', async () => {
const { routes } = goodRoutes()
assert.equal(install.isInstalled('uo'), false)
await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) })
assert.equal(install.isInstalled('uo'), true)
assert.equal(await install.removeDir('uo'), true)
assert.equal(install.isInstalled('uo'), false)
// Removing what is not there is not an error — an uninstall of a module whose
// directory was already deleted by hand should still tidy up the row.
assert.equal(await install.removeDir('uo'), false)
})
test('purgeFile resolves the manifest declaration, and refuses one that escapes', async () => {
const dir = path.join(tmpRoot, 'uo')
fs.mkdirSync(path.join(dir, 'server', 'db'), { recursive: true })
fs.writeFileSync(path.join(dir, 'server', 'db', 'purge.sql'), 'DROP TABLE IF EXISTS x;')
const write = (purge) => fs.writeFileSync(
path.join(dir, 'module.json'),
JSON.stringify({ id: 'uo', name: 'UO', version: '1.0.0', purge }),
)
write('server/db/purge.sql')
assert.equal(install.purgeFile('uo'), path.join(dir, 'server', 'db', 'purge.sql'))
// The same containment rule the loader applies to client.entry: a manifest may
// not point core at a file outside the module it belongs to.
write('../../../../etc/passwd')
assert.equal(install.purgeFile('uo'), null)
// Declared but absent, and not declared at all, are both "nothing to run".
write('server/db/missing.sql')
assert.equal(install.purgeFile('uo'), null)
write(undefined)
assert.equal(install.purgeFile('uo'), null)
})
test('purgeFile is null for a module that is not on the volume', () => {
assert.equal(install.purgeFile('nothere'), null)
})

153
server/test/moduleLifecycle.test.js
View File

@@ -125,6 +125,11 @@ function fakeModel(seed = []) {
if (!row || row.state === 'disabled') return
Object.assign(row, { state: 'startup_failed', failureStage: stage, failureReason: reason })
},
async disable(id) {
calls.push(`disable:${id}`)
const row = rows.get(id)
if (row) Object.assign(row, { state: 'disabled', failureStage: null, failureReason: null })
},
}
return model
}
@@ -374,3 +379,151 @@ test('a hook that throws does not stop the ones behind it', async () => {
await assert.doesNotReject(() => lifecycle.shutdown({ modules: loader }))
assert.deepEqual(noted(file), ['shutdown:zzz', 'shutdown:aaa'])
})
// ── stop(): the admin panel's Disable ──────────────────────────────────────
//
// Phase 4, §2.7.2 decision 3. Phase 2's disable moved a record and left the
// module running; the whole point of these tests is that it no longer does.
test('stop runs that one module\'s onShutdown and leaves the others alone', async () => {
const file = path.join(tmpRoot, 'log.txt')
writeModule('aaa', { boot: '', shutdown: '', log: file })
writeModule('bbb', { boot: '', shutdown: '', log: file })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
fs.writeFileSync(file, '')
const result = await lifecycle.stop('aaa', { modules: loader, model })
assert.deepEqual(result, { stopped: true, error: null })
// Only aaa. This is the difference from shutdown(), which runs everything.
assert.deepEqual(noted(file), ['shutdown:aaa'])
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
assert.equal(stateOf(loader, 'bbb').state, 'started', 'the other module keeps running')
assert.equal(model.rows.get('aaa').state, 'disabled')
})
test('stop moves the record only after the hook has run', async () => {
// While onShutdown runs, the module is still `started` — the only state in
// which its routes and the world it is tearing down agree with each other. The
// module reports its own view of itself, so a record moved too early shows up
// here as `disabled` instead of `started`.
const file = path.join(tmpRoot, 'log.txt')
const dir = path.join(tmpRoot, 'aaa')
fs.mkdirSync(dir, { recursive: true })
fs.writeFileSync(path.join(dir, 'module.json'), JSON.stringify({
id: 'aaa', name: 'A', version: '1.0.0', coreApi: '^1.0.0', server: 'index.js',
}))
fs.writeFileSync(path.join(dir, 'index.js'), `
module.exports = (ctx, api) => {
api.onBoot(async () => {})
api.onShutdown(async () => {
const loader = require(${JSON.stringify(require.resolve('../src/modules/loader'))})
const me = loader.list().find((m) => m.id === 'aaa')
require('fs').appendFileSync(${JSON.stringify(file)}, 'state-during-hook:' + me.state + ${JSON.stringify('\n')})
})
}`)
const loader = freshLoader(tmpRoot)
await lifecycle.boot({ modules: loader, model: fakeModel() })
await lifecycle.stop('aaa', { modules: loader, model: fakeModel([{ id: 'aaa', state: 'started' }]) })
assert.deepEqual(noted(file), ['state-during-hook:started'])
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
})
test('a hook that throws does not prevent the disable', async () => {
// The opposite of the boot path's rule, deliberately. There, a failure means
// the module never became safe to use; here, the operator has asked for it to
// stop answering and a module that could not close cleanly is a reason to log
// loudly, not a reason to leave it serving.
writeModule('aaa', { boot: '', shutdown: 'throw new Error("socket stuck")' })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
const result = await lifecycle.stop('aaa', { modules: loader, model })
assert.equal(result.stopped, false)
assert.match(result.error, /socket stuck/)
assert.equal(stateOf(loader, 'aaa').state, 'disabled', 'disabled anyway')
assert.equal(model.rows.get('aaa').state, 'disabled')
})
test('a hook that hangs costs its budget, and the module is still disabled', async () => {
writeModule('aaa', { boot: '', shutdown: 'await new Promise(() => {})' })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
const started = Date.now()
const result = await lifecycle.stop('aaa', { modules: loader, model, budgetMs: 50 })
assert.equal(result.stopped, false)
assert.match(result.error, /budget/)
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
assert.ok(Date.now() - started < 2000)
})
test('stopping a module with no onShutdown still disables it', async () => {
// A hookless module has nothing to run and must still stop answering, or the
// guard and the row disagree about what is serving.
writeModule('aaa', { boot: '' })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
const result = await lifecycle.stop('aaa', { modules: loader, model })
assert.deepEqual(result, { stopped: false, error: null })
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
assert.equal(model.rows.get('aaa').state, 'disabled')
})
test('stopping a module whose onBoot failed does not run its onShutdown', async () => {
// Same rule shutdownHooks() applies: a module that never finished warming up
// has a half-built world its onShutdown was not written for. It is still
// disabled — it just is not asked to tear anything down.
const file = path.join(tmpRoot, 'log.txt')
writeModule('aaa', { boot: 'throw new Error("no")', shutdown: '', log: file })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
const before = noted(file)
const result = await lifecycle.stop('aaa', { modules: loader, model })
assert.equal(result.stopped, false)
// Compared against what was already there rather than against an empty file:
// `noted` splits, so a blanked file reads as [''] and an emptiness assertion
// would pass for the wrong reason.
assert.deepEqual(noted(file), before, 'no new hook output')
assert.ok(!noted(file).includes('shutdown:aaa'))
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
})
test('stopping an unknown id writes the row and does not throw', async () => {
// A row can exist for a module that is not on the volume, and disabling it is
// exactly what an operator would do about that.
writeModule('aaa', { boot: '' })
const loader = freshLoader(tmpRoot)
const model = fakeModel([{ id: 'ghost', state: 'startup_failed' }])
await assert.doesNotReject(() => lifecycle.stop('ghost', { modules: loader, model }))
assert.equal(model.rows.get('ghost').state, 'disabled')
})
test('a row that will not write does not stop the module from being disabled', async () => {
// The same rule the boot path follows: bookkeeping failure is not the
// operation failing. The guard is what stops traffic, and it has already moved.
writeModule('aaa', { boot: '', shutdown: '' })
const loader = freshLoader(tmpRoot)
const model = fakeModel()
await lifecycle.boot({ modules: loader, model })
model.disable = async () => { throw new Error('database is gone') }
await assert.doesNotReject(() => lifecycle.stop('aaa', { modules: loader, model }))
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
})

65
server/test/moduleSchema.test.js
View File

@@ -25,7 +25,7 @@ const assert = require('node:assert/strict')
const express = require('express')
const db = require('../src/utils/db')
const { replayFragments } = require('../src/modules/schema')
const { replayFragments, runPurge } = require('../src/modules/schema')
const { splitStatements } = require('../src/utils/sqlStatements')
const { startApp } = require('./_helper')
@@ -245,3 +245,66 @@ test('replay is skipped, not thrown, when no scan happened in this process', asy
assert.deepEqual(rec.ran, [])
assert.equal(loader.isLoaded(), false)
})
// ── runPurge: the destructive twin ─────────────────────────────────────────
//
// Phase 4, slice 1. Same splitter, same pool, same serial execution as the
// replay above — pointed the other way. The two differences are deliberate and
// are what these tests are for.
test('purge runs every statement in the file, in order', async () => {
const file = path.join(tmpRoot, 'purge.sql')
fs.writeFileSync(file, [
'-- drop everything this module owns',
'DROP TABLE IF EXISTS mod_b;',
'DROP TABLE IF EXISTS mod_a;',
"DELETE FROM settings WHERE `key` = 'mod_thing';",
].join('\n'))
const rec = recorder()
const ran = await runPurge(file, { query: rec.query })
assert.equal(ran, 3)
assert.match(rec.ran[0], /DROP TABLE IF EXISTS mod_b/)
assert.match(rec.ran[2], /DELETE FROM settings/)
})
test('purge is not held to the fragment allowlist — DROP is the point of it', async () => {
// §2.6's leading-verb allowlist exists because a fragment replays on every
// boot. purge.sql never does, which is exactly why it is the one file a module
// may put a DROP in.
const file = path.join(tmpRoot, 'purge.sql')
fs.writeFileSync(file, 'DROP TABLE IF EXISTS mod_a;\nTRUNCATE TABLE mod_b;')
const rec = recorder()
assert.equal(await runPurge(file, { query: rec.query }), 2)
})
test('purge THROWS on failure, unlike the replay', async () => {
// A replay failure is one module failing to start, which the site survives by
// 503ing it. A purge failure is an operator's explicit destructive request not
// having happened — reporting success would leave them believing data is gone
// when it is not.
const file = path.join(tmpRoot, 'purge.sql')
fs.writeFileSync(file, 'DROP TABLE IF EXISTS mod_a;\nDROP TABLE mod_missing;')
const query = async (sql) => {
if (sql.includes('mod_missing')) throw new Error("Unknown table 'mod_missing'")
}
await assert.rejects(
() => runPurge(file, { query }),
// The message names WHICH statement stopped it, because a purge is not
// transactional — the ones before it have already committed and the operator
// needs to know where it got to.
/purge failed at statement 2 of 2: Unknown table 'mod_missing'/,
)
})
test('an empty purge file runs nothing and does not throw', async () => {
const file = path.join(tmpRoot, 'purge.sql')
fs.writeFileSync(file, '-- nothing to drop yet\n')
const rec = recorder()
assert.equal(await runPurge(file, { query: rec.query }), 0)
assert.deepEqual(rec.ran, [])
})

52
server/test/modules.model.test.js
View File

@@ -38,7 +38,17 @@ modulesDb.getOne = async (id) => (rows.has(id) ? [rows.get(id)] : [])
modulesDb.upsert = async ({ id, name, version, source, sha256 }) => {
const existing = rows.get(id)
if (existing) {
Object.assign(existing, { name, version, source: source ?? null, sha256: sha256 ?? null })
// COALESCE, matching the real statement: a value overwrites, a NULL leaves
// what is there. This fake used to assign unconditionally — faithfully
// reproducing the defect it was supposed to be able to catch, which is why
// the boot refresh nulling an install's provenance survived until a browser
// showed it.
Object.assign(existing, {
name,
version,
source: source ?? existing.source ?? null,
sha256: sha256 ?? existing.sha256 ?? null,
})
return { affectedRows: 1 }
}
rows.set(id, {
@@ -116,6 +126,46 @@ test('a hand-placed module records with no provenance', async () => {
assert.equal(mod.sha256, null)
})
test('the boot refresh does not wipe an install\'s provenance', async () => {
// The defect a browser found in Phase 4, and the reason the upsert COALESCEs.
//
// lifecycle.boot() re-records every scanned module with NO source and NO
// sha256, because a hand-placed directory genuinely has neither. That refresh
// used to write both columns as NULL, so an admin-panel install's provenance
// survived exactly until the restart the install asked for — and the screen
// then described a module installed from a URL as "placed on the volume by
// hand". Nothing could see it until Phase 4: no caller had ever passed a
// non-null value.
await modules.recordInstalled({
id: 'uo',
name: 'Ultima Online',
version: '1.0.0',
source: 'https://gitea.example/x/uo-1.0.0.json',
sha256: 'b'.repeat(64),
})
// What the next boot does.
const after = await modules.recordInstalled({ id: 'uo', name: 'Ultima Online', version: '1.0.0' })
assert.equal(after.source, 'https://gitea.example/x/uo-1.0.0.json')
assert.equal(after.sha256, 'b'.repeat(64))
})
test('a re-install from a new URL does replace the provenance', async () => {
// The other half: COALESCE must not make the columns write-once, or an upgrade
// would for ever show where the FIRST version came from.
await modules.recordInstalled({
id: 'uo', name: 'Ultima Online', version: '1.0.0', source: 'https://old/x.json', sha256: 'c'.repeat(64),
})
const after = await modules.recordInstalled({
id: 'uo', name: 'Ultima Online', version: '2.0.0', source: 'https://new/y.json', sha256: 'd'.repeat(64),
})
assert.equal(after.source, 'https://new/y.json')
assert.equal(after.sha256, 'd'.repeat(64))
assert.equal(after.version, '2.0.0')
})
test('recordInstalled refuses a manifest missing id, name or version', async () => {
await assert.rejects(
() => modules.recordInstalled({ id: 'uo', version: '1.0.0' }),