Compare commits
4 Commits
87230c879a
...
feature/mo
| Author | SHA1 | Date | |
|---|---|---|---|
| 732927a6bb | |||
| b30e82cde2 | |||
| 2cb549e9e5 | |||
| adff20be7b |
30
.env.example
30
.env.example
@@ -33,8 +33,8 @@ LOG_FILE=app.log
|
||||
# BRAND_NAME / BRAND_CONTACT_EMAIL.
|
||||
BRAND_NAME=Runic Gateway
|
||||
BRAND_SHORT_NAME=Runic Gateway
|
||||
BRAND_TAGLINE=an independent private Ultima Online shard
|
||||
BRAND_DESCRIPTION=Runic Gateway — an independent private Ultima Online shard. News, screenshots, guides, and community notes.
|
||||
BRAND_TAGLINE=an independent game community
|
||||
BRAND_DESCRIPTION=Runic Gateway — an independent game community. News, screenshots, guides, and community notes.
|
||||
BRAND_CONTACT_EMAIL=
|
||||
BRAND_URL=
|
||||
# Accent color — drives the web theme's --accent and the Discord embed color.
|
||||
@@ -107,20 +107,18 @@ CLIENT_ORIGIN=http://localhost:5173
|
||||
BOT_INTERNAL_URL=http://bot:4100
|
||||
BOT_INTERNAL_KEY=change-me-to-a-long-random-string
|
||||
|
||||
# uo-link sidecar — the HTTP + WebSocket bridge to the ServUO game server. The
|
||||
# website ingests its live event feed and proxies its read queries/commands
|
||||
# (shard status, online players, player-vendor sales, IDOC houses, character
|
||||
# sheets, account linking, town-crier). In production the sidecar + shard run on
|
||||
# a DIFFERENT host from the website, so both URLs are configurable. The
|
||||
# shared-secret auth token is NOT an env var — it is entered in the admin panel
|
||||
# (Shard page) and stored encrypted in the DB (same pattern as the Discord bot
|
||||
# token). These URLs are just defaults; the admin can override them at runtime.
|
||||
UOLINK_BASE_URL=http://127.0.0.1:8080
|
||||
UOLINK_WS_URL=ws://127.0.0.1:8080/ws
|
||||
# Wire protocol this build speaks (3 = Protocol 3.0). Only a fallback for a site
|
||||
# with nothing saved yet — the admin panel's pinned value wins — but set it lower
|
||||
# if you deliberately run an older sidecar.
|
||||
UOLINK_PROTOCOL=3
|
||||
# ─── Installed modules ───
|
||||
# A module is a directory on the modules volume (see MODULES_DIR in
|
||||
# server/.env.example); everything about a specific game lives in one, and core
|
||||
# knows nothing about any of them. A module may read its own env vars, and they
|
||||
# belong here because Compose passes this file to the container.
|
||||
#
|
||||
# RunicGateway/Module-uo, for example, reads UOLINK_BASE_URL / UOLINK_WS_URL /
|
||||
# UOLINK_PROTOCOL as the defaults for its connection to a uo-link sidecar, and
|
||||
# TOWNCRIER_DURATION_SEC for its news leg. Its README documents them; they are
|
||||
# left out here rather than half-copied, because a copy of another repo's
|
||||
# settings is a copy that goes stale silently. With no module installed, none of
|
||||
# this applies and the site runs as core.
|
||||
|
||||
# ─── Push notifications (M7) — self-hosted ntfy UnifiedPush relay ───
|
||||
# The `ntfy` compose service and the backend's push fan-out (opt-in notifications
|
||||
|
||||
260
README.md
260
README.md
@@ -8,16 +8,19 @@
|
||||
[](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
|
||||
[](https://sonar.whitlocktech.com/dashboard?id=runic-gateway-website)
|
||||
|
||||
Public site, wiki, and protected admin panel for a private Ultima Online shard — a
|
||||
full-stack app in one repo. Branding is instance-configurable via `BRAND_*` (see
|
||||
[Branding](#branding)); **UOMysticmoon** is the first instance.
|
||||
Public site, wiki, and protected admin panel for a game community — a full-stack app
|
||||
in one repo. Everything specific to a *particular* game lives in an installable
|
||||
module, not here. Branding is instance-configurable via `BRAND_*` (see
|
||||
[Branding](#branding)); **UOMysticmoon**, an Ultima Online shard, is the first
|
||||
instance, and its game half is
|
||||
[RunicGateway/Module-uo](https://gitea.whitlocktech.com/RunicGateway/Module-uo).
|
||||
|
||||
A full-stack app in one repo:
|
||||
|
||||
- **Backend** — Node.js + Express REST API (layered `router → controller → model → db`), MariaDB, a provider-agnostic session layer (JWT cookie for web, bearer tokens for mobile, pluggable SSO).
|
||||
- **Frontend** — React + Vite single-page app (public site, wiki, and the admin panel), dark "gothic" theme (Cinzel + Georgia).
|
||||
- **Deploy** — Docker Compose (app + MariaDB) behind a reverse proxy (Pangolin, Nginx, Caddy, Traefik, …). Express serves the built SPA in production.
|
||||
- **Shard link** — a live bridge to the in-game ServUO shard through the **uo-link** sidecar ([RunicGateway/link](https://gitea.whitlocktech.com/RunicGateway/link)): the site ingests a live event feed and makes server-side REST calls to show shard status, economy, staff presence, IDOCs, live activity, and per-character sheets. See [Shard integration (uo-link)](#shard-integration-uo-link).
|
||||
- **Modules** — the game-specific half of a site is a module dropped onto a volume: it adds routes, database tables, nav entries and whole SPA pages without this repo knowing anything about the game. See [Modules](#modules).
|
||||
|
||||
The design reference is [BACKEND_DESIGN.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/BACKEND_DESIGN.md) (API contract, schema, security), in the [**RunicGateway/docs**](https://gitea.whitlocktech.com/RunicGateway/docs) repo — where all project documentation now lives.
|
||||
|
||||
@@ -37,7 +40,7 @@ The design reference is [BACKEND_DESIGN.md](https://gitea.whitlocktech.com/Runic
|
||||
- [Pages & routes](#pages--routes)
|
||||
- [API endpoints](#api-endpoints)
|
||||
- [API documentation (Swagger)](#api-documentation-swagger)
|
||||
- [Shard integration (uo-link)](#shard-integration-uo-link)
|
||||
- [Modules](#modules)
|
||||
- [Environment variables](#environment-variables)
|
||||
- [Security](#security)
|
||||
- [Logging](#logging)
|
||||
@@ -48,8 +51,8 @@ The design reference is [BACKEND_DESIGN.md](https://gitea.whitlocktech.com/Runic
|
||||
## Architecture
|
||||
|
||||
How the pieces fit together — the React SPA and native app talk to one Express backend
|
||||
(`router → controller → model → db`), which persists to MariaDB and bridges to the live
|
||||
game world only through the **uo-link** sidecar. The shard itself is never internet-facing.
|
||||
(`router → controller → model → db`), which persists to MariaDB. Anything that knows
|
||||
what game this site is about lives in an installed module, on the right of the diagram.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
@@ -69,32 +72,29 @@ flowchart TB
|
||||
subgraph backend["server/ — Express backend"]
|
||||
direction TB
|
||||
mw["Middleware<br/>helmet · siteMode · noindex<br/>rateLimit · loginProtection · botScore · validate"]
|
||||
router["Router /api/v1<br/>auth (web · mobile · sso) · public · admin"]
|
||||
router["Router /api/v1<br/>auth (web · mobile · sso) · public · admin · player"]
|
||||
ctrl["Controllers"]
|
||||
auth["Session layer (auth/)<br/>sessionService · JWT/cookie · bearer · SSO+PKCE"]
|
||||
model["Models (.model + .db)<br/>raw parameterized SQL — no ORM"]
|
||||
sse["SSE fan-out<br/>public stream (allowlist) · admin stream (sensitive)"]
|
||||
|
||||
subgraph shardutil["Shard integration (utils/)"]
|
||||
ingest["shardIngest.js<br/>WS ingest dispatcher"]
|
||||
restcli["uoLinkClient.js<br/>REST client (never throws)"]
|
||||
end
|
||||
|
||||
loader["modules/loader.js<br/>scans the volume · mounts · registries · lifecycle"]
|
||||
secret["secretBox.js<br/>AES-256-GCM secrets at rest"]
|
||||
end
|
||||
|
||||
bot["bot/<br/>Discord bot"]
|
||||
end
|
||||
|
||||
db[("MariaDB<br/>users · posts · wiki · settings · activity<br/>mobileSessions · authProviders · userIdentities<br/>uoLinkConfig · shard_online/economy/houses/events")]
|
||||
db[("MariaDB<br/>users · posts · wiki · settings · activity<br/>mobileSessions · authProviders · userIdentities<br/>installed_modules · <module>_*")]
|
||||
|
||||
%% ---------- Shard side ----------
|
||||
subgraph shardside["Game shard (never internet-facing)"]
|
||||
%% ---------- Module side ----------
|
||||
subgraph modside["modules/<id>/ — installed, not built (e.g. Module-uo)"]
|
||||
direction TB
|
||||
sidecar["uo-link sidecar<br/>(Rust) — the only bridge exposed"]
|
||||
servuo["ServUO shard<br/>(C# plugin)"]
|
||||
modsrv["server/ — routers, models, schema fragment<br/>reaches core only through ctx"]
|
||||
modcli["client/dist/entry.js — prebuilt ESM chunk<br/>React shared via window.__rg"]
|
||||
end
|
||||
|
||||
game["The game<br/>whatever the module talks to<br/>(for Module-uo: a ServUO shard,<br/>via the uo-link sidecar)"]
|
||||
|
||||
%% ---------- Edges ----------
|
||||
browser <-->|"same-origin JSON + SSE (cookie)"| mw
|
||||
mobile -->|"REST (bearer access/refresh)"| mw
|
||||
@@ -104,40 +104,42 @@ flowchart TB
|
||||
mw --> router --> ctrl
|
||||
ctrl --> auth
|
||||
ctrl --> model
|
||||
ctrl --> restcli
|
||||
ctrl --> sse
|
||||
auth --> model
|
||||
model <--> db
|
||||
auth -. reads/writes secrets .-> secret
|
||||
restcli -. reads config/token .-> secret
|
||||
ingest --> model
|
||||
ingest --> sse
|
||||
sse -->|"live events"| browser
|
||||
bot -->|"messages"| discord
|
||||
bot <--> db
|
||||
|
||||
restcli -->|"REST: /char /roster /economy /history · /link/confirm · /towncrier"| sidecar
|
||||
sidecar -->|"WebSocket live event feed (bearer + X-UOLink-Version)"| ingest
|
||||
servuo -->|"loopback TCP 127.0.0.1:7788<br/>newline-delimited JSON (shard dials out)"| sidecar
|
||||
loader -->|"mounts under /api/v1/<tier>/<prefix>"| router
|
||||
loader -->|"require() + register(ctx, api)"| modsrv
|
||||
modsrv -->|"ctx.db · ctx.push · ctx.activity …"| model
|
||||
modsrv <--> game
|
||||
browser -->|"<script type=module> injected by htmlShell"| modcli
|
||||
|
||||
%% ---------- Styling ----------
|
||||
classDef ext fill:#2d2233,stroke:#7a5c94,color:#e8dff0;
|
||||
classDef store fill:#1f2d2a,stroke:#4c8c7d,color:#dff0ea;
|
||||
classDef bridge fill:#2d2620,stroke:#94764c,color:#f0e6d8;
|
||||
class idp,discord ext;
|
||||
classDef mod fill:#2d2620,stroke:#94764c,color:#f0e6d8;
|
||||
class idp,discord,game ext;
|
||||
class db store;
|
||||
class sidecar,servuo bridge;
|
||||
class modsrv,modcli mod;
|
||||
```
|
||||
|
||||
- **One backend, layered.** Every request flows `middleware → router → controller → model → db`.
|
||||
Web browsers authenticate with an httpOnly JWT cookie; the native app uses short-lived bearer
|
||||
access tokens plus rotated refresh tokens; SSO (Google/Discord/OIDC) is link-only and PKCE-guarded.
|
||||
All three surfaces produce the *same* session via the session layer.
|
||||
- **The shard is never reachable.** The ServUO shard *dials out* over loopback TCP to the uo-link
|
||||
sidecar; only the sidecar is exposed, and only the backend talks to it. The REST client
|
||||
(`uoLinkClient.js`) never throws, so the site degrades gracefully when the shard is down.
|
||||
- **Sensitive events stay private.** Ingested game events fan out to browsers over two SSE channels —
|
||||
a public allowlist stream and an admin-only stream that adds staff audit / cheat / login events.
|
||||
- **Core knows nothing about any game.** Routes, tables, nav entries, SPA pages and push streams for
|
||||
a specific game arrive from a module the operator installed. Core provides the seams; the module
|
||||
fills them. See [Modules](#modules).
|
||||
- **A module that fails must never take the site down.** The loader catches failures across a
|
||||
module's whole lifecycle and marks that one module `startup_failed`; the site comes up with its
|
||||
routes and nav absent, and the admin panel says why.
|
||||
- **Sensitive events stay private.** Events fan out to browsers over two SSE channels — a public
|
||||
allowlist stream and an admin-only stream that adds staff audit / cheat / login events. Which
|
||||
event kinds are public is decided by the module that publishes them, and core enforces the split.
|
||||
|
||||
---
|
||||
|
||||
@@ -164,12 +166,13 @@ website/
|
||||
│ │ ├─ server.js bootstrap: ensure schema → seed → listen (0.0.0.0)
|
||||
│ │ ├─ app.js middleware + static SPA + routes
|
||||
│ │ ├─ auth/ session layer: session.service · token (JWT/cookies) · session.middleware · ssoState (PKCE/CSRF) · providers/ (base · oauth2 · google · discord · genericOidc · registry)
|
||||
│ │ ├─ router/v1/ auth (web · mobile · sso) / public / admin route groups
|
||||
│ │ ├─ model/ users · posts · wiki · settings · activity · mobileSessions · authProviders · userIdentities (.model + .db)
|
||||
│ │ ├─ router/v1/ auth (web · mobile · sso) / public / admin / player route groups
|
||||
│ │ ├─ model/ users · posts · wiki · settings · activity · mobileSessions · authProviders · userIdentities · modules (.model + .db)
|
||||
│ │ ├─ modules/ loader (scan · validate · mount) · registries (the seams) · lifecycle (boot/shutdown + reconcile)
|
||||
│ │ ├─ middleware/ siteMode · noindex · rateLimit · loginProtection · botScore · validate
|
||||
│ │ └─ utils/ auth (compat facade) · totp (2FA) · secretBox (AES-GCM secrets) · db (pool) · mailer · logger
|
||||
│ │ └─ utils/ auth (compat facade) · totp (2FA) · secretBox (AES-GCM secrets) · db (pool) · mailer · logger · htmlShell
|
||||
│ ├─ db/ schema.sql + seed.js
|
||||
│ ├─ swagger/ swagger.js (OpenAPI generator config) + swagger-output.json (generated spec)
|
||||
│ ├─ swagger/ swagger.js (generator config) · swagger-output.json (generated, core only) · docsSpec.js (merges module fragments at request time)
|
||||
│ └─ .env.example
|
||||
├─ client/ React + Vite SPA
|
||||
│ ├─ src/
|
||||
@@ -178,9 +181,11 @@ website/
|
||||
│ │ ├─ routes/admin/ AdminLogin (password + TOTP + SSO buttons), AdminLayout, views/ (Dashboard, Posts, Wiki, Settings, Activity, Bot Activity, Authentication, Users, Account) + editors
|
||||
│ │ ├─ components/ SiteHeader, SiteFooter, layout, guards, Modal, ProviderIcon (inline SSO SVGs), …
|
||||
│ │ ├─ contexts/ AuthContext, SiteContext
|
||||
│ │ ├─ modules/ the client registry: routes · nav · slots · feature gates · window.__rg
|
||||
│ │ ├─ api/client.js fetch wrapper (sends cookies)
|
||||
│ │ └─ styles/theme.css design tokens
|
||||
│ └─ public/assets/img/ hero image
|
||||
├─ modules/ installed modules, one directory each — a Docker bind mount; empty here
|
||||
├─ Dockerfile builds client → serves via Express
|
||||
├─ docker-compose.yml app + MariaDB
|
||||
├─ .env.example root env (used by Compose)
|
||||
@@ -306,7 +311,7 @@ npm start # node server → serves API + SPA at http://localhost:3
|
||||
| `/site/screenshots` | Screenshot gallery |
|
||||
| `/site/five-on-friday` | Five on Friday |
|
||||
| `/site/newsletter` · `/site/newsletter/:id` | Newsletter list + issue |
|
||||
| `/site/about` · `/site/status` | About · Shard status |
|
||||
| `/site/about` · `/site/status` | About · Site status |
|
||||
| `/wiki` · `/wiki/:slug` | Wiki landing + article (auto table-of-contents) |
|
||||
|
||||
**Admin** (cookie auth, `noindex`):
|
||||
@@ -324,6 +329,13 @@ npm start # node server → serves API + SPA at http://localhost:3
|
||||
| `/admin/users` | User management |
|
||||
| `/admin/account` | Account security (self-service TOTP two-factor + linked SSO accounts) |
|
||||
|
||||
**Player** (any signed-in account, `noindex`): `/player` and its self-service views. Staff are a
|
||||
superset of players and reach these too.
|
||||
|
||||
An installed module adds its own pages under `/<id>/*`, `/admin/<id>/*` and `/player/<id>/*` — for
|
||||
Module-uo that is `/uo/shard`, `/admin/uo/link`, `/player/uo/characters` and the rest. Core does not
|
||||
know their names; they arrive with the module and are interleaved into the nav.
|
||||
|
||||
---
|
||||
|
||||
## API endpoints
|
||||
@@ -335,9 +347,15 @@ npm start # node server → serves API + SPA at http://localhost:3
|
||||
| SSO | `/api/v1/auth` (`providers` — public discovery; `sso/:provider/start`, `sso/:provider/link`, `sso/:provider/callback`) | redirect flow |
|
||||
| Public | `/api/v1/public` (`settings`, `status`, `posts/:category`, `posts/:category/:idOrSlug`, `wiki`, `wiki/:slug`, `contact`) | none |
|
||||
| Admin | `/api/v1/admin` (`dashboard`, `site-mode`, `posts`, `posts/upload`, `wiki`, `settings`, `activity`, `bot-activity`, `bot-activity/unban`, `auth/providers` (CRUD), `users`, `account`, `account/totp/*`, `account/identities`) | cookie (admin) |
|
||||
| Public · Shard | `/api/v1/public/shard` (`status`, `feed`, `economy`, `online`, `idoc`, `stream`) | none |
|
||||
| Player · Shard | `/api/v1/player/shard` (`link`, `accounts`, `roster/:account`, `vendors/:account`, `char/:serial`, `sales`) | cookie/bearer (player) |
|
||||
| Admin · Shard | `/api/v1/admin/shard` (self linking, same as player) · `/api/v1/admin/uo-link` (`config`, `towncrier`, `stream`) | cookie (staff / admin) |
|
||||
| Player | `/api/v1/player` (`me`, credentials, 2FA, identities, appeals) | cookie/bearer (any signed-in account) |
|
||||
| Modules | `/api/v1/public/modules` — id, name, version and capabilities of the modules currently serving | none |
|
||||
|
||||
**Module routes are not in this table**, because they are not core's. An installed module mounts
|
||||
under `/api/v1/public/<prefix>`, `/api/v1/admin/<prefix>` and `/api/v1/player/<prefix>`; which
|
||||
prefixes exist depends on what is installed. Module-uo, for instance, serves 72 routes under
|
||||
`/shard`, `/atlas` and `/uo-link` — see its own
|
||||
[`routes.manifest.json`](https://gitea.whitlocktech.com/RunicGateway/Module-uo/src/branch/main/routes.manifest.json).
|
||||
On a running instance, `/api/docs` lists everything, core and modules together.
|
||||
|
||||
Post categories (URL form): `news`, `five-on-friday`, `newsletter`, `screenshots`.
|
||||
`authMethod` on a session ∈ `local · totp · mobile · google · discord · oidc`.
|
||||
@@ -380,6 +398,28 @@ npm run swagger # → server/swagger/swagger-output.json
|
||||
If the generated spec is missing, the server logs a warning and simply disables `/api/docs` (it does
|
||||
not crash).
|
||||
|
||||
**The committed spec is core only, and the served one is not.** swagger-autogen is *static
|
||||
analysis* — it parses `src/app.js` as text and follows the literal `app.use(…)` chain — so it can
|
||||
see neither an installed module (which arrives on a volume long after the image was built, and
|
||||
mounts through a call no parser can follow) nor an extension slot (whose router is created empty and
|
||||
filled later). Both are handled by merging a **fragment**:
|
||||
|
||||
- **Extension slots** contribute at generation time, from `server/swagger/slotSpecs.js`, so they are
|
||||
in the committed file.
|
||||
- **Modules** contribute at request time, from the `swagger-fragment.json` each one ships, merged by
|
||||
`server/swagger/docsSpec.js`. So `/api/docs.json` on a running instance describes more than
|
||||
`npm run swagger` produces here, and `swagger-output.json` stays reproducible on any machine
|
||||
regardless of what is installed.
|
||||
|
||||
**Core wins every key collision** — a module cannot redefine a core path, tag or schema by shipping
|
||||
one with the same name; the collision is logged and the module's version dropped.
|
||||
|
||||
One thing worth knowing if you edit an annotation: swagger-autogen **reports a broken one and then
|
||||
succeeds anyway**, dropping it. `npm run swagger` now captures those diagnostics and fails, which is
|
||||
how two annotations that had been silently documenting an empty request body were found. If it
|
||||
rejects yours, the usual causes are an object literal a brace short, or a `"` or backtick inside a
|
||||
single-quoted description (it re-quotes both to `'` before evaluating).
|
||||
|
||||
### The route manifest (frozen URL surface)
|
||||
|
||||
`server/routes.manifest.json` is a generated, sorted `{ method, path }` list of every route the two
|
||||
@@ -412,94 +452,75 @@ annotated routes appear), the manifest records reality.
|
||||
|
||||
---
|
||||
|
||||
## Shard integration (uo-link)
|
||||
## Modules
|
||||
|
||||
The site is wired to the live in-game world through **uo-link**, a standalone sidecar service that
|
||||
runs next to the ServUO shard. Its source lives in a separate repo:
|
||||
**[RunicGateway/link](https://gitea.whitlocktech.com/RunicGateway/link)**. uo-link speaks the shard's internals and
|
||||
exposes a small, authenticated HTTP + WebSocket API; this website is a *client* of it. The shard
|
||||
itself is never exposed to the internet — only the sidecar is, and only the website's backend talks
|
||||
to it.
|
||||
**Everything specific to a game is a module.** Core has no idea what an "account", a "character" or
|
||||
a "shard" is; it provides seams, and a module fills them. That is what makes one image able to run a
|
||||
site for any game rather than for Ultima Online in particular.
|
||||
|
||||
### Setting up the shard side
|
||||
The design of record is
|
||||
[MODULE_SYSTEM.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_SYSTEM.md);
|
||||
the normative contract — the one to read before writing a module — is
|
||||
[MODULE_API.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_API.md).
|
||||
The worked example is [RunicGateway/Module-uo](https://gitea.whitlocktech.com/RunicGateway/Module-uo),
|
||||
which is where everything this README used to describe under *Shard integration (uo-link)* now
|
||||
lives: the sidecar client, the ingest dispatcher, account linking, the town crier, the spawn atlas,
|
||||
and every page that renders them.
|
||||
|
||||
You do not build or place any of it by hand. The
|
||||
**[Runic Gateway installer](https://gitea.whitlocktech.com/RunicGateway/installer)** runs on the
|
||||
shard host, deploys the ServUO plugin and the uo-link sidecar as a matched, protocol-checked pair,
|
||||
registers the sidecar as a service, and ends by printing the four values this site needs:
|
||||
### An operator never builds anything
|
||||
|
||||
That constraint shapes the whole design. Installing a module is the WordPress-plugin experience — an
|
||||
admin-panel action, or a directory dropped onto the `modules/` volume — because production runs a
|
||||
prebuilt, pull-only image with no toolchain in it. So a module ships **assembled**: its client half
|
||||
is a prebuilt ESM chunk that resolves React from a `window.__rg` global core owns (an import map
|
||||
would have to be inline, and the CSP is `script-src 'self'`), and its one runtime dependency travels
|
||||
inside the tarball.
|
||||
|
||||
```
|
||||
Base URL http://<shard-host>:8080
|
||||
WebSocket URL ws://<shard-host>:8080/ws
|
||||
Protocol version 3
|
||||
Auth token 4f9c…
|
||||
modules/
|
||||
└─ uo/ one directory per module; the id is the directory name
|
||||
├─ module.json id, version, coreApi range, mounts, extensions, capabilities
|
||||
├─ swagger-fragment.json merged into /api/docs.json while the module is running
|
||||
├─ server/ routers, models, and an idempotent schema.sql fragment
|
||||
└─ client/dist/entry.js the prebuilt chunk, injected by utils/htmlShell.js
|
||||
```
|
||||
|
||||
Paste them into **Admin → Shard** here and the bridge is live. The operator guide is
|
||||
[installer/INSTALL.md](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/installer/INSTALL.md);
|
||||
its [Appendix A](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/installer/INSTALL.md#appendix-a--installing-by-hand)
|
||||
is the same deployment done by hand, still supported, for a host that cannot run the binary or a
|
||||
developer working from a source tree.
|
||||
`modules/` is a bind mount in `docker-compose.yml`, so placing a directory there by hand is a
|
||||
supported install. The directory is tracked in git (via its README) on purpose: Docker recreates a
|
||||
*missing* bind-mount source as `root:root`, and the container is uid 1000.
|
||||
|
||||
Nothing here needs the shard to exist: with no sidecar configured the site renders normally and
|
||||
shows the shard offline.
|
||||
### What a module gets, and what it may not do
|
||||
|
||||
### How it works
|
||||
At boot, `app.js` scans the volume synchronously, validates each `module.json`, and calls the
|
||||
module's `register(ctx, api)`:
|
||||
|
||||
- **`ctx` is everything core hands over** — the database, the logger, settings, the session reader,
|
||||
push, the secret box, the middleware, the rate-limit factory, the activity log, and **express
|
||||
itself**. A module lives outside `server/`, so Node's resolver never reaches core's
|
||||
`node_modules`; anything it must share has to be handed to it, or there would be two Expresses and
|
||||
two Reacts in one process.
|
||||
- **`api` is everything it may register** — routes (one prefix per tier), an extension slot fill,
|
||||
notification streams, a news-announce leg, a post hook, and `onBoot`/`onShutdown`.
|
||||
- **It may not reach into core's tree**, mount outside its declared prefixes, or create tables
|
||||
outside its `<id>_` prefix. Each of those is checked, in the module's CI and again by the loader.
|
||||
|
||||
Two things are guaranteed regardless of what a module does. **A failure never takes the site down**:
|
||||
the loader catches everything from `require` to `onBoot`, marks that module `startup_failed`, and
|
||||
the site comes up with its routes and nav absent and the reason on the admin screen. And **no URL of
|
||||
core's may move** — a module that displaced one is caught by the frozen route manifest, which is
|
||||
generated from a real core with the module loaded.
|
||||
|
||||
### What is running right now
|
||||
|
||||
```
|
||||
ServUO shard ──▶ uo-link sidecar (RunicGateway/link) ──▶ website backend ──▶ browser
|
||||
REST + WebSocket, bearer-auth ingest + REST same-origin JSON/SSE
|
||||
GET /api/v1/public/modules
|
||||
{ "modules": [ { "id": "uo", "name": "Ultima Online", "version": "0.3.0",
|
||||
"capabilities": ["shard", "atlas", "market", …] } ] }
|
||||
```
|
||||
|
||||
- **Connection is admin-managed, not env.** The sidecar's base URL, WebSocket URL, shared-secret
|
||||
token, and protocol version are stored in the database (`uoLinkConfig`), edited from the
|
||||
**Admin → Shard** panel. The token is **encrypted at rest** (AES-256-GCM) and is **write-only** in
|
||||
the API — it is never returned to any client and never sent to the browser. Every call the backend
|
||||
makes carries `Authorization: Bearer <token>` and an `X-UOLink-Version` header (a protocol
|
||||
mismatch fails fast with `409` instead of being mis-parsed).
|
||||
- **Live ingest (WebSocket).** When enabled, the backend opens an outbound WebSocket to the sidecar
|
||||
and receives a stream of game events — `mob.login`/`logout`, `char.vitals`, `economy.supply`,
|
||||
`vendor.sale`, `player.death`/`murdered`, `house.decay` (IDOC), staff `audit.*`/`cheat.*`,
|
||||
`link.request`, and `server.hello`/`shutdown`. A single dispatcher (`utils/shardIngest.js`) routes
|
||||
each event: state-changing kinds update `shard_online` / `shard_economy` / `shard_houses`; notable
|
||||
kinds are appended to an append-only `shard_events` log; high-frequency kinds (vitals, supply
|
||||
ticks) only update state and are not logged. A changed boot id on `server.hello` is detected as a
|
||||
restart and stale "online" rows are cleared. On reconnect the backend backfills missed events via
|
||||
the sidecar's `/history`.
|
||||
- **Live round-trips (REST).** For point-in-time reads the backend calls the sidecar directly —
|
||||
`/char/serial/:serial`, `/roster/:account`, `/vendors/:account`, `/economy`, `/history` — plus
|
||||
commands `/link/confirm` and `/towncrier`. The REST client (`utils/uoLinkClient.js`) **never
|
||||
throws**: every call returns `{ ok, data, status }`, so a shard that is down or mid-restart
|
||||
degrades to a `503`/retry banner instead of a 500.
|
||||
- **Fan-out to the browser.** Ingested events are pushed to browsers over **Server-Sent Events**.
|
||||
Two channels exist: a **public** stream carrying only a safe allowlist of kinds, and an
|
||||
**admin-only** stream that also includes sensitive kinds (staff audit, cheat detection, login
|
||||
attempts, IPs). Sensitive kinds can never leak onto the public channel.
|
||||
|
||||
### Account linking
|
||||
|
||||
A player (or staff member) proves ownership of a game account without sharing any game credentials:
|
||||
|
||||
1. In game, the player runs **`[link`** and receives a one-time code.
|
||||
2. On the website (Player portal, or Admin → Account for staff) they enter the code.
|
||||
3. The backend confirms the code with the sidecar (`POST /link/confirm`), which permanently tags the
|
||||
game account with the website user id, and mirrors the link locally in `shard_account_links`.
|
||||
|
||||
That mirror is the authorization basis for character reads: roster/vendor/character-sheet endpoints
|
||||
are **ownership-checked** so a user only sees accounts they linked. **Admins may view any
|
||||
character**; players and editor/moderator staff are limited to their own linked accounts.
|
||||
|
||||
### What each audience sees
|
||||
|
||||
| Surface | Endpoints | Who | Data |
|
||||
|---|---|---|---|
|
||||
| **Public** | `/api/v1/public/shard/*` (`status`, `feed`, `economy`, `online`, `idoc`, `stream`) | anyone | Shard up/down, gold-supply series, IDOC houses, a curated live feed, and **"Staff online"** — only players whose account is linked to a **staff** user (admin/editor/moderator), shown with name + map location. Linked *players* are never listed publicly; no vitals or account are exposed. |
|
||||
| **Player** | `/api/v1/player/shard/*` (`link`, `accounts`, `roster/:account`, `vendors/:account`, `char/:serial`, `sales`) | logged-in player | Their own linked accounts: character rosters, character sheets, player-vendor snapshots, and recent vendor sales. |
|
||||
| **Admin** | `/api/v1/admin/shard/*` (self-linking, same as player) · `/api/v1/admin/uo-link/*` (`config`, `towncrier`, `stream`) | staff / admin | Staff link their own accounts like players; **admins** additionally read *any* character's data, edit the sidecar connection config, publish/remove **town-crier** messages, and subscribe to the full event stream (incl. audit/cheat). |
|
||||
|
||||
The sidecar URL and token are set once in **Admin → Shard**; if uo-link is not configured (or the
|
||||
shard is offline), every shard surface degrades gracefully — the public page still renders, showing
|
||||
the shard as offline.
|
||||
Anonymous, database-free, never site-mode gated, and **`started` modules only** — a module that is
|
||||
disabled or failed is absent, exactly as its routes and its nav already are. Clients feature-detect
|
||||
against it; they do not use it to decide what to load (the HTML shell injects each chunk's tag).
|
||||
|
||||
---
|
||||
|
||||
@@ -534,15 +555,14 @@ Copy `.env.example` (Compose) or `server/.env.example` (local) and fill in. **`.
|
||||
| `CLIENT_ORIGIN` | `http://localhost:5173` | enables CORS in dev only |
|
||||
| `LOG_LEVEL` / `FILE_LOG_LEVEL` | `info` / `debug` | console / file verbosity |
|
||||
| `LOG_TO_FILE` / `LOG_DIR` / `LOG_FILE` | `true` / `<server>/logs` / `app.log` | log file (bind-mounted to `./logs` in Docker) |
|
||||
| `ANNOUNCE_POLL_MS` | `15000` | how often the news-announcement dispatcher sweeps `announce_jobs` for due/retry legs (town crier + Discord) |
|
||||
| `TOWNCRIER_DURATION_SEC` | `3600` | how long a news post's in-game town-crier message stays up (≤ `86400`) |
|
||||
| `ANNOUNCE_POLL_MS` | `15000` | how often the news-announcement dispatcher sweeps `announce_jobs` for legs that are due or retrying. Which legs exist is up to what has registered one — Discord is core's; a module may add its own |
|
||||
|
||||
---
|
||||
|
||||
## Branding
|
||||
|
||||
Instance identity is data, not code — set via `BRAND_*` env vars, so one prebuilt
|
||||
image can run as any shard. With none set, everything renders as **Runic Gateway**.
|
||||
image can run as any community. With none set, everything renders as **Runic Gateway**.
|
||||
|
||||
| Var | What |
|
||||
|---|---|
|
||||
|
||||
@@ -99,13 +99,15 @@ CLIENT_ORIGIN=http://localhost:5173
|
||||
BOT_INTERNAL_URL=http://localhost:4100
|
||||
BOT_INTERNAL_KEY=dev-only-change-me-bot-key
|
||||
|
||||
# News announcement pipeline (published news post -> in-game town crier + Discord
|
||||
# #news). The dispatcher is an in-process poller; these tune it. Links in the
|
||||
# News announcement pipeline (published news post -> every registered delivery
|
||||
# leg). The dispatcher is an in-process poller; this tunes it. Links in the
|
||||
# announcements use APP_BASE_URL (set above), so set that in production too.
|
||||
# ANNOUNCE_POLL_MS how often the dispatcher sweeps for due/retry legs
|
||||
# TOWNCRIER_DURATION_SEC how long the in-game town-crier message stays up (<= 86400)
|
||||
#
|
||||
# Which legs exist depends on what has registered one: Discord (#news) is core's,
|
||||
# and an installed module may add its own. A module's leg brings its own settings
|
||||
# with it -- module-uo's in-game town crier reads TOWNCRIER_DURATION_SEC, which is
|
||||
# documented in that module rather than here, because core has no town crier.
|
||||
ANNOUNCE_POLL_MS=15000
|
||||
TOWNCRIER_DURATION_SEC=3600
|
||||
|
||||
# Push notifications (M7) — opt-in fan-out to the Android app via a self-hosted
|
||||
# ntfy UnifiedPush relay (docs/android/PLAN.md §11). The publisher POSTs
|
||||
@@ -122,8 +124,24 @@ TOWNCRIER_DURATION_SEC=3600
|
||||
# NTFY_PUBLISH_TOKEN Optional bearer token for backend->ntfy publishes (off by default).
|
||||
# Leave NTFY_BASE_URL unset in local dev to allow any public HTTPS endpoint
|
||||
# (private/loopback hosts are always rejected). Without NTFY_PUBLIC_URL /
|
||||
# NTFY_ALLOWED_ORIGINS the app shows push as unavailable for the shard.
|
||||
# NTFY_ALLOWED_ORIGINS the app shows push as unavailable for this instance.
|
||||
# NTFY_BASE_URL=https://ntfy.example.com
|
||||
# NTFY_PUBLIC_URL=https://ntfy.example.com
|
||||
# NTFY_ALLOWED_ORIGINS=https://ntfy.example.com
|
||||
# NTFY_PUBLISH_TOKEN=
|
||||
|
||||
# Modules (MODULE_SYSTEM.md §2.5) — where installable modules live, and where
|
||||
# they may be installed from.
|
||||
# MODULES_DIR Directory the loader scans at require time. Defaults to
|
||||
# <repo>/modules; docker-compose.yml sets it to /app/modules,
|
||||
# which is the bind mount that makes it meaningful.
|
||||
# MODULE_SOURCE_HOSTS BOOTSTRAP ONLY. Comma-separated hostnames the admin panel
|
||||
# may install a module from, seeded into the `module_source_hosts`
|
||||
# setting the first time the site boots without one. From then
|
||||
# on the SETTING is authoritative and is edited in
|
||||
# Admin → Modules — changing this variable on an existing
|
||||
# deployment does nothing, deliberately, so a redeploy cannot
|
||||
# silently undo an operator's choice. Installs are https-only
|
||||
# and an empty list forbids all of them.
|
||||
# MODULES_DIR=/app/modules
|
||||
# MODULE_SOURCE_HOSTS=gitea.whitlocktech.com
|
||||
|
||||
@@ -23,6 +23,17 @@ const DEFAULT_SETTINGS = {
|
||||
site_title: brand.name,
|
||||
// Android App Links opt-in — off until an admin enables it (docs/android/APP_LINKS.md).
|
||||
mobile_app_links_enabled: 'false',
|
||||
// Hosts a module may be installed from (MODULE_SYSTEM.md §2.7.2 decision 6).
|
||||
//
|
||||
// The environment BOOTSTRAPS this and does not own it: seedDefault is an
|
||||
// INSERT IGNORE, so the variable supplies a sane default on a fresh install
|
||||
// and never reaches back in to overwrite what an admin later chose in
|
||||
// Admin → Modules. Changing MODULE_SOURCE_HOSTS on an existing deployment is
|
||||
// therefore a no-op, which is the intended behaviour and not an oversight.
|
||||
//
|
||||
// An empty stored value forbids every install rather than allowing every host
|
||||
// — the safe direction for a setting someone might blank by accident.
|
||||
module_source_hosts: process.env.MODULE_SOURCE_HOSTS || 'gitea.whitlocktech.com',
|
||||
}
|
||||
|
||||
// Starter wiki sections (editable later via the admin panel).
|
||||
|
||||
68
server/package-lock.json
generated
68
server/package-lock.json
generated
@@ -27,6 +27,7 @@
|
||||
"sanitize-html": "^2.17.5",
|
||||
"speakeasy": "^2.0.0",
|
||||
"swagger-ui-express": "^5.0.1",
|
||||
"tar": "^7.5.22",
|
||||
"ws": "^8.21.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
@@ -34,6 +35,18 @@
|
||||
"swagger-autogen": "^2.23.7"
|
||||
}
|
||||
},
|
||||
"node_modules/@isaacs/fs-minipass": {
|
||||
"version": "4.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@isaacs/fs-minipass/-/fs-minipass-4.0.1.tgz",
|
||||
"integrity": "sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w==",
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"minipass": "^7.0.4"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@scarf/scarf": {
|
||||
"version": "1.4.0",
|
||||
"resolved": "https://registry.npmjs.org/@scarf/scarf/-/scarf-1.4.0.tgz",
|
||||
@@ -330,6 +343,15 @@
|
||||
"fsevents": "~2.3.2"
|
||||
}
|
||||
},
|
||||
"node_modules/chownr": {
|
||||
"version": "3.0.0",
|
||||
"resolved": "https://registry.npmjs.org/chownr/-/chownr-3.0.0.tgz",
|
||||
"integrity": "sha512-+IxzY9BZOQd/XuYPRmrvEVjF/nqj5kgT4kEq7VofrDoM1MxoRjEWkrCC3EtLi59TVawxTAn+orJwFQcrqEN1+g==",
|
||||
"license": "BlueOak-1.0.0",
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/cliui": {
|
||||
"version": "6.0.0",
|
||||
"resolved": "https://registry.npmjs.org/cliui/-/cliui-6.0.0.tgz",
|
||||
@@ -1488,6 +1510,27 @@
|
||||
"url": "https://github.com/sponsors/isaacs"
|
||||
}
|
||||
},
|
||||
"node_modules/minipass": {
|
||||
"version": "7.1.3",
|
||||
"resolved": "https://registry.npmjs.org/minipass/-/minipass-7.1.3.tgz",
|
||||
"integrity": "sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==",
|
||||
"license": "BlueOak-1.0.0",
|
||||
"engines": {
|
||||
"node": ">=16 || 14 >=14.17"
|
||||
}
|
||||
},
|
||||
"node_modules/minizlib": {
|
||||
"version": "3.1.0",
|
||||
"resolved": "https://registry.npmjs.org/minizlib/-/minizlib-3.1.0.tgz",
|
||||
"integrity": "sha512-KZxYo1BUkWD2TVFLr0MQoM8vUUigWD3LlD83a/75BqC+4qE0Hb1Vo5v1FgcfaNXvfXzr+5EhQ6ing/CaBijTlw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"minipass": "^7.1.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 18"
|
||||
}
|
||||
},
|
||||
"node_modules/morgan": {
|
||||
"version": "1.11.0",
|
||||
"resolved": "https://registry.npmjs.org/morgan/-/morgan-1.11.0.tgz",
|
||||
@@ -2254,6 +2297,22 @@
|
||||
"express": ">=4.0.0 || >=5.0.0-beta"
|
||||
}
|
||||
},
|
||||
"node_modules/tar": {
|
||||
"version": "7.5.22",
|
||||
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.22.tgz",
|
||||
"integrity": "sha512-MFO/QzvtAOmJbkhOaCTvbGcFN9L9b+JunIsDwaKljSOdcLMea3NJ1k9Usz/rjdfSXTq4dfzfeS7W4p4YOAAHeA==",
|
||||
"license": "BlueOak-1.0.0",
|
||||
"dependencies": {
|
||||
"@isaacs/fs-minipass": "^4.0.0",
|
||||
"chownr": "^3.0.0",
|
||||
"minipass": "^7.1.2",
|
||||
"minizlib": "^3.1.0",
|
||||
"yallist": "^5.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/to-regex-range": {
|
||||
"version": "5.0.1",
|
||||
"resolved": "https://registry.npmjs.org/to-regex-range/-/to-regex-range-5.0.1.tgz",
|
||||
@@ -2414,6 +2473,15 @@
|
||||
"integrity": "sha512-JKhqTOwSrqNA1NY5lSztJ1GrBiUodLMmIZuLiDaMRJ+itFd+ABVE8XBjOvIWL+rSqNDC74LCSFmlb/U4UZ4hJQ==",
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/yallist": {
|
||||
"version": "5.0.0",
|
||||
"resolved": "https://registry.npmjs.org/yallist/-/yallist-5.0.0.tgz",
|
||||
"integrity": "sha512-YgvUTfwqyc7UXVMrB+SImsVYSmTS8X/tSrtdNZMImM+n7+QTriRXyXim0mBrTXNeqzVF0KWGgHPeiyViFFrNDw==",
|
||||
"license": "BlueOak-1.0.0",
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/yargs": {
|
||||
"version": "15.4.1",
|
||||
"resolved": "https://registry.npmjs.org/yargs/-/yargs-15.4.1.tgz",
|
||||
|
||||
@@ -38,6 +38,7 @@
|
||||
"sanitize-html": "^2.17.5",
|
||||
"speakeasy": "^2.0.0",
|
||||
"swagger-ui-express": "^5.0.1",
|
||||
"tar": "^7.5.22",
|
||||
"ws": "^8.21.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
|
||||
@@ -427,6 +427,90 @@
|
||||
"validate"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/modules",
|
||||
"handlers": 2,
|
||||
"gates": [
|
||||
"noindex",
|
||||
"requireAuth"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules",
|
||||
"handlers": 4,
|
||||
"gates": [
|
||||
"noindex",
|
||||
"requireAuth",
|
||||
"middleware",
|
||||
"validate"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/modules/:id",
|
||||
"handlers": 5,
|
||||
"gates": [
|
||||
"noindex",
|
||||
"requireAuth",
|
||||
"middleware",
|
||||
"validate"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules/:id/disable",
|
||||
"handlers": 4,
|
||||
"gates": [
|
||||
"noindex",
|
||||
"requireAuth",
|
||||
"middleware",
|
||||
"validate"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules/:id/enable",
|
||||
"handlers": 4,
|
||||
"gates": [
|
||||
"noindex",
|
||||
"requireAuth",
|
||||
"middleware",
|
||||
"validate"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules/:id/purge",
|
||||
"handlers": 4,
|
||||
"gates": [
|
||||
"noindex",
|
||||
"requireAuth",
|
||||
"middleware",
|
||||
"validate"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules/restart",
|
||||
"handlers": 2,
|
||||
"gates": [
|
||||
"noindex",
|
||||
"requireAuth"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/modules/sources",
|
||||
"handlers": 4,
|
||||
"gates": [
|
||||
"noindex",
|
||||
"requireAuth",
|
||||
"middleware",
|
||||
"validate"
|
||||
]
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/pages",
|
||||
|
||||
@@ -177,6 +177,38 @@
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/moderation/user/:discordId/notes"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/modules"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/modules/:id"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules/:id/disable"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules/:id/enable"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules/:id/purge"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/modules/restart"
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/modules/sources"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/pages"
|
||||
|
||||
@@ -111,15 +111,22 @@ app.use(
|
||||
)
|
||||
|
||||
// ── API docs (Swagger UI) ─────────────────────────────────────────────
|
||||
// Interactive OpenAPI docs at /api/docs, raw spec at /api/docs.json. The spec
|
||||
// is generated from route annotations by `npm run swagger` (server/swagger/).
|
||||
// Interactive OpenAPI docs at /api/docs, raw spec at /api/docs.json. Core's own
|
||||
// routes are generated from their annotations by `npm run swagger`
|
||||
// (server/swagger/) and committed; an installed module's routes cannot be —
|
||||
// swagger-autogen is static analysis and a module arrives on the volume after the
|
||||
// image was built — so each module ships its own fragment and they are merged
|
||||
// HERE, per request, over core's committed spec (docs/website/MODULE_API.md §6.1a).
|
||||
// Loaded lazily and guarded so a missing spec never crashes the server.
|
||||
try {
|
||||
// eslint-disable-next-line global-require
|
||||
/* eslint-disable global-require */
|
||||
const swaggerSpec = require('../swagger/swagger-output.json')
|
||||
const { docsSpec } = require('../swagger/docsSpec')
|
||||
/* eslint-enable global-require */
|
||||
|
||||
app.get('/api/docs.json', (req, res) => {
|
||||
// #swagger.ignore = true
|
||||
res.json(swaggerSpec)
|
||||
res.json(docsSpec(swaggerSpec))
|
||||
})
|
||||
// swagger-ui-express injects an inline bootstrap script and inline styles, which
|
||||
// the global 'self'-only script-src would block — relax CSP for this route only.
|
||||
@@ -132,10 +139,18 @@ try {
|
||||
'upgrade-insecure-requests': null,
|
||||
},
|
||||
})
|
||||
app.use('/api/docs', swaggerCsp, swaggerUi.serve, swaggerUi.setup(swaggerSpec, {
|
||||
// `setup()` is called PER REQUEST rather than once here, because the document it
|
||||
// renders is not fixed at boot: a module reaching `started` (or failing to) adds
|
||||
// or removes paths, and a UI bound to the spec as it looked while app.js was
|
||||
// still being required would show core's routes for the life of the process
|
||||
// while /api/docs.json showed the merged set. `docsSpec` is cached on the
|
||||
// loader's state version, so the repeated call costs a comparison.
|
||||
const swaggerOpts = {
|
||||
customSiteTitle: `${brand.name} API docs`,
|
||||
swaggerOptions: { persistAuthorization: true },
|
||||
}))
|
||||
}
|
||||
app.use('/api/docs', swaggerCsp, swaggerUi.serve, (req, res, next) =>
|
||||
swaggerUi.setup(docsSpec(swaggerSpec), swaggerOpts)(req, res, next))
|
||||
} catch (err) {
|
||||
errLog.error('Swagger spec not found — run `npm run swagger` to generate it. API docs disabled.', {
|
||||
message: err.message,
|
||||
|
||||
@@ -17,6 +17,23 @@ const getOne = (id) => query(`SELECT ${COLS} FROM installed_modules WHERE id = ?
|
||||
// module must not silently disable it, and re-installing a disabled one must not
|
||||
// silently switch it back on. A brand-new row lands in `installed`, the transient
|
||||
// state the next restart resolves.
|
||||
// Provenance is COALESCEd, and that is the whole difference between this
|
||||
// working and not.
|
||||
//
|
||||
// `lifecycle.boot()` re-records every module it scanned with no source and no
|
||||
// sha256 — a directory placed on the volume by hand genuinely has neither, and
|
||||
// the boot has no way to know where one came from. With a plain
|
||||
// `source = VALUES(source)` that refresh overwrote both columns with NULL on
|
||||
// EVERY boot, so an admin-panel install's provenance survived exactly until the
|
||||
// restart that install asked for. Nothing could catch it before Phase 4: until
|
||||
// then no caller ever passed a non-null value, and lifecycle.js's comment
|
||||
// asserting that "recordInstalled leaves what it is not given" was a description
|
||||
// of an intention rather than of this statement.
|
||||
//
|
||||
// COALESCE makes that comment true: a value overwrites, a NULL leaves what is
|
||||
// there. The cost is that re-placing a DIFFERENT bundle by hand over a row that
|
||||
// was once installed from a URL keeps the old provenance — which is wrong but
|
||||
// stale, and strictly better than the alternative, which was wrong and blank.
|
||||
const upsert = ({ id, name, version, source, sha256 }) =>
|
||||
query(
|
||||
`INSERT INTO installed_modules (id, name, version, source, sha256, state)
|
||||
@@ -24,8 +41,8 @@ const upsert = ({ id, name, version, source, sha256 }) =>
|
||||
ON DUPLICATE KEY UPDATE
|
||||
name = VALUES(name),
|
||||
version = VALUES(version),
|
||||
source = VALUES(source),
|
||||
sha256 = VALUES(sha256)`,
|
||||
source = COALESCE(VALUES(source), source),
|
||||
sha256 = COALESCE(VALUES(sha256), sha256)`,
|
||||
[id, name, version, source ?? null, sha256 ?? null],
|
||||
)
|
||||
|
||||
|
||||
221
server/src/modules/archive.js
Normal file
221
server/src/modules/archive.js
Normal file
@@ -0,0 +1,221 @@
|
||||
// ── The hardened bundle extractor ──────────────────────────────────────────
|
||||
//
|
||||
// Phase 4, slice 1 of docs/website/MODULE_SYSTEM.md §2.7.2. This is the part of
|
||||
// the install path that handles input an attacker chose, and it is separated
|
||||
// from install.js so that it can be tested against crafted archives without a
|
||||
// network, a database or a filesystem layout.
|
||||
//
|
||||
// **The download is not the dangerous part.** An allowlisted host, a declared
|
||||
// sha256 and a size cap between them settle where the bytes came from and that
|
||||
// they are the bytes that were published. Unpacking is different: the ARCHIVE
|
||||
// chooses the filenames, and core writes into a directory bind-mounted from the
|
||||
// host (MODULE_SYSTEM.md §2.5), so an escape is not confined to the container.
|
||||
//
|
||||
// The rule this file follows is **reject, never sanitise.** node-tar will
|
||||
// happily strip a leading `/` and drop a `..` for you, which turns a hostile
|
||||
// archive into a slightly different archive that then gets installed. An archive
|
||||
// that needs correcting is an archive that should not be trusted, so anything on
|
||||
// the list below fails the whole install and nothing is written.
|
||||
//
|
||||
// - an absolute path, POSIX (`/etc/…`) or Windows (`C:\…`, `\\server\…`)
|
||||
// - any `..` segment, anywhere
|
||||
// - anything that is not a regular file or a directory — so no symlinks, no
|
||||
// hardlinks, no devices, no FIFOs. A module bundle has no legitimate use for
|
||||
// any of them, and every one is a documented escape primitive
|
||||
// - more than one top-level entry
|
||||
// - more entries, or more unpacked bytes, than the caps below
|
||||
//
|
||||
// That third rule deserves its own note, because it is why the `tar` dependency
|
||||
// is pinned forward rather than merely present: **the majority of node-tar's
|
||||
// published advisories are hardlink or symlink path traversal**, several of them
|
||||
// through interpretation differences in PAX and GNU long-name headers rather
|
||||
// than through anything the calling code did wrong. Refusing those entry types
|
||||
// outright means this extractor is not relying on the library to get their
|
||||
// containment right. The remaining advisories are parser denial-of-service, and
|
||||
// those are what the caps and the two-pass shape address.
|
||||
//
|
||||
// Two passes, deliberately: `inspect()` reads the archive and decides, and only
|
||||
// an archive that survived that is handed to `extract()`. A `filter` callback
|
||||
// during extraction cannot reject — by the time it is asked about entry 400, the
|
||||
// first 399 are already on disk.
|
||||
|
||||
const fs = require('fs')
|
||||
const fsp = require('fs/promises')
|
||||
const path = require('path')
|
||||
const tar = require('tar')
|
||||
|
||||
// Caps. Generous against a real bundle (module-uo's is a few megabytes, a few
|
||||
// hundred files) and small enough that a decompression bomb is refused rather
|
||||
// than paged in. Both are checked DURING the inspect pass, not after it, so a
|
||||
// hostile archive stops being read at the limit instead of at its end.
|
||||
const MAX_BYTES = 128 * 1024 * 1024
|
||||
const MAX_ENTRIES = 20000
|
||||
|
||||
// tar entry types worth naming. node-tar reports these as strings; anything not
|
||||
// in this set is refused, including the ones no one has thought of, which is the
|
||||
// point of an allowlist here rather than a list of the types known to be bad.
|
||||
const ALLOWED_TYPES = new Set(['File', 'Directory'])
|
||||
|
||||
class ArchiveError extends Error {
|
||||
constructor(message) {
|
||||
super(message)
|
||||
this.name = 'ArchiveError'
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Is this entry path safe to unpack anywhere?
|
||||
*
|
||||
* Returns a reason string, or null when the path is fine. Written as a reason
|
||||
* rather than a boolean because the operator pasting a URL needs to be told what
|
||||
* was wrong with what they were served, and "the archive is invalid" is not that.
|
||||
*/
|
||||
function pathProblem(entryPath) {
|
||||
const p = String(entryPath)
|
||||
|
||||
// NUL is not a path character. node-tar has had uncaught-exception advisories
|
||||
// for NUL bytes inside PAX records, so this is checked here rather than left
|
||||
// to the parser.
|
||||
if (p.includes('\0')) return 'an entry path contains a NUL byte'
|
||||
|
||||
// A backslash is a path separator on the platform this may be unpacked on, so
|
||||
// an entry that contains one is not the single path component it looks like.
|
||||
if (p.includes('\\')) return `an entry path contains a backslash: "${p}"`
|
||||
|
||||
if (p.startsWith('/')) return `an entry path is absolute: "${p}"`
|
||||
// Drive-relative and UNC. Both are the subject of their own node-tar
|
||||
// advisories, and neither is refused by a leading-slash check.
|
||||
if (/^[a-zA-Z]:/.test(p)) return `an entry path is drive-absolute: "${p}"`
|
||||
|
||||
const segments = p.split('/')
|
||||
if (segments.includes('..')) return `an entry path escapes upward: "${p}"`
|
||||
|
||||
return null
|
||||
}
|
||||
|
||||
/**
|
||||
* Read an archive without unpacking it, and decide whether it may be unpacked.
|
||||
*
|
||||
* @param {string} file absolute path to the .tar.gz
|
||||
* @returns {Promise<{root: string, entries: number, bytes: number}>} the single
|
||||
* top-level directory name, and what is inside it.
|
||||
* @throws {ArchiveError} on anything in this file's header list.
|
||||
*/
|
||||
async function inspect(file) {
|
||||
const roots = new Set()
|
||||
let entries = 0
|
||||
let bytes = 0
|
||||
// The first problem found, kept rather than thrown from inside the callback:
|
||||
// throwing out of `onentry` escapes through the parser's stream machinery and
|
||||
// arrives as an unhelpful wrapped error, when it arrives at all.
|
||||
let problem = null
|
||||
|
||||
const note = (message) => {
|
||||
if (!problem) problem = message
|
||||
}
|
||||
|
||||
await tar.t({
|
||||
file,
|
||||
onentry(entry) {
|
||||
if (problem) return
|
||||
|
||||
entries += 1
|
||||
if (entries > MAX_ENTRIES) {
|
||||
note(`the archive has more than ${MAX_ENTRIES} entries`)
|
||||
return
|
||||
}
|
||||
|
||||
if (!ALLOWED_TYPES.has(entry.type)) {
|
||||
// The message names the type because "symbolic link" is a much more
|
||||
// useful thing for an operator to read than "invalid entry".
|
||||
note(`the archive contains a ${entry.type} ("${entry.path}") — a module bundle may only contain files and directories`)
|
||||
return
|
||||
}
|
||||
|
||||
const bad = pathProblem(entry.path)
|
||||
if (bad) {
|
||||
note(bad)
|
||||
return
|
||||
}
|
||||
|
||||
// A negative or absurd size is a parser-confusion primitive in its own
|
||||
// right; clamping at zero keeps the running total honest.
|
||||
bytes += Math.max(0, Number(entry.size) || 0)
|
||||
if (bytes > MAX_BYTES) {
|
||||
note(`the archive unpacks to more than ${Math.round(MAX_BYTES / 1024 / 1024)} MB`)
|
||||
return
|
||||
}
|
||||
|
||||
const [root] = String(entry.path).split('/')
|
||||
if (root) roots.add(root)
|
||||
},
|
||||
})
|
||||
|
||||
if (problem) throw new ArchiveError(problem)
|
||||
if (entries === 0) throw new ArchiveError('the archive is empty')
|
||||
if (roots.size !== 1) {
|
||||
// A bundle is one directory. More than one top-level entry means `strip: 1`
|
||||
// below would silently merge or discard things, and a bundle that needs
|
||||
// interpreting is not a bundle.
|
||||
throw new ArchiveError(
|
||||
`the archive must contain exactly one top-level directory, found ${roots.size}` +
|
||||
(roots.size > 1 ? ` (${[...roots].slice(0, 4).join(', ')})` : ''),
|
||||
)
|
||||
}
|
||||
|
||||
return { root: [...roots][0], entries, bytes }
|
||||
}
|
||||
|
||||
/**
|
||||
* Unpack an inspected archive into `dest`, stripping its single top level.
|
||||
*
|
||||
* The top-level directory is stripped rather than kept, because its name belongs
|
||||
* to whoever published the bundle — module-uo's release workflow packs
|
||||
* `module-uo-<version>/`, not `uo/` — while the directory it lands in is core's
|
||||
* decision and has to be the module id the loader scans for. What is inside the
|
||||
* bundle is the module; what the wrapper is called is packaging.
|
||||
*
|
||||
* `dest` must not exist. Callers unpack to a temporary directory and move it
|
||||
* into place, so a failure halfway through leaves nothing for the next boot's
|
||||
* scan to find.
|
||||
*/
|
||||
async function extract(file, dest) {
|
||||
if (fs.existsSync(dest)) throw new ArchiveError(`extraction target already exists: ${dest}`)
|
||||
await fsp.mkdir(dest, { recursive: true })
|
||||
|
||||
await tar.x({
|
||||
file,
|
||||
cwd: dest,
|
||||
strip: 1,
|
||||
// Belt and braces on top of inspect(): the library's own refusal to write
|
||||
// outside cwd, and its refusal to overwrite through a link. Neither is
|
||||
// load-bearing here — inspect() has already rejected every entry that could
|
||||
// exercise them — but a second lock on a door that is already locked costs
|
||||
// nothing, and this is the door.
|
||||
preservePaths: false,
|
||||
// Reject rather than warn on anything the parser itself objects to.
|
||||
strict: true,
|
||||
})
|
||||
|
||||
return dest
|
||||
}
|
||||
|
||||
/**
|
||||
* Inspect, then extract. The only entry point install.js uses.
|
||||
*/
|
||||
async function unpack(file, dest) {
|
||||
const stats = await inspect(file)
|
||||
await extract(file, dest)
|
||||
return stats
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
ArchiveError,
|
||||
inspect,
|
||||
extract,
|
||||
unpack,
|
||||
pathProblem,
|
||||
MAX_BYTES,
|
||||
MAX_ENTRIES,
|
||||
ALLOWED_TYPES,
|
||||
}
|
||||
421
server/src/modules/install.js
Normal file
421
server/src/modules/install.js
Normal file
@@ -0,0 +1,421 @@
|
||||
// ── Installing and removing a module bundle ────────────────────────────────
|
||||
//
|
||||
// Phase 4, slice 1 of docs/website/MODULE_SYSTEM.md §2.7.2 — the consumer half
|
||||
// of a release the module's own CI already publishes. Nothing here touches the
|
||||
// database or the loader: this file moves bytes onto the volume and off it, and
|
||||
// the caller (router/v1/admin/modules.controller.js) writes down what happened.
|
||||
//
|
||||
// The shape of an install, and why it is this shape:
|
||||
//
|
||||
// 1. The operator pastes the URL of a release's install manifest — a small
|
||||
// JSON document naming the artifact, its sha256 and its size (decision 2).
|
||||
// There is no catalog, because a catalog would make core's release cadence
|
||||
// decide which modules can exist.
|
||||
// 2. Every URL fetched — the manifest, the artifact, and every redirect hop —
|
||||
// is checked against the admin-managed host allowlist (decision 6). That is
|
||||
// what keeps a pasted URL from also being an SSRF primitive.
|
||||
// 3. The artifact is streamed to a temporary file under a byte cap, hashed as
|
||||
// it arrives, and compared against the manifest's `sha256`. The hash is the
|
||||
// trust anchor; releases are unsigned and say so.
|
||||
// 4. modules/archive.js inspects the archive in full before a single byte is
|
||||
// unpacked, and only then unpacks it — into a TEMPORARY directory.
|
||||
// 5. The unpacked tree's own `module.json` must agree with the manifest about
|
||||
// what it is. A manifest that promises `uo` and delivers something else is
|
||||
// refused rather than installed under the name it was promised.
|
||||
// 6. Only then is anything moved into `modules/<id>/`, and the directory it
|
||||
// replaces is kept aside until the move has succeeded.
|
||||
//
|
||||
// **Nothing is written into the modules directory until every check has passed**,
|
||||
// and that is not belt-and-braces. Verified against node-tar 7.5.22 while writing
|
||||
// this: extracting an archive whose fourth member escapes upward throws — and
|
||||
// leaves the first three members on disk. A loader scans that directory at
|
||||
// require time on the next boot; a half-unpacked module is a module.
|
||||
//
|
||||
// One thing this file deliberately does NOT do: mount anything. Installing puts
|
||||
// a directory on the volume, and §1.12 means the volume is read at require time,
|
||||
// so the module appears when the process restarts. The admin screen offers that
|
||||
// restart (decision 1); it is not implied here.
|
||||
|
||||
const crypto = require('crypto')
|
||||
const fs = require('fs')
|
||||
const fsp = require('fs/promises')
|
||||
const path = require('path')
|
||||
const { Readable, Transform } = require('stream')
|
||||
const { pipeline } = require('stream/promises')
|
||||
|
||||
const archive = require('./archive')
|
||||
const loader = require('./loader')
|
||||
|
||||
const log = require('../utils/logger')('modules')
|
||||
|
||||
// An install manifest is a small JSON document. A megabyte of it is not a
|
||||
// manifest, and reading it into memory unbounded is the one place this file
|
||||
// would otherwise trust a remote length.
|
||||
const MAX_MANIFEST_BYTES = 256 * 1024
|
||||
// The artifact cap is the archive's own unpacked cap — a compressed bundle
|
||||
// larger than what it is allowed to unpack to has nothing to offer.
|
||||
const MAX_ARTIFACT_BYTES = archive.MAX_BYTES
|
||||
const FETCH_TIMEOUT_MS = 30000
|
||||
// Gitea serves a release asset through at least one redirect. Following them is
|
||||
// necessary; following them blindly is how an allowlist gets bypassed, so each
|
||||
// hop is re-checked and the chain is bounded.
|
||||
const MAX_REDIRECTS = 5
|
||||
|
||||
// Same id rule the loader enforces when scanning, restated rather than imported
|
||||
// so an install cannot put a directory on the volume that the loader would then
|
||||
// refuse to look at.
|
||||
const ID = /^[a-z][a-z0-9-]{1,31}$/
|
||||
const SHA256 = /^[0-9a-f]{64}$/
|
||||
|
||||
class InstallError extends Error {
|
||||
constructor(message, { status = 400 } = {}) {
|
||||
super(message)
|
||||
this.name = 'InstallError'
|
||||
// Carried so the controller can answer 400 for "your URL is wrong" and 502
|
||||
// for "the host you named misbehaved" without re-deriving it from the text.
|
||||
this.status = status
|
||||
}
|
||||
}
|
||||
|
||||
// ── The allowlist ──────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Parse the stored allowlist setting into hostnames.
|
||||
*
|
||||
* Comma or whitespace separated, case-insensitive, empty entries dropped. An
|
||||
* empty list means nothing may be installed from anywhere — a refusal, never a
|
||||
* wildcard. That direction matters: a setting an admin accidentally blanks
|
||||
* should stop installs, not permit every host on the internet.
|
||||
*/
|
||||
function parseHosts(value) {
|
||||
return String(value || '')
|
||||
.split(/[,\s]+/)
|
||||
.map((h) => h.trim().toLowerCase())
|
||||
.filter(Boolean)
|
||||
}
|
||||
|
||||
/**
|
||||
* Check one URL against the allowlist, and return it parsed.
|
||||
*
|
||||
* `https` only. A plaintext fetch of code this process is going to execute is
|
||||
* not something an operator should be able to opt into by typing a URL, and the
|
||||
* sha256 does not help — whoever can rewrite the artifact in flight can rewrite
|
||||
* the manifest that declares its hash.
|
||||
*/
|
||||
function checkUrl(raw, hosts) {
|
||||
let url
|
||||
try {
|
||||
url = new URL(String(raw))
|
||||
} catch {
|
||||
throw new InstallError(`"${raw}" is not a valid URL`)
|
||||
}
|
||||
if (url.protocol !== 'https:') {
|
||||
throw new InstallError(`only https URLs may be installed from (got "${url.protocol}")`)
|
||||
}
|
||||
if (!hosts.length) {
|
||||
throw new InstallError(
|
||||
'no module source hosts are allowed — set one in Admin → Modules before installing',
|
||||
)
|
||||
}
|
||||
if (!hosts.includes(url.hostname.toLowerCase())) {
|
||||
throw new InstallError(
|
||||
`"${url.hostname}" is not an allowed module source host (allowed: ${hosts.join(', ')})`,
|
||||
)
|
||||
}
|
||||
return url
|
||||
}
|
||||
|
||||
// ── Fetching ───────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* GET a URL, following redirects by hand so every hop is re-checked.
|
||||
*
|
||||
* `fetch`'s own redirect following would take the first hop off the allowlist
|
||||
* and the rest wherever it was pointed, which is exactly the hole the allowlist
|
||||
* exists to close.
|
||||
*
|
||||
* `fetchImpl` is the same injection seam `replayFragments({query})` and
|
||||
* `lifecycle.boot({model})` use, and it exists for the same reason: the rules
|
||||
* this file enforces — https only, allowlisted host, allowlisted REDIRECT host,
|
||||
* bounded body, matching hash — are all decisions about a response, and testing
|
||||
* them against a real TLS server would mean testing Node's certificate handling
|
||||
* instead. Production never passes it.
|
||||
*
|
||||
* @returns {Promise<Response>} a response whose body has not been read.
|
||||
*/
|
||||
async function get(rawUrl, hosts, fetchImpl = fetch) {
|
||||
let url = checkUrl(rawUrl, hosts)
|
||||
|
||||
for (let hop = 0; hop <= MAX_REDIRECTS; hop += 1) {
|
||||
let res
|
||||
try {
|
||||
res = await fetchImpl(url, {
|
||||
redirect: 'manual',
|
||||
signal: AbortSignal.timeout(FETCH_TIMEOUT_MS),
|
||||
headers: { accept: '*/*' },
|
||||
})
|
||||
} catch (err) {
|
||||
throw new InstallError(`could not reach ${url.hostname}: ${err.message}`, { status: 502 })
|
||||
}
|
||||
|
||||
if (res.status >= 300 && res.status < 400) {
|
||||
const location = res.headers.get('location')
|
||||
if (!location) throw new InstallError(`${url.hostname} redirected without a location`, { status: 502 })
|
||||
// Resolved against the current URL, then re-checked — a relative redirect
|
||||
// is normal and a cross-host one is the thing being guarded against.
|
||||
url = checkUrl(new URL(location, url).toString(), hosts)
|
||||
continue
|
||||
}
|
||||
|
||||
if (!res.ok) {
|
||||
throw new InstallError(`${url.href} returned ${res.status}`, { status: 502 })
|
||||
}
|
||||
return res
|
||||
}
|
||||
|
||||
throw new InstallError(`too many redirects (more than ${MAX_REDIRECTS})`, { status: 502 })
|
||||
}
|
||||
|
||||
/** Read a bounded response body as text. */
|
||||
async function readText(res, cap, what) {
|
||||
const declared = Number(res.headers.get('content-length'))
|
||||
if (Number.isFinite(declared) && declared > cap) {
|
||||
throw new InstallError(`${what} is larger than ${cap} bytes`)
|
||||
}
|
||||
const buf = Buffer.from(await res.arrayBuffer())
|
||||
// Checked again after reading: content-length is the server's claim, not a
|
||||
// limit it is obliged to honour.
|
||||
if (buf.length > cap) throw new InstallError(`${what} is larger than ${cap} bytes`)
|
||||
return buf.toString('utf8')
|
||||
}
|
||||
|
||||
/**
|
||||
* Stream a response body to a file, hashing as it goes and stopping at the cap.
|
||||
*
|
||||
* The hash is computed from the bytes that were written rather than by re-reading
|
||||
* the file, so there is no window in which the file could differ from what was
|
||||
* verified.
|
||||
*/
|
||||
async function download(res, dest, cap) {
|
||||
const hash = crypto.createHash('sha256')
|
||||
let bytes = 0
|
||||
|
||||
const body = Readable.fromWeb(res.body)
|
||||
const counter = new Transform({
|
||||
transform(chunk, _enc, cb) {
|
||||
bytes += chunk.length
|
||||
if (bytes > cap) {
|
||||
cb(new InstallError(`the artifact is larger than ${Math.round(cap / 1024 / 1024)} MB`))
|
||||
return
|
||||
}
|
||||
hash.update(chunk)
|
||||
cb(null, chunk)
|
||||
},
|
||||
})
|
||||
|
||||
await pipeline(body, counter, fs.createWriteStream(dest))
|
||||
return { sha256: hash.digest('hex'), bytes }
|
||||
}
|
||||
|
||||
// ── The install manifest ───────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Fetch and validate a release's install manifest.
|
||||
*
|
||||
* The shape is the one module-uo's release workflow already writes:
|
||||
* `{schema, id, name, version, coreApi, artifact, url, sha256, size}`. Only the
|
||||
* fields this side needs are required — a module publisher may carry more.
|
||||
*/
|
||||
async function fetchManifest(manifestUrl, hosts, fetchImpl = fetch) {
|
||||
const res = await get(manifestUrl, hosts, fetchImpl)
|
||||
const text = await readText(res, MAX_MANIFEST_BYTES, 'the install manifest')
|
||||
|
||||
let manifest
|
||||
try {
|
||||
manifest = JSON.parse(text)
|
||||
} catch (err) {
|
||||
throw new InstallError(`the install manifest is not valid JSON: ${err.message}`)
|
||||
}
|
||||
if (!manifest || typeof manifest !== 'object' || Array.isArray(manifest)) {
|
||||
throw new InstallError('the install manifest is not a JSON object')
|
||||
}
|
||||
|
||||
const { id, name, version, sha256 } = manifest
|
||||
if (!ID.test(String(id || ''))) {
|
||||
throw new InstallError(`the install manifest has an invalid module id: ${JSON.stringify(id)}`)
|
||||
}
|
||||
if (!name || !version) throw new InstallError('the install manifest is missing name or version')
|
||||
if (!SHA256.test(String(sha256 || '').toLowerCase())) {
|
||||
throw new InstallError('the install manifest has no valid sha256 for its artifact')
|
||||
}
|
||||
|
||||
// `url` is the absolute artifact URL the publisher wrote; `artifact` is its
|
||||
// filename. Prefer the URL, fall back to resolving the filename beside the
|
||||
// manifest — which is where a release's assets sit — so a manifest that
|
||||
// travelled without its absolute URL still installs.
|
||||
const artifactUrl = new URL(manifest.url || manifest.artifact || '', manifestUrl).toString()
|
||||
|
||||
return {
|
||||
id: String(id),
|
||||
name: String(name),
|
||||
version: String(version),
|
||||
sha256: String(sha256).toLowerCase(),
|
||||
size: Number(manifest.size) || null,
|
||||
artifactUrl,
|
||||
coreApi: manifest.coreApi ? String(manifest.coreApi) : null,
|
||||
}
|
||||
}
|
||||
|
||||
// ── Paths on the volume ────────────────────────────────────────────────────
|
||||
|
||||
/** Where a module with this id lives. Rejects an id that is not one. */
|
||||
function moduleDir(id) {
|
||||
if (!ID.test(String(id || ''))) throw new InstallError(`invalid module id: ${JSON.stringify(id)}`)
|
||||
return path.join(loader.dir(), String(id))
|
||||
}
|
||||
|
||||
/** Is there a directory for this module on the volume right now? */
|
||||
function isInstalled(id) {
|
||||
try {
|
||||
return fs.statSync(moduleDir(id)).isDirectory()
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The absolute path of a module's `purge.sql`, or null.
|
||||
*
|
||||
* Read from the module's own `module.json` on disk rather than from the loader,
|
||||
* because purge has to work for a module that never loaded — a `startup_failed`
|
||||
* one is exactly when an operator wants its tables gone.
|
||||
*/
|
||||
function purgeFile(id) {
|
||||
try {
|
||||
const manifest = JSON.parse(fs.readFileSync(path.join(moduleDir(id), 'module.json'), 'utf8'))
|
||||
if (!manifest.purge) return null
|
||||
const file = path.resolve(moduleDir(id), manifest.purge)
|
||||
// The same containment check the loader applies to `client.entry`: a
|
||||
// manifest may not point core at a file outside the module.
|
||||
if (!file.startsWith(moduleDir(id) + path.sep)) return null
|
||||
return fs.existsSync(file) ? file : null
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
// ── Install ────────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Download, verify, unpack and install a module from an install-manifest URL.
|
||||
*
|
||||
* Never leaves a partial module on the volume: everything happens under a
|
||||
* scratch directory that is removed on any failure, and the move into place is
|
||||
* the last step.
|
||||
*
|
||||
* @param {object} args
|
||||
* @param {string} args.url the install manifest URL the admin pasted
|
||||
* @param {string[]} args.hosts the allowlist, already parsed
|
||||
* @returns {Promise<{id,name,version,sha256,source,bytes,replaced}>}
|
||||
*/
|
||||
async function install({ url, hosts, fetchImpl = fetch }) {
|
||||
const manifest = await fetchManifest(url, hosts, fetchImpl)
|
||||
const target = moduleDir(manifest.id)
|
||||
const scratch = await fsp.mkdtemp(path.join(loader.dir(), `.install-${manifest.id}-`))
|
||||
const tarball = path.join(scratch, 'bundle.tar.gz')
|
||||
const unpacked = path.join(scratch, 'unpacked')
|
||||
|
||||
try {
|
||||
const res = await get(manifest.artifactUrl, hosts, fetchImpl)
|
||||
const { sha256, bytes } = await download(res, tarball, MAX_ARTIFACT_BYTES)
|
||||
|
||||
if (sha256 !== manifest.sha256) {
|
||||
// The whole trust model in one comparison. Deliberately does not name the
|
||||
// computed hash in a way that reads like a value to copy into the
|
||||
// manifest — a mismatch means stop, not reconcile.
|
||||
throw new InstallError(
|
||||
`the downloaded artifact does not match the sha256 in the install manifest — refusing to install`,
|
||||
)
|
||||
}
|
||||
if (manifest.size && bytes !== manifest.size) {
|
||||
throw new InstallError(`the artifact is ${bytes} bytes but the manifest declares ${manifest.size}`)
|
||||
}
|
||||
|
||||
await archive.unpack(tarball, unpacked)
|
||||
|
||||
// What the bundle says it is, checked against what the manifest promised.
|
||||
let inner
|
||||
try {
|
||||
inner = JSON.parse(await fsp.readFile(path.join(unpacked, 'module.json'), 'utf8'))
|
||||
} catch {
|
||||
throw new InstallError('the bundle has no readable module.json at its root')
|
||||
}
|
||||
if (inner.id !== manifest.id) {
|
||||
throw new InstallError(
|
||||
`the bundle declares module id "${inner.id}" but the install manifest promised "${manifest.id}"`,
|
||||
)
|
||||
}
|
||||
if (inner.version !== manifest.version) {
|
||||
throw new InstallError(
|
||||
`the bundle declares version "${inner.version}" but the install manifest promised "${manifest.version}"`,
|
||||
)
|
||||
}
|
||||
|
||||
// The swap. The outgoing directory is moved aside rather than deleted first,
|
||||
// so a failed rename leaves the previous version recoverable instead of
|
||||
// leaving no module at all — the same reasoning the installer repo applies
|
||||
// to a ServUO tree.
|
||||
const replaced = fs.existsSync(target)
|
||||
const aside = `${target}.replaced-${Date.now()}`
|
||||
if (replaced) await fsp.rename(target, aside)
|
||||
try {
|
||||
await fsp.rename(unpacked, target)
|
||||
} catch (err) {
|
||||
if (replaced) await fsp.rename(aside, target).catch(() => {})
|
||||
throw err
|
||||
}
|
||||
if (replaced) await fsp.rm(aside, { recursive: true, force: true })
|
||||
|
||||
log.info(`installed module "${manifest.id}" v${manifest.version}`, {
|
||||
source: url,
|
||||
sha256: manifest.sha256,
|
||||
bytes,
|
||||
replaced,
|
||||
})
|
||||
|
||||
return { ...manifest, source: url, bytes, replaced }
|
||||
} finally {
|
||||
await fsp.rm(scratch, { recursive: true, force: true }).catch(() => {})
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove a module's directory from the volume.
|
||||
*
|
||||
* Returns whether there was one. Does not touch the database, does not run
|
||||
* purge.sql, and does not stop the running module — the controller sequences
|
||||
* all three, because the order matters and only it knows what the operator
|
||||
* asked for.
|
||||
*/
|
||||
async function removeDir(id) {
|
||||
const dir = moduleDir(id)
|
||||
if (!fs.existsSync(dir)) return false
|
||||
await fsp.rm(dir, { recursive: true, force: true })
|
||||
log.info(`removed module directory for "${id}"`)
|
||||
return true
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
InstallError,
|
||||
parseHosts,
|
||||
checkUrl,
|
||||
get,
|
||||
fetchManifest,
|
||||
install,
|
||||
removeDir,
|
||||
moduleDir,
|
||||
isInstalled,
|
||||
purgeFile,
|
||||
MAX_MANIFEST_BYTES,
|
||||
MAX_ARTIFACT_BYTES,
|
||||
}
|
||||
@@ -101,10 +101,16 @@ async function boot({ modules, model } = {}) {
|
||||
id: m.id,
|
||||
name: m.name,
|
||||
version: m.version,
|
||||
// Null provenance is what a hand-placed directory looks like. An install
|
||||
// performed through the admin panel (§2.5, a later phase) writes the row
|
||||
// with its source and hash first; this refresh deliberately does not
|
||||
// overwrite either, because recordInstalled leaves what it is not given.
|
||||
// Null provenance is what a hand-placed directory looks like, and it is
|
||||
// all this step can honestly say: a scan finds a directory, never where it
|
||||
// came from. An install through the admin panel writes source and sha256
|
||||
// first, and this refresh must not undo that.
|
||||
//
|
||||
// It used to. `upsert` assigned both columns unconditionally, so every
|
||||
// boot nulled them and an install's provenance survived only until the
|
||||
// restart it asked for. The statement now COALESCEs — see the note on
|
||||
// modules.db.js's upsert. Nothing could have caught it before Phase 4:
|
||||
// this was the only caller, and it has never had a value to pass.
|
||||
}))
|
||||
}
|
||||
|
||||
@@ -205,4 +211,67 @@ async function shutdown({ modules, budgetMs = SHUTDOWN_BUDGET_MS } = {}) {
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { boot, shutdown, SHUTDOWN_BUDGET_MS }
|
||||
/**
|
||||
* Stop ONE module and mark it disabled — the admin panel's Disable (§2.7.2
|
||||
* decision 3).
|
||||
*
|
||||
* Phase 2 built disable as a pure state flip: the record moved to `disabled` and
|
||||
* the dispatch guard started answering 404. That makes a module invisible, not
|
||||
* stopped. Everything a module does that is not a response to a request — the
|
||||
* sockets and timers its `onBoot` armed — carried on running, so an operator
|
||||
* disabling a module *because* it was misbehaving got nothing until the next
|
||||
* restart, which is the one thing the button was supposed to save them.
|
||||
*
|
||||
* So the hook runs first, and the state moves after it: while `onShutdown` is
|
||||
* running the module is still `started`, which is the only state in which its
|
||||
* own routes and the things it is tearing down are consistent with each other.
|
||||
* The hook gets the same budget the exit path gives it.
|
||||
*
|
||||
* A hook that throws does NOT stop the disable. The operator asked for this
|
||||
* module to stop answering; a module that could not close cleanly is a reason to
|
||||
* log loudly, not a reason to leave it serving. That is the opposite of the boot
|
||||
* path's rule, and deliberately: there, a failure means the module never became
|
||||
* safe to use.
|
||||
*
|
||||
* **Enable is not the mirror of this, and there is no `start(id)` beside it.**
|
||||
* There is no `onBoot` re-dispatch, and MODULE_API.md has never promised the
|
||||
* hooks are re-entrant — no module author has written `onBoot` to be safe to run
|
||||
* twice in one process. Re-enabling therefore moves the row and waits for a
|
||||
* restart, which the admin screen offers.
|
||||
*
|
||||
* @returns {Promise<{stopped: boolean, error: string|null}>} whether a hook ran.
|
||||
*/
|
||||
async function stop(id, { modules, model, budgetMs = SHUTDOWN_BUDGET_MS } = {}) {
|
||||
/* eslint-disable global-require */
|
||||
const loader = modules || require('./loader')
|
||||
const rows = model || require('../model/modules/modules.model')
|
||||
/* eslint-enable global-require */
|
||||
|
||||
let error = null
|
||||
let stopped = false
|
||||
|
||||
if (loader.isLoaded()) {
|
||||
const target = loader.stopHook(id)
|
||||
if (target) {
|
||||
try {
|
||||
await withBudget(target.hook, budgetMs)
|
||||
stopped = true
|
||||
log.info(`module "${id}" stopped by an operator`)
|
||||
} catch (err) {
|
||||
error = err.message
|
||||
log.warn(`module "${id}" onShutdown failed or timed out while being disabled — disabling anyway`, {
|
||||
error: err.message,
|
||||
})
|
||||
}
|
||||
}
|
||||
// Unconditional, and after the hook: this is what makes its routes and its
|
||||
// client chunk answer 404 (§4.5). A module that is not loaded in this
|
||||
// process has no record to move, and setState ignores an unknown id.
|
||||
loader.setState(id, 'disabled')
|
||||
}
|
||||
|
||||
await safe(`disabling module "${id}"`, () => rows.disable(id))
|
||||
return { stopped, error }
|
||||
}
|
||||
|
||||
module.exports = { boot, shutdown, stop, SHUTDOWN_BUDGET_MS }
|
||||
|
||||
@@ -74,6 +74,13 @@ const MANIFEST_KEYS = new Set([
|
||||
const modules = new Map()
|
||||
let loaded = false
|
||||
|
||||
// Bumped by every state CHANGE. One consumer today: the merged OpenAPI document
|
||||
// at /api/docs.json, which is built from the fragments of `started` modules and
|
||||
// so has to be rebuilt when that set moves (§6.1a). A counter rather than an
|
||||
// event, because the question a cache asks is "is what I have still current",
|
||||
// and a number answers it without anyone having to remember to subscribe.
|
||||
let stateVersion = 0
|
||||
|
||||
// ── ctx ────────────────────────────────────────────────────────────────────
|
||||
|
||||
// Everything a module may reach in core, and nothing else (§2.3). Required
|
||||
@@ -709,6 +716,7 @@ function setState(id, state, { stage = null, reason = null } = {}) {
|
||||
if (!RECORD_STATES.has(state)) throw new Error(`unknown module state "${state}"`)
|
||||
const record = modules.get(id)
|
||||
if (!record) return
|
||||
if (record.state !== state) stateVersion += 1
|
||||
record.state = state
|
||||
record.stage = state === 'startup_failed' ? stage : null
|
||||
record.reason = state === 'startup_failed' ? reason : null
|
||||
@@ -788,6 +796,31 @@ function shutdownHooks() {
|
||||
.reverse()
|
||||
}
|
||||
|
||||
/**
|
||||
* One module's `onShutdown`, for stopping it on its own rather than at exit.
|
||||
*
|
||||
* Phase 4 (§2.7.2 decision 3) gave the admin panel's Disable a real meaning.
|
||||
* Until then, disabling flipped this record's state and the dispatch guard began
|
||||
* answering 404 — which made the module invisible without making it stop. A
|
||||
* module's `onBoot` is where it opens its sockets and arms its timers, and none
|
||||
* of that is reachable through a URL, so an operator disabling a misbehaving
|
||||
* module got no relief from it at all until the next restart.
|
||||
*
|
||||
* `started` only, the same rule shutdownHooks() applies and for the same reason:
|
||||
* a module whose `onBoot` threw has a half-built world its `onShutdown` was
|
||||
* never written to tear down. Returns null when there is nothing to run — which
|
||||
* covers "not started", "no hook", and "no such module", none of which is an
|
||||
* error the caller can act on differently.
|
||||
*
|
||||
* @returns {{id: string, hook: Function}|null}
|
||||
*/
|
||||
function stopHook(id) {
|
||||
assertLoaded('stopHook')
|
||||
const record = modules.get(id)
|
||||
if (!record || record.state !== 'started' || !record.hooks.onShutdown) return null
|
||||
return { id: record.id, hook: record.hooks.onShutdown }
|
||||
}
|
||||
|
||||
/**
|
||||
* Every schema fragment waiting to be replayed, in scan order.
|
||||
*
|
||||
@@ -845,6 +878,39 @@ function clientEntryUrls() {
|
||||
.map((r) => r.client.entryUrl)
|
||||
}
|
||||
|
||||
/**
|
||||
* Every started module's OpenAPI fragment, in scan order.
|
||||
*
|
||||
* `started` only, matching clientEntryUrls() rather than clientChunks(): the
|
||||
* merged document is built when it is asked for, at which point the state is
|
||||
* known, and documenting a module that is 503ing every one of those paths would
|
||||
* send a client somewhere it cannot go.
|
||||
*
|
||||
* The filename is fixed by §2.8 — `swagger-fragment.json` in the bundle root —
|
||||
* rather than declared in `module.json`, so a module cannot point core at
|
||||
* something else. A module that ships none is simply absent: registering routes
|
||||
* without documenting them is checked in the module's OWN CI (§2.8), where the
|
||||
* routes are known; core has no way to tell the difference here between a module
|
||||
* with no routes and one that forgot.
|
||||
*
|
||||
* @returns {{id: string, file: string}[]}
|
||||
*/
|
||||
function specFragments() {
|
||||
assertLoaded('specFragments')
|
||||
return [...modules.values()]
|
||||
.filter((r) => r.state === 'started')
|
||||
.map((r) => ({ id: r.id, file: path.join(r.dir, 'swagger-fragment.json') }))
|
||||
.filter((f) => fs.existsSync(f.file))
|
||||
}
|
||||
|
||||
/**
|
||||
* How many times a module's state has CHANGED in this process.
|
||||
*
|
||||
* A cache key, and nothing more: hold the value you built with, compare, rebuild
|
||||
* when it differs. It says nothing about which module moved or where to.
|
||||
*/
|
||||
const version = () => stateVersion
|
||||
|
||||
/** Absolute path of the modules directory. */
|
||||
const dir = () => MODULES_DIR
|
||||
|
||||
@@ -855,8 +921,11 @@ module.exports = {
|
||||
fragments,
|
||||
bootable,
|
||||
shutdownHooks,
|
||||
stopHook,
|
||||
clientChunks,
|
||||
clientEntryUrls,
|
||||
specFragments,
|
||||
version,
|
||||
isLoaded,
|
||||
dir,
|
||||
}
|
||||
|
||||
@@ -81,4 +81,48 @@ async function replayFragments({ query, modules } = {}) {
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { replayFragments }
|
||||
/**
|
||||
* Run one module's `purge.sql` — the destructive twin of the replay above
|
||||
* (MODULE_SYSTEM.md §2.5, and §2.7.2 decision 5 for when it is offered).
|
||||
*
|
||||
* It lives beside replayFragments because they are the same operation pointed in
|
||||
* opposite directions: a file of statements the module ships, split by the same
|
||||
* splitter, run serially on the same pool. Keeping them together is what makes
|
||||
* it obvious that the fragment's leading-verb allowlist does NOT apply here —
|
||||
* `purge.sql` is the one file a module may put a DROP in, precisely because it
|
||||
* is the one file that never runs on a boot.
|
||||
*
|
||||
* Unlike the replay, this **throws**. A replay failure is one module failing to
|
||||
* start, which the site survives by 503ing that module; a purge failure is an
|
||||
* operator's explicit destructive request not having happened, and reporting
|
||||
* success for that would leave them believing data is gone when it is not.
|
||||
*
|
||||
* Statements run serially and are not wrapped in a transaction, for the reason
|
||||
* the replay's header already gives: MariaDB commits DDL implicitly, so there is
|
||||
* no rollback to have. A purge that fails halfway has dropped some tables — the
|
||||
* error names the statement that stopped it, and running it again is safe
|
||||
* because a purge script is required to be idempotent in the same way a fragment
|
||||
* is (`DROP TABLE IF EXISTS`).
|
||||
*
|
||||
* @param {string} file absolute path to the module's purge.sql
|
||||
* @param {object} [deps] injection seam for tests
|
||||
* @returns {Promise<number>} how many statements ran
|
||||
*/
|
||||
async function runPurge(file, { query } = {}) {
|
||||
// eslint-disable-next-line global-require
|
||||
const run = query || require('../utils/db').query
|
||||
|
||||
const statements = splitStatements(fs.readFileSync(file, 'utf8'))
|
||||
let ran = 0
|
||||
for (const statement of statements) {
|
||||
try {
|
||||
await run(statement)
|
||||
ran += 1
|
||||
} catch (err) {
|
||||
throw new Error(`purge failed at statement ${ran + 1} of ${statements.length}: ${err.message}`)
|
||||
}
|
||||
}
|
||||
return ran
|
||||
}
|
||||
|
||||
module.exports = { replayFragments, runPurge }
|
||||
|
||||
@@ -30,6 +30,7 @@ const pagesRouter = require('./pages.router')
|
||||
const emailRouter = require('./email.router')
|
||||
const discordBotRouter = require('./discordBot.router')
|
||||
const settingsRouter = require('./settings.router')
|
||||
const modulesRouter = require('./modules.router')
|
||||
const dashboardRouter = require('./dashboard.router')
|
||||
|
||||
const adminRouter = express.Router()
|
||||
@@ -73,6 +74,11 @@ adminRouter.use('/pages', pagesRouter)
|
||||
adminRouter.use('/email', emailRouter)
|
||||
adminRouter.use('/discord-bot', discordBotRouter)
|
||||
adminRouter.use('/settings', settingsRouter)
|
||||
// The Modules screen (Phase 4). Core's, not a module's — and it has to be
|
||||
// core's: it is how a module gets onto the volume in the first place. Mounted
|
||||
// here alongside the other configuration capabilities, and admin-only per route
|
||||
// rather than at this line, so the gate sits next to what it is guarding.
|
||||
adminRouter.use('/modules', modulesRouter)
|
||||
|
||||
// The two singletons that own no path segment of their own: GET /dashboard and
|
||||
// PUT /site-mode. Mounted at the group root, last, exactly where the residual
|
||||
|
||||
@@ -19,7 +19,7 @@ invitesRouter.post(
|
||||
// #swagger.tags = ['Admin · Invites']
|
||||
// #swagger.summary = 'Create and email an account invite at a chosen access level'
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["email","role"], properties: { email: { type: "string" }, role: { type: "string" } } } } } */
|
||||
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["email","role"], properties: { email: { type: "string" }, role: { type: "string" } } } } } } */
|
||||
/* #swagger.responses[201] = { description: 'Invite created', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
|
||||
/* #swagger.responses[400] = { description: 'Validation error', content: { "application/json": { schema: { $ref: "#/components/schemas/ValidationError" } } } } */
|
||||
adminOnly,
|
||||
|
||||
397
server/src/router/v1/admin/modules.controller.js
Normal file
397
server/src/router/v1/admin/modules.controller.js
Normal file
@@ -0,0 +1,397 @@
|
||||
// ── Admin: installed modules ───────────────────────────────────────────────
|
||||
//
|
||||
// Phase 4, slice 1 of docs/website/MODULE_SYSTEM.md §2.7.2. Admin-only, and more
|
||||
// so than anything else in this directory: installing a module puts JavaScript on
|
||||
// the volume that core will `require` into its own process on the next boot. That
|
||||
// is the feature — it is what "an operator never builds anything" means (§1.14) —
|
||||
// but it is worth being plain that this controller is remote code execution with
|
||||
// an audit trail, not a settings screen.
|
||||
//
|
||||
// What guards it, in the order an attacker would meet them:
|
||||
//
|
||||
// 1. `requireRole('admin')` on every route, on top of the group's staff gate.
|
||||
// 2. An `https`-only host allowlist, re-checked on every redirect hop, so a
|
||||
// pasted URL cannot be pointed at the compose network or a metadata service
|
||||
// (install.js).
|
||||
// 3. The sha256 the release published, compared against the bytes that arrived.
|
||||
// 4. A full inspection of the archive before a byte of it is unpacked, and an
|
||||
// unpack into a scratch directory that is only moved into place once the
|
||||
// bundle has agreed with the manifest about what it is (archive.js).
|
||||
// 5. Every action here writes to core's one audit log.
|
||||
//
|
||||
// The one thing this file cannot do is mount anything. §1.12 makes the volume the
|
||||
// mounting source of truth, read once at require time, so install and uninstall
|
||||
// take effect on the next boot — which is why `restart` is a route here rather
|
||||
// than a sentence in a tooltip (decision 1).
|
||||
|
||||
const modules = require('../../../model/modules/modules.model')
|
||||
const activity = require('../../../model/activity/activity.model')
|
||||
const settings = require('../../../model/settings/settings.model')
|
||||
const loader = require('../../../modules/loader')
|
||||
const lifecycle = require('../../../modules/lifecycle')
|
||||
const install = require('../../../modules/install')
|
||||
// A namespace import, like every other require in this file, and not
|
||||
// `const { runPurge } = …`: destructuring at require time captures the function
|
||||
// rather than the module, which makes it the one dependency here that cannot be
|
||||
// substituted. That matters because the two tests worth having about purge are
|
||||
// about the ORDER it runs in relative to the directory being removed.
|
||||
const schema = require('../../../modules/schema')
|
||||
|
||||
const log = require('../../../utils/logger')('admin-modules')
|
||||
|
||||
// The allowlist setting. Seeded from MODULE_SOURCE_HOSTS on first boot and
|
||||
// admin-managed from then on (decision 6) — db/seed.js writes it once and never
|
||||
// overwrites it, so changing the variable later does not silently reach in and
|
||||
// undo an operator's choice.
|
||||
const HOSTS_KEY = 'module_source_hosts'
|
||||
|
||||
// A hostname, not a URL: no scheme, no path, no port, no wildcard. Deliberately
|
||||
// strict — every character allowed here is a character that can appear in the
|
||||
// host of a URL this server will fetch and execute the contents of.
|
||||
const HOSTNAME = /^[a-z0-9]([a-z0-9-]*[a-z0-9])?(\.[a-z0-9]([a-z0-9-]*[a-z0-9])?)*$/
|
||||
|
||||
async function allowedHosts() {
|
||||
return install.parseHosts(await settings.get(HOSTS_KEY))
|
||||
}
|
||||
|
||||
/**
|
||||
* One module, as the admin screen needs it.
|
||||
*
|
||||
* Three sources have to be reconciled, and which one answers which question is
|
||||
* the whole of §2.4:
|
||||
*
|
||||
* - the ROW says what the operator decided and what the last boot recorded;
|
||||
* - the LOADER says what is mounted and answering right now;
|
||||
* - the VOLUME says whether there is still a directory there at all.
|
||||
*
|
||||
* They can legitimately disagree, and the screen has to show that rather than
|
||||
* pick a winner. A row `enabled` with a loader state of `disabled` is a module
|
||||
* the operator has just switched back on and which is waiting for a restart —
|
||||
* exactly the case decision 3 creates, and it would be a lie to render it as
|
||||
* either "running" or "off".
|
||||
*/
|
||||
function present(row, live, onVolume) {
|
||||
return {
|
||||
id: row ? row.id : live.id,
|
||||
name: row ? row.name : live.name,
|
||||
version: row ? row.version : live.version,
|
||||
// What the database records.
|
||||
state: row ? row.state : null,
|
||||
failureStage: row ? row.failureStage : (live && live.stage) || null,
|
||||
failureReason: row ? row.failureReason : (live && live.reason) || null,
|
||||
source: row ? row.source : null,
|
||||
sha256: row ? row.sha256 : null,
|
||||
installedAt: row ? row.installedAt : null,
|
||||
startedAt: row ? row.startedAt : null,
|
||||
// What is actually mounted in this process, and what it is answering.
|
||||
liveState: live ? live.state : null,
|
||||
// The version RUNNING, which is not always the version installed: an upgrade
|
||||
// writes new files and a new row while the old code stays loaded until the
|
||||
// restart. Without this the screen would report the new version as
|
||||
// "Running", which is the same lie in a different place.
|
||||
liveVersion: live ? live.version : null,
|
||||
capabilities: live ? live.capabilities : [],
|
||||
// What is on the volume.
|
||||
onVolume,
|
||||
canPurge: onVolume && Boolean(install.purgeFile(row ? row.id : live.id)),
|
||||
}
|
||||
}
|
||||
|
||||
// GET /admin/modules — every module core knows about, from all three sources,
|
||||
// plus the source allowlist the install form needs.
|
||||
async function list(req, res) {
|
||||
try {
|
||||
const rows = await modules.list()
|
||||
// The loader throws rather than returning [] before load() has run (§7.6),
|
||||
// and this controller is reachable from a process where that is true —
|
||||
// `npm run seed` never gets here, but a test harness might.
|
||||
const live = loader.isLoaded() ? loader.list() : []
|
||||
const byId = new Map(live.map((m) => [m.id, m]))
|
||||
|
||||
const seen = new Set()
|
||||
const out = []
|
||||
for (const row of rows) {
|
||||
seen.add(row.id)
|
||||
out.push(present(row, byId.get(row.id) || null, install.isInstalled(row.id)))
|
||||
}
|
||||
// A directory on the volume that has no row yet — a hand-placed install
|
||||
// before its first boot. It has to be listed, or the screen would show
|
||||
// nothing for a module whose routes are already being served.
|
||||
for (const m of live) {
|
||||
if (!seen.has(m.id)) out.push(present(null, m, true))
|
||||
}
|
||||
|
||||
return res.json({ modules: out, sourceHosts: await allowedHosts() })
|
||||
} catch (err) {
|
||||
log.error('list modules', err)
|
||||
return res.status(500).json({ message: 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
// POST /admin/modules — install (or upgrade) from an install-manifest URL.
|
||||
async function create(req, res) {
|
||||
const url = String(req.body.url || '').trim()
|
||||
try {
|
||||
const hosts = await allowedHosts()
|
||||
const result = await install.install({ url, hosts })
|
||||
|
||||
// Provenance is written here and nowhere else: the boot reconcile records a
|
||||
// module with NULL source/sha256 and leaves what it is not given, precisely
|
||||
// so that a refresh cannot overwrite what an install knew (lifecycle.js).
|
||||
const row = await modules.recordInstalled({
|
||||
id: result.id,
|
||||
name: result.name,
|
||||
version: result.version,
|
||||
source: result.source,
|
||||
sha256: result.sha256,
|
||||
})
|
||||
|
||||
await activity.log({
|
||||
req,
|
||||
userId: req.user.id,
|
||||
action: 'module.install',
|
||||
detail: { id: result.id, version: result.version, source: url, sha256: result.sha256, replaced: result.replaced },
|
||||
})
|
||||
log.warn('module installed — it will mount on the next restart', {
|
||||
id: result.id,
|
||||
version: result.version,
|
||||
by: req.user.username,
|
||||
})
|
||||
|
||||
return res.status(201).json({ module: row, restartRequired: true, replaced: result.replaced })
|
||||
} catch (err) {
|
||||
if (err.name === 'InstallError' || err.name === 'ArchiveError') {
|
||||
// The operator pasted a URL and something about what came back was wrong.
|
||||
// The message is the useful part and is written to be read by them.
|
||||
log.warn('module install refused', { url, reason: err.message })
|
||||
return res.status(err.status || 400).json({ message: err.message })
|
||||
}
|
||||
log.error('install module', err)
|
||||
return res.status(500).json({ message: 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
// POST /admin/modules/:id/enable — switch a module back on, for the next boot.
|
||||
//
|
||||
// Deliberately does NOT touch the loader's record. Disable ran the module's
|
||||
// onShutdown (decision 3), and there is no onBoot re-dispatch to undo that: a
|
||||
// module whose sockets were closed and timers cleared cannot be made to serve
|
||||
// again by flipping a flag, and pretending otherwise would put it back on the
|
||||
// nav with a torn-down world behind it. The row moves; the restart starts it.
|
||||
async function enable(req, res) {
|
||||
const { id } = req.params
|
||||
try {
|
||||
const row = await modules.enable(id)
|
||||
if (!row) return res.status(404).json({ message: 'No such module.' })
|
||||
|
||||
await activity.log({ req, userId: req.user.id, action: 'module.enable', detail: { id } })
|
||||
return res.json({ module: row, restartRequired: true })
|
||||
} catch (err) {
|
||||
if (err.name === 'ModuleStateError') return res.status(409).json({ message: err.message })
|
||||
log.error('enable module', err)
|
||||
return res.status(500).json({ message: 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
// POST /admin/modules/:id/disable — stop it now.
|
||||
//
|
||||
// The one action on this screen that takes effect without a restart, and the
|
||||
// reason it does is that it is the one an operator reaches for when something is
|
||||
// going wrong. Its routes answer 404 from the moment this returns, and its
|
||||
// onShutdown has already run.
|
||||
async function disable(req, res) {
|
||||
const { id } = req.params
|
||||
try {
|
||||
const current = await modules.get(id)
|
||||
if (!current) return res.status(404).json({ message: 'No such module.' })
|
||||
|
||||
const { stopped, error } = await lifecycle.stop(id)
|
||||
const row = await modules.get(id)
|
||||
|
||||
await activity.log({
|
||||
req,
|
||||
userId: req.user.id,
|
||||
action: 'module.disable',
|
||||
detail: { id, hookRan: stopped, hookError: error },
|
||||
})
|
||||
log.warn('module disabled by an operator', { id, hookRan: stopped, by: req.user.username })
|
||||
|
||||
// `shutdownError` is reported rather than swallowed: the module IS disabled
|
||||
// either way, and an operator whose module could not close cleanly should be
|
||||
// told so while they still have the logs to look at.
|
||||
return res.json({ module: row, stopped, shutdownError: error })
|
||||
} catch (err) {
|
||||
log.error('disable module', err)
|
||||
return res.status(500).json({ message: 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
// DELETE /admin/modules/:id[?purge=true] — uninstall.
|
||||
//
|
||||
// Non-destructive by default (§2.5): the directory goes, the row stays
|
||||
// `disabled`, and the module's tables and data are left alone.
|
||||
//
|
||||
// The purge option is here rather than as a follow-up action because it cannot
|
||||
// be a follow-up action (decision 5): `purge.sql` is a file inside the directory
|
||||
// this is about to delete, so after an uninstall there is nothing left to purge
|
||||
// with. Ticking the box is the last moment the file exists.
|
||||
//
|
||||
// The order below is the whole of it, and each step depends on the one above:
|
||||
// purge while the SQL is still readable, stop while the code is still loaded,
|
||||
// then delete.
|
||||
async function remove(req, res) {
|
||||
const { id } = req.params
|
||||
const purge = req.query.purge === 'true' || req.query.purge === '1'
|
||||
try {
|
||||
const current = await modules.get(id)
|
||||
const onVolume = install.isInstalled(id)
|
||||
if (!current && !onVolume) return res.status(404).json({ message: 'No such module.' })
|
||||
|
||||
let purged = null
|
||||
if (purge) {
|
||||
const file = install.purgeFile(id)
|
||||
if (!file) {
|
||||
return res.status(400).json({
|
||||
message: 'This module ships no purge.sql, so its data cannot be deleted. Uninstall without purging instead.',
|
||||
})
|
||||
}
|
||||
purged = await schema.runPurge(file)
|
||||
}
|
||||
|
||||
// Stop it before its files vanish. A module whose directory is deleted out
|
||||
// from under a running onShutdown is being asked to tear down a world whose
|
||||
// code may already be half-unreadable — and its sockets would otherwise stay
|
||||
// open until the restart, holding a connection on behalf of a module that no
|
||||
// longer exists on disk.
|
||||
await lifecycle.stop(id)
|
||||
const removed = await install.removeDir(id)
|
||||
|
||||
// A purge leaves nothing: no directory, no tables, no data. Keeping a
|
||||
// `disabled` row for that is a tombstone with nothing to offer and a Purge
|
||||
// button that would fail. A plain uninstall keeps its row, which is what
|
||||
// makes the retained data visible and reinstallable.
|
||||
if (purge) await modules.remove(id)
|
||||
|
||||
await activity.log({
|
||||
req,
|
||||
userId: req.user.id,
|
||||
action: purge ? 'module.purge' : 'module.uninstall',
|
||||
detail: { id, purged, removed },
|
||||
})
|
||||
log.warn(`module ${purge ? 'uninstalled and purged' : 'uninstalled'}`, {
|
||||
id,
|
||||
statements: purged,
|
||||
by: req.user.username,
|
||||
})
|
||||
|
||||
return res.json({ id, removed, purged, restartRequired: true })
|
||||
} catch (err) {
|
||||
log.error('uninstall module', err)
|
||||
return res.status(500).json({ message: err.message || 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
// POST /admin/modules/:id/purge — drop a still-installed module's data.
|
||||
//
|
||||
// Refuses unless the module is already disabled, and that guard is the point:
|
||||
// dropping the tables under a module that is still serving requests leaves it
|
||||
// answering out of a world that no longer exists. Disabling first is one click
|
||||
// and makes the destructive step happen against something that has stopped.
|
||||
async function purge(req, res) {
|
||||
const { id } = req.params
|
||||
try {
|
||||
const current = await modules.get(id)
|
||||
if (!current) return res.status(404).json({ message: 'No such module.' })
|
||||
if (current.state !== 'disabled') {
|
||||
return res.status(409).json({
|
||||
message: 'Disable this module before purging its data, so nothing is serving out of the tables being dropped.',
|
||||
})
|
||||
}
|
||||
|
||||
const file = install.purgeFile(id)
|
||||
if (!file) {
|
||||
return res.status(400).json({ message: 'This module ships no purge.sql, so its data cannot be deleted.' })
|
||||
}
|
||||
|
||||
const statements = await schema.runPurge(file)
|
||||
await activity.log({ req, userId: req.user.id, action: 'module.purge', detail: { id, statements } })
|
||||
log.warn('module data purged', { id, statements, by: req.user.username })
|
||||
|
||||
return res.json({ id, purged: statements })
|
||||
} catch (err) {
|
||||
log.error('purge module', err)
|
||||
return res.status(500).json({ message: err.message || 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
// PUT /admin/modules/sources — the host allowlist.
|
||||
async function setSources(req, res) {
|
||||
const hosts = install.parseHosts(req.body.hosts)
|
||||
const bad = hosts.find((h) => !HOSTNAME.test(h))
|
||||
if (bad) return res.status(400).json({ message: `"${bad}" is not a valid hostname.` })
|
||||
|
||||
try {
|
||||
const before = await allowedHosts()
|
||||
await settings.set(HOSTS_KEY, hosts.join(','), req.user.id)
|
||||
await activity.log({
|
||||
req,
|
||||
userId: req.user.id,
|
||||
action: 'module.sources',
|
||||
detail: { before, after: hosts },
|
||||
})
|
||||
log.warn('module source allowlist changed', { before, after: hosts, by: req.user.username })
|
||||
return res.json({ sourceHosts: hosts })
|
||||
} catch (err) {
|
||||
log.error('set module sources', err)
|
||||
return res.status(500).json({ message: 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
// POST /admin/modules/restart — restart the server process.
|
||||
//
|
||||
// Decision 1. Install, uninstall and re-enable all only take effect at boot
|
||||
// because §1.12 reads the volume at require time, and §2.4 promises recovery
|
||||
// "with no shell access to the box" — which a banner saying "please restart your
|
||||
// container" does not deliver.
|
||||
//
|
||||
// It reaches server.js's existing SIGTERM handler rather than doing the work
|
||||
// itself: that handler stops the modules, the workers and the listeners in the
|
||||
// right order and closes the pool and the log file before exiting 0, and going
|
||||
// through it means there is exactly one graceful-shutdown path that this route
|
||||
// cannot drift from.
|
||||
//
|
||||
// It gets there by EMITTING the event, not by signalling the process, and that
|
||||
// is not a detail. `process.kill(process.pid, 'SIGTERM')` is what this did
|
||||
// first, and it works on Linux — but **Windows has no POSIX signals, and Node
|
||||
// documents SIGTERM there as unconditional termination of the target process**.
|
||||
// So on a Windows host the restart killed the server outright: no module
|
||||
// `onShutdown`, no pool close, no log flush. Verified by running it — the
|
||||
// process was gone and the shutdown handler had logged nothing.
|
||||
//
|
||||
// `process.on('SIGTERM', …)` is an ordinary EventEmitter listener, so
|
||||
// `process.emit('SIGTERM')` invokes exactly the same handler on every platform
|
||||
// without involving the OS at all. Deployment is Linux containers and would
|
||||
// never have shown this; development is not.
|
||||
//
|
||||
// What brings the process BACK is the supervisor, not this. The shipped
|
||||
// docker-compose.yml declares `restart: unless-stopped` on `app`, which restarts
|
||||
// on a clean exit as well as a crash. A bare `npm start` does not come back, and
|
||||
// the screen says so before it asks.
|
||||
function restart(req, res) {
|
||||
log.warn('restart requested from the admin panel', { by: req.user.username })
|
||||
|
||||
// Logged and answered first. Once the signal is raised the response has no
|
||||
// listener left to flush through, so the operator would be told nothing.
|
||||
res.status(202).json({ restarting: true })
|
||||
|
||||
activity
|
||||
.log({ req, userId: req.user.id, action: 'module.restart', detail: {} })
|
||||
.catch((err) => log.error('failed to record the restart in the audit log', err))
|
||||
.finally(() => {
|
||||
// A beat, so the 202 is on the wire. `unref` so this timer is not itself
|
||||
// something keeping the process alive.
|
||||
setTimeout(() => process.emit('SIGTERM'), 250).unref()
|
||||
})
|
||||
}
|
||||
|
||||
module.exports = { list, create, enable, disable, remove, purge, setSources, restart, HOSTS_KEY }
|
||||
143
server/src/router/v1/admin/modules.router.js
Normal file
143
server/src/router/v1/admin/modules.router.js
Normal file
@@ -0,0 +1,143 @@
|
||||
// Admin · Modules — install, enable, disable, uninstall, purge and restart.
|
||||
//
|
||||
// Mounted at /api/v1/admin/modules by admin/index.js, which has already applied
|
||||
// `noindex, isLoggedIn, staffOnly`. Every route here re-gates to `admin`: an
|
||||
// editor or moderator has no business installing code into the server process,
|
||||
// and the group gate alone would let them.
|
||||
//
|
||||
// Route order matters in one place. `/restart` and `/sources` are declared
|
||||
// BEFORE the `/:id/...` routes, because express matches in declaration order and
|
||||
// a module whose id was `restart` would otherwise shadow — or be shadowed by —
|
||||
// the literal path. The id pattern below makes that unreachable in practice; the
|
||||
// ordering makes it unreachable by construction.
|
||||
|
||||
const express = require('express')
|
||||
const { body, param, query } = require('express-validator')
|
||||
|
||||
const controller = require('./modules.controller')
|
||||
const { requireRole } = require('../../../utils/auth')
|
||||
const validate = require('../../../middleware/validate')
|
||||
|
||||
const modulesRouter = express.Router()
|
||||
const adminOnly = requireRole('admin')
|
||||
|
||||
// The loader's own id rule (MODULE_API.md §2.1). Applied at the edge so a
|
||||
// traversal-shaped id never reaches a path join, even though install.js checks
|
||||
// it again — this one produces a 400 with a readable message, that one is the
|
||||
// guarantee.
|
||||
const ID = /^[a-z][a-z0-9-]{1,31}$/
|
||||
|
||||
modulesRouter.get(
|
||||
'/',
|
||||
// #swagger.tags = ['Admin · Modules']
|
||||
// #swagger.summary = 'List installed modules, their live state, and the source allowlist'
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
/* #swagger.responses[200] = { description: 'Modules and the install source allowlist', content: { "application/json": { schema: { type: "object", properties: { modules: { type: "array", items: { type: "object", additionalProperties: true } }, sourceHosts: { type: "array", items: { type: "string" } } } } } } } */
|
||||
adminOnly,
|
||||
controller.list,
|
||||
)
|
||||
|
||||
modulesRouter.post(
|
||||
'/',
|
||||
// #swagger.tags = ['Admin · Modules']
|
||||
// #swagger.summary = 'Install or upgrade a module from a release install-manifest URL'
|
||||
// #swagger.description = 'Downloads the artifact the manifest names, verifies its sha256, inspects the archive in full and unpacks it onto the modules volume. The module mounts on the next restart.'
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["url"], properties: { url: { type: "string", description: "https URL of the release install manifest, on an allowed host" } } } } } } */
|
||||
/* #swagger.responses[201] = { description: 'Installed — restart to mount it', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
|
||||
/* #swagger.responses[400] = { description: 'The URL, the manifest, the hash or the archive was refused', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
/* #swagger.responses[502] = { description: 'The source host could not be reached or answered badly', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
adminOnly,
|
||||
body('url').isString().trim().isLength({ min: 1, max: 2048 }),
|
||||
validate,
|
||||
controller.create,
|
||||
)
|
||||
|
||||
modulesRouter.put(
|
||||
'/sources',
|
||||
// #swagger.tags = ['Admin · Modules']
|
||||
// #swagger.summary = 'Replace the allowlist of hosts modules may be installed from'
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["hosts"], properties: { hosts: { type: "string", description: "Comma- or space-separated hostnames. An empty list forbids all installs." } } } } } } */
|
||||
/* #swagger.responses[200] = { description: 'The new allowlist', content: { "application/json": { schema: { type: "object", properties: { sourceHosts: { type: "array", items: { type: "string" } } } } } } } */
|
||||
/* #swagger.responses[400] = { description: 'One of the entries is not a hostname', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
adminOnly,
|
||||
body('hosts').isString().isLength({ max: 2048 }),
|
||||
validate,
|
||||
controller.setSources,
|
||||
)
|
||||
|
||||
modulesRouter.post(
|
||||
'/restart',
|
||||
// #swagger.tags = ['Admin · Modules']
|
||||
// #swagger.summary = 'Restart the server process so module changes take effect'
|
||||
// #swagger.description = 'Runs the same graceful shutdown a SIGTERM does. The process is brought back by the supervisor, which the shipped docker-compose.yml provides; a bare `npm start` will not come back.'
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
/* #swagger.responses[202] = { description: 'Shutting down', content: { "application/json": { schema: { type: "object", properties: { restarting: { type: "boolean" } } } } } } */
|
||||
adminOnly,
|
||||
controller.restart,
|
||||
)
|
||||
|
||||
modulesRouter.post(
|
||||
'/:id/enable',
|
||||
// #swagger.tags = ['Admin · Modules']
|
||||
// #swagger.summary = 'Enable a module (takes effect on the next restart)'
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Module id.' }
|
||||
/* #swagger.responses[200] = { description: 'Enabled — restart to start it', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
|
||||
/* #swagger.responses[404] = { description: 'No such module', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
adminOnly,
|
||||
param('id').matches(ID),
|
||||
validate,
|
||||
controller.enable,
|
||||
)
|
||||
|
||||
modulesRouter.post(
|
||||
'/:id/disable',
|
||||
// #swagger.tags = ['Admin · Modules']
|
||||
// #swagger.summary = 'Stop a module now — runs its onShutdown, then its routes answer 404'
|
||||
// #swagger.description = 'The only module action that takes effect without a restart. Re-enabling needs one, because there is no onBoot re-dispatch.'
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Module id.' }
|
||||
/* #swagger.responses[200] = { description: 'Disabled', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
|
||||
/* #swagger.responses[404] = { description: 'No such module', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
adminOnly,
|
||||
param('id').matches(ID),
|
||||
validate,
|
||||
controller.disable,
|
||||
)
|
||||
|
||||
modulesRouter.post(
|
||||
'/:id/purge',
|
||||
// #swagger.tags = ['Admin · Modules']
|
||||
// #swagger.summary = "Run a disabled module’s purge.sql, dropping its tables and data"
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Module id.' }
|
||||
/* #swagger.responses[200] = { description: 'Purged', content: { "application/json": { schema: { type: "object", properties: { id: { type: "string" }, purged: { type: "integer" } } } } } } */
|
||||
/* #swagger.responses[400] = { description: 'The module ships no purge.sql', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
/* #swagger.responses[409] = { description: 'The module must be disabled first', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
adminOnly,
|
||||
param('id').matches(ID),
|
||||
validate,
|
||||
controller.purge,
|
||||
)
|
||||
|
||||
modulesRouter.delete(
|
||||
'/:id',
|
||||
// #swagger.tags = ['Admin · Modules']
|
||||
// #swagger.summary = 'Uninstall a module, optionally deleting its data too'
|
||||
// #swagger.description = "Removes the module directory and leaves its row disabled. With purge=true it also runs purge.sql first — which is the only moment it can, since purge.sql lives inside the directory being deleted."
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Module id.' }
|
||||
// #swagger.parameters['purge'] = { in: 'query', required: false, schema: { type: 'boolean' }, description: "Also run the module’s purge.sql and drop its row. Destructive and irreversible." }
|
||||
/* #swagger.responses[200] = { description: 'Uninstalled — restart to unmount it', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
|
||||
/* #swagger.responses[400] = { description: 'Purge was asked for and the module ships no purge.sql', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
/* #swagger.responses[404] = { description: 'No such module', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
adminOnly,
|
||||
param('id').matches(ID),
|
||||
query('purge').optional().isIn(['true', 'false', '1', '0']),
|
||||
validate,
|
||||
controller.remove,
|
||||
)
|
||||
|
||||
module.exports = modulesRouter
|
||||
@@ -33,7 +33,7 @@ inviteRouter.post(
|
||||
// #swagger.tags = ['Auth']
|
||||
// #swagger.summary = 'Accept an email invite (creates the account at the invited role)'
|
||||
// #swagger.description = 'Creates the website user at the invite’s pre-assigned role and logs them in (sets the session cookie). Bypasses the player_registration gate — the invite is its own authority. Rate limited + honeypot-guarded like registration.'
|
||||
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["username","password"], properties: { username: { type: "string" }, password: { type: "string" } } } } } */
|
||||
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["username","password"], properties: { username: { type: "string" }, password: { type: "string" } } } } } } */
|
||||
/* #swagger.responses[200] = { description: 'Account created and session issued', content: { "application/json": { schema: { $ref: "#/components/schemas/LoginResponse" } } } } */
|
||||
/* #swagger.responses[400] = { description: 'Validation error', content: { "application/json": { schema: { $ref: "#/components/schemas/ValidationError" } } } } */
|
||||
/* #swagger.responses[404] = { description: 'Invalid or expired invite', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
|
||||
100
server/swagger/docsSpec.js
Normal file
100
server/swagger/docsSpec.js
Normal file
@@ -0,0 +1,100 @@
|
||||
// ── The OpenAPI document core actually serves ──────────────────────────────
|
||||
//
|
||||
// `swagger-output.json` is core's own routes and only core's own routes: it is
|
||||
// generated by `npm run swagger` on a developer's machine and committed, so it
|
||||
// must come out the same regardless of which modules that developer happened to
|
||||
// have checked out. A module's routes cannot be in it, and not merely because
|
||||
// nobody put them there — a module arrives on a volume long after the image was
|
||||
// built, and core never has its sources to analyse.
|
||||
//
|
||||
// So the document served at `/api/docs.json` is assembled at REQUEST time: core's
|
||||
// committed spec, plus the `swagger-fragment.json` of every started module
|
||||
// (docs/website/MODULE_API.md §2.8 and §6.1a). This file is that assembly.
|
||||
//
|
||||
// **Core always wins a key collision.** `mergeFragment` enforces it and reports
|
||||
// what it dropped. A module cannot redefine a core path, tag or schema by shipping
|
||||
// one with the same name — which is why §6.1a tells modules to namespace the
|
||||
// schemas they define (`UoShardStatus`) while referencing core's shared ones
|
||||
// (`Error`) by core's name: the first would collide and lose, the second resolves
|
||||
// here, in the merged document, which is the only place both exist.
|
||||
//
|
||||
// **Cached, keyed on the loader's state version.** Building the document reads a
|
||||
// file per module and deep-copies a 5,000-line spec; `/api/docs` is a page an
|
||||
// operator opens occasionally and a crawler may hit repeatedly. The cache is
|
||||
// invalidated by any module state CHANGE — which is what "started modules only"
|
||||
// depends on, and the only input here that can move without a restart.
|
||||
|
||||
const fs = require('fs')
|
||||
|
||||
const modules = require('../src/modules/loader')
|
||||
const createLogger = require('../src/utils/logger')
|
||||
const { mergeFragment } = require('./mergeSpec')
|
||||
|
||||
const log = createLogger('swagger')
|
||||
|
||||
let cached = null
|
||||
let cachedVersion = -1
|
||||
|
||||
/**
|
||||
* Core's spec with every started module's fragment merged over it.
|
||||
*
|
||||
* Never throws: `/api/docs.json` answering with core's routes alone is a worse
|
||||
* document than the full one, but it is a document. A fragment that is missing,
|
||||
* unreadable or not JSON costs that module its paths and nothing else — the same
|
||||
* bargain §4.4 makes everywhere else, where one module's failure is never the
|
||||
* site's.
|
||||
*
|
||||
* @param {object} coreSpec the committed swagger-output.json — never mutated
|
||||
* @returns {object}
|
||||
*/
|
||||
function docsSpec(coreSpec) {
|
||||
// Before app.js has called modules.load(), asking is a mis-ordered boot rather
|
||||
// than a core with nothing installed (§7.6) — but this is a request handler, and
|
||||
// 500ing the docs page over it would be the wrong trade. Core's own spec is the
|
||||
// honest answer to "what is documented" at that point anyway.
|
||||
if (!modules.isLoaded()) return coreSpec
|
||||
|
||||
const version = modules.version()
|
||||
if (cached && cachedVersion === version) return cached
|
||||
|
||||
// A structural copy, because mergeFragment writes into what it is given and
|
||||
// `coreSpec` is a require()d JSON module: mutating it would make the merge
|
||||
// cumulative across rebuilds and permanent for the life of the process.
|
||||
const spec = JSON.parse(JSON.stringify(coreSpec))
|
||||
spec.paths = spec.paths || {}
|
||||
spec.tags = spec.tags || []
|
||||
spec.components = spec.components || {}
|
||||
spec.components.schemas = spec.components.schemas || {}
|
||||
|
||||
for (const { id, file } of modules.specFragments()) {
|
||||
let fragment
|
||||
try {
|
||||
fragment = JSON.parse(fs.readFileSync(file, 'utf8'))
|
||||
} catch (err) {
|
||||
log.warn('module OpenAPI fragment could not be read — its routes will be undocumented', {
|
||||
module: id,
|
||||
file,
|
||||
error: err.message,
|
||||
})
|
||||
continue
|
||||
}
|
||||
const before = Object.keys(spec.paths).length
|
||||
mergeFragment(spec, fragment, `module ${id}`)
|
||||
log.debug('merged module OpenAPI fragment', {
|
||||
module: id,
|
||||
paths: Object.keys(spec.paths).length - before,
|
||||
})
|
||||
}
|
||||
|
||||
cached = spec
|
||||
cachedVersion = version
|
||||
return spec
|
||||
}
|
||||
|
||||
/** Test seam: forget the cached document. */
|
||||
function reset() {
|
||||
cached = null
|
||||
cachedVersion = -1
|
||||
}
|
||||
|
||||
module.exports = { docsSpec, reset }
|
||||
File diff suppressed because it is too large
Load Diff
@@ -33,8 +33,11 @@ const doc = {
|
||||
title: `${brand.name} API`,
|
||||
version: pkg.version,
|
||||
description:
|
||||
`REST API for the ${brand.name} website, wiki and admin panel — a private ` +
|
||||
'Ultima Online shard.\n\n' +
|
||||
`REST API for the ${brand.name} website, wiki and admin panel.\n\n` +
|
||||
'This document is core. Installed modules add their own paths, tags and ' +
|
||||
'schemas to it at request time from the fragment each one ships, so ' +
|
||||
'`/api/docs.json` on a running instance describes more than `npm run swagger` ' +
|
||||
'generates here (docs/website/MODULE_API.md §6.1a).\n\n' +
|
||||
'### Authentication\n' +
|
||||
`- **Web / admin panel** uses an httpOnly session cookie (\`${COOKIE_NAME}\`) issued by ` +
|
||||
'`POST /api/v1/auth/login` (plus `/login/totp` when 2FA is enabled).\n' +
|
||||
@@ -47,27 +50,33 @@ const doc = {
|
||||
{ url: '/', description: 'Same-origin (current host)' },
|
||||
{ url: 'http://localhost:3000', description: 'Local development' },
|
||||
],
|
||||
// Core's tags only. A module contributes its own in its fragment, and they are
|
||||
// merged in beside these — the four game-specific ones that used to sit here
|
||||
// (`Public · Shard`, `Public · Atlas`, `Player · Shard`, `Admin · Shard`) went
|
||||
// with the routes they group, and arrive back from module-uo on any instance
|
||||
// that has it installed.
|
||||
tags: [
|
||||
{ name: 'Health', description: 'Liveness probe' },
|
||||
{ name: 'Auth', description: 'Web session login/logout (cookie + TOTP)' },
|
||||
{ name: 'Auth · Me', description: 'The signed-in account: profile, notification streams and devices' },
|
||||
{ name: 'Auth · Mobile', description: 'Native bearer-token login, refresh and logout' },
|
||||
{ name: 'Auth · SSO', description: 'OAuth2 / OIDC provider discovery and redirect flow' },
|
||||
{ name: 'Public', description: 'Unauthenticated site content (settings, posts, wiki, contact)' },
|
||||
{ name: 'Public · Shard', description: 'Live shard data ingested from the uo-link sidecar (status, feed, economy, IDOC, characters)' },
|
||||
{ name: 'Public · Atlas', description: 'Spawn atlas / bestiary — static shard content parsed from the shard\'s own ServUO tree, independent of the sidecar' },
|
||||
{ name: 'Admin · Account', description: 'Self-service account security (2FA, linked identities)' },
|
||||
{ name: 'Player', description: 'Self-service player accounts (register, credentials, 2FA, linked identities)' },
|
||||
{ name: 'Player · Shard', description: 'Link an in-game account and read its roster / vendors (uo-link)' },
|
||||
{ name: 'Player · Appeals', description: 'Player-submitted moderation appeals' },
|
||||
{ name: 'Settings', description: 'Site-wide settings any authenticated account may read (nav overrides)' },
|
||||
{ name: 'Admin · Dashboard', description: 'Dashboard summary and site mode' },
|
||||
{ name: 'Admin · Posts', description: 'News / five-on-friday / newsletter / screenshots + uploads' },
|
||||
{ name: 'Admin · Pages', description: 'Editable static site pages' },
|
||||
{ name: 'Admin · Wiki', description: 'Wiki pages, categories, tags and revisions' },
|
||||
{ name: 'Admin · Settings', description: 'Site settings (admin only)' },
|
||||
{ name: 'Admin · Email', description: 'Outbound email configuration and delivery test (admin only)' },
|
||||
{ name: 'Admin · Invites', description: 'Registration invites — issue, list and revoke' },
|
||||
{ name: 'Admin · Moderation', description: 'Player reports, appeals and moderator actions' },
|
||||
{ name: 'Admin · Activity', description: 'Admin activity log' },
|
||||
{ name: 'Admin · Bot Activity', description: 'Bot-scoring/ban state and emergency unban (admin only)' },
|
||||
{ name: 'Admin · Discord Bot', description: 'Discord bot token/config and live status (admin only)' },
|
||||
{ name: 'Admin · Shard', description: 'uo-link sidecar connection config, live status and town crier (admin only)' },
|
||||
{ name: 'Admin · Auth Providers', description: 'SSO provider configuration (admin only)' },
|
||||
{ name: 'Admin · Users', description: 'User management (admin only)' },
|
||||
],
|
||||
@@ -918,584 +927,6 @@ const doc = {
|
||||
removed: { type: 'boolean', description: 'Whether the IP had an entry that was cleared.', example: true },
|
||||
},
|
||||
},
|
||||
// ── uo-link shard data ──────────────────────────────────────────────
|
||||
ShardStatus: {
|
||||
type: 'object',
|
||||
description: 'Public shard status (GET /public/shard/status).',
|
||||
properties: {
|
||||
enabled: { type: 'boolean', example: true },
|
||||
status: { type: 'string', example: 'connected', description: 'connected | reconnecting | disconnected | error' },
|
||||
pluginConnected: { type: 'boolean', description: 'Is the shard link up right now?', example: true },
|
||||
lastEventAt: { type: 'string', format: 'date-time', nullable: true },
|
||||
onlineCount: { type: 'integer', example: 12 },
|
||||
economy: { $ref: '#/components/schemas/ShardEconomyPoint' },
|
||||
},
|
||||
},
|
||||
ShardEvent: {
|
||||
type: 'object',
|
||||
description: 'A logged shard event.',
|
||||
properties: {
|
||||
id: { type: 'integer', example: 4821 },
|
||||
kind: { type: 'string', example: 'vendor.sale' },
|
||||
t: { type: 'integer', description: 'Event time, epoch ms.', example: 1783720195626 },
|
||||
bootId: { type: 'string', nullable: true, example: 'boot-abc123' },
|
||||
payload: { type: 'object', additionalProperties: true, description: 'The full event object.' },
|
||||
createdAt: { type: 'string', format: 'date-time' },
|
||||
},
|
||||
},
|
||||
ShardEconomyPoint: {
|
||||
type: 'object',
|
||||
nullable: true,
|
||||
description: 'One gold-supply sample.',
|
||||
properties: {
|
||||
accounts: { type: 'integer', nullable: true, example: 240 },
|
||||
gold: { type: 'integer', nullable: true, example: 1028983421 },
|
||||
t: { type: 'integer', description: 'Sample time, epoch ms.', example: 1783720000000 },
|
||||
},
|
||||
},
|
||||
ShardOnlinePlayer: {
|
||||
type: 'object',
|
||||
description: 'A LINKED player online now (only accounts linked to a website user are listed).',
|
||||
properties: {
|
||||
serial: { type: 'string', example: '0x24C' },
|
||||
name: { type: 'string', example: 'Darrow' },
|
||||
map: { type: 'string', nullable: true, example: 'Trammel' },
|
||||
x: { type: 'integer', nullable: true, example: 1402 },
|
||||
y: { type: 'integer', nullable: true, example: 1604 },
|
||||
z: { type: 'integer', nullable: true, example: 0 },
|
||||
},
|
||||
},
|
||||
ShardVendorSale: {
|
||||
type: 'object',
|
||||
description: 'A player-vendor sale (visible only to the linked owner).',
|
||||
properties: {
|
||||
t: { type: 'integer', description: 'Sale time, epoch ms.', example: 1783720195626 },
|
||||
itemType: { type: 'string', example: 'Longsword' },
|
||||
amount: { type: 'integer', example: 1 },
|
||||
price: { type: 'integer', example: 100 },
|
||||
commission: { type: 'integer', nullable: true, example: 5 },
|
||||
ownerAcct: { type: 'string', example: 'whitlocktech' },
|
||||
},
|
||||
},
|
||||
ShardHouse: {
|
||||
type: 'object',
|
||||
description: 'A house at its current decay stage.',
|
||||
properties: {
|
||||
serial: { type: 'string', example: '0x4004705F' },
|
||||
stage: { type: 'string', example: 'IDOC' },
|
||||
map: { type: 'string', nullable: true, example: 'Trammel' },
|
||||
x: { type: 'integer', nullable: true },
|
||||
y: { type: 'integer', nullable: true },
|
||||
z: { type: 'integer', nullable: true },
|
||||
region: { type: 'string', nullable: true },
|
||||
name: { type: 'string', nullable: true, example: 'An Unnamed House' },
|
||||
ownerSerial: { type: 'string', nullable: true },
|
||||
ownerAcct: { type: 'string', nullable: true },
|
||||
builtOn: { type: 'string', format: 'date-time', nullable: true },
|
||||
lastRefreshed: { type: 'string', format: 'date-time', nullable: true },
|
||||
isIdoc: { type: 'boolean', example: true },
|
||||
updatedAt: { type: 'string', format: 'date-time' },
|
||||
},
|
||||
},
|
||||
ShardPointsBoard: {
|
||||
type: 'object',
|
||||
description:
|
||||
"One point system's leaderboard (Protocol 3.0 points.board). The shard carries ~25 separate point currencies; each publishes its own board. The display name may arrive as a literal string, a cliloc id, or both — resolve clilocs client-side.",
|
||||
properties: {
|
||||
system: { type: 'string', example: 'QueensLoyalty', description: "The shard's PointsType name; the board's stable key." },
|
||||
nameString: { type: 'string', nullable: true, example: "Queen's Loyalty" },
|
||||
nameNumber: { type: 'integer', nullable: true, example: 1114938, description: 'Cliloc id, 0 when the name is a literal.' },
|
||||
maxPoints: { type: 'integer', nullable: true, example: 30000 },
|
||||
players: { type: 'integer', nullable: true, example: 842, description: 'Players actually holding points in this system.' },
|
||||
showOnGump: { type: 'boolean', example: true, description: "The shard's own 'is this player-facing?' flag." },
|
||||
top: {
|
||||
type: 'array',
|
||||
description: 'The ranked players, best first. Capped by the shard (10 by default). Empty when nobody has scored yet.',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
rank: { type: 'integer', example: 1 },
|
||||
serial: { type: 'string', example: '0x1A2B' },
|
||||
name: { type: 'string', example: 'Darrow', description: 'Omitted when the leaderboards `name` field is gated above the caller.' },
|
||||
points: { type: 'integer', example: 29500 },
|
||||
},
|
||||
},
|
||||
},
|
||||
t: { type: 'integer', nullable: true, description: 'Frame time, epoch ms.' },
|
||||
updatedAt: { type: 'string', format: 'date-time' },
|
||||
},
|
||||
},
|
||||
ShardMarketLocation: {
|
||||
type: 'object',
|
||||
nullable: true,
|
||||
description:
|
||||
"Where a vendor is standing. ONE nested object rather than flat map/x/y/region because it is one admin-configurable field (`market.location`) — the whole object is omitted when that field is gated above the caller.",
|
||||
properties: {
|
||||
map: { type: 'string', nullable: true, example: 'Trammel' },
|
||||
x: { type: 'integer', nullable: true, example: 1421 },
|
||||
y: { type: 'integer', nullable: true, example: 1699 },
|
||||
z: { type: 'integer', nullable: true, example: 0 },
|
||||
region: { type: 'string', nullable: true, example: 'Britain' },
|
||||
house: { type: 'string', nullable: true, example: "Darrow's Villa", description: "The house SIGN's name, not the house type. Null for a vendor standing outside one." },
|
||||
},
|
||||
},
|
||||
ShardMarketListing: {
|
||||
type: 'object',
|
||||
description:
|
||||
'One priced listing on a player vendor, carrying enough of its shop to be actionable without a second request.',
|
||||
properties: {
|
||||
serial: { type: 'string', example: '0x40012ABC' },
|
||||
itemId: { type: 'integer', example: 3922, description: 'ItemID (the art/graphic id).' },
|
||||
hue: { type: 'integer', example: 0 },
|
||||
amount: { type: 'integer', example: 1 },
|
||||
price: { type: 'integer', example: 25000 },
|
||||
name: { type: 'string', nullable: true, description: "The item's own literal name, set by a player. Null for most items." },
|
||||
cliloc: { type: 'integer', nullable: true, example: 1023721, description: "The item's LabelNumber." },
|
||||
displayName: {
|
||||
type: 'string',
|
||||
nullable: true,
|
||||
example: 'quarter staff',
|
||||
description: 'Resolved server-side from `name` (preferred, being player-set and more specific) else `cliloc`. Null on a shard with no cliloc table configured — render the item id.',
|
||||
},
|
||||
child: { type: 'boolean', example: false, description: 'Priced by an enclosing container rather than itself, exactly as the in-game Vendor Search reports it.' },
|
||||
vendor: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
serial: { type: 'string', example: '0x40001234' },
|
||||
shopName: { type: 'string', nullable: true, example: "Darrow's Bargains" },
|
||||
ownerSerial: { type: 'string', nullable: true, example: '0x1A2B', description: 'Omitted when the market `ownerSerial` field is gated above the caller.' },
|
||||
ownerName: { type: 'string', nullable: true, example: 'Darrow', description: 'Omitted when the market `ownerName` field is gated above the caller.' },
|
||||
location: { $ref: '#/components/schemas/ShardMarketLocation' },
|
||||
updatedAt: { type: 'string', format: 'date-time', description: 'When the shard last published this shop.' },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
ShardMarketPage: {
|
||||
type: 'object',
|
||||
description: 'A page of marketplace listings plus the unpaginated total and the staleness stamp.',
|
||||
properties: {
|
||||
listings: { type: 'array', items: { $ref: '#/components/schemas/ShardMarketListing' } },
|
||||
total: { type: 'integer', example: 1284, description: 'Matching listings, ignoring paging.' },
|
||||
limit: { type: 'integer', example: 50 },
|
||||
offset: { type: 'integer', example: 0 },
|
||||
vendors: { type: 'integer', example: 137, description: 'Vendors in the whole index.' },
|
||||
staleAt: {
|
||||
type: 'string',
|
||||
format: 'date-time',
|
||||
nullable: true,
|
||||
description: 'The OLDEST vendor row. The shard sweeps vendors round-robin, so the index can be a full cycle behind and a client must say so rather than implying live prices.',
|
||||
},
|
||||
},
|
||||
},
|
||||
ShardMarketVendor: {
|
||||
type: 'object',
|
||||
description: 'One player vendor and its listings.',
|
||||
properties: {
|
||||
serial: { type: 'string', example: '0x40001234' },
|
||||
shopName: { type: 'string', nullable: true, example: "Darrow's Bargains" },
|
||||
ownerSerial: { type: 'string', nullable: true },
|
||||
ownerName: { type: 'string', nullable: true, example: 'Darrow' },
|
||||
location: { $ref: '#/components/schemas/ShardMarketLocation' },
|
||||
count: { type: 'integer', example: 250, description: 'Listings the shard published for this shop.' },
|
||||
total: { type: 'integer', example: 3104, description: 'Listings the shop actually holds.' },
|
||||
truncated: { type: 'boolean', example: true, description: '`total` exceeds `count` — the shop holds more than the shard publishes per frame.' },
|
||||
updatedAt: { type: 'string', format: 'date-time' },
|
||||
items: { type: 'array', items: { $ref: '#/components/schemas/ShardMarketListing' } },
|
||||
},
|
||||
},
|
||||
ShardMarketMeta: {
|
||||
type: 'object',
|
||||
description: 'Marketplace size, staleness and the filter options a client needs to build its UI.',
|
||||
properties: {
|
||||
vendors: { type: 'integer', example: 137 },
|
||||
items: { type: 'integer', example: 18422 },
|
||||
staleAt: { type: 'string', format: 'date-time', nullable: true },
|
||||
freshAt: { type: 'string', format: 'date-time', nullable: true },
|
||||
maps: { type: 'array', items: { type: 'string' }, example: ['Felucca', 'Trammel'], description: "Facets that actually hold vendors. From the shard's own data — never a hardcoded list." },
|
||||
regions: { type: 'array', items: { type: 'string' }, example: ['Britain', 'Luna'] },
|
||||
},
|
||||
},
|
||||
ShardFeatures: {
|
||||
type: 'object',
|
||||
description:
|
||||
"The shard features the caller may reach, plus the audience rung they resolved to. Drives client nav so it never renders a link that would 403.",
|
||||
properties: {
|
||||
level: {
|
||||
type: 'string',
|
||||
enum: ['anonymous', 'logged_in', 'player', 'staff', 'admin'],
|
||||
example: 'anonymous',
|
||||
},
|
||||
features: {
|
||||
type: 'array',
|
||||
items: { type: 'string' },
|
||||
example: ['status', 'activity', 'champs', 'guilds', 'governors', 'houses', 'presence'],
|
||||
},
|
||||
},
|
||||
},
|
||||
ShardFeatureVisibility: {
|
||||
type: 'object',
|
||||
description: 'Visibility settings for one shard feature.',
|
||||
properties: {
|
||||
enabled: { type: 'boolean', example: true },
|
||||
audience: {
|
||||
type: 'string',
|
||||
enum: ['anonymous', 'logged_in', 'player', 'staff', 'admin'],
|
||||
description: 'Minimum rung that may reach this feature. Each rung implies the ones below it.',
|
||||
example: 'anonymous',
|
||||
},
|
||||
stream: {
|
||||
type: 'boolean',
|
||||
description: "Whether this feature's event kinds fan out over SSE at all.",
|
||||
example: true,
|
||||
},
|
||||
fieldRules: {
|
||||
type: 'object',
|
||||
additionalProperties: { type: 'string' },
|
||||
description:
|
||||
'Per-field rung overrides for the sensitive fields this feature exposes. acct / webId are admin-only always and are rejected here.',
|
||||
example: { location: 'staff' },
|
||||
},
|
||||
},
|
||||
},
|
||||
ShardVisibilityConfig: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
ladder: {
|
||||
type: 'array',
|
||||
items: { type: 'string' },
|
||||
example: ['anonymous', 'logged_in', 'player', 'staff', 'admin'],
|
||||
},
|
||||
lockedFields: { type: 'array', items: { type: 'string' }, example: ['acct', 'webId'] },
|
||||
defaults: {
|
||||
type: 'object',
|
||||
additionalProperties: { $ref: '#/components/schemas/ShardFeatureVisibility' },
|
||||
},
|
||||
features: {
|
||||
type: 'object',
|
||||
additionalProperties: { $ref: '#/components/schemas/ShardFeatureVisibility' },
|
||||
},
|
||||
},
|
||||
},
|
||||
ShardVisibilityUpdate: {
|
||||
type: 'object',
|
||||
required: ['features'],
|
||||
properties: {
|
||||
features: {
|
||||
type: 'object',
|
||||
additionalProperties: { $ref: '#/components/schemas/ShardFeatureVisibility' },
|
||||
example: { market: { enabled: true, audience: 'player', stream: false, fieldRules: { ownerName: 'player' } } },
|
||||
},
|
||||
},
|
||||
},
|
||||
// ── Spawn atlas (Protocol 3.0 Part C) ────────────────────────────────
|
||||
// Static shard content, parsed from the shard's own ServUO tree. Nothing
|
||||
// here comes from the sidecar, so it stays populated while the shard is
|
||||
// down. Facet names are whatever the shard's files declare — the examples
|
||||
// below are stock ServUO, not a fixed list.
|
||||
AtlasCreature: {
|
||||
type: 'object',
|
||||
description: 'A creature in the bestiary. `places`/`points`/`alsoHere` are present only on the single-creature route.',
|
||||
properties: {
|
||||
slug: { type: 'string', example: 'lizardman' },
|
||||
name: { type: 'string', example: 'Lizardman' },
|
||||
total: { type: 'integer', description: 'How many can be alive at once, summed across every spawner.', example: 214 },
|
||||
points: { type: 'integer', description: 'How many spawners mention this creature.', example: 62 },
|
||||
facets: {
|
||||
type: 'object',
|
||||
additionalProperties: { type: 'integer' },
|
||||
description: "This creature's share per facet.",
|
||||
example: { Felucca: 96, Trammel: 88, Tokuno: 30 },
|
||||
},
|
||||
art: { type: 'string', nullable: true, description: 'Operator-supplied art under uploads/atlas/. NULL on a fresh import — the repo ships no creature art.' },
|
||||
places: {
|
||||
type: 'array',
|
||||
description: 'Where it spawns, aggregated by resolved place. The answer the atlas exists to give.',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
facet: { type: 'string', example: 'Trammel' },
|
||||
label: { type: 'string', description: 'Resolved region, else nearest landmark group, else "Wilderness".', example: 'Shrines' },
|
||||
spawners: { type: 'integer', example: 7 },
|
||||
maxAlive: { type: 'integer', example: 21 },
|
||||
},
|
||||
},
|
||||
},
|
||||
spawners: {
|
||||
type: 'array',
|
||||
description: 'The individual spawners. Named separately from `points` (the count) so one key never means two things.',
|
||||
items: { $ref: '#/components/schemas/AtlasSpawner' },
|
||||
},
|
||||
spawnersTruncated: { type: 'boolean', description: 'True when the spawner list was cut at the requested bound.', example: false },
|
||||
alsoHere: {
|
||||
type: 'array',
|
||||
description: 'Creatures sharing a spawner with this one.',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
slug: { type: 'string', example: 'lizardman-warrior' },
|
||||
name: { type: 'string', example: 'Lizardman Warrior' },
|
||||
shared: { type: 'integer', example: 12 },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
AtlasSpawner: {
|
||||
type: 'object',
|
||||
description: 'One ServUO spawner, with the place its coordinates resolved to.',
|
||||
properties: {
|
||||
id: { type: 'integer' },
|
||||
facet: { type: 'string', example: 'Felucca' },
|
||||
name: { type: 'string', nullable: true, description: "The spawner's own name in the ServUO file." },
|
||||
x: { type: 'integer', example: 5411 },
|
||||
y: { type: 'integer', example: 1234 },
|
||||
width: { type: 'integer' },
|
||||
height: { type: 'integer' },
|
||||
range: { type: 'integer', description: 'Spawn radius.' },
|
||||
maxCount: { type: 'integer', description: 'How many of THIS creature this spawner keeps alive.', example: 3 },
|
||||
minDelay: { type: 'integer', description: 'Respawn window, in SECONDS. Normalised at parse time — the source stores minutes or seconds per record, decided by its own DelayInSec flag.', example: 300 },
|
||||
maxDelay: { type: 'integer', example: 600 },
|
||||
todStart: { type: 'integer', description: 'Meaningless unless todMode is non-zero.' },
|
||||
todEnd: { type: 'integer' },
|
||||
todMode: { type: 'integer' },
|
||||
region: { type: 'string', nullable: true, example: 'Despise' },
|
||||
landmark: { type: 'string', nullable: true, example: 'Covetous' },
|
||||
label: { type: 'string', description: 'Region, else landmark group, else "Wilderness".', example: 'Despise' },
|
||||
},
|
||||
},
|
||||
AtlasCreaturePage: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
total: { type: 'integer', description: 'Matching creatures before pagination.', example: 800 },
|
||||
limit: { type: 'integer', example: 50 },
|
||||
offset: { type: 'integer', example: 0 },
|
||||
creatures: { type: 'array', items: { $ref: '#/components/schemas/AtlasCreature' } },
|
||||
},
|
||||
},
|
||||
AtlasRegion: {
|
||||
type: 'object',
|
||||
description: 'A named region, flattened out of the shard\'s nested Regions.xml.',
|
||||
properties: {
|
||||
facet: { type: 'string', example: 'Felucca' },
|
||||
name: { type: 'string', example: 'Despise' },
|
||||
type: { type: 'string', nullable: true, description: 'ServUO region class.', example: 'DungeonRegion' },
|
||||
priority: { type: 'integer', example: 50 },
|
||||
parent: { type: 'string', nullable: true, example: 'Britain' },
|
||||
rects: {
|
||||
type: 'array',
|
||||
description: 'The rectangles that placed each spawn point.',
|
||||
items: { type: 'object', additionalProperties: true },
|
||||
},
|
||||
},
|
||||
},
|
||||
AtlasLandmark: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
facet: { type: 'string', example: 'Trammel' },
|
||||
name: { type: 'string', example: 'Level 1' },
|
||||
group: { type: 'string', nullable: true, description: 'Innermost enclosing parent — the label worth showing.', example: 'Covetous' },
|
||||
x: { type: 'integer', example: 5411 },
|
||||
y: { type: 'integer', example: 1234 },
|
||||
z: { type: 'integer', example: 0 },
|
||||
},
|
||||
},
|
||||
AtlasChampion: {
|
||||
type: 'object',
|
||||
description: 'A CONFIGURED champion altar. Not the live board — see GET /public/shard/champs for that.',
|
||||
properties: {
|
||||
slug: { type: 'string', example: 'felucca-deceit' },
|
||||
name: { type: 'string', example: 'Deceit' },
|
||||
group: { type: 'string', nullable: true, description: 'Spawn group; one altar active per group.', example: 'Dungeons' },
|
||||
type: { type: 'string', nullable: true, description: 'NULL when the champion is drawn at activation.', example: 'UnholyTerror' },
|
||||
randomType: { type: 'boolean', example: false },
|
||||
facet: { type: 'string', example: 'Felucca' },
|
||||
x: { type: 'integer' },
|
||||
y: { type: 'integer' },
|
||||
z: { type: 'integer' },
|
||||
radius: { type: 'integer', example: 60 },
|
||||
label: { type: 'string', nullable: true, example: 'Deceit' },
|
||||
},
|
||||
},
|
||||
AtlasMeta: {
|
||||
type: 'object',
|
||||
description: 'What atlas is loaded. Game-world facts only: the ServUO path, source hashes and any pending refresh are operator detail and live on the admin status route.',
|
||||
properties: {
|
||||
importedAt: { type: 'string', format: 'date-time', nullable: true },
|
||||
generatedAt: { type: 'string', format: 'date-time', nullable: true },
|
||||
counts: {
|
||||
type: 'object',
|
||||
nullable: true,
|
||||
additionalProperties: true,
|
||||
example: { facets: 6, points: 6455, creatures: 800, regions: 387, landmarks: 558, champions: 25, unresolvedPoints: 1086 },
|
||||
},
|
||||
facets: { type: 'array', items: { type: 'string' }, example: ['Felucca', 'Ilshenar', 'Malas', 'TerMur', 'Tokuno', 'Trammel'] },
|
||||
},
|
||||
},
|
||||
AtlasStatus: {
|
||||
type: 'object',
|
||||
description: 'Admin view of atlas state: where the tree is, whether it is readable, whether it has drifted from what is loaded, and any refresh staged for review.',
|
||||
properties: {
|
||||
configured: { type: 'boolean', example: true },
|
||||
path: { type: 'string', example: '/srv/servuo' },
|
||||
treeReadable: { type: 'boolean', example: true },
|
||||
drift: { type: 'boolean', nullable: true, description: 'True when the tree\'s source hashes differ from the loaded atlas. NULL when the tree could not be read.', example: false },
|
||||
facets: { type: 'array', items: { type: 'string' } },
|
||||
importedAt: { type: 'string', format: 'date-time', nullable: true },
|
||||
counts: { type: 'object', nullable: true, additionalProperties: true },
|
||||
pending: {
|
||||
type: 'object',
|
||||
nullable: true,
|
||||
description: 'A refresh that was parsed but NOT applied because it would remove a facet. `status` is pending or rejected.',
|
||||
additionalProperties: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
AtlasRefreshResult: {
|
||||
type: 'object',
|
||||
description: 'Outcome of a refresh. Reported rather than thrown, so an unreadable tree is an answer and not a 500.',
|
||||
properties: {
|
||||
status: {
|
||||
type: 'string',
|
||||
enum: ['skipped', 'unavailable', 'unchanged', 'imported', 'needsReview', 'failed', 'rejected', 'none'],
|
||||
example: 'imported',
|
||||
},
|
||||
reason: { type: 'string', nullable: true },
|
||||
path: { type: 'string', nullable: true },
|
||||
counts: { type: 'object', nullable: true, additionalProperties: true },
|
||||
addedFacets: { type: 'array', items: { type: 'string' } },
|
||||
removedFacets: { type: 'array', items: { type: 'string' } },
|
||||
},
|
||||
},
|
||||
ClilocStatus: {
|
||||
type: 'object',
|
||||
description:
|
||||
'Admin view of cliloc state: where the converted file is, whether it is readable, how many entries are loaded, and whether the file has drifted from them. `configured: false` is a supported state — item names then render as ids.',
|
||||
properties: {
|
||||
configured: { type: 'boolean', example: true },
|
||||
path: { type: 'string', example: '/srv/uo-client' },
|
||||
file: { type: 'string', nullable: true, description: 'The file actually resolved, when the path is a directory.', example: '/srv/uo-client/clilocs.tsv' },
|
||||
fileReadable: { type: 'boolean', example: true },
|
||||
problem: { type: 'string', nullable: true, description: 'Why the file cannot be used, when it cannot. Set (with code COMPRESSED) for a readable-but-unconverted client file.', example: null },
|
||||
code: { type: 'string', nullable: true, description: 'Machine-readable cause of `problem`.', enum: ['NO_PATH', 'NOT_FOUND', 'NO_FILE', 'UNREADABLE', 'COMPRESSED'] },
|
||||
drift: { type: 'boolean', nullable: true, description: 'True when any source hash differs from the loaded table. NULL when the sources could not be read or are not usable.', example: false },
|
||||
count: { type: 'integer', description: 'Entries currently loaded.', example: 67496 },
|
||||
sources: {
|
||||
type: 'array',
|
||||
items: { type: 'string' },
|
||||
description: 'Every source found now, root-relative, base first then overlays in merge order.',
|
||||
example: ['clilocs.plain', 'custom/uomysticmoon.tsv'],
|
||||
},
|
||||
loadedSources: {
|
||||
type: 'array',
|
||||
nullable: true,
|
||||
description: 'What each source contributed at the last import.',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
label: { type: 'string', example: 'custom/uomysticmoon.tsv' },
|
||||
kind: { type: 'string', enum: ['base', 'custom'], example: 'custom' },
|
||||
entries: { type: 'integer', example: 37 },
|
||||
added: { type: 'integer', description: 'Ids this source introduced.', example: 25 },
|
||||
overrode: { type: 'integer', description: 'Ids it replaced from an earlier source.', example: 12 },
|
||||
},
|
||||
},
|
||||
},
|
||||
missingSources: {
|
||||
type: 'array',
|
||||
items: { type: 'string' },
|
||||
description: 'Sources loaded previously and now absent. An import refuses these without `approve`.',
|
||||
example: [],
|
||||
},
|
||||
importedAt: { type: 'string', format: 'date-time', nullable: true },
|
||||
sourceBytes: { type: 'integer', nullable: true, example: 4973525 },
|
||||
},
|
||||
},
|
||||
ClilocRefreshResult: {
|
||||
type: 'object',
|
||||
description:
|
||||
'Outcome of a cliloc refresh. Reported rather than thrown, so a missing or compressed file is an answer and not a 500.',
|
||||
properties: {
|
||||
status: {
|
||||
type: 'string',
|
||||
enum: ['skipped', 'unavailable', 'unchanged', 'imported', 'needsReview', 'failed'],
|
||||
description: '`needsReview` means a previously-loaded source has vanished and nothing was applied; re-run with `approve` to accept it.',
|
||||
example: 'imported',
|
||||
},
|
||||
reason: { type: 'string', nullable: true },
|
||||
code: {
|
||||
type: 'string',
|
||||
nullable: true,
|
||||
description: 'Machine-readable cause. `COMPRESSED` means the client\'s own Cliloc.enu was supplied instead of a converted one.',
|
||||
enum: ['NO_PATH', 'NOT_FOUND', 'NO_FILE', 'UNREADABLE', 'COMPRESSED', 'TRUNCATED', 'EMPTY', 'NOT_BUFFER'],
|
||||
},
|
||||
path: { type: 'string', nullable: true },
|
||||
file: { type: 'string', nullable: true },
|
||||
count: { type: 'integer', nullable: true, description: 'Entries stored (blank strings are dropped).', example: 67496 },
|
||||
parsed: { type: 'integer', nullable: true, description: 'Entries read across every source before blanks were dropped.', example: 123527 },
|
||||
blank: { type: 'integer', nullable: true, example: 55994 },
|
||||
sources: {
|
||||
type: 'array',
|
||||
nullable: true,
|
||||
description: 'Per-source breakdown: what each file contributed and how much of it overrode an earlier source.',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
label: { type: 'string' },
|
||||
kind: { type: 'string', enum: ['base', 'custom'] },
|
||||
entries: { type: 'integer' },
|
||||
added: { type: 'integer' },
|
||||
overrode: { type: 'integer' },
|
||||
},
|
||||
},
|
||||
},
|
||||
missingSources: {
|
||||
type: 'array',
|
||||
nullable: true,
|
||||
items: { type: 'string' },
|
||||
description: 'On `needsReview`: the sources that vanished. Nothing was applied.',
|
||||
},
|
||||
acceptedMissing: {
|
||||
type: 'array',
|
||||
nullable: true,
|
||||
items: { type: 'string' },
|
||||
description: 'On `imported` with `approve`: the vanished sources the admin accepted.',
|
||||
},
|
||||
},
|
||||
},
|
||||
ShardLinkRequest: {
|
||||
type: 'object',
|
||||
required: ['code'],
|
||||
properties: {
|
||||
code: { type: 'string', description: 'The one-time code shown by [link in game.', example: 'AB12CD' },
|
||||
},
|
||||
},
|
||||
ShardLinkResult: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
linked: { type: 'boolean', example: true },
|
||||
account: { type: 'string', example: 'whitlocktech' },
|
||||
},
|
||||
},
|
||||
ShardLink: {
|
||||
type: 'object',
|
||||
description: 'A linked in-game account (GET /player/shard/accounts).',
|
||||
properties: {
|
||||
account: { type: 'string', example: 'whitlocktech' },
|
||||
userId: { type: 'integer', example: 42 },
|
||||
charName: { type: 'string', nullable: true, example: 'Darrow' },
|
||||
linkedAt: { type: 'string', format: 'date-time' },
|
||||
},
|
||||
},
|
||||
TownCrierRequest: {
|
||||
type: 'object',
|
||||
required: ['id', 'lines'],
|
||||
properties: {
|
||||
id: { type: 'string', maxLength: 64, description: 'Re-posting the same id replaces the prior entry.', example: 'news-42' },
|
||||
lines: { type: 'array', items: { type: 'string', maxLength: 200 }, example: ['Hear ye!', 'Market tax is now 5%.'] },
|
||||
durationSec: { type: 'integer', minimum: 1, maximum: 86400, example: 3600 },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
@@ -1549,8 +980,37 @@ function normalizePaths(spec) {
|
||||
process.env.DB_HOST = process.env.DB_HOST || '127.0.0.1'
|
||||
process.env.DB_PORT = process.env.DB_PORT || '59999'
|
||||
|
||||
// ── An annotation swagger-autogen cannot parse is DROPPED, not failed ──────
|
||||
//
|
||||
// It `console.error`s "Syntax error" or "out of structure", skips that one
|
||||
// annotation, and prints `Success` in green. Nothing was listening, so the tree
|
||||
// had been carrying a broken one — `POST /api/v1/admin/invites` documented with an
|
||||
// EMPTY request body — for as long as it had existed. The same class turned up
|
||||
// four more times in module-uo, whose annotations came from here.
|
||||
//
|
||||
// Two ways one breaks, both of them invisible in review: an object literal a
|
||||
// brace short, and a `"` or a backtick inside a single-quoted description
|
||||
// (swagger-autogen re-quotes both to `'` before evaluating, which ends the string
|
||||
// early). Capturing the diagnostics is the only way to be told.
|
||||
const swaggerComplaints = []
|
||||
const realConsoleError = console.error
|
||||
console.error = (...args) => {
|
||||
const line = args.map(String).join(' ')
|
||||
if (/syntax error|out of structure/i.test(line)) swaggerComplaints.push(line.trim())
|
||||
else realConsoleError(...args)
|
||||
}
|
||||
|
||||
/* eslint-disable global-require */
|
||||
swaggerAutogen(outputFile, routes, doc)
|
||||
.then(() => {
|
||||
console.error = realConsoleError
|
||||
if (swaggerComplaints.length > 0) {
|
||||
throw new Error(
|
||||
`swagger: ${swaggerComplaints.length} annotation(s) could not be parsed and were DROPPED ` +
|
||||
`— the spec would be missing what they described:\n ${swaggerComplaints.join('\n ')}`,
|
||||
)
|
||||
}
|
||||
})
|
||||
.then(() => require('./slotSpecs').mergeSlotSpecs(outputFile))
|
||||
.then(() => {
|
||||
const written = JSON.parse(fs.readFileSync(outputFile, 'utf8'))
|
||||
@@ -1561,6 +1021,7 @@ swaggerAutogen(outputFile, routes, doc)
|
||||
return require('../src/utils/db').close()
|
||||
})
|
||||
.catch((err) => {
|
||||
console.error = realConsoleError
|
||||
process.stderr.write(`${err.stack || err.message}\n`)
|
||||
process.exit(1)
|
||||
})
|
||||
|
||||
476
server/test/adminModules.test.js
Normal file
476
server/test/adminModules.test.js
Normal file
@@ -0,0 +1,476 @@
|
||||
// ── Admin · Modules: the delivery surface ──────────────────────────────────
|
||||
//
|
||||
// Phase 4, slice 1 of MODULE_SYSTEM.md §2.7.2. The controller is tested directly
|
||||
// with a mock `res` and stubbed models — the same shape adminUsers.test.js uses —
|
||||
// because what is interesting here is not the HTTP plumbing but the ORDER of
|
||||
// operations and which of the three sources of truth answers which question.
|
||||
//
|
||||
// Two of these tests exist to pin decisions that are easy to "fix" back into
|
||||
// being wrong:
|
||||
//
|
||||
// - **enable must not touch the loader.** Disable ran the module's onShutdown;
|
||||
// there is no onBoot re-dispatch, so flipping the record back would put a
|
||||
// module with closed sockets and cleared timers back on the nav.
|
||||
// - **purge must run before the directory is removed.** purge.sql lives inside
|
||||
// that directory. Reorder those two lines and the feature silently stops
|
||||
// working, with a 200 and no data deleted.
|
||||
//
|
||||
// Point the DB at a closed port BEFORE requiring anything that builds the pool.
|
||||
process.env.DB_HOST = '127.0.0.1'
|
||||
process.env.DB_PORT = '59999'
|
||||
|
||||
const { test, beforeEach, after } = require('node:test')
|
||||
const assert = require('node:assert/strict')
|
||||
|
||||
const ctrl = require('../src/router/v1/admin/modules.controller')
|
||||
const modules = require('../src/model/modules/modules.model')
|
||||
const activity = require('../src/model/activity/activity.model')
|
||||
const settings = require('../src/model/settings/settings.model')
|
||||
const loader = require('../src/modules/loader')
|
||||
const lifecycle = require('../src/modules/lifecycle')
|
||||
const install = require('../src/modules/install')
|
||||
const schema = require('../src/modules/schema')
|
||||
const db = require('../src/utils/db')
|
||||
|
||||
after(() => db.close())
|
||||
|
||||
function mockRes() {
|
||||
return {
|
||||
statusCode: 200,
|
||||
body: null,
|
||||
status(c) { this.statusCode = c; return this },
|
||||
json(b) { this.body = b; return this },
|
||||
}
|
||||
}
|
||||
|
||||
const req = (extra = {}) => ({
|
||||
user: { id: 1, username: 'admin' },
|
||||
params: {},
|
||||
query: {},
|
||||
body: {},
|
||||
...extra,
|
||||
})
|
||||
|
||||
// Everything the controller reaches for, replaced wholesale per test. Restored
|
||||
// from these originals rather than from a snapshot taken mid-run, so one test
|
||||
// leaking a stub cannot quietly become another test's fixture.
|
||||
const originals = {
|
||||
modules: { ...modules },
|
||||
activity: { log: activity.log },
|
||||
settings: { get: settings.get, set: settings.set },
|
||||
loader: { isLoaded: loader.isLoaded, list: loader.list },
|
||||
lifecycle: { stop: lifecycle.stop },
|
||||
install: {
|
||||
install: install.install,
|
||||
isInstalled: install.isInstalled,
|
||||
purgeFile: install.purgeFile,
|
||||
removeDir: install.removeDir,
|
||||
},
|
||||
schema: { runPurge: schema.runPurge },
|
||||
}
|
||||
|
||||
let logged
|
||||
|
||||
beforeEach(() => {
|
||||
Object.assign(modules, originals.modules)
|
||||
Object.assign(activity, originals.activity)
|
||||
Object.assign(settings, originals.settings)
|
||||
Object.assign(loader, originals.loader)
|
||||
Object.assign(lifecycle, originals.lifecycle)
|
||||
Object.assign(install, originals.install)
|
||||
Object.assign(schema, originals.schema)
|
||||
|
||||
logged = []
|
||||
activity.log = async (entry) => { logged.push(entry) }
|
||||
settings.get = async () => 'gitea.whitlocktech.com'
|
||||
loader.isLoaded = () => true
|
||||
loader.list = () => []
|
||||
})
|
||||
|
||||
// ── list ───────────────────────────────────────────────────────────────────
|
||||
|
||||
test('list reconciles the row, the loader and the volume without picking a winner', async () => {
|
||||
// The case §2.4 creates and decision 3 makes routine: the row says `enabled`
|
||||
// because the operator just switched it back on, the loader still says
|
||||
// `disabled` because its onShutdown has run and there is no way back without a
|
||||
// restart. Rendering either one alone would be a lie.
|
||||
modules.list = async () => [{
|
||||
id: 'uo', name: 'UO', version: '1.0.0', state: 'enabled',
|
||||
failureStage: null, failureReason: null, source: 'https://x/y.json', sha256: 'a'.repeat(64),
|
||||
installedAt: null, startedAt: null,
|
||||
}]
|
||||
loader.list = () => [{ id: 'uo', name: 'UO', version: '1.0.0', state: 'disabled', stage: null, reason: null, capabilities: ['shard'] }]
|
||||
install.isInstalled = () => true
|
||||
install.purgeFile = () => '/modules/uo/server/db/purge.sql'
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.list(req(), res)
|
||||
|
||||
const [m] = res.body.modules
|
||||
assert.equal(m.state, 'enabled', 'what the operator decided')
|
||||
assert.equal(m.liveState, 'disabled', 'what is actually answering')
|
||||
assert.equal(m.onVolume, true)
|
||||
assert.equal(m.canPurge, true)
|
||||
assert.deepEqual(m.capabilities, ['shard'])
|
||||
assert.deepEqual(res.body.sourceHosts, ['gitea.whitlocktech.com'])
|
||||
})
|
||||
|
||||
test('list includes a module on the volume that has no row yet', async () => {
|
||||
// A hand-placed directory before its first boot. §2.5 keeps that a supported
|
||||
// install, and its routes are already being served — a screen showing nothing
|
||||
// for it would be showing the wrong thing.
|
||||
modules.list = async () => []
|
||||
loader.list = () => [{ id: 'byhand', name: 'By Hand', version: '0.1.0', state: 'started', stage: null, reason: null, capabilities: [] }]
|
||||
install.isInstalled = () => true
|
||||
install.purgeFile = () => null
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.list(req(), res)
|
||||
|
||||
assert.equal(res.body.modules.length, 1)
|
||||
assert.equal(res.body.modules[0].id, 'byhand')
|
||||
assert.equal(res.body.modules[0].state, null, 'no row means no recorded state, not a guessed one')
|
||||
assert.equal(res.body.modules[0].liveState, 'started')
|
||||
})
|
||||
|
||||
test('list survives a process where the loader never scanned', async () => {
|
||||
loader.isLoaded = () => false
|
||||
loader.list = () => { throw new Error('modules.list() before modules.load()') }
|
||||
modules.list = async () => [{ id: 'uo', name: 'UO', version: '1', state: 'disabled' }]
|
||||
install.isInstalled = () => false
|
||||
install.purgeFile = () => null
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.list(req(), res)
|
||||
|
||||
assert.equal(res.statusCode, 200)
|
||||
assert.equal(res.body.modules[0].liveState, null)
|
||||
})
|
||||
|
||||
// ── install ────────────────────────────────────────────────────────────────
|
||||
|
||||
test('install records provenance and says a restart is needed', async () => {
|
||||
const calls = []
|
||||
install.install = async ({ url, hosts }) => {
|
||||
calls.push({ url, hosts })
|
||||
return { id: 'uo', name: 'UO', version: '1.0.0', sha256: 'b'.repeat(64), source: url, replaced: false }
|
||||
}
|
||||
modules.recordInstalled = async (row) => { calls.push(row); return { ...row, state: 'installed' } }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.create(req({ body: { url: 'https://gitea.whitlocktech.com/x/uo.json' } }), res)
|
||||
|
||||
assert.equal(res.statusCode, 201)
|
||||
assert.equal(res.body.restartRequired, true)
|
||||
assert.deepEqual(calls[0].hosts, ['gitea.whitlocktech.com'], 'the allowlist comes from the setting')
|
||||
// Provenance is written HERE and nowhere else — the boot reconcile records a
|
||||
// module with null source/sha256 and leaves what it is not given.
|
||||
assert.equal(calls[1].source, 'https://gitea.whitlocktech.com/x/uo.json')
|
||||
assert.equal(calls[1].sha256, 'b'.repeat(64))
|
||||
assert.equal(logged[0].action, 'module.install')
|
||||
})
|
||||
|
||||
test('an install refusal is reported to the operator, with its own status', async () => {
|
||||
install.install = async () => {
|
||||
const err = new Error('"evil.net" is not an allowed module source host')
|
||||
err.name = 'InstallError'
|
||||
err.status = 400
|
||||
throw err
|
||||
}
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.create(req({ body: { url: 'https://evil.net/x.json' } }), res)
|
||||
|
||||
assert.equal(res.statusCode, 400)
|
||||
// The message is the useful part: the operator pasted a URL and needs to know
|
||||
// what was wrong with what came back.
|
||||
assert.match(res.body.message, /not an allowed module source host/)
|
||||
assert.equal(logged.length, 0, 'a refused install is not an audit-log entry')
|
||||
})
|
||||
|
||||
test('an unreachable host is a 502, not a 400', async () => {
|
||||
install.install = async () => {
|
||||
const err = new Error('could not reach x: timeout')
|
||||
err.name = 'InstallError'
|
||||
err.status = 502
|
||||
throw err
|
||||
}
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.create(req({ body: { url: 'https://gitea.whitlocktech.com/x.json' } }), res)
|
||||
assert.equal(res.statusCode, 502)
|
||||
})
|
||||
|
||||
test('an unexpected failure is a 500 and does not leak its message', async () => {
|
||||
install.install = async () => { throw new Error('ENOENT /some/internal/path') }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.create(req({ body: { url: 'https://gitea.whitlocktech.com/x.json' } }), res)
|
||||
|
||||
assert.equal(res.statusCode, 500)
|
||||
assert.equal(res.body.message, 'Internal Server Error')
|
||||
})
|
||||
|
||||
// ── enable / disable ───────────────────────────────────────────────────────
|
||||
|
||||
test('enable moves the row and does NOT touch the loader', async () => {
|
||||
// The decision-3 invariant. Re-enabling cannot restart a module: its
|
||||
// onShutdown has run, and MODULE_API.md has never promised onBoot is safe to
|
||||
// run twice. Flipping the record would put it back on the nav with a
|
||||
// torn-down world behind it.
|
||||
let setStateCalled = false
|
||||
loader.setState = () => { setStateCalled = true }
|
||||
modules.enable = async (id) => ({ id, state: 'enabled' })
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.enable(req({ params: { id: 'uo' } }), res)
|
||||
|
||||
assert.equal(res.body.module.state, 'enabled')
|
||||
assert.equal(res.body.restartRequired, true)
|
||||
assert.equal(setStateCalled, false, 'enable must not move the in-memory record')
|
||||
assert.equal(logged[0].action, 'module.enable')
|
||||
loader.setState = originals.loader.setState
|
||||
})
|
||||
|
||||
test('enabling a module with no row is a 404', async () => {
|
||||
modules.enable = async () => null
|
||||
const res = mockRes()
|
||||
await ctrl.enable(req({ params: { id: 'ghost' } }), res)
|
||||
assert.equal(res.statusCode, 404)
|
||||
})
|
||||
|
||||
test('an illegal transition is a 409, not a 500', async () => {
|
||||
modules.enable = async () => {
|
||||
const err = new Error("module 'uo': cannot move from 'x' to 'enabled'")
|
||||
err.name = 'ModuleStateError'
|
||||
throw err
|
||||
}
|
||||
const res = mockRes()
|
||||
await ctrl.enable(req({ params: { id: 'uo' } }), res)
|
||||
assert.equal(res.statusCode, 409)
|
||||
})
|
||||
|
||||
test('disable stops the module and reports whether the hook ran', async () => {
|
||||
const calls = []
|
||||
modules.get = async (id) => ({ id, state: 'started' })
|
||||
lifecycle.stop = async (id) => { calls.push(id); return { stopped: true, error: null } }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.disable(req({ params: { id: 'uo' } }), res)
|
||||
|
||||
assert.deepEqual(calls, ['uo'])
|
||||
assert.equal(res.body.stopped, true)
|
||||
// No restart: this is the one action that takes effect immediately, and it is
|
||||
// the one an operator reaches for when something is going wrong.
|
||||
assert.equal(res.body.restartRequired, undefined)
|
||||
assert.equal(logged[0].action, 'module.disable')
|
||||
})
|
||||
|
||||
test('a shutdown hook that failed is reported rather than swallowed', async () => {
|
||||
modules.get = async (id) => ({ id, state: 'started' })
|
||||
lifecycle.stop = async () => ({ stopped: false, error: 'socket would not close' })
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.disable(req({ params: { id: 'uo' } }), res)
|
||||
|
||||
// It IS disabled either way; the operator should be told it did not close
|
||||
// cleanly while they still have the logs in front of them.
|
||||
assert.equal(res.statusCode, 200)
|
||||
assert.match(res.body.shutdownError, /socket would not close/)
|
||||
})
|
||||
|
||||
// ── uninstall and purge ────────────────────────────────────────────────────
|
||||
|
||||
test('uninstall purges BEFORE it removes the directory', async () => {
|
||||
// The ordering that makes decision 5 work at all: purge.sql is a file inside
|
||||
// the directory being deleted. Swap these two and the endpoint still answers
|
||||
// 200 and deletes nothing.
|
||||
const order = []
|
||||
modules.get = async (id) => ({ id, state: 'started' })
|
||||
install.isInstalled = () => true
|
||||
install.purgeFile = () => '/modules/uo/server/db/purge.sql'
|
||||
schema.runPurge = async () => { order.push('purge'); return 12 }
|
||||
lifecycle.stop = async () => { order.push('stop'); return { stopped: true, error: null } }
|
||||
install.removeDir = async () => { order.push('removeDir'); return true }
|
||||
modules.remove = async () => { order.push('removeRow') }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.remove(req({ params: { id: 'uo' }, query: { purge: 'true' } }), res)
|
||||
|
||||
assert.deepEqual(order, ['purge', 'stop', 'removeDir', 'removeRow'])
|
||||
assert.equal(res.body.purged, 12)
|
||||
assert.equal(res.body.restartRequired, true)
|
||||
assert.equal(logged[0].action, 'module.purge')
|
||||
})
|
||||
|
||||
test('a plain uninstall keeps the row and does not purge', async () => {
|
||||
const order = []
|
||||
modules.get = async (id) => ({ id, state: 'started' })
|
||||
install.isInstalled = () => true
|
||||
schema.runPurge = async () => { order.push('purge'); return 1 }
|
||||
lifecycle.stop = async () => { order.push('stop'); return { stopped: true, error: null } }
|
||||
install.removeDir = async () => { order.push('removeDir'); return true }
|
||||
modules.remove = async () => { order.push('removeRow') }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.remove(req({ params: { id: 'uo' } }), res)
|
||||
|
||||
// §2.5's default: the directory goes, the data stays, and the disabled row is
|
||||
// what keeps the retained data visible and the module reinstallable.
|
||||
assert.deepEqual(order, ['stop', 'removeDir'])
|
||||
assert.equal(res.body.purged, null)
|
||||
assert.equal(logged[0].action, 'module.uninstall')
|
||||
})
|
||||
|
||||
test('asking to purge a module that ships no purge.sql refuses instead of pretending', async () => {
|
||||
modules.get = async (id) => ({ id, state: 'started' })
|
||||
install.isInstalled = () => true
|
||||
install.purgeFile = () => null
|
||||
let removed = false
|
||||
install.removeDir = async () => { removed = true; return true }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.remove(req({ params: { id: 'uo' }, query: { purge: 'true' } }), res)
|
||||
|
||||
assert.equal(res.statusCode, 400)
|
||||
assert.match(res.body.message, /ships no purge.sql/)
|
||||
// Nothing happened. The operator asked for the module AND its data to go; the
|
||||
// data cannot go, so doing half of it silently would be the worst answer.
|
||||
assert.equal(removed, false)
|
||||
})
|
||||
|
||||
test('uninstalling something that is neither on the volume nor in a row is a 404', async () => {
|
||||
modules.get = async () => null
|
||||
install.isInstalled = () => false
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.remove(req({ params: { id: 'ghost' } }), res)
|
||||
assert.equal(res.statusCode, 404)
|
||||
})
|
||||
|
||||
test('standalone purge refuses while the module is still running', async () => {
|
||||
// Dropping the tables under a module that is still serving leaves it answering
|
||||
// out of a world that no longer exists. Disabling first is one click.
|
||||
modules.get = async (id) => ({ id, state: 'started' })
|
||||
let ran = false
|
||||
schema.runPurge = async () => { ran = true; return 1 }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.purge(req({ params: { id: 'uo' } }), res)
|
||||
|
||||
assert.equal(res.statusCode, 409)
|
||||
assert.match(res.body.message, /Disable this module before purging/)
|
||||
assert.equal(ran, false)
|
||||
})
|
||||
|
||||
test('standalone purge runs on a disabled module', async () => {
|
||||
modules.get = async (id) => ({ id, state: 'disabled' })
|
||||
install.purgeFile = () => '/modules/uo/server/db/purge.sql'
|
||||
schema.runPurge = async () => 7
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.purge(req({ params: { id: 'uo' } }), res)
|
||||
|
||||
assert.equal(res.body.purged, 7)
|
||||
assert.equal(logged[0].action, 'module.purge')
|
||||
})
|
||||
|
||||
// ── the allowlist ──────────────────────────────────────────────────────────
|
||||
|
||||
test('setSources stores a normalised list and audits the change', async () => {
|
||||
let stored = null
|
||||
settings.set = async (key, value) => { stored = { key, value } }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.setSources(req({ body: { hosts: 'Gitea.Example.com, releases.example.org' } }), res)
|
||||
|
||||
assert.deepEqual(res.body.sourceHosts, ['gitea.example.com', 'releases.example.org'])
|
||||
assert.equal(stored.key, ctrl.HOSTS_KEY)
|
||||
assert.equal(stored.value, 'gitea.example.com,releases.example.org')
|
||||
// Before AND after: this setting decides what code the site will execute, so
|
||||
// the audit entry has to say what it used to be.
|
||||
assert.equal(logged[0].action, 'module.sources')
|
||||
assert.deepEqual(logged[0].detail.before, ['gitea.whitlocktech.com'])
|
||||
})
|
||||
|
||||
test('setSources refuses anything that is not a bare hostname', async () => {
|
||||
let stored = false
|
||||
settings.set = async () => { stored = true }
|
||||
|
||||
for (const bad of ['https://x.com', 'x.com/path', 'x.com:8443', '*.x.com', 'x_y.com']) {
|
||||
const res = mockRes()
|
||||
// eslint-disable-next-line no-await-in-loop
|
||||
await ctrl.setSources(req({ body: { hosts: bad } }), res)
|
||||
assert.equal(res.statusCode, 400, `${bad} should be refused`)
|
||||
}
|
||||
assert.equal(stored, false)
|
||||
})
|
||||
|
||||
test('an empty allowlist is storable, and means no installs', async () => {
|
||||
// Not a wildcard, and not an error: "nothing may be installed" is a position
|
||||
// an operator is entitled to take.
|
||||
let stored = null
|
||||
settings.set = async (key, value) => { stored = value }
|
||||
|
||||
const res = mockRes()
|
||||
await ctrl.setSources(req({ body: { hosts: '' } }), res)
|
||||
|
||||
assert.equal(res.statusCode, 200)
|
||||
assert.deepEqual(res.body.sourceHosts, [])
|
||||
assert.equal(stored, '')
|
||||
})
|
||||
|
||||
// ── restart ────────────────────────────────────────────────────────────────
|
||||
|
||||
// These listen for the SIGTERM EVENT rather than stubbing `process.kill`, and
|
||||
// that is the whole point of them now.
|
||||
//
|
||||
// The first version of this route called `process.kill(process.pid, 'SIGTERM')`
|
||||
// and the first version of these tests stubbed `process.kill` and asserted it
|
||||
// had been called with SIGTERM. Both passed. Both were wrong: Windows has no
|
||||
// POSIX signals, and Node documents SIGTERM there as unconditional termination —
|
||||
// so on a Windows host the route killed the server outright, with no module
|
||||
// `onShutdown`, no pool close and no log flush. A stub of `process.kill` cannot
|
||||
// see that, because what it asserts is precisely the call whose MEANING differs
|
||||
// by platform.
|
||||
//
|
||||
// Asserting on the event closes the gap: it is what server.js's handler is
|
||||
// actually subscribed to, so a test passing here means the handler would run.
|
||||
function onceSigterm() {
|
||||
return new Promise((resolve, reject) => {
|
||||
const timer = setTimeout(() => {
|
||||
process.removeListener('SIGTERM', handler)
|
||||
reject(new Error('no SIGTERM was emitted within 1s'))
|
||||
}, 1000)
|
||||
function handler() {
|
||||
clearTimeout(timer)
|
||||
process.removeListener('SIGTERM', handler)
|
||||
resolve(true)
|
||||
}
|
||||
process.on('SIGTERM', handler)
|
||||
})
|
||||
}
|
||||
|
||||
test('restart answers first, then triggers the one graceful-shutdown path', async () => {
|
||||
const fired = onceSigterm()
|
||||
const res = mockRes()
|
||||
|
||||
ctrl.restart(req(), res)
|
||||
|
||||
// Answered synchronously: once the shutdown starts there is no listener left
|
||||
// to flush a response through, so the operator would be told nothing.
|
||||
assert.equal(res.statusCode, 202)
|
||||
assert.equal(res.body.restarting, true)
|
||||
|
||||
assert.equal(await fired, true)
|
||||
assert.equal(logged[0].action, 'module.restart')
|
||||
})
|
||||
|
||||
test('a failure to write the audit entry does not cancel the restart', async () => {
|
||||
activity.log = async () => { throw new Error('database is gone') }
|
||||
const fired = onceSigterm()
|
||||
|
||||
ctrl.restart(req(), mockRes())
|
||||
|
||||
assert.equal(await fired, true)
|
||||
})
|
||||
252
server/test/moduleArchive.test.js
Normal file
252
server/test/moduleArchive.test.js
Normal file
@@ -0,0 +1,252 @@
|
||||
// modules/archive.js — the hardened bundle extractor (MODULE_SYSTEM.md §2.7.2).
|
||||
//
|
||||
// Every rejection case is exercised against a REAL archive rather than a mocked
|
||||
// tar parser, because the thing being tested is what the parser reports for a
|
||||
// given sequence of bytes. A fake that returns `{type: 'SymbolicLink'}` proves
|
||||
// only that the `if` is spelled correctly.
|
||||
//
|
||||
// The hostile archives are written here as raw ustar headers instead of being
|
||||
// produced with `tar`, for two reasons that both bit during this slice:
|
||||
//
|
||||
// 1. `ln -s` needs a privilege Windows does not hand out by default, so a
|
||||
// symlink fixture built with the shell is a fixture that silently is not
|
||||
// one, and the test passes for the wrong reason on the machine most of this
|
||||
// work happens on.
|
||||
// 2. GNU tar will not emit `../escape` or `/etc/passwd` as a member name — it
|
||||
// strips them and tells you so. The archives worth defending against are
|
||||
// exactly the ones a cooperative archiver refuses to produce.
|
||||
//
|
||||
// A ustar header is 512 bytes of fixed-offset fields, so writing one is less
|
||||
// code than persuading a tool to misbehave.
|
||||
|
||||
const test = require('node:test')
|
||||
const assert = require('node:assert/strict')
|
||||
const fs = require('fs')
|
||||
const os = require('os')
|
||||
const path = require('path')
|
||||
const zlib = require('zlib')
|
||||
|
||||
const archive = require('../src/modules/archive')
|
||||
|
||||
// ── A minimal ustar writer ─────────────────────────────────────────────────
|
||||
|
||||
const BLOCK = 512
|
||||
|
||||
function octal(value, width) {
|
||||
// ustar numeric fields are NUL-terminated octal, right-aligned with zeros.
|
||||
return Number(value).toString(8).padStart(width - 1, '0') + '\0'
|
||||
}
|
||||
|
||||
/**
|
||||
* One 512-byte header plus its padded data.
|
||||
*
|
||||
* @param {object} entry
|
||||
* @param {string} entry.name member path
|
||||
* @param {string} [entry.type] '0' file · '5' dir · '2' symlink · '1' hardlink
|
||||
* · '3' char dev · '6' FIFO
|
||||
* @param {string} [entry.linkname] target, for the link types
|
||||
* @param {string} [entry.body] file contents
|
||||
* @param {number} [entry.size] declared size — defaults to the body's, and
|
||||
* may be set independently to build a header
|
||||
* that lies about its payload
|
||||
*/
|
||||
function member({ name, type = '0', linkname = '', body = '', size = null }) {
|
||||
const header = Buffer.alloc(BLOCK, 0)
|
||||
const data = Buffer.from(body, 'utf8')
|
||||
const declared = size === null ? data.length : size
|
||||
|
||||
header.write(name, 0, 100, 'utf8')
|
||||
header.write(octal(0o644, 8), 100, 8, 'ascii') // mode
|
||||
header.write(octal(0, 8), 108, 8, 'ascii') // uid
|
||||
header.write(octal(0, 8), 116, 8, 'ascii') // gid
|
||||
header.write(octal(declared, 12), 124, 12, 'ascii')
|
||||
header.write(octal(0, 12), 136, 12, 'ascii') // mtime
|
||||
header.write(' ', 148, 8, 'ascii') // checksum field is spaces while summing
|
||||
header.write(type, 156, 1, 'ascii')
|
||||
header.write(linkname, 157, 100, 'utf8')
|
||||
header.write('ustar\0', 257, 6, 'ascii')
|
||||
header.write('00', 263, 2, 'ascii')
|
||||
|
||||
let sum = 0
|
||||
for (const byte of header) sum += byte
|
||||
header.write(`${sum.toString(8).padStart(6, '0')}\0 `, 148, 8, 'ascii')
|
||||
|
||||
const padding = Buffer.alloc((BLOCK - (data.length % BLOCK)) % BLOCK, 0)
|
||||
return Buffer.concat([header, data, padding])
|
||||
}
|
||||
|
||||
/** Gzip a set of members into a .tar.gz on disk, and return its path. */
|
||||
function writeArchive(dir, filename, members) {
|
||||
const tarball = Buffer.concat([
|
||||
...members.map(member),
|
||||
Buffer.alloc(BLOCK * 2, 0), // two zero blocks end the archive
|
||||
])
|
||||
const file = path.join(dir, filename)
|
||||
fs.writeFileSync(file, zlib.gzipSync(tarball))
|
||||
return file
|
||||
}
|
||||
|
||||
/** A well-formed bundle: one top-level directory, ordinary files inside it. */
|
||||
function goodMembers(root = 'module-uo-1.0.0') {
|
||||
return [
|
||||
{ name: `${root}/`, type: '5' },
|
||||
{ name: `${root}/module.json`, body: '{"id":"uo","name":"UO","version":"1.0.0"}' },
|
||||
{ name: `${root}/server/`, type: '5' },
|
||||
{ name: `${root}/server/index.js`, body: 'module.exports = () => {}\n' },
|
||||
]
|
||||
}
|
||||
|
||||
// One scratch directory for the whole file, removed at the end.
|
||||
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'rg-archive-'))
|
||||
test.after(() => fs.rmSync(tmp, { recursive: true, force: true }))
|
||||
|
||||
/** Assert that inspecting `members` fails, and that the message says why. */
|
||||
async function rejects(name, members, matcher) {
|
||||
const file = writeArchive(tmp, `${name}.tar.gz`, members)
|
||||
await assert.rejects(
|
||||
() => archive.inspect(file),
|
||||
(err) => {
|
||||
assert.equal(err.name, 'ArchiveError', `expected an ArchiveError, got ${err.name}: ${err.message}`)
|
||||
assert.match(err.message, matcher)
|
||||
return true
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
// ── What a good bundle does ────────────────────────────────────────────────
|
||||
|
||||
test('inspect accepts a well-formed bundle and reports its single root', async () => {
|
||||
const file = writeArchive(tmp, 'good.tar.gz', goodMembers())
|
||||
const stats = await archive.inspect(file)
|
||||
|
||||
assert.equal(stats.root, 'module-uo-1.0.0')
|
||||
assert.equal(stats.entries, 4)
|
||||
assert.ok(stats.bytes > 0)
|
||||
})
|
||||
|
||||
test('extract strips the top level, so the bundle lands as the module id', async () => {
|
||||
// The wrapper directory is the publisher's naming (module-uo's release
|
||||
// workflow packs `module-uo-<version>/`); the directory it lands in is core's,
|
||||
// and has to be the id the loader scans for.
|
||||
const file = writeArchive(tmp, 'strip.tar.gz', goodMembers())
|
||||
const dest = path.join(tmp, 'unpacked-uo')
|
||||
|
||||
await archive.unpack(file, dest)
|
||||
|
||||
assert.ok(fs.existsSync(path.join(dest, 'module.json')), 'module.json should be at the root of the destination')
|
||||
assert.ok(fs.existsSync(path.join(dest, 'server', 'index.js')))
|
||||
assert.ok(!fs.existsSync(path.join(dest, 'module-uo-1.0.0')), 'the wrapper directory should not survive')
|
||||
})
|
||||
|
||||
test('extract refuses a destination that already exists', async () => {
|
||||
const file = writeArchive(tmp, 'exists.tar.gz', goodMembers())
|
||||
const dest = path.join(tmp, 'already-there')
|
||||
fs.mkdirSync(dest)
|
||||
|
||||
await assert.rejects(() => archive.extract(file, dest), /already exists/)
|
||||
})
|
||||
|
||||
// ── What a hostile bundle does ─────────────────────────────────────────────
|
||||
|
||||
test('an absolute member path is refused', async () => {
|
||||
await rejects('absolute', [
|
||||
{ name: 'mod/', type: '5' },
|
||||
{ name: '/etc/cron.d/pwned', body: '* * * * * root sh\n' },
|
||||
], /absolute/)
|
||||
})
|
||||
|
||||
test('a drive-absolute member path is refused', async () => {
|
||||
// Its own node-tar advisory, and invisible to a leading-slash check.
|
||||
await rejects('drive', [
|
||||
{ name: 'mod/', type: '5' },
|
||||
{ name: 'C:\\Windows\\Temp\\pwned', body: 'x' },
|
||||
], /backslash|drive-absolute/)
|
||||
})
|
||||
|
||||
test('an upward-escaping member path is refused', async () => {
|
||||
await rejects('escape', [
|
||||
{ name: 'mod/', type: '5' },
|
||||
{ name: 'mod/../../../etc/passwd', body: 'root::0:0\n' },
|
||||
], /escapes upward/)
|
||||
})
|
||||
|
||||
test('a symlink member is refused, and the message names the type', async () => {
|
||||
await rejects('symlink', [
|
||||
{ name: 'mod/', type: '5' },
|
||||
{ name: 'mod/passwd', type: '2', linkname: '/etc/passwd' },
|
||||
], /SymbolicLink/)
|
||||
})
|
||||
|
||||
test('a hardlink member is refused', async () => {
|
||||
// The single most-published node-tar escape primitive. Refusing the type
|
||||
// outright is what keeps this file from depending on the library getting
|
||||
// hardlink containment right.
|
||||
await rejects('hardlink', [
|
||||
{ name: 'mod/', type: '5' },
|
||||
{ name: 'mod/shadow', type: '1', linkname: '../../../etc/shadow' },
|
||||
], /Link/)
|
||||
})
|
||||
|
||||
test('a device node is refused', async () => {
|
||||
await rejects('device', [
|
||||
{ name: 'mod/', type: '5' },
|
||||
{ name: 'mod/zero', type: '3', linkname: '' },
|
||||
], /may only contain files and directories/)
|
||||
})
|
||||
|
||||
test('a FIFO is refused', async () => {
|
||||
await rejects('fifo', [
|
||||
{ name: 'mod/', type: '5' },
|
||||
{ name: 'mod/pipe', type: '6' },
|
||||
], /may only contain files and directories/)
|
||||
})
|
||||
|
||||
test('two top-level directories are refused', async () => {
|
||||
await rejects('two-roots', [
|
||||
{ name: 'mod-a/', type: '5' },
|
||||
{ name: 'mod-a/module.json', body: '{}' },
|
||||
{ name: 'mod-b/', type: '5' },
|
||||
{ name: 'mod-b/module.json', body: '{}' },
|
||||
], /exactly one top-level directory, found 2/)
|
||||
})
|
||||
|
||||
test('a bundle whose declared sizes exceed the cap is refused before it is read to the end', async () => {
|
||||
// The header lies: it declares a gigabyte and carries nothing. That is the
|
||||
// decompression-bomb shape, and the point is that inspect() decides on the
|
||||
// DECLARED size without ever materialising the payload.
|
||||
await rejects('bomb', [
|
||||
{ name: 'mod/', type: '5' },
|
||||
{ name: 'mod/big', size: archive.MAX_BYTES + 1, body: '' },
|
||||
], /unpacks to more than/)
|
||||
})
|
||||
|
||||
test('an empty archive is refused', async () => {
|
||||
await rejects('empty', [], /empty/)
|
||||
})
|
||||
|
||||
// ── The path check on its own ──────────────────────────────────────────────
|
||||
//
|
||||
// pathProblem is exported so the cases that are awkward to express as archive
|
||||
// bytes can still be asserted directly.
|
||||
|
||||
test('pathProblem accepts ordinary bundle paths', () => {
|
||||
for (const ok of ['mod/module.json', 'mod/server/router/x.js', 'mod/a.b-c_d/e.js']) {
|
||||
assert.equal(archive.pathProblem(ok), null, `${ok} should be accepted`)
|
||||
}
|
||||
})
|
||||
|
||||
test('pathProblem rejects the escape shapes', () => {
|
||||
assert.match(archive.pathProblem('/etc/passwd'), /absolute/)
|
||||
assert.match(archive.pathProblem('C:/Windows/x'), /drive-absolute/)
|
||||
assert.match(archive.pathProblem('mod/../../x'), /escapes upward/)
|
||||
assert.match(archive.pathProblem('..'), /escapes upward/)
|
||||
assert.match(archive.pathProblem('mod\\x'), /backslash/)
|
||||
assert.match(archive.pathProblem('mod/\0/x'), /NUL byte/)
|
||||
})
|
||||
|
||||
test('pathProblem does not reject a filename that merely contains two dots', () => {
|
||||
// `..` is a SEGMENT, not a substring — a file called `version..js` is fine,
|
||||
// and a check written with `includes('..')` would refuse it.
|
||||
assert.equal(archive.pathProblem('mod/version..js'), null)
|
||||
assert.equal(archive.pathProblem('mod/..hidden'), null)
|
||||
})
|
||||
205
server/test/moduleDocsSpec.test.js
Normal file
205
server/test/moduleDocsSpec.test.js
Normal file
@@ -0,0 +1,205 @@
|
||||
// ── /api/docs.json, with a module installed ────────────────────────────────
|
||||
//
|
||||
// The request-time half of docs/website/MODULE_API.md §6.1's settled decision.
|
||||
// `swagger-output.json` is core's own routes and cannot be anything else: it is
|
||||
// generated on a developer's machine and committed, so it has to come out the
|
||||
// same regardless of what they had checked out, and a module arrives on the
|
||||
// volume long after the image was built. The module's routes therefore reach the
|
||||
// document only here, from the fragment it ships (§2.8, §6.1a).
|
||||
//
|
||||
// This boots the REAL app against a throwaway module directory, because the two
|
||||
// things worth locking are properties of the served document rather than of the
|
||||
// merge helper: that a started module's paths are IN it, and that core wins.
|
||||
//
|
||||
// The modules directory is written and MODULES_DIR set BEFORE app.js is required
|
||||
// — the scan is synchronous and happens during that require.
|
||||
|
||||
process.env.DB_HOST = '127.0.0.1'
|
||||
process.env.DB_PORT = '59999'
|
||||
|
||||
const fs = require('fs')
|
||||
const os = require('os')
|
||||
const path = require('path')
|
||||
|
||||
const { test, before, after } = require('node:test')
|
||||
const assert = require('node:assert/strict')
|
||||
|
||||
const tmpRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'rg-module-docs-'))
|
||||
const moduleDir = path.join(tmpRoot, 'atlas')
|
||||
fs.mkdirSync(path.join(moduleDir, 'client', 'dist'), { recursive: true })
|
||||
fs.writeFileSync(path.join(moduleDir, 'client', 'dist', 'entry.js'), 'export const hello = 1\n')
|
||||
fs.writeFileSync(
|
||||
path.join(moduleDir, 'module.json'),
|
||||
JSON.stringify({
|
||||
id: 'atlas',
|
||||
name: 'Atlas',
|
||||
version: '1.0.0',
|
||||
coreApi: '^1.0.0',
|
||||
client: { entry: 'client/dist/entry.js' },
|
||||
}),
|
||||
)
|
||||
fs.writeFileSync(
|
||||
path.join(moduleDir, 'swagger-fragment.json'),
|
||||
JSON.stringify({
|
||||
paths: {
|
||||
'/api/v1/public/atlas/creatures': { get: { tags: ['Public · Atlas'], summary: 'List creatures' } },
|
||||
// The collision case, and the one that matters: a module trying to
|
||||
// redefine a path core already declares. Core wins and the module's
|
||||
// version is dropped (§6.1a) — a module cannot rewrite core's docs.
|
||||
'/api/v1/public/settings': { get: { summary: 'MODULE OVERRIDE' } },
|
||||
},
|
||||
tags: [{ name: 'Public · Atlas', description: 'from the module' }],
|
||||
components: {
|
||||
schemas: {
|
||||
AtlasCreature: { type: 'object' },
|
||||
// Same shape of collision, one section down.
|
||||
Error: { type: 'string', description: 'MODULE OVERRIDE' },
|
||||
},
|
||||
},
|
||||
}),
|
||||
)
|
||||
process.env.MODULES_DIR = tmpRoot
|
||||
|
||||
/* eslint-disable global-require */
|
||||
const app = require('../src/app')
|
||||
const loader = require('../src/modules/loader')
|
||||
const db = require('../src/utils/db')
|
||||
const coreSpec = require('../swagger/swagger-output.json')
|
||||
const { docsSpec, reset } = require('../swagger/docsSpec')
|
||||
/* eslint-enable global-require */
|
||||
|
||||
let server
|
||||
let base
|
||||
|
||||
before(async () => {
|
||||
server = await new Promise((resolve) => {
|
||||
const s = app.listen(0, '127.0.0.1', () => resolve(s))
|
||||
})
|
||||
base = `http://127.0.0.1:${server.address().port}`
|
||||
})
|
||||
|
||||
after(async () => {
|
||||
server.closeAllConnections()
|
||||
await new Promise((resolve) => server.close(resolve))
|
||||
await db.close()
|
||||
fs.rmSync(tmpRoot, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
const fetchSpec = async () => {
|
||||
const res = await fetch(`${base}/api/docs.json`)
|
||||
assert.equal(res.status, 200)
|
||||
return res.json()
|
||||
}
|
||||
|
||||
test('a module that is not started contributes nothing', async () => {
|
||||
// It is `registered` here: loaded cleanly, onBoot not yet dispatched. Its
|
||||
// routes answer 503 in that state, so documenting them would send a client
|
||||
// somewhere it cannot go — the same reason the client chunk's <script> tag is
|
||||
// withheld until `started`.
|
||||
reset()
|
||||
const spec = await fetchSpec()
|
||||
assert.equal(spec.paths['/api/v1/public/atlas/creatures'], undefined)
|
||||
})
|
||||
|
||||
test('a started module\'s paths, tags and schemas are in the served document', async () => {
|
||||
loader.setState('atlas', 'started')
|
||||
const spec = await fetchSpec()
|
||||
|
||||
assert.equal(spec.paths['/api/v1/public/atlas/creatures'].get.summary, 'List creatures')
|
||||
assert.ok(spec.tags.some((t) => t.name === 'Public · Atlas'))
|
||||
assert.deepEqual(spec.components.schemas.AtlasCreature, { type: 'object' })
|
||||
})
|
||||
|
||||
test('core wins every collision, in every section', async () => {
|
||||
loader.setState('atlas', 'started')
|
||||
const spec = await fetchSpec()
|
||||
|
||||
assert.notEqual(spec.paths['/api/v1/public/settings'].get.summary, 'MODULE OVERRIDE')
|
||||
assert.notEqual(spec.components.schemas.Error.description, 'MODULE OVERRIDE')
|
||||
assert.deepEqual(spec.components.schemas.Error, coreSpec.components.schemas.Error)
|
||||
})
|
||||
|
||||
test('the committed spec is never mutated by the merge', async () => {
|
||||
// `swagger-output.json` is a require()d JSON module, so one in-place merge
|
||||
// would be permanent for the life of the process AND cumulative across
|
||||
// rebuilds — a module's paths surviving its own uninstall.
|
||||
loader.setState('atlas', 'started')
|
||||
await fetchSpec()
|
||||
assert.equal(coreSpec.paths['/api/v1/public/atlas/creatures'], undefined)
|
||||
assert.equal(coreSpec.components.schemas.AtlasCreature, undefined)
|
||||
})
|
||||
|
||||
test('a state change rebuilds the document rather than serving the cached one', async () => {
|
||||
loader.setState('atlas', 'started')
|
||||
assert.ok((await fetchSpec()).paths['/api/v1/public/atlas/creatures'])
|
||||
|
||||
loader.setState('atlas', 'disabled')
|
||||
assert.equal((await fetchSpec()).paths['/api/v1/public/atlas/creatures'], undefined)
|
||||
|
||||
loader.setState('atlas', 'started')
|
||||
assert.ok((await fetchSpec()).paths['/api/v1/public/atlas/creatures'])
|
||||
})
|
||||
|
||||
test('an unreadable fragment costs that module its paths and nothing else', async () => {
|
||||
// §4.4's bargain, applied here: one module's failure is never the site's. A
|
||||
// docs page that 500s is strictly worse than one missing a module's routes.
|
||||
loader.setState('atlas', 'started')
|
||||
const file = path.join(moduleDir, 'swagger-fragment.json')
|
||||
const good = fs.readFileSync(file, 'utf8')
|
||||
fs.writeFileSync(file, 'not json {')
|
||||
try {
|
||||
reset()
|
||||
const spec = await fetchSpec()
|
||||
assert.equal(spec.paths['/api/v1/public/atlas/creatures'], undefined)
|
||||
assert.ok(spec.paths['/api/v1/public/settings'], 'core\'s own paths must survive')
|
||||
} finally {
|
||||
fs.writeFileSync(file, good)
|
||||
reset()
|
||||
}
|
||||
})
|
||||
|
||||
test('a module with no fragment at all is simply absent', async () => {
|
||||
// Registering routes without documenting them is checked in the MODULE's CI,
|
||||
// where the routes are known. Core cannot tell a module with no routes from
|
||||
// one that forgot, so it does not guess.
|
||||
loader.setState('atlas', 'started')
|
||||
const file = path.join(moduleDir, 'swagger-fragment.json')
|
||||
const good = fs.readFileSync(file, 'utf8')
|
||||
fs.rmSync(file)
|
||||
try {
|
||||
reset()
|
||||
const spec = await fetchSpec()
|
||||
assert.equal(spec.paths['/api/v1/public/atlas/creatures'], undefined)
|
||||
assert.ok(Object.keys(spec.paths).length > 100)
|
||||
} finally {
|
||||
fs.writeFileSync(file, good)
|
||||
reset()
|
||||
}
|
||||
})
|
||||
|
||||
test('before the loader has scanned, core\'s own spec is the answer', () => {
|
||||
// §7.6 makes every other accessor THROW when asked before modules.load(), so a
|
||||
// mis-ordered boot cannot be mistaken for an empty install. A request handler
|
||||
// is the exception: 500ing the docs page over it would be the wrong trade, and
|
||||
// core's routes are the honest answer to "what is documented" at that point.
|
||||
reset()
|
||||
const stub = { isLoaded: () => false }
|
||||
const original = Object.getOwnPropertyDescriptor(loader, 'isLoaded')
|
||||
Object.defineProperty(loader, 'isLoaded', { value: stub.isLoaded, configurable: true })
|
||||
try {
|
||||
assert.equal(docsSpec(coreSpec), coreSpec)
|
||||
} finally {
|
||||
Object.defineProperty(loader, 'isLoaded', original)
|
||||
reset()
|
||||
}
|
||||
})
|
||||
|
||||
test('the interactive UI is rebuilt per request, not bound to boot\'s document', async () => {
|
||||
// Bound once at require time, /api/docs would show core's routes for the life
|
||||
// of the process while /api/docs.json showed the merged set.
|
||||
loader.setState('atlas', 'started')
|
||||
reset()
|
||||
const withModule = await fetch(`${base}/api/docs/`)
|
||||
assert.equal(withModule.status, 200)
|
||||
assert.match(await withModule.text(), /swagger/i)
|
||||
})
|
||||
448
server/test/moduleInstall.test.js
Normal file
448
server/test/moduleInstall.test.js
Normal file
@@ -0,0 +1,448 @@
|
||||
// modules/install.js — fetching, verifying and placing a module bundle.
|
||||
//
|
||||
// Phase 4, slice 1 of MODULE_SYSTEM.md §2.7.2. The rules under test are all
|
||||
// refusals, and each one is a step an attacker would otherwise walk through:
|
||||
// a non-https URL, a host that is not allowed, a REDIRECT to a host that is not
|
||||
// allowed, a body larger than declared, a hash that does not match, an archive
|
||||
// that does not agree with the manifest about what it is.
|
||||
//
|
||||
// `fetchImpl` is injected rather than a TLS server being stood up, for the same
|
||||
// reason `replayFragments` takes a `query`: what is being tested is what this
|
||||
// file decides about a response, and a real server would mostly test Node's
|
||||
// certificate handling. The responses below are real `Response` objects with
|
||||
// real bodies, so the streaming, the hashing and the byte cap are exercised for
|
||||
// real — only the transport is stubbed.
|
||||
process.env.DB_HOST = '127.0.0.1'
|
||||
process.env.DB_PORT = '59999'
|
||||
|
||||
const crypto = require('crypto')
|
||||
const fs = require('fs')
|
||||
const os = require('os')
|
||||
const path = require('path')
|
||||
const zlib = require('zlib')
|
||||
|
||||
const { test, beforeEach, after } = require('node:test')
|
||||
const assert = require('node:assert/strict')
|
||||
|
||||
const db = require('../src/utils/db')
|
||||
|
||||
after(() => db.close())
|
||||
|
||||
const HOSTS = ['releases.example.com']
|
||||
const MANIFEST_URL = 'https://releases.example.com/mod/uo-1.0.0.json'
|
||||
|
||||
let tmpRoot
|
||||
let install
|
||||
|
||||
/**
|
||||
* A fresh install.js bound to a fresh modules directory.
|
||||
*
|
||||
* install.js reads `loader.dir()`, which is resolved once at require time from
|
||||
* MODULES_DIR — so both have to be re-required per test, exactly as
|
||||
* moduleLifecycle.test.js does.
|
||||
*/
|
||||
function freshInstall(dir) {
|
||||
process.env.MODULES_DIR = dir
|
||||
delete require.cache[require.resolve('../src/modules/loader')]
|
||||
delete require.cache[require.resolve('../src/modules/install')]
|
||||
// eslint-disable-next-line global-require
|
||||
return require('../src/modules/install')
|
||||
}
|
||||
|
||||
// ── Building a bundle to serve ─────────────────────────────────────────────
|
||||
|
||||
const BLOCK = 512
|
||||
|
||||
function octal(value, width) {
|
||||
return Number(value).toString(8).padStart(width - 1, '0') + '\0'
|
||||
}
|
||||
|
||||
function member({ name, type = '0', body = '' }) {
|
||||
const header = Buffer.alloc(BLOCK, 0)
|
||||
const data = Buffer.from(body, 'utf8')
|
||||
header.write(name, 0, 100, 'utf8')
|
||||
header.write(octal(0o644, 8), 100, 8, 'ascii')
|
||||
header.write(octal(0, 8), 108, 8, 'ascii')
|
||||
header.write(octal(0, 8), 116, 8, 'ascii')
|
||||
header.write(octal(data.length, 12), 124, 12, 'ascii')
|
||||
header.write(octal(0, 12), 136, 12, 'ascii')
|
||||
header.write(' ', 148, 8, 'ascii')
|
||||
header.write(type, 156, 1, 'ascii')
|
||||
header.write('ustar\0', 257, 6, 'ascii')
|
||||
header.write('00', 263, 2, 'ascii')
|
||||
let sum = 0
|
||||
for (const byte of header) sum += byte
|
||||
header.write(`${sum.toString(8).padStart(6, '0')}\0 `, 148, 8, 'ascii')
|
||||
const padding = Buffer.alloc((BLOCK - (data.length % BLOCK)) % BLOCK, 0)
|
||||
return Buffer.concat([header, data, padding])
|
||||
}
|
||||
|
||||
/** A bundle tarball whose root directory is named the way a release names it. */
|
||||
function bundle({ id = 'uo', version = '1.0.0', extra = [] } = {}) {
|
||||
const root = `module-${id}-${version}`
|
||||
return zlib.gzipSync(Buffer.concat([
|
||||
member({ name: `${root}/`, type: '5' }),
|
||||
member({
|
||||
name: `${root}/module.json`,
|
||||
body: JSON.stringify({ id, name: 'Ultima Online', version, coreApi: '^1.0.0', server: 'server/index.js' }),
|
||||
}),
|
||||
member({ name: `${root}/server/`, type: '5' }),
|
||||
member({ name: `${root}/server/index.js`, body: 'module.exports = () => {}\n' }),
|
||||
...extra.map(member),
|
||||
Buffer.alloc(BLOCK * 2, 0),
|
||||
]))
|
||||
}
|
||||
|
||||
const sha256 = (buf) => crypto.createHash('sha256').update(buf).digest('hex')
|
||||
|
||||
/**
|
||||
* A fake transport serving one manifest and one artifact.
|
||||
*
|
||||
* `routes` maps an absolute URL to either a Buffer/string body or
|
||||
* `{ status, location }` for a redirect, so a test can describe exactly what the
|
||||
* remote host does without describing how it does it.
|
||||
*/
|
||||
function fakeFetch(routes) {
|
||||
const seen = []
|
||||
const impl = async (url) => {
|
||||
const href = String(url)
|
||||
seen.push(href)
|
||||
const route = routes[href]
|
||||
if (route === undefined) return new Response('not found', { status: 404 })
|
||||
if (route && route.status) {
|
||||
return new Response(null, {
|
||||
status: route.status,
|
||||
headers: route.location ? { location: route.location } : {},
|
||||
})
|
||||
}
|
||||
return new Response(route, { status: 200 })
|
||||
}
|
||||
impl.seen = seen
|
||||
return impl
|
||||
}
|
||||
|
||||
/** The manifest a release publishes, with whatever a test wants to change. */
|
||||
function manifestFor(tarball, overrides = {}) {
|
||||
return JSON.stringify({
|
||||
schema: 1,
|
||||
id: 'uo',
|
||||
name: 'Ultima Online',
|
||||
version: '1.0.0',
|
||||
coreApi: '^1.0.0',
|
||||
artifact: 'uo-1.0.0.tar.gz',
|
||||
url: 'https://releases.example.com/mod/uo-1.0.0.tar.gz',
|
||||
sha256: sha256(tarball),
|
||||
size: tarball.length,
|
||||
...overrides,
|
||||
})
|
||||
}
|
||||
|
||||
/** The happy-path pair: a valid manifest and the artifact it describes. */
|
||||
function goodRoutes(options = {}) {
|
||||
const tarball = bundle(options)
|
||||
return {
|
||||
tarball,
|
||||
routes: {
|
||||
[MANIFEST_URL]: manifestFor(tarball, options.manifest || {}),
|
||||
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
tmpRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'rg-install-'))
|
||||
install = freshInstall(tmpRoot)
|
||||
})
|
||||
|
||||
// ── The allowlist ──────────────────────────────────────────────────────────
|
||||
|
||||
test('parseHosts accepts comma and whitespace separation, and lower-cases', () => {
|
||||
assert.deepEqual(install.parseHosts('a.com, B.com\nc.com'), ['a.com', 'b.com', 'c.com'])
|
||||
assert.deepEqual(install.parseHosts(''), [])
|
||||
assert.deepEqual(install.parseHosts(null), [])
|
||||
})
|
||||
|
||||
test('an empty allowlist forbids everything rather than allowing everything', () => {
|
||||
// The direction matters: a setting someone blanks by accident must stop
|
||||
// installs, not open the door to every host on the internet.
|
||||
assert.throws(() => install.checkUrl('https://releases.example.com/x.json', []), /no module source hosts are allowed/)
|
||||
})
|
||||
|
||||
test('only https may be installed from', () => {
|
||||
// The sha256 is no help against a plaintext fetch: whoever can rewrite the
|
||||
// artifact in flight can rewrite the manifest that declares its hash.
|
||||
assert.throws(() => install.checkUrl('http://releases.example.com/x.json', HOSTS), /only https/)
|
||||
assert.throws(() => install.checkUrl('file:///etc/passwd', HOSTS), /only https/)
|
||||
})
|
||||
|
||||
test('a host that is not on the allowlist is refused, and the message says which are', () => {
|
||||
assert.throws(
|
||||
() => install.checkUrl('https://evil.example.net/x.json', HOSTS),
|
||||
/"evil.example.net" is not an allowed module source host \(allowed: releases.example.com\)/,
|
||||
)
|
||||
})
|
||||
|
||||
test('the allowlist is matched on the host, not on a substring of the URL', () => {
|
||||
// `https://evil.com/?x=releases.example.com` must not pass, and neither must a
|
||||
// subdomain nobody listed.
|
||||
assert.throws(() => install.checkUrl('https://evil.com/?releases.example.com', HOSTS), /not an allowed/)
|
||||
assert.throws(() => install.checkUrl('https://sub.releases.example.com/x', HOSTS), /not an allowed/)
|
||||
assert.throws(() => install.checkUrl('https://releases.example.com.evil.net/x', HOSTS), /not an allowed/)
|
||||
})
|
||||
|
||||
// ── Redirects ──────────────────────────────────────────────────────────────
|
||||
|
||||
test('a redirect is followed, and re-checked against the allowlist', async () => {
|
||||
const { tarball } = goodRoutes()
|
||||
const impl = fakeFetch({
|
||||
[MANIFEST_URL]: { status: 302, location: 'https://releases.example.com/cdn/uo-1.0.0.json' },
|
||||
'https://releases.example.com/cdn/uo-1.0.0.json': manifestFor(tarball),
|
||||
})
|
||||
|
||||
const manifest = await install.fetchManifest(MANIFEST_URL, HOSTS, impl)
|
||||
|
||||
assert.equal(manifest.id, 'uo')
|
||||
assert.deepEqual(impl.seen, [MANIFEST_URL, 'https://releases.example.com/cdn/uo-1.0.0.json'])
|
||||
})
|
||||
|
||||
test('a redirect to a host that is not allowed is refused', async () => {
|
||||
// The hole the allowlist exists to close. `fetch`'s own redirect following
|
||||
// would check the first hop and then go wherever it was pointed — which is
|
||||
// why get() follows them by hand.
|
||||
const impl = fakeFetch({
|
||||
[MANIFEST_URL]: { status: 302, location: 'http://169.254.169.254/latest/meta-data/' },
|
||||
})
|
||||
|
||||
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /only https/)
|
||||
})
|
||||
|
||||
test('a redirect loop is bounded rather than followed forever', async () => {
|
||||
const impl = fakeFetch({
|
||||
[MANIFEST_URL]: { status: 302, location: MANIFEST_URL },
|
||||
})
|
||||
|
||||
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /too many redirects/)
|
||||
})
|
||||
|
||||
// ── The install manifest ───────────────────────────────────────────────────
|
||||
|
||||
test('a manifest that is not JSON is refused with a readable reason', async () => {
|
||||
const impl = fakeFetch({ [MANIFEST_URL]: 'this is not json' })
|
||||
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /not valid JSON/)
|
||||
})
|
||||
|
||||
test('a manifest with an invalid module id is refused', async () => {
|
||||
// The loader would refuse to scan a directory called `../etc`, but the point
|
||||
// is that one is never created: the id becomes a path segment.
|
||||
for (const id of ['../etc', 'UO', '', 'x', 'a'.repeat(40)]) {
|
||||
const impl = fakeFetch({ [MANIFEST_URL]: JSON.stringify({ id, name: 'n', version: '1', sha256: 'a'.repeat(64) }) })
|
||||
// eslint-disable-next-line no-await-in-loop
|
||||
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /invalid module id/, `id ${JSON.stringify(id)}`)
|
||||
}
|
||||
})
|
||||
|
||||
test('a manifest with no usable sha256 is refused', async () => {
|
||||
const impl = fakeFetch({
|
||||
[MANIFEST_URL]: JSON.stringify({ id: 'uo', name: 'n', version: '1.0.0', sha256: 'not-a-hash' }),
|
||||
})
|
||||
await assert.rejects(() => install.fetchManifest(MANIFEST_URL, HOSTS, impl), /no valid sha256/)
|
||||
})
|
||||
|
||||
test('the artifact URL is resolved relative to the manifest when it carries no absolute one', async () => {
|
||||
const { tarball } = goodRoutes()
|
||||
const impl = fakeFetch({ [MANIFEST_URL]: manifestFor(tarball, { url: undefined }) })
|
||||
|
||||
const manifest = await install.fetchManifest(MANIFEST_URL, HOSTS, impl)
|
||||
|
||||
// Release assets sit beside their manifest, so a manifest that travelled
|
||||
// without its absolute URL still resolves to the right place.
|
||||
assert.equal(manifest.artifactUrl, 'https://releases.example.com/mod/uo-1.0.0.tar.gz')
|
||||
})
|
||||
|
||||
// ── The install ────────────────────────────────────────────────────────────
|
||||
|
||||
test('a good bundle installs into modules/<id>, stripped of its wrapper', async () => {
|
||||
const { routes, tarball } = goodRoutes()
|
||||
|
||||
const result = await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) })
|
||||
|
||||
assert.equal(result.id, 'uo')
|
||||
assert.equal(result.version, '1.0.0')
|
||||
assert.equal(result.sha256, sha256(tarball))
|
||||
assert.equal(result.source, MANIFEST_URL)
|
||||
assert.equal(result.replaced, false)
|
||||
|
||||
// The directory is named for the module id, not for the archive's root — the
|
||||
// loader scans for the former and the publisher chose the latter.
|
||||
const dir = path.join(tmpRoot, 'uo')
|
||||
assert.ok(fs.existsSync(path.join(dir, 'module.json')))
|
||||
assert.ok(fs.existsSync(path.join(dir, 'server', 'index.js')))
|
||||
assert.ok(!fs.existsSync(path.join(tmpRoot, 'module-uo-1.0.0')))
|
||||
})
|
||||
|
||||
test('a hash that does not match refuses the install and writes nothing', async () => {
|
||||
const { routes } = goodRoutes()
|
||||
// The bytes are fine; the manifest lies about them. Which is the same thing an
|
||||
// artifact swapped after publication looks like.
|
||||
routes[MANIFEST_URL] = manifestFor(Buffer.from('different'), {})
|
||||
|
||||
await assert.rejects(
|
||||
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
|
||||
/does not match the sha256/,
|
||||
)
|
||||
assert.deepEqual(fs.readdirSync(tmpRoot), [], 'nothing may be left on the volume')
|
||||
})
|
||||
|
||||
test('a bundle whose module.json disagrees with the manifest is refused', async () => {
|
||||
// A manifest promising `uo` and delivering something else would otherwise be
|
||||
// installed into `modules/uo/` under a name it is not.
|
||||
const tarball = bundle({ id: 'rust', version: '1.0.0' })
|
||||
const routes = {
|
||||
[MANIFEST_URL]: manifestFor(tarball),
|
||||
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
|
||||
}
|
||||
|
||||
await assert.rejects(
|
||||
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
|
||||
/declares module id "rust" but the install manifest promised "uo"/,
|
||||
)
|
||||
assert.deepEqual(fs.readdirSync(tmpRoot), [])
|
||||
})
|
||||
|
||||
test('a bundle whose version disagrees with the manifest is refused', async () => {
|
||||
const tarball = bundle({ id: 'uo', version: '9.9.9' })
|
||||
const routes = {
|
||||
[MANIFEST_URL]: manifestFor(tarball),
|
||||
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
|
||||
}
|
||||
|
||||
await assert.rejects(
|
||||
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
|
||||
/declares version "9.9.9"/,
|
||||
)
|
||||
})
|
||||
|
||||
test('an artifact whose length differs from the declared size is refused', async () => {
|
||||
const tarball = bundle()
|
||||
const routes = {
|
||||
[MANIFEST_URL]: manifestFor(tarball, { size: tarball.length + 10 }),
|
||||
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
|
||||
}
|
||||
|
||||
await assert.rejects(
|
||||
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
|
||||
/but the manifest declares/,
|
||||
)
|
||||
})
|
||||
|
||||
test('a hostile archive is refused, and nothing of it reaches the volume', async () => {
|
||||
// The case node-tar alone does NOT cover: it throws on the escaping member,
|
||||
// but only once it reaches it — the members before it are already on disk.
|
||||
// archive.inspect() decides before extract() runs, and the unpack happens in a
|
||||
// scratch directory that is removed either way.
|
||||
const tarball = bundle({
|
||||
extra: [
|
||||
{ name: 'module-uo-1.0.0/../../ESCAPED.txt', body: 'escaped' },
|
||||
],
|
||||
})
|
||||
const routes = {
|
||||
[MANIFEST_URL]: manifestFor(tarball),
|
||||
'https://releases.example.com/mod/uo-1.0.0.tar.gz': tarball,
|
||||
}
|
||||
|
||||
await assert.rejects(
|
||||
() => install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) }),
|
||||
/escapes upward/,
|
||||
)
|
||||
assert.deepEqual(fs.readdirSync(tmpRoot), [], 'no scratch directory, no partial module, no escape')
|
||||
})
|
||||
|
||||
test('an upgrade replaces the directory and reports that it did', async () => {
|
||||
const first = goodRoutes()
|
||||
await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(first.routes) })
|
||||
fs.writeFileSync(path.join(tmpRoot, 'uo', 'STALE.txt'), 'from the old version')
|
||||
|
||||
const second = goodRoutes({ version: '2.0.0', manifest: { version: '2.0.0' } })
|
||||
second.routes[MANIFEST_URL] = manifestFor(second.tarball, { version: '2.0.0' })
|
||||
const result = await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(second.routes) })
|
||||
|
||||
assert.equal(result.replaced, true)
|
||||
assert.equal(result.version, '2.0.0')
|
||||
// A replace, not a merge: a file the previous version left behind must not
|
||||
// survive into the new one, or an upgrade quietly keeps dead code loadable.
|
||||
assert.ok(!fs.existsSync(path.join(tmpRoot, 'uo', 'STALE.txt')))
|
||||
assert.deepEqual(fs.readdirSync(tmpRoot), ['uo'], 'the aside copy is cleaned up')
|
||||
})
|
||||
|
||||
test('a failed upgrade leaves the previous version in place', async () => {
|
||||
const first = goodRoutes()
|
||||
await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(first.routes) })
|
||||
|
||||
const badTarball = bundle({ id: 'uo', version: '2.0.0', extra: [{ name: 'evil/', type: '5' }] })
|
||||
await assert.rejects(() => install.install({
|
||||
url: MANIFEST_URL,
|
||||
hosts: HOSTS,
|
||||
fetchImpl: fakeFetch({
|
||||
[MANIFEST_URL]: manifestFor(badTarball, { version: '2.0.0' }),
|
||||
'https://releases.example.com/mod/uo-1.0.0.tar.gz': badTarball,
|
||||
}),
|
||||
}))
|
||||
|
||||
// The installed module is untouched: the swap is the last step, so a bundle
|
||||
// rejected before it never got near the live directory.
|
||||
assert.ok(fs.existsSync(path.join(tmpRoot, 'uo', 'module.json')))
|
||||
assert.equal(JSON.parse(fs.readFileSync(path.join(tmpRoot, 'uo', 'module.json'), 'utf8')).version, '1.0.0')
|
||||
assert.deepEqual(fs.readdirSync(tmpRoot), ['uo'])
|
||||
})
|
||||
|
||||
// ── The volume ─────────────────────────────────────────────────────────────
|
||||
|
||||
test('moduleDir refuses an id that is not one', () => {
|
||||
for (const id of ['../escape', 'a/b', '', 'UO', '.']) {
|
||||
assert.throws(() => install.moduleDir(id), /invalid module id/, `id ${JSON.stringify(id)}`)
|
||||
}
|
||||
assert.equal(install.moduleDir('uo'), path.join(tmpRoot, 'uo'))
|
||||
})
|
||||
|
||||
test('isInstalled and removeDir report what they did', async () => {
|
||||
const { routes } = goodRoutes()
|
||||
assert.equal(install.isInstalled('uo'), false)
|
||||
|
||||
await install.install({ url: MANIFEST_URL, hosts: HOSTS, fetchImpl: fakeFetch(routes) })
|
||||
assert.equal(install.isInstalled('uo'), true)
|
||||
|
||||
assert.equal(await install.removeDir('uo'), true)
|
||||
assert.equal(install.isInstalled('uo'), false)
|
||||
// Removing what is not there is not an error — an uninstall of a module whose
|
||||
// directory was already deleted by hand should still tidy up the row.
|
||||
assert.equal(await install.removeDir('uo'), false)
|
||||
})
|
||||
|
||||
test('purgeFile resolves the manifest declaration, and refuses one that escapes', async () => {
|
||||
const dir = path.join(tmpRoot, 'uo')
|
||||
fs.mkdirSync(path.join(dir, 'server', 'db'), { recursive: true })
|
||||
fs.writeFileSync(path.join(dir, 'server', 'db', 'purge.sql'), 'DROP TABLE IF EXISTS x;')
|
||||
|
||||
const write = (purge) => fs.writeFileSync(
|
||||
path.join(dir, 'module.json'),
|
||||
JSON.stringify({ id: 'uo', name: 'UO', version: '1.0.0', purge }),
|
||||
)
|
||||
|
||||
write('server/db/purge.sql')
|
||||
assert.equal(install.purgeFile('uo'), path.join(dir, 'server', 'db', 'purge.sql'))
|
||||
|
||||
// The same containment rule the loader applies to client.entry: a manifest may
|
||||
// not point core at a file outside the module it belongs to.
|
||||
write('../../../../etc/passwd')
|
||||
assert.equal(install.purgeFile('uo'), null)
|
||||
|
||||
// Declared but absent, and not declared at all, are both "nothing to run".
|
||||
write('server/db/missing.sql')
|
||||
assert.equal(install.purgeFile('uo'), null)
|
||||
write(undefined)
|
||||
assert.equal(install.purgeFile('uo'), null)
|
||||
})
|
||||
|
||||
test('purgeFile is null for a module that is not on the volume', () => {
|
||||
assert.equal(install.purgeFile('nothere'), null)
|
||||
})
|
||||
@@ -125,6 +125,11 @@ function fakeModel(seed = []) {
|
||||
if (!row || row.state === 'disabled') return
|
||||
Object.assign(row, { state: 'startup_failed', failureStage: stage, failureReason: reason })
|
||||
},
|
||||
async disable(id) {
|
||||
calls.push(`disable:${id}`)
|
||||
const row = rows.get(id)
|
||||
if (row) Object.assign(row, { state: 'disabled', failureStage: null, failureReason: null })
|
||||
},
|
||||
}
|
||||
return model
|
||||
}
|
||||
@@ -374,3 +379,151 @@ test('a hook that throws does not stop the ones behind it', async () => {
|
||||
await assert.doesNotReject(() => lifecycle.shutdown({ modules: loader }))
|
||||
assert.deepEqual(noted(file), ['shutdown:zzz', 'shutdown:aaa'])
|
||||
})
|
||||
|
||||
// ── stop(): the admin panel's Disable ──────────────────────────────────────
|
||||
//
|
||||
// Phase 4, §2.7.2 decision 3. Phase 2's disable moved a record and left the
|
||||
// module running; the whole point of these tests is that it no longer does.
|
||||
|
||||
test('stop runs that one module\'s onShutdown and leaves the others alone', async () => {
|
||||
const file = path.join(tmpRoot, 'log.txt')
|
||||
writeModule('aaa', { boot: '', shutdown: '', log: file })
|
||||
writeModule('bbb', { boot: '', shutdown: '', log: file })
|
||||
const loader = freshLoader(tmpRoot)
|
||||
const model = fakeModel()
|
||||
|
||||
await lifecycle.boot({ modules: loader, model })
|
||||
fs.writeFileSync(file, '')
|
||||
|
||||
const result = await lifecycle.stop('aaa', { modules: loader, model })
|
||||
|
||||
assert.deepEqual(result, { stopped: true, error: null })
|
||||
// Only aaa. This is the difference from shutdown(), which runs everything.
|
||||
assert.deepEqual(noted(file), ['shutdown:aaa'])
|
||||
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
|
||||
assert.equal(stateOf(loader, 'bbb').state, 'started', 'the other module keeps running')
|
||||
assert.equal(model.rows.get('aaa').state, 'disabled')
|
||||
})
|
||||
|
||||
test('stop moves the record only after the hook has run', async () => {
|
||||
// While onShutdown runs, the module is still `started` — the only state in
|
||||
// which its routes and the world it is tearing down agree with each other. The
|
||||
// module reports its own view of itself, so a record moved too early shows up
|
||||
// here as `disabled` instead of `started`.
|
||||
const file = path.join(tmpRoot, 'log.txt')
|
||||
const dir = path.join(tmpRoot, 'aaa')
|
||||
fs.mkdirSync(dir, { recursive: true })
|
||||
fs.writeFileSync(path.join(dir, 'module.json'), JSON.stringify({
|
||||
id: 'aaa', name: 'A', version: '1.0.0', coreApi: '^1.0.0', server: 'index.js',
|
||||
}))
|
||||
fs.writeFileSync(path.join(dir, 'index.js'), `
|
||||
module.exports = (ctx, api) => {
|
||||
api.onBoot(async () => {})
|
||||
api.onShutdown(async () => {
|
||||
const loader = require(${JSON.stringify(require.resolve('../src/modules/loader'))})
|
||||
const me = loader.list().find((m) => m.id === 'aaa')
|
||||
require('fs').appendFileSync(${JSON.stringify(file)}, 'state-during-hook:' + me.state + ${JSON.stringify('\n')})
|
||||
})
|
||||
}`)
|
||||
|
||||
const loader = freshLoader(tmpRoot)
|
||||
await lifecycle.boot({ modules: loader, model: fakeModel() })
|
||||
await lifecycle.stop('aaa', { modules: loader, model: fakeModel([{ id: 'aaa', state: 'started' }]) })
|
||||
|
||||
assert.deepEqual(noted(file), ['state-during-hook:started'])
|
||||
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
|
||||
})
|
||||
|
||||
test('a hook that throws does not prevent the disable', async () => {
|
||||
// The opposite of the boot path's rule, deliberately. There, a failure means
|
||||
// the module never became safe to use; here, the operator has asked for it to
|
||||
// stop answering and a module that could not close cleanly is a reason to log
|
||||
// loudly, not a reason to leave it serving.
|
||||
writeModule('aaa', { boot: '', shutdown: 'throw new Error("socket stuck")' })
|
||||
const loader = freshLoader(tmpRoot)
|
||||
const model = fakeModel()
|
||||
|
||||
await lifecycle.boot({ modules: loader, model })
|
||||
const result = await lifecycle.stop('aaa', { modules: loader, model })
|
||||
|
||||
assert.equal(result.stopped, false)
|
||||
assert.match(result.error, /socket stuck/)
|
||||
assert.equal(stateOf(loader, 'aaa').state, 'disabled', 'disabled anyway')
|
||||
assert.equal(model.rows.get('aaa').state, 'disabled')
|
||||
})
|
||||
|
||||
test('a hook that hangs costs its budget, and the module is still disabled', async () => {
|
||||
writeModule('aaa', { boot: '', shutdown: 'await new Promise(() => {})' })
|
||||
const loader = freshLoader(tmpRoot)
|
||||
const model = fakeModel()
|
||||
|
||||
await lifecycle.boot({ modules: loader, model })
|
||||
const started = Date.now()
|
||||
const result = await lifecycle.stop('aaa', { modules: loader, model, budgetMs: 50 })
|
||||
|
||||
assert.equal(result.stopped, false)
|
||||
assert.match(result.error, /budget/)
|
||||
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
|
||||
assert.ok(Date.now() - started < 2000)
|
||||
})
|
||||
|
||||
test('stopping a module with no onShutdown still disables it', async () => {
|
||||
// A hookless module has nothing to run and must still stop answering, or the
|
||||
// guard and the row disagree about what is serving.
|
||||
writeModule('aaa', { boot: '' })
|
||||
const loader = freshLoader(tmpRoot)
|
||||
const model = fakeModel()
|
||||
|
||||
await lifecycle.boot({ modules: loader, model })
|
||||
const result = await lifecycle.stop('aaa', { modules: loader, model })
|
||||
|
||||
assert.deepEqual(result, { stopped: false, error: null })
|
||||
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
|
||||
assert.equal(model.rows.get('aaa').state, 'disabled')
|
||||
})
|
||||
|
||||
test('stopping a module whose onBoot failed does not run its onShutdown', async () => {
|
||||
// Same rule shutdownHooks() applies: a module that never finished warming up
|
||||
// has a half-built world its onShutdown was not written for. It is still
|
||||
// disabled — it just is not asked to tear anything down.
|
||||
const file = path.join(tmpRoot, 'log.txt')
|
||||
writeModule('aaa', { boot: 'throw new Error("no")', shutdown: '', log: file })
|
||||
const loader = freshLoader(tmpRoot)
|
||||
const model = fakeModel()
|
||||
|
||||
await lifecycle.boot({ modules: loader, model })
|
||||
const before = noted(file)
|
||||
const result = await lifecycle.stop('aaa', { modules: loader, model })
|
||||
|
||||
assert.equal(result.stopped, false)
|
||||
// Compared against what was already there rather than against an empty file:
|
||||
// `noted` splits, so a blanked file reads as [''] and an emptiness assertion
|
||||
// would pass for the wrong reason.
|
||||
assert.deepEqual(noted(file), before, 'no new hook output')
|
||||
assert.ok(!noted(file).includes('shutdown:aaa'))
|
||||
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
|
||||
})
|
||||
|
||||
test('stopping an unknown id writes the row and does not throw', async () => {
|
||||
// A row can exist for a module that is not on the volume, and disabling it is
|
||||
// exactly what an operator would do about that.
|
||||
writeModule('aaa', { boot: '' })
|
||||
const loader = freshLoader(tmpRoot)
|
||||
const model = fakeModel([{ id: 'ghost', state: 'startup_failed' }])
|
||||
|
||||
await assert.doesNotReject(() => lifecycle.stop('ghost', { modules: loader, model }))
|
||||
assert.equal(model.rows.get('ghost').state, 'disabled')
|
||||
})
|
||||
|
||||
test('a row that will not write does not stop the module from being disabled', async () => {
|
||||
// The same rule the boot path follows: bookkeeping failure is not the
|
||||
// operation failing. The guard is what stops traffic, and it has already moved.
|
||||
writeModule('aaa', { boot: '', shutdown: '' })
|
||||
const loader = freshLoader(tmpRoot)
|
||||
const model = fakeModel()
|
||||
await lifecycle.boot({ modules: loader, model })
|
||||
model.disable = async () => { throw new Error('database is gone') }
|
||||
|
||||
await assert.doesNotReject(() => lifecycle.stop('aaa', { modules: loader, model }))
|
||||
assert.equal(stateOf(loader, 'aaa').state, 'disabled')
|
||||
})
|
||||
|
||||
@@ -25,7 +25,7 @@ const assert = require('node:assert/strict')
|
||||
const express = require('express')
|
||||
|
||||
const db = require('../src/utils/db')
|
||||
const { replayFragments } = require('../src/modules/schema')
|
||||
const { replayFragments, runPurge } = require('../src/modules/schema')
|
||||
const { splitStatements } = require('../src/utils/sqlStatements')
|
||||
const { startApp } = require('./_helper')
|
||||
|
||||
@@ -245,3 +245,66 @@ test('replay is skipped, not thrown, when no scan happened in this process', asy
|
||||
assert.deepEqual(rec.ran, [])
|
||||
assert.equal(loader.isLoaded(), false)
|
||||
})
|
||||
|
||||
// ── runPurge: the destructive twin ─────────────────────────────────────────
|
||||
//
|
||||
// Phase 4, slice 1. Same splitter, same pool, same serial execution as the
|
||||
// replay above — pointed the other way. The two differences are deliberate and
|
||||
// are what these tests are for.
|
||||
|
||||
test('purge runs every statement in the file, in order', async () => {
|
||||
const file = path.join(tmpRoot, 'purge.sql')
|
||||
fs.writeFileSync(file, [
|
||||
'-- drop everything this module owns',
|
||||
'DROP TABLE IF EXISTS mod_b;',
|
||||
'DROP TABLE IF EXISTS mod_a;',
|
||||
"DELETE FROM settings WHERE `key` = 'mod_thing';",
|
||||
].join('\n'))
|
||||
const rec = recorder()
|
||||
|
||||
const ran = await runPurge(file, { query: rec.query })
|
||||
|
||||
assert.equal(ran, 3)
|
||||
assert.match(rec.ran[0], /DROP TABLE IF EXISTS mod_b/)
|
||||
assert.match(rec.ran[2], /DELETE FROM settings/)
|
||||
})
|
||||
|
||||
test('purge is not held to the fragment allowlist — DROP is the point of it', async () => {
|
||||
// §2.6's leading-verb allowlist exists because a fragment replays on every
|
||||
// boot. purge.sql never does, which is exactly why it is the one file a module
|
||||
// may put a DROP in.
|
||||
const file = path.join(tmpRoot, 'purge.sql')
|
||||
fs.writeFileSync(file, 'DROP TABLE IF EXISTS mod_a;\nTRUNCATE TABLE mod_b;')
|
||||
const rec = recorder()
|
||||
|
||||
assert.equal(await runPurge(file, { query: rec.query }), 2)
|
||||
})
|
||||
|
||||
test('purge THROWS on failure, unlike the replay', async () => {
|
||||
// A replay failure is one module failing to start, which the site survives by
|
||||
// 503ing it. A purge failure is an operator's explicit destructive request not
|
||||
// having happened — reporting success would leave them believing data is gone
|
||||
// when it is not.
|
||||
const file = path.join(tmpRoot, 'purge.sql')
|
||||
fs.writeFileSync(file, 'DROP TABLE IF EXISTS mod_a;\nDROP TABLE mod_missing;')
|
||||
const query = async (sql) => {
|
||||
if (sql.includes('mod_missing')) throw new Error("Unknown table 'mod_missing'")
|
||||
}
|
||||
|
||||
await assert.rejects(
|
||||
() => runPurge(file, { query }),
|
||||
// The message names WHICH statement stopped it, because a purge is not
|
||||
// transactional — the ones before it have already committed and the operator
|
||||
// needs to know where it got to.
|
||||
/purge failed at statement 2 of 2: Unknown table 'mod_missing'/,
|
||||
)
|
||||
})
|
||||
|
||||
test('an empty purge file runs nothing and does not throw', async () => {
|
||||
const file = path.join(tmpRoot, 'purge.sql')
|
||||
fs.writeFileSync(file, '-- nothing to drop yet\n')
|
||||
const rec = recorder()
|
||||
|
||||
assert.equal(await runPurge(file, { query: rec.query }), 0)
|
||||
assert.deepEqual(rec.ran, [])
|
||||
})
|
||||
|
||||
@@ -38,7 +38,17 @@ modulesDb.getOne = async (id) => (rows.has(id) ? [rows.get(id)] : [])
|
||||
modulesDb.upsert = async ({ id, name, version, source, sha256 }) => {
|
||||
const existing = rows.get(id)
|
||||
if (existing) {
|
||||
Object.assign(existing, { name, version, source: source ?? null, sha256: sha256 ?? null })
|
||||
// COALESCE, matching the real statement: a value overwrites, a NULL leaves
|
||||
// what is there. This fake used to assign unconditionally — faithfully
|
||||
// reproducing the defect it was supposed to be able to catch, which is why
|
||||
// the boot refresh nulling an install's provenance survived until a browser
|
||||
// showed it.
|
||||
Object.assign(existing, {
|
||||
name,
|
||||
version,
|
||||
source: source ?? existing.source ?? null,
|
||||
sha256: sha256 ?? existing.sha256 ?? null,
|
||||
})
|
||||
return { affectedRows: 1 }
|
||||
}
|
||||
rows.set(id, {
|
||||
@@ -116,6 +126,46 @@ test('a hand-placed module records with no provenance', async () => {
|
||||
assert.equal(mod.sha256, null)
|
||||
})
|
||||
|
||||
test('the boot refresh does not wipe an install\'s provenance', async () => {
|
||||
// The defect a browser found in Phase 4, and the reason the upsert COALESCEs.
|
||||
//
|
||||
// lifecycle.boot() re-records every scanned module with NO source and NO
|
||||
// sha256, because a hand-placed directory genuinely has neither. That refresh
|
||||
// used to write both columns as NULL, so an admin-panel install's provenance
|
||||
// survived exactly until the restart the install asked for — and the screen
|
||||
// then described a module installed from a URL as "placed on the volume by
|
||||
// hand". Nothing could see it until Phase 4: no caller had ever passed a
|
||||
// non-null value.
|
||||
await modules.recordInstalled({
|
||||
id: 'uo',
|
||||
name: 'Ultima Online',
|
||||
version: '1.0.0',
|
||||
source: 'https://gitea.example/x/uo-1.0.0.json',
|
||||
sha256: 'b'.repeat(64),
|
||||
})
|
||||
|
||||
// What the next boot does.
|
||||
const after = await modules.recordInstalled({ id: 'uo', name: 'Ultima Online', version: '1.0.0' })
|
||||
|
||||
assert.equal(after.source, 'https://gitea.example/x/uo-1.0.0.json')
|
||||
assert.equal(after.sha256, 'b'.repeat(64))
|
||||
})
|
||||
|
||||
test('a re-install from a new URL does replace the provenance', async () => {
|
||||
// The other half: COALESCE must not make the columns write-once, or an upgrade
|
||||
// would for ever show where the FIRST version came from.
|
||||
await modules.recordInstalled({
|
||||
id: 'uo', name: 'Ultima Online', version: '1.0.0', source: 'https://old/x.json', sha256: 'c'.repeat(64),
|
||||
})
|
||||
const after = await modules.recordInstalled({
|
||||
id: 'uo', name: 'Ultima Online', version: '2.0.0', source: 'https://new/y.json', sha256: 'd'.repeat(64),
|
||||
})
|
||||
|
||||
assert.equal(after.source, 'https://new/y.json')
|
||||
assert.equal(after.sha256, 'd'.repeat(64))
|
||||
assert.equal(after.version, '2.0.0')
|
||||
})
|
||||
|
||||
test('recordInstalled refuses a manifest missing id, name or version', async () => {
|
||||
await assert.rejects(
|
||||
() => modules.recordInstalled({ id: 'uo', version: '1.0.0' }),
|
||||
|
||||
Reference in New Issue
Block a user