RunicGateway/website
3
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Activity

Compare commits

base: RunicGateway:spike/module-atlas
Branches Tags
RunicGateway:main RunicGateway:edge RunicGateway:feat/asset-bridge-p2 RunicGateway:feature/engagement-uo-triggers RunicGateway:feature/module-docker-path RunicGateway:feature/module-install-service RunicGateway:spike/module-atlas RunicGateway:ci/gitea-actions-image-build RunicGateway:feature/moderation-dashboard
...
compare: RunicGateway:91964c8898dbaa6589c1796df3b5e3100679c024
Branches Tags
RunicGateway:main RunicGateway:edge RunicGateway:feat/asset-bridge-p2 RunicGateway:feature/engagement-uo-triggers RunicGateway:feature/module-docker-path RunicGateway:feature/module-install-service RunicGateway:spike/module-atlas RunicGateway:ci/gitea-actions-image-build RunicGateway:feature/moderation-dashboard

27 Commits
spike/modu ... 91964c8898

Author SHA1 Message Date
Colby Whitlock
91964c8898 Merge pull request 'feat(modules): client extension slots (phase 3, slice 2)' (#138) from feature/module-client-slots into edge 
Reviewed-on: #138
 2026-08-11 21:53:41 +00:00
wtclaude
d667565ae7 refactor(modules): move core's UO page content behind the two slots
All checks were successful
PR Checks / client-build (pull_request) Successful in 26s
Details
PR Checks / server-tests (pull_request) Successful in 29s
Details
PR Checks / bot-install (pull_request) Successful in 8m46s
Details 
Core declares site.footer.status and admin.users.detail in main.jsx and fills
both itself, under owner id `core` -- the client twin of registries.registerCore()
and the same trick useShardFlags already uses. The rendered page is unchanged;
what changes is that the content now arrives the way a module's will.

The footer's Shard Status link becomes ShardStatusLink.jsx, and UserDetail's six
UO sections become UserShardSections.jsx. Both are files rather than inline
markup so that the client half of phase 3 deletes a registration and a file
instead of editing a core page under extraction pressure -- which is also what
proves the mechanism before anything depends on it.

The user-detail slot is handed userId and not scope. api.admin.userShard is a UO
binding that leaves core with the client half, so a slot passing it would hand a
module something core is about to delete; an extension builds its own client for
the routes it registered at the other end. Core's own fill now does exactly what
the module will.

Verified in a browser against a real chunk (MODULE_API.md 7.7): a throwaway
module fills both slots and renders its own label and target in the footer with
core's linkStyle, and receives userId on the admin page; a deliberate render
failure is contained to that one spot with the slot named in the console; core's
own fills leave the pages byte-identical to before; and with no module installed
both slots render nothing. Zero CSP reports throughout.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-11 16:45:22 -05:00
wtclaude
1d1350558b feat(modules): client extension slots (phase 3, slice 2) 
The client twin of the server's declareSlot/registerExtension, and the same rule
in both halves: core declares a slot, only core declares one, and at most one
module fills it. Core renders <Slot name> and gets nothing back when the slot is
unfilled, so an instance with no module installed renders exactly what it
rendered before -- the same untouched-path guarantee withModuleNav makes.

A slot is named for a PLACE, never for a meaning. Core supplies the position and
the styling; the label, the target, the data and whether anything renders at all
are the module's. The moment core types a slot by its content it has re-acquired
the game semantics phase 3 exists to remove.

This is the one place the client registry is not fail-open. An unknown slot, a
non-component and a second fill all throw, matching checkExtensionShape
server-side, because a dropped nav row costs a link the viewer can reach another
way while a silently dropped extension is invisible to everyone including its
author. A throw is always a programming error and never a race: core declares in
its own bundle and every module chunk is a deferred script injected after it.

Reading stays fail-safe -- undeclared and unfilled both read null -- and a
filling component renders inside an error boundary. That asymmetry is where the
client differs from the server: a module route that throws costs the module's own
page, but an extension throws inside CORE's, and the whole reason core keeps
ownership of that page is that it stays usable.

Core decorates a slot through <Slot wrap>, not by asking whether it is filled.
The obvious alternative is right about the unfilled case and wrong about the
failed one -- the extension is filled, so the separator renders, and then the
component throws into the boundary and leaves the separator behind on its own.
wrap puts core's decoration inside the boundary where it shares the extension's
fate. Found in a browser, with the footer's separator, which is the only place
either could have been found.

MODULE_API_VERSION 1.1.0 -> 1.2.0, both halves: the two state ONE version.
Contract: docs/website/MODULE_API.md 3.7.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-11 16:45:08 -05:00
Colby Whitlock
7a236cad8b Merge pull request 'refactor(modules)!: move the UO server half out to module-uo (phase 3, slice 1)' (#137) from feature/module-extract-server into edge 
Reviewed-on: #137
 2026-08-11 21:06:23 +00:00
wtclaude
f5e6025dcc test: re-point core's suite at what core still owns
All checks were successful
PR Checks / bot-install (pull_request) Successful in 18s
Details
PR Checks / client-build (pull_request) Successful in 27s
Details
PR Checks / server-tests (pull_request) Successful in 29s
Details 
25 of 82 test files left with the module. Three that core keeps needed splitting
rather than moving, and the split is the boundary in each case.

announceJobs.test.js keeps the announce PIPELINE -- the shared backoff schedule,
the parent-status rollup, core's Discord leg -- and loses the town-crier text
building and classification, which are a module's leg. pushDispatch.test.js
keeps the SSRF guard and publish() delivering a content-free tickle, and loses
mapShardEvent and the shard fan-out, which are a module's catalog.

playerRouteAccess.test.js is the one worth explaining. It guards a real past bug
-- an admin 403'd off their own characters -- and it did so through
/player/shard/accounts, which is now module-owned. The guarantee it protects is
CORE's, though: /player/* is role-agnostic self-service, staff are a superset of
players. So it stays here and asserts that through /player/appeals, a core route
with the same gate. Moving it would have left core with no test of its own tier
rule, which is precisely what regressed once before.

The remaining updates are core's own tests catching up: ctx has four more
members, registerCore now registers only what core owns (one stream, one leg, no
filled slot), and the extension-slot test asks for the DECLARED slot's router
rather than the filled one, since core declares it and a module fills it. The
gated-surface floor drops from >100 to >50 -- it is there so a filter matching
nothing fails loudly, not to track core's exact route count.

616 core tests and 160 client tests pass; the module's own suite is 351.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-11 12:08:26 -05:00
wtclaude
39d731d87a refactor(modules)!: move the UO server half out to module-uo 
40 files, ~9,674 lines, 27 of 68 tables. Core no longer contains anything that
knows what a shard is.

BREAKING for a deployment only in the sense that the module must be installed
for these URLs to answer -- no URL moved. routes.manifest.json goes 228 -> 158
public routes here, and the 70 that left reappear byte-identical when the module
is loaded: verified by generating the manifest against core+module and diffing
it against the pre-extraction file. Zero missing, zero added, and routes.guards
identical across all 228, so no auth gate moved either.

The five tier mounts are gone from public/admin/player index.js and are still
served: the loader mounts them onto the same routers after every core mount.
That ordering is also what keeps the prefixes unclaimable -- the collision check
asks the live router what core owns, so a second module claiming /shard is
rejected against the mounts actually present rather than against a list.

server.js loses its eight UO call sites to the module's onBoot/onShutdown.
schema.sql loses its 27 shard_*/uo_link_* statements; the two that FK into users
are why the fragment replays AFTER core's schema, and no core table ever
referenced a module table, which is what makes core still able to boot alone.

Verified against a running server with the module installed: it loads, mounts
five prefixes, replays 35 statements, warms up and reaches `started`; public
shard and atlas routes answer 200 with real data (800 creatures, 6,455
spawners), admin and player answer 401 from core's tier gates, and the extension
slot answers at /admin/users/:id/shard/*. Core's own SPA renders the shard and
atlas pages unchanged against the module-served API, with no console errors and
no CSP reports.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-11 12:08:03 -05:00
wtclaude
f50541f374 feat(modules): ctx additions and the post-hook registry (API 1.1.0) 
Everything the extraction needed from core that ctx did not already offer.
Additions only, so minor.

ctx.activity.log, because an admin action a module performs has to land in
core's one audit log or the trail has a hole exactly where a module operates the
game -- a module keeping its own log would be a second place to look, which in
practice means a place nobody looks. Write-only; reading the log is the admin
panel's job and it spans every actor.

ctx.users.getById, one function for one caller: the admin.users.detail slot
router needs the user its prefix names. ctx.site.baseUrl, because a module has
to build absolute links and §2.7 forbids it reading core's APP_BASE_URL -- a
getter, not a captured string, so it cannot go stale against the env.

ctx.middleware.rateLimit is core's makeLimiter, plus accountChangeLimiter handed
over whole. The split is deliberate: a module states its own window and cap
because it knows what its endpoints cost, and takes the plumbing from core so
there is one express-rate-limit in the process and one place a breach is logged.
accountChangeLimiter is shared policy -- core's /auth/me and /player/account sit
behind the same counter -- so a module's account-change route has to land IN it
rather than beside it. marketLimiter was UO policy living in core's file and
leaves with the route it guards.

registerPostHook is the fourth registry, and the last thing binding core to the
module. Core's post controller called newsGump.syncPost directly: core's CMS
naming a UO file. It now publishes what it already knows and a subscriber
decides what to do with it. Not folded into registerAnnounceLeg, which fires on
the same transition, because a leg is a one-shot DELIVERY with retry and
classification while a post hook maintains idempotent STATE, runs on delete as
well as save, and refreshes silently on an edit.

Also fixes a real loader defect the extraction exposed: schema table names were
matched against the RAW file, so a fragment whose header says "every CREATE
TABLE carries IF NOT EXISTS" was rejected for a prefix violation on a table
called `carries`. module-uo's fragment hit exactly that. Both scans now read
split statements, which strip comments -- the same class of bug as a boundary
check failing on its own documentation.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-11 12:07:45 -05:00
Colby Whitlock
b649345484 Merge pull request 'feat(modules): mount the modules directory as a volume (phase 2, PR 9)' (#136) from feature/module-compose-volume into edge 
Reviewed-on: #136
 2026-08-11 05:58:18 +00:00
wtclaude
a103e0ce10 feat(modules): mount the modules directory as a volume (phase 2, PR 9)
All checks were successful
PR Checks / bot-install (pull_request) Successful in 21s
Details
PR Checks / client-build (pull_request) Successful in 32s
Details
PR Checks / server-tests (pull_request) Successful in 1m39s
Details 
Closes Phase 2. Modules live on a mount, never in the image — that is what
lets an operator add one to a pull-only deployment without building anything.

`./modules` is a bind mount rather than a named volume: placing a module
directory by hand is a supported install (MODULE_SYSTEM.md §2.5), and that has
to be doable from the host rather than through `docker cp`. Read-write, because
the admin panel's install/uninstall unpacks and removes directories there.

The directory is tracked via its README so it exists in the checkout with the
operator's own ownership — Docker recreates a missing bind-mount source as
root:root, which the container user could not then write. `.dockerignore`
excludes it so a module in the builder's working tree can never ship inside an
image.

Also corrects the route-manifest generator's list of filesystem-conditional
mounts, which never picked up `/modules` when PR 7 added it. Comment only; the
generator filters on an allowlist, so its behaviour was already right.

Verified against a real container, not just a parsed compose file: image
carries an empty node-owned /app/modules despite a module in the build context;
a module on the bind mount loads, mounts, replays and reaches `started`;
`/api/v1/public/modules` lists it; the chunk serves from the entry's directory
only (server source and module.json 404) with `no-cache`; the injected tag
follows core's bundle; and in Chrome the page renders on first paint inside
core's PublicLayout with its nav row interleaved into core's public nav, under
enforced `script-src 'self'` with zero CSP reports and no console errors.
Removing the directory by hand reconciles the row to `startup_failed`/`require`
and leaves core healthy with no injection.

933 server + 160 client tests pass, manifest unchanged at 230 routes, swagger
regenerates byte-identical.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-11 00:48:48 -05:00
Colby Whitlock
0a3f1eb9fa Merge pull request 'feat(modules): interleave module nav items and derive moderator confinement (phase 2, PR 8)' (#135) from feature/module-nav-interleave into edge 
Reviewed-on: #135
 2026-08-11 05:29:40 +00:00
wtclaude
a45a3d120a feat(modules): interleave module nav, derive moderator confinement
All checks were successful
PR Checks / bot-install (pull_request) Successful in 17s
Details
PR Checks / server-tests (pull_request) Successful in 1m34s
Details
PR Checks / client-build (pull_request) Successful in 8m58s
Details 
Phase 2, PR 8 of docs/website/MODULE_SYSTEM.md 2.7 - the nav half PR 7
deferred, plus the two seams 1.4 and 1.5 asked for.

withModuleNav (client/src/modules/nav.js) merges an installed module's rows
into core's three navs BEFORE the admin-override merge, and that ordering is
the design. applyNavOverrides and buildPublicNav are keyed by `to` and drop
any key their base array does not declare, so rows appended after the merge
would be unorderable, unrelabellable and unhideable in Admin - Navigation.
Today's UO rows are all three of those things, so appending would make the
extraction a visible regression for anyone who has ever edited their nav.
Merging first means a module row is an ordinary row downstream: nothing in
navOverrides.js, NavEditor.jsx or the layouts knows a module exists.

MOD_PATHS is gone. Moderator visibility and the redirect that confines a
moderator both derive from each row's own `roles`, in the new plain-JS
lib/adminNav.js (plain so the DOM-less runner can reach it). Two rows move,
both toward what the server already permitted: Dashboard, whose roles had
always named moderator, and My Characters, which is ungated self-service.

That also fixes a defect predating the module system. The redirect was a
THIRD hardcoded list - three path prefixes against MOD_PATHS' five paths -
and they disagreed about /admin/houses, so a moderator who clicked Houses in
their own sidebar was bounced back to Moderation. The derived allow-list is
computed from the BASE nav, never the override-merged one: an override is
presentation and must not move an authorization boundary either way.

The feature seam (modules/features.jsx + modules/featureGate.js) resolves a
row's `feature` against the provider its OWN module registered, so the
namespace comes from the registration and no string carries a parsed prefix.
Core registers useShardFlags under the owner id `core` - the client twin of
registries.registerCore() - so the ten shard-gated header rows already run
through the seam and Phase 3 deletes a registration instead of rewriting
SiteHeader. Every unknown fails open: no provider, a null answer while a
fetch is in flight, or a junk return all show the link, because the server is
the gate and hiding a page from someone entitled to it is the worse mistake.

933 server tests (unchanged - this PR is client-only), 160 client tests
(+37). routes.manifest.json unchanged at 230 routes; the OpenAPI spec
regenerates byte-identical.

Re-ran the MODULE_API.md 7.7 browser smoke, since this is the seam that rule
exists for. A throwaway module registering nav in all three areas and a
provider granting one flag and withholding another: the row lands inside
core's Moderation group rather than an appended block, the withheld row does
not render, a moderator reaches both /admin/houses and the module's admin
page, and an admin can relabel a module row and have it persist and apply.
Zero CSP reports, zero console errors.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-10 23:42:02 -05:00
Colby Whitlock
e3c999b704 Merge pull request 'feat(modules): the client registry, window.__rg and the chunk's script injection' (#134) from feature/module-client-registry into edge 
Reviewed-on: #134
 2026-08-11 04:01:59 +00:00
wtclaude
e0927bc255 feat(modules): the client registry, window.__rg and the chunk's script injection
All checks were successful
PR Checks / bot-install (pull_request) Successful in 21s
Details
PR Checks / client-build (pull_request) Successful in 30s
Details
PR Checks / server-tests (pull_request) Successful in 1m37s
Details 
Phase 2, PR 7 of docs/website/MODULE_SYSTEM.md 2.7 — the client half's
delivery. A module's prebuilt chunk is served, injected, handed core's React
and its UI kit, and its routes are rendered by App.jsx. The registry is empty
on a bare core, so nothing an operator can see changes.

Client:
  - modules/registry.js — registerRoutes/registerNav/registerFeatureProvider,
    with the URL namespace written by core, never by the module
  - modules/shared.js — window.__rg: React, react-dom/client, react-router-dom,
    react/jsx-runtime, the registry, the seven-member UI kit and the request
    primitive, frozen
  - App.jsx reads routesFor for all three areas; nav consumption is PR 8
  - main.jsx publishes the global, then mounts on DOMContentLoaded

Server:
  - the loader validates client.entry and publishes clientChunks() and
    clientEntryUrls(); an entry in the module root is rejected, because the
    directory it sits in is what gets served
  - app.js mounts each chunk at /modules/<id>/ behind the module's state guard
    with no-cache; anything else under /modules is a 404, not the SPA shell
  - htmlShell injects the tag before </body>, so core's bundle runs first
    wherever a bundler puts it

Found by loading a real chunk in a browser, and fixed here: core mounted before
any module chunk had evaluated, because document.readyState during a deferred
script is 'interactive', not 'loading'. Every test passed against that build.
The smoke is written down in MODULE_API.md 7.7.

933 server tests (+23), 123 client tests (+14). routes.manifest.json unchanged
at 230 routes; the OpenAPI spec regenerates byte-identical.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-10 22:54:16 -05:00
Colby Whitlock
fe83c91ba9 Merge pull request 'feat(modules): publish the installed-module list at /api/v1/public/modules' (#133) from feature/module-public-endpoint into edge 
Reviewed-on: #133
 2026-08-11 03:16:49 +00:00
wtclaude
291c30f6ff feat(modules): publish the installed-module list at /api/v1/public/modules
All checks were successful
PR Checks / client-build (pull_request) Successful in 26s
Details
PR Checks / server-tests (pull_request) Successful in 1m33s
Details
PR Checks / bot-install (pull_request) Successful in 8m45s
Details 
Phase 2, PR 6 of docs/website/MODULE_SYSTEM.md 2.7 — the first module-system
URL a client can see. The SPA and the Android app feature-detect against the
capabilities a module declares; the shape is settled in MODULE_API.md 2.9.

Four decisions, and what is absent from the payload is most of the design:

* started modules only. A module that is disabled or failed to load is
  ABSENT, exactly as 4.4 already leaves its routes and its nav absent, so a
  client renders a site without that capability rather than advertising one
  that 503s.
* no state, failure_stage or failure_reason. Where a module broke belongs to
  the admin Modules screen, and the reason is an exception string from inside
  core — not anonymous-visitor business.
* no client chunk URL. htmlShell injects a script tag per started module
  (3.1.3), so the browser is handed the tag rather than a URL to fetch. This
  endpoint feature-detects; it does not load. MODULE_SYSTEM 2.6 step 4 is
  amended to match (API 6.7).
* no siteMode gate and no database — the same class as /public/status and
  /public/version, so a client can still feature-detect during maintenance.

It is a capability router of its own rather than a fifth singleton in
site.router.js, and that is load-bearing: the loader's prefix-collision probe
reads the live tier stack and skips root-mounted layers, because a use('/', ...)
matches every path. A route inside the root-mounted site router would be
invisible to it — mounting use('/modules', ...) is what makes "no module may
claim /modules" a rule the loader enforces.

910 tests pass (+9, every one on the boundary — what must NOT appear).
routes.manifest.json gains exactly the one route and routes.guards.json records
it with an empty gates list, which is itself the assertion that it is ungated.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-10 22:03:22 -05:00
Colby Whitlock
85f563fc16 Merge pull request 'feat(modules): boot/shutdown hook dispatch and the installed_modules reconcile' (#132) from feature/module-lifecycle into edge 
Reviewed-on: #132
Reviewed-by: Colby Whitlock <whitlocktech@gmail.com>
 2026-08-11 02:34:46 +00:00
wtclaude
32ed8e4411 fix(test): stop the suite reaching a real database, and make it exit
All checks were successful
PR Checks / client-build (pull_request) Successful in 25s
Details
PR Checks / server-tests (pull_request) Successful in 1m34s
Details
PR Checks / bot-install (pull_request) Successful in 8m45s
Details 
`npm test` never terminated. Twenty-two test files omitted the two lines that
point the pool at a dead port, so utils/db.js -- which builds its mariadb pool at
require time and calls dotenv.config() itself -- picked up server/.env and opened
five live connections to the developer's MariaDB. The tests still passed, because
they stub their models and never issue a query; the only symptoms were a process
that never exited and five connections held for as long as it lived. Thirty
stranded workers is 150 connections, which is the whole server's limit, and that
is the "too many connections" this workspace has hit before.

The convention was right and only ever as good as the next test file's memory of
it, so it moves into the harness: test/_setup.js is loaded with --require by the
npm script, ahead of the test file it hosts, which is the only moment early
enough to matter. It pins the dead port -- dotenv does not overwrite an existing
variable, so an explicit DB_PORT= still wins for anyone who wants a live database
-- and closes the pool after the file's tests, so the process exits at once
instead of waiting out the driver's connect retries. The per-file preambles stay:
they keep `node --test test/one.test.js` safe on its own.

Two supporting fixes:

- db.close() is idempotent. pool.end() throws "pool is already closed" on a
  second call, and closing twice is now normal rather than exceptional -- the
  harness closes the pool for every file on top of the suites that close it
  themselves, and a SIGINT followed by a SIGTERM already reached the shutdown
  handler twice.
- test/_helper.js's close() destroys open connections. server.close() only stops
  accepting and waits for existing connections to end, and node's global fetch
  keeps its sockets alive, so the listener outlived the test that created it --
  invisible until now, because the pool was holding the process open anyway.

announceJobs.test.js alone: 120s+ hang -> 0.35s. The whole suite now finishes in
~75s where it previously did not finish at all: 901 tests, 901 pass, verified
three times on CI's exact platform (node:20 on Linux, via Docker).

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-10 21:27:47 -05:00
wtclaude
21196466ed feat(modules): boot/shutdown hook dispatch and the installed_modules reconcile
All checks were successful
PR Checks / bot-install (pull_request) Successful in 23s
Details
PR Checks / server-tests (pull_request) Successful in 1m44s
Details
PR Checks / client-build (pull_request) Successful in 9m1s
Details 
Phase 2, PR 5 of docs/website/MODULE_SYSTEM.md 2.7. api.onBoot/api.onShutdown
stop throwing, server.js gains one call on each side, and the 2.4 state machine
finally runs against real outcomes -- which is what makes 4.5's `disabled` 404
leg reachable for the first time.

Dispatch and reconcile live in src/modules/lifecycle.js rather than in the
loader, for the reason the schema replay does: routeManifest.js and swagger.js
both require app.js against a dead pool, so the loader may not reach the
database. The two halves meet at exactly one function, loader.setState(), so the
in-memory record the dispatch guard reads and the row the admin panel reads are
moved together and cannot disagree.

Four decisions, all recorded in MODULE_API.md 2.5 and 4.4:

- The loader classifies its failures by 4.3 step, so failure_stage says where a
  module broke instead of being a column nothing ever filled. The four steps
  readManifest covers in one pass label themselves; the rest are inferred from
  how far load() had got, and an unlabelled throw is recorded against the step
  that was running rather than guessed at.
- A row whose directory is gone is marked startup_failed rather than left
  claiming `enabled` -- the boot reset has just moved it there, and a row
  claiming to be enabled for a module that is not on the volume is the one state
  that is simply untrue. An uninstall leaves `disabled`, which the reset never
  touches, so this catches only a hand-deleted directory.
- Core's eight UO boot call sites stay in server.js until Phase 3. Unlike a
  registered announce leg, a boot call site already has somewhere to live, so
  moving it now would be extraction done early in a phase whose exit criterion
  is that nothing changes.
- onBoot gets no timeout. Shutdown races a SIGKILL and boot does not, and a slow
  onBoot delaying the listener is the contract's promise to a module that must
  warm up before it serves.

The operator's switch wins over everything: a disabled module is guarded, not
booted, and does not have its failure re-recorded, or an outcome would silently
switch it back on next boot. Every database write in the reconcile is
individually caught -- a row that will not update is worse reporting, never a
failed boot.

900 tests pass (17 new). routes.manifest.json is unchanged at 229 routes and the
OpenAPI spec regenerates byte-identical.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-10 20:32:00 -05:00
Colby Whitlock
39eaae90a8 Merge pull request 'feat(modules): the three de-entanglement registries, with core as the registrant' (#131) from feature/module-registries into edge 
Reviewed-on: #131
 2026-08-10 23:18:57 +00:00
wtclaude
97f19b4221 docs(modules): note that registerExtension's spec-file argument is core-only
All checks were successful
PR Checks / bot-install (pull_request) Successful in 18s
Details
PR Checks / client-build (pull_request) Successful in 26s
Details
PR Checks / server-tests (pull_request) Successful in 1m40s
Details 
Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-10 17:53:59 -05:00
wtclaude
6195c76d61 feat(modules): the three de-entanglement registries, with core as the registrant
All checks were successful
PR Checks / client-build (pull_request) Successful in 23s
Details
PR Checks / server-tests (pull_request) Successful in 1m39s
Details
PR Checks / bot-install (pull_request) Successful in 8m49s
Details 
Phase 2 PR 4 of docs/website/MODULE_SYSTEM.md §2.7. Adds server/src/modules/registries.js
and moves core's own notification streams, announce leg and users-detail routes
behind it, so the three seams §1.8 and §1.9 named are exercised on every boot
before any module depends on them.

Registering is validate-then-commit per registrant: the loader stages what a
module claims and the second pass commits it, so a module that throws halfway
through register() — or fails checkDeclared after it — leaves nothing behind.
That is the registry-side twin of PR 2's second-pass mount rule.

Four decisions, all the recommended option:

- announce legs became a child table. `announce_job_legs` replaces the
  towncrier_*/discord_* column groups, so the leg set is data: core registers
  `discord`, module-uo will register `towncrier`, and a module cannot ALTER a
  core table to add its own. Backfill is guarded on information_schema (a
  SELECT of a dropped column is a parse error, not a runtime one) and the
  columns go with DROP COLUMN IF EXISTS. Verified against the live dev DB:
  three legacy jobs migrated faithfully, three replays, no duplicates.
- `mapEvent` dropped from registerNotificationStreams. §1.8 already inverts the
  push path so a module owns fromShardEvent and calls core's publish() with a
  stream id it resolved; a second mapping mechanism was a leftover. The public
  safety filter, the kinds it reads and the streams it protects now live in one
  file and move together.
- core registers through the same staging area a module uses, via an explicit
  registries.registerCore() in app.js before modules.load().
- core's six /admin/users/:id/shard/* paths now go through the
  `admin.users.detail` slot, and getUser moved back to admin.controller.js.

Found on the way, and the reason two build tools changed:

- scripts/routeManifest.js could not decode a parameterised mount. Its
  unwinder expected `(?:([^\/]+?))`; express 4.22 emits `(?:\/([^/]+?))` with
  the separator inside the group. The branch had never run. It threw rather
  than guessing, which is what it is for.
- swagger-autogen cannot follow a route into an extension slot — the slot's
  router is created by declareSlot() and filled later, so there is no literal
  mount for a static parse. Regenerating deleted 407 lines and printed
  `Swagger-autogen: Success`, the spike's exact failure (MODULE_API.md §7.4).
  swagger/slotSpecs.js generates a fragment per filled slot and re-roots it at
  the prefix the router actually hangs at in the live app — read from the
  express stack via routeManifest's own mountPath, so the manifest and the spec
  cannot disagree. swagger/mergeSpec.js is the merge helper core owes for
  module fragments anyway (§6.1a), proved here against core's own slot first.

884 tests pass (856 before). routes.manifest.json is unchanged at 229 routes.
The OpenAPI spec diff is two lines of intent: the retry endpoint's summary, and
its `leg` no longer being a fixed enum.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-10 17:47:59 -05:00
Colby Whitlock
bd749d4f1f Merge pull request 'feat(modules): replay module schema fragments after core's (phase 2, PR 3)' (#130) from feat/modules-schema into edge 
Reviewed-on: #130
 2026-08-10 22:06:44 +00:00
wtclaude
2892d01b24 feat(modules): replay module schema fragments after core's
All checks were successful
PR Checks / bot-install (pull_request) Successful in 22s
Details
PR Checks / client-build (pull_request) Successful in 28s
Details
PR Checks / server-tests (pull_request) Successful in 10m9s
Details 
Phase 2, PR 3 of docs/website/MODULE_SYSTEM.md 2.7. ensureSchema() now replays
every installed module's schema fragment immediately after core's schema.sql,
per MODULE_API.md 2.6.

The work splits across two files on the line of whether a database is needed to
know the answer. loader.js VALIDATES a fragment at load time, before anything is
mounted, because every rule 2.6 states about the SQL is knowable by reading it;
a module that breaks one never mounts (4.4, left column). modules/schema.js
EXECUTES it, so the only failures there are the ones the database alone could
report, and those are post-mount and answer 503 (4.4, right column).

Validation is a leading-verb allowlist -- CREATE, ALTER, INSERT, UPDATE, the
four core's own schema.sql uses -- rather than the DROP denylist 2.6 words it
as. A fragment is replayed on every boot, so TRUNCATE and DELETE would empty a
table at each restart and RENAME would fail at the second one; a denylist only
ever bans what somebody thought of. A CREATE TABLE missing IF NOT EXISTS is
rejected for the same reason: it works once and fails every boot after, which
presents to an operator as a module that broke on restart.

The splitter moves to utils/sqlStatements.js so core's schema and a fragment are
split by literally the same code, which is what 2.6 promises. It is its own file
rather than an export of utils/db.js because the loader validates fragments at
require time and must not drag the mariadb pool into app.js's require chain.

The replay sits outside ensureSchema's wait-for-the-database retry loop: a
fragment that throws is one module's failure, not a signal the database is
coming up, and retrying core's whole schema nine more times over one module's
bad SQL would turn a 503'd module into a two-minute boot.

Found while wiring it: db/seed.js calls ensureSchema() standalone for
`npm run seed`, without ever requiring app.js, so the loader has not scanned and
fragments()'s 7.6 throw would have broken seeding outright. The replay asks
isLoaded() and logs the skip rather than swallowing it -- a booting server
quietly getting no module tables is the thing 7.6 exists to prevent.

Verification:

- 856 server tests pass, 14 new. moduleSchema.test.js injects the query fn, so
  the exact statements and their order are asserted with the pool at a dead port
  like every other suite.
- routes.manifest.json and routes.guards.json diffs are zero lines, 229 routes
  -- the phase 2 exit criterion. swagger-output.json regenerates byte-identical.
- Run for real against the local MariaDB with two fixture modules: a good
  fragment created its table, applied its ALTER and seeded its row; a fragment
  whose SQL passes validation but the server rejects (`id NOTATYPE`) marked only
  that module startup_failed, its route answering 503 while the other answered
  200; a second ensureSchema on the same database was a clean no-op.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-10 16:56:58 -05:00
Colby Whitlock
7780fb033b Merge pull request 'feat(modules): the filesystem module loader (phase 2, PR 2)' (#129) from feat/modules-loader into edge 
Reviewed-on: #129
 2026-08-10 21:31:24 +00:00
wtclaude
ec1ca7e794 feat(modules): the filesystem module loader
All checks were successful
PR Checks / bot-install (pull_request) Successful in 16s
Details
PR Checks / client-build (pull_request) Successful in 24s
Details
PR Checks / server-tests (pull_request) Successful in 10m4s
Details 
Phase 2 PR 2 of docs/website/MODULE_SYSTEM.md 2.7. Adds
server/src/modules/{loader,semver,version}.js: the synchronous scan of
MODULES_DIR, manifest validation, prefix and table-name collision
rejection, per-module try/catch and the mount into the three tier
routers behind the MODULE_API.md 4.5 dispatch guard.

Two decisions the contract left open, both now written up there:

- The load trigger is one explicit modules.load(tierRouters) call in
  app.js, not a lazy scan (API 7.6). Accessors throw until it has run,
  because "no modules installed" is a real answer a caller must not be
  handed by accident.
- Whether core owns a prefix is asked of the live tier routers via
  express's own layer.match(), skipping root-mounted layers, rather than
  a hardcoded table -- the spike's was already stale when written
  (API 4.3).

Mounting is a second pass after every module is validated. Doing it
inside the scan loop makes the first module's layers indistinguishable
from core's, so the second module claiming a taken prefix is told it
collided with core and the module-versus-module check is unreachable.

registerExtension/NotificationStreams/AnnounceLeg and onBoot/onShutdown
throw "not available until phase 2 PR 4/5" rather than no-op; an
accepting stub would let a module believe it had registered something.
No schema replay, no boot dispatch, no installed_modules reconcile --
those are PRs 3 and 5, and until PR 5 a record's state is in memory only.

No module ships on the volume, so nothing an operator or client can see
changes: 842 tests pass, routes.manifest.json is unchanged at 229 routes
and swagger-output.json regenerates byte-identical.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-10 14:35:27 -05:00
Colby Whitlock
dcf6ef1886 Merge pull request 'feat(modules): installed_modules and the module state machine (phase 2, PR 1)' (#128) from feat/modules-state into edge 
Reviewed-on: #128
 2026-08-10 19:04:58 +00:00
wtclaude
3add0063bf feat(modules): installed_modules and the module state machine
All checks were successful
PR Checks / bot-install (pull_request) Successful in 17s
Details
PR Checks / client-build (pull_request) Successful in 27s
Details
PR Checks / server-tests (pull_request) Successful in 1m35s
Details 
Phase 2 PR 1 of the module system (docs/website/MODULE_SYSTEM.md 2.7). The
table and the state machine only: no loader, no routes, no boot wiring, so
nothing an operator or a client can see changes and the route manifest diff
is zero lines.

The five states of 2.4 live in one `state` column: installed -> enabled ->
started, with disabled and startup_failed as the recoverable ones. The row is
a record of what happened, never the source of truth for what is mounted --
the loader scans the filesystem at require time, before the database is
reachable (MODULE_API.md 4.1), which is what keeps routes.manifest.json
generatable against a dead database.

Two rules the model owns and the boot path will lean on:

- Every boot resets each non-disabled row to `enabled` and clears its
  recorded failure, so a startup_failed module is retried on the next restart
  and a fixed one recovers with no admin-panel visit. `disabled` is the one
  operator decision rather than outcome, so it survives untouched -- and a
  disabled module's failure is a no-op, never a re-enable.
- A failure carries the stage it happened at, and every non-failing
  transition clears it, so a running module can never show a stale reason.

An illegal move throws instead of writing a row that misrepresents the state,
except on the two boot-path softenings noted above, because one module's
failure must never become everybody's.

22 model tests over an in-memory fake; the SQL and the DDL were round-tripped
against a real MariaDB separately.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-10 06:35:57 -05:00
153 changed files with 8100 additions and 19927 deletions
Expand all files Collapse all files

3
.dockerignore
View File

@@ -10,5 +10,8 @@ uploads
server/logs
logs
*.log
# Installed modules are mounted at runtime, never baked into the image. Without
# this a module in the builder's working tree would ship inside every image.
modules
.DS_Store
Thumbs.db

7
.gitignore vendored
View File

@@ -21,6 +21,13 @@ uploads/
server/logs/
logs/
# Installed modules (docs/website/MODULE_SYSTEM.md). Core ships no module, so
# anything here is an operator's install or a developer's scratch copy. The
# directory itself IS tracked, via its README: docker-compose.yml bind-mounts it,
# and a missing bind-mount source is recreated by Docker as root-owned.
modules/*
!modules/README.md
# Operator-supplied spawn atlas artwork. Creature art is never committed: sprites
# are extracted from the operator's own UO client .mul/.uop files and are theirs,
# not ours to redistribute. The images live under server/uploads/atlas/, already

6
Dockerfile
View File

@@ -21,6 +21,12 @@ RUN if [ -f client/package.json ]; then \
# Persistent uploads + logs live on mounted volumes.
RUN mkdir -p /app/uploads /app/logs && chown -R node:node /app/uploads /app/logs
# Installed modules are mounted in too (docker-compose.yml), and .dockerignore
# keeps any local modules/ OUT of the image — a module must never be baked in.
# The directory is still created here so a container run without the mount finds
# an empty, writable modules dir rather than no directory at all.
RUN mkdir -p /app/modules && chown node:node /app/modules
USER node
EXPOSE 3000

12
README.md
View File

@@ -222,6 +222,10 @@ IMAGE_TAG=sha-042a151 docker compose pull && docker compose up -d
- Health check: `GET http://localhost:3000/api/health` → `{ "status": "ok" }`
- Logs: `docker compose logs -f app` (and `./logs/app.log` on the host)
- Stop: `docker compose down` (add `-v` to also wipe the database + uploads volumes)
- Modules: installed into `./modules` on the host (bind-mounted to `/app/modules`), never baked into
the image — an operator adds one to a pull-only deployment without building anything. Adding or
removing one takes a `docker compose restart app`; the scan is synchronous at startup. See
[`modules/README.md`](modules/README.md).
**Build the images locally instead of pulling** (offline, or to test an unmerged change) — overlay
the dev file, which adds `build:` back:
@@ -391,9 +395,10 @@ npm run routes:manifest -- --check # exit 1 if either file is stale (what CI ru
The generator walks the live Express stack (runtime introspection, not source parsing — a route's path
sits on the line *after* `router.get(`, which defeats greps) and keeps only
`/api/**` and `/.well-known/**` plus the internal listener. The SPA catch-all, `/uploads` and `/brand`
are filesystem-conditional static mounts, not API contract, so they are excluded and the output does
not depend on whether the client has been built.
`/api/**` and `/.well-known/**` plus the internal listener. The SPA catch-all, `/uploads`, `/brand`
and installed modules' `/modules/<id>` chunks are filesystem-conditional static mounts, not API
contract, so they are excluded and the output depends neither on whether the client has been built
nor on which modules are mounted.
Two generated files, two very different meanings:
@@ -507,6 +512,7 @@ Copy `.env.example` (Compose) or `server/.env.example` (local) and fill in. **`.
| `NODE_ENV` | `production` | |
| `PORT` | `3000` | server listens on `0.0.0.0:PORT` |
| `UPLOAD_DIR` | `<server>/uploads` | where post images are written (`/app/uploads`, volume-mounted, in Compose) |
| `MODULES_DIR` | `<repo>/modules` | where installed modules are scanned from (`/app/modules`, bind-mounted, in Compose) |
| `DB_HOST` / `DB_PORT` | `db` / `3306` | `db` in Compose; `127.0.0.1` for local dev |
| `DB_NAME` / `DB_USER` / `DB_PASSWORD` | `runic_gateway` / `runic` / — | app database credentials |
| `DB_ROOT_PASSWORD` | — | MariaDB root (Compose only) |

326
client/src/App.jsx
View File

@@ -5,6 +5,8 @@ import MaintenanceGate from './components/MaintenanceGate.jsx'
import RequireAuth from './components/RequireAuth.jsx'
import RequirePlayer from './components/RequirePlayer.jsx'
import RoleGate from './components/RoleGate.jsx'
import { routesFor } from './modules/registry.js'
import { ModuleFeaturesProvider } from './modules/features.jsx'
// Public
import Portal from './routes/public/Portal.jsx'
@@ -79,156 +81,198 @@ export default function App() {
return (
<AuthProvider>
<SiteProvider>
<Routes>
{/* Landing hero — always public, even in maintenance mode. The hero is
itself the pre-launch "coming soon" page, so it sits outside the
MaintenanceGate and every visitor sees it regardless of auth/site mode. */}
<Route path="/" element={<Portal />} />
{/* Inside the auth and site contexts, because a feature provider is a
hook that may well read either — the shard one does, indirectly, by
asking an endpoint whose answer depends on the session. Outside the
routes, so the nav in every layout is filtered by the same gate and
the provider hooks are called once for the whole app rather than
once per screen. */}
<ModuleFeaturesProvider>
<Routes>
{/* Landing hero — always public, even in maintenance mode. The hero is
itself the pre-launch "coming soon" page, so it sits outside the
MaintenanceGate and every visitor sees it regardless of auth/site mode. */}
<Route path="/" element={<Portal />} />
{/* Rest of the public site — gated by maintenance mode (admins preview through it) */}
<Route
element={
<MaintenanceGate>
<Outlet />
</MaintenanceGate>
}
>
<Route path="/site" element={<Website />} />
<Route path="/site/news" element={<News />} />
<Route path="/site/screenshots" element={<Screenshots />} />
<Route path="/site/five-on-friday" element={<FiveOnFriday />} />
<Route path="/site/newsletter" element={<Newsletter />} />
<Route path="/site/newsletter/:id" element={<NewsletterIssue />} />
<Route path="/site/about" element={<About />} />
<Route path="/site/status" element={<Status />} />
<Route path="/site/shard" element={<Shard />} />
<Route path="/site/shard/activity" element={<ShardActivity />} />
<Route path="/site/champs" element={<ChampSpawns />} />
<Route path="/site/guilds" element={<Guilds />} />
<Route path="/site/governors" element={<Governors />} />
<Route path="/site/houses" element={<Houses />} />
<Route path="/site/rules" element={<Rules />} />
<Route path="/site/atlas" element={<Atlas />} />
<Route path="/site/atlas/:slug" element={<AtlasCreature />} />
<Route path="/site/leaderboards" element={<Leaderboards />} />
<Route path="/site/market" element={<Market />} />
<Route path="/site/market/vendors/:serial" element={<MarketVendor />} />
<Route path="/wiki" element={<Wiki />} />
<Route path="/wiki/:slug" element={<WikiArticle />} />
{/* CMS pages: top-level /:slug, matched only after the named routes
above (React Router ranks static routes over this dynamic one). */}
<Route path="/:slug" element={<CmsPage />} />
</Route>
{/* Draft-preview link (token-gated). Outside the maintenance gate so a
preview link works regardless of site mode. */}
<Route path="/preview/:id/:token" element={<CmsPage preview />} />
{/* Admin */}
<Route path="/admin/login" element={<AdminLogin />} />
<Route
path="/admin"
element={
<RequireAuth>
<AdminLayout />
</RequireAuth>
}
>
<Route index element={<Dashboard />} />
<Route path="posts" element={<PostsAdmin />} />
<Route path="pages" element={<PagesAdmin />} />
<Route path="pages/new" element={<PageBuilder />} />
<Route path="pages/:id" element={<PageBuilder />} />
<Route path="wiki" element={<WikiAdmin />} />
<Route path="hero" element={<HeroEditor />} />
{/* Theme editing writes an admin-only settings key; the route sits
behind the same RoleGate as the sidebar entry that reaches it,
and PUT/DELETE /admin/settings is admin-only server-side too. */}
{/* Rest of the public site — gated by maintenance mode (admins preview through it) */}
<Route
path="appearance"
element={
<RoleGate roles={['admin']}>
<AppearanceAdmin />
</RoleGate>
}
/>
{/* Same reasoning as Appearance: the nav overrides are an admin-only
settings key, so the route carries the same RoleGate as the
sidebar entry that reaches it. */}
<Route
path="navigation"
element={
<RoleGate roles={['admin']}>
<NavEditor />
</RoleGate>
}
/>
<Route path="settings" element={<SettingsAdmin />} />
<Route
path="moderation"
element={
<RoleGate roles={['admin', 'moderator']}>
<MaintenanceGate>
<Outlet />
</RoleGate>
</MaintenanceGate>
}
>
<Route index element={<Moderation />} />
<Route path="user/:discordId" element={<ModerationUser />} />
<Route path="appeals" element={<Appeals />} />
<Route path="/site" element={<Website />} />
<Route path="/site/news" element={<News />} />
<Route path="/site/screenshots" element={<Screenshots />} />
<Route path="/site/five-on-friday" element={<FiveOnFriday />} />
<Route path="/site/newsletter" element={<Newsletter />} />
<Route path="/site/newsletter/:id" element={<NewsletterIssue />} />
<Route path="/site/about" element={<About />} />
<Route path="/site/status" element={<Status />} />
<Route path="/site/shard" element={<Shard />} />
<Route path="/site/shard/activity" element={<ShardActivity />} />
<Route path="/site/champs" element={<ChampSpawns />} />
<Route path="/site/guilds" element={<Guilds />} />
<Route path="/site/governors" element={<Governors />} />
<Route path="/site/houses" element={<Houses />} />
<Route path="/site/rules" element={<Rules />} />
<Route path="/site/atlas" element={<Atlas />} />
<Route path="/site/atlas/:slug" element={<AtlasCreature />} />
<Route path="/site/leaderboards" element={<Leaderboards />} />
<Route path="/site/market" element={<Market />} />
<Route path="/site/market/vendors/:serial" element={<MarketVendor />} />
<Route path="/wiki" element={<Wiki />} />
<Route path="/wiki/:slug" element={<WikiArticle />} />
{/* Installed modules' public pages, namespaced `/<id>/…` — the
registry prefixes the segment, so a module cannot spell its way
out of it (docs/website/MODULE_API.md §3.3). Declared before the
CMS catch-all below: React Router ranks a static segment over a
dynamic one, so the order is not what saves us, but keeping the
two adjacent makes the relationship visible to whoever adds the
next route here. */}
{routesFor('public').map((r) => (
<Route key={r.path} path={`/${r.path}`} element={r.element} />
))}
{/* CMS pages: top-level /:slug, matched only after the named routes
above (React Router ranks static routes over this dynamic one). */}
<Route path="/:slug" element={<CmsPage />} />
</Route>
<Route path="activity" element={<ActivityAdmin />} />
<Route path="bot-activity" element={<BotActivityAdmin />} />
<Route path="discord-bot" element={<DiscordBotAdmin />} />
<Route path="shard" element={<ShardAdmin />} />
<Route path="shard-visibility" element={<ShardVisibility />} />
<Route path="shard-atlas" element={<SpawnAtlasAdmin />} />
<Route
path="shard-ops"
element={
<RoleGate roles={['admin', 'moderator']}>
<ShardOps />
</RoleGate>
}
/>
<Route
path="houses"
element={
<RoleGate roles={['admin', 'moderator']}>
<HousesAdmin />
</RoleGate>
}
/>
<Route path="characters" element={<AdminCharacters />} />
<Route path="characters/:serial" element={<AdminCharacter />} />
<Route path="auth-providers" element={<AuthProvidersAdmin />} />
<Route path="users" element={<UsersAdmin />} />
<Route path="users/:id" element={<UserDetail />} />
<Route path="invites" element={<InvitesAdmin />} />
<Route path="account" element={<AccountAdmin />} />
<Route path="*" element={<Navigate to="/admin" replace />} />
</Route>
{/* Player portal */}
<Route path="/account/login" element={<PlayerLogin />} />
<Route path="/account/register" element={<PlayerRegister />} />
<Route path="/account/forgot" element={<ForgotPassword />} />
<Route path="/account/reset/:token" element={<ResetPassword />} />
<Route path="/invite/:token" element={<AcceptInvite />} />
<Route
element={
<RequirePlayer>
<PlayerPortalLayout />
</RequirePlayer>
}
>
<Route path="/player" element={<PlayerCharacters />} />
<Route path="/player/char/:serial" element={<PlayerCharacter />} />
<Route path="/account" element={<PlayerAccount />} />
<Route path="/account/appeals" element={<PlayerAppeals />} />
</Route>
{/* Draft-preview link (token-gated). Outside the maintenance gate so a
preview link works regardless of site mode. */}
<Route path="/preview/:id/:token" element={<CmsPage preview />} />
<Route path="*" element={<Navigate to="/" replace />} />
</Routes>
{/* Admin */}
<Route path="/admin/login" element={<AdminLogin />} />
<Route
path="/admin"
element={
<RequireAuth>
<AdminLayout />
</RequireAuth>
}
>
<Route index element={<Dashboard />} />
<Route path="posts" element={<PostsAdmin />} />
<Route path="pages" element={<PagesAdmin />} />
<Route path="pages/new" element={<PageBuilder />} />
<Route path="pages/:id" element={<PageBuilder />} />
<Route path="wiki" element={<WikiAdmin />} />
<Route path="hero" element={<HeroEditor />} />
{/* Theme editing writes an admin-only settings key; the route sits
behind the same RoleGate as the sidebar entry that reaches it,
and PUT/DELETE /admin/settings is admin-only server-side too. */}
<Route
path="appearance"
element={
<RoleGate roles={['admin']}>
<AppearanceAdmin />
</RoleGate>
}
/>
{/* Same reasoning as Appearance: the nav overrides are an admin-only
settings key, so the route carries the same RoleGate as the
sidebar entry that reaches it. */}
<Route
path="navigation"
element={
<RoleGate roles={['admin']}>
<NavEditor />
</RoleGate>
}
/>
<Route path="settings" element={<SettingsAdmin />} />
<Route
path="moderation"
element={
<RoleGate roles={['admin', 'moderator']}>
<Outlet />
</RoleGate>
}
>
<Route index element={<Moderation />} />
<Route path="user/:discordId" element={<ModerationUser />} />
<Route path="appeals" element={<Appeals />} />
</Route>
<Route path="activity" element={<ActivityAdmin />} />
<Route path="bot-activity" element={<BotActivityAdmin />} />
<Route path="discord-bot" element={<DiscordBotAdmin />} />
<Route path="shard" element={<ShardAdmin />} />
<Route path="shard-visibility" element={<ShardVisibility />} />
<Route path="shard-atlas" element={<SpawnAtlasAdmin />} />
<Route
path="shard-ops"
element={
<RoleGate roles={['admin', 'moderator']}>
<ShardOps />
</RoleGate>
}
/>
<Route
path="houses"
element={
<RoleGate roles={['admin', 'moderator']}>
<HousesAdmin />
</RoleGate>
}
/>
<Route path="characters" element={<AdminCharacters />} />
<Route path="characters/:serial" element={<AdminCharacter />} />
<Route path="auth-providers" element={<AuthProvidersAdmin />} />
<Route path="users" element={<UsersAdmin />} />
<Route path="users/:id" element={<UserDetail />} />
<Route path="invites" element={<InvitesAdmin />} />
<Route path="account" element={<AccountAdmin />} />
{/* Installed modules' admin pages, at /admin/<id>/…, already inside
RequireAuth + AdminLayout. A module cannot supply its own auth
wrapper — only an optional { roles }, which core applies as the
same RoleGate its own routes above use, so the sidebar and the
route table cannot disagree about who may see what. Before the
`*` redirect, which would otherwise swallow every one of them. */}
{routesFor('admin').map((r) => (
<Route
key={r.path}
path={r.path}
element={r.gate ? <RoleGate roles={r.gate.roles}>{r.element}</RoleGate> : r.element}
/>
))}
<Route path="*" element={<Navigate to="/admin" replace />} />
</Route>
{/* Player portal */}
<Route path="/account/login" element={<PlayerLogin />} />
<Route path="/account/register" element={<PlayerRegister />} />
<Route path="/account/forgot" element={<ForgotPassword />} />
<Route path="/account/reset/:token" element={<ResetPassword />} />
<Route path="/invite/:token" element={<AcceptInvite />} />
<Route
element={
<RequirePlayer>
<PlayerPortalLayout />
</RequirePlayer>
}
>
<Route path="/player" element={<PlayerCharacters />} />
<Route path="/player/char/:serial" element={<PlayerCharacter />} />
<Route path="/account" element={<PlayerAccount />} />
<Route path="/account/appeals" element={<PlayerAppeals />} />
{/* Installed modules' player-portal pages, at /player/<id>/…. This
group's own routes are absolute (its layout route has no path),
so the prefix is written here rather than inherited — the one
place the three areas do not read alike. */}
{routesFor('player').map((r) => (
<Route
key={r.path}
path={`/player/${r.path}`}
element={r.gate ? <RoleGate roles={r.gate.roles}>{r.element}</RoleGate> : r.element}
/>
))}
</Route>
<Route path="*" element={<Navigate to="/" replace />} />
</Routes>
</ModuleFeaturesProvider>
</SiteProvider>
</AuthProvider>
)

11
client/src/api/client.js
View File

@@ -42,6 +42,17 @@ function safeParse(text) {
}
}
// The request PRIMITIVE, exported for installed modules and handed to them on
// `window.__rg.api` (docs/website/MODULE_API.md §3.5). Core owns the fetch
// semantics — same-origin /api/v1, cookies included, JSON in and out, ApiError
// on a non-2xx — and nothing above them: a module owns the paths it calls,
// because it owns the routes at the other end.
//
// The `api` object below stays core's own binding surface. Its `atlas` and
// `shard` namespaces are module bindings that only still live here because
// Phase 3 has not moved them yet.
export { req as request }
export const api = {
// ----- auth -----
me: () => req('/auth/me'),

24
client/src/components/ShardStatusLink.jsx Normal file
View File

@@ -0,0 +1,24 @@
// ── Core's fill for the `site.footer.status` extension slot ────────────────
//
// Phase 3, slice 2 of docs/website/MODULE_SYSTEM.md §2.7.1; the contract is
// MODULE_API.md §3.7.
//
// This is the whole of what used to be four lines inline in SiteFooter.jsx, and
// it is a file now for one reason: `/site/shard` is a UO page, so the link goes
// when the client half goes, and core should be deleting a registration rather
// than editing its footer under extraction pressure.
//
// Note what core kept and what it handed over. Core owns the position in the row
// and the separator around it, and passes `linkStyle` so the row stays visually
// one row. The label, the destination, and the decision to render at all are
// this file's — which is exactly the division a module inherits.
import { Link } from 'react-router-dom'
export default function ShardStatusLink({ linkStyle }) {
return (
<Link to="/site/shard" style={linkStyle}>
Shard Status
</Link>
)
}

21
client/src/components/SiteFooter.jsx
View File

@@ -1,5 +1,14 @@
import { Link } from 'react-router-dom'
import { useSite } from '../contexts/SiteContext.jsx'
import Slot from '../modules/Slot.jsx'
const FOOTER_SLOT = 'site.footer.status'
// Handed to the extension rather than left for it to guess. A module rendering
// its own link in this row should look like the row, and the alternative is
// every module restating core's colours and then drifting from them the next
// time this footer is themed.
const LINK_STYLE = { color: 'var(--accent)', textDecoration: 'none' }
export default function SiteFooter() {
const { contactEmail, siteTitle } = useSite()
@@ -36,10 +45,14 @@ export default function SiteFooter() {
<a href={`mailto:${contactEmail}`} style={{ color: 'var(--accent)', textDecoration: 'none' }}>
{contactEmail}
</a>
&nbsp;·&nbsp;
<Link to="/site/shard" style={{ color: 'var(--accent)', textDecoration: 'none' }}>
Shard Status
</Link>
{/* A module's spot in the footer, and core supplies only the
position and the styling: the label, the target and whether
anything renders at all are the module's (MODULE_API.md §3.7).
The separator goes through `wrap` rather than sitting beside the
slot, so it shares the extension's fate — no module installed and
a module whose link throws both render nothing here, rather than
the second leaving a stray middot behind. */}
<Slot name={FOOTER_SLOT} linkStyle={LINK_STYLE} wrap={(link) => <>&nbsp;·&nbsp;{link}</>} />
&nbsp;·&nbsp;
<Link to="/admin/login" style={{ color: '#5d6b7d', textDecoration: 'none' }}>
Admin

31
client/src/components/SiteHeader.jsx
View File

@@ -4,22 +4,28 @@ import MoonDot from './MoonDot.jsx'
import BrandLogo from './BrandLogo.jsx'
import { useAuth } from '../contexts/AuthContext.jsx'
import { useSite } from '../contexts/SiteContext.jsx'
import { useShardFeatures, canSee } from '../lib/useShardFeatures.js'
import NavDropdown from './NavDropdown.jsx'
import { buildPublicNav, pruneNav } from '../lib/navOverrides.js'
import { parseJsonSetting } from '../lib/settingsJson.js'
import { withModuleNav } from '../modules/nav.js'
import { useFeatureGate } from '../modules/features.jsx'
// One consistent top nav for the whole public site. Every page gets the same
// main links plus an auth-aware entry on the right (Sign in / My Account / Admin).
//
// Entries carrying a `feature` are shard surfaces an admin can disable or gate
// to a higher audience (Admin -> Shard Visibility). They are hidden when this
// viewer can't reach them, so we never render a link that would 403. The gate
// itself is server-side; this is only about not advertising a dead end.
// Entries carrying a `feature` are surfaces an admin can disable or gate to a
// higher audience (Admin -> Shard Visibility). They are hidden when this viewer
// can't reach them, so we never render a link that would 403. The gate itself is
// server-side; this is only about not advertising a dead end. Which module
// answers for a given flag is the registry's business now, not this file's —
// core registers `useShardFlags` for the ten below and Phase 3 hands them over
// (modules/featureGate.js).
//
// Exported because Admin -> Navigation edits this list. It stays declared here,
// with this component as its owner: the editor may only relabel, reorder and
// hide what it finds, and `to`/`feature` are never its to change (§7).
// hide what it finds, and `to`/`feature` are never its to change (§7). An
// installed module's rows join it in `withModuleNav` below — before the override
// merge, so an admin can edit those rows exactly as they edit these.
export const NAV = [
{ label: 'Home', to: '/', end: true },
{ label: 'News', to: '/site/news' },
@@ -48,7 +54,12 @@ const linkStyle = ({ isActive }) => ({
export default function SiteHeader() {
const { user, loading } = useAuth()
const { siteTitle, settings } = useSite()
const shardFeatures = useShardFeatures()
const isVisible = useFeatureGate()
// Core's rows plus every installed module's. Computed once: the registry is
// fixed before the first render and there is no unregistering, so this cannot
// change during a session (modules/nav.js).
const baseNav = useMemo(() => withModuleNav(NAV, 'public'), [])
// An admin may relabel, reorder and hide these entries from Admin →
// Navigation, and may group them into dropdown sections alongside links of
@@ -62,9 +73,9 @@ export default function SiteHeader() {
// • with no stored row this is the coded NAV, in code order, so an
// untouched instance renders exactly what it renders today.
const nav = useMemo(() => {
const tree = buildPublicNav(NAV, parseJsonSetting(settings.nav_public))
return pruneNav(tree, (item) => !item.feature || canSee(shardFeatures, item.feature))
}, [settings.nav_public, shardFeatures])
const tree = buildPublicNav(baseNav, parseJsonSetting(settings.nav_public))
return pruneNav(tree, isVisible)
}, [baseNav, settings.nav_public, isVisible])
// Where the auth entry points: staff → admin, player → portal, else sign in.
let account

64
client/src/lib/adminNav.js Normal file
View File

@@ -0,0 +1,64 @@
// Who may see a row of the admin sidebar, and where that lets them go.
//
// Plain JS, in its own file, for two reasons. It is shared — AdminLayout renders
// by it and Admin -> Navigation builds its palette by it (THEMING_AND_NAV.md
// §8.1), and a second copy of this answer is exactly the thing this file exists
// to abolish. And it is the closest thing in the client to an authorization
// decision, so it belongs somewhere the test runner can reach, which a .jsx file
// is not.
//
// **A row's own `roles` is the whole answer.** Until Phase 2 PR 8 this was
// `roles` AND a hardcoded `MOD_PATHS` list of five paths that confined
// moderators, AND a third prefix list in the redirect effect that disagreed with
// both (docs/website/MODULE_SYSTEM.md §1.4). A module's rows could never be
// added to a list core hardcodes, which is what forced the derivation — but the
// lists had already drifted from each other without a module in sight.
/**
* Can a viewer with this role see this row?
*
* Applied AFTER the override merge in both callers: an override is presentation
* and this is the boundary, so an override saying `hidden: false` on a row this
* role cannot see still shows nothing (THEMING_AND_NAV.md §7).
*
* A row with no `roles` is visible to everyone who reached the admin area at
* all — that is the self-service case (Account, My Characters), and staff are a
* superset of players.
*/
export function navItemVisibleTo(item, role) {
return !item.roles || item.roles.includes(role)
}
/**
* The paths a viewer with this role may reach, derived from the rows they see.
*
* Takes the BASE nav, never the override-merged one: an override must not be
* able to move this boundary in either direction. Hiding a row from a
* moderator's sidebar must not also bar them from the page behind it, and
* un-hiding one must not admit them to a page their role does not carry.
*
* @param {Array<{items: Array}>} baseNav the grouped admin nav
* @param {string} role
* @returns {Array<{to: string, exact: boolean}>}
*/
export function allowedPathsFor(baseNav, role) {
return (Array.isArray(baseNav) ? baseNav : [])
.flatMap((g) => g.items || [])
.filter((item) => navItemVisibleTo(item, role))
.map((item) => ({ to: item.to, exact: item.end === true }))
}
/**
* Is this pathname one of them?
*
* A row carrying `end` matches exactly — `/admin` is the dashboard, not a prefix
* of the whole admin area, and treating it as one would let every path through.
* Every other row also covers its sub-routes, which is what keeps
* `/admin/moderation/appeals/12` and a module's detail pages reachable without
* anyone listing them.
*/
export function isAllowedPath(pathname, allowed) {
return (allowed || []).some(({ to, exact }) =>
exact ? pathname === to : pathname === to || pathname.startsWith(`${to}/`),
)
}

5
client/src/lib/navOverrides.js
View File

@@ -20,7 +20,10 @@
// Two shapes are supported, because two exist:
// flat [{ to, label, ... }] — public header, player portal
// grouped [{ title?, items: [{ to, label, ... }] }] — admin sidebar
function isGrouped(nav) {
// Exported for modules/nav.js, which has to answer the same question about the
// same array a moment earlier — one implementation, so the interleave and the
// merge can never disagree about which shape they are looking at.
export function isGrouped(nav) {
return nav.length > 0 && nav.every((g) => g && Array.isArray(g.items))
}

12
client/src/lib/useShardFeatures.js
View File

@@ -56,3 +56,15 @@ export function useShardFeatures() {
export function canSee(features, name) {
return !features || features.set.has(name)
}
// The same answer in the shape core's generic feature seam takes: a Set-like of
// the flags this viewer may see, or null while we do not know yet
// (modules/featureGate.js). Core registers THIS as the provider for the `uo`
// namespace (main.jsx), so the ten shard-gated rows in the public header are
// already resolved through the module seam rather than beside it — when Phase 3
// moves those rows into the module, the registration moves with this file and
// core is left with nothing to delete.
export function useShardFlags() {
const features = useShardFeatures()
return features ? features.set : null
}

101
client/src/main.jsx
View File

@@ -2,12 +2,99 @@ import React from 'react'
import { createRoot } from 'react-dom/client'
import { BrowserRouter } from 'react-router-dom'
import App from './App.jsx'
import { publishSharedDependencies } from './modules/shared.js'
import { declareSlot, registerExtension, registerFeatureProvider } from './modules/registry.js'
import { useShardFlags } from './lib/useShardFeatures.js'
import ShardStatusLink from './components/ShardStatusLink.jsx'
import UserShardSections from './routes/admin/views/UserShardSections.jsx'
import './styles/theme.css'
createRoot(document.getElementById('root')).render(
<React.StrictMode>
<BrowserRouter>
<App />
</BrowserRouter>
</React.StrictMode>,
)
// Publish window.__rg BEFORE rendering and before any module chunk evaluates.
// Installed modules arrive as `<script type="module" src="/modules/<id>/…">`
// tags the server injects into the shell (server/src/utils/htmlShell.js), placed
// after this bundle's own tag; module scripts execute in document order, so they
// resolve their externals against the global this call sets up
// (docs/website/MODULE_API.md §3.2).
publishSharedDependencies()
// Core registers through the same seam a module uses, and registers FIRST — the
// client twin of the server's `registries.registerCore()` (MODULE_SYSTEM.md
// §1.9). The ten shard-gated rows in the public header are core's only because
// Phase 3 has not moved them yet; routing them through the registry now means
// SiteHeader holds one mechanism instead of two, and the extraction becomes a
// deletion rather than a rewrite made under extraction pressure.
//
// The owner id is `core`, which is what a nav row with no `moduleId` resolves
// against (modules/featureGate.js). The namespace is `uo`, so a module that
// wants to read these flags — the `uo` module itself, once it owns them — asks
// for them by the name they will always have had.
registerFeatureProvider('core', 'uo', useShardFlags)
// ── Extension slots (MODULE_API.md §3.7) ───────────────────────────────────
//
// Declared HERE, in core's own bundle, which is what makes the ordering a fact
// rather than a hope: module chunks are deferred scripts the shell injects after
// this one (§3.1), so a module can never reach registerExtension before the slot
// it names exists. "Unknown slot" therefore always means a typo or a version
// skew, never a load-order accident — which is why that case throws.
//
// Both slots are named for a PLACE, not for a meaning. `site.footer.status` is
// the spot in the footer's info row, not a declaration that core knows what a
// game server's status is; the label, the target and whether anything renders at
// all belong to whoever fills it. A slot typed by its content would put game
// semantics back into core, which is the thing Phase 3 takes out.
declareSlot('site.footer.status')
// Deliberately the same name as the server's slot (MODULE_API.md §2.4): one
// resource, one extension point, two halves. The module with routes under
// /api/v1/admin/users/:id is the module with something to show on that page.
declareSlot('admin.users.detail')
// And core fills both itself, under owner id `core`, with the components that
// were inline in SiteFooter.jsx and UserDetail.jsx until this slice. The page
// renders exactly what it rendered before, and the mechanism is exercised by
// core's own content from the day it lands rather than first proved by the
// change that depends on it. Slice 3 deletes these three lines and the two files
// they name, and the module registers the same two slots on its way in.
registerExtension('core', 'site.footer.status', ShardStatusLink)
registerExtension('core', 'admin.users.detail', UserShardSections)
// Render on DOMContentLoaded rather than immediately, and that is the one line
// of core's boot the module system changes.
//
// Deferred scripts — which every `type="module"` script is — execute in document
// order and ALL of them finish before DOMContentLoaded fires. Waiting for that
// event is therefore the guarantee that every installed module has registered
// its routes before React reads the registry: no loading state, no re-render,
// and no ordering race between core's bundle and a module's. A module chunk that
// 404s or throws does not hold the event back, so a broken module costs its own
// pages and not the site.
//
// The readyState check below is `'complete'`, and it is not the obvious
// `'loading'`. A DEFERRED script — which every `type="module"` script is — runs
// after the document has been parsed, so by the time this line executes
// readyState is already `'interactive'`; DOMContentLoaded has NOT fired yet and
// still comes after every deferred script. Testing for `'loading'` therefore
// mounts immediately, before any module chunk has evaluated, and a module's
// routes are missing from the very first render — which looks exactly like a
// module that failed to load: its URL falls through to core's catch-all and
// redirects home. Found by loading a real chunk in a browser; no unit test in
// this repo can see it.
//
// `'complete'` is only reached after `load`, which is strictly later than any
// static deferred script, so this branch is the genuine "the event has already
// been and gone" case and not a wrong guess about our own timing.
function mount() {
createRoot(document.getElementById('root')).render(
<React.StrictMode>
<BrowserRouter>
<App />
</BrowserRouter>
</React.StrictMode>,
)
}
if (document.readyState === 'complete') {
mount()
} else {
document.addEventListener('DOMContentLoaded', mount, { once: true })
}

68
client/src/modules/Slot.jsx Normal file
View File

@@ -0,0 +1,68 @@
// ── <Slot> — where core renders a module's content ─────────────────────────
//
// Phase 3, slice 2 of docs/website/MODULE_SYSTEM.md §2.7.1; the normative
// contract is docs/website/MODULE_API.md §3.7.
//
// The read side of registry.js's extension slots. Core puts one of these where a
// module may contribute to a core page, and gets back either the filling
// component with the props core passed, or nothing at all.
//
// **Nothing at all is the important half.** An instance with no module installed
// renders the identical page it renders today, which is the same untouched-path
// guarantee `withModuleNav` makes for nav — and the reason a core layout can
// place a slot without also acquiring an empty-state to design.
import React from 'react'
import { extensionFor } from './registry.js'
/**
* Contain a module's render failure to the module's own section.
*
* This is where the client differs from the server, deliberately. A module
* *route* that throws costs the module's own page and core does not need to care.
* An extension throws inside CORE's page — the admin's user detail, the site
* footer — and the whole reason core keeps ownership of that page is that it
* stays usable. So a slot renders nothing and logs, rather than taking the
* surrounding page down with it.
*
* A class because that is what React gives us: there is no hook form of
* componentDidCatch, and this is the only error boundary core has.
*/
class SlotBoundary extends React.Component {
constructor(props) {
super(props)
this.state = { failed: false }
}
static getDerivedStateFromError() {
return { failed: true }
}
componentDidCatch(error) {
// Named so the console says whose fault it is: a blank section with an
// anonymous stack is how a module bug becomes core's support ticket.
console.error(`[modules] extension in slot "${this.props.name}" threw and was dropped`, error)
}
render() {
return this.state.failed ? null : this.props.children
}
}
/**
* @param {string} name the slot id, declared by core in main.jsx
* @param {function} [wrap] core markup that only makes sense AROUND a rendered
* extension — a separator, a heading, a rule. Called with the extension's
* element and rendered inside the boundary, so it shares the extension's fate:
* an unfilled slot and a failed one both render nothing at all, decoration
* included. Found in a browser, because the obvious alternative — asking
* whether the slot is filled and rendering the separator alongside — is right
* about the unfilled case and leaves a stray separator behind on the failed one.
* @param {object} props everything else is handed to the filling component
*/
export default function Slot({ name, wrap, ...props }) {
const Extension = extensionFor(name)
if (!Extension) return null
const element = <Extension {...props} />
return <SlotBoundary name={name}>{wrap ? wrap(element) : element}</SlotBoundary>
}

56
client/src/modules/featureGate.js Normal file
View File

@@ -0,0 +1,56 @@
// Which nav rows a viewer may see, when the answer belongs to a module.
//
// Phase 2, PR 8 of docs/website/MODULE_SYSTEM.md §2.7 (§1.5 states the problem);
// the contract is docs/website/MODULE_API.md §3.3.
//
// Ten of the sixteen rows in the public header carry a `feature`, and every one
// of them is a shard surface an admin can disable or gate to a higher audience.
// The provider that answers those questions — `useShardFeatures` — moves out
// with the module, so core cannot keep calling it directly and still be a core.
// It keeps a generic seam instead, and the module fills it.
//
// **The namespace comes from the registration, not from the string.** A row's
// `feature` is resolved by the provider its OWN module registered, so a module
// author writes `feature: 'status'` exactly as it reads today: nothing parses a
// prefix, and a typo'd namespace is not a thing that can exist. Core's own rows
// carry no `moduleId` and resolve against the owner id `core`, which is what
// core registers `useShardFeatures` under until Phase 3 moves those rows into
// the module and they arrive stamped `uo` instead.
//
// Everything here fails OPEN, and that is deliberate and unchanged from
// useShardFeatures' own posture: this is presentation, the gate is server-side
// (a disabled feature 404s and an out-of-rung one 403s whether or not a link was
// rendered), so an unknown answer shows the link rather than blanking the nav.
// The one thing a UI mistake must never do here is hide a page from someone
// entitled to it.
/**
* The predicate the layouts filter their nav with.
*
* @param {Map<string, {has: (name: string) => boolean} | null | undefined>} flagsByOwner
* one entry per registered provider, keyed by the id of the module that
* registered it. The value is whatever that provider's hook returned this
* render: a Set-like of the flags this viewer may see, or `null` while the
* answer is still in flight.
* @returns {(item: object) => boolean}
*/
export function buildFeatureGate(flagsByOwner) {
return function isVisible(item) {
if (!item || !item.feature) return true
const owner = item.moduleId ?? 'core'
// No provider for this owner: the row names a flag nothing answers for. That
// is the no-module-installed case — no core row carries a `feature` once the
// module is out — and it is a correct no-op rather than a hidden row.
if (!flagsByOwner || !flagsByOwner.has(owner)) return true
const flags = flagsByOwner.get(owner)
// Still loading, or a provider that returned something unusable. Both are
// "we do not know yet", and both show the link.
if (!flags || typeof flags.has !== 'function') return true
return flags.has(item.feature)
}
}
/** The gate an area with no providers gets: everything is visible. */
export const OPEN_GATE = () => true
export default buildFeatureGate

65
client/src/modules/features.jsx Normal file
View File

@@ -0,0 +1,65 @@
import { createContext, useContext, useMemo, useState } from 'react'
import { featureProviders } from './registry.js'
import { buildFeatureGate, OPEN_GATE } from './featureGate.js'
// The React half of the feature seam. The decision logic is featureGate.js,
// which is plain JS and therefore testable in a runner with no DOM; this file is
// wiring, the same split registry.js and shared.js already use.
//
// **Calling a hook per provider inside a loop is the point, and it is legal
// here.** The rules of hooks require the same hooks in the same order on every
// render of a component — not a statically known list. The provider list is
// fixed before the first render (registration happens while module chunks
// evaluate, and main.jsx does not mount until DOMContentLoaded), there is no
// unregistering, and the snapshot below freezes it per component instance
// anyway. So the loop's length cannot change between renders of this provider,
// which is the actual requirement.
//
// A provider hook returns a Set-like of the flags this viewer may see, or `null`
// while it is still fetching. Core knows nothing else about it: what a flag
// means, how it is fetched, and what it is gated on are all the module's.
const FeatureGateContext = createContext(OPEN_GATE)
export function ModuleFeaturesProvider({ children }) {
// Snapshotted once. useState's initialiser runs on the first render only, so
// even a provider that somehow registered late cannot change this instance's
// hook count mid-life — it would be ignored until the next mount, which is a
// far better failure than a crashed render.
const [providers] = useState(featureProviders)
// eslint-disable-next-line react-hooks/rules-of-hooks -- fixed-length list, see above
const values = providers.map((provider) => provider.hook())
const gate = useMemo(
() => {
const byOwner = new Map()
// First registration wins for a given owner: a module that registers two
// namespaces answers its own nav rows from the first, rather than from
// whichever happened to be stored last.
providers.forEach((provider, i) => {
if (!byOwner.has(provider.id)) byOwner.set(provider.id, values[i])
})
return buildFeatureGate(byOwner)
},
// One dependency per provider — a fixed-length list, for the same reason the
// hook loop above is fixed-length.
// eslint-disable-next-line react-hooks/exhaustive-deps
[providers, ...values],
)
return <FeatureGateContext.Provider value={gate}>{children}</FeatureGateContext.Provider>
}
/**
* The predicate to filter nav rows with: `(item) => boolean`, true when the row
* carries no `feature` or when its module says this viewer may see it.
*
* Outside a provider it is the open gate, so a component rendered in isolation
* (a test, a preview) shows its whole nav rather than none of it.
*/
export function useFeatureGate() {
return useContext(FeatureGateContext)
}
export default ModuleFeaturesProvider

174
client/src/modules/nav.js Normal file
View File

@@ -0,0 +1,174 @@
// The interleave of module nav items into core's nav.
//
// Phase 2, PR 8 of docs/website/MODULE_SYSTEM.md §2.7 (§1.4 states the problem);
// the normative contract is docs/website/MODULE_API.md §3.3.
//
// **Module items join the BASE array, before anything else happens to it.** That
// is the whole design of this file and the override merge next door forces it:
// `applyNavOverrides` / `buildPublicNav` are keyed by `to` and drop any key the
// base array does not declare (lib/navOverrides.js — deliberately, so a deleted
// route cannot leave a stale row doing something unexpected later). Append
// module items *after* that merge and they are unreachable to Admin →
// Navigation: unorderable, unrelabellable, unhideable. Today's UO rows are all
// three of those things, so appending would make the extraction a visible
// regression for every operator who has ever touched their nav.
//
// So the pipeline gains one step at the front and nothing else changes:
//
// withModuleNav(NAV, area) → admin overrides → role/feature filter → rendered
//
// and the filter stays last, which is what keeps it the boundary an override
// cannot cross (THEMING_AND_NAV.md §7). MODULE_API.md §3.3 wrote those last two
// the other way round; the code is right and the contract was amended.
//
// The result is that a module row is, to everything downstream, an ordinary row.
// Nothing in navOverrides.js, NavEditor.jsx or the layouts knows a module exists.
import { navFor } from './registry.js'
import { isGrouped } from '../lib/navOverrides.js'
// Rows with no group of their own are collected under this key. A Symbol rather
// than a string so it cannot collide with a group an admin or a module names.
const UNGROUPED = Symbol('ungrouped')
/**
* Sort by effective position, where a row that asked for nothing keeps the index
* it already had. Three tie-breaks, in this order: an explicit `order` beats a
* coincidental index (the module said "third", so third), and two explicit
* orders keep registration order, which `navFor` has already put in scan order.
*
* The same rule byOrder/place use in lib/navOverrides.js, and it has to be — an
* admin who then drags that row is editing the position this produced.
*/
function place(entries) {
return entries
.map((entry, index) => ({ ...entry, index }))
.sort((a, b) => a.key - b.key || Number(b.explicit) - Number(a.explicit) || a.index - b.index)
.map(({ item }) => item)
}
function entryFor(item, fallbackKey) {
return { item, key: item.order ?? fallbackKey, explicit: item.order !== undefined }
}
function coreEntries(items) {
return items.map((item, index) => ({ item, key: index, explicit: false }))
}
/** The `to`s a base nav already claims, flat or grouped. */
function claimedPaths(baseNav, grouped) {
return new Set(grouped ? baseNav.flatMap((g) => g.items.map((i) => i.to)) : baseNav.map((i) => i.to))
}
/**
* Drop a module row whose `to` is already on the nav, and say so.
*
* Not a policy about where a module may link — it is that `to` is the KEY the
* override layer stores under and React renders by. Two rows sharing one would
* give an admin a single editor row that silently moves both, and a duplicate
* key in the rendered list. Dropping the newcomer keeps core's row, which is the
* one any existing override was written against.
*
* Fail-safe like every other read in this area: the offending row goes, its
* neighbours stay.
*/
function withoutCollisions(items, claimed) {
const out = []
for (const item of items) {
if (!item || typeof item.to !== 'string' || !item.to) continue
if (claimed.has(item.to)) {
console.warn(
`[modules] nav item "${item.to}" from module "${item.moduleId}" collides with an existing row and was dropped`,
)
continue
}
claimed.add(item.to)
out.push(item)
}
return out
}
// The flat navs — the public header and the player portal.
//
// No groups, so `order` is a position in the one list: core rows are keyed by
// their index and a module row by the `order` it asked for. A module row with no
// order appends after the coded ones, in registration order, rather than jumping
// to the front on a 0 default — the same choice buildPublicNav makes for an
// admin-created link.
function mergeFlat(baseNav, items) {
return place([...coreEntries(baseNav), ...items.map((item, i) => entryFor(item, baseNav.length + i))])
}
// The grouped nav — the admin sidebar.
//
// `group` names an existing core group and the row lands inside it: Moderation
// and System, where today's UO rows already sit (§1.4). An unknown group name
// creates a group at the end rather than dropping the row — a typo must cost a
// position, never a link. A row with no `group` at all lands in a trailing
// untitled group, which renders as ungrouped links; core does not invent a
// display title out of a module id.
//
// An ungrouped row is NOT folded into one of core's own untitled groups
// (Dashboard's, Account's): those are furniture pinned to the top and bottom of
// the sidebar, and a module page does not belong beside "Account".
//
// A group created here is a group as far as everything downstream is concerned,
// including as a destination in Admin → Navigation's "move to section" control:
// `readOverrides` builds its set of legal destinations from the base nav it is
// handed, which is this one.
function mergeGrouped(baseNav, items) {
const titles = new Set(baseNav.map((g) => g.title).filter((t) => typeof t === 'string'))
const into = new Map() // existing group title → rows
const fresh = new Map() // new group title (or UNGROUPED) → rows, first-seen order
for (const item of items) {
const named = typeof item.group === 'string' && item.group ? item.group : null
const key = named ?? UNGROUPED
const bucket = named !== null && titles.has(named) ? into : fresh
if (!bucket.has(key)) bucket.set(key, [])
bucket.get(key).push(item)
}
const kept = baseNav.map((g) => {
const incoming = into.get(g.title)
if (!incoming) return g
return {
...g,
items: place([...coreEntries(g.items), ...incoming.map((item, i) => entryFor(item, g.items.length + i))]),
}
})
const created = [...fresh.entries()].map(([key, rows]) => {
const items_ = place(rows.map((item, i) => entryFor(item, i)))
return key === UNGROUPED ? { items: items_ } : { title: key, items: items_ }
})
return [...kept, ...created]
}
/**
* The base nav a layout should render: core's coded array with every installed
* module's rows for this area interleaved into it.
*
* Returns `baseNav` ITSELF when no module registered anything for this area, so
* an instance with no modules installed renders the identical array it renders
* today — the same "untouched path" guarantee applyNavOverrides makes, and what
* makes a `useMemo` with an empty dependency list around this call honest.
*
* Safe to call once per component and cache: registration completes before the
* first render (main.jsx waits for DOMContentLoaded — MODULE_API.md §3.1) and
* there is no unregistering, so this answer cannot change during a session.
*
* @param {Array} baseNav the coded NAV, flat or grouped
* @param {'public'|'admin'|'player'} area
* @returns {Array} a nav of the same shape
*/
export function withModuleNav(baseNav, area) {
if (!Array.isArray(baseNav)) return []
const grouped = isGrouped(baseNav)
const items = withoutCollisions(navFor(area), claimedPaths(baseNav, grouped))
if (items.length === 0) return baseNav
return grouped ? mergeGrouped(baseNav, items) : mergeFlat(baseNav, items)
}
export default withModuleNav

221
client/src/modules/registry.js Normal file
View File

@@ -0,0 +1,221 @@
// ── The client-side module registry ────────────────────────────────────────
//
// Phase 2, PR 7 of docs/website/MODULE_SYSTEM.md §2.7. The normative contract is
// docs/website/MODULE_API.md §3.3; where the two disagree, the contract wins.
//
// A module's prebuilt chunk registers its routes, its nav entries and its feature
// provider here, and core reads them back. This is the client twin of the
// server's modules/loader.js — with one structural difference worth stating,
// because it is what makes the file this short: core *hands* the registry to the
// module (on `window.__rg`, see shared.js) rather than discovering it. There is
// nothing to scan, nothing to validate a manifest against, and no failure mode
// where half a module is registered.
//
// **Timing is the whole design.** Module chunks are `<script type="module" src>`
// tags the server injects before `</body>` (server/src/utils/htmlShell.js), after
// core's own bundle. Module scripts are deferred, so they evaluate after that
// bundle has run — which is where `window.__rg` is published — and all of them
// finish before DOMContentLoaded. main.jsx waits for that same event before
// calling render(), so registration is complete before React reads any of this.
//
// That is what buys the simplicity here: registration is a plain synchronous
// write with no subscribers, not an observable store, because nothing can
// register after the first render. If that ever stops being true it changes in
// this file and in main.jsx, not in a dozen consumers.
//
// What PR 7 wires up is `routesFor` (App.jsx). `navFor` and `featureProviderFor`
// are stored and returned faithfully but core does not read them yet — PR 8 adds
// the nav interleave and the feature-provider seam. Storing them is not the kind
// of accepting stub the server's registries refused to be: nothing is discarded
// here, so a module that registers nav in this core gets it back from `navFor`.
const routes = { public: [], admin: [], player: [] }
const nav = { public: [], admin: [], player: [] }
const providers = new Map()
// slot name → { Component, filledBy }.
const slots = new Map()
const registered = new Set()
const AREAS = ['public', 'admin', 'player']
function assertArea(area, call) {
if (!AREAS.includes(area)) throw new Error(`${call}: unknown area "${area}"`)
}
/**
* Route components, by area.
*
* @param {string} id the module id — the URL segment its routes are namespaced under
* @param {{public?: Array, admin?: Array, player?: Array}} byArea
* each entry `{ path, element, gate? }`. `path` is relative to the module's
* namespace; core prefixes it and mounts it inside the area's existing wrapper
* (`/<id>/…` under MaintenanceGate, `/admin/<id>/…` under RequireAuth +
* AdminLayout, `/player/<id>/…` under RequirePlayer + PlayerPortalLayout).
* `gate` is an optional `{ roles: [...] }` that core applies as its own
* RoleGate — a module cannot supply an auth wrapper, because the sidebar and
* the route table have to agree about who may see what (§3.3).
*/
export function registerRoutes(id, byArea) {
for (const [area, list] of Object.entries(byArea || {})) {
assertArea(area, 'registerRoutes')
for (const route of list || []) {
// Prefixed HERE rather than by the module: a module cannot claim a path
// outside its own namespace however it spells `path` — a leading `/`, a
// trailing one, or several — because it never gets to write the segment
// its routes hang under.
const path = `${id}/${String(route.path || '').replace(/^\/+/, '')}`.replace(/\/+$/, '')
routes[area].push({ ...route, path, moduleId: id })
}
}
registered.add(id)
}
/**
* Nav entries, interleaved into CORE groups rather than appended as a block.
*
* Today's UO items sit inside core's own Moderation and System groups; a "UO"
* group at the bottom of the sidebar would be a visible regression on the day
* the module is extracted (MODULE_SYSTEM.md §1.4). `group` names an existing
* core group, `order` sorts within it, and an unknown group name appends rather
* than dropping the item — a mis-typed group must cost a position, never a link.
*
* @param {string} id
* @param {{area: string, items: Array<{label, to, group?, order?, roles?, feature?}>}} spec
*/
export function registerNav(id, spec) {
const { area, items } = spec || {}
assertArea(area, 'registerNav')
for (const item of items || []) nav[area].push({ ...item, moduleId: id })
registered.add(id)
}
/**
* The hook that answers "which of this module's features may this viewer see".
*
* Core keeps a generic flag context and owns none of the semantics — `uo` fills
* its namespace with today's `useShardFeatures` (MODULE_SYSTEM.md §1.5). With no
* module installed the nav filter is a correct no-op, because no core nav item
* carries a `feature` today.
*/
export function registerFeatureProvider(id, namespace, hook) {
providers.set(namespace, { id, hook })
registered.add(id)
}
// ── Extension slots (§3.7) ─────────────────────────────────────────────────
//
// The client twin of the server's declareSlot/registerExtension, and the same
// rule in both halves: core declares a slot, ONLY core declares one, and at most
// one module fills it. Core renders `<Slot name>` (Slot.jsx) and gets nothing
// back when the slot is unfilled — so an instance with no module installed
// renders exactly what it renders today.
//
// A slot is named for a PLACE, never for a meaning. `site.footer.status` is a
// position in the footer and the styling that goes with it; the label, the
// target, the data and whether anything renders at all are the module's. The
// moment core types a slot by its content it has re-acquired the game semantics
// this whole extraction removes.
/**
* @param {string} name the slot id. Core-only — deliberately not on the
* `registry` object handed to modules.
*/
export function declareSlot(name) {
if (slots.has(name)) throw new Error(`extension slot "${name}" already declared`)
slots.set(name, { Component: null, filledBy: null })
}
/**
* Fill a declared slot with a component.
*
* **This is the one place the client registry is not fail-open**, and the
* asymmetry is deliberate. A dropped nav row costs a link the viewer can reach
* another way; a silently dropped extension is invisible to everyone including
* its author. So an unknown slot, a non-component, and a second fill all throw —
* exactly as the server's checkExtensionShape does.
*
* A throw here is always a programming error and never a race, because
* declaration structurally precedes filling: core declares in main.jsx, inside
* its own bundle, and every module chunk is a deferred script injected after it
* (§3.1).
*/
export function registerExtension(id, slot, Component) {
const entry = slots.get(slot)
if (!entry) throw new Error(`registerExtension: unknown extension slot "${slot}"`)
if (typeof Component !== 'function') throw new Error(`registerExtension: ${slot} is not a component`)
if (entry.filledBy) throw new Error(`extension slot "${slot}" is already filled by "${entry.filledBy}"`)
entry.Component = Component
entry.filledBy = id
registered.add(id)
}
/**
* The filling component, or null.
*
* Read by Slot.jsx and nothing else — deliberately. There is no `hasExtension`
* for a core layout to branch on, because a layout that asks whether a slot is
* filled and then renders its own decoration alongside gets the *failed* case
* wrong: the extension is filled, so the decoration renders, and the component
* then throws into the boundary leaving the decoration behind on its own. Core
* decorates through `<Slot wrap>` instead, which puts the decoration inside the
* boundary where it shares the extension's fate. (Found in a browser, with the
* footer's separator.)
*
* Undeclared and unfilled both read null: reading is fail-safe, and only writing
* is strict.
*/
export const extensionFor = (slot) => (slots.get(slot) || {}).Component || null
export const routesFor = (area) => routes[area] || []
// Sorted by the `order` a module asked for. Array#sort is stable in every engine
// this ships to, so two modules asking for the same slot keep load order —
// which is alphabetical by id, the same order the server scans in (§4.2).
export const navFor = (area) =>
[...(nav[area] || [])].sort((a, b) => (a.order ?? 100) - (b.order ?? 100))
export const featureProviderFor = (namespace) => providers.get(namespace)
/**
* Every registered provider, for core's feature context to call.
*
* Exported from the module but deliberately NOT a member of the `registry`
* object below: a module asks for a namespace it knows the name of, and has no
* business enumerating what everyone else registered. Core needs the list
* because it has to call each hook — unconditionally, in a fixed order, at the
* top of a component (modules/features.jsx).
*/
export const featureProviders = () =>
[...providers.entries()].map(([namespace, { id, hook }]) => ({ id, namespace, hook }))
export const registeredIds = () => [...registered]
/** Test seam. Nothing in the app calls this — there is no unregistering. */
export function _reset() {
for (const area of AREAS) {
routes[area].length = 0
nav[area].length = 0
}
providers.clear()
// Declarations go too, unlike the server's, where a slot is declared once at
// require time by the router that owns it. Core declares its slots in
// main.jsx — the one file no test loads — so on this side there is nothing
// declared at import time for a surviving declaration to protect.
slots.clear()
registered.clear()
}
// The object handed to modules on window.__rg.registry. Deliberately the write
// calls plus the read ones: a module reading `routesFor` is how it finds out
// another module is installed, which is the only supported form of module-to-
// module awareness (there is no dependency resolution).
export const registry = {
registerRoutes,
registerNav,
registerFeatureProvider,
registerExtension,
routesFor,
navFor,
featureProviderFor,
registeredIds,
}

96
client/src/modules/shared.js Normal file
View File

@@ -0,0 +1,96 @@
// ── window.__rg — the shared-dependency global ─────────────────────────────
//
// Phase 2, PR 7 of docs/website/MODULE_SYSTEM.md §2.7; the normative shape is
// docs/website/MODULE_API.md §3.2.
//
// A module's client half is a PREBUILT ESM chunk — the operator never builds
// anything (MODULE_SYSTEM.md §1.14) — served same-origin and loaded under
// `script-src 'self'` with no 'unsafe-inline'. That combination is what rules out
// an import map: an import map has to be an inline `<script type="importmap">`,
// and the policy forbids inline scripts outright. So the shared dependencies ride
// on a global, and the module's Rollup externals are aliased to two-line shims
// that re-export from it (§3.6).
//
// **There is exactly one React in the page and core owns it.** A module that
// bundled its own would get a second hook dispatcher and fail at its first
// useState. That is the same rule the server half enforces for `express` and
// `express-validator` on `ctx`, and for the same reason: anything shared between
// core and a module is owned by core and HANDED OVER, never resolved by the
// module.
import * as react from 'react'
import * as reactDom from 'react-dom/client'
import * as router from 'react-router-dom'
// The automatic JSX runtime, and it is not decoration. A module's bundler
// compiles every .jsx file to imports from `react/jsx-runtime` under the modern
// default, and those have to resolve to CORE's React like every other import.
// Without it here a module would have to build with `jsxRuntime: 'classic'`;
// with it, a module uses the default its tooling already assumes.
import * as jsxRuntime from 'react/jsx-runtime'
import { registry } from './registry.js'
import { MODULE_API_VERSION } from './version.js'
import PublicLayout from '../components/PublicLayout.jsx'
import PageHeader from '../components/PageHeader.jsx'
import { Loading, ErrorState, EmptyState } from '../components/PageState.jsx'
import { useAsync } from '../lib/useAsync.js'
import { useAuth } from '../contexts/AuthContext.jsx'
import { useSite } from '../contexts/SiteContext.jsx'
import { request, ApiError } from '../api/client.js'
// The UI kit is CURATED AND CLOSED (§3.4), not a re-export of components/. These
// seven are what the smallest UO page already needs beyond React and the router:
// without them a module either reaches into core's tree — violating the
// zero-import rule the whole boundary rests on — or ships its own copies, which
// means a module page that does not look like the site it is installed in, and
// that drifts further every time core's layout changes.
//
// Adding a member is a MINOR MODULE_API_VERSION bump; changing a member's props
// is a MAJOR one. That is a real constraint on core's own refactoring and it is
// the price of the boundary being worth anything.
//
// `AdminPage` appears in §3.4's table and is deliberately absent: core has no
// such component — admin views are plain markup inside AdminLayout — and
// inventing one to satisfy a table would be a core change with no consumer until
// Phase 3. The contract is amended rather than the code padded, and adding it
// later costs a minor bump, which is exactly the case the versioning is for.
const ui = {
PublicLayout,
PageHeader,
Loading,
ErrorState,
EmptyState,
useAsync,
useAuth,
useSite,
}
// The request PRIMITIVE, not the `api` object (§3.5). `api.atlas` and `api.shard`
// are module bindings that only still live in core's client because Phase 3 has
// not moved them; a module builds its own namespace over `request` and owns the
// paths it calls — which is right, because it owns the routes at the other end.
const api = { request, ApiError }
/**
* Publish `window.__rg`. Called by main.jsx before it renders, and before any
* module chunk evaluates.
*
* Frozen, one level down as well as at the top: the object a module reaches for
* its React is not somewhere a module gets to leave something for the next one.
* Cross-module communication is a thing the contract does not have, and an
* unfrozen global is how a codebase acquires one by accident.
*/
export function publishSharedDependencies() {
window.__rg = Object.freeze({
version: MODULE_API_VERSION,
react,
reactDom,
router,
jsxRuntime,
registry,
ui: Object.freeze(ui),
api: Object.freeze(api),
})
return window.__rg
}

23
client/src/modules/version.js Normal file
View File

@@ -0,0 +1,23 @@
// The client's copy of MODULE_API_VERSION. It must equal the server's
// (server/src/modules/version.js) — the two halves version ONE contract
// (docs/website/MODULE_API.md §1.1), and a module checks whichever half it is
// talking to: `coreApi` against the server's at load time, `window.__rg.version`
// against the client's before it registers anything.
//
// Duplicated rather than fetched, and that is deliberate. The value has to be on
// `window.__rg` before the first module chunk evaluates, which is earlier than
// any network round trip could answer — a fetched version would mean either an
// await before render or a module reading `undefined`. The cost of the copy is
// that the two files can drift, so a test asserts they agree
// (client/test/moduleRegistry.test.js) rather than trusting a bump to remember
// both.
// 1.2.0 — `registry` gained `registerExtension` and core gained extension slots
// (MODULE_API.md §3.7). The first change to window.__rg since 1.0.0, and an
// addition: a module that never fills a slot is unaffected. The server half is
// untouched and bumps anyway, for the reason below.
// 1.1.0 — the server's ctx gained activity.log, users.getById, site.baseUrl and
// the rate-limit factory (MODULE_API.md §2.3). Nothing on window.__rg changed,
// but the two halves state ONE version: a module declares a single coreApi range
// and is served one chunk, so a client that claimed 1.0.0 while the server
// answered 1.1.0 would be two answers to one question.
export const MODULE_API_VERSION = '1.2.0'

71
client/src/routes/admin/AdminLayout.jsx
View File

@@ -6,6 +6,9 @@ import { useAuth } from '../../contexts/AuthContext.jsx'
import { useSite } from '../../contexts/SiteContext.jsx'
import { applyNavOverrides } from '../../lib/navOverrides.js'
import { useNavOverrides } from '../../lib/useNavOverrides.js'
import { withModuleNav } from '../../modules/nav.js'
import { useFeatureGate } from '../../modules/features.jsx'
import { navItemVisibleTo, allowedPathsFor, isAllowedPath } from '../../lib/adminNav.js'
// Small inline stroke icons (16px, currentColor) — same style as ProviderIcon.
// One shared frame keeps them terse; each item just supplies its path(s).
@@ -104,10 +107,6 @@ export const NAV = [
const COLLAPSE_KEY = 'admin.nav.collapsed'
// Moderators only get the moderation section (Discord + in-game ops) + their
// own account security.
const MOD_PATHS = ['/admin/moderation', '/admin/moderation/appeals', '/admin/shard-ops', '/admin/houses', '/admin/account']
// The one row an override may never hide: the nav editor itself, which is the
// only screen that can un-hide anything. The write path already refuses it
// (server/src/utils/navOverrides.js) and the editor's own toggle is disabled —
@@ -123,15 +122,11 @@ function keepEditorReachable(overrides) {
return { ...overrides, [UNHIDEABLE]: rest }
}
// Who may see a sidebar row. The single authority for that question: the layout
// applies it after the override merge (overrides are presentation, this is the
// boundary — §7), and Admin -> Navigation applies it to build its palette, so an
// admin is never offered a row they cannot themselves see (§8.1).
export function navItemVisibleTo(item, role) {
if (item.roles && !item.roles.includes(role)) return false
if (role === 'moderator') return MOD_PATHS.includes(item.to)
return true
}
// Who may see a sidebar row, and where that lets them go, both derived from the
// row's own `roles` — lib/adminNav.js, which is where the two hardcoded path
// lists this component used to carry went (MODULE_SYSTEM.md §1.4). Re-exported
// because Admin -> Navigation has always imported it from here.
export { navItemVisibleTo }
const TITLES = {
'/admin': 'Dashboard',
@@ -159,6 +154,19 @@ const TITLES = {
'/admin/account': 'Account Security',
}
// An installed module's admin pages are not in TITLES and cannot be — core does
// not know what they are called. Their nav row does, so the row is the title:
// the longest matching module row wins, so a detail page under a section titles
// as that section rather than falling through to a bare "Admin". Restricted to
// rows a module registered, which is what keeps every core path resolving
// through TITLES and sectionTitle exactly as it does today.
function moduleTitle(baseNav, pathname) {
return baseNav
.flatMap((g) => g.items)
.filter((i) => i.moduleId && (pathname === i.to || pathname.startsWith(`${i.to}/`)))
.sort((a, b) => b.to.length - a.to.length)[0]?.label
}
// Fallback page title for dynamic sub-routes not in the exact-match TITLES map.
function sectionTitle(pathname) {
if (pathname.startsWith('/admin/moderation')) return 'Moderation'
@@ -186,7 +194,15 @@ export default function AdminLayout() {
const navOverrides = useNavOverrides()
const navigate = useNavigate()
const location = useLocation()
const title = TITLES[location.pathname] || sectionTitle(location.pathname)
const isVisible = useFeatureGate()
// Core's rows plus every installed module's, before the override merge sees
// them — so a module row is editable in Admin -> Navigation like any other
// (modules/nav.js). Computed once: the registry is fixed before the first
// render and nothing unregisters.
const baseNav = useMemo(() => withModuleNav(NAV, 'admin'), [])
const title =
TITLES[location.pathname] || moduleTitle(baseNav, location.pathname) || sectionTitle(location.pathname)
// The hero canvas editor needs room — let it use the full content width.
const wide = location.pathname === '/admin/hero'
const modeDot = mode === 'live' ? 'var(--mode-live)' : 'var(--mode-maint)'
@@ -200,11 +216,17 @@ export default function AdminLayout() {
// NAV itself and this is exactly the code that ran before the feature.
const navGroups = useMemo(
() =>
applyNavOverrides(NAV, keepEditorReachable(navOverrides.nav_admin))
.map((g) => ({ ...g, items: g.items.filter((item) => navItemVisibleTo(item, user?.role)) }))
applyNavOverrides(baseNav, keepEditorReachable(navOverrides.nav_admin))
.map((g) => ({
...g,
// `isVisible` is a no-op for every core row — none carries a `feature`
// — and is applied here so that a module row which does carry one is
// gated on the sidebar rather than silently advertised.
items: g.items.filter((item) => navItemVisibleTo(item, user?.role) && isVisible(item)),
}))
// Drop any now-empty group so an empty category header never renders.
.filter((g) => g.items.length > 0),
[navOverrides.nav_admin, user?.role],
[baseNav, navOverrides.nav_admin, user?.role, isVisible],
)
// Accordion: track which titled categories are collapsed. Persist across
@@ -231,17 +253,22 @@ export default function AdminLayout() {
g.title && g.items.some((i) => (i.end ? location.pathname === i.to : location.pathname.startsWith(i.to)))
)?.title
// Where a moderator may go, from the same `roles` that decide what they see.
// It used to be a third hardcoded list — a prefix check over three paths —
// which disagreed with the sidebar's own five-path allowlist: `/admin/houses`
// was on the sidebar and not in the redirect, so a moderator who clicked
// Houses in their own nav was bounced straight back to Moderation. One
// derivation cannot disagree with itself, which is the point of deriving it.
const allowed = useMemo(() => allowedPathsFor(baseNav, user?.role), [baseNav, user?.role])
// Confine a moderator who deep-links (or is redirected to the index) to a page
// outside their remit — the API would 403 anyway, so send them to their home.
useEffect(() => {
if (!isModerator) return
const p = location.pathname
const allowed =
p.startsWith('/admin/moderation') || p.startsWith('/admin/shard-ops') || p === '/admin/account'
if (!allowed) {
if (!isAllowedPath(location.pathname, allowed)) {
navigate('/admin/moderation', { replace: true })
}
}, [isModerator, location.pathname, navigate])
}, [isModerator, location.pathname, navigate, allowed])
// Keep the admin out of search indexes (belt-and-suspenders with robots.txt).
useEffect(() => {

39
client/src/routes/admin/views/NavEditor.jsx
View File

@@ -13,7 +13,8 @@ import { Loading, ErrorState } from '../../../components/PageState.jsx'
import { api } from '../../../api/client.js'
import { useAuth } from '../../../contexts/AuthContext.jsx'
import { useSite } from '../../../contexts/SiteContext.jsx'
import { useShardFeatures, canSee } from '../../../lib/useShardFeatures.js'
import { withModuleNav } from '../../../modules/nav.js'
import { useFeatureGate } from '../../../modules/features.jsx'
import { buildNavRows, buildNavOverrides, buildPublicNav, buildPublicNavOverrides } from '../../../lib/navOverrides.js'
import PublicNavTree from './PublicNavTree.jsx'
import { parseJsonSetting } from '../../../lib/settingsJson.js'
@@ -244,7 +245,7 @@ export function Row({ row, id, destinations, destination, onDestination, onChang
export default function NavEditor() {
const { user } = useAuth()
const { refresh: refreshSite } = useSite()
const shardFeatures = useShardFeatures()
const isVisible = useFeatureGate()
const [tab, setTab] = useState('nav_public')
// Per nav: the editable groups, the overrides as loaded (so a row this admin
// cannot see survives their save), and whether a settings row exists at all.
@@ -255,25 +256,39 @@ export default function NavEditor() {
const [saved, setSaved] = useState('')
const [dirty, setDirty] = useState({})
// The palette: each base nav, filtered to what THIS admin can see (§8.1). The
// public nav's gates are the shard-feature ones; the admin nav's are roles.
// The player portal has no gates at all.
// The nav as coded, unfiltered. The palette below is what this admin may EDIT;
// this is what still EXISTS, and the two are different questions. Saving needs
// both: an entry for a row their palette filtered out must be carried through
// rather than reset, and only an entry for a route the code no longer declares
// at all should be dropped.
const fullNavs = { nav_public: PUBLIC_NAV, nav_admin: ADMIN_NAV, nav_player: PLAYER_NAV }
//
// Each nav is the coded array with every installed module's rows already
// interleaved (modules/nav.js) — the same array the layout renders, which is
// what makes a module row editable here at all: the override merge is keyed by
// `to` and drops a key the base it is handed does not declare, so a nav built
// from core alone would silently discard every stored override on a module row
// the moment it was saved.
const fullNavs = useMemo(
() => ({
nav_public: withModuleNav(PUBLIC_NAV, 'public'),
nav_admin: withModuleNav(ADMIN_NAV, 'admin'),
nav_player: withModuleNav(PLAYER_NAV, 'player'),
}),
[],
)
// The palette: each base nav, filtered to what THIS admin can see (§8.1). Two
// gates, and neither is core's own opinion any more — `roles` on a row, and
// the owning module's answer for a row that names a `feature`.
const palettes = useMemo(
() => ({
nav_public: PUBLIC_NAV.filter((item) => !item.feature || canSee(shardFeatures, item.feature)),
nav_admin: ADMIN_NAV.map((g) => ({ ...g, items: g.items.filter((i) => navItemVisibleTo(i, user?.role)) })).filter(
(g) => g.items.length > 0,
),
nav_player: PLAYER_NAV,
nav_public: fullNavs.nav_public.filter(isVisible),
nav_admin: fullNavs.nav_admin
.map((g) => ({ ...g, items: g.items.filter((i) => navItemVisibleTo(i, user?.role) && isVisible(i)) }))
.filter((g) => g.items.length > 0),
nav_player: fullNavs.nav_player.filter(isVisible),
}),
[shardFeatures, user?.role],
[fullNavs, isVisible, user?.role],
)
useEffect(() => {

23
client/src/routes/admin/views/PostEditor.jsx
View File

@@ -177,14 +177,15 @@ const delStyle = {
}
// ── Announcement status panel ────────────────────────────────────────────────
// Shows the town-crier + Discord delivery state for a published news post and
// offers a per-leg retry (useful after fixing the sidecar / news channel without
// re-publishing). Only rendered for news posts in edit mode; renders nothing
// until the post has actually been announced (no job row yet → nothing to show).
const LEG_META = {
towncrier: { label: 'In-game town crier' },
discord: { label: 'Discord #news' },
}
// Shows each delivery leg's state for a published news post and offers a per-leg
// retry (useful after fixing the sidecar / news channel without re-publishing).
// Only rendered for news posts in edit mode; renders nothing until the post has
// actually been announced (no job row yet → nothing to show).
//
// The legs and their labels come from the JOB, not from a constant here: which
// legs exist is decided by what the server has registered, so an installed module
// brings its own leg and this panel renders it with no client change
// (docs/website/MODULE_SYSTEM.md §1.8).
const STATUS_STYLE = {
done: { color: '#7bbf8f', label: 'delivered' },
pending: { color: '#d9b84a', label: 'pending' },
@@ -227,14 +228,12 @@ function AnnouncePanel({ postId }) {
return (
<div style={panelStyle}>
<span className="field-label" style={{ marginBottom: 2 }}>Announcement</span>
{['towncrier', 'discord'].map((leg) => {
const status = job[`${leg}_status`]
const err = job[`${leg}_last_error`]
{(job.legs || []).map(({ leg, label, status, last_error: err }) => {
const s = STATUS_STYLE[status] || STATUS_STYLE.pending
return (
<div key={leg} style={{ display: 'flex', flexDirection: 'column', gap: 3 }}>
<div style={{ display: 'flex', alignItems: 'center', gap: 8 }}>
<span className="sans" style={{ fontSize: '0.85rem', minWidth: 140 }}>{LEG_META[leg].label}</span>
<span className="sans" style={{ fontSize: '0.85rem', minWidth: 140 }}>{label}</span>
<span className="sans" style={{ fontSize: '0.8rem', color: s.color, fontWeight: 600 }}>● {s.label}</span>
{status !== 'done' && (
<button

147
client/src/routes/admin/views/UserDetail.jsx
View File

@@ -1,17 +1,16 @@
import { useCallback, useEffect, useMemo, useState } from 'react'
import { useCallback, useEffect, useState } from 'react'
import { useParams, Link } from 'react-router-dom'
import { Loading, ErrorState } from '../../../components/PageState.jsx'
import { useAsync } from '../../../lib/useAsync.js'
import { dateTime, ago } from '../../../lib/format.js'
import { dateTime } from '../../../lib/format.js'
import { api } from '../../../api/client.js'
import CharacterStats from '../../../components/CharacterStats.jsx'
import GameAccounts from '../../../components/GameAccounts.jsx'
import VendorSales from '../../../components/VendorSales.jsx'
import Slot from '../../../modules/Slot.jsx'
// Admin read-only view of one user's shard (uo-link) footprint: linked game
// accounts + character rosters, currently-online characters, houses (incl.
// IDOC) and recent vendor sales — everything scoped to that user's accounts.
// Reached from the Users table's "View" action; Edit stays a separate modal.
// Admin view of one user: who they are, their security posture (trusted devices
// and MFA), and then whatever the installed module contributes about them —
// today core's own UO footprint, via the `admin.users.detail` extension slot
// (MODULE_API.md §3.7). Reached from the Users table's "View" action; Edit stays
// a separate modal.
const ROLE_BADGE = {
admin: 'badge-admin',
@@ -28,114 +27,6 @@ function SectionTitle({ children }) {
)
}
// Currently-online characters on the user's accounts, with where they are. The
// per-character Online/Offline badge lives in the roster; this adds location.
function OnlineNow({ scope }) {
const { data } = useAsync(() => scope.online(), [scope])
if (!data) return null
return (
<section style={{ borderTop: '1px solid var(--line-soft)', marginTop: 30, paddingTop: 22 }}>
<SectionTitle>Online now</SectionTitle>
{data.length === 0 ? (
<p className="sans dim" style={{ margin: 0, fontSize: '0.86rem' }}>No characters online right now.</p>
) : (
<ul style={{ listStyle: 'none', margin: 0, padding: 0, display: 'flex', flexDirection: 'column', gap: 8 }}>
{data.map((c) => (
<li key={c.serial} className="sans" style={{ display: 'flex', justifyContent: 'space-between', gap: 12, fontSize: '0.9rem', color: 'var(--ink)' }}>
<span style={{ display: 'inline-flex', alignItems: 'center', gap: 8, minWidth: 0 }}>
<span style={{ width: 8, height: 8, borderRadius: '50%', background: '#7fd0a4', boxShadow: '0 0 6px #7fd0a4', flex: 'none' }} />
<span style={{ color: 'var(--head)' }}>{c.name || '(unnamed)'}</span>
</span>
<span className="dim" style={{ flex: 'none', fontSize: '0.8rem' }}>
{c.map != null ? `map ${c.map} · ${c.x}, ${c.y}` : '—'}
</span>
</li>
))}
</ul>
)}
</section>
)
}
// Shard "standing": city governorships held and guilds led by this user's
// accounts (both reliable current-state lookups). Renders nothing when empty.
function Standing({ scope }) {
const { data } = useAsync(() => scope.standing(), [scope])
if (!data) return null
const govs = data.governorOf || []
const guilds = data.guildsLed || []
if (govs.length === 0 && guilds.length === 0) return null
return (
<section style={{ borderTop: '1px solid var(--line-soft)', marginTop: 30, paddingTop: 22 }}>
<SectionTitle>Standing</SectionTitle>
<div style={{ display: 'flex', flexWrap: 'wrap', gap: 8 }}>
{govs.map((g) => (
<span key={`gov-${g.city}`} className="sans" style={{ fontSize: '0.78rem', padding: '4px 10px', borderRadius: 999, border: '1px solid #c9a24b55', color: '#c9a24b' }}>
Governor of {g.city}
</span>
))}
{guilds.map((g) => (
<span key={`guild-${g.id}`} className="sans" style={{ fontSize: '0.78rem', padding: '4px 10px', borderRadius: 999, border: '1px solid var(--accent)', color: 'var(--accent)' }}>
Guildmaster{g.abbr ? `, [${g.abbr}]` : ''} {g.name}
</span>
))}
</div>
</section>
)
}
// One house row — the many optional detail fields are gathered here so the
// Houses list stays a simple map.
function HouseRow({ house: h }) {
const location = h.region || (h.map != null ? `map ${h.map}` : 'unknown')
const coords = h.x != null ? ` · ${h.x}, ${h.y}` : ''
const owner = h.ownerAcct ? ` · ${h.ownerAcct}` : ''
const shares = h.coOwners || h.friends ? ` · ${h.coOwners || 0} co-owners, ${h.friends || 0} friends` : ''
return (
<li
style={{ display: 'flex', justifyContent: 'space-between', gap: 12, alignItems: 'baseline', padding: '12px 14px', border: '1px solid var(--line)', borderRadius: 10, background: 'rgba(255,255,255,0.02)' }}
>
<div style={{ minWidth: 0 }}>
<div className="sans" style={{ color: 'var(--head)', fontSize: '0.95rem' }}>
{h.name || 'Unnamed house'}
{h.isIdoc && <span className="badge" style={{ marginLeft: 8, background: '#5b2020', color: '#f0c8c2' }}>IDOC</span>}
</div>
<div className="sans dim" style={{ fontSize: '0.78rem', marginTop: 2 }}>
{location}
{coords}
{owner}
{shares}
</div>
</div>
<div className="sans dim" style={{ flex: 'none', fontSize: '0.78rem', textAlign: 'right' }}>
{(h.decay || h.stage) ? <div style={{ color: h.isIdoc ? '#e0928a' : 'var(--muted)' }}>{h.decay || h.stage}</div> : null}
{h.price != null ? <div style={{ fontVariantNumeric: 'tabular-nums' }}>{Number(h.price).toLocaleString()} gp</div> : null}
{h.lastRefreshed ? <div>refreshed {ago(h.lastRefreshed)}</div> : null}
</div>
</li>
)
}
// Houses owned by the user's accounts, IDOC first (flagged).
function Houses({ scope }) {
const { data } = useAsync(() => scope.houses(), [scope])
if (!data) return null
return (
<section style={{ borderTop: '1px solid var(--line-soft)', marginTop: 30, paddingTop: 22 }}>
<SectionTitle>Houses</SectionTitle>
{data.length === 0 ? (
<p className="sans dim" style={{ margin: 0, fontSize: '0.86rem' }}>No houses recorded for this user’s accounts.</p>
) : (
<ul style={{ listStyle: 'none', margin: 0, padding: 0, display: 'flex', flexDirection: 'column', gap: 10 }}>
{data.map((h) => (
<HouseRow key={h.serial} house={h} />
))}
</ul>
)}
</section>
)
}
// Admin security controls for one user: their trusted devices (view + revoke) and
// an MFA reset for a locked-out user. Every action is audit-logged server-side.
function SecurityAdmin({ userId }) {
@@ -244,25 +135,8 @@ function SecurityAdmin({ userId }) {
)
}
function ShardSections({ scope }) {
return (
<>
<CharacterStats scope={scope} />
<SectionTitle>Linked accounts &amp; characters</SectionTitle>
<GameAccounts scope={scope} readOnly moderation onUnlink={scope.unlink} charTo={(serial) => `/admin/characters/${serial}`} />
<Standing scope={scope} />
<OnlineNow scope={scope} />
<Houses scope={scope} />
<VendorSales fetchSales={scope.sales} />
</>
)
}
export default function UserDetail() {
const { id } = useParams()
// Memoize so the child components' effects (keyed on `scope`) don't refetch
// on every render.
const scope = useMemo(() => api.admin.userShard(id), [id])
const { loading, error, data: user } = useAsync(() => api.admin.getUser(id), [id])
if (loading) return <Loading />
@@ -296,7 +170,10 @@ export default function UserDetail() {
</div>
<SecurityAdmin userId={id} />
<ShardSections scope={scope} />
{/* Whatever the installed module has to say about this user, or nothing
at all. Core's own UO sections fill it today (UserShardSections.jsx,
registered in main.jsx) — MODULE_API.md §3.7. */}
<Slot name="admin.users.detail" userId={id} />
</section>
)
}

163
client/src/routes/admin/views/UserShardSections.jsx Normal file
View File

@@ -0,0 +1,163 @@
// ── Core's fill for the `admin.users.detail` extension slot ────────────────
//
// Phase 3, slice 2 of docs/website/MODULE_SYSTEM.md §2.7.1. Every section below
// is UO, and every one of them leaves core with the client half in slice 3 —
// this file exists so that when they do, core deletes a registration and a file
// instead of unpicking a page.
//
// Core registers it through the same seam a module uses
// (`registerExtension('core', …)` in main.jsx), which is the client twin of the
// server's `registries.registerCore()` and the same trick `useShardFlags`
// already uses for the feature seam. The mechanism is therefore exercised by
// core's own content from the day it lands, rather than first proved by the
// change that depends on it.
//
// The slot hands over `userId` and nothing else — deliberately, not `scope`.
// `api.admin.userShard` is a UO binding that leaves core in slice 3, so a slot
// that passed it would be handing a module something core is about to delete.
// An extension builds its own client for the routes it registered at the other
// end (MODULE_API.md §3.5), and this file does exactly what the module will.
import { useMemo } from 'react'
import { useAsync } from '../../../lib/useAsync.js'
import { ago } from '../../../lib/format.js'
import { api } from '../../../api/client.js'
import CharacterStats from '../../../components/CharacterStats.jsx'
import GameAccounts from '../../../components/GameAccounts.jsx'
import VendorSales from '../../../components/VendorSales.jsx'
// Its own copy, not an export from UserDetail.jsx: six lines of presentational
// furniture that is not in the §3.4 kit, so a module filling this slot would
// vendor the same thing. Core's copy stays behind with core's own security
// panel, which is the other caller.
function SectionTitle({ children }) {
return (
<div className="field-label" style={{ marginBottom: 12, marginTop: 4 }}>
{children}
</div>
)
}
// Currently-online characters on the user's accounts, with where they are. The
// per-character Online/Offline badge lives in the roster; this adds location.
function OnlineNow({ scope }) {
const { data } = useAsync(() => scope.online(), [scope])
if (!data) return null
return (
<section style={{ borderTop: '1px solid var(--line-soft)', marginTop: 30, paddingTop: 22 }}>
<SectionTitle>Online now</SectionTitle>
{data.length === 0 ? (
<p className="sans dim" style={{ margin: 0, fontSize: '0.86rem' }}>No characters online right now.</p>
) : (
<ul style={{ listStyle: 'none', margin: 0, padding: 0, display: 'flex', flexDirection: 'column', gap: 8 }}>
{data.map((c) => (
<li key={c.serial} className="sans" style={{ display: 'flex', justifyContent: 'space-between', gap: 12, fontSize: '0.9rem', color: 'var(--ink)' }}>
<span style={{ display: 'inline-flex', alignItems: 'center', gap: 8, minWidth: 0 }}>
<span style={{ width: 8, height: 8, borderRadius: '50%', background: '#7fd0a4', boxShadow: '0 0 6px #7fd0a4', flex: 'none' }} />
<span style={{ color: 'var(--head)' }}>{c.name || '(unnamed)'}</span>
</span>
<span className="dim" style={{ flex: 'none', fontSize: '0.8rem' }}>
{c.map != null ? `map ${c.map} · ${c.x}, ${c.y}` : '—'}
</span>
</li>
))}
</ul>
)}
</section>
)
}
// Shard "standing": city governorships held and guilds led by this user's
// accounts (both reliable current-state lookups). Renders nothing when empty.
function Standing({ scope }) {
const { data } = useAsync(() => scope.standing(), [scope])
if (!data) return null
const govs = data.governorOf || []
const guilds = data.guildsLed || []
if (govs.length === 0 && guilds.length === 0) return null
return (
<section style={{ borderTop: '1px solid var(--line-soft)', marginTop: 30, paddingTop: 22 }}>
<SectionTitle>Standing</SectionTitle>
<div style={{ display: 'flex', flexWrap: 'wrap', gap: 8 }}>
{govs.map((g) => (
<span key={`gov-${g.city}`} className="sans" style={{ fontSize: '0.78rem', padding: '4px 10px', borderRadius: 999, border: '1px solid #c9a24b55', color: '#c9a24b' }}>
Governor of {g.city}
</span>
))}
{guilds.map((g) => (
<span key={`guild-${g.id}`} className="sans" style={{ fontSize: '0.78rem', padding: '4px 10px', borderRadius: 999, border: '1px solid var(--accent)', color: 'var(--accent)' }}>
Guildmaster{g.abbr ? `, [${g.abbr}]` : ''} {g.name}
</span>
))}
</div>
</section>
)
}
// One house row — the many optional detail fields are gathered here so the
// Houses list stays a simple map.
function HouseRow({ house: h }) {
const location = h.region || (h.map != null ? `map ${h.map}` : 'unknown')
const coords = h.x != null ? ` · ${h.x}, ${h.y}` : ''
const owner = h.ownerAcct ? ` · ${h.ownerAcct}` : ''
const shares = h.coOwners || h.friends ? ` · ${h.coOwners || 0} co-owners, ${h.friends || 0} friends` : ''
return (
<li
style={{ display: 'flex', justifyContent: 'space-between', gap: 12, alignItems: 'baseline', padding: '12px 14px', border: '1px solid var(--line)', borderRadius: 10, background: 'rgba(255,255,255,0.02)' }}
>
<div style={{ minWidth: 0 }}>
<div className="sans" style={{ color: 'var(--head)', fontSize: '0.95rem' }}>
{h.name || 'Unnamed house'}
{h.isIdoc && <span className="badge" style={{ marginLeft: 8, background: '#5b2020', color: '#f0c8c2' }}>IDOC</span>}
</div>
<div className="sans dim" style={{ fontSize: '0.78rem', marginTop: 2 }}>
{location}
{coords}
{owner}
{shares}
</div>
</div>
<div className="sans dim" style={{ flex: 'none', fontSize: '0.78rem', textAlign: 'right' }}>
{(h.decay || h.stage) ? <div style={{ color: h.isIdoc ? '#e0928a' : 'var(--muted)' }}>{h.decay || h.stage}</div> : null}
{h.price != null ? <div style={{ fontVariantNumeric: 'tabular-nums' }}>{Number(h.price).toLocaleString()} gp</div> : null}
{h.lastRefreshed ? <div>refreshed {ago(h.lastRefreshed)}</div> : null}
</div>
</li>
)
}
// Houses owned by the user's accounts, IDOC first (flagged).
function Houses({ scope }) {
const { data } = useAsync(() => scope.houses(), [scope])
if (!data) return null
return (
<section style={{ borderTop: '1px solid var(--line-soft)', marginTop: 30, paddingTop: 22 }}>
<SectionTitle>Houses</SectionTitle>
{data.length === 0 ? (
<p className="sans dim" style={{ margin: 0, fontSize: '0.86rem' }}>No houses recorded for this user’s accounts.</p>
) : (
<ul style={{ listStyle: 'none', margin: 0, padding: 0, display: 'flex', flexDirection: 'column', gap: 10 }}>
{data.map((h) => (
<HouseRow key={h.serial} house={h} />
))}
</ul>
)}
</section>
)
}
export default function UserShardSections({ userId }) {
// Memoized so the child components' effects (keyed on `scope`) don't refetch
// on every render — the same reason UserDetail memoized it before this moved.
const scope = useMemo(() => api.admin.userShard(userId), [userId])
return (
<>
<CharacterStats scope={scope} />
<SectionTitle>Linked accounts &amp; characters</SectionTitle>
<GameAccounts scope={scope} readOnly moderation onUnlink={scope.unlink} charTo={(serial) => `/admin/characters/${serial}`} />
<Standing scope={scope} />
<OnlineNow scope={scope} />
<Houses scope={scope} />
<VendorSales fetchSales={scope.sales} />
</>
)
}

16
client/src/routes/player/PlayerPortalLayout.jsx
View File

@@ -6,6 +6,8 @@ import { useAuth } from '../../contexts/AuthContext.jsx'
import { useSite } from '../../contexts/SiteContext.jsx'
import { applyNavOverrides } from '../../lib/navOverrides.js'
import { useNavOverrides } from '../../lib/useNavOverrides.js'
import { withModuleNav } from '../../modules/nav.js'
import { useFeatureGate } from '../../modules/features.jsx'
// Shared shell for the logged-in player portal. Uses the same sidebar shell as
// Admin (icon nav, sticky content header, footer sign-out) so the two logged-in
@@ -35,9 +37,10 @@ const IconGear = () => <Icon><circle cx="12" cy="12" r="3" /><path d="M12 2v3M12
const IconShield = () => <Icon><path d="M12 3l7 3v5c0 5-3.5 8-7 10-3.5-2-7-5-7-10V6z" /><path d="M9 12l2 2 4-4" /></Icon>
// Exported because Admin -> Navigation edits this list. It stays declared here;
// the editor may only relabel, reorder and hide what it finds (§7). No row
// carries a gate — every player sees all three — so the merged result is what
// renders, with no filter after it.
// the editor may only relabel, reorder and hide what it finds (§7). No CORE row
// carries a gate — every player sees all three — but an installed module's rows
// join this list before the merge and may carry a `feature`, so the filter after
// it is not dead code.
export const NAV = [
{ to: '/player', label: 'Characters', end: true, icon: IconUser },
{ to: '/account/appeals', label: 'Appeals', icon: IconShield },
@@ -69,7 +72,12 @@ export default function PlayerPortalLayout() {
const { user, logout } = useAuth()
const { siteTitle } = useSite()
const navOverrides = useNavOverrides()
const nav = useMemo(() => applyNavOverrides(NAV, navOverrides.nav_player), [navOverrides.nav_player])
const isVisible = useFeatureGate()
const baseNav = useMemo(() => withModuleNav(NAV, 'player'), [])
const nav = useMemo(
() => applyNavOverrides(baseNav, navOverrides.nav_player).filter(isVisible),
[baseNav, navOverrides.nav_player, isVisible],
)
const navigate = useNavigate()
const location = useLocation()
const title =

125
client/test/adminNav.test.js Normal file
View File

@@ -0,0 +1,125 @@
import { test } from 'node:test'
import assert from 'node:assert/strict'
import { navItemVisibleTo, allowedPathsFor, isAllowedPath } from '../src/lib/adminNav.js'
// Moderator confinement, derived from each row's `roles` (Phase 2 PR 8 —
// MODULE_SYSTEM.md §1.4). This replaced two hardcoded path lists that had
// drifted apart from each other, so the tests worth having are the ones that
// pin what a moderator may now see and reach, and the shape of the match.
// The real sidebar, trimmed to the rows that decide something here.
const NAV = [
{ items: [{ to: '/admin', label: 'Dashboard', end: true, roles: ['admin', 'editor', 'moderator'] }] },
{
title: 'Moderation',
items: [
{ to: '/admin/moderation', label: 'Moderation', roles: ['admin', 'moderator'] },
{ to: '/admin/moderation/appeals', label: 'Appeals', roles: ['admin', 'moderator'] },
{ to: '/admin/shard-ops', label: 'In-Game Ops', roles: ['admin', 'moderator'] },
{ to: '/admin/houses', label: 'Houses', roles: ['admin', 'moderator'] },
],
},
{
title: 'System',
items: [
{ to: '/admin/users', label: 'Users', roles: ['admin'] },
{ to: '/admin/settings', label: 'Settings', roles: ['admin'] },
],
},
{
items: [
{ to: '/admin/characters', label: 'My Characters' },
{ to: '/admin/account', label: 'Account' },
],
},
]
const visibleTo = (role) =>
NAV.flatMap((g) => g.items)
.filter((i) => navItemVisibleTo(i, role))
.map((i) => i.to)
test('a row with no roles is visible to every staff role', () => {
// Self-service: staff are a superset of players, so a moderator reaching their
// own characters is not a privilege, it is the thing every account has.
for (const role of ['admin', 'editor', 'moderator']) {
assert.equal(navItemVisibleTo({ to: '/admin/account' }, role), true)
}
})
test('a role not named on the row cannot see it', () => {
assert.equal(navItemVisibleTo({ to: '/admin/users', roles: ['admin'] }, 'moderator'), false)
assert.equal(navItemVisibleTo({ to: '/admin/users', roles: ['admin'] }, 'admin'), true)
// An unknown or absent role sees only the ungated rows.
assert.equal(navItemVisibleTo({ to: '/admin/users', roles: ['admin'] }, undefined), false)
assert.equal(navItemVisibleTo({ to: '/admin/account' }, undefined), true)
})
test('what a moderator sees is exactly the moderation section, plus self-service', () => {
// The two additions the derivation makes over the old MOD_PATHS list are
// Dashboard — whose roles have always named moderator, so the two lists
// disagreed — and My Characters. Both are already permitted server-side.
assert.deepEqual(visibleTo('moderator'), [
'/admin',
'/admin/moderation',
'/admin/moderation/appeals',
'/admin/shard-ops',
'/admin/houses',
'/admin/characters',
'/admin/account',
])
})
test('an admin still sees everything and an editor still sees nothing extra', () => {
assert.equal(visibleTo('admin').length, NAV.flatMap((g) => g.items).length)
assert.deepEqual(visibleTo('editor'), ['/admin', '/admin/characters', '/admin/account'])
})
test('a row with `end` matches exactly — the dashboard is not a prefix', () => {
// The bug this shape exists to prevent: treating `/admin` as a prefix would
// make every path in the admin area allowed for anyone who can see Dashboard.
const allowed = allowedPathsFor(NAV, 'moderator')
assert.equal(isAllowedPath('/admin', allowed), true)
assert.equal(isAllowedPath('/admin/users', allowed), false)
assert.equal(isAllowedPath('/admin/users/12', allowed), false)
})
test('every other row covers its own sub-routes', () => {
const allowed = allowedPathsFor(NAV, 'moderator')
assert.equal(isAllowedPath('/admin/moderation/appeals/12', allowed), true)
assert.equal(isAllowedPath('/admin/characters/0x4001', allowed), true)
})
test('a sibling path that merely shares a prefix is NOT covered', () => {
const allowed = allowedPathsFor(NAV, 'moderator')
// `/admin/houses-secret` starts with `/admin/houses` as a string; the match is
// on path segments, so it does not start with `/admin/houses/`.
assert.equal(isAllowedPath('/admin/houses-secret', allowed), false)
assert.equal(isAllowedPath('/admin/houses/42', allowed), true)
})
test('Houses is reachable, which is the defect the derivation fixed', () => {
// The redirect used to allow only /admin/moderation*, /admin/shard-ops* and
// /admin/account, while the sidebar showed Houses — so a moderator clicking a
// row in their own nav was bounced back to Moderation.
const allowed = allowedPathsFor(NAV, 'moderator')
assert.equal(isAllowedPath('/admin/houses', allowed), true)
})
test('a module row a moderator may see is reachable without core listing it', () => {
// The reason this is derived at all: core cannot hardcode a path it has never
// heard of, and a module row arrives with `roles` like any other.
const withModule = [
...NAV,
{ title: 'Shard', items: [{ to: '/admin/uo/shard-ops', label: 'Ops', roles: ['admin', 'moderator'], moduleId: 'uo' }] },
]
const allowed = allowedPathsFor(withModule, 'moderator')
assert.equal(isAllowedPath('/admin/uo/shard-ops', allowed), true)
assert.equal(isAllowedPath('/admin/uo/shard-ops/queue', allowed), true)
})
test('a nav that is not there does not throw', () => {
assert.deepEqual(allowedPathsFor(null, 'moderator'), [])
assert.equal(isAllowedPath('/admin', undefined), false)
})

85
client/test/featureGate.test.js Normal file
View File

@@ -0,0 +1,85 @@
import { test } from 'node:test'
import assert from 'node:assert/strict'
import { buildFeatureGate, OPEN_GATE } from '../src/modules/featureGate.js'
// The feature seam's decision logic (MODULE_SYSTEM.md §1.5, MODULE_API.md §3.3).
// Every branch here fails OPEN, and that is the property under test as much as
// the happy path: this is presentation, the server is the gate, and a UI mistake
// that hides a page from someone entitled to it is worse in every case than one
// that shows a link which then 403s.
const flags = (...names) => new Set(names)
test('a row with no feature is always visible', () => {
const gate = buildFeatureGate(new Map([['uo', flags()]]))
assert.equal(gate({ to: '/site/news' }), true)
})
test('a core row resolves against the owner id `core`', () => {
// Core's ten shard-gated rows carry no moduleId, and core registers its
// provider under `core` (main.jsx) precisely so they resolve without one.
const gate = buildFeatureGate(new Map([['core', flags('atlas')]]))
assert.equal(gate({ to: '/site/atlas', feature: 'atlas' }), true)
assert.equal(gate({ to: '/site/market', feature: 'market' }), false)
})
test('a module row resolves against ITS module, not another one', () => {
const gate = buildFeatureGate(
new Map([
['uo', flags('atlas')],
['rust', flags('market')],
]),
)
assert.equal(gate({ to: '/uo/atlas', feature: 'atlas', moduleId: 'uo' }), true)
// `market` is a flag the OTHER module grants. Resolution is by registration,
// so there is no string a module can write to borrow it.
assert.equal(gate({ to: '/uo/market', feature: 'market', moduleId: 'uo' }), false)
assert.equal(gate({ to: '/rust/market', feature: 'market', moduleId: 'rust' }), true)
})
test('no provider for the owner shows the row', () => {
// The no-module-installed case, and the reason the filter is a correct no-op
// on a bare core rather than a nav that renders nothing.
const gate = buildFeatureGate(new Map())
assert.equal(gate({ to: '/site/atlas', feature: 'atlas' }), true)
assert.equal(gate({ to: '/uo/atlas', feature: 'atlas', moduleId: 'uo' }), true)
})
test('a provider still loading shows the row', () => {
// useShardFlags returns null until its fetch lands. Blanking the nav on every
// page load and filling it in a moment later is the behaviour this avoids.
const gate = buildFeatureGate(new Map([['core', null]]))
assert.equal(gate({ to: '/site/atlas', feature: 'atlas' }), true)
})
test('a provider that returned something unusable shows the row', () => {
for (const bad of [undefined, 42, 'atlas', {}, []]) {
const gate = buildFeatureGate(new Map([['core', bad]]))
assert.equal(gate({ to: '/site/atlas', feature: 'atlas' }), true, `failed closed on ${JSON.stringify(bad)}`)
}
})
test('an array-backed provider is not silently treated as a Set', () => {
// `[].has` does not exist, so this is the unusable case above rather than a
// membership test that quietly always fails. Asserted so that a future
// "helpful" normalisation knows it changed a documented behaviour.
const gate = buildFeatureGate(new Map([['core', ['atlas']]]))
assert.equal(gate({ to: '/site/atlas', feature: 'atlas' }), true)
})
test('a missing map, or a junk row, shows rather than throws', () => {
assert.equal(buildFeatureGate(null)({ feature: 'atlas' }), true)
assert.equal(buildFeatureGate(new Map())(null), true)
assert.equal(buildFeatureGate(new Map())(undefined), true)
})
test('any Set-like satisfies a provider — core does not require a Set', () => {
const gate = buildFeatureGate(new Map([['uo', { has: (name) => name === 'ruleset' }]]))
assert.equal(gate({ feature: 'ruleset', moduleId: 'uo' }), true)
assert.equal(gate({ feature: 'champs', moduleId: 'uo' }), false)
})
test('the open gate is what a component outside the provider gets', () => {
assert.equal(OPEN_GATE({ feature: 'anything' }), true)
})

187
client/test/moduleNav.test.js Normal file
View File

@@ -0,0 +1,187 @@
import { test, beforeEach } from 'node:test'
import assert from 'node:assert/strict'
import { withModuleNav } from '../src/modules/nav.js'
import { registerNav, _reset } from '../src/modules/registry.js'
import { applyNavOverrides, buildPublicNav } from '../src/lib/navOverrides.js'
// The interleave of module nav rows into core's nav (MODULE_API.md §3.3, Phase 2
// PR 8). Tested against the real merge next door rather than in isolation,
// because the property that matters is a relationship between the two: a module
// row has to be indistinguishable from a core row to everything downstream, and
// the way to prove that is to run the downstream thing on it.
const PUBLIC = [
{ label: 'Home', to: '/', end: true },
{ label: 'News', to: '/site/news' },
{ label: 'About', to: '/site/about' },
]
const ADMIN = [
{ items: [{ to: '/admin', label: 'Dashboard', end: true, roles: ['admin', 'moderator'] }] },
{ title: 'Moderation', items: [{ to: '/admin/moderation', label: 'Moderation' }] },
{ title: 'System', items: [{ to: '/admin/users', label: 'Users' }, { to: '/admin/settings', label: 'Settings' }] },
{ items: [{ to: '/admin/account', label: 'Account' }] },
]
beforeEach(() => _reset())
test('with no module installed the base array is returned unchanged', () => {
// Identity, not a copy: this is what makes the useMemo in each layout honest,
// and what guarantees an instance with no modules renders what it renders now.
assert.equal(withModuleNav(PUBLIC, 'public'), PUBLIC)
assert.equal(withModuleNav(ADMIN, 'admin'), ADMIN)
})
test('a flat nav places a module row by the order it asked for', () => {
registerNav('uo', { area: 'public', items: [{ label: 'Atlas', to: '/uo/atlas', order: 1 }] })
assert.deepEqual(
withModuleNav(PUBLIC, 'public').map((i) => i.label),
['Home', 'Atlas', 'News', 'About'],
)
})
test('a flat row with no order appends rather than jumping to the front', () => {
// The 0-default trap: `order ?? 0` would put an unordered row first, which is
// the one place a module could take over the nav without asking for anything.
registerNav('uo', { area: 'public', items: [{ label: 'Atlas', to: '/uo/atlas' }] })
assert.deepEqual(
withModuleNav(PUBLIC, 'public').map((i) => i.label),
['Home', 'News', 'About', 'Atlas'],
)
})
test('an explicit order beats a core row that merely sits at that index', () => {
registerNav('uo', { area: 'public', items: [{ label: 'Atlas', to: '/uo/atlas', order: 2 }] })
const labels = withModuleNav(PUBLIC, 'public').map((i) => i.label)
assert.deepEqual(labels, ['Home', 'News', 'Atlas', 'About'])
})
test('an admin row lands INSIDE the core group it names', () => {
registerNav('uo', {
area: 'admin',
items: [
{ label: 'In-Game Ops', to: '/admin/uo/shard-ops', group: 'Moderation', order: 30 },
{ label: 'Shard', to: '/admin/uo/link', group: 'System', order: 0 },
],
})
const nav = withModuleNav(ADMIN, 'admin')
assert.deepEqual(nav.map((g) => g.title), [undefined, 'Moderation', 'System', undefined])
assert.deepEqual(nav[1].items.map((i) => i.label), ['Moderation', 'In-Game Ops'])
// order 0 puts it above both core rows, which is the whole point of the field.
assert.deepEqual(nav[2].items.map((i) => i.label), ['Shard', 'Users', 'Settings'])
})
test('an unknown group appends a new group instead of dropping the row', () => {
// A typo must cost a position, never a link.
registerNav('uo', { area: 'admin', items: [{ label: 'Atlas', to: '/admin/uo/atlas', group: 'Moderaton' }] })
const nav = withModuleNav(ADMIN, 'admin')
assert.equal(nav.length, ADMIN.length + 1)
assert.deepEqual(nav.at(-1), { title: 'Moderaton', items: [{ label: 'Atlas', to: '/admin/uo/atlas', group: 'Moderaton', moduleId: 'uo' }] })
})
test('an admin row with no group gets a trailing untitled group of its own', () => {
// NOT folded into one of core's untitled groups: those are Dashboard at the
// top and Account at the bottom, and a module page belongs beside neither.
registerNav('uo', { area: 'admin', items: [{ label: 'Atlas', to: '/admin/uo/atlas' }] })
const nav = withModuleNav(ADMIN, 'admin')
assert.equal(nav.length, ADMIN.length + 1)
assert.equal(nav.at(-1).title, undefined)
assert.deepEqual(nav.at(-1).items.map((i) => i.label), ['Atlas'])
assert.deepEqual(nav[0].items.map((i) => i.label), ['Dashboard'])
assert.deepEqual(nav[3].items.map((i) => i.label), ['Account'])
})
test('a row whose `to` collides with a core row is dropped, not rendered twice', () => {
// `to` is the key the override layer stores under and React renders by. Two
// rows sharing one would give an admin a single editor row that moves both.
const warnings = []
const warn = console.warn
console.warn = (msg) => warnings.push(msg)
try {
registerNav('uo', {
area: 'public',
items: [{ label: 'Not News', to: '/site/news' }, { label: 'Atlas', to: '/uo/atlas' }],
})
const nav = withModuleNav(PUBLIC, 'public')
assert.deepEqual(nav.map((i) => i.label), ['Home', 'News', 'About', 'Atlas'])
assert.equal(warnings.length, 1)
assert.match(warnings[0], /\/site\/news.*collides/)
} finally {
console.warn = warn
}
})
test('two modules cannot claim the same path either', () => {
const warn = console.warn
console.warn = () => {}
try {
registerNav('aa', { area: 'public', items: [{ label: 'First', to: '/shared' }] })
registerNav('zz', { area: 'public', items: [{ label: 'Second', to: '/shared' }] })
const labels = withModuleNav(PUBLIC, 'public').map((i) => i.label)
assert.deepEqual(labels, ['Home', 'News', 'About', 'First'])
} finally {
console.warn = warn
}
})
test('a module row carries its moduleId through, which is how the gate finds it', () => {
registerNav('uo', { area: 'public', items: [{ label: 'Atlas', to: '/uo/atlas', feature: 'atlas' }] })
const row = withModuleNav(PUBLIC, 'public').at(-1)
assert.equal(row.moduleId, 'uo')
assert.equal(row.feature, 'atlas')
})
test('areas do not leak into one another', () => {
registerNav('uo', { area: 'admin', items: [{ label: 'Shard', to: '/admin/uo/link', group: 'System' }] })
assert.equal(withModuleNav(PUBLIC, 'public'), PUBLIC)
})
// ── The relationship that is the actual requirement ───────────────────────
test('an admin override applies to a module row exactly as to a core row', () => {
// The reason the interleave happens BEFORE the merge and not after: the merge
// drops any key its base array does not declare, so appending module rows
// afterwards would make every one of them unorderable, unrelabellable and
// unhideable — a visible regression the day the UO rows leave core.
registerNav('uo', { area: 'public', items: [{ label: 'Atlas', to: '/uo/atlas' }] })
const base = withModuleNav(PUBLIC, 'public')
const merged = applyNavOverrides(base, {
'/uo/atlas': { label: 'Bestiary', order: 0 },
'/site/news': { order: 3 },
})
assert.deepEqual(merged.map((i) => i.label), ['Bestiary', 'Home', 'About', 'News'])
})
test('an override can hide a module row, and the public tree can section it', () => {
registerNav('uo', { area: 'public', items: [{ label: 'Atlas', to: '/uo/atlas' }, { label: 'Market', to: '/uo/market' }] })
const base = withModuleNav(PUBLIC, 'public')
const hidden = buildPublicNav(base, { '/uo/atlas': { hidden: true } })
assert.equal(hidden.some((n) => n.to === '/uo/atlas'), false)
const sectioned = buildPublicNav(base, {
items: { '/uo/market': { section: 'sec_shard' } },
sections: [{ id: 'sec_shard', label: 'Shard', order: 0 }],
})
assert.equal(sectioned[0].kind, 'section')
assert.deepEqual(sectioned[0].items.map((i) => i.to), ['/uo/market'])
})
test('a module row can be moved between admin groups by an override', () => {
registerNav('uo', { area: 'admin', items: [{ label: 'Shard', to: '/admin/uo/link', group: 'System' }] })
const base = withModuleNav(ADMIN, 'admin')
const merged = applyNavOverrides(base, { '/admin/uo/link': { group: 'Moderation' } })
assert.deepEqual(merged[1].items.map((i) => i.to), ['/admin/moderation', '/admin/uo/link'])
assert.deepEqual(merged[2].items.map((i) => i.to), ['/admin/users', '/admin/settings'])
})
test('a group a module created is itself a legal override destination', () => {
// Falls out of building the destination set from the base nav it is handed —
// recorded because it is the kind of thing that would otherwise be discovered
// by an admin finding a section they cannot move anything into.
registerNav('uo', { area: 'admin', items: [{ label: 'Atlas', to: '/admin/uo/atlas', group: 'Shard' }] })
const base = withModuleNav(ADMIN, 'admin')
const merged = applyNavOverrides(base, { '/admin/users': { group: 'Shard' } })
assert.deepEqual(merged.at(-1).items.map((i) => i.to), ['/admin/uo/atlas', '/admin/users'])
})

188
client/test/moduleRegistry.test.js Normal file
View File

@@ -0,0 +1,188 @@
import { test, beforeEach } from 'node:test'
import assert from 'node:assert/strict'
import fs from 'node:fs'
import path from 'node:path'
import { fileURLToPath } from 'node:url'
import {
registry,
registerRoutes,
registerNav,
registerFeatureProvider,
routesFor,
navFor,
featureProviderFor,
featureProviders,
registeredIds,
_reset,
} from '../src/modules/registry.js'
import { MODULE_API_VERSION } from '../src/modules/version.js'
// The client-side module registry (docs/website/MODULE_API.md §3.3). Tested in
// isolation from React, like the nav-override merge next door, because the
// property worth proving has nothing to do with rendering: a module gets exactly
// the URL namespace core gave it, however it spells the paths it registers.
//
// window.__rg itself (modules/shared.js) is not tested here — it imports .jsx and
// there is no DOM in this runner. What it publishes is React, the router and
// core components: a wiring test would assert that an import statement imported
// something. Phase 1's spike proved the half that can actually fail, which is a
// real chunk resolving its externals against the global in a browser under an
// enforced CSP.
beforeEach(() => _reset())
test('a module route is namespaced under the module id', () => {
registerRoutes('uo', { public: [{ path: 'atlas', element: 'ATLAS' }] })
assert.deepEqual(
routesFor('public').map((r) => r.path),
['uo/atlas'],
)
})
test('a module cannot spell its way out of its namespace', () => {
// Whatever the module writes, the segment it lands under is core's to choose:
// leading slashes, several of them, a trailing one, or nothing at all.
registerRoutes('uo', {
public: [
{ path: '/atlas' },
{ path: '//atlas/creatures' },
{ path: 'atlas/' },
{ path: '' },
],
})
assert.deepEqual(
routesFor('public').map((r) => r.path),
['uo/atlas', 'uo/atlas/creatures', 'uo/atlas', 'uo'],
)
})
test('a path is namespaced, not sanitised — traversal stays a literal segment', () => {
// `..` is not stripped, and does not need to be: React Router matches path
// patterns literally, so `/uo/../admin` is a route nothing navigates to rather
// than a route that resolves somewhere else. Asserted so that a future
// "cleanup" that starts resolving these knows it changed a behaviour.
registerRoutes('uo', { public: [{ path: '../admin' }] })
assert.deepEqual(routesFor('public')[0].path, 'uo/../admin')
})
test('routes keep their gate and carry the owning module id', () => {
registerRoutes('uo', {
admin: [{ path: 'shard-ops', element: 'OPS', gate: { roles: ['admin', 'moderator'] } }],
})
const [route] = routesFor('admin')
assert.deepEqual(route.gate, { roles: ['admin', 'moderator'] })
assert.equal(route.moduleId, 'uo')
assert.equal(route.element, 'OPS')
})
test('the three areas are kept apart', () => {
registerRoutes('uo', {
public: [{ path: 'atlas' }],
admin: [{ path: 'link' }],
player: [{ path: 'chars' }],
})
assert.equal(routesFor('public').length, 1)
assert.equal(routesFor('admin').length, 1)
assert.equal(routesFor('player').length, 1)
// An area nobody registered is an empty list, never undefined: App.jsx maps
// over all three unconditionally.
_reset()
for (const area of ['public', 'admin', 'player']) assert.deepEqual(routesFor(area), [])
})
test('an unknown area throws rather than being dropped', () => {
// Loudly, because the alternative is a module whose pages simply never appear
// and no indication anywhere of why.
assert.throws(() => registerRoutes('uo', { publik: [{ path: 'atlas' }] }), /unknown area/)
assert.throws(() => registerNav('uo', { area: 'sidebar', items: [] }), /unknown area/)
assert.equal(registeredIds().length, 0)
})
test('nav items sort by order, and equal orders keep load order', () => {
registerNav('aa', { area: 'admin', items: [{ label: 'Second', to: '/a', order: 30 }] })
registerNav('zz', { area: 'admin', items: [{ label: 'Third', to: '/z', order: 30 }] })
registerNav('mm', { area: 'admin', items: [{ label: 'First', to: '/m', order: 10 }] })
assert.deepEqual(
navFor('admin').map((i) => i.label),
['First', 'Second', 'Third'],
)
})
test('a nav item with no order sorts after the ones that asked for a place', () => {
registerNav('uo', {
area: 'public',
items: [{ label: 'Unordered', to: '/u' }, { label: 'Early', to: '/e', order: 5 }],
})
assert.deepEqual(
navFor('public').map((i) => i.label),
['Early', 'Unordered'],
)
})
test('a feature provider is stored under its namespace, with its owner', () => {
const hook = () => ({ atlas: true })
registerFeatureProvider('uo', 'shard', hook)
assert.deepEqual(featureProviderFor('shard'), { id: 'uo', hook })
assert.equal(featureProviderFor('nothing'), undefined)
})
test('providers can be enumerated in registration order, with their owner', () => {
// Core's feature context has to CALL each of these, as a hook, in a fixed
// order — so it needs the list, and it needs the owner id to resolve a nav
// row whose `moduleId` says who it belongs to (modules/features.jsx).
const uo = () => null
const rust = () => null
registerFeatureProvider('uo', 'shard', uo)
registerFeatureProvider('rust', 'server', rust)
assert.deepEqual(featureProviders(), [
{ id: 'uo', namespace: 'shard', hook: uo },
{ id: 'rust', namespace: 'server', hook: rust },
])
})
test('enumerating providers is NOT part of the module-facing surface', () => {
// A module asks for a namespace it knows the name of; enumerating what
// everyone else registered is core's business, so `featureProviders` is a
// module export and not a member of window.__rg.registry.
assert.equal(registry.featureProviders, undefined)
assert.equal(typeof featureProviders, 'function')
})
test('every registration marks the module registered', () => {
registerRoutes('a', { public: [{ path: 'x' }] })
registerNav('b', { area: 'public', items: [] })
registerFeatureProvider('c', 'ns', () => {})
assert.deepEqual(registeredIds().sort(), ['a', 'b', 'c'])
})
test('the registry object handed to modules exposes the whole surface', () => {
// window.__rg.registry is the ONLY way a module reaches any of this, so a
// member missing from the object is a member that does not exist.
assert.deepEqual(Object.keys(registry).sort(), [
'featureProviderFor',
'navFor',
'registerExtension',
'registerFeatureProvider',
'registerNav',
'registerRoutes',
'registeredIds',
'routesFor',
])
})
test('the client and server halves declare the same MODULE_API_VERSION', () => {
// The value is duplicated because it has to be on window.__rg before the first
// module chunk evaluates, which is earlier than a fetch could answer. This is
// the test that pays for the copy: a bump that edits one file fails here
// instead of shipping a core whose two halves disagree about the contract they
// implement.
const here = path.dirname(fileURLToPath(import.meta.url))
const server = fs.readFileSync(
path.join(here, '..', '..', 'server', 'src', 'modules', 'version.js'),
'utf8',
)
const match = server.match(/MODULE_API_VERSION\s*=\s*'([^']+)'/)
assert.ok(match, 'server/src/modules/version.js no longer declares MODULE_API_VERSION as a literal')
assert.equal(MODULE_API_VERSION, match[1])
})

94
client/test/moduleSlots.test.js Normal file
View File

@@ -0,0 +1,94 @@
import { test, beforeEach } from 'node:test'
import assert from 'node:assert/strict'
import {
registry,
declareSlot,
registerExtension,
extensionFor,
registeredIds,
_reset,
} from '../src/modules/registry.js'
// Client extension slots (docs/website/MODULE_API.md §3.7) — the client twin of
// the server's declareSlot/registerExtension.
//
// The registry half only. `<Slot>` itself renders, and there is no DOM in this
// runner, so what it does with what these functions return — including the error
// boundary — is proved by the §7.7 browser smoke instead. Everything below is a
// rule that can be stated without rendering anything, and every one of them can
// be got wrong in a way a browser check would not obviously catch.
beforeEach(() => _reset())
const Fake = () => null
const Other = () => null
test('an unfilled slot reads as nothing', () => {
// The guarantee core's layouts rest on: place a slot, install no module, and
// the page renders what it rendered before.
declareSlot('site.footer.status')
assert.equal(extensionFor('site.footer.status'), null)
})
test('an undeclared slot reads as nothing rather than throwing', () => {
// Reading is core's side and stays fail-safe: a typo in a layout costs that
// spot, not the page. Only WRITING is strict, which is the next test.
assert.equal(extensionFor('nope'), null)
})
test('a module fills a declared slot and core reads it back', () => {
declareSlot('admin.users.detail')
registerExtension('uo', 'admin.users.detail', Fake)
assert.equal(extensionFor('admin.users.detail'), Fake)
assert.deepEqual(registeredIds(), ['uo'])
})
test('filling an unknown slot throws, naming the slot', () => {
// This is the one place the client registry is NOT fail-open, and the reason
// is asymmetry of consequence: a dropped nav row costs a link the viewer can
// reach another way, a silently dropped extension is invisible to everyone
// including its author. Declaration structurally precedes filling (§3.1), so
// this can only ever be a typo or a version skew.
assert.throws(() => registerExtension('uo', 'site.footer.sttaus', Fake), /unknown extension slot "site\.footer\.sttaus"/)
})
test('a non-component fill throws', () => {
declareSlot('site.footer.status')
assert.throws(() => registerExtension('uo', 'site.footer.status', { render: true }), /is not a component/)
})
test('a second module cannot take a filled slot, and the first keeps it', () => {
// Matches the server's rule exactly (registries.js): first fill wins, second
// is an error. The second half of the assertion is the one that matters — a
// rejected fill must not have half-replaced the incumbent.
declareSlot('admin.users.detail')
registerExtension('uo', 'admin.users.detail', Fake)
assert.throws(() => registerExtension('other', 'admin.users.detail', Other), /already filled by "uo"/)
assert.equal(extensionFor('admin.users.detail'), Fake)
})
test('declaring a slot twice throws', () => {
// Core-side programming error: two owners for one position means whichever
// module registered first wins by file order.
declareSlot('site.footer.status')
assert.throws(() => declareSlot('site.footer.status'), /already declared/)
})
test('core fills a slot through the same seam a module uses', () => {
// The client twin of registries.registerCore(). Core is a registrant with an
// id like any other, which is what makes slice 3 a deletion: the module
// registers the same slot and core drops its line.
declareSlot('site.footer.status')
registerExtension('core', 'site.footer.status', Fake)
assert.deepEqual(registeredIds(), ['core'])
})
test('declareSlot and extensionFor are not on the module-facing registry', () => {
// Declaring is core's alone (§3.7), and reading who filled a slot is core's
// too — the same line featureProviders() draws. registerExtension IS on the
// object, because filling is the whole point.
assert.equal(registry.declareSlot, undefined)
assert.equal(registry.extensionFor, undefined)
assert.equal(typeof registry.registerExtension, 'function')
})

23
docker-compose.yml
View File

@@ -35,6 +35,10 @@ services:
DB_HOST: db
UPLOAD_DIR: /app/uploads
LOG_DIR: /app/logs
# Where the loader scans for installed modules. Same path the code already
# defaults to (<repo>/modules, and the repo is /app in the image), set
# explicitly because the bind mount below is what makes it meaningful.
MODULES_DIR: /app/modules
depends_on:
db:
condition: service_healthy
@@ -47,6 +51,25 @@ services:
# the image, so this mount only matters for custom brand images. Create
# ./brand/ on the host and drop assets in; read-only in the container.
- ./brand:/app/brand:ro
# Installed modules (docs/website/MODULE_SYSTEM.md). Modules live on a
# mount, NEVER in the image: that is what lets an operator add one to a
# pull-only deployment without building anything. A bind mount rather than
# a named volume because placing a module directory by hand is a supported
# install — `tar -xf uo-1.0.0.tgz -C ./modules` then restart — and that has
# to be doable from the host, not through `docker cp`.
#
# Read-WRITE: the admin panel's install/uninstall unpacks and removes
# directories here from inside the container.
#
# `modules/` is tracked (it ships a README) so the directory exists in the
# checkout with the operator's own ownership. Do not delete it — Docker
# would recreate a missing bind-mount source as root:root and the container
# user could no longer write it. If the app runs as a uid that does not own
# ./modules, `chown 1000:1000 modules` on the host.
#
# Adding or removing a module takes a RESTART: the scan is synchronous at
# require time (MODULE_API.md §4.1), so nothing here is picked up live.
- ./modules:/app/modules
# Only the PUBLIC API port (3000) is published. The internal server<->bot
# port (INTERNAL_PORT, default 3001) is deliberately NOT listed here, so it
# stays reachable only over the private compose network — Pangolin/the public

75
modules/README.md Normal file
View File

@@ -0,0 +1,75 @@
# Installed modules
This directory is bind-mounted into the container at `/app/modules` (see
`docker-compose.yml`). It is where **installed modules** live — the game-specific
routes, tables, screens and nav that are not part of core. Design of record:
[`docs/website/MODULE_SYSTEM.md`](../../docs/website/MODULE_SYSTEM.md); the
normative contract module authors build against is
[`docs/website/MODULE_API.md`](../../docs/website/MODULE_API.md).
**Core ships no module.** This directory is empty in a fresh checkout, and the
site runs cleanly that way — an empty `modules/` is the normal state for bare
core, not a misconfiguration. Everything below is ignored by git except this
README, which exists so the directory itself is tracked: `docker-compose.yml`
bind-mounts it, and Docker recreates a *missing* bind-mount source as a
root-owned directory the container user cannot write.
## Layout
One directory per module, named for its id, each holding a prebuilt bundle:
```
modules/
uo/
module.json # the manifest the loader reads
server/index.js # registers routes, streams, hooks
server/db/schema.sql # tables, replayed every boot
server/db/purge.sql # only ever run by an explicit purge
client/dist/entry.js # prebuilt ESM chunk, served at /modules/uo/
```
**Nothing here is compiled by the operator.** A module arrives already built —
that is the whole point of the design. There is no install step that runs a
bundler, and none that needs one.
## Installing a module
Two supported paths, both writing the same `installed_modules` row:
- **The admin panel** downloads the bundle from the module's release, verifies it
against its `sha256`, and unpacks it here.
- **By hand**, for a compose-managed host: unpack the bundle into a directory
named for the module id, e.g. `tar -xf uo-1.0.0.tgz -C ./modules`.
Either way, **adding or removing a module takes a restart.** The loader scans
this directory synchronously at startup (`MODULE_API.md` §4.1); nothing placed
here is picked up by a running server.
```
docker compose restart app
```
On boot each module is validated, mounted, its schema fragment replayed and its
`onBoot` hook run — reaching `started`, or `startup_failed` with the stage and
reason recorded. A module that fails to start does not stop the site: core, and
every other module, carry on without it.
## Uninstalling
Removing a directory and restarting is enough to stop a module serving. Note that
this is *not* the same as an uninstall through the admin panel, which also marks
the row `disabled` — a directory that simply vanishes leaves a row claiming to be
enabled, which the loader records as `startup_failed`.
A module's **tables and data are retained** in both cases. Dropping them is a
separate, explicit, destructive purge; it is never bundled into an uninstall.
## Ownership
The container runs as uid 1000 (`node`) and the admin panel writes here, so the
app must be able to write this directory. It is created by your checkout, with
your ownership. If they differ:
```
chown -R 1000:1000 modules
```

24
server/db/data/spawnAtlas.art.example.json
View File

@@ -1,24 +0,0 @@
{
"_comment": [
"OPTIONAL operator-supplied creature art for the spawn atlas. Copy this file to",
"spawnAtlas.art.json (same directory) and edit it, then restart the server or run",
"`npm run atlas:import` — the art map is read on every atlas refresh.",
"",
"This project ships NO creature artwork and never will. UO sprites live in your",
"own client's .mul/.uop files and are yours to extract, not ours to redistribute.",
"If you want art on the atlas pages, export it yourself (UOFiddler, ClassicUO's",
"tooling, or any art extractor), drop the images under server/uploads/atlas/, and",
"map each creature slug to its file name here.",
"",
"Both spawnAtlas.art.json and server/uploads/ are gitignored, so neither the map",
"nor the images can be committed by accident.",
"",
"Keys are creature slugs, as reported by the atlas API and derived from the type",
"names in your own shard's Spawns/*.xml. Values are file names relative to",
"server/uploads/atlas/. Any creature with no entry here simply renders without",
"art — that is the default and fully supported state, not a degraded one."
],
"lizardman": "lizardman.png",
"orc": "orc.png",
"dragon": "dragon.png"
}

711
server/db/schema.sql
View File

@@ -1,6 +1,13 @@
-- Runic Gateway database schema (MariaDB)
-- Run automatically by the MariaDB container (docker-entrypoint-initdb.d) on a
-- fresh volume, and idempotently by ensureSchema() on every server boot.
--
-- CORE ONLY. The 27 shard_* / uo_link_* tables left with module-uo in Phase 3
-- and live in its schema fragment, which core replays immediately after this
-- file (MODULE_API.md 2.6). Two of them carry a foreign key INTO users, which
-- is why that order matters and why the reverse -- a core table referencing a
-- module table -- must never appear here: it would make core unable to boot
-- without a module installed.
CREATE TABLE IF NOT EXISTS users (
id INT AUTO_INCREMENT PRIMARY KEY,
@@ -343,395 +350,11 @@ CREATE TABLE IF NOT EXISTS email_config (
CONSTRAINT chk_email_config_singleton CHECK (id = 1)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- ── uo-link sidecar ────────────────────────────────────────────────────────
-- Connection config for the uo-link sidecar (the HTTP + WebSocket bridge to the
-- ServUO shard). Singleton row (id = 1), mirroring bot_config/email_config: the
-- DB only ever holds the AES-256-GCM-encrypted shared-secret auth token, never
-- plaintext, and it is only decrypted server-side (to call the sidecar). It is
-- never returned to the admin UI — responses expose only `hasToken`. base_url is
-- the REST endpoint, ws_url the live-feed endpoint; both are configurable because
-- in production the sidecar runs on a different host from the website. `status`/
-- `plugin_connected`/`last_event_at`/`boot_id` mirror the sidecar's last-known
-- state for the admin panel between polls; `boot_id` tracks server.hello.bootId
-- so a shard restart can be detected (and caches dropped).
CREATE TABLE IF NOT EXISTS uo_link_config (
id INT PRIMARY KEY DEFAULT 1,
base_url VARCHAR(255) NULL,
ws_url VARCHAR(255) NULL,
auth_token_enc TEXT NULL,
protocol INT NOT NULL DEFAULT 3,
enabled TINYINT(1) NOT NULL DEFAULT 0,
status VARCHAR(20) NOT NULL DEFAULT 'disconnected',
status_detail VARCHAR(500) NULL,
plugin_connected TINYINT(1) NOT NULL DEFAULT 0,
last_event_at DATETIME NULL,
boot_id VARCHAR(64) NULL,
updated_by INT NULL,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
CONSTRAINT fk_uo_link_config_user FOREIGN KEY (updated_by) REFERENCES users(id) ON DELETE SET NULL,
CONSTRAINT chk_uo_link_config_singleton CHECK (id = 1)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Append-only log of notable shard events ingested from the uo-link WebSocket
-- feed. The site OWNS this data (it does not query the sidecar's SQLite): the WS
-- client writes here, and the public/admin read endpoints + live feeds read from
-- here. Only "notable" kinds are logged (sales, deaths, murders, mob.killed,
-- IDOC transitions, quests, skill.gain, fame/karma, audit.*, cheat.*, link.*,
-- server.*). High-frequency kinds (char.vitals, economy.supply) are NOT logged
-- here — they update shard_online / shard_economy instead, keeping the log lean.
-- dedupe_key = sha256(kind + t + stable-json(payload)) truncated to 40 hex chars
-- (fits CHAR(40)); with the UNIQUE index it makes INSERT IGNORE idempotent so
-- WS-reconnect backfill never double-inserts.
CREATE TABLE IF NOT EXISTS shard_events (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
kind VARCHAR(48) NOT NULL,
t BIGINT NOT NULL, -- event time, epoch ms (from the sidecar)
boot_id VARCHAR(64) NULL, -- shard boot id at ingest (server.hello.bootId)
payload JSON NOT NULL, -- the full event object
dedupe_key CHAR(40) NOT NULL UNIQUE,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
INDEX idx_shard_events_kind_t (kind, t),
INDEX idx_shard_events_t (t)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Current online players. Upserted on mob.login, refreshed on char.vitals, and
-- removed on mob.logout. Cleared wholesale when the shard restarts (a new
-- server.hello.bootId). web_id is the linked website user id (present when the
-- account is linked), so the roster can be correlated to site accounts.
CREATE TABLE IF NOT EXISTS shard_online (
serial VARCHAR(20) NOT NULL PRIMARY KEY, -- mobile serial (opaque hex key)
name VARCHAR(120) NULL,
acct VARCHAR(120) NULL,
web_id INT NULL,
map VARCHAR(40) NULL,
x INT NULL,
y INT NULL,
z INT NULL,
hits INT NULL,
hits_max INT NULL,
mana INT NULL,
mana_max INT NULL,
stam INT NULL,
stam_max INT NULL,
str INT NULL,
dex INT NULL,
`int` INT NULL,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_shard_online_acct (acct)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Total-gold-supply time series (from the periodic economy.supply event). Kept
-- append-only so the public status page can render a supply-over-time sparkline.
CREATE TABLE IF NOT EXISTS shard_economy (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
accounts INT NULL, -- number of accounts included in the total
gold BIGINT NULL, -- total gold supply across all accounts
t BIGINT NOT NULL, -- sample time, epoch ms
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
INDEX idx_shard_economy_t (t)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Current decay stage per house, upserted on house.decay. is_idoc is a derived
-- flag (stage == 'IDOC') so the public "houses in danger" list is a cheap
-- indexed lookup rather than a scan.
CREATE TABLE IF NOT EXISTS shard_houses (
serial VARCHAR(20) NOT NULL PRIMARY KEY,
stage VARCHAR(24) NULL, -- Somewhat | Fairly | Greatly | IDOC | Collapsed | ...
map VARCHAR(40) NULL,
x INT NULL,
y INT NULL,
z INT NULL,
region VARCHAR(120) NULL,
name VARCHAR(160) NULL,
owner_serial VARCHAR(20) NULL,
owner_acct VARCHAR(120) NULL,
built_on DATETIME NULL,
last_refreshed DATETIME NULL,
is_idoc TINYINT(1) NOT NULL DEFAULT 0,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_shard_houses_idoc (is_idoc)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Site-side mirror of in-game-account → website-user links. The sidecar is the
-- source of truth (it tags the game account with the websiteUserId on
-- /link/confirm); this table mirrors it so the player portal can list a user's
-- linked accounts and enforce ownership on roster/vendor reads without a shard
-- round-trip. account is unique (one game account maps to at most one site user);
-- a single user may link several game accounts.
CREATE TABLE IF NOT EXISTS shard_account_links (
account VARCHAR(120) NOT NULL PRIMARY KEY,
user_id INT NOT NULL,
char_name VARCHAR(120) NULL,
linked_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT fk_shard_links_user FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
INDEX idx_shard_links_user (user_id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Current champion-spawn board, upserted on champ.update and removed on
-- champ.remove. Mirrors the sidecar's /champs projection into our own store so
-- the public Champions page (and its live deltas) survive a shard outage, the
-- same way shard_online / shard_houses do. Three families share one table, told
-- apart by `category` (champion | mini | sea); category-specific fields (level,
-- kills, boss, restartAt, hits, …) live in the JSON `payload` so the schema does
-- not have to model every variant.
CREATE TABLE IF NOT EXISTS shard_champs (
serial VARCHAR(20) NOT NULL PRIMARY KEY, -- controller/mobile serial (opaque hex)
category VARCHAR(16) NULL, -- champion | mini | sea
type VARCHAR(80) NULL,
name VARCHAR(120) NULL,
status VARCHAR(16) NULL, -- active | cooldown | dormant
active TINYINT(1) NOT NULL DEFAULT 0,
map VARCHAR(40) NULL,
x INT NULL,
y INT NULL,
z INT NULL,
boss_up TINYINT(1) NOT NULL DEFAULT 0,
payload JSON NOT NULL, -- the full champ.update object
t BIGINT NULL, -- event time, epoch ms
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_shard_champs_category (category)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Current open help-page (support ticket) queue, upserted on page.new/page.updated
-- and removed on page.closed. Snapshotted authoritatively from the sidecar's
-- GET /pages on every (re)connect. page_id is the sender's serial (one page per
-- player). Staff-only data — served on the admin channel, never public.
CREATE TABLE IF NOT EXISTS shard_pages (
page_id VARCHAR(20) NOT NULL PRIMARY KEY, -- sender serial (one page per player)
type VARCHAR(40) NULL, -- Bug | Stuck | Account | Question | ...
sender_name VARCHAR(120) NULL,
sender_acct VARCHAR(120) NULL,
web_id INT NULL, -- linked website user id, if any
message TEXT NULL,
map VARCHAR(40) NULL,
x INT NULL,
y INT NULL,
z INT NULL,
sent_ms BIGINT NULL, -- when the page was opened, epoch ms
handled TINYINT(1) NOT NULL DEFAULT 0, -- a staffer claimed it in game
handler VARCHAR(120) NULL,
payload JSON NOT NULL, -- the full page.new/updated object
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_shard_pages_handled (handled)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Guild roster board (Protocol 2.0). Upserted on guild.update (a full-state
-- snapshot emitted only on change) and removed on guild.remove. The leader is an
-- actor object flattened into leader_* columns; the full event is kept in
-- `payload` for anything not hoisted. Mirrors the sidecar's GET /guilds
-- projection into our store so the public Guilds page survives a shard outage.
CREATE TABLE IF NOT EXISTS shard_guilds (
id INT NOT NULL PRIMARY KEY, -- in-game guild id
name VARCHAR(120) NULL,
abbr VARCHAR(24) NULL,
members INT NULL,
online INT NULL,
alliance VARCHAR(120) NULL,
leader_serial VARCHAR(20) NULL,
leader_name VARCHAR(120) NULL,
leader_acct VARCHAR(120) NULL,
leader_web_id INT NULL,
payload JSON NOT NULL, -- the full guild.update object
t BIGINT NULL, -- event time, epoch ms
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_shard_guilds_name (name)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Town-governor board (Protocol 2.0, City Loyalty). One row per city, upserted on
-- city.update (full-state, emitted only on change; there is no remove event since
-- the set of cities is fixed). governor / governorElect are actor objects
-- flattened into columns; the full event is kept in `payload`. Empty on shards
-- that do not run the City Loyalty system.
CREATE TABLE IF NOT EXISTS shard_governors (
city VARCHAR(40) NOT NULL PRIMARY KEY, -- Britain | Moonglow | ...
governor_serial VARCHAR(20) NULL,
governor_name VARCHAR(120) NULL,
governor_acct VARCHAR(120) NULL,
governor_web_id INT NULL,
elect_serial VARCHAR(20) NULL,
elect_name VARCHAR(120) NULL,
elect_acct VARCHAR(120) NULL,
election_phase VARCHAR(16) NULL, -- none | nominate | vote | pending
candidates INT NULL,
auto_pick_at DATETIME NULL,
payload JSON NOT NULL, -- the full city.update object
t BIGINT NULL, -- event time, epoch ms
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Governor term history — the "who governed when" ledger behind the Governors
-- board. Captured from day one (history cannot be backfilled) on every observed
-- governor CHANGE: the open term (ended_at IS NULL) is closed and a new one
-- opened. `votes` stays NULL — the city.update feed exposes only the candidate
-- COUNT and election phase, not per-candidate tallies, so we record who governed
-- and when (reliable) and never fabricate vote numbers. The look-back UI ("who
-- were all the governors of Britain?") reads this table.
CREATE TABLE IF NOT EXISTS shard_governor_terms (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
city VARCHAR(40) NOT NULL,
governor_serial VARCHAR(20) NULL,
governor_name VARCHAR(120) NULL,
governor_acct VARCHAR(120) NULL,
governor_web_id INT NULL,
started_at BIGINT NOT NULL, -- term start, epoch ms
ended_at BIGINT NULL, -- term end epoch ms (NULL = current)
votes INT NULL, -- not in the feed (reserved)
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
INDEX idx_shard_gov_terms_city (city, started_at),
INDEX idx_shard_gov_terms_open (city, ended_at)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Online-population snapshot (Protocol 2.0). Singleton row (id = 1) holding the
-- latest presence.online aggregate: total count plus per-facet and per-region
-- breakdown maps (stored as JSON). Distinct from shard_online (per-player) — this
-- is the rolled-up headcount the public "Players Online" widget renders. The
-- time series, if ever needed, is available from GET /history?kind=presence.online.
CREATE TABLE IF NOT EXISTS shard_presence (
id INT PRIMARY KEY DEFAULT 1,
count INT NOT NULL DEFAULT 0,
by_facet JSON NULL, -- { "Felucca": 12, "Trammel": 30 }
by_region JSON NULL, -- { "Britain": 18, "Wilderness": 9 }
t BIGINT NULL, -- snapshot time, epoch ms
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
CONSTRAINT chk_shard_presence_singleton CHECK (id = 1)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- The shard's published ruleset (Protocol 3.0 world.ruleset). Singleton row
-- (id = 1) holding the latest frame: expansion, which optional systems are on,
-- skill/stat caps, account and house limits, champion scroll rules, the
-- save/restart schedule. The shard re-emits it on every sidecar connect, so this
-- row is simply overwritten; `rev` is the shard's own FNV-1a of the body, which
-- distinguishes "same ruleset, re-sent on reconnect" from "an operator changed a
-- .cfg". No row at all means the shard has never published one — served as null,
-- which the rules page renders differently from a published ruleset.
CREATE TABLE IF NOT EXISTS shard_ruleset (
id INT PRIMARY KEY DEFAULT 1,
rev VARCHAR(32) NULL,
expansion VARCHAR(16) NULL, -- hoisted for cheap display
payload JSON NOT NULL, -- the whole world.ruleset frame
t BIGINT NULL, -- frame time, epoch ms
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
CONSTRAINT chk_shard_ruleset_singleton CHECK (id = 1)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Points/loyalty leaderboards (Protocol 3.0 points.board). One row per point
-- system, keyed by the shard's own PointsType name. The shard publishes ~25 of
-- these (Queen's Loyalty, Void Pool, the nine city loyalties, …), each a standing
-- players accumulate over months.
--
-- The top-N list stays inside `payload` rather than being normalized into a
-- shard_points_entries table. It is a fixed-size list (10 by default) that is only
-- ever read whole, exactly like shard_governors.candidates — normalizing it would
-- buy nothing until something needs a per-character reverse lookup, and a
-- character's own standings already ride inside char.profile instead.
--
-- No delete path: the shard's set of systems is fixed at startup, so there is no
-- points.remove to mirror.
CREATE TABLE IF NOT EXISTS shard_points_boards (
system VARCHAR(48) PRIMARY KEY, -- PointsType name, e.g. QueensLoyalty
name VARCHAR(128) NULL, -- resolved display name, if the shard sent a literal
name_cliloc INT NULL, -- cliloc id when the name is a TextDefinition number
max_points BIGINT NULL,
players INT NULL, -- players actually holding points in this system
show_on_gump TINYINT(1) NOT NULL DEFAULT 1, -- the shard's own "is this player-facing?" flag
payload JSON NOT NULL, -- the whole points.board frame, incl. `top`
t BIGINT NULL, -- frame time, epoch ms
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Player-vendor market index (Protocol 3.0 vendor.listing). One row per player
-- vendor and one per priced listing, so the site can offer the search the in-game
-- Vendor Search gump offers — from outside the game.
--
-- The shard sweeps vendors round-robin and emits one AUTHORITATIVE frame per
-- vendor, so ingest is delete-then-insert of that vendor's items inside one
-- transaction (see shardMarket.db.js). No foreign key from items to vendors, in
-- keeping with every other shard_* table: the ingest transaction is what keeps
-- them consistent, and an FK would turn a malformed frame into a failed write
-- rather than a dropped row.
--
-- Only vendors whose owner left the in-game Vendor Search flag ON are ever sent,
-- so a player who hid their shop in game is hidden here too — see BridgeMarket.cs.
CREATE TABLE IF NOT EXISTS shard_vendors (
serial VARCHAR(20) NOT NULL PRIMARY KEY, -- "0x40001234"
shop_name VARCHAR(160) NULL,
owner_serial VARCHAR(20) NULL,
owner_name VARCHAR(64) NULL,
map VARCHAR(40) NULL,
x INT NULL,
y INT NULL,
z INT NULL,
region VARCHAR(80) NULL,
house VARCHAR(160) NULL, -- the house SIGN's name, not the house type
item_count INT NOT NULL DEFAULT 0, -- listings published in the frame
item_total INT NOT NULL DEFAULT 0, -- listings the shop actually holds
truncated TINYINT(1) NOT NULL DEFAULT 0, -- item_total > item_count
t BIGINT NULL, -- frame time, epoch ms
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_shard_vendors_owner (owner_name),
INDEX idx_shard_vendors_map (map),
INDEX idx_shard_vendors_region (region),
-- The market page's staleness banner is MIN(updated_at) over this column: the
-- round-robin sweep means the oldest row is how far behind the index can be.
INDEX idx_shard_vendors_updated (updated_at)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- One priced listing. Unlike the points board's top-N — a fixed-size list read
-- whole — these are the searchable rows the whole feature exists for, so they are
-- normalized rather than left inside a payload column, and there is no payload
-- column on shard_vendors at all.
--
-- `display_name` is DENORMALIZED at ingest: the shard sends `cliloc` (the item's
-- LabelNumber) and, rarely, a literal `name`, and resolving 50 clilocs per page
-- at query time would make the cliloc table a join on the hot path AND make
-- search-by-name impossible. Resolving once on write buys the index. It is
-- re-resolved in bulk after a cliloc import, because the diff sweep will not
-- re-send an unchanged shop just because the site learned what its items are
-- called.
CREATE TABLE IF NOT EXISTS shard_vendor_items (
id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
vendor_serial VARCHAR(20) NOT NULL,
serial VARCHAR(20) NOT NULL,
item_id INT NOT NULL DEFAULT 0, -- ItemID (the art/graphic id)
hue INT NOT NULL DEFAULT 0,
amount INT NOT NULL DEFAULT 1,
price BIGINT NOT NULL DEFAULT 0,
name VARCHAR(160) NULL, -- the item's literal Name, null for most
cliloc INT NULL, -- LabelNumber, resolved against shard_clilocs
display_name VARCHAR(160) NULL, -- resolved at ingest; what search matches
child TINYINT(1) NOT NULL DEFAULT 0, -- priced by an enclosing container, not itself
INDEX idx_shard_vendor_items_vendor (vendor_serial),
INDEX idx_shard_vendor_items_price (price),
INDEX idx_shard_vendor_items_item (item_id),
INDEX idx_shard_vendor_items_name (display_name),
-- Search filters on name and sorts on price; the composite covers the common
-- "cheapest matching X" without a filesort over the whole table.
INDEX idx_shard_vendor_items_name_price (display_name, price)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Per-feature visibility for every shard-derived surface (Protocol 3.0). One row
-- per feature; an absent row means "use the compiled default", and the compiled
-- defaults reproduce the behavior that shipped before v3 — so an empty table is
-- a no-op. See utils/shardVisibility.js for the catalog and the ladder, and
-- docs/link/v3.md §3 for the contract.
--
-- audience the minimum rung on anonymous < logged_in < player < staff < admin
-- stream whether this feature's kinds fan out over SSE at all (the market
-- index ships with this off: no page needs a live firehose of
-- whole vendor inventories)
-- field_rules {"<field>": "<rung>"} for SENSITIVE fields only. `acct` and
-- `webId` are admin-only always and are rejected here — they are
-- not in-game visible and are deliberately not configurable.
CREATE TABLE IF NOT EXISTS shard_feature_visibility (
feature VARCHAR(48) NOT NULL PRIMARY KEY,
enabled TINYINT(1) NOT NULL DEFAULT 1,
audience VARCHAR(20) NOT NULL DEFAULT 'anonymous',
stream TINYINT(1) NOT NULL DEFAULT 1,
field_rules JSON NULL,
updated_by INT NULL,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Admin email invites (Protocol 2.0 provisioning). A staff member invites someone
-- by email at a pre-chosen access level; the invitee accepts via a tokened link,
@@ -1106,35 +729,79 @@ CREATE TABLE IF NOT EXISTS pages (
-- Announcement pipeline. One row per publish event of a news post; the table
-- doubles as the job queue (a light in-process poller — utils/announceWorker.js
-- — sweeps it for due legs). Two INDEPENDENT delivery legs so a Discord outage
-- never blocks or retries the in-game town-crier leg and vice versa. `status` is
-- a derived rollup of the two legs (see announceJobs.logic.js): done when both
-- legs done, failed when both exhausted, partial in between. Each leg tracks its
-- own attempt count, last error, and next-due time for exponential backoff.
-- post_id is INT (matches posts.id) and cascades so deleting a post reaps its
-- jobs. posts.announce_job_id points back at the latest row for admin lookups.
-- — sweeps it for due legs). `status` is a derived rollup of the legs (see
-- announceJobs.logic.js): done when every leg is done, failed when every leg is
-- exhausted, partial in between. post_id is INT (matches posts.id) and cascades
-- so deleting a post reaps its jobs. posts.announce_job_id points back at the
-- latest row for admin lookups.
CREATE TABLE IF NOT EXISTS announce_jobs (
id INT AUTO_INCREMENT PRIMARY KEY,
post_id INT NOT NULL,
status ENUM('pending','partial','done','failed') NOT NULL DEFAULT 'pending',
towncrier_status ENUM('pending','done','failed') NOT NULL DEFAULT 'pending',
towncrier_attempts SMALLINT NOT NULL DEFAULT 0,
towncrier_last_error TEXT NULL,
towncrier_next_attempt_at DATETIME NULL,
discord_status ENUM('pending','done','failed') NOT NULL DEFAULT 'pending',
discord_attempts SMALLINT NOT NULL DEFAULT 0,
discord_last_error TEXT NULL,
discord_next_attempt_at DATETIME NULL,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
CONSTRAINT fk_announce_jobs_post FOREIGN KEY (post_id) REFERENCES posts(id) ON DELETE CASCADE,
INDEX idx_announce_due (towncrier_status, towncrier_next_attempt_at),
INDEX idx_announce_due_discord (discord_status, discord_next_attempt_at)
CONSTRAINT fk_announce_jobs_post FOREIGN KEY (post_id) REFERENCES posts(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- One row per delivery leg per job. INDEPENDENT by design: a Discord outage never
-- blocks or retries another leg, and each leg tracks its own attempt count, last
-- error and next-due time for exponential backoff.
--
-- This is a child table rather than a pair of leg-prefixed column groups on
-- announce_jobs because the leg set is DATA now, not schema: core registers
-- `discord`, module-uo registers `towncrier`, and a module for another game
-- registers its own — through modules/registries.js's registerAnnounceLeg
-- (MODULE_SYSTEM.md §1.8). A module cannot ALTER a core table, so a leg that
-- needed its own columns could never come from a module at all. `leg` is a plain
-- VARCHAR and not an ENUM for the same reason.
CREATE TABLE IF NOT EXISTS announce_job_legs (
job_id INT NOT NULL,
leg VARCHAR(64) NOT NULL,
status ENUM('pending','done','failed') NOT NULL DEFAULT 'pending',
attempts SMALLINT NOT NULL DEFAULT 0,
last_error TEXT NULL,
next_attempt_at DATETIME NULL,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (job_id, leg),
CONSTRAINT fk_announce_job_legs_job FOREIGN KEY (job_id) REFERENCES announce_jobs(id) ON DELETE CASCADE,
INDEX idx_announce_leg_due (status, next_attempt_at)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Carry the two hardcoded leg column groups over to the child table, once. Guarded
-- on the OLD columns still existing (via information_schema, since a plain SELECT
-- of a dropped column is a parse error, not a runtime one) and on there being no
-- row already, so replaying this file on every boot is a no-op after the first.
-- Deleting this block once every deployment has booted it is safe.
SET @has_legacy_legs := (
SELECT COUNT(*) FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'announce_jobs'
AND COLUMN_NAME = 'towncrier_status'
);
SET @sql := IF(@has_legacy_legs > 0,
'INSERT IGNORE INTO announce_job_legs (job_id, leg, status, attempts, last_error, next_attempt_at)
SELECT id, ''towncrier'', towncrier_status, towncrier_attempts, towncrier_last_error, towncrier_next_attempt_at FROM announce_jobs
UNION ALL
SELECT id, ''discord'', discord_status, discord_attempts, discord_last_error, discord_next_attempt_at FROM announce_jobs',
'DO 0');
PREPARE stmt FROM @sql; EXECUTE stmt; DEALLOCATE PREPARE stmt;
-- MariaDB's IF EXISTS makes this idempotent, so it replays cleanly like the rest
-- of the file. It is the one DROP in core's schema, and it is deliberate: leaving
-- the columns would leave `towncrier` in a core file, which Phase 3's acceptance
-- grep forbids (MODULE_SYSTEM.md §2.7).
ALTER TABLE announce_jobs
DROP COLUMN IF EXISTS towncrier_status,
DROP COLUMN IF EXISTS towncrier_attempts,
DROP COLUMN IF EXISTS towncrier_last_error,
DROP COLUMN IF EXISTS towncrier_next_attempt_at,
DROP COLUMN IF EXISTS discord_status,
DROP COLUMN IF EXISTS discord_attempts,
DROP COLUMN IF EXISTS discord_last_error,
DROP COLUMN IF EXISTS discord_next_attempt_at,
DROP INDEX IF EXISTS idx_announce_due,
DROP INDEX IF EXISTS idx_announce_due_discord;
-- ── Spawn atlas (Protocol 3.0 Part C) ───────────────────────────────────────
-- Static shard CONTENT, not live shard state: what spawns where, which regions
-- and landmarks exist, and which champion altars are configured. Nothing here
@@ -1148,174 +815,48 @@ CREATE TABLE IF NOT EXISTS announce_jobs (
-- One row per spawnable type, aggregated across the world. `total` is the sum of
-- each type's own MX across every point that spawns it (how many exist at once);
-- `facets` is a per-facet point count, so the facet filter and "where does this
-- live" both answer without touching shard_spawn_points.
CREATE TABLE IF NOT EXISTS shard_spawn_creatures (
slug VARCHAR(120) NOT NULL PRIMARY KEY, -- slugified class name; the /atlas/:slug key
name VARCHAR(120) NOT NULL, -- display spelling chosen by the build
total INT NOT NULL DEFAULT 0,
points INT NOT NULL DEFAULT 0,
facets JSON NULL, -- { "Felucca": 171, "Trammel": 160, ... }
-- Operator-supplied artwork, always NULL on a fresh import. The repo ships no
-- creature art: sprites live in the operator's own client .mul/.uop files and
-- are theirs to extract and place under uploads/atlas/. The UI renders without
-- art when this is NULL, which is the normal case.
art VARCHAR(255) NULL,
-- Plain INDEX, deliberately NOT FULLTEXT: ~800 rows makes a LIKE scan free,
-- and FULLTEXT's min-token-length would break searches for names like "orc".
INDEX idx_shard_spawn_creatures_name (name)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- One row per spawner. `region`/`landmark` are the resolved place name — the
-- point-in-rect transform that turns "5411,1234" into "Despise" — and `label` is
-- the resolved display string (region, else landmark, else 'Wilderness').
CREATE TABLE IF NOT EXISTS shard_spawn_points (
id INT AUTO_INCREMENT PRIMARY KEY,
facet VARCHAR(40) NOT NULL,
name VARCHAR(120) NULL, -- the ServUO spawner's own name
x INT NOT NULL,
y INT NOT NULL,
width INT NOT NULL DEFAULT 0,
height INT NOT NULL DEFAULT 0,
spawn_range INT NOT NULL DEFAULT 0, -- `range` is reserved in MariaDB
max_count INT NOT NULL DEFAULT 0,
min_delay INT NOT NULL DEFAULT 0,
max_delay INT NOT NULL DEFAULT 0,
tod_start INT NOT NULL DEFAULT 0, -- meaningless unless tod_mode <> 0
tod_end INT NOT NULL DEFAULT 0,
tod_mode INT NOT NULL DEFAULT 0,
region VARCHAR(120) NULL,
landmark VARCHAR(120) NULL,
label VARCHAR(120) NOT NULL DEFAULT 'Wilderness',
INDEX idx_shard_spawn_points_facet (facet),
INDEX idx_shard_spawn_points_label (label)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- The many-to-many between the two above: one spawner commonly carries several
-- types (a single Trammel point spawns six), each with its own max. This is how
-- /atlas/creatures/:slug finds the places a creature appears.
CREATE TABLE IF NOT EXISTS shard_spawn_point_types (
point_id INT NOT NULL,
slug VARCHAR(120) NOT NULL, -- → shard_spawn_creatures.slug (no FK)
max_count INT NOT NULL DEFAULT 1,
PRIMARY KEY (point_id, slug),
INDEX idx_shard_spawn_point_types_slug (slug)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Named regions from Data/Regions.xml, flattened out of their nesting. `rects`
-- holds the region's rectangles; `priority` and rect area are what resolved each
-- spawn point at build time, kept here so the admin drift check can re-derive.
CREATE TABLE IF NOT EXISTS shard_regions (
id INT AUTO_INCREMENT PRIMARY KEY,
facet VARCHAR(40) NOT NULL,
name VARCHAR(120) NOT NULL,
type VARCHAR(80) NULL, -- ServUO region class
priority INT NOT NULL DEFAULT 0,
parent VARCHAR(120) NULL, -- enclosing named region, if any
rects JSON NULL,
INDEX idx_shard_regions_facet (facet),
INDEX idx_shard_regions_name (name)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Points of interest from Data/Locations/*.xml. `grp` is the innermost enclosing
-- parent ("Covetous"), which is the label worth showing — "Covetous" reads
-- better than the individual marker "Level 1". (`group` is reserved in SQL.)
CREATE TABLE IF NOT EXISTS shard_landmarks (
id INT AUTO_INCREMENT PRIMARY KEY,
facet VARCHAR(40) NOT NULL,
name VARCHAR(120) NOT NULL,
grp VARCHAR(120) NULL,
x INT NOT NULL,
y INT NOT NULL,
z INT NOT NULL DEFAULT 0,
INDEX idx_shard_landmarks_facet (facet),
INDEX idx_shard_landmarks_name (name)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Configured champion altars from Config/ChampionSpawns.xml. This is static
-- roster data ("there is an Unholy Terror altar in Deceit") and is distinct from
-- the live champ.update feed in shard_champs ("it is on level 3 right now").
CREATE TABLE IF NOT EXISTS shard_champion_spawns (
slug VARCHAR(160) NOT NULL PRIMARY KEY, -- facet-name, e.g. "felucca-deceit"
name VARCHAR(120) NOT NULL,
grp VARCHAR(80) NULL, -- spawn group; one active per group
type VARCHAR(80) NULL, -- '' when randomised per activation
random_type TINYINT(1) NOT NULL DEFAULT 0,
facet VARCHAR(40) NOT NULL,
x INT NOT NULL,
y INT NOT NULL,
z INT NOT NULL DEFAULT 0,
radius INT NOT NULL DEFAULT 0,
label VARCHAR(120) NULL, -- resolved place name
INDEX idx_shard_champion_spawns_facet (facet)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- UO's localization table: cliloc id -> display string. Items carry a
-- `LabelNumber` rather than a name, so without this the site can only render
-- `id 1023721` where the game shows "quarter staff". The shard has always sent
-- the id (char.profile's `cliloc`, and one per marketplace listing) — the number
-- was never the missing piece, the table was.
-- Installed modules (module system, docs/website/MODULE_SYSTEM.md §2.4). One row
-- per module the operator has installed onto the modules volume, keyed by the
-- module id from its module.json — the same id that names the directory, the URL
-- segment and the client registry key.
--
-- Sourced from a file the OPERATOR converts once from their own UO client and
-- points the site at (docs/website/CLILOCS.md); nothing derived from the client
-- is committed, the same rule the spawn atlas and the creature art map follow.
-- A shard with no cliloc file configured simply renders item ids, which is what
-- it did before this table existed.
-- This table is a RECORD of what happened, never the source of truth for what is
-- mounted: the loader scans the filesystem at require time, before the database is
-- reachable (MODULE_API.md §4.1), so the URL surface is a property of the volume
-- and not of a row here. What the row decides is whether a mounted module answers
-- (`disabled` ⇒ its guard 404s, §4.5) and what the admin panel shows after a
-- failure.
--
-- `text` is TEXT, not VARCHAR: real tables top out around 12 KB for the long
-- property descriptions, and truncating them silently would be worse than
-- storing them. Item NAMES are all short — the index that matters for search is
-- on the denormalized `shard_vendor_items.display_name`, not here.
CREATE TABLE IF NOT EXISTS shard_clilocs (
number INT NOT NULL PRIMARY KEY,
flag SMALLINT NOT NULL DEFAULT 0,
text TEXT NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Singleton (id = 1) describing the cliloc table currently loaded: the source
-- file, its sha256, the entry count and the parser version. The boot path
-- compares the stored hash against the file on disk and skips the parse when
-- they match, which is every restart that did not follow a client patch.
CREATE TABLE IF NOT EXISTS shard_cliloc_meta (
id TINYINT NOT NULL PRIMARY KEY DEFAULT 1,
payload JSON NOT NULL,
imported_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
CONSTRAINT chk_shard_cliloc_meta_singleton CHECK (id = 1)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Singleton (id = 1) describing the artifact currently loaded: when it was
-- built, its counts, and a sha256 per ServUO source file. The admin drift check
-- compares this against db/data/spawnAtlas.meta.json to report when the database
-- is behind the committed artifact.
CREATE TABLE IF NOT EXISTS shard_atlas_meta (
id TINYINT NOT NULL PRIMARY KEY DEFAULT 1,
payload JSON NOT NULL,
imported_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
CONSTRAINT chk_shard_atlas_meta_singleton CHECK (id = 1)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Singleton (id = 1) holding an atlas refresh that was parsed but deliberately
-- NOT applied, because it would remove a facet the site currently serves.
-- `state` is the §2.4 machine in one column: installed → enabled → started, with
-- disabled and startup_failed as the recoverable states. `installed` is the
-- transient state between an install writing the row and the restart that starts
-- it. On every boot each non-disabled row is reset to `enabled` and re-attempted
-- (so a fixed module recovers on restart, with no panel visit needed), then the
-- load outcome writes `started` or `startup_failed`. Only `disabled` survives a
-- boot untouched — it is the operator's decision, not an outcome.
--
-- Losing a facet is the signature of a half-copied or mid-update ServUO tree as
-- much as of a real map change, and boot cannot tell the two apart — so the
-- refresh is staged here for a human instead of being applied. Startup is never
-- blocked by it: the site comes up serving the atlas it already had.
-- failure_stage/failure_reason are §4.4's recorded reason, one of the seven
-- validation steps of §4.3 plus `boot`. Both are cleared by every transition that
-- is not a failure, so a stale reason can never be shown against a running module.
--
-- Only the DECISION is stored, not the parsed world: `payload` holds the source
-- hashes and the facet diff (a few KB), and approving re-parses the tree. That
-- keeps a multi-megabyte blob out of the database and guarantees the applied
-- atlas matches the tree as it is at approval time, not as it was at boot.
--
-- `rejected` is remembered against those exact source hashes so a declined
-- refresh does not re-prompt on every restart; changing the tree changes the
-- hashes and asks again.
CREATE TABLE IF NOT EXISTS shard_atlas_pending (
id TINYINT NOT NULL PRIMARY KEY DEFAULT 1,
status ENUM('pending','rejected') NOT NULL DEFAULT 'pending',
payload JSON NOT NULL, -- source hashes + facet diff
detected_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
CONSTRAINT chk_shard_atlas_pending_singleton CHECK (id = 1)
-- source/sha256 are install provenance (§2.5): the release the bundle came from and
-- the digest that was verified before unpacking. Both NULL for a directory placed
-- on the volume by hand, which stays supported.
CREATE TABLE IF NOT EXISTS installed_modules (
id VARCHAR(32) NOT NULL PRIMARY KEY, -- module.json id; names the directory
name VARCHAR(128) NOT NULL, -- human label for the admin Modules screen
version VARCHAR(32) NOT NULL, -- module.json version (semver)
state ENUM('installed','enabled','disabled','started','startup_failed')
NOT NULL DEFAULT 'installed',
failure_stage VARCHAR(32) NULL, -- manifest|core_api|mounts|extensions|schema|require|register|boot
failure_reason TEXT NULL, -- the recorded reason, shown in the admin panel
source VARCHAR(255) NULL, -- release URL the bundle came from
sha256 CHAR(64) NULL, -- verified bundle digest
installed_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
started_at DATETIME NULL, -- last successful start
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_installed_modules_state (state)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Migrations for databases created before the wiki upgrade. Each statement uses
@@ -1369,23 +910,6 @@ ALTER TABLE wiki_pages ADD FULLTEXT INDEX IF NOT EXISTS idx_wiki_search (title,
ALTER TABLE posts ADD COLUMN IF NOT EXISTS announced_at DATETIME NULL;
ALTER TABLE posts ADD COLUMN IF NOT EXISTS announce_job_id INT NULL;
-- House registry (Protocol 2.0). The house.update full-state feed carries richer
-- fields than the house.decay transition feed shard_houses was built for. Rather
-- than a second table for one entity, extend shard_houses: house.update writes the
-- registry columns below (owner display name, co-owner/friend counts, placement
-- price, decay level name) while house.decay keeps owning `stage`/`is_idoc`. Each
-- upsert only touches its own columns, so the two feeds never clobber each other.
-- `price` is the placement value, NOT a "for sale" flag (stock ServUO has none).
ALTER TABLE shard_houses ADD COLUMN IF NOT EXISTS owner_name VARCHAR(120) NULL;
ALTER TABLE shard_houses ADD COLUMN IF NOT EXISTS co_owners INT NULL;
ALTER TABLE shard_houses ADD COLUMN IF NOT EXISTS friends INT NULL;
ALTER TABLE shard_houses ADD COLUMN IF NOT EXISTS price BIGINT NULL;
ALTER TABLE shard_houses ADD COLUMN IF NOT EXISTS decay VARCHAR(24) NULL;
-- Distinguishes a full registry row (seen via house.update) from a decay-only row,
-- so the public Houses browser can list registered houses without pulling in rows
-- we only ever saw an IDOC transition for.
ALTER TABLE shard_houses ADD COLUMN IF NOT EXISTS in_registry TINYINT(1) NOT NULL DEFAULT 0;
-- Mobile device sessions (M9): a friendly label the app may send at login, and
-- the last time this session token was issued/used, for the "Active Devices"
-- self-service list. Both nullable and additive; existing rows get them here.
@@ -1397,19 +921,4 @@ ALTER TABLE mobile_refresh_tokens ADD COLUMN IF NOT EXISTS last_used_at DATETIME
-- trust token. A boolean only — the token is returned over that app→server call
-- and never persisted here (only its sha256 lands in trusted_devices).
ALTER TABLE mobile_auth_sessions ADD COLUMN IF NOT EXISTS trust_device TINYINT(1) NOT NULL DEFAULT 0;
-- Protocol 3.0 cutover: this build speaks wire protocol 3 (world.ruleset,
-- points.board, vendor.listing), so the pinned version an existing install
-- carries has to move with it — a 2 against a v3 sidecar 409s every REST call
-- and closes the WS on ws.hello. MODIFY fixes the column default for installs
-- created before the bump (idempotent, like the other MODIFYs here).
ALTER TABLE uo_link_config MODIFY COLUMN protocol INT NOT NULL DEFAULT 3;
-- The row itself is admin-editable, and schema.sql runs on EVERY boot, so this
-- must be one-shot: an operator who deliberately pins an older sidecar in
-- Admin → Shard has to stay pinned. The marker row in `settings` is what makes
-- it fire once — written after the UPDATE, and on a fresh install (no
-- uo_link_config row yet) it is simply written with nothing to update.
UPDATE uo_link_config SET protocol = 3
WHERE id = 1 AND protocol < 3
AND NOT EXISTS (SELECT 1 FROM settings WHERE `key` = 'uo_link_protocol_3_migrated');
INSERT IGNORE INTO settings (`key`, value) VALUES ('uo_link_protocol_3_migrated', '1');
INSERT IGNORE INTO settings (`key`, value) VALUES ('uo_link_protocol_3_migrated', '1');

3
server/package.json
View File

@@ -9,8 +9,7 @@
"seed": "node db/seed.js",
"swagger": "node swagger/swagger.js",
"routes:manifest": "node scripts/routeManifest.js",
"atlas:import": "node scripts/importSpawnAtlas.js",
"test": "node --test"
"test": "node --test --require ./test/_setup.js"
},
"keywords": [
"express",

658
server/routes.guards.json
View File

@@ -635,272 +635,6 @@
"multerMiddleware"
]
},
{
"method": "POST",
"path": "/api/v1/admin/shard/account",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/shard/accounts",
"handlers": 1,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "GET",
"path": "/api/v1/admin/shard/atlas",
"handlers": 2,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "POST",
"path": "/api/v1/admin/shard/atlas/approve",
"handlers": 2,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "POST",
"path": "/api/v1/admin/shard/atlas/import",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "PUT",
"path": "/api/v1/admin/shard/atlas/path",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "POST",
"path": "/api/v1/admin/shard/atlas/reject",
"handlers": 2,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "GET",
"path": "/api/v1/admin/shard/audit",
"handlers": 2,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "POST",
"path": "/api/v1/admin/shard/ban",
"handlers": 7,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "POST",
"path": "/api/v1/admin/shard/broadcast",
"handlers": 5,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/shard/char/:serial",
"handlers": 3,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/shard/clilocs",
"handlers": 2,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "POST",
"path": "/api/v1/admin/shard/clilocs/import",
"handlers": 5,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "PUT",
"path": "/api/v1/admin/shard/clilocs/path",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/shard/houses",
"handlers": 2,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "POST",
"path": "/api/v1/admin/shard/kick",
"handlers": 5,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "POST",
"path": "/api/v1/admin/shard/link",
"handlers": 3,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/shard/pages",
"handlers": 2,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "POST",
"path": "/api/v1/admin/shard/pages/:id/close",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "POST",
"path": "/api/v1/admin/shard/pages/:id/respond",
"handlers": 6,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/shard/roster/:account",
"handlers": 3,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/shard/sales",
"handlers": 1,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "POST",
"path": "/api/v1/admin/shard/unban",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/shard/vendors/:account",
"handlers": 3,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/shard/visibility",
"handlers": 2,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "PUT",
"path": "/api/v1/admin/shard/visibility",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "PUT",
"path": "/api/v1/admin/site-mode",
@@ -912,57 +646,6 @@
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/uo-link/config",
"handlers": 2,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "PUT",
"path": "/api/v1/admin/uo-link/config",
"handlers": 8,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/uo-link/stream",
"handlers": 2,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "POST",
"path": "/api/v1/admin/uo-link/towncrier",
"handlers": 7,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "DELETE",
"path": "/api/v1/admin/uo-link/towncrier/:id",
"handlers": 4,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "POST",
"path": "/api/v1/admin/uploads",
@@ -1037,72 +720,6 @@
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/users/:id/shard/accounts",
"handlers": 3,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/users/:id/shard/houses",
"handlers": 3,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "DELETE",
"path": "/api/v1/admin/users/:id/shard/link/:account",
"handlers": 5,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/users/:id/shard/online",
"handlers": 3,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/users/:id/shard/sales",
"handlers": 3,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/admin/users/:id/shard/standing",
"handlers": 3,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "DELETE",
"path": "/api/v1/admin/users/:id/trusted-devices",
@@ -1798,146 +1415,6 @@
"requireAuth"
]
},
{
"method": "POST",
"path": "/api/v1/player/shard/account",
"handlers": 5,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/player/shard/accounts",
"handlers": 1,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "GET",
"path": "/api/v1/player/shard/char/:serial",
"handlers": 3,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/player/shard/houses",
"handlers": 1,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "POST",
"path": "/api/v1/player/shard/link",
"handlers": 3,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/player/shard/roster/:account",
"handlers": 3,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/player/shard/sales",
"handlers": 1,
"gates": [
"noindex",
"requireAuth"
]
},
{
"method": "GET",
"path": "/api/v1/player/shard/vendors/:account",
"handlers": 3,
"gates": [
"noindex",
"requireAuth",
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/public/atlas/champions",
"handlers": 5,
"gates": [
"middleware",
"validate",
"siteMode"
]
},
{
"method": "GET",
"path": "/api/v1/public/atlas/creatures",
"handlers": 8,
"gates": [
"middleware",
"validate",
"siteMode"
]
},
{
"method": "GET",
"path": "/api/v1/public/atlas/creatures/:slug",
"handlers": 7,
"gates": [
"middleware",
"validate",
"siteMode"
]
},
{
"method": "GET",
"path": "/api/v1/public/atlas/landmarks",
"handlers": 6,
"gates": [
"middleware",
"validate",
"siteMode"
]
},
{
"method": "GET",
"path": "/api/v1/public/atlas/meta",
"handlers": 3,
"gates": [
"siteMode"
]
},
{
"method": "GET",
"path": "/api/v1/public/atlas/regions",
"handlers": 6,
"gates": [
"middleware",
"validate",
"siteMode"
]
},
{
"method": "POST",
"path": "/api/v1/public/contact",
@@ -1947,6 +1424,12 @@
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/public/modules",
"handlers": 1,
"gates": []
},
{
"method": "GET",
"path": "/api/v1/public/pages/:id/preview/:token",
@@ -1983,135 +1466,6 @@
"handlers": 1,
"gates": []
},
{
"method": "GET",
"path": "/api/v1/public/shard/champs",
"handlers": 2,
"gates": []
},
{
"method": "GET",
"path": "/api/v1/public/shard/economy",
"handlers": 4,
"gates": [
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/public/shard/features",
"handlers": 1,
"gates": []
},
{
"method": "GET",
"path": "/api/v1/public/shard/feed",
"handlers": 5,
"gates": [
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/public/shard/governors",
"handlers": 2,
"gates": []
},
{
"method": "GET",
"path": "/api/v1/public/shard/governors/:city/history",
"handlers": 5,
"gates": [
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/public/shard/guilds",
"handlers": 2,
"gates": []
},
{
"method": "GET",
"path": "/api/v1/public/shard/houses",
"handlers": 2,
"gates": []
},
{
"method": "GET",
"path": "/api/v1/public/shard/idoc",
"handlers": 2,
"gates": []
},
{
"method": "GET",
"path": "/api/v1/public/shard/market",
"handlers": 13,
"gates": [
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/public/shard/market/meta",
"handlers": 2,
"gates": []
},
{
"method": "GET",
"path": "/api/v1/public/shard/market/vendors/:serial",
"handlers": 7,
"gates": [
"middleware",
"validate"
]
},
{
"method": "GET",
"path": "/api/v1/public/shard/online",
"handlers": 2,
"gates": []
},
{
"method": "GET",
"path": "/api/v1/public/shard/points",
"handlers": 2,
"gates": []
},
{
"method": "GET",
"path": "/api/v1/public/shard/points/:system",
"handlers": 2,
"gates": []
},
{
"method": "GET",
"path": "/api/v1/public/shard/presence",
"handlers": 2,
"gates": []
},
{
"method": "GET",
"path": "/api/v1/public/shard/ruleset",
"handlers": 2,
"gates": []
},
{
"method": "GET",
"path": "/api/v1/public/shard/status",
"handlers": 2,
"gates": []
},
{
"method": "GET",
"path": "/api/v1/public/shard/stream",
"handlers": 1,
"gates": []
},
{
"method": "GET",
"path": "/api/v1/public/status",

284
server/routes.manifest.json
View File

@@ -257,134 +257,10 @@
"method": "POST",
"path": "/api/v1/admin/settings/brand-asset/:slot"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/account"
},
{
"method": "GET",
"path": "/api/v1/admin/shard/accounts"
},
{
"method": "GET",
"path": "/api/v1/admin/shard/atlas"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/atlas/approve"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/atlas/import"
},
{
"method": "PUT",
"path": "/api/v1/admin/shard/atlas/path"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/atlas/reject"
},
{
"method": "GET",
"path": "/api/v1/admin/shard/audit"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/ban"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/broadcast"
},
{
"method": "GET",
"path": "/api/v1/admin/shard/char/:serial"
},
{
"method": "GET",
"path": "/api/v1/admin/shard/clilocs"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/clilocs/import"
},
{
"method": "PUT",
"path": "/api/v1/admin/shard/clilocs/path"
},
{
"method": "GET",
"path": "/api/v1/admin/shard/houses"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/kick"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/link"
},
{
"method": "GET",
"path": "/api/v1/admin/shard/pages"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/pages/:id/close"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/pages/:id/respond"
},
{
"method": "GET",
"path": "/api/v1/admin/shard/roster/:account"
},
{
"method": "GET",
"path": "/api/v1/admin/shard/sales"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/unban"
},
{
"method": "GET",
"path": "/api/v1/admin/shard/vendors/:account"
},
{
"method": "GET",
"path": "/api/v1/admin/shard/visibility"
},
{
"method": "PUT",
"path": "/api/v1/admin/shard/visibility"
},
{
"method": "PUT",
"path": "/api/v1/admin/site-mode"
},
{
"method": "GET",
"path": "/api/v1/admin/uo-link/config"
},
{
"method": "PUT",
"path": "/api/v1/admin/uo-link/config"
},
{
"method": "GET",
"path": "/api/v1/admin/uo-link/stream"
},
{
"method": "POST",
"path": "/api/v1/admin/uo-link/towncrier"
},
{
"method": "DELETE",
"path": "/api/v1/admin/uo-link/towncrier/:id"
},
{
"method": "POST",
"path": "/api/v1/admin/uploads"
@@ -413,30 +289,6 @@
"method": "POST",
"path": "/api/v1/admin/users/:id/mfa/reset"
},
{
"method": "GET",
"path": "/api/v1/admin/users/:id/shard/accounts"
},
{
"method": "GET",
"path": "/api/v1/admin/users/:id/shard/houses"
},
{
"method": "DELETE",
"path": "/api/v1/admin/users/:id/shard/link/:account"
},
{
"method": "GET",
"path": "/api/v1/admin/users/:id/shard/online"
},
{
"method": "GET",
"path": "/api/v1/admin/users/:id/shard/sales"
},
{
"method": "GET",
"path": "/api/v1/admin/users/:id/shard/standing"
},
{
"method": "DELETE",
"path": "/api/v1/admin/users/:id/trusted-devices"
@@ -721,66 +573,14 @@
"method": "GET",
"path": "/api/v1/player/appeals/eligible"
},
{
"method": "POST",
"path": "/api/v1/player/shard/account"
},
{
"method": "GET",
"path": "/api/v1/player/shard/accounts"
},
{
"method": "GET",
"path": "/api/v1/player/shard/char/:serial"
},
{
"method": "GET",
"path": "/api/v1/player/shard/houses"
},
{
"method": "POST",
"path": "/api/v1/player/shard/link"
},
{
"method": "GET",
"path": "/api/v1/player/shard/roster/:account"
},
{
"method": "GET",
"path": "/api/v1/player/shard/sales"
},
{
"method": "GET",
"path": "/api/v1/player/shard/vendors/:account"
},
{
"method": "GET",
"path": "/api/v1/public/atlas/champions"
},
{
"method": "GET",
"path": "/api/v1/public/atlas/creatures"
},
{
"method": "GET",
"path": "/api/v1/public/atlas/creatures/:slug"
},
{
"method": "GET",
"path": "/api/v1/public/atlas/landmarks"
},
{
"method": "GET",
"path": "/api/v1/public/atlas/meta"
},
{
"method": "GET",
"path": "/api/v1/public/atlas/regions"
},
{
"method": "POST",
"path": "/api/v1/public/contact"
},
{
"method": "GET",
"path": "/api/v1/public/modules"
},
{
"method": "GET",
"path": "/api/v1/public/pages/:id/preview/:token"
@@ -801,82 +601,6 @@
"method": "GET",
"path": "/api/v1/public/settings"
},
{
"method": "GET",
"path": "/api/v1/public/shard/champs"
},
{
"method": "GET",
"path": "/api/v1/public/shard/economy"
},
{
"method": "GET",
"path": "/api/v1/public/shard/features"
},
{
"method": "GET",
"path": "/api/v1/public/shard/feed"
},
{
"method": "GET",
"path": "/api/v1/public/shard/governors"
},
{
"method": "GET",
"path": "/api/v1/public/shard/governors/:city/history"
},
{
"method": "GET",
"path": "/api/v1/public/shard/guilds"
},
{
"method": "GET",
"path": "/api/v1/public/shard/houses"
},
{
"method": "GET",
"path": "/api/v1/public/shard/idoc"
},
{
"method": "GET",
"path": "/api/v1/public/shard/market"
},
{
"method": "GET",
"path": "/api/v1/public/shard/market/meta"
},
{
"method": "GET",
"path": "/api/v1/public/shard/market/vendors/:serial"
},
{
"method": "GET",
"path": "/api/v1/public/shard/online"
},
{
"method": "GET",
"path": "/api/v1/public/shard/points"
},
{
"method": "GET",
"path": "/api/v1/public/shard/points/:system"
},
{
"method": "GET",
"path": "/api/v1/public/shard/presence"
},
{
"method": "GET",
"path": "/api/v1/public/shard/ruleset"
},
{
"method": "GET",
"path": "/api/v1/public/shard/status"
},
{
"method": "GET",
"path": "/api/v1/public/shard/stream"
},
{
"method": "GET",
"path": "/api/v1/public/status"

127
server/scripts/importSpawnAtlas.js
View File

@@ -1,127 +0,0 @@
#!/usr/bin/env node
//
// Refresh the spawn atlas from a ServUO tree, from the command line.
//
// npm run atlas:import # use the configured path
// npm run atlas:import -- --servuo <path> # override it for this run
// npm run atlas:import -- --force # reimport even if unchanged
// npm run atlas:import -- --approve # apply a staged refresh
// npm run atlas:import -- --status # report without changing anything
//
// The server does this itself on every boot (see `shardAtlas.refreshOnBoot`), so
// this is for operators who want to apply a map change without a restart, and
// for approving a refresh that was staged because it would remove a facet.
//
// All the logic lives in `src/model/shardAtlas/shardAtlas.model.js`; this file
// is argument parsing and output formatting.
const db = () => require('../src/utils/db')
function parseArgs(argv) {
const args = {}
for (let i = 0; i < argv.length; i += 1) {
const flag = argv[i]
if (flag === '--servuo') args.servuo = argv[++i]
else if (flag === '--force') args.force = true
else if (flag === '--approve') args.approve = true
else if (flag === '--reject') args.reject = true
else if (flag === '--status') args.status = true
else if (flag === '--help' || flag === '-h') args.help = true
}
return args
}
const USAGE = `
Refresh the spawn atlas from a ServUO tree.
node scripts/importSpawnAtlas.js [options]
--servuo <path> Use this tree for this run instead of the configured path.
--force Reimport even when the source files are unchanged.
--approve Apply a refresh that was staged for removing a facet.
--reject Keep the current atlas and dismiss the staged refresh.
--status Report atlas and source state; change nothing.
With no options this imports only if the tree differs from what is loaded.
`
function describe(result) {
switch (result.status) {
case 'skipped':
return (
'No ServUO path configured — nothing to import.\n' +
'Set one with SERVUO_PATH, the admin panel, or --servuo <path>.\n'
)
case 'unavailable':
return `ServUO tree unavailable: ${result.reason}\n`
case 'unchanged':
return `Atlas is already up to date${result.reason ? ` (${result.reason})` : ''}.\n`
case 'needsReview': {
return (
'Refresh NOT applied — it would remove ' +
`${result.removedFacets.length} facet(s): ${result.removedFacets.join(', ')}.\n` +
'This is what a half-copied or mid-update tree looks like, so it has been\n' +
'staged for review. The current atlas is unchanged.\n' +
'Apply it with --approve, or dismiss it with --reject.\n'
)
}
case 'imported': {
const c = result.counts
const added = result.addedFacets?.length ? ` Added facets: ${result.addedFacets.join(', ')}.` : ''
const removed = result.removedFacets?.length
? ` Removed facets: ${result.removedFacets.join(', ')}.`
: ''
return (
`Atlas imported: ${c.points} points, ${c.creatures} creatures, ` +
`${c.pointTypes} point/type rows, ${c.regions} regions, ` +
`${c.landmarks} landmarks, ${c.champions} champion altars.${added}${removed}\n`
)
}
case 'failed':
return `Atlas refresh failed: ${result.reason}\n`
default:
return `${JSON.stringify(result, null, 2)}\n`
}
}
async function main() {
const args = parseArgs(process.argv.slice(2))
if (args.help) {
process.stdout.write(USAGE)
return
}
const shardAtlas = require('../src/model/shardAtlas/shardAtlas.model')
// `--servuo` is a per-run override and deliberately does NOT persist to the
// configured path; changing where the atlas permanently reads from is an
// admin action, not a side effect of a one-off import.
const override = { path: args.servuo ?? '' }
if (args.status) {
process.stdout.write(`${JSON.stringify(await shardAtlas.status(override), null, 2)}\n`)
return
}
if (args.reject) {
process.stdout.write(`${JSON.stringify(await shardAtlas.rejectPending(), null, 2)}\n`)
return
}
const result = args.approve
? await shardAtlas.approvePending(override)
: await shardAtlas.refresh({ ...override, force: Boolean(args.force) })
process.stdout.write(describe(result))
if (result.status === 'failed') process.exitCode = 1
}
if (require.main === module) {
main()
.catch((err) => {
process.stderr.write(`atlas:import failed: ${err.message}\n`)
process.exitCode = 1
})
.finally(() => db().close())
}
module.exports = { describe, parseArgs }

31
server/scripts/routeManifest.js
View File

@@ -16,10 +16,11 @@
* derived (only annotated routes appear) and documents intent; this records reality.
*
* Scope: only `/api/**` and `/.well-known/**` from the public app, plus everything
* on the internal app. Three mounts in app.js are *filesystem* conditional — the SPA
* catch-all `GET *`, the `/brand` static mount and swagger-ui's `/api/docs` static
* assets — so including them would make the output depend on whether CI had built
* the client. Static mounts are not API contract.
* on the internal app. Four mounts in app.js are *filesystem* conditional — the SPA
* catch-all `GET *`, the `/brand` static mount, installed modules' `/modules/<id>`
* chunks and swagger-ui's `/api/docs` static assets — so including them would make
* the output depend on whether CI had built the client, or on which modules were
* mounted. Static mounts are not API contract.
*
* Usage:
* npm run routes:manifest # write server/routes.manifest.json (+ guards)
@@ -56,7 +57,8 @@ const GUARDS_COMMENT =
'`npm run routes:manifest`.'
// Only these prefixes are contract. Everything else the public app serves (SPA
// shell, /uploads, /brand, swagger-ui assets) is static delivery, not API surface.
// shell, /uploads, /brand, /modules, swagger-ui assets) is static delivery, not
// API surface.
const PUBLIC_PREFIXES = ['/api/', '/.well-known/']
/**
@@ -64,9 +66,16 @@ const PUBLIC_PREFIXES = ['/api/', '/.well-known/']
*
* Express keeps no copy of the mount string, only the compiled regexp. For a
* literal mount (`/api/v1`) that is `^\/api\/v1\/?(?=\/|$)`; a parameterised mount
* contributes one `(?:([^\/]+?))` group per entry in `layer.keys`. Unwinding both
* gets us back to `/api/v1` and `/thing/:id` respectively. `fast_slash` is
* express's marker for a router mounted at the root, which contributes nothing.
* contributes one group per entry in `layer.keys`, and the separator before the
* parameter lives INSIDE that group — express 4.22 compiles `use('/:id', r)` to
* `^(?:\/([^/]+?))\/?(?=\/|$)`. Unwinding both gets us back to `/api/v1` and
* `/:id` respectively. `fast_slash` is express's marker for a router mounted at
* the root, which contributes nothing.
*
* The parameterised branch went unexercised until the `admin.users.detail`
* extension slot mounted a router at `/:id` (MODULE_SYSTEM.md §1.9), and it was
* wrong: it expected the group as `(?:([^\/]+?))`, with the slash outside and the
* class escaped. It threw rather than guessing, which is exactly what it is for.
*/
function mountPath(layer) {
const re = layer.regexp
@@ -79,9 +88,11 @@ function mountPath(layer) {
const keys = layer.keys || []
let i = 0
src = src.replace(/\(\?:\(\[\^\\\/\]\+\?\)\)/g, () => {
// `\/` optional and the `/` in the class optionally escaped, so this survives a
// path-to-regexp that emits either shape.
src = src.replace(/\((?:\?:)?(\\\/)?\(\[\^\\?\/\]\+\?\)\)/g, (_m, slash) => {
const key = keys[i++]
return key ? `:${key.name}` : ':param'
return `${slash ? '/' : ''}:${key ? key.name : 'param'}`
})
// Whatever is left should be a literal path with regexp-escaped separators.

71
server/src/app.js
View File

@@ -10,6 +10,8 @@ require('dotenv').config()
const swaggerUi = require('swagger-ui-express')
const apiRouter = require('./router/api.router')
const modules = require('./modules/loader')
const registries = require('./modules/registries')
const wellKnown = require('./router/wellKnown.controller')
const cspReport = require('./router/cspReport.controller')
const brand = require('./config/brand')
@@ -154,8 +156,77 @@ app.get(
app.post(csp.REPORT_PATH, cspReportLimiter, ...cspReport.parsers, cspReport.receive)
app.use('/api', apiRouter)
// ── Installed modules ─────────────────────────────────────────────────
// Discover, validate and mount whatever is on the modules volume
// (docs/website/MODULE_API.md Part 4). One explicit call, here and nowhere else:
// the loader has no lazy self-scan, so there is exactly one place that decides
// when modules are discovered, and reading the module list before this line is
// an error rather than a silent empty answer (§7.6).
//
// Position is load-bearing, in both directions. It is AFTER `/api` is mounted,
// so every core prefix is already on the tier routers when the collision check
// asks them what core owns — and so first-match-wins means a module physically
// cannot shadow a core route. It is BEFORE the `/api` 404 below, so a module
// route reaches its handler instead of the catch-all.
//
// The three requires resolve from cache to the very routers v1.router.js
// mounted; this is a reference to them, not a second copy.
//
// registerCore() first, and for the same reason the loader runs after `/api`: a
// module's collision checks are asked against what is ALREADY registered, so
// core's streams, its announce leg and its extension-slot fill have to be there
// before the first module registers anything (MODULE_SYSTEM.md §1.8).
registries.registerCore()
modules.load({
public: require('./router/v1/public'),
admin: require('./router/v1/admin'),
player: require('./router/v1/player'),
})
app.use('/api', (req, res) => res.status(404).json({ message: 'Not found' }))
// Installed modules' prebuilt client chunks, at /modules/<id>/ — same-origin, so
// `script-src 'self'` admits them with no nonce and no inline script
// (docs/website/MODULE_API.md §3.1). Three properties, each load-bearing:
//
// • The static root is the directory the ENTRY sits in, never the module root.
// One express.static over a module root would publish its server source, its
// module.json and its schema fragment; the loader rejects an entry that would
// make those the same directory.
// • Behind the module's own state guard, so a failed module's chunk is 503 and
// a disabled one's is 404 — the same answers its API gives, for the same
// reason: the browser should not be running the client half of something the
// server half has stopped serving.
// • `fallthrough: false`, so a missing file is a 404 here rather than falling
// through to the SPA catch-all and answering a `<script src>` with the index
// shell, which the browser then rejects on its MIME type instead.
//
// Vite's library build emits an unhashed `entry.js`, so `no-cache` (revalidate,
// not "do not store") is what stops an upgraded module serving yesterday's chunk
// out of the disk cache.
for (const chunk of modules.clientChunks()) {
app.use(
chunk.url,
chunk.guard,
express.static(chunk.dir, {
fallthrough: false,
setHeaders: (res) => {
res.set('Cache-Control', 'no-cache')
res.set('X-Content-Type-Options', 'nosniff')
},
}),
)
}
// Everything else under /modules is a 404, not the SPA shell. The namespace
// belongs to installed modules' chunks — an unknown module id or a file a module
// does not ship is a missing file, and answering a `<script src>` with an HTML
// page turns that into a MIME-type refusal in the console with a 200 in the
// network tab. It also keeps the namespace's boundary a fact of the app rather
// than of whichever catch-all happens to be mounted after it.
app.use('/modules', (req, res) => res.status(404).json({ message: 'Not found' }))
// ── /.well-known ──────────────────────────────────────────────────────
// Android App Links verification file at the web root (M9 follow-up). Mounted
// before the SPA catch-all so it returns JSON, not the index shell. 404s unless

26
server/src/config/coreStreams.js Normal file
View File

@@ -0,0 +1,26 @@
// ── Core's own push-notification streams ───────────────────────────────────
//
// What is left of config/notificationStreams.js once the shard-derived catalog
// moved to config/shardStreams.js (MODULE_SYSTEM.md §1.8: push INFRASTRUCTURE is
// core, the CATALOG is content). Exactly one stream is core's: `news.post` is
// produced by the website's own posts path, not by any game feed.
//
// Registered through modules/registries.js like any module's, and read back
// through it — nothing imports this file to get "the catalog", because the
// catalog is core's plus every module's.
//
// The payload that ever leaves the server is a CONTENT-FREE tickle
// ({ stream, ref }); the app wakes and PULLS the real, ownership-checked content
// over the authenticated API (docs/android/PLAN.md §11).
const STREAMS = [
{
id: 'news.post',
label: 'News posts',
description: 'New news / Five-on-Friday / newsletter posts.',
personal: false,
requiresLinkedAccount: false,
},
]
module.exports = { STREAMS }

168
server/src/config/notificationStreams.js
View File

@@ -1,168 +0,0 @@
// ── Push-notification stream catalog + event → stream mapping ───────────────
//
// The single source of truth for which streams a user can subscribe to, and how
// a shard event maps onto them. Two families:
// • public / opt-in — no linked game account required; delivered to every
// subscriber. Drawn ONLY from the SSE public allowlist
// (utils/shardBroadcast PUBLIC_KINDS) — a sensitive kind
// can never produce a public push.
// • personal / owner-keyed — require a linked game account; delivered ONLY to
// the owning user's devices (resolved from the event's
// game account via shardLinks), never fanned out publicly.
//
// The payload the relay ever carries is a CONTENT-FREE tickle ({ stream, ref });
// `ref` is an opaque hint (serial / city / timestamp) the app uses to pull the
// real, ownership-checked content over the authenticated API. So even a leaked
// ntfy topic reveals nothing (docs/android/PLAN.md §11).
const { PUBLIC_KINDS } = require('../utils/shardBroadcast')
// The subscribable catalog. `news.post` is produced by the website's own posts
// path (not the shard feed) — see utils/pushDispatch — so it has no mapShardEvent
// case; every other stream is shard-derived below.
const STREAMS = [
{
id: 'news.post',
label: 'News posts',
description: 'New news / Five-on-Friday / newsletter posts.',
personal: false,
requiresLinkedAccount: false,
},
{
id: 'server.status',
label: 'Server up / down',
description: 'The shard comes online or goes offline.',
personal: false,
requiresLinkedAccount: false,
},
{
id: 'idoc.warning',
label: 'IDOC warnings',
description: 'A house falls into its final (IDOC) decay stage.',
personal: false,
requiresLinkedAccount: false,
},
{
id: 'champ.start',
label: 'Champion spawn starts',
description: 'A champion spawn becomes active.',
personal: false,
requiresLinkedAccount: false,
},
{
id: 'governor.election',
label: 'Governor elections',
description: 'A town elects a new governor.',
personal: false,
requiresLinkedAccount: false,
},
{
id: 'vendor.sale',
label: 'Your vendor sold an item',
description: 'One of your player vendors made a sale.',
personal: true,
requiresLinkedAccount: true,
},
{
id: 'house.idoc',
label: 'Your house entered IDOC',
description: 'One of your houses fell into its final decay stage.',
personal: true,
requiresLinkedAccount: true,
},
{
id: 'account.login',
label: 'A login to your account',
description: 'An authentication attempt against your game account.',
personal: true,
requiresLinkedAccount: true,
},
]
const STREAM_IDS = new Set(STREAMS.map((s) => s.id))
const isValidStream = (id) => STREAM_IDS.has(id)
const PERSONAL_STREAMS = new Set(STREAMS.filter((s) => s.personal).map((s) => s.id))
// Per-process transition state so full-state upserts (champ.update / city.update
// are upserts, not discrete "started"/"elected" events — see docs/link
// PROTOCOL_2 §383) only fire once, on an actual transition. Injectable so tests
// pass a fresh tracker; a module-level default backs the live dispatcher.
function createTracker() {
return { champActive: new Map(), cityGovernor: new Map() }
}
const defaultTracker = createTracker()
// Per-kind mappers, each pushing 0+ targets onto `out` (and updating `tracker`
// for the upsert-transition kinds). Split out of mapShardEvent so that function
// stays a trivial dispatch + the public-safety filter.
const serverStatusUp = (event, tracker, out) =>
out.push({ streamId: 'server.status', ref: `up:${event.bootId || ''}` })
const serverStatusDown = (event, tracker, out) => out.push({ streamId: 'server.status', ref: 'down' })
const EVENT_MAPPERS = {
'server.hello': serverStatusUp,
'server.shutdown': serverStatusDown,
'server.crashed': serverStatusDown,
'house.decay': (event, tracker, out) => {
if (String(event.to).toUpperCase() !== 'IDOC') return
const ref = String(event.serial ?? '')
out.push({ streamId: 'idoc.warning', ref }) // public — location only
if (event.ownerAcct) {
out.push({ streamId: 'house.idoc', ref, ownerAccount: event.ownerAcct }) // personal
}
},
'champ.update': (event, tracker, out) => {
const { serial } = event
if (serial == null) return
const wasActive = tracker.champActive.get(serial) === true
const isActive = event.active === true
tracker.champActive.set(serial, isActive)
if (isActive && !wasActive) out.push({ streamId: 'champ.start', ref: String(serial) })
},
'champ.remove': (event, tracker) => {
if (event.serial != null) tracker.champActive.delete(event.serial)
},
'city.update': (event, tracker, out) => {
const { city } = event
if (!city) return
const gov = event.governor && event.governor.serial != null ? String(event.governor.serial) : null
const prev = tracker.cityGovernor.get(city)
tracker.cityGovernor.set(city, gov)
// Only a real transition to a new governor, and never on first sight
// (prev === undefined) so a reconnect snapshot isn't read as an election.
if (prev !== undefined && gov && gov !== prev) {
out.push({ streamId: 'governor.election', ref: String(city) })
}
},
'vendor.sale': (event, tracker, out) => {
if (event.ownerAcct) {
out.push({ streamId: 'vendor.sale', ref: String(event.t ?? ''), ownerAccount: event.ownerAcct })
}
},
'account.login.attempt': (event, tracker, out) => {
if (event.acct) {
out.push({ streamId: 'account.login', ref: String(event.t ?? ''), ownerAccount: event.acct })
}
},
}
// Map one shard event → an array of targets ({ streamId, ref, ownerAccount? }).
// May yield 0, 1, or 2 targets (an owner house.decay produces both the public
// idoc.warning and the personal house.idoc). Pure given `tracker`.
function mapShardEvent(event, tracker = defaultTracker) {
if (!event || typeof event.kind !== 'string') return []
const kind = event.kind
const out = []
const mapper = EVENT_MAPPERS[kind]
if (mapper) mapper(event, tracker, out)
// Defense in depth: a PUBLIC (non-personal) target may only ride a public-safe
// kind. Personal targets are owner-keyed and delivered solely to the owner, so
// they are exempt from the public allowlist (that is the whole point of the
// owner-keyed split). This guarantees a sensitive kind can never leak publicly
// even if a future mapping case is added carelessly.
return out.filter((t) => (PERSONAL_STREAMS.has(t.streamId) ? true : PUBLIC_KINDS.has(kind)))
}
module.exports = { STREAMS, isValidStream, mapShardEvent, createTracker, PERSONAL_STREAMS }

22
server/src/middleware/rateLimit.js
View File

@@ -118,19 +118,6 @@ const passwordResetConfirmLimiter = makeLimiter({
message: 'Too many attempts. Please try again later.',
})
// The player-vendor market search. The first genuinely expensive PUBLIC endpoint
// on the site: every call is a LIKE scan plus a COUNT over the listings table,
// which on a large shard is the biggest table there is, and it is anonymous by
// default. Generous for a human browsing shops (a typed search is debounced to
// one request, and paging is a click), tight enough that it cannot be used as a
// cheap way to load the database.
const marketLimiter = makeLimiter({
windowMs: 60 * 1000,
max: 60,
label: 'market',
message: 'Too many searches. Please slow down.',
})
// CSP violation reports. Unauthenticated by necessity (browsers send them with no
// session), and every accepted report writes a log line — so an attacker who can get
// a victim to load a page could otherwise use it as a log-flood amplifier. Generous
@@ -144,6 +131,14 @@ const cspReportLimiter = makeLimiter({
})
module.exports = {
// Exported for modules (MODULE_API.md 2.3, added in API 1.1.0). A module
// writes its own policy -- the window and the cap are its business, since it
// knows what its endpoints cost -- but it takes the PLUMBING from here: one
// express-rate-limit in the process, one store, and one place limit breaches
// are logged. A module resolving the package itself would get a second store,
// and a limit enforced by two independent counters is not the limit either of
// them states.
makeLimiter,
loginLimiter,
registerLimiter,
accountChangeLimiter,
@@ -154,6 +149,5 @@ module.exports = {
mobileSsoExchangeLimiter,
passwordResetRequestLimiter,
passwordResetConfirmLimiter,
marketLimiter,
cspReportLimiter,
}

114
server/src/model/announceJobs/announceJobs.db.js
View File

@@ -1,26 +1,68 @@
// ── Announcement pipeline: SQL ─────────────────────────────────────────────
//
// Two tables since PR 4 (MODULE_SYSTEM.md §1.8): `announce_jobs` is one row per
// publish event, `announce_job_legs` one row per delivery leg of that job. The
// leg set is registered rather than fixed, so a leg is a stored VALUE now instead
// of a group of leg-prefixed columns — which is what lets a module bring its own
// leg without altering a core table.
//
// Every read returns the job with a `legs` array attached, so a caller never has
// to remember to fetch the second table.
const { query } = require('../../utils/db')
const COLS =
'id, post_id, status, ' +
'towncrier_status, towncrier_attempts, towncrier_last_error, towncrier_next_attempt_at, ' +
'discord_status, discord_attempts, discord_last_error, discord_next_attempt_at, ' +
'created_at, updated_at'
const COLS = 'id, post_id, status, created_at, updated_at'
const LEG_COLS = 'job_id, leg, status, attempts, last_error, next_attempt_at'
// Whitelist so a `leg` value can be interpolated into a column name safely — it
// never comes from raw user input, but keep the guard explicit.
const LEGS = ['towncrier', 'discord']
function assertLeg(leg) {
if (!LEGS.includes(leg)) throw new Error(`unknown announce leg: ${leg}`)
async function legsFor(jobIds) {
if (jobIds.length === 0) return new Map()
const marks = jobIds.map(() => '?').join(', ')
const rows = await query(
`SELECT ${LEG_COLS} FROM announce_job_legs WHERE job_id IN (${marks}) ORDER BY job_id, leg`,
jobIds,
)
const byJob = new Map(jobIds.map((id) => [id, []]))
for (const row of rows) byJob.get(row.job_id).push(row)
return byJob
}
async function create(postId) {
async function attachLegs(jobs) {
const byJob = await legsFor(jobs.map((j) => j.id))
for (const job of jobs) job.legs = byJob.get(job.id) || []
return jobs
}
// Create a job and its leg rows in one go. `legs` is the registered leg id list —
// an empty list is legal and yields a job with nothing to deliver.
async function create(postId, legs = []) {
const res = await query('INSERT INTO announce_jobs (post_id) VALUES (?)', [postId])
return res.insertId
const jobId = Number(res.insertId)
if (legs.length > 0) {
const values = legs.map(() => '(?, ?)').join(', ')
await query(
`INSERT INTO announce_job_legs (job_id, leg) VALUES ${values}`,
legs.flatMap((leg) => [jobId, leg]),
)
}
return jobId
}
// Add any registered legs this job is missing. A job enqueued before a module was
// installed has no row for that module's leg, and without this it could never
// deliver one — the worker only ever sees rows that exist.
async function ensureLegs(jobId, legs = []) {
if (legs.length === 0) return
const values = legs.map(() => '(?, ?)').join(', ')
await query(
`INSERT IGNORE INTO announce_job_legs (job_id, leg) VALUES ${values}`,
legs.flatMap((leg) => [jobId, leg]),
)
}
async function findById(id) {
const rows = await query(`SELECT ${COLS} FROM announce_jobs WHERE id = ? LIMIT 1`, [id])
return rows[0] || null
if (rows.length === 0) return null
return (await attachLegs(rows))[0]
}
async function findByPostId(postId) {
@@ -28,36 +70,38 @@ async function findByPostId(postId) {
`SELECT ${COLS} FROM announce_jobs WHERE post_id = ? ORDER BY id DESC LIMIT 1`,
[postId],
)
return rows[0] || null
if (rows.length === 0) return null
return (await attachLegs(rows))[0]
}
// Jobs with at least one leg that is due now: pending and either never scheduled
// (next_attempt_at IS NULL — a fresh enqueue) or past its backoff time.
// (next_attempt_at IS NULL — a fresh enqueue) or past its backoff time. Returns
// whole jobs with every leg attached; the worker decides which legs to run, so
// this stays one query regardless of how many legs are registered.
async function findDue(now = new Date(), limit = 25) {
return query(
`SELECT ${COLS} FROM announce_jobs
WHERE (towncrier_status = 'pending'
AND (towncrier_next_attempt_at IS NULL OR towncrier_next_attempt_at <= ?))
OR (discord_status = 'pending'
AND (discord_next_attempt_at IS NULL OR discord_next_attempt_at <= ?))
ORDER BY id ASC
const rows = await query(
`SELECT ${COLS} FROM announce_jobs j
WHERE EXISTS (
SELECT 1 FROM announce_job_legs l
WHERE l.job_id = j.id
AND l.status = 'pending'
AND (l.next_attempt_at IS NULL OR l.next_attempt_at <= ?))
ORDER BY j.id ASC
LIMIT ?`,
[now, now, limit],
[now, limit],
)
return attachLegs(rows)
}
// Update one leg's columns. `fields` uses leg-agnostic keys (status, attempts,
// lastError, nextAttemptAt); we map them onto the leg-prefixed columns.
async function updateLeg(id, leg, { status, attempts, lastError, nextAttemptAt }) {
assertLeg(leg)
// Update one leg's row. `leg` is a bound VALUE, not an interpolated column name —
// the reason the old leg allowlist that guarded that interpolation is gone. A
// module's leg id could not have passed it anyway.
async function updateLeg(jobId, leg, { status, attempts, lastError, nextAttemptAt }) {
await query(
`UPDATE announce_jobs SET
${leg}_status = ?,
${leg}_attempts = ?,
${leg}_last_error = ?,
${leg}_next_attempt_at = ?
WHERE id = ?`,
[status, attempts, lastError ?? null, nextAttemptAt ?? null, id],
`UPDATE announce_job_legs SET
status = ?, attempts = ?, last_error = ?, next_attempt_at = ?
WHERE job_id = ? AND leg = ?`,
[status, attempts, lastError ?? null, nextAttemptAt ?? null, jobId, leg],
)
}
@@ -65,4 +109,4 @@ async function setStatus(id, status) {
await query('UPDATE announce_jobs SET status = ? WHERE id = ?', [status, id])
}
module.exports = { LEGS, create, findById, findByPostId, findDue, updateLeg, setStatus }
module.exports = { create, ensureLegs, findById, findByPostId, findDue, updateLeg, setStatus }

120
server/src/model/announceJobs/announceJobs.logic.js
View File

@@ -1,87 +1,38 @@
// ── Announcement pipeline: pure logic ──────────────────────────────────────
//
// No DB, no network — just the decisions the worker and model make, kept here so
// they are unit-testable in isolation (server/test/announceJobs.test.js):
// • buildTownCrierText — turn a post into sidecar-safe town-crier lines
// • classifyTownCrier / classifyDiscord — map a dispatch result to done / retry
// / terminal, so a data problem fails fast and a transient outage retries
// No DB, no network — just the LEG-AGNOSTIC decisions the worker and model make,
// kept here so they are unit-testable in isolation (server/test/announceJobs.test.js):
// • scheduleAfter — exponential backoff schedule + the attempt cap
// • rollupStatus — derive the parent job status from the two legs
const { deriveExcerpt } = require('../../utils/sanitizeHtml')
// Sidecar town-crier caps, mirrored from the admin route validation
// (admin/uoLink.router.js: lines isArray({ max: 8 }), lines.* isLength({ max: 200 })).
// We pre-truncate to these so a published post never bounces with towncrier.error.
const MAX_LINES = 8
const MAX_LINE_LEN = 200
// • rollupStatus — derive the parent job status from its legs
// • legError — squeeze a client result into one error line
// • baseUrl / articleUrl — the public link an announcement carries
//
// What used to be here and is not any more: `buildTownCrierText`,
// `classifyTownCrier` and `classifyDiscord`. A leg's own text-building and result
// classification belong to the leg, and a leg is a registration now
// (MODULE_SYSTEM.md §1.8) — they live in utils/shardAnnounce.js and
// utils/discordAnnounce.js. This file is what every leg shares.
// Backoff between retries, indexed by attempts-so-far. Six attempts spread over
// ~a couple of hours; after the last one a leg is marked failed and surfaced in
// the post's admin panel. Shared by both legs.
// the post's admin panel. Shared by every leg.
const BACKOFF_MS = [30_000, 120_000, 600_000, 1_800_000, 3_600_000, 7_200_000]
const MAX_ATTEMPTS = BACKOFF_MS.length
// Trim to a hard length, appending an ellipsis only when something was cut.
function clamp(value, max) {
const s = String(value == null ? '' : value)
.replace(/\s+/g, ' ')
.trim()
if (s.length <= max) return s
return `${s.slice(0, max - 1).trimEnd()}…`
// The site's public base, used to build the link an announcement carries.
function baseUrl() {
return (process.env.APP_BASE_URL || 'http://localhost:5173').replace(/\/+$/, '')
}
// The public link that goes in the announcement. News has no per-post route
// (App.jsx only has the /site/news list), so we link the list — matches the
// pre-pipeline Discord announce behavior.
function articleUrl(baseUrl) {
return `${String(baseUrl || '').replace(/\/+$/, '')}/site/news`
}
// Build the town-crier lines: title, a one-line excerpt, then the URL. Each line
// is clamped to the sidecar's per-line cap and the whole thing to the line-count
// cap. Falls back to a stripped body excerpt when the post has no excerpt.
function buildTownCrierText(post, { baseUrl } = {}) {
const title = clamp(post.title, MAX_LINE_LEN)
const excerptSource = post.excerpt || deriveExcerpt(post.body, MAX_LINE_LEN) || ''
const lines = [title]
const excerpt = clamp(excerptSource, MAX_LINE_LEN)
if (excerpt) lines.push(excerpt)
const url = clamp(articleUrl(baseUrl), MAX_LINE_LEN)
if (url) lines.push(url)
return lines.filter(Boolean).slice(0, MAX_LINES)
}
// ── Result classification ──────────────────────────────────────────────────
// Both clients return { ok, status, error }. Map that to one of:
// done — delivered, mark the leg done
// retry — transient (shard restarting, bot down, network); back off + retry
// terminal — will never succeed as-is (over caps, bad auth/config); fail now
function classifyTownCrier(result) {
if (result && result.ok) return { outcome: 'done' }
const status = result ? result.status : 0
// 400 = over the line/duration caps (a data problem — do NOT retry).
// 401 = token mismatch, 409 = protocol mismatch (both config problems).
if (status === 400 || status === 401 || status === 409) {
return { outcome: 'terminal', error: legError(result) }
}
// 503 (shard not connected), 504 (shard timeout), 0 (network/timeout / not
// configured yet), and any other 5xx are transient — retry.
return { outcome: 'retry', error: legError(result) }
}
function classifyDiscord(result) {
if (result && result.ok) return { outcome: 'done' }
// The bot's /internal/announce collapses failures (503 = not connected,
// 400 = no news channel configured) without surfacing Discord's own
// retry_after, so there is no reliable terminal signal to key on here. Retry
// every failure on the shared backoff; a genuine config problem simply
// exhausts its attempts and lands as `failed` in the admin panel, where the
// per-leg retry button re-runs it after the channel is set.
return { outcome: 'retry', error: legError(result) }
function articleUrl(base) {
return `${String(base || '').replace(/\/+$/, '')}/site/news`
}
// Every leg's client returns { ok, status, data, error }. Squeeze a failure into
// the one line stored in announce_job_legs.last_error and shown in the panel.
function legError(result) {
if (!result) return 'no response'
if (result.status) {
@@ -99,29 +50,32 @@ function scheduleAfter(attempts) {
return BACKOFF_MS[Math.min(attempts - 1, BACKOFF_MS.length - 1)]
}
// Parent job status derived from the two leg statuses:
// done — both legs delivered
// failed — both legs gave up
// partial — at least one leg reached a terminal state while the other has not
// matched it (still pending/retrying, or the opposite terminal state)
// pending — neither leg is terminal yet
function rollupStatus(towncrierStatus, discordStatus) {
if (towncrierStatus === 'done' && discordStatus === 'done') return 'done'
if (towncrierStatus === 'failed' && discordStatus === 'failed') return 'failed'
// Parent job status derived from its leg statuses:
// done — every leg delivered
// failed — every leg gave up
// partial — at least one leg reached a terminal state without all of them
// agreeing (some still pending/retrying, or a mix of done and failed)
// pending — no leg is terminal yet
//
// Takes the list of leg statuses rather than two named arguments, because the leg
// set is registered rather than fixed (MODULE_SYSTEM.md §1.8). No legs at all
// rolls up `done`: with nothing registered there is nothing left to deliver, and
// leaving such jobs `pending` would pile up rows the worker never touches.
function rollupStatus(statuses) {
const list = Array.isArray(statuses) ? statuses : []
const terminal = (s) => s === 'done' || s === 'failed'
if (terminal(towncrierStatus) || terminal(discordStatus)) return 'partial'
if (list.every((s) => s === 'done')) return 'done'
if (list.every((s) => s === 'failed')) return 'failed'
if (list.some(terminal)) return 'partial'
return 'pending'
}
module.exports = {
MAX_LINES,
MAX_LINE_LEN,
MAX_ATTEMPTS,
BACKOFF_MS,
buildTownCrierText,
baseUrl,
articleUrl,
classifyTownCrier,
classifyDiscord,
legError,
scheduleAfter,
rollupStatus,
}

55
server/src/model/announceJobs/announceJobs.model.js
View File

@@ -2,19 +2,21 @@
//
// Sits between the DB rows and the worker: creates jobs on publish, records each
// leg's outcome, keeps the parent `status` rollup in sync, stamps the post's
// announced_at when both legs land, and resets a leg for the admin retry button.
// The pure decisions (backoff, rollup, classification) live in .logic.js.
// announced_at when every leg lands, and resets a leg for the admin retry button.
// The pure decisions (backoff, rollup) live in .logic.js; which legs exist at all
// is modules/registries.js's answer, not this file's (MODULE_SYSTEM.md §1.8).
const db = require('./announceJobs.db')
const logic = require('./announceJobs.logic')
const registries = require('../../modules/registries')
const posts = require('../posts/posts.model')
const log = require('../../utils/logger')('announce')
// Enqueue an announcement for a freshly-published news post: one job row (both
// legs pending, due immediately) plus a back-pointer on the post so the admin
// panel can find it. Returns the new job id.
// Enqueue an announcement for a freshly-published news post: one job row, one leg
// row per registered leg (all pending, due immediately), plus a back-pointer on
// the post so the admin panel can find it. Returns the new job id.
async function enqueue(postId) {
const jobId = await db.create(postId)
const jobId = await db.create(postId, registries.announceLegIds())
await posts.linkAnnounceJob(postId, jobId)
log.info('announce job enqueued', { jobId, postId })
return jobId
@@ -43,12 +45,13 @@ async function enqueueIfNeeded(post, transition) {
}
}
// Record a leg's dispatch outcome and refresh the rollup. `outcome` is one of
// logic.classify*'s results: 'done' | 'retry' | 'terminal'. For 'retry' we bump
// the attempt count and schedule the next run (or fail the leg once the cap is
// hit). Returns the updated job row.
// Record a leg's dispatch outcome and refresh the rollup. `outcome` is one of a
// leg's classify() results: 'done' | 'retry' | 'terminal'. For 'retry' we bump the
// attempt count and schedule the next run (or fail the leg once the cap is hit).
// Returns the updated job row.
async function recordOutcome(job, leg, { outcome, error }) {
const attempts = Number(job[`${leg}_attempts`]) || 0
const row = (job.legs || []).find((l) => l.leg === leg)
const attempts = Number(row && row.attempts) || 0
if (outcome === 'done') {
await db.updateLeg(job.id, leg, { status: 'done', attempts, lastError: null, nextAttemptAt: null })
@@ -71,12 +74,12 @@ async function recordOutcome(job, leg, { outcome, error }) {
return refreshStatus(job.id)
}
// Recompute and persist the parent status from the two legs; stamp the post's
// announced_at the moment both legs have delivered.
// Recompute and persist the parent status from the legs; stamp the post's
// announced_at the moment every leg has delivered.
async function refreshStatus(jobId) {
const job = await db.findById(jobId)
if (!job) return null
const status = logic.rollupStatus(job.towncrier_status, job.discord_status)
const status = logic.rollupStatus(job.legs.map((l) => l.status))
if (status !== job.status) await db.setStatus(jobId, status)
job.status = status
if (status === 'done') {
@@ -93,16 +96,33 @@ async function refreshStatus(jobId) {
// the worker pick it up on the next tick. Resets the attempt count so a retry
// after a config fix gets a full budget again.
async function resetLeg(postId, leg) {
if (!db.LEGS.includes(leg)) throw new Error(`unknown announce leg: ${leg}`)
if (!registries.announceLeg(leg)) throw new Error(`unknown announce leg: ${leg}`)
const job = await db.findByPostId(postId)
if (!job) return null
// A job enqueued before this leg was registered has no row for it; create it so
// the retry button works on an existing post after a module is installed.
await db.ensureLegs(job.id, [leg])
await db.updateLeg(job.id, leg, { status: 'pending', attempts: 0, lastError: null, nextAttemptAt: null })
log.info('announce leg reset for retry', { jobId: job.id, postId, leg })
return refreshStatus(job.id)
// Labelled, because this is the response body the admin panel re-renders from.
return withLabels(await refreshStatus(job.id))
}
// Decorate a job's legs with the label their registration carries, so the admin
// panel renders a module's leg with a real name and no client change
// (MODULE_SYSTEM.md §1.8). An unregistered leg — a stale row from a module that
// was since removed — keeps its id as the label rather than disappearing.
function withLabels(job) {
if (!job) return job
job.legs = (job.legs || []).map((l) => {
const registered = registries.announceLeg(l.leg)
return { ...l, label: registered ? registered.label : l.leg }
})
return job
}
async function getByPostId(postId) {
return db.findByPostId(postId)
return withLabels(await db.findByPostId(postId))
}
module.exports = {
@@ -113,4 +133,5 @@ module.exports = {
refreshStatus,
resetLeg,
getByPostId,
withLabels,
}

59
server/src/model/modules/modules.db.js Normal file
View File

@@ -0,0 +1,59 @@
const { query } = require('../../utils/db')
// SQL for installed_modules — the module system's record of what is installed and
// what happened to it on the last boot (db/schema.sql, docs/website/MODULE_SYSTEM.md
// §2.4). Rows are keyed by module id. All state rules live in modules.model.js;
// this file only moves rows.
const COLS = `id, name, version, state, failure_stage, failure_reason,
source, sha256, installed_at, started_at, updated_at`
const listAll = () => query(`SELECT ${COLS} FROM installed_modules ORDER BY id`)
const getOne = (id) => query(`SELECT ${COLS} FROM installed_modules WHERE id = ?`, [id])
// Write (or refresh) the row for an installed module. A re-install or an upgrade
// updates the metadata and deliberately leaves `state` alone: upgrading an enabled
// module must not silently disable it, and re-installing a disabled one must not
// silently switch it back on. A brand-new row lands in `installed`, the transient
// state the next restart resolves.
const upsert = ({ id, name, version, source, sha256 }) =>
query(
`INSERT INTO installed_modules (id, name, version, source, sha256, state)
VALUES (?, ?, ?, ?, ?, 'installed')
ON DUPLICATE KEY UPDATE
name = VALUES(name),
version = VALUES(version),
source = VALUES(source),
sha256 = VALUES(sha256)`,
[id, name, version, source ?? null, sha256 ?? null],
)
// Move one row to a new state. `failureStage`/`failureReason` are written on every
// call — a non-failing transition passes nulls, which is what clears a stale reason
// off a module that has since come up. `stampStarted` sets started_at to now.
const setState = ({ id, state, failureStage = null, failureReason = null, stampStarted = false }) =>
query(
`UPDATE installed_modules
SET state = ?, failure_stage = ?, failure_reason = ?
${stampStarted ? ', started_at = CURRENT_TIMESTAMP' : ''}
WHERE id = ?`,
[state, failureStage, failureReason, id],
)
// Boot reset: every row the operator has not disabled goes back to `enabled` with
// no failure recorded, so the load that follows writes this boot's outcome rather
// than leaving the last one on display. `disabled` is untouched — it is a decision,
// not an outcome.
const resetForBoot = () =>
query(
`UPDATE installed_modules
SET state = 'enabled', failure_stage = NULL, failure_reason = NULL
WHERE state <> 'disabled'`,
)
// Drop the row entirely. Only the explicit purge does this (§2.5); a plain
// uninstall disables the module and keeps its row and its data.
const remove = (id) => query('DELETE FROM installed_modules WHERE id = ?', [id])
module.exports = { listAll, getOne, upsert, setState, resetForBoot, remove }

183
server/src/model/modules/modules.model.js Normal file
View File

@@ -0,0 +1,183 @@
// The module state machine (docs/website/MODULE_SYSTEM.md §2.4, MODULE_API.md §4.4).
//
// installed ──► enabled ──► started
// │ │
// │ └──► startup_failed ──┐
// │ │ (retry)
// └──────────────► disabled ◄─────────┘
//
// One row per installed module, one column holding the state. The rules that make
// the machine mean anything live here, not in the SQL:
//
// - `installed` is transient. An install writes the row; the restart that follows
// resolves it to `started` or `startup_failed` (§2.5).
// - `disabled` is the only state a boot leaves alone. It is the operator's
// decision; every other state is an outcome and is recomputed each boot by
// beginBoot(). That is what makes a fixed module recover on restart without
// anyone visiting the admin panel.
// - A failure is recorded with the stage it happened at, and every non-failing
// transition clears it — a running module can never show a stale reason.
//
// What this table does NOT decide is which routes exist. The loader scans the
// filesystem at require time, before the database is reachable (MODULE_API.md §4.1),
// so a disabled module is still mounted and simply guarded (§4.5). Keeping the URL
// surface a property of the volume is what lets routes.manifest.json be generated
// off a dead database.
const db = require('./modules.db')
const STATES = ['installed', 'enabled', 'disabled', 'started', 'startup_failed']
// The stage a failure happened at: MODULE_API.md §4.3's seven validation steps,
// plus `boot` for an onBoot hook that threw (§2.5).
const FAILURE_STAGES = [
'manifest',
'core_api',
'mounts',
'extensions',
'schema',
'require',
'register',
'boot',
]
class ModuleStateError extends Error {
constructor(code, message) {
super(message)
this.name = 'ModuleStateError'
this.code = code
}
}
// Legal moves, keyed by target state. Anything not listed is a bug in the caller
// and throws rather than writing a row that misrepresents what happened.
const ALLOWED_FROM = {
// Enabling is the recovery path as well as the first step: a disabled module the
// operator switches back on, and a startup_failed one they retry, both land here.
enabled: ['installed', 'enabled', 'disabled', 'startup_failed', 'started'],
// The operator may disable a module in any state, including one that is running.
disabled: STATES,
// Reached from `enabled` on a normal boot, and from `installed` on the first boot
// after an install (or for a directory placed on the volume by hand, whose row is
// written moments earlier in the same boot).
started: ['installed', 'enabled'],
// Failure always precedes `started` in the lifecycle; `started` is accepted so a
// late failure can still be recorded truthfully rather than dropped.
startup_failed: ['installed', 'enabled', 'started'],
}
// row → API shape.
function serialize(row) {
if (!row) return null
return {
id: row.id,
name: row.name,
version: row.version,
state: row.state,
failureStage: row.failure_stage ?? null,
failureReason: row.failure_reason ?? null,
source: row.source ?? null,
sha256: row.sha256 ?? null,
installedAt: row.installed_at ?? null,
startedAt: row.started_at ?? null,
updatedAt: row.updated_at ?? null,
}
}
async function list() {
const rows = await db.listAll()
return rows.map(serialize)
}
async function get(id) {
const rows = await db.getOne(id)
return serialize(rows[0])
}
// Record an install (or a re-install / upgrade). Metadata is refreshed; the state is
// left as it is, so upgrading an enabled module does not switch it off and
// re-installing a disabled one does not switch it on. A new row lands in `installed`.
async function recordInstalled({ id, name, version, source = null, sha256 = null }) {
if (!id || !name || !version) {
throw new ModuleStateError('invalid_module', 'id, name and version are required')
}
await db.upsert({ id, name, version, source, sha256 })
return get(id)
}
// Start of boot: clear the last boot's outcomes so what is on display after this
// boot is what this boot did. Leaves `disabled` rows alone (see the header).
// Returns the number of rows reset.
async function beginBoot() {
const res = await db.resetForBoot()
return res?.affectedRows ?? 0
}
// Apply one transition, after checking it is legal for the row's current state.
// A row that does not exist is not an error the caller can act on — a module can be
// present on the volume with no row at all — so it returns null and writes nothing.
async function transition(id, target, { failureStage = null, failureReason = null } = {}) {
const current = await get(id)
if (!current) return null
const allowed = ALLOWED_FROM[target]
if (!allowed.includes(current.state)) {
throw new ModuleStateError(
'illegal_transition',
`module '${id}': cannot move from '${current.state}' to '${target}'`,
)
}
await db.setState({
id,
state: target,
failureStage,
failureReason,
stampStarted: target === 'started',
})
return get(id)
}
const enable = (id) => transition(id, 'enabled')
const disable = (id) => transition(id, 'disabled')
const markStarted = (id) => transition(id, 'started')
// Record a failure at a named stage. Two deliberate softenings, both because this is
// called from the boot path where throwing would turn one module's failure into
// everybody's (MODULE_API.md §4.4 — the failing module fails alone):
//
// - a `disabled` row is a no-op. The operator switched it off; a broken module
// they already disabled is not news, and overwriting their decision with an
// outcome would silently re-enable it on the next boot.
// - an unrecognised stage is recorded as `require` rather than rejected, so a
// miscategorised failure still reaches the admin panel with its reason intact.
async function markStartupFailed(id, { stage, reason }) {
const current = await get(id)
if (!current || current.state === 'disabled') return current
return transition(id, 'startup_failed', {
failureStage: FAILURE_STAGES.includes(stage) ? stage : 'require',
failureReason: String(reason ?? 'unknown error').slice(0, 4000),
})
}
// Purge only (§2.5). A plain uninstall disables the module and keeps its row, so its
// data survives and the admin panel can still show what was there.
async function remove(id) {
await db.remove(id)
}
module.exports = {
STATES,
FAILURE_STAGES,
ModuleStateError,
list,
get,
recordInstalled,
beginBoot,
enable,
disable,
markStarted,
markStartupFailed,
remove,
}

6
server/src/model/notificationSubs/notificationSubs.model.js
View File

@@ -1,8 +1,10 @@
// Per-user push-notification subscriptions (which streams a user opted into;
// applied to every device they register). The catalog is config/notificationStreams.
// applied to every device they register). The catalog is core's plus every
// installed module's, so it is read back through modules/registries rather than
// from a config file (MODULE_SYSTEM.md §1.8).
const db = require('./notificationSubs.db')
const { isValidStream } = require('../../config/notificationStreams')
const { isValidStream } = require('../../modules/registries')
const getForUser = async (userId) => (await db.listByUser(userId)).map((r) => r.stream_id)

390
server/src/model/shardAtlas/shardAtlas.db.js
View File

@@ -1,390 +0,0 @@
const { pool, query } = require('../../utils/db')
// Raw SQL for the spawn atlas. Every table here is IMPORT-OWNED: `replaceAtlas`
// empties and refills all six inside one transaction, and nothing else in the
// codebase writes to them. There are no foreign keys, consistent with every
// other shard_* table.
const BATCH = 500
const ATLAS_TABLES = [
'shard_spawn_point_types',
'shard_spawn_points',
'shard_spawn_creatures',
'shard_regions',
'shard_landmarks',
'shard_champion_spawns',
]
async function insertBatched(conn, sql, rows) {
for (let i = 0; i < rows.length; i += BATCH) {
await conn.batch(sql, rows.slice(i, i + BATCH))
}
return rows.length
}
/**
* Replace the entire atlas in one transaction.
*
* All-or-nothing on purpose: a failed reload must leave the previous atlas
* intact rather than a half-loaded world, since a partially-imported atlas is
* indistinguishable from a real one to anyone reading it.
*
* `DELETE`, not `TRUNCATE` — `TRUNCATE` is DDL in MariaDB and implicitly
* commits, which would defeat exactly that guarantee. At ~7k rows the cost of
* `DELETE` is irrelevant.
*/
async function replaceAtlas(atlas, art = {}) {
const conn = await pool.getConnection()
const counts = {}
try {
await conn.beginTransaction()
for (const table of ATLAS_TABLES) await conn.query(`DELETE FROM ${table}`)
counts.creatures = await insertBatched(
conn,
'INSERT INTO shard_spawn_creatures (slug, name, total, points, facets, art) VALUES (?,?,?,?,?,?)',
atlas.creatures.map((c) => [
c.slug,
c.name,
c.total ?? 0,
c.points ?? 0,
JSON.stringify(c.facets ?? {}),
art[c.slug] ?? null,
]),
)
counts.regions = await insertBatched(
conn,
'INSERT INTO shard_regions (facet, name, type, priority, parent, rects) VALUES (?,?,?,?,?,?)',
atlas.regions.map((r) => [
r.facet,
r.name,
r.type || null,
r.priority ?? 0,
r.parent || null,
JSON.stringify(r.rects ?? []),
]),
)
counts.landmarks = await insertBatched(
conn,
'INSERT INTO shard_landmarks (facet, name, grp, x, y, z) VALUES (?,?,?,?,?,?)',
atlas.landmarks.map((l) => [
l.facet,
l.name,
l.group || null,
l.x ?? 0,
l.y ?? 0,
l.z ?? 0,
]),
)
counts.champions = await insertBatched(
conn,
'INSERT INTO shard_champion_spawns ' +
'(slug, name, grp, type, random_type, facet, x, y, z, radius, label) ' +
'VALUES (?,?,?,?,?,?,?,?,?,?,?)',
atlas.champions.map((c) => [
c.slug,
c.name,
c.group || null,
c.type || null,
c.randomType ? 1 : 0,
c.facet,
c.x ?? 0,
c.y ?? 0,
c.z ?? 0,
c.radius ?? 0,
c.label || null,
]),
)
// Point ids are assigned explicitly rather than left to AUTO_INCREMENT: the
// join rows need to know them and `conn.batch()` reports no usable insertId
// for a multi-row insert. Safe because this transaction just emptied the
// table and nothing else writes to it.
counts.points = await insertBatched(
conn,
'INSERT INTO shard_spawn_points ' +
'(id, facet, name, x, y, width, height, spawn_range, max_count, min_delay, max_delay, ' +
'tod_start, tod_end, tod_mode, region, landmark, label) ' +
'VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)',
atlas.points.map((p, i) => [
i + 1,
p.facet,
p.name,
p.x,
p.y,
p.width ?? 0,
p.height ?? 0,
p.range ?? 0,
p.maxCount ?? 0,
p.minDelay ?? 0,
p.maxDelay ?? 0,
p.todStart ?? 0,
p.todEnd ?? 0,
p.todMode ?? 0,
p.region,
p.landmark,
p.label || 'Wilderness',
]),
)
counts.pointTypes = await insertBatched(
conn,
'INSERT INTO shard_spawn_point_types (point_id, slug, max_count) VALUES (?,?,?)',
atlas.pointTypes,
)
await conn.query(
'INSERT INTO shard_atlas_meta (id, payload) VALUES (1, ?) ' +
'ON DUPLICATE KEY UPDATE payload = VALUES(payload), imported_at = CURRENT_TIMESTAMP',
[JSON.stringify({ ...atlas.meta, importedCounts: counts })],
)
// A completed import answers whatever was pending.
await conn.query('DELETE FROM shard_atlas_pending')
await conn.commit()
return counts
} catch (err) {
await conn.rollback().catch(() => {})
throw err
} finally {
conn.release()
}
}
async function getMeta() {
const rows = await query('SELECT payload, imported_at FROM shard_atlas_meta WHERE id = 1')
if (rows.length === 0) return null
const payload = typeof rows[0].payload === 'string' ? JSON.parse(rows[0].payload) : rows[0].payload
return { ...payload, importedAt: rows[0].imported_at }
}
/** Facet names currently loaded, used to detect a facet disappearing. */
async function getFacets() {
const rows = await query('SELECT DISTINCT facet FROM shard_spawn_points ORDER BY facet')
return rows.map((row) => row.facet)
}
// ── Pending review ─────────────────────────────────────────────────────────
async function getPending() {
const rows = await query('SELECT payload, status, detected_at FROM shard_atlas_pending WHERE id = 1')
if (rows.length === 0) return null
const payload = typeof rows[0].payload === 'string' ? JSON.parse(rows[0].payload) : rows[0].payload
return { ...payload, status: rows[0].status, detectedAt: rows[0].detected_at }
}
async function setPending(payload, status = 'pending') {
return query(
'INSERT INTO shard_atlas_pending (id, status, payload) VALUES (1, ?, ?) ' +
'ON DUPLICATE KEY UPDATE status = VALUES(status), payload = VALUES(payload), ' +
'detected_at = CURRENT_TIMESTAMP',
[status, JSON.stringify(payload)],
)
}
async function clearPending() {
return query('DELETE FROM shard_atlas_pending')
}
// ── Reads (the public /atlas surface) ──────────────────────────────────────
//
// Every read here is a plain indexed query over ~7k rows and is served entirely
// from MariaDB: the atlas is static shard content, so nothing on this path
// touches the sidecar and nothing degrades when the shard is down.
//
// A facet filter is expressed as EXISTS over the points, never as a JSON path
// built from caller input. `shard_spawn_creatures.facets` is a JSON object keyed
// by facet name, and matching a key means either concatenating the name into a
// path or handing it to JSON_SEARCH — whose search string treats `%` and `_` as
// wildcards, so `?facet=%` would quietly match everything. The join is exact and
// uses the indexes that already exist.
const CREATURE_FACET_EXISTS = `EXISTS (
SELECT 1 FROM shard_spawn_point_types t
JOIN shard_spawn_points p ON p.id = t.point_id
WHERE t.slug = c.slug AND p.facet = ?
)`
// Build the WHERE for a creature search. `q` is a substring match on the display
// name — a LIKE scan, which is free at ~800 rows and, unlike FULLTEXT, has no
// minimum token length to break a search for "orc".
function creatureWhere({ q, facet }) {
const where = []
const params = []
if (q) {
where.push('c.name LIKE ?')
params.push(`%${q}%`)
}
if (facet) {
where.push(CREATURE_FACET_EXISTS)
params.push(facet)
}
return { sql: where.length ? `WHERE ${where.join(' AND ')}` : '', params }
}
async function countCreatures({ q = '', facet = '' } = {}) {
const { sql, params } = creatureWhere({ q, facet })
const rows = await query(`SELECT COUNT(*) AS n FROM shard_spawn_creatures c ${sql}`, params)
return rows[0] ? Number(rows[0].n) : 0
}
function listCreatures({ q = '', facet = '', limit = 50, offset = 0 } = {}) {
const { sql, params } = creatureWhere({ q, facet })
return query(
`SELECT c.slug, c.name, c.total, c.points, c.facets, c.art
FROM shard_spawn_creatures c
${sql}
ORDER BY c.total DESC, c.name ASC
LIMIT ? OFFSET ?`,
[...params, limit, offset],
)
}
async function getCreature(slug) {
const rows = await query(
'SELECT slug, name, total, points, facets, art FROM shard_spawn_creatures WHERE slug = ?',
[slug],
)
return rows[0] || null
}
/**
* Where a creature spawns, grouped by resolved place.
*
* This is the answer the atlas exists to give — "lizardman → Shrines,
* Isamu-Jima, Yew" — so it is aggregated in SQL rather than by summing 6,455
* point rows in Node.
*/
function listCreaturePlaces(slug, { facet = '' } = {}) {
const params = [slug]
let facetSql = ''
if (facet) {
facetSql = 'AND p.facet = ?'
params.push(facet)
}
return query(
`SELECT p.facet, p.label, COUNT(*) AS spawners, SUM(t.max_count) AS max_alive
FROM shard_spawn_point_types t
JOIN shard_spawn_points p ON p.id = t.point_id
WHERE t.slug = ? ${facetSql}
GROUP BY p.facet, p.label
ORDER BY spawners DESC, p.facet ASC, p.label ASC`,
params,
)
}
/** The individual spawners for a creature, newest-largest first. Bounded. */
function listCreaturePoints(slug, { facet = '', limit = 200 } = {}) {
const params = [slug]
let facetSql = ''
if (facet) {
facetSql = 'AND p.facet = ?'
params.push(facet)
}
params.push(limit)
return query(
`SELECT p.id, p.facet, p.name, p.x, p.y, p.width, p.height, p.spawn_range,
p.min_delay, p.max_delay, p.tod_start, p.tod_end, p.tod_mode,
p.region, p.landmark, p.label, t.max_count
FROM shard_spawn_point_types t
JOIN shard_spawn_points p ON p.id = t.point_id
WHERE t.slug = ? ${facetSql}
ORDER BY t.max_count DESC, p.facet ASC, p.label ASC, p.id ASC
LIMIT ?`,
params,
)
}
/** Every other creature sharing a spawner with this one. */
function listCreatureCompanions(slug, { limit = 24 } = {}) {
return query(
`SELECT o.slug, c.name, COUNT(*) AS shared
FROM shard_spawn_point_types t
JOIN shard_spawn_point_types o ON o.point_id = t.point_id AND o.slug <> t.slug
JOIN shard_spawn_creatures c ON c.slug = o.slug
WHERE t.slug = ?
GROUP BY o.slug, c.name
ORDER BY shared DESC, c.name ASC
LIMIT ?`,
[slug, limit],
)
}
function listRegions({ facet = '', q = '' } = {}) {
const where = []
const params = []
if (facet) {
where.push('facet = ?')
params.push(facet)
}
if (q) {
where.push('name LIKE ?')
params.push(`%${q}%`)
}
return query(
`SELECT facet, name, type, priority, parent, rects
FROM shard_regions
${where.length ? `WHERE ${where.join(' AND ')}` : ''}
ORDER BY facet ASC, name ASC`,
params,
)
}
function listLandmarks({ facet = '', q = '' } = {}) {
const where = []
const params = []
if (facet) {
where.push('facet = ?')
params.push(facet)
}
if (q) {
where.push('(name LIKE ? OR grp LIKE ?)')
params.push(`%${q}%`, `%${q}%`)
}
return query(
`SELECT facet, name, grp, x, y, z
FROM shard_landmarks
${where.length ? `WHERE ${where.join(' AND ')}` : ''}
ORDER BY facet ASC, grp ASC, name ASC`,
params,
)
}
function listChampions({ facet = '' } = {}) {
const params = []
let where = ''
if (facet) {
where = 'WHERE facet = ?'
params.push(facet)
}
return query(
`SELECT slug, name, grp, type, random_type, facet, x, y, z, radius, label
FROM shard_champion_spawns
${where}
ORDER BY facet ASC, name ASC`,
params,
)
}
module.exports = {
replaceAtlas,
getMeta,
getFacets,
getPending,
setPending,
clearPending,
countCreatures,
listCreatures,
getCreature,
listCreaturePlaces,
listCreaturePoints,
listCreatureCompanions,
listRegions,
listLandmarks,
listChampions,
}

485
server/src/model/shardAtlas/shardAtlas.model.js
View File

@@ -1,485 +0,0 @@
const fs = require('fs')
const path = require('path')
const db = require('./shardAtlas.db')
const settings = require('../settings/settings.model')
const { slugify } = require('../../utils/spawnAtlasParse')
const {
AtlasSourceError,
PARSER_VERSION,
buildAtlas,
hashSources,
sameSources,
} = require('../../utils/spawnAtlasSource')
const log = require('../../utils/logger')('shardAtlas')
// The spawn atlas, refreshed from the shard's own ServUO tree.
//
// The tree is the single source of truth. Nothing is precomputed and committed,
// because a shard's maps change over its lifetime — facets get added, replaced
// or renamed — and a snapshot in the repo would go stale against the world
// players actually see. So the atlas is re-derived on every boot.
//
// Two rules govern the boot path:
//
// 1. **It never blocks startup.** No configured path, an unreadable path, a
// malformed file, a database error — all of it is caught and logged. The
// site comes up either way, serving whatever atlas it already had.
// 2. **A facet disappearing is not applied automatically.** Losing a facet is
// the signature of a half-copied or mid-update tree as much as of a real
// map change, and the two are indistinguishable from here. The refresh is
// staged for a human instead, and an admin approves or rejects it.
//
// Everything else — new facets, renamed regions, changed spawns — applies
// straight away, because none of it can silently destroy data an operator would
// miss.
const SETTING_KEY = 'spawn_atlas_servuo_path'
/**
* Where the ServUO tree lives.
*
* The admin setting wins over the environment so an operator can point the
* atlas at a different tree without a redeploy, matching how the rest of the
* shard integration is admin-managed rather than env-configured. `SERVUO_PATH`
* remains as the deploy-time default, since the path usually describes a mount
* that the deployment sets up.
*/
async function getServuoPath() {
try {
const configured = await settings.get(SETTING_KEY)
if (configured && String(configured).trim() !== '') return String(configured).trim()
} catch {
// Settings unavailable is not fatal — fall through to the env default.
}
const fromEnv = process.env.SERVUO_PATH
return fromEnv && fromEnv.trim() !== '' ? fromEnv.trim() : ''
}
async function setServuoPath(value, updatedBy = null) {
return settings.set(SETTING_KEY, String(value ?? '').trim(), updatedBy)
}
/**
* Optional operator-supplied art map, `{ "<slug>": "<file under uploads/atlas/>" }`.
*
* Never committed and never shipped — creature sprites come out of the
* operator's own client `.mul`/`.uop` files, which are theirs, not ours to
* redistribute. Absent (the normal case) every `art` stays NULL and the UI
* renders text-only.
*/
function loadArtMap(dir = path.join(__dirname, '..', '..', '..', 'db', 'data')) {
try {
const file = path.join(dir, 'spawnAtlas.art.json')
if (!fs.existsSync(file)) return {}
const map = JSON.parse(fs.readFileSync(file, 'utf8'))
return map && typeof map === 'object' ? map : {}
} catch (err) {
log.warn('spawn atlas art map could not be read', { error: err.message })
return {}
}
}
/**
* Flatten each point's types into `shard_spawn_point_types` rows.
*
* A spawner may legitimately list the same type twice, and the primary key is
* (point_id, slug), so duplicates collapse to the larger max rather than
* failing the insert.
*/
function pointTypeRows(points) {
const rows = []
points.forEach((point, i) => {
const bySlug = new Map()
for (const entry of point.types ?? []) {
const slug = slugify(entry.type)
if (slug === '') continue
bySlug.set(slug, Math.max(bySlug.get(slug) ?? 0, entry.max ?? 1))
}
for (const [slug, max] of bySlug) rows.push([i + 1, slug, max])
})
return rows
}
async function applyAtlas(atlas) {
return db.replaceAtlas({ ...atlas, pointTypes: pointTypeRows(atlas.points) }, loadArtMap())
}
/**
* Refresh the atlas from the configured ServUO tree.
*
* Returns a result describing what happened rather than throwing, so the caller
* — including the boot path — can log it and move on:
*
* `skipped` no path configured
* `unavailable` path configured but unreadable / missing required files
* `unchanged` source hashes match the loaded atlas; nothing parsed
* `imported` parsed and applied
* `needsReview` parsed, but a facet would be lost; staged for an admin
* `failed` parsed or applied and something went wrong
*
* `force` skips the hash check (an admin asking for a reimport) and `approve`
* additionally accepts facet loss (an admin approving a staged refresh).
*/
/**
* Was the loaded atlas built by THIS parser?
*
* An atlas imported before `parserVersion` existed reports undefined, which is
* correctly "no" — those are exactly the ones carrying the old readings.
*/
const currentParser = (meta) => meta?.parserVersion === PARSER_VERSION
async function refresh({ force = false, approve = false, path: pathOverride = '' } = {}) {
// An explicit override wins outright — it is a one-off "use this tree", and it
// must not be silently overruled by the configured path the way an env default
// would be.
const root = pathOverride.trim() !== '' ? pathOverride.trim() : await getServuoPath()
if (root === '') return { status: 'skipped', reason: 'no ServUO path configured' }
let hashes
try {
hashes = hashSources(root)
} catch (err) {
if (err instanceof AtlasSourceError) {
return { status: 'unavailable', reason: err.message, code: err.code, path: root }
}
return { status: 'failed', reason: err.message, path: root }
}
const meta = await db.getMeta().catch(() => null)
const loaded = meta?.source
? Object.fromEntries(Object.entries(meta.source).map(([label, v]) => [label, v.sha256]))
: null
// Two things make a loaded atlas stale: the tree changed, or the PARSER did.
// Only checking the tree would strand an install whose maps never change on
// whatever an older build derived — a corrected parse would ship and never
// reach the data.
if (!force && sameSources(hashes, loaded) && currentParser(meta)) {
return { status: 'unchanged', path: root }
}
// A rejected refresh must not re-prompt on every boot. It stays rejected until
// the tree changes again, at which point the hashes differ and it is a new
// decision.
const pending = await db.getPending().catch(() => null)
if (!approve && !force && pending?.status === 'rejected' && sameSources(hashes, pending.hashes)) {
return { status: 'unchanged', path: root, reason: 'refresh previously rejected' }
}
let atlas
try {
atlas = buildAtlas(root)
} catch (err) {
return { status: 'failed', reason: err.message, path: root }
}
const currentFacets = await db.getFacets().catch(() => [])
const incomingFacets = atlas.facets
const removedFacets = currentFacets.filter((facet) => !incomingFacets.includes(facet))
const addedFacets = incomingFacets.filter((facet) => !currentFacets.includes(facet))
// Losing a facet is indistinguishable here from a half-copied tree, so it is
// staged rather than applied — but startup is never blocked by it.
if (removedFacets.length > 0 && !approve) {
const summary = {
hashes,
path: root,
currentFacets,
incomingFacets,
removedFacets,
addedFacets,
counts: atlas.meta.counts,
}
await db.setPending(summary, 'pending').catch((err) => {
log.warn('could not stage spawn atlas refresh', { error: err.message })
})
return { status: 'needsReview', ...summary }
}
try {
const counts = await applyAtlas(atlas)
return { status: 'imported', path: root, counts, addedFacets, removedFacets }
} catch (err) {
return { status: 'failed', reason: err.message, path: root }
}
}
/** Admin approved a staged refresh: apply it, facet loss and all. */
async function approvePending(options = {}) {
return refresh({ ...options, approve: true, force: true })
}
/**
* Admin rejected a staged refresh: keep the current atlas and remember the
* decision against those exact source hashes, so it does not re-prompt every
* boot. A further change to the tree produces different hashes and asks again.
*/
async function rejectPending() {
const pending = await db.getPending()
if (!pending) return { status: 'none' }
await db.setPending({ ...pending, rejectedAt: new Date().toISOString() }, 'rejected')
return { status: 'rejected' }
}
/** Everything the admin panel needs to describe atlas state. */
async function status({ path: pathOverride = '' } = {}) {
const root = pathOverride.trim() !== '' ? pathOverride.trim() : await getServuoPath()
const [meta, pending, facets] = await Promise.all([
db.getMeta().catch(() => null),
db.getPending().catch(() => null),
db.getFacets().catch(() => []),
])
let treeReadable = false
let drift = null
if (root !== '') {
try {
const hashes = hashSources(root)
treeReadable = true
const loaded = meta?.source
? Object.fromEntries(Object.entries(meta.source).map(([l, v]) => [l, v.sha256]))
: null
// Same question `refresh` asks: an import picks something up when either
// the tree or the parser has moved on.
drift = !sameSources(hashes, loaded) || !currentParser(meta)
} catch {
treeReadable = false
}
}
return {
configured: root !== '',
path: root,
treeReadable,
drift,
facets,
importedAt: meta?.importedAt ?? null,
counts: meta?.counts ?? null,
pending,
}
}
/**
* Boot hook. Best-effort by contract: it logs and returns, never throws, so a
* missing tree or a bad file can never stop the site coming up.
*/
async function refreshOnBoot() {
try {
const result = await refresh()
switch (result.status) {
case 'imported':
log.info('spawn atlas refreshed from ServUO tree', {
...result.counts,
added: result.addedFacets,
})
break
case 'needsReview':
log.warn(
'spawn atlas refresh staged for admin review — a facet would be removed; ' +
'the existing atlas is unchanged',
{ removed: result.removedFacets, added: result.addedFacets },
)
break
case 'unavailable':
log.warn('spawn atlas source unavailable', { reason: result.reason, path: result.path })
break
case 'failed':
log.warn('spawn atlas refresh failed', { reason: result.reason })
break
default:
break
}
return result
} catch (err) {
log.warn('spawn atlas refresh errored', { error: err.message })
return { status: 'failed', reason: err.message }
}
}
// ── Reads ──────────────────────────────────────────────────────────────────
//
// The shapes the /public/atlas endpoints serve. Rows are camelCased here rather
// than in the controller, for the same reason shardState does it: the column
// names are an implementation detail of the import, and the browser contract
// should not move when a column is renamed.
const jsonOr = (value, fallback) => {
if (value == null) return fallback
if (typeof value !== 'string') return value
try {
return JSON.parse(value)
} catch {
return fallback
}
}
const shapeCreature = (row) => ({
slug: row.slug,
name: row.name,
// `total` is the summed MaxCount across every spawner (how many can be alive
// at once); `points` is how many spawners mention it. They answer different
// questions and the UI shows both.
total: row.total,
points: row.points,
facets: jsonOr(row.facets, {}),
art: row.art || null,
})
const shapePlace = (row) => ({
facet: row.facet,
label: row.label,
spawners: Number(row.spawners) || 0,
maxAlive: Number(row.max_alive) || 0,
})
const shapePoint = (row) => ({
id: row.id,
facet: row.facet,
name: row.name || null,
x: row.x,
y: row.y,
width: row.width,
height: row.height,
range: row.spawn_range,
maxCount: row.max_count,
minDelay: row.min_delay,
maxDelay: row.max_delay,
todStart: row.tod_start,
todEnd: row.tod_end,
todMode: row.tod_mode,
region: row.region || null,
landmark: row.landmark || null,
label: row.label,
})
/**
* Paginated creature search. Returns the page plus the unpaginated total, so
* the UI can say "showing 50 of 800" without a second round trip.
*/
async function searchCreatures({ q = '', facet = '', limit = 50, offset = 0 } = {}) {
const [rows, total] = await Promise.all([
db.listCreatures({ q, facet, limit, offset }),
db.countCreatures({ q, facet }),
])
return { total, limit, offset, creatures: rows.map(shapeCreature) }
}
/**
* One creature: its totals, the places it spawns (the aggregate the atlas
* exists for), the individual spawners, and what else shares those spawners.
*
* `null` when the slug is unknown — the controller turns that into a 404.
*/
async function getCreature(slug, { facet = '', points = 200 } = {}) {
const row = await db.getCreature(slug)
if (!row) return null
const [places, pointRows, alsoHere] = await Promise.all([
db.listCreaturePlaces(slug, { facet }),
db.listCreaturePoints(slug, { facet, limit: points }),
db.listCreatureCompanions(slug),
])
return {
...shapeCreature(row),
places: places.map(shapePlace),
// `spawners`, not `points`: shapeCreature already uses `points` for the
// COUNT of spawners, and reusing the key for the list of them would make the
// same field a number on the search route and an array here.
spawners: pointRows.map(shapePoint),
// Bounded by the query, so a creature on hundreds of spawners returns a page
// rather than the world.
spawnersTruncated: pointRows.length >= points,
alsoHere: alsoHere.map((r) => ({
slug: r.slug,
name: r.name,
shared: Number(r.shared) || 0,
})),
}
}
async function listRegions(opts = {}) {
const rows = await db.listRegions(opts)
return rows.map((r) => ({
facet: r.facet,
name: r.name,
type: r.type || null,
priority: r.priority,
parent: r.parent || null,
rects: jsonOr(r.rects, []),
}))
}
async function listLandmarks(opts = {}) {
const rows = await db.listLandmarks(opts)
return rows.map((r) => ({
facet: r.facet,
name: r.name,
group: r.grp || null,
x: r.x,
y: r.y,
z: r.z,
}))
}
async function listChampions(opts = {}) {
const rows = await db.listChampions(opts)
return rows.map((r) => ({
slug: r.slug,
name: r.name,
group: r.grp || null,
// '' on the wire means "randomised at activation"; `randomType` says so
// explicitly rather than making the client infer it from an empty string.
type: r.type || null,
randomType: !!r.random_type,
facet: r.facet,
x: r.x,
y: r.y,
z: r.z,
radius: r.radius,
label: r.label || null,
}))
}
/**
* What is loaded: the facet list, the counts, and when it was imported.
*
* Deliberately does NOT report the source path, the per-file hashes or whether
* a refresh is pending. Those describe the operator's filesystem, and this is a
* public endpoint; the admin status route carries them instead.
*/
async function publicMeta() {
const [meta, facets] = await Promise.all([
db.getMeta().catch(() => null),
db.getFacets().catch(() => []),
])
return {
importedAt: meta?.importedAt ?? null,
generatedAt: meta?.generatedAt ?? null,
// The parse counts, not the row counts: `unresolvedPoints` is what lets the
// page state its own placement accuracy instead of implying it is complete.
counts: meta?.counts ?? null,
facets,
}
}
const listFacets = () => db.getFacets()
module.exports = {
refresh,
refreshOnBoot,
approvePending,
rejectPending,
status,
getServuoPath,
setServuoPath,
pointTypeRows,
loadArtMap,
SETTING_KEY,
searchCreatures,
getCreature,
listRegions,
listLandmarks,
listChampions,
listFacets,
publicMeta,
}

108
server/src/model/shardClilocs/shardClilocs.db.js
View File

@@ -1,108 +0,0 @@
const { pool, query } = require('../../utils/db')
// Raw SQL for the cliloc table. `shard_clilocs` is IMPORT-OWNED: `replaceAll`
// empties and refills it inside one transaction, and nothing else in the
// codebase writes to it. No foreign keys, consistent with every other shard_*
// table.
const BATCH = 1000
/**
* Replace the entire cliloc table in one transaction.
*
* All-or-nothing on purpose: a failed reload must leave the previous table
* intact rather than a half-loaded one, because a partially-imported cliloc
* table is indistinguishable from a complete one to anyone reading it — you
* would just see some items named and some not, which is also what "no table at
* all" looks like.
*
* `DELETE`, not `TRUNCATE` — `TRUNCATE` is DDL in MariaDB and implicitly
* commits, which would defeat exactly that guarantee. (The same trap the spawn
* atlas import documents; at ~123k rows `DELETE` is still well under a second.)
*/
async function replaceAll(entries, meta) {
const conn = await pool.getConnection()
try {
await conn.beginTransaction()
await conn.query('DELETE FROM shard_clilocs')
// Blank entries are dropped rather than stored. Roughly HALF of a real
// cliloc table is empty strings — ids the client reserves and never uses —
// and a row that resolves to no name is indistinguishable from no row at
// all to every caller. Dropping them halves the table (123,490 → ~67,500)
// and, more importantly, makes the binary and text imports converge on
// identical content: the binary format carries the blanks explicitly and a
// text export may or may not, depending on the tool.
//
// Later duplicates win. Merging across sources already happened upstream in
// `readCliloc`, so in practice this collapses nothing — it is kept because
// the plain format permits a repeated id WITHIN one file and the client's
// own loader resolves it the same way (its dictionary assignment
// overwrites). Without it, a file the game itself would load happily would
// fail the batch insert on a primary-key collision.
const byNumber = new Map()
let blank = 0
for (const entry of entries) {
if (!Number.isInteger(entry.number)) continue
if (String(entry.text ?? '').trim() === '') {
blank++
continue
}
byNumber.set(entry.number, entry)
}
const rows = [...byNumber.values()].map((e) => [e.number, e.flag ?? 0, e.text])
for (let i = 0; i < rows.length; i += BATCH) {
await conn.batch('INSERT INTO shard_clilocs (number, flag, text) VALUES (?,?,?)', rows.slice(i, i + BATCH))
}
await conn.query(
'INSERT INTO shard_cliloc_meta (id, payload) VALUES (1, ?) ' +
'ON DUPLICATE KEY UPDATE payload = VALUES(payload), imported_at = CURRENT_TIMESTAMP',
[JSON.stringify({ ...meta, count: rows.length })],
)
await conn.commit()
return { count: rows.length, blank, duplicates: entries.length - blank - rows.length }
} catch (err) {
await conn.rollback().catch(() => {})
throw err
} finally {
conn.release()
}
}
async function getMeta() {
const rows = await query('SELECT payload, imported_at FROM shard_cliloc_meta WHERE id = 1')
if (rows.length === 0) return null
const payload = typeof rows[0].payload === 'string' ? JSON.parse(rows[0].payload) : rows[0].payload
return { ...payload, importedAt: rows[0].imported_at }
}
/**
* Look up a batch of ids.
*
* Batched rather than one-at-a-time because every caller has a LIST: a character
* sheet resolves a dozen equipment ids at once, and a page of marketplace
* listings resolves fifty. `IN (...)` with generated placeholders keeps it one
* round trip and one parameterized statement.
*/
async function lookup(numbers) {
if (!Array.isArray(numbers) || numbers.length === 0) return []
const ids = [...new Set(numbers.filter((n) => Number.isInteger(n)))]
if (ids.length === 0) return []
const placeholders = ids.map(() => '?').join(',')
return query(`SELECT number, text FROM shard_clilocs WHERE number IN (${placeholders})`, ids)
}
async function count() {
const rows = await query('SELECT COUNT(*) AS n FROM shard_clilocs')
return Number(rows[0]?.n) || 0
}
module.exports = {
replaceAll,
getMeta,
lookup,
count,
}

368
server/src/model/shardClilocs/shardClilocs.model.js
View File

@@ -1,368 +0,0 @@
const db = require('./shardClilocs.db')
const settings = require('../settings/settings.model')
const { displayText } = require('../../utils/clilocParse')
const {
ClilocFormatError,
ClilocSourceError,
PARSER_VERSION,
hashSources,
sameSources,
missingSources,
readCliloc,
} = require('../../utils/clilocSource')
const log = require('../../utils/logger')('shardClilocs')
// The cliloc table — UO's id → display-string map, refreshed from a file the
// operator converts once from their own client.
//
// Why the site holds this at all: items on the wire carry a `LabelNumber`, not a
// name. `char.profile.equipment` has always sent `cliloc`, and every marketplace
// listing sends one too. Without the table the UI can only print `id 1023721`
// where the game prints "quarter staff".
//
// Two rules govern the boot path, both inherited from the spawn atlas:
//
// 1. **It never blocks startup.** No configured path, an unreadable file, a
// wrong-format file, a database error — all caught and logged. The site
// comes up either way, serving whatever table it already had (or none, in
// which case the UI falls back to item ids exactly as it did before).
// 2. **Nothing client-derived is committed.** The table is built from the
// operator's own file at a configured path. The repo ships no strings.
//
// The table is built from a SET of sources — the converted client table plus
// every operator-maintained overlay beside it — because shards edit items and
// add new ones, and those carry cliloc ids no stock client table has. All of
// them are re-read on every boot and hash-gated together, so adding one custom
// item never means re-exporting a 5 MB client file. Later sources win.
//
// That set is also why this has the atlas's escalation, in a lighter form. A
// single corrupt file fails the parse loudly, but a source that has simply
// VANISHED parses perfectly and imports a table quietly missing everything it
// contributed — the same ambiguity (real change vs half-copied mount) the atlas
// stages a facet removal for. So a disappearing source is refused and reported
// rather than applied.
//
// It is lighter than the atlas's because it needs to be: the atlas stores a
// pending decision in its own table and adds approve/reject endpoints, whereas
// here the decision is a single boolean an admin passes to the import they were
// already going to run. Re-parsing at approval time — the property that makes
// the atlas store only the decision — is automatic when there is nothing stored.
const SETTING_KEY = 'cliloc_client_path'
/**
* Where the converted cliloc file lives.
*
* The admin setting wins over the environment so an operator can repoint it
* without a redeploy, matching how the rest of the shard integration is
* admin-managed rather than env-configured. `UO_CLIENT_PATH` remains as the
* deploy-time default, since the path usually describes a mount the deployment
* sets up.
*/
async function getClientPath() {
try {
const configured = await settings.get(SETTING_KEY)
if (configured && String(configured).trim() !== '') return String(configured).trim()
} catch {
// Settings unavailable is not fatal — fall through to the env default.
}
const fromEnv = process.env.UO_CLIENT_PATH
return fromEnv && fromEnv.trim() !== '' ? fromEnv.trim() : ''
}
async function setClientPath(value, updatedBy = null) {
const result = await settings.set(SETTING_KEY, String(value ?? '').trim(), updatedBy)
invalidate()
return result
}
// ── Refresh ────────────────────────────────────────────────────────────────
/** Was the loaded table built by THIS parser? */
const currentParser = (meta) => meta?.parserVersion === PARSER_VERSION
/**
* Refresh the cliloc table from the configured file.
*
* Returns a result describing what happened rather than throwing, so the caller
* — including the boot path — can log it and move on:
*
* `skipped` no path configured
* `unavailable` path configured but missing / unreadable / not a cliloc file
* `unchanged` source hashes match the loaded table; nothing parsed
* `imported` parsed and applied
* `needsReview` a previously-present source has vanished; NOT applied
* `failed` parsed or applied and something went wrong
*
* `force` skips the hash check (an admin asking for a reimport). `approve`
* additionally accepts a vanished source.
*/
async function refresh({ force = false, approve = false, path: pathOverride = '' } = {}) {
// An explicit override wins outright — a one-off "use this file", which must
// not be silently overruled by the configured path the way an env default is.
const configured = pathOverride.trim() !== '' ? pathOverride.trim() : await getClientPath()
if (configured === '') return { status: 'skipped', reason: 'no cliloc path configured' }
let fingerprint
try {
fingerprint = hashSources(configured)
} catch (err) {
if (err instanceof ClilocSourceError) {
return { status: 'unavailable', reason: err.message, code: err.code, path: configured }
}
return { status: 'failed', reason: err.message, path: configured }
}
const meta = await db.getMeta().catch(() => null)
// Two things make a loaded table stale: any source changed, or the PARSER did.
// Only checking the sources would strand an install whose client never patches
// on whatever an older build derived.
if (!force && sameSources(fingerprint.hashes, meta?.hashes) && currentParser(meta)) {
return {
status: 'unchanged',
path: configured,
file: fingerprint.file,
count: meta.count ?? null,
customCount: fingerprint.customCount,
}
}
// A source that was there last import and is not there now is refused, not
// applied — an unmounted volume and a deliberate deletion look identical from
// here, and the wrong guess silently drops every name that file contributed.
const gone = missingSources(fingerprint.hashes, meta?.hashes)
if (gone.length > 0 && !approve) {
return {
status: 'needsReview',
reason: `${gone.length} previously-loaded cliloc source(s) are missing; the existing table is unchanged`,
missingSources: gone,
path: configured,
file: fingerprint.file,
}
}
let parsed
try {
parsed = readCliloc(configured)
} catch (err) {
if (err instanceof ClilocFormatError || err instanceof ClilocSourceError) {
return { status: 'unavailable', reason: err.message, code: err.code, path: configured }
}
return { status: 'failed', reason: err.message, path: configured }
}
try {
const applied = await db.replaceAll(parsed.entries, parsed.source)
invalidate()
return {
status: 'imported',
path: configured,
file: parsed.source.file,
count: applied.count,
parsed: parsed.entries.length,
blank: applied.blank,
// Per-source breakdown: how many entries each file contributed and how
// many of them overrode something already merged. An operator who adds an
// overlay wants to see it took effect, and "overrode: 0" on a file meant
// to re-label stock items says it did not.
sources: parsed.source.sources,
acceptedMissing: gone.length > 0 ? gone : undefined,
}
} catch (err) {
return { status: 'failed', reason: err.message, path: configured }
}
}
/**
* Boot hook. Best-effort by contract: it logs and returns, never throws, so a
* missing or malformed cliloc file can never stop the site coming up.
*/
async function refreshOnBoot() {
try {
const result = await refresh()
switch (result.status) {
case 'imported':
log.info('cliloc table refreshed', {
file: result.file,
count: result.count,
overlays: (result.sources || []).filter((s) => s.kind === 'custom').length,
})
break
case 'needsReview':
log.warn(
'cliloc refresh staged for admin review — a previously-loaded source is missing; ' +
'the existing table is unchanged',
{ missing: result.missingSources },
)
break
case 'unavailable':
// Deliberately a warning, not an error: an operator who has not supplied
// a cliloc file is in a supported state (the UI shows item ids), and the
// most common cause — pointing at the client's own compressed file —
// needs the reason spelled out rather than a stack trace.
log.warn('cliloc source unavailable (item names will show as ids)', {
reason: result.reason,
code: result.code,
path: result.path,
})
break
case 'failed':
log.warn('cliloc refresh failed', { reason: result.reason })
break
default:
break
}
return result
} catch (err) {
log.warn('cliloc refresh errored', { error: err.message })
return { status: 'failed', reason: err.message }
}
}
/** Everything the admin panel needs to describe cliloc state. */
async function status({ path: pathOverride = '' } = {}) {
const configured = pathOverride.trim() !== '' ? pathOverride.trim() : await getClientPath()
const meta = await db.getMeta().catch(() => null)
const loaded = await db.count().catch(() => 0)
let fileReadable = false
let file = null
let drift = null
let problem = null
let code = null
let sources = []
let missing = []
if (configured !== '') {
try {
const fingerprint = hashSources(configured)
fileReadable = true
file = fingerprint.file
sources = Object.keys(fingerprint.hashes)
missing = missingSources(fingerprint.hashes, meta?.hashes)
// A compressed file is readable but not importable, and the panel has to
// say so HERE — otherwise pointing at an unconverted client directory
// reports a healthy file with pending drift ("ready to import") and the
// operator only finds out when the import fails. `drift` stays null
// because comparing hashes with an unusable file answers nothing.
if (fingerprint.compressed) {
problem =
'This is a compressed (Mythic-format) cliloc file, which the site cannot read. ' +
'Convert it to the plain format first — see docs/website/CLILOCS.md.'
code = 'COMPRESSED'
} else {
drift = !sameSources(fingerprint.hashes, meta?.hashes) || !currentParser(meta)
}
} catch (err) {
fileReadable = false
problem = err.message
code = err.code ?? null
}
}
return {
configured: configured !== '',
path: configured,
file,
fileReadable,
problem,
code,
drift,
count: loaded,
// Every source found now (base first, then overlays), what each contributed
// at the last import, and any that have since vanished — which is the state
// an import will refuse without `approve`.
sources,
loadedSources: meta?.sources ?? null,
missingSources: missing,
importedAt: meta?.importedAt ?? null,
sourceBytes: meta?.bytes ?? null,
}
}
// ── Lookup ─────────────────────────────────────────────────────────────────
//
// Resolution happens SERVER-SIDE, not in the browser. Two reasons: the table is
// ~123k rows and shipping it to a client would dwarf every page that uses it,
// and the Android app consumes the same JSON and would otherwise need its own
// copy. Callers get names, not ids-plus-a-table.
// A small write-through cache in front of the table. Item ids repeat heavily —
// one page of listings is mostly the same few hundred clilocs, and a character
// sheet re-resolves the same gear on every view — so this turns the steady state
// into zero queries. Capped so a pathological caller cannot grow it without
// bound; on overflow it is dropped wholesale rather than evicted entry-by-entry,
// which is cheap and correct for a table that only changes on reimport.
const CACHE_MAX = 20000
let cache = new Map()
function invalidate() {
cache = new Map()
}
/**
* Resolve a batch of cliloc ids to display strings.
*
* Returns a `Map<number, string>` holding only the ids that resolved to
* something displayable — an id with no row, or one whose text is nothing but
* interpolated arguments we do not have, is simply absent. Callers fall back to
* whatever they had (the item id), so "missing" and "unnamed" collapse into one
* branch at the call site.
*
* Never throws: a cliloc lookup is decoration on someone's character sheet, and
* a database blip must not fail the sheet.
*/
async function resolveMany(numbers) {
const out = new Map()
if (!Array.isArray(numbers)) return out
const wanted = [...new Set(numbers.filter((n) => Number.isInteger(n) && n > 0))]
if (wanted.length === 0) return out
const missing = []
for (const number of wanted) {
if (cache.has(number)) {
const hit = cache.get(number)
if (hit !== '') out.set(number, hit)
} else {
missing.push(number)
}
}
if (missing.length > 0) {
try {
const rows = await db.lookup(missing)
const found = new Map(rows.map((r) => [Number(r.number), displayText(r.text)]))
if (cache.size + missing.length > CACHE_MAX) invalidate()
for (const number of missing) {
// Cache the miss too ('' meaning "no usable name"), so an id absent from
// the table does not re-query on every page view.
const text = found.get(number) ?? ''
cache.set(number, text)
if (text !== '') out.set(number, text)
}
} catch (err) {
log.warn('cliloc lookup failed', { message: err.message })
}
}
return out
}
/** Single-id convenience. Returns `null` when there is no usable name. */
async function resolve(number) {
const found = await resolveMany([number])
return found.get(number) ?? null
}
module.exports = {
SETTING_KEY,
getClientPath,
setClientPath,
refresh,
refreshOnBoot,
status,
resolveMany,
resolve,
invalidate,
}

46
server/src/model/shardEvents/shardEvents.db.js
View File

@@ -1,46 +0,0 @@
const { query } = require('../../utils/db')
// INSERT IGNORE on the UNIQUE dedupe_key — a re-ingested event (WS-reconnect
// backfill overlap) is silently skipped rather than duplicated. Returns true if
// a new row was actually inserted.
async function insertIgnore({ kind, t, bootId, payload, dedupeKey }) {
const res = await query(
`INSERT IGNORE INTO shard_events (kind, t, boot_id, payload, dedupe_key)
VALUES (?, ?, ?, ?, ?)`,
[kind, t, bootId || null, JSON.stringify(payload), dedupeKey],
)
return res.affectedRows > 0
}
// Recent events, newest first. Filter by a single `kind`, or an allowlist of
// `kinds` (IN clause) — the public feed uses the allowlist so it can never leak
// staff/sensitive kinds. limit is clamped by the model.
async function list({ kind, kinds, limit }) {
// An allowlist that resolved to NOTHING means "serve nothing" — never "serve
// everything". Falling through to the unfiltered query below would have turned
// a fully-gated visibility config into a full dump of the event log, staff
// audit and cheat detections included.
if (kinds && kinds.length === 0) return []
if (kinds && kinds.length) {
const placeholders = kinds.map(() => '?').join(', ')
return query(
`SELECT id, kind, t, boot_id, payload, created_at
FROM shard_events WHERE kind IN (${placeholders}) ORDER BY t DESC LIMIT ?`,
[...kinds, limit],
)
}
if (kind) {
return query(
`SELECT id, kind, t, boot_id, payload, created_at
FROM shard_events WHERE kind = ? ORDER BY t DESC LIMIT ?`,
[kind, limit],
)
}
return query(
`SELECT id, kind, t, boot_id, payload, created_at
FROM shard_events ORDER BY t DESC LIMIT ?`,
[limit],
)
}
module.exports = { insertIgnore, list }

61
server/src/model/shardEvents/shardEvents.model.js
View File

@@ -1,61 +0,0 @@
// Append-only shard event log. The WS ingest dispatcher calls append() for the
// notable kinds; the public/admin read endpoints call list(). The DB layer only
// sees an already-computed dedupe_key so INSERT IGNORE is idempotent across
// WS-reconnect backfill.
const crypto = require('crypto')
const db = require('./shardEvents.db')
const MAX_LIMIT = 1000
const DEFAULT_LIMIT = 100
// Stable stringify — keys sorted — so the dedupe hash is independent of the
// property order the sidecar happened to serialize with.
function stableStringify(value) {
if (value === null || typeof value !== 'object') return JSON.stringify(value)
if (Array.isArray(value)) return `[${value.map(stableStringify).join(',')}]`
const keys = Object.keys(value).sort()
const entries = keys.map((k) => `${JSON.stringify(k)}:${stableStringify(value[k])}`)
return `{${entries.join(',')}}`
}
// dedupe_key = sha256(kind + t + stable-json(payload)), truncated to 40 hex chars.
// This is a content fingerprint for idempotent INSERT IGNORE, not a security value,
// but we use SHA-256 rather than SHA-1 anyway; the truncation keeps it inside the
// CHAR(40) column (160 bits is ample collision resistance for dedupe). Two identical
// events (same kind, same timestamp, same body) collapse to one row.
function dedupeKey(kind, t, payload) {
return crypto
.createHash('sha256')
.update(`${kind}|${t}|${stableStringify(payload)}`)
.digest('hex')
.slice(0, 40)
}
// Append one event. Returns true if a new row was inserted (false = deduped).
async function append({ kind, t, bootId, payload }) {
return db.insertIgnore({ kind, t, bootId, payload, dedupeKey: dedupeKey(kind, t, payload) })
}
function normalizeLimit(limit) {
const n = Number(limit)
if (!Number.isFinite(n) || n <= 0) return DEFAULT_LIMIT
return Math.min(Math.floor(n), MAX_LIMIT)
}
// Recent events, newest first. Each row's JSON payload is parsed back to an
// object. `kinds` (array) restricts to an allowlist; `kind` filters a single kind.
async function list({ kind, kinds, limit } = {}) {
const rows = await db.list({ kind, kinds, limit: normalizeLimit(limit) })
return rows.map((row) => ({
id: row.id,
kind: row.kind,
t: row.t,
bootId: row.boot_id || null,
// mariadb returns JSON columns as strings on some versions; parse defensively.
payload: typeof row.payload === 'string' ? JSON.parse(row.payload) : row.payload,
createdAt: row.created_at,
}))
}
module.exports = { append, list, dedupeKey }

42
server/src/model/shardLinks/shardLinks.db.js
View File

@@ -1,42 +0,0 @@
const { query } = require('../../utils/db')
const COLS = 'account, user_id, char_name, linked_at'
// Upsert a link. account is the PK, so a re-link moves the account to the new
// user (the sidecar already treats /link/confirm as authoritative).
async function upsert({ account, userId, charName }) {
await query(
`INSERT INTO shard_account_links (account, user_id, char_name)
VALUES (?, ?, ?)
ON DUPLICATE KEY UPDATE user_id = VALUES(user_id), char_name = VALUES(char_name)`,
[account, userId, charName || null],
)
return getByAccount(account)
}
async function getByAccount(account) {
const rows = await query(`SELECT ${COLS} FROM shard_account_links WHERE account = ? LIMIT 1`, [account])
return rows[0] || null
}
const listByUser = (userId) =>
query(`SELECT ${COLS} FROM shard_account_links WHERE user_id = ? ORDER BY linked_at DESC`, [userId])
async function isOwnedBy(account, userId) {
const rows = await query(
'SELECT 1 FROM shard_account_links WHERE account = ? AND user_id = ? LIMIT 1',
[account, userId],
)
return rows.length > 0
}
const remove = (account, userId) =>
query('DELETE FROM shard_account_links WHERE account = ? AND user_id = ?', [account, userId])
// Drop the mirror for an account regardless of which user held it — used to
// reconcile when the tie is severed at the source (an in-game [unlink →
// account.unlinked event, or a site-side DELETE /link/{account}).
const removeByAccount = (account) =>
query('DELETE FROM shard_account_links WHERE account = ?', [account])
module.exports = { upsert, getByAccount, listByUser, isOwnedBy, remove, removeByAccount }

37
server/src/model/shardLinks/shardLinks.model.js
View File

@@ -1,37 +0,0 @@
// Site-side mirror of in-game-account → website-user links. The sidecar owns the
// authoritative link (it tags the game account on /link/confirm); this model
// records it locally so the player portal can list links and enforce ownership.
const db = require('./shardLinks.db')
function toSafe(row) {
if (!row) return null
return {
account: row.account,
userId: row.user_id,
charName: row.char_name || null,
linkedAt: row.linked_at,
}
}
async function link({ account, userId, charName }) {
return toSafe(await db.upsert({ account, userId, charName }))
}
async function listForUser(userId) {
const rows = await db.listByUser(userId)
return rows.map(toSafe)
}
const ownsAccount = (account, userId) => db.isOwnedBy(account, userId)
async function getByAccount(account) {
return toSafe(await db.getByAccount(account))
}
const unlink = (account, userId) => db.remove(account, userId)
// Drop the local mirror for an account (source-of-truth severed elsewhere).
const removeByAccount = (account) => db.removeByAccount(account)
module.exports = { link, listForUser, ownsAccount, getByAccount, unlink, removeByAccount }

299
server/src/model/shardMarket/shardMarket.db.js
View File

@@ -1,299 +0,0 @@
const { pool, query } = require('../../utils/db')
// Raw SQL for the player-vendor market index (Protocol 3.0 vendor.listing).
//
// Two tables, both INGEST-OWNED: `shard_vendors` (one row per shop) and
// `shard_vendor_items` (one row per priced listing). Nothing else in the codebase
// writes to either. No foreign keys, consistent with every other shard_* table.
// Insert batch size for one vendor's listings. A shop is capped at
// MarketMaxListings (250 by default) on the shard side, so in practice this is
// one batch — it exists for the operator who raised that cap.
const BATCH = 500
// LIKE wildcards in user input. `%` and `_` are not special to the parameterized
// query — they are special to LIKE itself — so a search for "50% off" would
// otherwise match everything containing "50" and a search for "_" would match
// every single-character name. Escaped with a backslash, which is MariaDB's
// default LIKE escape (no ESCAPE clause needed).
const likeTerm = (q) => `%${String(q).replace(/[\\%_]/g, (c) => `\\${c}`)}%`
/**
* Replace one vendor's whole row and listing set, in one transaction.
*
* Delete-then-insert rather than a diff, because the frame is AUTHORITATIVE for
* that vendor: the shard's sweep only emits a shop whose contents, prices or
* location moved, and when it does it sends the whole shop. Reconciling it item
* by item would be more code for the same result and would leave sold items
* behind on any path the reconciliation missed.
*
* All-or-nothing matters here for a specific reason: the two writes are "the
* shop" and "what is in it", and a failure between them leaves a shop advertising
* an inventory it no longer has (or none at all) — visibly wrong on the page, and
* indistinguishable from a genuinely empty shop.
*/
async function replaceVendor(vendor, items) {
const conn = await pool.getConnection()
try {
await conn.beginTransaction()
await conn.query(
`INSERT INTO shard_vendors
(serial, shop_name, owner_serial, owner_name, map, x, y, z, region, house,
item_count, item_total, truncated, t)
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?)
ON DUPLICATE KEY UPDATE shop_name = VALUES(shop_name), owner_serial = VALUES(owner_serial),
owner_name = VALUES(owner_name), map = VALUES(map), x = VALUES(x), y = VALUES(y),
z = VALUES(z), region = VALUES(region), house = VALUES(house),
item_count = VALUES(item_count), item_total = VALUES(item_total),
truncated = VALUES(truncated), t = VALUES(t),
-- Touched explicitly rather than left to ON UPDATE CURRENT_TIMESTAMP:
-- MariaDB does not fire that when every column is written back
-- unchanged, and a shop that is re-published identically is still
-- FRESHLY CONFIRMED. Without this the staleness banner would age a
-- perfectly current shop forever.
updated_at = CURRENT_TIMESTAMP`,
[
vendor.serial,
vendor.shopName ?? null,
vendor.ownerSerial ?? null,
vendor.ownerName ?? null,
vendor.map ?? null,
Number.isFinite(vendor.x) ? vendor.x : null,
Number.isFinite(vendor.y) ? vendor.y : null,
Number.isFinite(vendor.z) ? vendor.z : null,
vendor.region ?? null,
vendor.house ?? null,
items.length,
Number.isFinite(vendor.itemTotal) ? vendor.itemTotal : items.length,
vendor.truncated ? 1 : 0,
Number.isFinite(vendor.t) ? vendor.t : null,
],
)
await conn.query('DELETE FROM shard_vendor_items WHERE vendor_serial = ?', [vendor.serial])
const rows = items.map((i) => [
vendor.serial,
i.serial,
i.itemId,
i.hue,
i.amount,
i.price,
i.name,
i.cliloc,
i.displayName,
i.child ? 1 : 0,
])
for (let i = 0; i < rows.length; i += BATCH) {
await conn.batch(
`INSERT INTO shard_vendor_items
(vendor_serial, serial, item_id, hue, amount, price, name, cliloc, display_name, child)
VALUES (?,?,?,?,?,?,?,?,?,?)`,
rows.slice(i, i + BATCH),
)
}
await conn.commit()
return { items: rows.length }
} catch (err) {
await conn.rollback().catch(() => {})
throw err
} finally {
conn.release()
}
}
/** Drop one vendor and its listings (vendor.listing.remove). */
async function removeVendor(serial) {
const conn = await pool.getConnection()
try {
await conn.beginTransaction()
await conn.query('DELETE FROM shard_vendor_items WHERE vendor_serial = ?', [serial])
await conn.query('DELETE FROM shard_vendors WHERE serial = ?', [serial])
await conn.commit()
} catch (err) {
await conn.rollback().catch(() => {})
throw err
} finally {
conn.release()
}
}
// ── Search ─────────────────────────────────────────────────────────────────
//
// The unit of a search RESULT is a listing, not a vendor: "who sells a vanquishing
// kryss and for how much" is the question, and answering it per vendor would make
// the caller flatten the shops back out. The vendor's columns ride along on the
// join so a result row is self-contained.
function searchWhere({ q, minPrice, maxPrice, itemId, map, region }) {
const where = ['i.price > 0']
const params = []
if (q) {
// Both the resolved display name and the item's own literal, because an item
// with a player-set name (most of what is actually worth searching for on a
// player-run shard) may have a generic cliloc.
where.push('(i.display_name LIKE ? OR i.name LIKE ?)')
params.push(likeTerm(q), likeTerm(q))
}
if (Number.isFinite(minPrice)) {
where.push('i.price >= ?')
params.push(minPrice)
}
if (Number.isFinite(maxPrice)) {
where.push('i.price <= ?')
params.push(maxPrice)
}
if (Number.isFinite(itemId)) {
where.push('i.item_id = ?')
params.push(itemId)
}
if (map) {
where.push('v.map = ?')
params.push(map)
}
if (region) {
where.push('v.region = ?')
params.push(region)
}
return { sql: `WHERE ${where.join(' AND ')}`, params }
}
// Whitelisted, because this interpolates into the statement. `recent` sorts by
// the vendor's freshness, which is the only way to see what has just been listed
// on a shard whose sweep is minutes wide.
const SORTS = {
price_asc: 'i.price ASC, i.id ASC',
price_desc: 'i.price DESC, i.id ASC',
recent: 'v.updated_at DESC, i.id ASC',
}
async function searchListings({ q, minPrice, maxPrice, itemId, map, region, sort, limit, offset }) {
const { sql, params } = searchWhere({ q, minPrice, maxPrice, itemId, map, region })
const order = SORTS[sort] || SORTS.price_asc
const rows = await query(
`SELECT i.serial, i.item_id, i.hue, i.amount, i.price, i.name, i.cliloc, i.display_name, i.child,
v.serial AS vendor_serial, v.shop_name, v.owner_serial, v.owner_name,
v.map, v.x, v.y, v.z, v.region, v.house, v.updated_at
FROM shard_vendor_items i
JOIN shard_vendors v ON v.serial = i.vendor_serial
${sql}
ORDER BY ${order}
LIMIT ? OFFSET ?`,
[...params, limit, offset],
)
const counted = await query(
`SELECT COUNT(*) AS n
FROM shard_vendor_items i
JOIN shard_vendors v ON v.serial = i.vendor_serial
${sql}`,
params,
)
return { rows, total: Number(counted[0]?.n) || 0 }
}
async function getVendor(serial) {
const rows = await query(
`SELECT serial, shop_name, owner_serial, owner_name, map, x, y, z, region, house,
item_count, item_total, truncated, t, updated_at
FROM shard_vendors WHERE serial = ?`,
[serial],
)
return rows[0] || null
}
async function listVendorItems(serial, { limit, offset }) {
return query(
`SELECT serial, item_id, hue, amount, price, name, cliloc, display_name, child
FROM shard_vendor_items
WHERE vendor_serial = ?
ORDER BY price ASC, id ASC
LIMIT ? OFFSET ?`,
[serial, limit, offset],
)
}
/**
* What the market page's header needs: how big the index is, and how stale it may
* be. `staleAt` is the OLDEST vendor row — the round-robin sweep means a shop can
* be a full cycle behind, and the page says so rather than implying live prices.
*/
async function meta() {
const rows = await query(
`SELECT COUNT(*) AS vendors, MIN(updated_at) AS stale_at, MAX(updated_at) AS fresh_at
FROM shard_vendors`,
)
const items = await query('SELECT COUNT(*) AS n FROM shard_vendor_items')
return {
vendors: Number(rows[0]?.vendors) || 0,
items: Number(items[0]?.n) || 0,
staleAt: rows[0]?.stale_at || null,
freshAt: rows[0]?.fresh_at || null,
}
}
/** The distinct facets and regions holding vendors — drives the page's filters. */
async function listPlaces() {
const maps = await query(
'SELECT DISTINCT map FROM shard_vendors WHERE map IS NOT NULL ORDER BY map',
)
const regions = await query(
'SELECT DISTINCT region FROM shard_vendors WHERE region IS NOT NULL ORDER BY region',
)
return { maps: maps.map((r) => r.map), regions: regions.map((r) => r.region) }
}
// ── Cliloc re-resolution ───────────────────────────────────────────────────
/**
* One page of listings whose name still needs resolving, for the bulk pass that
* runs after a cliloc import.
*
* Keyed on `id > after` rather than OFFSET: the pass updates the very rows it is
* scanning, and an OFFSET walk over a table being rewritten skips rows. Every
* row with a cliloc is re-read, not just the unresolved ones, because an import
* can also CHANGE a name — a shard overlay relabelling a stock item is the whole
* reason overlays exist.
*/
async function listResolvableItems(after, limit) {
return query(
`SELECT id, cliloc, name, display_name
FROM shard_vendor_items
WHERE cliloc IS NOT NULL AND cliloc > 0 AND id > ?
ORDER BY id
LIMIT ?`,
[after, limit],
)
}
/** Write back a batch of re-resolved display names. */
async function updateDisplayNames(pairs) {
if (pairs.length === 0) return 0
const conn = await pool.getConnection()
try {
await conn.batch('UPDATE shard_vendor_items SET display_name = ? WHERE id = ?', pairs)
return pairs.length
} finally {
conn.release()
}
}
module.exports = {
replaceVendor,
removeVendor,
searchListings,
getVendor,
listVendorItems,
meta,
listPlaces,
listResolvableItems,
updateDisplayNames,
likeTerm,
}

329
server/src/model/shardMarket/shardMarket.model.js
View File

@@ -1,329 +0,0 @@
// ── Player-vendor market index (Protocol 3.0 vendor.listing) ───────────────
//
// The shard-wide shop index: what every player vendor is selling, for how much,
// and where it is standing. This is the website's half of the search the in-game
// Vendor Search gump offers — the same data, the same opt-out, reachable without
// logging in to the game.
//
// Ingest is per-vendor and authoritative: the shard's round-robin sweep emits one
// `vendor.listing` frame per shop whose contents, prices or location moved, and
// the frame is the whole shop (see docs/link/v3.md §8 and BridgeMarket.cs). This
// module normalizes it into shard_vendors + shard_vendor_items and, crucially,
// resolves each listing's cliloc to a DISPLAY NAME on the way in — a search for
// "kryss" is a search over names, and the shard only ever sends numbers.
const db = require('./shardMarket.db')
const clilocs = require('../shardClilocs/shardClilocs.model')
const log = require('../../utils/logger')('shard-market')
// Defense in depth on top of the shard's own MarketMaxListings cap. The shard is
// trusted, but it is a separately-versioned component: a frame from a plugin
// whose cap was raised (or a shard running modified scripts) must not be able to
// turn one ingest into an unbounded transaction.
const MAX_ITEMS_PER_VENDOR = 5000
// Column widths in schema.sql. Truncating here rather than letting MariaDB do it
// keeps the behavior the same in strict mode, where an over-length value is an
// ERROR and would fail the whole vendor rather than shortening one name.
const MAX_NAME = 160
const MAX_SHOP = 160
const MAX_OWNER = 64
const MAX_MAP = 40
const MAX_REGION = 80
const MAX_SERIAL = 20
const clip = (value, max) => {
if (value == null) return null
const s = String(value)
return s.length > max ? s.slice(0, max) : s
}
const int = (value, fallback = 0) => {
const n = Number(value)
return Number.isFinite(n) ? Math.trunc(n) : fallback
}
// ── Ingest ─────────────────────────────────────────────────────────────────
/**
* Flatten one `vendor.listing` frame into the row shapes the DB layer wants.
*
* `location` arrives as a nested object rather than flat map/x/y/region, and that
* shape is load-bearing rather than cosmetic: the visibility projection matches
* literal JSON keys, so ONE `market.location` rule can hide a vendor's
* whereabouts only if `location` is a single key on both the live frame and the
* stored read model. Flattening it here for storage and re-nesting it on read is
* what keeps that true on both paths.
*
* Exported for tests — it is the part with rules in it, and it is pure.
*/
function flattenFrame(ev) {
const loc = (ev && ev.location) || {}
return {
serial: clip(ev.serial, MAX_SERIAL),
shopName: clip(ev.shopName, MAX_SHOP),
ownerSerial: clip(ev.ownerSerial, MAX_SERIAL),
ownerName: clip(ev.ownerName, MAX_OWNER),
map: clip(loc.map, MAX_MAP),
x: Number.isFinite(loc.x) ? Math.trunc(loc.x) : null,
y: Number.isFinite(loc.y) ? Math.trunc(loc.y) : null,
z: Number.isFinite(loc.z) ? Math.trunc(loc.z) : null,
region: clip(loc.region, MAX_REGION),
house: clip(loc.house, MAX_SHOP),
// What the SHOP holds, which is not what the frame carries when it was
// truncated. Kept apart so the page can say "showing 250 of 3,104" rather
// than presenting a partial shop as a complete one.
itemTotal: int(ev.total, int(ev.count, 0)),
truncated: ev.truncated === true,
t: Number.isFinite(ev.t) ? ev.t : null,
}
}
/**
* Resolve each listing's display name.
*
* Order of preference is the item's own literal `name` first, then the cliloc.
* That is the opposite of what "resolve the id" suggests and it is right: a
* literal name only exists because a player set one ("Bob's vanquishing kryss"),
* and it is strictly more specific than the generic cliloc the item still
* carries.
*
* One batched lookup per frame rather than per item; `resolveMany` is cached and
* never throws, so a cliloc table that is missing entirely just leaves
* `displayName` null and the page renders item ids, exactly as it did before the
* table existed.
*/
async function shapeItems(ev) {
const raw = Array.isArray(ev.items) ? ev.items.slice(0, MAX_ITEMS_PER_VENDOR) : []
const wanted = raw
.map((i) => int(i && i.cliloc, 0))
.filter((n) => n > 0)
const names = await clilocs.resolveMany(wanted)
return raw
.filter((i) => i && i.serial)
.map((i) => {
const literal = clip(i.name, MAX_NAME)
const cliloc = int(i.cliloc, 0) || null
return {
serial: clip(i.serial, MAX_SERIAL),
itemId: int(i.itemId, 0),
hue: int(i.hue, 0),
amount: int(i.amount, 1),
price: int(i.price, 0),
name: literal,
cliloc,
displayName: literal || (cliloc ? clip(names.get(cliloc) ?? null, MAX_NAME) : null),
child: i.child === true,
}
})
// Unpriced rows are inventory, not listings. The shard already drops them;
// this is the same rule enforced where the table is written, so a plugin that
// stops enforcing it cannot put un-buyable rows on the market page.
.filter((i) => i.price > 0)
}
/** Ingest one `vendor.listing` frame. */
async function upsertVendor(ev) {
if (!ev || !ev.serial) return
const vendor = flattenFrame(ev)
const items = await shapeItems(ev)
await db.replaceVendor(vendor, items)
}
/** Ingest one `vendor.listing.remove` frame. */
async function removeVendor(serial) {
if (!serial) return
await db.removeVendor(String(serial).slice(0, MAX_SERIAL))
}
// ── Read models ────────────────────────────────────────────────────────────
//
// `location` is re-nested (see flattenFrame) so the stored read model and the
// live wire frame present the same keys to the visibility projection.
const place = (r) => ({
map: r.map,
x: r.x,
y: r.y,
z: r.z,
region: r.region,
house: r.house,
})
// A listing as the search returns it: the item, plus enough of its shop to be
// actionable without a second request. `displayName` falls back to nothing rather
// than to a fabricated "Item 3922" — the client decides how to render an
// unresolved id, and inventing a name here would make it indistinguishable from
// a real one.
const shapeListing = (r) => ({
serial: r.serial,
itemId: r.item_id,
hue: r.hue,
amount: r.amount,
price: Number(r.price),
name: r.name,
cliloc: r.cliloc,
displayName: r.display_name,
child: !!r.child,
vendor: {
serial: r.vendor_serial,
shopName: r.shop_name,
ownerSerial: r.owner_serial,
ownerName: r.owner_name,
location: place(r),
updatedAt: r.updated_at,
},
})
const shapeVendor = (r) => ({
serial: r.serial,
shopName: r.shop_name,
ownerSerial: r.owner_serial,
ownerName: r.owner_name,
location: place(r),
count: r.item_count,
total: r.item_total,
truncated: !!r.truncated,
updatedAt: r.updated_at,
})
const shapeItem = (r) => ({
serial: r.serial,
itemId: r.item_id,
hue: r.hue,
amount: r.amount,
price: Number(r.price),
name: r.name,
cliloc: r.cliloc,
displayName: r.display_name,
child: !!r.child,
})
/**
* Search the index. Returns a page of LISTINGS (not vendors) plus the
* unpaginated total and the staleness stamp the page's banner needs.
*/
async function search({
q = '',
minPrice,
maxPrice,
itemId,
map = '',
region = '',
sort = 'price_asc',
limit = 50,
offset = 0,
} = {}) {
const { rows, total } = await db.searchListings({
q: q.trim(),
minPrice: Number.isFinite(minPrice) ? minPrice : undefined,
maxPrice: Number.isFinite(maxPrice) ? maxPrice : undefined,
itemId: Number.isFinite(itemId) ? itemId : undefined,
map: map.trim(),
region: region.trim(),
sort,
limit,
offset,
})
const info = await db.meta()
return {
listings: rows.map(shapeListing),
total,
limit,
offset,
// Repeated on every search response rather than left to a separate /meta
// call: the banner that says how old these prices are must age with the
// results it labels, and a client that fetched it once would keep showing a
// stamp from before the page it is looking at.
staleAt: info.staleAt,
vendors: info.vendors,
}
}
/** One shop and its listings. `null` when the index has never seen that serial. */
async function getVendor(serial, { limit = 250, offset = 0 } = {}) {
const row = await db.getVendor(serial)
if (!row) return null
const items = await db.listVendorItems(serial, { limit, offset })
return { ...shapeVendor(row), items: items.map(shapeItem) }
}
/** Index size, staleness, and the facet/region filter options. */
async function meta() {
const [info, places] = await Promise.all([db.meta(), db.listPlaces()])
return { ...info, ...places }
}
// ── Cliloc re-resolution ───────────────────────────────────────────────────
// Batch size for the post-import pass. Big enough that a 40k-row table is ~40
// round trips, small enough that a single batch is not a long-held connection.
const RESOLVE_BATCH = 1000
/**
* Re-resolve every listing's display name against the current cliloc table.
*
* Called after a cliloc import, and it has to be: the market's diff sweep will
* NOT re-send an unchanged shop just because the site learned what its items are
* called, so without this an operator who configures clilocs after the first
* market sweep sees item ids until every shop happens to change. That is the same
* class of staleness the spawn atlas avoids by re-parsing on boot — here the
* source of truth for names moved, not the data.
*
* Never throws. It is a cosmetic backfill on a table that is already serving; a
* failure means names stay as they were, which is exactly the pre-import state.
*/
async function refreshDisplayNames() {
let after = 0
let scanned = 0
let changed = 0
try {
for (;;) {
const rows = await db.listResolvableItems(after, RESOLVE_BATCH)
if (rows.length === 0) break
after = rows[rows.length - 1].id
scanned += rows.length
const names = await clilocs.resolveMany(rows.map((r) => Number(r.cliloc)))
const pairs = []
for (const row of rows) {
// The literal name still wins, so a re-resolution never overwrites a
// player-set name with the generic cliloc behind it.
const next = row.name
? clip(row.name, MAX_NAME)
: clip(names.get(Number(row.cliloc)) ?? null, MAX_NAME)
if (next !== row.display_name) pairs.push([next, row.id])
}
changed += await db.updateDisplayNames(pairs)
}
if (changed > 0) log.info('market display names refreshed', { scanned, changed })
return { scanned, changed }
} catch (err) {
log.warn('market display-name refresh failed', { message: err.message, scanned, changed })
return { scanned, changed, error: err.message }
}
}
module.exports = {
upsertVendor,
removeVendor,
search,
getVendor,
meta,
refreshDisplayNames,
flattenFrame,
shapeItems,
shapeListing,
shapeVendor,
MAX_ITEMS_PER_VENDOR,
}

369
server/src/model/shardState/shardState.db.js
View File

@@ -1,369 +0,0 @@
const { query } = require('../../utils/db')
// Shared upsert builder for the shard-state tables. Each is keyed on a single
// primary column (`pkCol` = pk); `fields` carries only the columns the model
// wants to write, so a partial refresh touches nothing else. `coalesce` keeps
// the prior column value when the incoming one is NULL (used by shard_online so a
// vitals frame that omits acct/name doesn't blank what mob.login set); otherwise
// the incoming value wins (VALUES()).
function upsertRow(table, pkCol, pk, fields, { coalesce = false } = {}) {
const cols = Object.keys(fields)
const allCols = [pkCol, ...cols]
const insertCols = allCols.map((c) => `\`${c}\``).join(', ')
const placeholders = allCols.map(() => '?').join(', ')
const rhs = coalesce
? (c) => `\`${c}\` = COALESCE(VALUES(\`${c}\`), \`${c}\`)`
: (c) => `\`${c}\` = VALUES(\`${c}\`)`
const updates = cols.map(rhs).join(', ')
return query(
`INSERT INTO ${table} (${insertCols}) VALUES (${placeholders})
ON DUPLICATE KEY UPDATE ${updates}`,
[pk, ...cols.map((c) => fields[c])],
)
}
// ── Online players ─────────────────────────────────────────────────────────
const ONLINE_COLS =
'serial, name, acct, web_id, map, x, y, z, hits, hits_max, mana, mana_max, stam, stam_max, str, dex, `int`, updated_at'
// Upsert one online player. `fields` already prepared by the model (only the
// columns it wants to write); serial is required and is the primary key.
// COALESCE variant: a char.vitals frame that omits acct/name must not blank what
// mob.login set, so an incoming NULL keeps the prior column value.
const upsertOnline = (serial, fields) =>
upsertRow('shard_online', 'serial', serial, fields, { coalesce: true })
const removeOnline = (serial) => query('DELETE FROM shard_online WHERE serial = ?', [serial])
const clearOnline = () => query('DELETE FROM shard_online')
async function countOnline() {
const rows = await query('SELECT COUNT(*) AS n FROM shard_online')
return rows[0] ? Number(rows[0].n) : 0
}
const listOnline = () =>
query(`SELECT ${ONLINE_COLS} FROM shard_online ORDER BY name ASC`)
// Online players on any of the given game accounts (admin: a user's linked
// accounts). Empty list short-circuits so we never emit `IN ()`.
const listOnlineByAccounts = (accounts) =>
accounts.length === 0
? Promise.resolve([])
: query(
`SELECT ${ONLINE_COLS} FROM shard_online
WHERE acct IN (${accounts.map(() => '?').join(', ')})
ORDER BY name ASC`,
accounts,
)
// Staff roles whose online presence is shown on the public Shard page. Players
// who link an account are NOT surfaced publicly — only staff opt into visibility
// by virtue of being staff.
const PUBLIC_ONLINE_ROLES = ['admin', 'editor', 'moderator']
// Online players whose game account is linked to a STAFF website user. Joined
// against shard_account_links (not the sidecar-supplied web_id) so a link takes
// effect immediately, regardless of whether the player has re-logged since
// linking, then through to users so only staff roles are surfaced publicly.
const listOnlineLinked = () => {
const cols = ONLINE_COLS.split(', ')
.map((c) => `o.${c}`)
.join(', ')
return query(
`SELECT ${cols}
FROM shard_online o
JOIN shard_account_links l ON l.account = o.acct
JOIN users u ON u.id = l.user_id
WHERE u.role IN (${PUBLIC_ONLINE_ROLES.map(() => '?').join(', ')})
ORDER BY o.name ASC`,
PUBLIC_ONLINE_ROLES,
)
}
// ── Economy supply series ────────────────────────────────────────────────
const insertEconomy = ({ accounts, gold, t }) =>
query('INSERT INTO shard_economy (accounts, gold, t) VALUES (?, ?, ?)', [
accounts ?? null,
gold ?? null,
t,
])
const listEconomy = (limit) =>
query('SELECT accounts, gold, t FROM shard_economy ORDER BY t DESC LIMIT ?', [limit])
async function latestEconomy() {
const rows = await query('SELECT accounts, gold, t FROM shard_economy ORDER BY t DESC LIMIT 1')
return rows[0] || null
}
// ── Houses / IDOC ────────────────────────────────────────────────────────
const HOUSE_COLS =
'serial, stage, map, x, y, z, region, name, owner_serial, owner_acct, built_on, last_refreshed, is_idoc, updated_at'
const upsertHouse = (serial, fields) => upsertRow('shard_houses', 'serial', serial, fields)
const listIdocHouses = () =>
query(`SELECT ${HOUSE_COLS} FROM shard_houses WHERE is_idoc = 1 ORDER BY updated_at DESC`)
// Houses owned by any of the given game accounts (admin: a user's linked
// accounts). IDOC houses first, then newest-refreshed. Empty list short-circuits.
const listHousesByAccounts = (accounts) =>
accounts.length === 0
? Promise.resolve([])
: query(
`SELECT ${HOUSE_REG_COLS} FROM shard_houses
WHERE owner_acct IN (${accounts.map(() => '?').join(', ')})
ORDER BY is_idoc DESC, updated_at DESC`,
accounts,
)
// ── House registry (Protocol 2.0 house.update / house.remove) ──────────────
// The registry columns extend HOUSE_COLS; a registry row is one we've seen via
// house.update (in_registry = 1), as opposed to a decay-only transition row.
const HOUSE_REG_COLS = `${HOUSE_COLS}, owner_name, co_owners, friends, price, decay, in_registry`
const removeHouse = (serial) => query('DELETE FROM shard_houses WHERE serial = ?', [serial])
// The full registered-house browser: every row we've seen via house.update.
const listRegistryHouses = () =>
query(`SELECT ${HOUSE_REG_COLS} FROM shard_houses WHERE in_registry = 1 ORDER BY name ASC`)
// ── Champion spawns ────────────────────────────────────────────────────────
const CHAMP_COLS =
'serial, category, type, name, status, active, map, x, y, z, boss_up, payload, t, updated_at'
const upsertChamp = (serial, fields) => upsertRow('shard_champs', 'serial', serial, fields)
const removeChamp = (serial) => query('DELETE FROM shard_champs WHERE serial = ?', [serial])
const clearChamps = () => query('DELETE FROM shard_champs')
// Ordered by name (matches the sidecar's /champs ordering).
const listChamps = () => query(`SELECT ${CHAMP_COLS} FROM shard_champs ORDER BY name ASC`)
// ── Help-page (support) queue ──────────────────────────────────────────────
const PAGE_COLS =
'page_id, type, sender_name, sender_acct, web_id, message, map, x, y, z, sent_ms, handled, handler, payload, updated_at'
async function upsertPage(pageId, fields) {
const cols = Object.keys(fields)
const allCols = ['page_id', ...cols]
const insertCols = allCols.map((c) => `\`${c}\``).join(', ')
const placeholders = allCols.map(() => '?').join(', ')
const updates = cols.map((c) => `\`${c}\` = VALUES(\`${c}\`)`).join(', ')
await query(
`INSERT INTO shard_pages (${insertCols}) VALUES (${placeholders})
ON DUPLICATE KEY UPDATE ${updates}`,
[pageId, ...cols.map((c) => fields[c])],
)
}
const removePage = (pageId) => query('DELETE FROM shard_pages WHERE page_id = ?', [pageId])
const clearPages = () => query('DELETE FROM shard_pages')
// Oldest-open first so the queue reads like a work list.
const listPages = () => query(`SELECT ${PAGE_COLS} FROM shard_pages ORDER BY sent_ms ASC`)
// ── Guild board (Protocol 2.0) ─────────────────────────────────────────────
const GUILD_COLS =
'id, name, abbr, members, online, alliance, leader_serial, leader_name, leader_acct, leader_web_id, payload, t, updated_at'
const upsertGuild = (id, fields) => upsertRow('shard_guilds', 'id', id, fields)
const removeGuild = (id) => query('DELETE FROM shard_guilds WHERE id = ?', [id])
const clearGuilds = () => query('DELETE FROM shard_guilds')
const listGuilds = () => query(`SELECT ${GUILD_COLS} FROM shard_guilds ORDER BY name ASC`)
// The guild an actor LEADS — matched on the current board (leader_serial or the
// linked leader_acct), so it reflects live state. Guild MEMBERSHIP for non-leaders
// is not modelled (the board carries only counts + leader), so we don't guess it.
const findGuildLedByActor = (serial, acct) =>
query(
`SELECT id, name, abbr, alliance, leader_name FROM shard_guilds
WHERE leader_serial = ? OR (leader_acct IS NOT NULL AND leader_acct = ?)
LIMIT 1`,
[serial ?? null, acct ?? null],
)
// Guilds led by any of the given game accounts (admin: a user's linked accounts).
const listGuildsLedByAccounts = (accounts) =>
accounts.length === 0
? Promise.resolve([])
: query(
`SELECT id, name, abbr, alliance, leader_name FROM shard_guilds
WHERE leader_acct IN (${accounts.map(() => '?').join(', ')})
ORDER BY name ASC`,
accounts,
)
// ── Governor board + term history (Protocol 2.0) ───────────────────────────
const GOV_COLS =
'city, governor_serial, governor_name, governor_acct, governor_web_id, elect_serial, elect_name, elect_acct, election_phase, candidates, auto_pick_at, payload, t, updated_at'
const upsertGovernor = (city, fields) => upsertRow('shard_governors', 'city', city, fields)
const listGovernors = () => query(`SELECT ${GOV_COLS} FROM shard_governors ORDER BY city ASC`)
// Cities whose current governor is one of the given game accounts (cross-link:
// does this user hold a governorship?). Empty list short-circuits.
const listGovernorshipsByAccounts = (accounts) =>
accounts.length === 0
? Promise.resolve([])
: query(
`SELECT ${GOV_COLS} FROM shard_governors
WHERE governor_acct IN (${accounts.map(() => '?').join(', ')})
ORDER BY city ASC`,
accounts,
)
// The single open term (ended_at IS NULL) for a city, if any.
async function currentGovernorTerm(city) {
const rows = await query(
'SELECT id, city, governor_serial, governor_name, governor_acct, governor_web_id, started_at, ended_at, votes FROM shard_governor_terms WHERE city = ? AND ended_at IS NULL ORDER BY started_at DESC LIMIT 1',
[city],
)
return rows[0] || null
}
const closeGovernorTerm = (id, endedAt) =>
query('UPDATE shard_governor_terms SET ended_at = ? WHERE id = ?', [endedAt, id])
const openGovernorTerm = ({ city, serial, name, acct, webId, startedAt }) =>
query(
`INSERT INTO shard_governor_terms
(city, governor_serial, governor_name, governor_acct, governor_web_id, started_at)
VALUES (?, ?, ?, ?, ?, ?)`,
[city, serial ?? null, name ?? null, acct ?? null, webId ?? null, startedAt],
)
const listGovernorTerms = (city, limit) =>
query(
'SELECT id, city, governor_serial, governor_name, governor_acct, governor_web_id, started_at, ended_at, votes FROM shard_governor_terms WHERE city = ? ORDER BY started_at DESC LIMIT ?',
[city, limit],
)
// ── Online-population snapshot (Protocol 2.0 presence.online) ───────────────
async function setPresence({ count, byFacet, byRegion, t }) {
await query(
`INSERT INTO shard_presence (id, count, by_facet, by_region, t) VALUES (1, ?, ?, ?, ?)
ON DUPLICATE KEY UPDATE count = VALUES(count), by_facet = VALUES(by_facet),
by_region = VALUES(by_region), t = VALUES(t)`,
[
Number.isFinite(count) ? count : 0,
byFacet ? JSON.stringify(byFacet) : null,
byRegion ? JSON.stringify(byRegion) : null,
Number.isFinite(t) ? t : null,
],
)
}
async function latestPresence() {
const rows = await query('SELECT count, by_facet, by_region, t FROM shard_presence WHERE id = 1')
return rows[0] || null
}
// ── Shard ruleset (Protocol 3.0 world.ruleset) ─────────────────────────────
// Singleton, same shape as shard_presence: the shard re-emits the whole frame on
// every connect, so there is nothing to merge — the latest one wins outright.
async function setRuleset({ rev, expansion, payload, t }) {
await query(
`INSERT INTO shard_ruleset (id, rev, expansion, payload, t) VALUES (1, ?, ?, ?, ?)
ON DUPLICATE KEY UPDATE rev = VALUES(rev), expansion = VALUES(expansion),
payload = VALUES(payload), t = VALUES(t)`,
[rev ?? null, expansion ?? null, payload, Number.isFinite(t) ? t : null],
)
}
async function getRuleset() {
const rows = await query(
'SELECT rev, expansion, payload, t, updated_at FROM shard_ruleset WHERE id = 1',
)
return rows[0] || null
}
// ── Points/loyalty boards (Protocol 3.0 points.board) ──────────────────────
// One row per point system. The shard only emits a system whose top N actually
// moved, so this is a sparse stream of overwrites; there is no delete, because
// the shard's set of systems is fixed at startup.
async function upsertPointsBoard({ system, name, nameCliloc, maxPoints, players, showOnGump, payload, t }) {
await query(
`INSERT INTO shard_points_boards
(system, name, name_cliloc, max_points, players, show_on_gump, payload, t)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON DUPLICATE KEY UPDATE name = VALUES(name), name_cliloc = VALUES(name_cliloc),
max_points = VALUES(max_points), players = VALUES(players),
show_on_gump = VALUES(show_on_gump), payload = VALUES(payload), t = VALUES(t)`,
[
system,
name ?? null,
Number.isFinite(nameCliloc) ? nameCliloc : null,
Number.isFinite(maxPoints) ? maxPoints : null,
Number.isFinite(players) ? players : null,
showOnGump ? 1 : 0,
payload,
Number.isFinite(t) ? t : null,
],
)
}
// Ordered by display name, falling back to the system key for a board whose name
// arrived as a bare cliloc — otherwise every unresolved board would sort together
// under NULL.
async function listPointsBoards() {
return query(
`SELECT system, name, name_cliloc, max_points, players, show_on_gump, payload, t, updated_at
FROM shard_points_boards ORDER BY COALESCE(name, system), system`,
)
}
async function getPointsBoard(system) {
const rows = await query(
`SELECT system, name, name_cliloc, max_points, players, show_on_gump, payload, t, updated_at
FROM shard_points_boards WHERE system = ?`,
[system],
)
return rows[0] || null
}
module.exports = {
upsertOnline,
removeOnline,
clearOnline,
countOnline,
listOnline,
listOnlineLinked,
listOnlineByAccounts,
insertEconomy,
listEconomy,
latestEconomy,
upsertHouse,
listIdocHouses,
listHousesByAccounts,
removeHouse,
listRegistryHouses,
upsertGuild,
removeGuild,
clearGuilds,
listGuilds,
findGuildLedByActor,
listGuildsLedByAccounts,
upsertGovernor,
listGovernors,
listGovernorshipsByAccounts,
currentGovernorTerm,
closeGovernorTerm,
openGovernorTerm,
listGovernorTerms,
setPresence,
latestPresence,
setRuleset,
getRuleset,
upsertPointsBoard,
listPointsBoards,
getPointsBoard,
upsertChamp,
removeChamp,
clearChamps,
listChamps,
upsertPage,
removePage,
clearPages,
listPages,
}

638
server/src/model/shardState/shardState.model.js
View File

@@ -1,638 +0,0 @@
// Live shard state derived from the WS feed: who is online, the gold-supply
// series, and per-house decay stage. The ingest dispatcher calls the write
// methods; the public read endpoints call the list/count methods. Writes take
// camelCase semantic objects and map to the snake_case columns; only the keys
// present are written (so a char.vitals refresh doesn't clobber login fields).
const db = require('./shardState.db')
const MAX_ECONOMY = 1000
// Small coercion helpers, kept out of the upsert builders below so those stay
// flat (each inline `?? null` / ternary otherwise adds to cognitive complexity).
const orNull = (v) => v ?? null
const toDate = (v) => (v ? new Date(v) : null)
// Owner is an actor object (or null for an abandoned house); flatten to columns.
const ownerFields = (owner) => ({
owner_serial: orNull(owner?.serial),
owner_acct: orNull(owner?.acct),
owner_name: orNull(owner?.name),
})
// Map a camelCase online descriptor to DB columns, dropping undefined keys so a
// partial refresh only touches the fields it carries.
function onlineFields(data) {
const map = {
name: data.name,
acct: data.acct,
web_id: data.webId,
map: data.map,
x: data.x,
y: data.y,
z: data.z,
hits: data.hits,
hits_max: data.hitsMax,
mana: data.mana,
mana_max: data.manaMax,
stam: data.stam,
stam_max: data.stamMax,
str: data.str,
dex: data.dex,
int: data.int,
}
const fields = {}
for (const [k, v] of Object.entries(map)) if (v !== undefined) fields[k] = v
return fields
}
// Upsert an online player (mob.login) or refresh their vitals (char.vitals).
async function upsertOnline(data) {
if (!data || !data.serial) return
await db.upsertOnline(data.serial, onlineFields(data))
}
const setOffline = (serial) => db.removeOnline(serial)
const clearOnline = () => db.clearOnline()
const onlineCount = () => db.countOnline()
function shapeOnline(r) {
return {
serial: r.serial,
name: r.name,
acct: r.acct,
webId: r.web_id,
map: r.map,
x: r.x,
y: r.y,
z: r.z,
hits: r.hits,
hitsMax: r.hits_max,
mana: r.mana,
manaMax: r.mana_max,
stam: r.stam,
stamMax: r.stam_max,
str: r.str,
dex: r.dex,
int: r.int,
updatedAt: r.updated_at,
}
}
// Only players whose account is linked to a website user (opt-in visibility).
async function listOnlineLinked() {
const rows = await db.listOnlineLinked()
return rows.map(shapeOnline)
}
async function listOnline() {
const rows = await db.listOnline()
return rows.map(shapeOnline)
}
// Append a gold-supply sample (economy.supply).
async function addEconomySample({ accounts, gold, t }) {
await db.insertEconomy({ accounts, gold, t })
}
async function listEconomy(limit = 100) {
const n = Math.min(Math.max(Number(limit) || 100, 1), MAX_ECONOMY)
const rows = await db.listEconomy(n)
// Return oldest → newest for charting.
return rows
.map((r) => ({ accounts: r.accounts, gold: r.gold == null ? null : Number(r.gold), t: r.t }))
.reverse()
}
async function latestEconomy() {
const r = await db.latestEconomy()
return r ? { accounts: r.accounts, gold: r.gold == null ? null : Number(r.gold), t: r.t } : null
}
// Upsert a house's decay stage (house.decay). is_idoc is derived from the stage.
async function upsertHouse(data) {
if (!data || !data.serial) return
const fields = {
stage: data.stage ?? null,
map: data.map ?? null,
x: data.x ?? null,
y: data.y ?? null,
z: data.z ?? null,
region: data.region ?? null,
name: data.name ?? null,
owner_serial: data.ownerSerial ?? null,
owner_acct: data.ownerAcct ?? null,
built_on: data.builtOn ? new Date(data.builtOn) : null,
last_refreshed: data.lastRefreshed ? new Date(data.lastRefreshed) : null,
is_idoc: String(data.stage).toUpperCase() === 'IDOC' ? 1 : 0,
}
await db.upsertHouse(data.serial, fields)
}
function shapeHouse(r) {
return {
serial: r.serial,
stage: r.stage,
map: r.map,
x: r.x,
y: r.y,
z: r.z,
region: r.region,
name: r.name,
ownerSerial: r.owner_serial,
ownerAcct: r.owner_acct,
// Registry fields (Protocol 2.0 house.update); undefined on decay-only rows.
ownerName: r.owner_name,
coOwners: r.co_owners,
friends: r.friends,
price: r.price == null ? null : Number(r.price),
decay: r.decay,
inRegistry: r.in_registry == null ? undefined : Boolean(r.in_registry),
builtOn: r.built_on,
lastRefreshed: r.last_refreshed,
isIdoc: Boolean(r.is_idoc),
updatedAt: r.updated_at,
}
}
async function listIdoc() {
const rows = await db.listIdocHouses()
return rows.map(shapeHouse)
}
// Houses owned by the given game accounts (admin: a user's linked accounts).
async function listHousesForAccounts(accounts) {
const rows = await db.listHousesByAccounts(accounts)
return rows.map(shapeHouse)
}
// ── House registry (Protocol 2.0 house.update / house.remove) ──────────────
// Richer per-house snapshot than the decay-transition feed. Writes only the
// registry columns (+ shared location/owner fields); is_idoc/stage stay owned by
// the house.decay path, so the two feeds never clobber each other. owner is an
// actor object (or null for an abandoned house).
async function upsertHouseRegistry(data) {
if (!data || !data.serial) return
const fields = {
name: orNull(data.name),
...ownerFields(data.owner || null),
co_owners: orNull(data.coOwners),
friends: orNull(data.friends),
price: orNull(data.price),
decay: orNull(data.decay),
region: orNull(data.region),
map: orNull(data.map),
x: orNull(data.x),
y: orNull(data.y),
z: orNull(data.z),
built_on: toDate(data.builtOn),
last_refreshed: toDate(data.lastRefreshed),
in_registry: 1,
}
await db.upsertHouse(data.serial, fields)
}
const removeHouse = (serial) => (serial ? db.removeHouse(serial) : Promise.resolve())
async function listHouses() {
const rows = await db.listRegistryHouses()
return rows.map(shapeHouse)
}
// Online players on the given game accounts (admin: a user's linked accounts).
async function listOnlineForAccounts(accounts) {
const rows = await db.listOnlineByAccounts(accounts)
return rows.map(shapeOnline)
}
// ── Champion spawns ────────────────────────────────────────────────────────
// Upsert a champ spawn's state (champ.update). The full event is stored in
// `payload` for the category-specific fields; a few columns are hoisted out for
// querying/ordering. is-boss-up is derived from bossUp (sea bosses are always up).
async function upsertChamp(ev) {
if (!ev || !ev.serial) return
await db.upsertChamp(ev.serial, {
category: orNull(ev.category),
type: orNull(ev.type),
name: orNull(ev.name),
status: orNull(ev.status),
active: ev.active ? 1 : 0,
map: orNull(ev.map),
x: orNull(ev.x),
y: orNull(ev.y),
z: orNull(ev.z),
boss_up: ev.bossUp ? 1 : 0,
payload: JSON.stringify(ev),
t: Number.isFinite(ev.t) ? ev.t : null,
})
}
const removeChamp = (serial) => (serial ? db.removeChamp(serial) : Promise.resolve())
const clearChamps = () => db.clearChamps()
// Return the stored champ.update payload (the shape the sidecar/UI expect),
// falling back to the hoisted columns if an older row lacks a payload.
function shapeChamp(r) {
const payload = typeof r.payload === 'string' ? safeJson(r.payload) : r.payload
return payload || {
kind: 'champ.update',
serial: r.serial,
category: r.category,
type: r.type,
name: r.name,
status: r.status,
active: Boolean(r.active),
map: r.map,
x: r.x,
y: r.y,
z: r.z,
bossUp: Boolean(r.boss_up),
t: r.t,
}
}
async function listChamps() {
const rows = await db.listChamps()
return rows.map(shapeChamp)
}
// Replace the whole board with a fresh snapshot (sidecar GET /champs on connect).
async function replaceChamps(spawns) {
await db.clearChamps()
for (const ev of spawns || []) await upsertChamp(ev)
}
// ── Help-page (support) queue ──────────────────────────────────────────────
// Upsert a page (page.new / page.updated). The `sender` actor object carries the
// name/acct/webId; the rest are top-level fields.
async function upsertPage(ev) {
const pageId = ev && (ev.pageId || (ev.sender && ev.sender.serial))
if (!pageId) return
const sender = ev.sender || {}
await db.upsertPage(pageId, {
type: orNull(ev.type),
sender_name: orNull(sender.name),
sender_acct: orNull(sender.acct),
web_id: orNull(sender.webId),
message: orNull(ev.message),
map: orNull(ev.map),
x: orNull(ev.x),
y: orNull(ev.y),
z: orNull(ev.z),
sent_ms: Number.isFinite(ev.sentMs) ? ev.sentMs : null,
handled: ev.handled ? 1 : 0,
handler: orNull(ev.handler),
payload: JSON.stringify(ev),
})
}
const removePage = (pageId) => (pageId ? db.removePage(pageId) : Promise.resolve())
const clearPages = () => db.clearPages()
function shapePage(r) {
const payload = typeof r.payload === 'string' ? safeJson(r.payload) : r.payload
return {
pageId: r.page_id,
type: r.type,
sender: { serial: r.page_id, name: r.sender_name, acct: r.sender_acct, webId: r.web_id },
message: r.message,
map: r.map,
x: r.x,
y: r.y,
z: r.z,
sentMs: r.sent_ms == null ? null : Number(r.sent_ms),
handled: Boolean(r.handled),
handler: r.handler,
updatedAt: r.updated_at,
// Keep the raw payload available for any field not hoisted above.
payload: payload || undefined,
}
}
async function listPages() {
const rows = await db.listPages()
return rows.map(shapePage)
}
// Replace the whole queue with a fresh snapshot (sidecar GET /pages on connect).
async function replacePages(pages) {
await db.clearPages()
for (const ev of pages || []) await upsertPage(ev)
}
// ── Guild board (Protocol 2.0) ─────────────────────────────────────────────
// Upsert a guild's roster snapshot (guild.update). The leader is an actor object
// flattened into leader_* columns; the full event lives in `payload`.
async function upsertGuild(ev) {
if (!ev || ev.id == null) return
const leader = ev.leader || {}
await db.upsertGuild(ev.id, {
name: ev.name ?? null,
abbr: ev.abbr ?? null,
members: ev.members ?? null,
online: ev.online ?? null,
alliance: ev.alliance ?? null,
leader_serial: leader.serial ?? null,
leader_name: leader.name ?? null,
leader_acct: leader.acct ?? null,
leader_web_id: leader.webId ?? null,
payload: JSON.stringify(ev),
t: Number.isFinite(ev.t) ? ev.t : null,
})
}
const removeGuild = (id) => (id == null ? Promise.resolve() : db.removeGuild(id))
const clearGuilds = () => db.clearGuilds()
function shapeGuild(r) {
const payload = typeof r.payload === 'string' ? safeJson(r.payload) : r.payload
return payload || {
kind: 'guild.update',
id: r.id,
name: r.name,
abbr: r.abbr,
members: r.members,
online: r.online,
alliance: r.alliance,
leader: r.leader_serial
? { serial: r.leader_serial, name: r.leader_name, acct: r.leader_acct, webId: r.leader_web_id }
: null,
t: r.t,
}
}
async function listGuilds() {
const rows = await db.listGuilds()
return rows.map(shapeGuild)
}
// Replace the board with a fresh snapshot (sidecar GET /guilds on connect).
async function replaceGuilds(guilds) {
await db.clearGuilds()
for (const ev of guilds || []) await upsertGuild(ev)
}
// The guild an actor leads (cross-link on the character sheet). Leadership only —
// see the db note; membership for rank-and-file isn't in the feed, so we return
// null rather than show a possibly-stale guess.
async function findGuildForActor({ serial, acct }) {
const rows = await db.findGuildLedByActor(serial ?? null, acct ?? null)
const g = rows[0]
if (!g) return null
return { id: g.id, name: g.name, abbr: g.abbr, alliance: g.alliance, role: 'leader' }
}
// Guilds led by any of a user's linked accounts (admin user-detail cross-link).
async function listGuildsLedForAccounts(accounts) {
const rows = await db.listGuildsLedByAccounts(accounts)
return rows.map((g) => ({ id: g.id, name: g.name, abbr: g.abbr, alliance: g.alliance, leaderName: g.leader_name }))
}
// ── Town governors (Protocol 2.0) ──────────────────────────────────────────
// Upsert a city's governance snapshot (city.update) AND capture term history.
// Term capture runs first (it reads the CURRENT open term to decide whether the
// governor changed) and is idempotent: a repeat/backfill of the same governor is a
// no-op, so it's safe to call on the live feed and on reconnect snapshots alike.
async function upsertGovernor(ev) {
if (!ev || !ev.city) return
await recordGovernorTransition(ev)
const gov = ev.governor
const elect = ev.governorElect
await db.upsertGovernor(ev.city, {
governor_serial: orNull(gov?.serial),
governor_name: orNull(gov?.name),
governor_acct: orNull(gov?.acct),
governor_web_id: orNull(gov?.webId),
elect_serial: orNull(elect?.serial),
elect_name: orNull(elect?.name),
elect_acct: orNull(elect?.acct),
election_phase: orNull(ev.electionPhase),
candidates: orNull(ev.candidates),
auto_pick_at: toDate(ev.autoPickAt),
payload: JSON.stringify(ev),
t: Number.isFinite(ev.t) ? ev.t : null,
})
}
// Close the open term and open a new one when the governor CHANGES. Idempotent:
// same governor as the open term ⇒ nothing happens (so backfill/duplicate
// city.update events never spawn spurious terms).
async function recordGovernorTransition(ev) {
const gov = ev.governor || null
const newSerial = gov ? gov.serial ?? null : null
const t = Number.isFinite(ev.t) ? ev.t : Date.now()
const open = await db.currentGovernorTerm(ev.city)
const openSerial = open ? open.governor_serial : null
if (open && openSerial === newSerial) return // unchanged — nothing to record
if (open) await db.closeGovernorTerm(open.id, t) // governor changed or seat vacated
if (newSerial) {
await db.openGovernorTerm({
city: ev.city,
serial: newSerial,
name: gov.name ?? null,
acct: gov.acct ?? null,
webId: gov.webId ?? null,
startedAt: t,
})
}
}
function shapeGovernor(r) {
const payload = typeof r.payload === 'string' ? safeJson(r.payload) : r.payload
return payload || {
kind: 'city.update',
city: r.city,
governor: r.governor_serial
? { serial: r.governor_serial, name: r.governor_name, acct: r.governor_acct, webId: r.governor_web_id }
: null,
governorElect: r.elect_serial
? { serial: r.elect_serial, name: r.elect_name, acct: r.elect_acct }
: null,
electionPhase: r.election_phase,
candidates: r.candidates,
t: r.t,
}
}
async function listGovernors() {
const rows = await db.listGovernors()
return rows.map(shapeGovernor)
}
// Cities the given game accounts currently govern (cross-link badge).
async function listGovernorshipsForAccounts(accounts) {
const rows = await db.listGovernorshipsByAccounts(accounts)
return rows.map(shapeGovernor)
}
// Term history for a city (look-back), newest first.
async function listGovernorHistory(city, limit = 100) {
const n = Math.min(Math.max(Number(limit) || 100, 1), 500)
const rows = await db.listGovernorTerms(city, n)
return rows.map((r) => ({
city: r.city,
governor: r.governor_serial
? { serial: r.governor_serial, name: r.governor_name, acct: r.governor_acct, webId: r.governor_web_id }
: null,
startedAt: r.started_at == null ? null : Number(r.started_at),
endedAt: r.ended_at == null ? null : Number(r.ended_at),
votes: r.votes,
}))
}
// Upsert governors without clearing (cities are fixed, no remove event); term
// capture inside upsertGovernor stays idempotent across reconnect snapshots.
async function replaceGovernors(cities) {
for (const ev of cities || []) await upsertGovernor(ev)
}
// ── Online-population snapshot (Protocol 2.0 presence.online) ───────────────
async function setPresence(ev) {
if (!ev) return
await db.setPresence({
count: ev.count,
byFacet: ev.byFacet || null,
byRegion: ev.byRegion || null,
t: ev.t,
})
}
async function latestPresence() {
const r = await db.latestPresence()
if (!r) return { count: 0, byFacet: {}, byRegion: {}, t: null }
const parse = (v) => (typeof v === 'string' ? safeJson(v) || {} : v || {})
return {
count: Number(r.count) || 0,
byFacet: parse(r.by_facet),
byRegion: parse(r.by_region),
t: r.t == null ? null : Number(r.t),
}
}
// ── Shard ruleset (Protocol 3.0 world.ruleset) ─────────────────────────────
//
// The whole frame is stored in `payload` and served back whole. Nothing is
// normalized out of it: it is a flat description of config read as one page, and
// splitting it into columns would mean a schema change every time the shard grows
// a new block. `rev` and `expansion` are hoisted only because they are cheap to
// index/display, following shard_champs' payload-plus-hoisted-columns pattern.
async function setRuleset(ev) {
if (!ev) return
await db.setRuleset({
rev: ev.rev ?? null,
expansion: ev.expansion ?? null,
payload: JSON.stringify(ev),
t: ev.t,
})
}
// The stored ruleset, or null when the shard has never published one (an old
// plugin, or Bridge.RulesetEnabled=false). Null is a real answer here — the page
// says "not published yet" rather than rendering an empty ruleset as if the shard
// had no rules — so it is deliberately not smoothed into {}.
async function getRuleset() {
const r = await db.getRuleset()
if (!r) return null
const payload = typeof r.payload === 'string' ? safeJson(r.payload) : r.payload
if (!payload) return null
return { ...payload, updatedAt: r.updated_at }
}
// ── Points/loyalty boards (Protocol 3.0 points.board) ──────────────────────
//
// The whole frame is stored in `payload`; the columns beside it are hoisted for
// listing and ordering only. The top-N list deliberately stays inside the payload
// (see schema.sql) — it is a fixed-size list read whole, like the governor board's
// candidates.
async function upsertPointsBoard(ev) {
if (!ev || !ev.system) return
await db.upsertPointsBoard({
system: String(ev.system).slice(0, 48),
name: ev.nameString ?? null,
nameCliloc: ev.nameNumber,
maxPoints: ev.maxPoints,
players: ev.players,
showOnGump: ev.showOnGump !== false,
payload: JSON.stringify(ev),
t: ev.t,
})
}
// A stored frame plus the freshness stamp. `top` is normalized to an array so a
// caller never has to guard it — a board with nobody on it is a real state (a
// system nobody has scored in yet), distinct from a system that was never
// published at all, which is absent from the table entirely.
function shapePointsBoard(r) {
const payload = (typeof r.payload === 'string' ? safeJson(r.payload) : r.payload) || {}
return {
...payload,
system: r.system,
top: Array.isArray(payload.top) ? payload.top : [],
updatedAt: r.updated_at,
}
}
async function listPointsBoards() {
const rows = await db.listPointsBoards()
return rows.map(shapePointsBoard)
}
async function getPointsBoard(system) {
const r = await db.getPointsBoard(system)
return r ? shapePointsBoard(r) : null
}
function safeJson(s) {
try {
return JSON.parse(s)
} catch {
return null
}
}
module.exports = {
upsertOnline,
setOffline,
clearOnline,
onlineCount,
listOnline,
listOnlineLinked,
listOnlineForAccounts,
addEconomySample,
listEconomy,
latestEconomy,
upsertHouse,
listIdoc,
listHousesForAccounts,
upsertHouseRegistry,
removeHouse,
listHouses,
upsertChamp,
removeChamp,
clearChamps,
listChamps,
replaceChamps,
upsertPage,
removePage,
clearPages,
listPages,
replacePages,
upsertGuild,
removeGuild,
clearGuilds,
listGuilds,
replaceGuilds,
findGuildForActor,
listGuildsLedForAccounts,
upsertGovernor,
listGovernors,
listGovernorshipsForAccounts,
listGovernorHistory,
replaceGovernors,
setPresence,
latestPresence,
setRuleset,
getRuleset,
upsertPointsBoard,
listPointsBoards,
getPointsBoard,
}

37
server/src/model/shardVisibility/shardVisibility.db.js
View File

@@ -1,37 +0,0 @@
const { query } = require('../../utils/db')
// One row per shard feature. Absent rows are fine — utils/shardVisibility.js
// compiles a default for every known feature and merges stored rows over it, so
// a fresh install with an empty table behaves exactly as the site did pre-v3.
const COLS = 'feature, enabled, audience, stream, field_rules, updated_by, updated_at'
const listAll = () => query(`SELECT ${COLS} FROM shard_feature_visibility`)
const getOne = (feature) =>
query(`SELECT ${COLS} FROM shard_feature_visibility WHERE feature = ?`, [feature])
// Upsert one feature's settings. `fieldRules` is stored as a JSON object of
// {field: rung}; the caller has already stripped locked fields and validated
// every rung against the ladder.
const upsert = ({ feature, enabled, audience, stream, fieldRules, updatedBy }) =>
query(
`INSERT INTO shard_feature_visibility (feature, enabled, audience, stream, field_rules, updated_by)
VALUES (?, ?, ?, ?, ?, ?)
ON DUPLICATE KEY UPDATE
enabled = VALUES(enabled),
audience = VALUES(audience),
stream = VALUES(stream),
field_rules = VALUES(field_rules),
updated_by = VALUES(updated_by)`,
[
feature,
enabled ? 1 : 0,
audience,
stream ? 1 : 0,
fieldRules == null ? null : JSON.stringify(fieldRules),
updatedBy ?? null,
],
)
module.exports = { listAll, getOne, upsert }

44
server/src/model/shardVisibility/shardVisibility.model.js
View File

@@ -1,44 +0,0 @@
// ── Shard feature visibility (model) ───────────────────────────────────────
//
// Thin row-shaping layer over shardVisibility.db. The policy — the ladder, the
// feature catalog, the locked fields, the kind→feature map — lives in
// utils/shardVisibility.js; this file only reads and writes rows.
const db = require('./shardVisibility.db')
// The `field_rules` JSON column comes back as a string on the mariadb driver.
function parseRules(raw) {
if (raw == null) return {}
if (typeof raw === 'object') return raw
try {
const parsed = JSON.parse(raw)
return parsed && typeof parsed === 'object' && !Array.isArray(parsed) ? parsed : {}
} catch {
return {}
}
}
const toSafe = (row) =>
row && {
feature: row.feature,
enabled: !!row.enabled,
audience: row.audience,
stream: row.stream == null ? null : !!row.stream,
fieldRules: parseRules(row.field_rules),
updatedBy: row.updated_by,
updatedAt: row.updated_at,
}
async function listAll() {
const rows = await db.listAll()
return rows.map(toSafe)
}
async function getOne(feature) {
const rows = await db.getOne(feature)
return toSafe(rows[0])
}
const upsert = (entry) => db.upsert(entry)
module.exports = { listAll, getOne, upsert }

7
server/src/model/uoLinkConfig/uoLinkConfig.db.js
View File

@@ -1,7 +0,0 @@
const singletonConfigDb = require('../singletonConfigDb')
const COLS =
'id, base_url, ws_url, auth_token_enc, protocol, enabled, status, status_detail, plugin_connected, last_event_at, boot_id, updated_by, created_at, updated_at'
// Singleton row (id = 1). See ../singletonConfigDb for the get/upsert contract.
module.exports = singletonConfigDb('uo_link_config', COLS)

88
server/src/model/uoLinkConfig/uoLinkConfig.model.js
View File

@@ -1,88 +0,0 @@
// uo-link sidecar connection config store. Mirrors botConfig/emailConfig: the DB
// layer only ever sees ciphertext, and only getWithToken() (used server-side to
// call the sidecar over REST/WS) decrypts it. The admin-facing getSafe() never
// includes the token — it exposes only `hasToken`. A blank `token` on save means
// "leave the existing token unchanged" (same convention as the other configs).
const db = require('./uoLinkConfig.db')
const secretBox = require('../../utils/secretBox')
// The wire protocol this build speaks (link/sidecar/src/main.rs PROTOCOL_VERSION).
// Only used before an admin has saved anything — the stored row wins once it exists,
// and UOLINK_PROTOCOL still overrides for an operator running an older sidecar.
const DEFAULT_PROTOCOL = Number(process.env.UOLINK_PROTOCOL) || 3
function toSafe(row) {
if (!row) {
return {
baseUrl: process.env.UOLINK_BASE_URL || null,
wsUrl: process.env.UOLINK_WS_URL || null,
protocol: DEFAULT_PROTOCOL,
enabled: false,
hasToken: false,
status: 'disconnected',
statusDetail: null,
pluginConnected: false,
lastEventAt: null,
bootId: null,
}
}
return {
baseUrl: row.base_url || null,
wsUrl: row.ws_url || null,
protocol: row.protocol || DEFAULT_PROTOCOL,
enabled: Boolean(row.enabled),
hasToken: Boolean(row.auth_token_enc),
status: row.status || 'disconnected',
statusDetail: row.status_detail || null,
pluginConnected: Boolean(row.plugin_connected),
lastEventAt: row.last_event_at || null,
bootId: row.boot_id || null,
}
}
async function getSafe() {
return toSafe(await db.get())
}
// Decrypted token included — server-side only (calling the sidecar's REST/WS
// API). Returns null when nothing has been saved yet.
async function getWithToken() {
const row = await db.get()
if (!row) return null
return { ...toSafe(row), token: row.auth_token_enc ? secretBox.decrypt(row.auth_token_enc) : null }
}
// Save admin-supplied config. `token` undefined or '' means "leave the existing
// token unchanged" (same convention as botConfig.save).
async function save({ baseUrl, wsUrl, token, protocol, enabled, updatedBy }) {
const fields = {}
if (baseUrl !== undefined) fields.base_url = baseUrl
if (wsUrl !== undefined) fields.ws_url = wsUrl
if (token) fields.auth_token_enc = secretBox.encrypt(token)
if (protocol !== undefined) fields.protocol = protocol
if (enabled !== undefined) fields.enabled = enabled ? 1 : 0
if (updatedBy !== undefined) fields.updated_by = updatedBy
const row = await db.upsert(fields)
return toSafe(row)
}
// Mirror the sidecar's last-reported connection state into the DB so the admin
// panel has something to show between polls and the public status endpoint can
// read it without a live round-trip.
async function recordStatus({ status, statusDetail, pluginConnected, lastEventAt, bootId }) {
const fields = {}
if (status !== undefined) fields.status = status
if (statusDetail !== undefined) fields.status_detail = statusDetail
if (pluginConnected !== undefined) fields.plugin_connected = pluginConnected ? 1 : 0
// lastEventAt may arrive as an ISO string (e.g. "2026-07-10T22:08:27Z"); the
// mariadb DATETIME parser rejects the "T"/"Z", so hand it a real Date (same
// fix as botConfig.recordStatus's last_connected_at).
if (lastEventAt !== undefined) fields.last_event_at = lastEventAt ? new Date(lastEventAt) : null
if (bootId !== undefined) fields.boot_id = bootId
if (Object.keys(fields).length === 0) return getSafe()
const row = await db.upsert(fields)
return toSafe(row)
}
module.exports = { getSafe, getWithToken, save, recordStatus }

208
server/src/modules/lifecycle.js Normal file
View File

@@ -0,0 +1,208 @@
// ── Module lifecycle dispatch ──────────────────────────────────────────────
//
// Phase 2, PR 5 of docs/website/MODULE_SYSTEM.md §2.7. Normative contract:
// docs/website/MODULE_API.md §2.5 (the hooks and when they run) and §4.4
// (failure is a state), plus MODULE_SYSTEM.md §2.4 (what a boot does to
// `installed_modules`).
//
// This is the database half of the loader, and it is a separate file for the
// same reason modules/schema.js is: `scripts/routeManifest.js` and
// `swagger/swagger.js` both require app.js with the pool pointed at a dead port,
// so loader.js may not reach the database. Everything here runs from server.js,
// after ensureSchema() has proved the database is up.
//
// The two halves meet at exactly one place — `loader.setState()` — so the
// in-memory record that the §4.5 dispatch guard reads and the row the admin
// panel reads are moved together and cannot disagree.
//
// Two properties this file exists to keep:
//
// 1. **A module's boot failure costs that module and nothing else** (§4.4).
// Its routes stay mounted and answer 503; the site comes up; the next
// module boots as if nothing happened.
// 2. **The row is a record of what happened, never the source of truth for
// what is mounted** (§2.4). Nothing here mounts, unmounts or re-scans. It
// reads the outcome of a scan that already happened and writes it down.
const log = require('../utils/logger')('modules')
// §2.5: shutdown races the process being killed, so a module that will not let
// go is logged and skipped rather than allowed to hang the exit. `onBoot` has no
// such budget on purpose — it delays the listener binding, which is the feature.
const SHUTDOWN_BUDGET_MS = 5000
/**
* Run one database call for one module without letting it become everyone's
* failure. Returns null on failure, having logged it.
*
* The boot path is the whole reason this exists. A row that will not update is
* bad — the admin panel shows the wrong thing — but it is strictly less bad than
* a site that will not start, and it must not stop the modules after it from
* booting.
*/
async function safe(what, fn) {
try {
return await fn()
} catch (err) {
log.error(`module bookkeeping failed: ${what}`, { error: err.message })
return null
}
}
/**
* Reconcile `installed_modules` with what the loader found, then run every
* surviving module's `onBoot`.
*
* Called once from server.js, after `ensureSchema()` (so a module's own tables
* exist) and `seedDefaults()`, and **before the HTTP listener binds** — a module
* that must not serve traffic until it has warmed a cache gets that for free
* (§2.5).
*
* The order of the four steps is the whole design:
*
* 1. `beginBoot()` clears the last boot's outcomes, so what is on display
* afterwards is what THIS boot did. `disabled` rows are left alone: that is
* an operator decision, not an outcome (§2.4).
* 2. Every module on the volume gets a row, written with null provenance if it
* does not have one — a directory placed on the volume by hand is a
* supported install (§2.4/§2.5), and without a row it could never be
* disabled or shown as failed.
* 3. Rows with no directory are marked failed, because step 1 has just reset
* them to `enabled` and a row claiming to be enabled for a module that is
* not there is the one state that is simply untrue. (A plain uninstall
* leaves `disabled`, which step 1 never touches, so this only catches a
* directory deleted by hand.)
* 4. The outcome each module already carries — disabled by the operator,
* failed during load or schema replay, or ready — is written down, and only
* then is `onBoot` dispatched.
*
* Never throws. @param {object} [deps] injection seam for tests.
*/
async function boot({ modules, model } = {}) {
/* eslint-disable global-require */
const loader = modules || require('./loader')
const rows = model || require('../model/modules/modules.model')
/* eslint-enable global-require */
// Not an error, and the same guard replayFragments carries: a process that
// never required app.js has no scan to reconcile against, and writing rows
// from an empty list would mark every installed module as missing.
if (!loader.isLoaded()) {
log.info('no module scan in this process — skipping module boot')
return
}
const scanned = loader.list()
await safe('resetting last boot\'s outcomes', () => rows.beginBoot())
for (const m of scanned) {
await safe(`recording module "${m.id}"`, () => rows.recordInstalled({
id: m.id,
name: m.name,
version: m.version,
// Null provenance is what a hand-placed directory looks like. An install
// performed through the admin panel (§2.5, a later phase) writes the row
// with its source and hash first; this refresh deliberately does not
// overwrite either, because recordInstalled leaves what it is not given.
}))
}
const stored = (await safe('reading module rows', () => rows.list())) || []
const onVolume = new Set(scanned.map((m) => m.id))
for (const row of stored) {
if (onVolume.has(row.id) || row.state === 'disabled') continue
await safe(`marking module "${row.id}" missing`, () => rows.markStartupFailed(row.id, {
stage: 'require',
reason: 'module directory not present on the volume',
}))
}
const disabled = new Set(stored.filter((r) => r.state === 'disabled').map((r) => r.id))
for (const m of scanned) {
// The operator's switch wins over everything, including a failure. Its
// routes 404 from here on (§4.5 — the leg that was unreachable until this
// PR), it is not booted, and its failure is not re-recorded: overwriting a
// deliberate `disabled` with an outcome would silently re-enable it on the
// next boot.
if (disabled.has(m.id)) {
loader.setState(m.id, 'disabled')
continue
}
if (m.state === 'startup_failed') {
// Already failed in load() or the schema replay — both of which ran before
// the database was available to write it down. This is where it lands.
await safe(`recording failure for "${m.id}"`, () => rows.markStartupFailed(m.id, {
stage: m.stage,
reason: m.reason,
}))
}
}
for (const { id, hook, ctx } of loader.bootable()) {
try {
// Awaited without a timeout, deliberately (§2.5): a slow onBoot delays the
// listener, which is the contract's promise to a module that must warm up
// before it serves. Core's own boot steps are awaited the same way.
if (hook) await hook(ctx)
loader.setState(id, 'started')
await safe(`marking module "${id}" started`, () => rows.markStarted(id))
log.info(`module "${id}" started`)
} catch (err) {
// §4.4's second column: the routes are already mounted, so they stay
// mounted and answer 503. A module that failed to warm up serving
// half-initialised data is worse than one that says it is down.
loader.setState(id, 'startup_failed', { stage: 'boot', reason: err.message })
await safe(`recording boot failure for "${id}"`, () => rows.markStartupFailed(id, {
stage: 'boot',
reason: err.message,
}))
log.error(`module "${id}" onBoot failed — its routes will answer 503`, {
reason: err.message,
})
}
}
}
/** Reject if `fn`'s promise has not settled within `ms`. */
function withBudget(fn, ms) {
let timer
const budget = new Promise((_, reject) => {
timer = setTimeout(() => reject(new Error(`onShutdown exceeded its ${ms}ms budget`)), ms)
})
return Promise.race([Promise.resolve().then(fn), budget]).finally(() => clearTimeout(timer))
}
/**
* Run every started module's `onShutdown`, in reverse registration order.
*
* Called from server.js's signal handler before anything core owns is closed, so
* a module still has a working database pool and push dispatcher to flush
* through. Reverse order is the mirror of boot order: a module that booted after
* another may be holding something the earlier one handed it.
*
* Never throws, and never hangs: each hook gets `SHUTDOWN_BUDGET_MS`, after
* which it is logged and abandoned. Abandoned, not cancelled — nothing can stop
* a promise that is still running — but the process is exiting anyway, and the
* alternative is a shard host where `systemctl stop` hangs until SIGKILL.
*/
async function shutdown({ modules, budgetMs = SHUTDOWN_BUDGET_MS } = {}) {
// eslint-disable-next-line global-require
const loader = modules || require('./loader')
if (!loader.isLoaded()) return
for (const { id, hook } of loader.shutdownHooks()) {
try {
await withBudget(hook, budgetMs)
log.info(`module "${id}" shut down`)
} catch (err) {
log.warn(`module "${id}" onShutdown failed or timed out — continuing`, {
error: err.message,
})
}
}
}
module.exports = { boot, shutdown, SHUTDOWN_BUDGET_MS }

855
server/src/modules/loader.js Normal file
View File

@@ -0,0 +1,855 @@
// ── The module loader ──────────────────────────────────────────────────────
//
// Phase 2, PR 2 of docs/website/MODULE_SYSTEM.md §2.7. The normative contract is
// docs/website/MODULE_API.md Part 4; where the two disagree, the contract wins.
//
// The one property this file exists to guarantee, and the reason it looks the
// way it does:
//
// **The filesystem is the mounting source of truth, and mounting is
// SYNCHRONOUS.** `scripts/routeManifest.js:38` and `swagger/swagger.js:29`
// both require app.js with the pool pointed at a dead port. A loader that
// awaited a database row before mounting would make every module route
// invisible to the frozen-URL-surface test (§1.12). So: readdirSync at require
// time, no database, no promises (§4.1).
//
// A module that fails ANYWHERE in this file fails alone. Nothing here may throw
// past its own try/catch — a bad module must cost the site its routes, never its
// boot (§4.4).
//
// PR 7 added the client half's server-side end: validating `client.entry` and
// publishing where the chunk lives (`clientChunks()`), so app.js can serve it and
// utils/htmlShell.js can inject its script tag. The loader resolves and validates;
// it does not mount, because the chunk hangs off the ROOT app rather than a tier
// router, and app.js is where core's own static mounts live.
//
// PR 3 added the fragment half of the schema story: this file VALIDATES a
// fragment (statement by statement, at load time, before anything is mounted)
// and publishes it through `fragments()`. Replaying it needs a database, so it
// belongs to modules/schema.js, which utils/db.js calls after core's schema.
//
// PR 5 added the lifecycle hooks a module registers here (`onBoot`/`onShutdown`)
// and the failure STAGE carried beside every reason. Dispatching those hooks and
// reconciling `installed_modules` need a database, so they live in
// modules/lifecycle.js for the same reason schema.js is a separate file: this one
// stays require-able against a dead pool.
const fs = require('fs')
const path = require('path')
const { MODULE_API_VERSION } = require('./version')
const semver = require('./semver')
const registries = require('./registries')
const { splitStatements } = require('../utils/sqlStatements')
const log = require('../utils/logger')('modules')
const REPO_ROOT = path.join(__dirname, '..', '..', '..')
const MODULES_DIR = process.env.MODULES_DIR || path.join(REPO_ROOT, 'modules')
// One segment, lowercase, no parameters. A module prefix that could contain a
// `/` or a `:` would let a module reach outside the slot it was given.
const ID = /^[a-z][a-z0-9-]{1,31}$/
const PREFIX = /^\/[a-z0-9][a-z0-9-]*$/
const TIERS = ['public', 'admin', 'player']
const MANIFEST_KEYS = new Set([
'id', 'name', 'version', 'coreApi', 'server', 'client',
'schema', 'purge', 'mounts', 'extensions', 'capabilities',
])
// Extension slots are declared by core, at require time, in the router that owns
// the resource (registries.declareSlot). The loader asks the registry which exist
// rather than keeping a list, for the same reason the prefix check probes the
// live tier routers: a second copy of the answer is a copy that drifts.
// id → record. Populated by load(), read by list().
const modules = new Map()
let loaded = false
// ── ctx ────────────────────────────────────────────────────────────────────
// Everything a module may reach in core, and nothing else (§2.3). Required
// lazily inside the factory rather than at file scope: this file is required by
// app.js, and hoisting these to the top would make the DB pool, the settings
// model and the upload directory startup-time dependencies of the loader itself.
function buildCtx(id, moduleRoot) {
/* eslint-disable global-require */
// The shared SERVER dependencies — the exact counterpart of window.__rg's
// react/react-dom/react-router on the client, and load-bearing for the same
// two reasons (§7.2).
//
// 1. A module lives at <repo>/modules/<id>/, OUTSIDE server/, so Node's
// resolver walks up from there and never sees server/node_modules. A
// module that required 'express' itself would fail to load — which is
// exactly how this was discovered.
// 2. Even if it resolved, a second copy of express in the process is a
// second Router prototype and a second set of instanceof checks. One
// express, owned by core, is the same rule as one React.
//
// The consequence for a module author is the same on both sides: declare these
// external, never bundle them, take them from what core hands you.
const express = require('express')
const validator = require('express-validator')
const db = require('../utils/db')
const settings = require('../model/settings/settings.model')
const posts = require('../model/posts/posts.model')
const auth = require('../utils/auth')
const pushDispatch = require('../utils/pushDispatch')
const secretBox = require('../utils/secretBox')
const createLogger = require('../utils/logger')
const { requireAuth, requireRole } = require('../auth/session.middleware')
const siteMode = require('../middleware/siteMode')
const validate = require('../middleware/validate')
const noindex = require('../middleware/noindex')
const uploads = require('../router/v1/admin/imageUpload')
const activity = require('../model/activity/activity.model')
const users = require('../model/users/users.model')
const { makeLimiter, accountChangeLimiter } = require('../middleware/rateLimit')
/* eslint-enable global-require */
// Narrowed on purpose (§2.3): utils/auth also re-exports signToken,
// setAuthCookie and the TOTP challenge primitives, and minting a session is
// core's job. A module that needs an identity needs to READ one.
const ctx = {
moduleId: id,
paths: { moduleRoot },
express,
validator,
db: { query: db.query, pool: db.pool },
log: (namespace) => createLogger(namespace ? `${id}:${namespace}` : id),
settings: {
get: settings.get,
set: settings.set,
getInstanceName: settings.getInstanceName,
},
auth: { getUserFromRequest: auth.getUserFromRequest },
push: { publish: pushDispatch.publish },
secretBox: { encrypt: secretBox.encrypt, decrypt: secretBox.decrypt },
middleware: {
requireAuth,
requireRole,
siteMode,
validate,
noindex,
// Rate limiting, added in API 1.1.0 as a factory plus one shared limiter.
//
// `rateLimit(options)` is core's `makeLimiter`: a module states its own
// window and cap — it knows what its endpoints cost — and takes the
// plumbing from core, so there is one express-rate-limit in the process,
// one store, and one place a breach is logged.
//
// `accountChangeLimiter` is handed over whole because it is genuinely
// shared policy: core's `/auth/me`, `/player/account` and
// `/player/appeals` are behind the same counter, and a module's
// account-change route has to land in it rather than beside it.
rateLimit: makeLimiter,
accountChangeLimiter,
},
uploads,
posts: {
listAll: posts.listAll,
getById: posts.getById,
linkAnnounceJob: posts.linkAnnounceJob,
markAnnounced: posts.markAnnounced,
},
// The admin audit log — WRITE only (§2.3, added in API 1.1.0). Core's one
// audit trail has to include the admin actions a module performs, or the
// trail has a hole exactly where a module operates the game. A module that
// kept its own log would be a second place to look, which in practice means
// a place nobody looks. `list` stays core's: reading the log is the admin
// panel's job, and it spans every actor.
activity: { log: activity.log },
// One function, for one caller: the `admin.users.detail` slot router needs
// the user its prefix names. Narrowed like `ctx.posts` — the users model
// exports creation, role changes and password handling, none of which is a
// module's business.
users: { getById: users.getById },
// Where this deployment is reachable, for a module that has to build an
// absolute link (an announcement carries one into a game window or a chat
// message, where a relative path means nothing). §2.7 forbids a module
// reading `process.env` for core configuration and this is core
// configuration, so core answers it. A getter, not a captured string: the
// value is read per call, so it cannot go stale against the env.
site: { get baseUrl() { return (process.env.APP_BASE_URL || 'http://localhost:5173').replace(/\/+$/, '') } },
}
// A guard against accident, not against a hostile module — the boundary is
// organisational, not a security boundary (MODULE_SYSTEM.md §2.2).
for (const value of Object.values(ctx)) {
if (value && typeof value === 'object') Object.freeze(value)
}
return Object.freeze(ctx)
}
// ── The registration api ───────────────────────────────────────────────────
// Collects what the module registers so validation can compare it against what
// module.json DECLARED. Declaration is the contract; a module that registers a
// prefix it did not declare is rejected, because module.json is what the admin
// panel, the collision check and the reviewer all read.
function buildApi(record) {
const once = (name) => {
if (record.called.has(name)) throw new Error(`${name}() called twice`)
record.called.add(name)
}
const hook = (name) => (fn) => {
once(name)
if (typeof fn !== 'function') throw new Error(`${name}: expected a function`)
record.hooks[name] = fn
}
return {
registerRoutes(mounts) {
once('registerRoutes')
if (!mounts || typeof mounts !== 'object') throw new Error('registerRoutes: expected an object')
for (const [tier, byPrefix] of Object.entries(mounts)) {
if (!TIERS.includes(tier)) throw new Error(`registerRoutes: unknown tier "${tier}"`)
for (const [prefix, router] of Object.entries(byPrefix)) {
if (!PREFIX.test(prefix)) throw new Error(`registerRoutes: bad prefix "${prefix}"`)
if (typeof router !== 'function') throw new Error(`registerRoutes: ${tier}${prefix} is not a router`)
record.routes[tier].set(prefix, router)
}
}
},
// The three de-entanglement registries (§2.4). They live in registries.js
// rather than here because core registers through the same staging area, and
// core has no `api` object.
//
// These STAGE. Nothing a module registers is visible to core until the
// second pass commits it, for the reason the second pass exists at all: a
// module that throws halfway through register(), or fails checkDeclared
// after it, must leave nothing behind. A half-registered stream catalog
// would be worse than a missing one — it would be a subscribable stream
// nothing will ever publish to.
registerExtension: record.staged.registerExtension,
registerNotificationStreams(streams) {
once('registerNotificationStreams')
record.staged.registerNotificationStreams(streams)
},
registerAnnounceLeg: record.staged.registerAnnounceLeg,
// The two lifecycle hooks (§2.5). Registered here, dispatched from
// lifecycle.js — this file runs with no database and the hooks run with one.
// Both are optional: a module with no warm-up and nothing to close simply
// never calls them.
onBoot: hook('onBoot'),
onShutdown: hook('onShutdown'),
}
}
// ── Validation ─────────────────────────────────────────────────────────────
/**
* Throw with the §4.3 step that failed attached.
*
* `installed_modules.failure_stage` exists so the admin panel can say *where* a
* module broke and not only what the message was, and the model enumerates the
* eight stages (`FAILURE_STAGES`). The steps that share one function — a
* manifest read that also checks `coreApi`, the mounts and the slots — cannot be
* told apart by position in load(), so they carry their own label; everything
* else is inferred from how far load() had got. An untagged error is recorded
* against the step that was running, never guessed at.
*/
function fail(stage, message) {
const err = new Error(message)
err.stage = stage
throw err
}
// Table names a module may create despite not carrying its own id as a prefix.
//
// module-uo's twenty-seven tables predate the module system by two years, and
// renaming live tables is a data migration this workstream deliberately does not
// do (MODULE_SYSTEM.md §1.6). Grandfathering them by an explicit, per-module
// allowlist keeps the prefix rule real for every module written after this one —
// the alternative, dropping the rule, would leave the first name collision to be
// discovered by a module silently adopting someone else's table.
const LEGACY_TABLE_PREFIXES = { uo: ['shard_', 'uo_link_'] }
const CREATE_TABLE = /CREATE\s+TABLE\s+(?:IF\s+NOT\s+EXISTS\s+)?[`"]?(\w+)[`"]?/gi
/** Table names core's own schema.sql declares — a module may not touch these. */
let coreTables = null
function coreTableNames() {
if (coreTables) return coreTables
coreTables = new Set()
try {
const sql = fs.readFileSync(path.join(__dirname, '..', '..', 'db', 'schema.sql'), 'utf8')
// Split first, for the same reason tablesOf() does: matching the raw file
// reads CREATE TABLE out of comments, and a phantom core table makes a
// module fail with a collision against something that does not exist.
for (const statement of splitStatements(sql)) {
for (const m of statement.matchAll(CREATE_TABLE)) coreTables.add(m[1].toLowerCase())
}
} catch (err) {
log.warn('could not read core schema for the table-collision check', { message: err.message })
}
return coreTables
}
// The only leading verbs a fragment may use — an allowlist, not a DROP denylist.
//
// §2.6 bans `DROP`, but a denylist only ever bans what somebody thought of, and
// core's own schema.sql needs exactly four verbs: CREATE, ALTER, INSERT, UPDATE.
// Anything else in a file that is REPLAYED ON EVERY BOOT is a mistake worth
// failing on — TRUNCATE and DELETE would empty a table every restart, RENAME
// would break on the second one, and GRANT/SET/USE are core's business, not a
// module's. CREATE covers CREATE INDEX as well as CREATE TABLE.
//
// This is a leading-verb check and says so: `ALTER TABLE x DROP COLUMN y` passes
// it. Catching that needs a SQL parser, which is a large dependency to take on
// for a rule whose real job is stopping the obvious foot-gun early.
const ALLOWED_VERBS = new Set(['CREATE', 'ALTER', 'INSERT', 'UPDATE'])
const CREATE_TABLE_ANY = /^CREATE\s+TABLE\s+(?:IF\s+NOT\s+EXISTS\s+)?/i
const CREATE_TABLE_GUARDED = /^CREATE\s+TABLE\s+IF\s+NOT\s+EXISTS\s+/i
/**
* Read and validate a module's schema fragment; return every table it declares.
*
* Validation happens HERE, at load time, and not in modules/schema.js where the
* fragment is replayed, because every rule §2.6 states is knowable by reading
* the file — no database required. Failing at load means a module with a bad
* fragment never mounts at all (§4.4's first column: routes and nav simply
* absent), rather than mounting, 503ing, and leaving whatever its fragment did
* manage to execute behind it.
*
* Throws if the file is unreadable or breaks a rule.
*/
function tablesOf(dir, manifest) {
if (!manifest.schema) return new Set()
const file = path.join(dir, manifest.schema)
const sql = fs.readFileSync(file, 'utf8')
const statements = splitStatements(sql)
for (const statement of statements) {
const verb = (statement.match(/^\w+/) || [''])[0].toUpperCase()
if (!ALLOWED_VERBS.has(verb)) {
throw new Error(`schema fragment statement starts with "${verb}" (allowed: ${[...ALLOWED_VERBS].join(', ')})`)
}
// A bare CREATE TABLE succeeds exactly once and fails every boot after it,
// which presents as a module that worked until the first restart.
if (CREATE_TABLE_ANY.test(statement) && !CREATE_TABLE_GUARDED.test(statement)) {
throw new Error('schema fragment has a CREATE TABLE without IF NOT EXISTS')
}
}
// Scanned over the SPLIT STATEMENTS, never the raw file. `splitStatements`
// strips `--` comments; the file does not, and a fragment that documents
// itself will say "every CREATE TABLE carries IF NOT EXISTS" in its header.
// Read raw, that yields a table called `carries`, and the module is rejected
// for a prefix violation on a table that does not exist — a message with no
// way back to the comment that caused it. module-uo's fragment hit exactly
// this on its first real load.
const tables = new Set()
for (const statement of statements) {
for (const m of statement.matchAll(CREATE_TABLE)) tables.add(m[1].toLowerCase())
}
return tables
}
function checkTableNames(id, tables) {
const allowed = LEGACY_TABLE_PREFIXES[id] || []
const core = coreTableNames()
for (const table of tables) {
if (core.has(table)) throw new Error(`schema fragment declares core table "${table}"`)
for (const other of modules.values()) {
if (other.tables.has(table)) {
throw new Error(`schema fragment declares "${table}", already owned by module "${other.id}"`)
}
}
const prefixed = table.startsWith(`${id}_`) || allowed.some((p) => table.startsWith(p))
if (!prefixed) throw new Error(`schema fragment table "${table}" is not prefixed "${id}_"`)
}
}
/**
* Does core already own this prefix in this tier?
*
* Asked of the LIVE tier router rather than a hardcoded list, so the check
* cannot drift the first time core adds a capability router — the spike's
* hardcoded table was already one prefix stale when it was written. Modules are
* loaded after every core mount, so the stack is complete by the time this runs,
* and `layer.match` is express's own matcher rather than a second-guess at its
* regexp grammar.
*
* Root-mounted layers are skipped: `use(noindex, requireAuth)` and the two
* `use('/', singletonRouter)` mounts match every path, and counting them would
* report every prefix as taken.
*/
function ownedByCore(tierRouter, prefix) {
return (tierRouter.stack || []).some(
(layer) => layer.regexp && !layer.regexp.fast_slash && layer.match(prefix),
)
}
// ── The client chunk ───────────────────────────────────────────────────────
// A chunk filename, and the same character set utils/htmlShell.js will accept in
// a script src. Two copies of the rule, deliberately: this one rejects the module
// at load time, that one refuses to write the tag. A validator three files away
// staying strict is not something an HTML attribute should depend on.
const CHUNK_FILE = /^[A-Za-z0-9][A-Za-z0-9._-]*\.js$/
/**
* Resolve and validate `client.entry` — where a module's prebuilt chunk lives on
* disk, and the URL it is served at (§3.1).
*
* The rule that matters most is the last one, and it is the one a reviewer would
* not think to ask for: the static mount is rooted at the DIRECTORY THE ENTRY IS
* IN, not at the module root. One `express.static` over a module root would
* publish its server source, its `module.json` and its schema fragment to the
* internet. So an entry sitting directly in the module root is rejected rather
* than quietly turning the whole module into a public directory.
*
* @returns {{dir: string, url: string, entryUrl: string}|null} null when the
* module ships no client half — a server-only module is perfectly normal.
*/
function resolveClient(dir, id, manifest) {
// Absent `client` is a server-only module. Present but empty is not the same
// thing: it states a client half and delivers none, which would be a module
// whose pages never load and nothing anywhere saying why.
if (manifest.client === undefined) return null
const { entry } = manifest.client
if (typeof entry !== 'string' || !entry.trim()) fail('manifest', 'client.entry must be a path')
const file = path.resolve(dir, entry)
// Containment before anything else: `../../server/src/config` resolves to a
// real, readable directory, and every check below it would pass.
if (file !== dir && !file.startsWith(dir + path.sep)) {
fail('manifest', `client.entry "${entry}" escapes the module directory`)
}
if (!CHUNK_FILE.test(path.basename(file))) {
fail('manifest', `client.entry "${entry}" must name a .js file`)
}
const chunkDir = path.dirname(file)
if (chunkDir === dir) {
fail('manifest', `client.entry "${entry}" must be in a subdirectory — its directory is served`)
}
if (!fs.existsSync(file)) fail('manifest', `client.entry "${entry}" is missing`)
return {
dir: chunkDir,
url: `/modules/${id}`,
entryUrl: `/modules/${id}/${path.basename(file)}`,
}
}
function readManifest(dir, id, tierRouters) {
const file = path.join(dir, 'module.json')
const manifest = JSON.parse(fs.readFileSync(file, 'utf8'))
for (const key of Object.keys(manifest)) {
// Rejected, not ignored: a typo'd key must be a loud failure rather than a
// silently inert setting the operator believes they configured.
if (!MANIFEST_KEYS.has(key)) fail('manifest', `unknown key "${key}" in module.json`)
}
if (!ID.test(manifest.id || '')) fail('manifest', `invalid id "${manifest.id}"`)
if (manifest.id !== id) fail('manifest', `id "${manifest.id}" does not match directory "${id}"`)
if (!manifest.version) fail('manifest', 'missing version')
if (!manifest.coreApi) fail('core_api', 'missing coreApi')
if (!semver.satisfies(MODULE_API_VERSION, manifest.coreApi)) {
fail('core_api', `needs core API ${manifest.coreApi}, this core is ${MODULE_API_VERSION}`)
}
for (const [tier, prefixes] of Object.entries(manifest.mounts || {})) {
if (!TIERS.includes(tier)) fail('mounts', `unknown tier "${tier}" in mounts`)
for (const prefix of prefixes) {
if (!PREFIX.test(prefix)) fail('mounts', `bad prefix "${prefix}" in mounts.${tier}`)
if (ownedByCore(tierRouters[tier], prefix)) {
fail('mounts', `prefix ${tier}${prefix} is owned by core`)
}
for (const other of modules.values()) {
if ((other.manifest.mounts?.[tier] || []).includes(prefix)) {
fail('mounts', `prefix ${tier}${prefix} already registered by module "${other.id}"`)
}
}
}
}
for (const slot of manifest.extensions || []) {
if (!registries.hasSlot(slot)) fail('extensions', `unknown extension slot "${slot}"`)
}
if (manifest.client !== undefined) {
if (typeof manifest.client !== 'object' || manifest.client === null || Array.isArray(manifest.client)) {
fail('manifest', 'client must be an object')
}
for (const key of Object.keys(manifest.client)) {
if (key !== 'entry') fail('manifest', `unknown key "client.${key}" in module.json`)
}
}
if (manifest.schema && !manifest.purge) {
// A module that can create tables and cannot drop them leaves an operator
// with orphaned data and no supported way to remove it.
fail('schema', 'declares schema but no purge')
}
if (manifest.purge && !fs.existsSync(path.join(dir, manifest.purge))) {
fail('schema', `purge file "${manifest.purge}" is missing`)
}
return manifest
}
// What the module registered must equal what it declared — in both directions.
function checkDeclared(record) {
const declared = record.manifest.mounts || {}
for (const tier of TIERS) {
const want = new Set(declared[tier] || [])
const got = new Set(record.routes[tier].keys())
for (const p of got) if (!want.has(p)) throw new Error(`registered ${tier}${p} without declaring it`)
for (const p of want) if (!got.has(p)) throw new Error(`declared ${tier}${p} but never registered it`)
}
}
// ── Load ───────────────────────────────────────────────────────────────────
/**
* Discover, validate, register and mount every module under MODULES_DIR.
*
* **Called exactly once, explicitly, from app.js**, after the three tier routers
* are required and before the app is exported. There is no lazy self-scan: the
* spike's was lazy and silent, so requiring the loader and reading the module
* list gave an empty array and no error (MODULE_API.md §7.6). Everything that
* reads the module list now throws until this has run.
*
* The ordering is not incidental. Core's mounts must already be on the tier
* routers, because that is what the prefix-collision check is asked about; and
* modules mount after them, so first-match-wins means a module could not shadow
* a core prefix even if the check were bypassed.
*
* Safe to call when the modules directory does not exist — that is the normal
* case for a bare core, and it is the state this PR ships in.
*
* @param {{public: Router, admin: Router, player: Router}} tierRouters
*/
function load(tierRouters) {
if (loaded) return
for (const tier of TIERS) {
if (!tierRouters || typeof tierRouters[tier] !== 'function') {
throw new Error(`modules.load: missing the "${tier}" tier router`)
}
}
loaded = true
let entries = []
try {
entries = fs.readdirSync(MODULES_DIR, { withFileTypes: true })
.filter((e) => e.isDirectory())
.map((e) => e.name)
.sort() // alphabetical: there is no dependency resolution, and any other
// order would imply a precedence nothing computes (§4.2)
} catch {
return // no modules directory is the normal case for a bare core
}
for (const id of entries) {
const dir = path.join(MODULES_DIR, id)
if (!fs.existsSync(path.join(dir, 'module.json'))) continue
const record = {
id,
dir,
manifest: null,
routes: { public: new Map(), admin: new Map(), player: new Map() },
staged: registries.stage(id),
tables: new Set(),
called: new Set(),
hooks: { onBoot: null, onShutdown: null },
client: null,
ctx: null,
state: 'installed',
stage: null,
reason: null,
}
// How far load() has got, so an untagged throw is recorded against the step
// that was actually running (§4.3's steps 5-7). The steps before it label
// themselves, because readManifest covers four of them in one pass.
let stage = 'manifest'
try {
record.manifest = readManifest(dir, id, tierRouters)
record.client = resolveClient(dir, id, record.manifest)
stage = 'schema'
record.tables = tablesOf(dir, record.manifest)
checkTableNames(id, record.tables)
if (record.manifest.server) {
const entry = path.join(dir, record.manifest.server)
stage = 'require'
// eslint-disable-next-line global-require, import/no-dynamic-require
const register = require(entry)
if (typeof register !== 'function') throw new Error(`${record.manifest.server} does not export a function`)
stage = 'register'
// Kept on the record, not discarded after register(): §2.5 hands the
// same ctx to onBoot, and building a second one would be a second frozen
// object claiming to be the same handle.
record.ctx = buildCtx(id, dir)
register(record.ctx, buildApi(record))
checkDeclared(record)
}
record.state = 'registered'
modules.set(id, record)
log.info(`registered module "${id}" v${record.manifest.version}`, {
mounts: record.manifest.mounts,
})
} catch (err) {
// A failure here is BEFORE any route was mounted, so this module's routes
// and nav are simply absent and the site comes up without it (§4.4).
record.state = 'startup_failed'
record.stage = err.stage || stage
record.reason = err.message
record.manifest = record.manifest || { id, version: 'unknown' }
modules.set(id, record)
log.error(`module "${id}" failed to load — continuing without it`, {
stage: record.stage,
reason: err.message,
})
}
}
// Mounting is a SECOND pass, after every module has been validated, and not
// because it reads better. `ownedByCore` asks the live tier router what is
// already on it, so mounting inside the loop would make the first module's
// layers indistinguishable from core's — the second module claiming a taken
// prefix would be told it collided with core, naming the wrong culprit, and
// the module-versus-module check below it could never be reached.
for (const record of modules.values()) {
if (record.state !== 'registered') continue
try {
// Commit what this module staged. Collisions with core or with an earlier
// module surface here, in scan order, and cost only this module.
registries.apply(record.staged.staged)
} catch (err) {
record.state = 'startup_failed'
record.stage = 'register'
record.reason = err.message
log.error(`module "${record.id}" failed to register — continuing without it`, {
reason: err.message,
})
continue // unmounted, exactly like a validation failure in the first pass
}
mount(record, tierRouters)
}
}
/**
* Mount one module's routers onto the tier routers, behind the dispatch guard.
*
* The guard is the other half of §4.4. A module that fails BEFORE this point has
* no routes at all; one that fails after — schema replay (PR 3), `onBoot`
* (PR 5) — keeps its URLs and answers 503, so `routes.manifest.json` never
* depends on whether a boot hook happened to succeed on the machine that
* generated it. `disabled` is 404 and unreachable until PR 5 wires
* `installed_modules` in; it is written here because the guard is the contract's
* §4.5, not a later addition.
*/
function mount(record, tierRouters) {
for (const tier of TIERS) {
for (const [prefix, router] of record.routes[tier]) {
tierRouters[tier].use(prefix, stateGuard(record), router)
}
}
}
/**
* The dispatch guard, as a middleware over the LIVE record.
*
* A closure over the record rather than over its state: everything mounts once,
* at boot, and the states that matter here are reached afterwards — the schema
* replay fails, `onBoot` throws, an admin disables the module. A guard that read
* the state at mount time would answer for the state a module was in before any
* of that happened.
*
* Used for a module's API routes and, since PR 7, for its client chunk: a module
* answering 503 on its API must not also be handing the browser the script that
* calls it, and one an admin has disabled should be as absent from the page as it
* is from the nav.
*/
function stateGuard(record) {
return (req, res, next) => {
if (record.state === 'startup_failed') {
return res.status(503).json({ message: 'Module unavailable' })
}
if (record.state === 'disabled') return res.status(404).json({ message: 'Not found' })
return next()
}
}
// ── State ──────────────────────────────────────────────────────────────────
// The states a loaded record may hold, deliberately a hardcoded subset rather
// than an import of model/modules/modules.model.js's STATES: that model reaches
// the database, and this file must stay require-able against a dead one.
// `installed` is not here because a record leaves load() resolved either way.
const RECORD_STATES = new Set(['registered', 'started', 'disabled', 'startup_failed'])
/**
* Move a loaded module to a new state — the POST-mount transitions.
*
* Called by whoever ran the step that failed or the step that succeeded, because
* only they can know: `ensureSchema()` replays the fragments (PR 3), and
* lifecycle.js runs `onBoot` and reconciles `installed_modules` (whose `disabled`
* rows are what make the guard's 404 leg reachable).
*
* The stage travels with the reason and is cleared by every non-failing move,
* for the same reason the database columns are (§2.4): a running module must
* never be able to show a stale failure.
*
* Unknown ids are ignored rather than thrown on: a module can be absent from the
* volume and still have a row, and a caller on the boot path must not turn that
* into everyone's failure.
*/
function setState(id, state, { stage = null, reason = null } = {}) {
if (!RECORD_STATES.has(state)) throw new Error(`unknown module state "${state}"`)
const record = modules.get(id)
if (!record) return
record.state = state
record.stage = state === 'startup_failed' ? stage : null
record.reason = state === 'startup_failed' ? reason : null
}
// ── Introspection ──────────────────────────────────────────────────────────
function assertLoaded(caller) {
if (!loaded) throw new Error(`modules.${caller}() before modules.load()`)
}
/**
* Has load() run in this process?
*
* The one legitimate reason to ask instead of just calling an accessor: a
* process that never required app.js and so has no module list to be wrong
* about. `npm run seed` (db/seed.js) is exactly that — it calls ensureSchema()
* standalone, and the fragment replay has to be able to tell "this is the seed
* script" from "the server booted and something is mis-ordered", which is the
* distinction §7.6's throw exists to preserve everywhere else.
*/
const isLoaded = () => loaded
/**
* Every module found on the volume, loaded or failed, in scan order.
*
* Throws rather than returning `[]` when load() has not run — the empty list is
* a real answer for a core with no modules installed, and a caller cannot tell
* the two apart (§7.6).
*/
function list() {
assertLoaded('list')
return [...modules.values()].map((r) => ({
id: r.id,
name: r.manifest.name || r.id,
version: r.manifest.version,
state: r.state,
stage: r.stage,
reason: r.reason,
capabilities: r.manifest.capabilities || [],
}))
}
/**
* The modules that are ready to be booted, with their hook, in scan order.
*
* `registered` only — the state a module holds between a clean load and its
* `onBoot`. One that failed validation or schema replay is not going to run, and
* one already `started` has run. A module with no `onBoot` is still listed: it
* has nothing to warm up, but it still has to reach `started` so the admin panel
* and `installed_modules` agree with the guard about what is serving.
*
* @returns {{id: string, hook: Function|null, ctx: object|null}[]}
*/
function bootable() {
assertLoaded('bootable')
return [...modules.values()]
.filter((r) => r.state === 'registered')
.map((r) => ({ id: r.id, hook: r.hooks.onBoot, ctx: r.ctx }))
}
/**
* The shutdown hooks to run, in REVERSE registration order (§2.5).
*
* `started` only. A module whose `onBoot` threw is mid-way through a warm-up it
* never finished, and calling its `onShutdown` would hand it a half-built world
* to tear down — the one thing worse than not closing cleanly. Reverse order is
* the same reasoning applied between modules rather than within one.
*
* @returns {{id: string, hook: Function}[]}
*/
function shutdownHooks() {
assertLoaded('shutdownHooks')
return [...modules.values()]
.filter((r) => r.state === 'started' && r.hooks.onShutdown)
.map((r) => ({ id: r.id, hook: r.hooks.onShutdown }))
.reverse()
}
/**
* Every schema fragment waiting to be replayed, in scan order.
*
* `registered` only: a module that failed validation must not get its tables
* created (it is not going to run), and one already `started` has had them. The
* absolute path is resolved here rather than handed out as a manifest-relative
* name, so the replay never has to know how a module directory is laid out.
*
* @returns {{id: string, file: string}[]}
*/
function fragments() {
assertLoaded('fragments')
return [...modules.values()]
.filter((r) => r.state === 'registered' && r.manifest.schema)
.map((r) => ({ id: r.id, file: path.join(r.dir, r.manifest.schema) }))
}
/**
* Every module that ships a client chunk, with where to serve it from and the
* guard to serve it behind — in scan order.
*
* Listed regardless of state, because mounting happens once at boot and the
* guard is what answers for the state at request time (the same arrangement the
* API routes have). A module that failed VALIDATION never reaches here at all:
* `record.client` is only resolved once the manifest passed.
*
* `dir` is the directory the entry sits in, never the module root — see
* resolveClient. app.js does the mounting; this file does not know about the
* root app.
*
* @returns {{id: string, dir: string, url: string, entryUrl: string, guard: Function}[]}
*/
function clientChunks() {
assertLoaded('clientChunks')
return [...modules.values()]
.filter((r) => r.client)
.map((r) => ({ id: r.id, ...r.client, guard: stateGuard(r) }))
}
/**
* The script URLs the HTML shell should inject, in scan order.
*
* `started` only, and that is the difference between this and clientChunks():
* the mount is a standing offer answered by a guard, while the tag is a decision
* taken per page render, when the state is already known. A module whose `onBoot`
* failed keeps its URLs and answers 503 on them — loading its client half would
* render its pages against a backend that cannot serve them.
*
* @returns {string[]}
*/
function clientEntryUrls() {
assertLoaded('clientEntryUrls')
return [...modules.values()]
.filter((r) => r.client && r.state === 'started')
.map((r) => r.client.entryUrl)
}
/** Absolute path of the modules directory. */
const dir = () => MODULES_DIR
module.exports = {
load,
list,
setState,
fragments,
bootable,
shutdownHooks,
clientChunks,
clientEntryUrls,
isLoaded,
dir,
}

435
server/src/modules/registries.js Normal file
View File

@@ -0,0 +1,435 @@
// ── The de-entanglement registries ─────────────────────────────────────────
//
// Phase 2, PR 4 of docs/website/MODULE_SYSTEM.md §2.7 — the three seams §1.8 and
// §1.9 identified, where core code and game-specific content are tangled in one
// file and a folder move cannot separate them. The normative contract is
// docs/website/MODULE_API.md §2.4.
//
// The three:
//
// 1. `registerExtension(slot, router)` — §1.9. Module routes hanging off a
// CORE resource (`/admin/users/:id`), so all six shard sub-paths keep their
// URLs while core never learns what "shard" means.
// 2. `registerNotificationStreams(streams)` — §1.8. The push-stream catalog:
// push INFRASTRUCTURE is core, this CATALOG is content.
// 3. `registerAnnounceLeg({ leg, label, dispatch, classify })` — §1.8. The news
// dispatcher's delivery legs; Discord is core, town crier is content.
//
// **Core registers through these functions too, and is the only registrant until
// Phase 3.** `registerCore()` below is called explicitly from app.js before
// `modules.load()` — explicit, never lazy, the same decision the loader's trigger
// took (MODULE_API.md §7.6). Core going through the same door is the point: a
// registry only core's hardcoded base bypasses is a registry whose first real
// exercise is a module, which is the drift this PR exists to prevent.
//
// **Registering is validate-then-commit, per registrant.** `apply()` checks every
// claim in a batch before it writes any of them, so a module that registers two
// streams and then throws — or fails a later validation step in the loader — has
// left nothing behind. That is the registry-side twin of the loader's second-pass
// mount rule: nothing a module claims takes effect until the module as a whole is
// known good.
//
// Nothing here reaches the database or the network. It is a require-time-safe
// collection of what core and modules have declared, read at request time.
const express = require('express')
const log = require('../utils/logger')('modules')
// ── State ──────────────────────────────────────────────────────────────────
// slot → { router, filledBy }. `router` is created when CORE DECLARES the slot
// and mounted immediately; registrants `use()` into it later. That indirection is
// not optional: users.router.js is required while app.js is being built, long
// before any module has been scanned, so the thing core mounts has to be a stable
// object that can still be empty.
const slots = new Map()
// Registration order, which is display order in the app's notifications screen.
const streams = []
const streamOwners = new Map() // stream id → owner id, for the collision message
// leg id → { owner, leg, label, dispatch, classify }
const legs = new Map()
// owner -> { onSaved?, onDeleted? }. Post hooks (§1.8, API 1.1.0). A Map keyed by
// owner rather than a flat list, so a registrant is a single subscription that
// can be reported and reasoned about as one thing — and so registering twice is
// a collision with a name attached rather than a silently doubled side effect.
const postHooks = new Map()
let coreRegistered = false
// Stream ids that predate the module system and may not carry their owner's
// prefix — the exact counterpart of the loader's LEGACY_TABLE_PREFIXES, for the
// exact same reason. These seven ids are stored in `notification_subs` rows and
// are read by a shipped Android client; renaming them in Phase 3 would be a data
// migration and a client break, so `uo` keeps them and the prefix rule stays real
// for every module written after it.
const LEGACY_STREAM_IDS = {
uo: [
'server.status', 'idoc.warning', 'champ.start', 'governor.election',
'vendor.sale', 'house.idoc', 'account.login',
],
}
// Likewise for announce legs: `towncrier` is a stored value in
// announce_job_legs.leg and the body of the admin retry endpoint.
const LEGACY_LEGS = { uo: ['towncrier'] }
const STREAM_ID = /^[a-z][a-z0-9]*(\.[a-z][a-z0-9]*)+$/
const LEG_ID = /^[a-z][a-z0-9.]{1,62}$/
// A module's claim must carry its id. Core's ids are its own namespace, and the
// grandfathered names are the ones that predate all of this.
function namespaced(owner, name, legacy) {
return owner === 'core' || name.startsWith(`${owner}.`) || (legacy[owner] || []).includes(name)
}
// ── Extension slots (§1.9) ─────────────────────────────────────────────────
/**
* Core declares an extension slot and gets the router to mount for it.
*
* ONLY core may declare a slot; a module may only fill one (MODULE_API.md §2.4).
* That asymmetry is why this is not on the `api` object handed to a module.
*
* `mergeParams` so the slot's router sees the parent's `:id`. Core's own routes
* on the resource are declared before the slot is mounted, so first-match-wins
* gives core the path conflict, as the contract requires.
*
* @returns {import('express').Router} mount this at the resource, once.
*/
function declareSlot(slot) {
if (slots.has(slot)) throw new Error(`extension slot "${slot}" already declared`)
const router = express.Router({ mergeParams: true })
slots.set(slot, { router, filledBy: null })
return router
}
/** Does this slot exist? The loader asks, to validate `extensions` in a manifest. */
const hasSlot = (slot) => slots.has(slot)
/** Who filled a slot, or null. */
const slotFilledBy = (slot) => (slots.get(slot) || {}).filledBy || null
/**
* Every FILLED slot, for the OpenAPI build step (swagger/slotSpecs.js).
*
* `router` is the slot's own stable router — the object mounted on the resource —
* so the build can find it in the live express stack and recover the prefix it
* hangs at without a hardcoded table.
*/
const filledSlots = () =>
[...slots.entries()]
.filter(([, e]) => e.filledBy)
.map(([slot, e]) => ({ slot, filledBy: e.filledBy, router: e.router, specFile: e.specFile || null }))
/**
* A DECLARED slot's stable router, filled or not.
*
* `filledSlots()` answers what the build needs — a filled slot has a spec file
* to generate a fragment from. This answers what a test needs: the slot exists
* from the moment core declares it at require time, and its position in the
* express stack has to stay findable whether or not a module has filled it.
* Before Phase 3 the two questions had the same answer, because core filled the
* only slot itself.
*/
const declaredSlotRouter = (slot) => (slots.get(slot) || {}).router || null
// ── Notification streams (§1.8) ────────────────────────────────────────────
/** The whole catalog, core's entries first, in registration order. */
const allStreams = () => streams.slice()
/** Is this a stream anyone registered? Gates a subscription write. */
const isValidStream = (id) => streamOwners.has(id)
/** Ids of the owner-keyed streams — those needing a linked game account. */
const personalStreams = () => new Set(streams.filter((s) => s.personal).map((s) => s.id))
// ── Post hooks (§1.8) ──────────────────────────────────────────────────────
// Core's CMS is the only writer of posts, and a module may need to mirror one
// somewhere core knows nothing about — module-uo keeps UO's in-game Town Cryer
// News gump in step with it. Before this existed, core's post controller
// required `utils/newsGump` directly, which is precisely the coupling the
// extraction had to remove: core's publish path naming a UO file.
//
// It is deliberately NOT folded into `registerAnnounceLeg`, which fires on the
// same transition. A leg is a one-shot DELIVERY with retry and classification;
// a post hook maintains idempotent STATE, has to run on delete as well as save,
// and refreshes silently on an edit. Overloading the leg would have meant a
// dispatch that must not be retried and a classify that means nothing.
/** Every registered hook, in registration order. */
const postHookEntries = () => [...postHooks.entries()].map(([owner, h]) => ({ owner, ...h }))
/**
* Fire `event` at every registered hook, one at a time, never throwing.
*
* Best-effort by contract, and awaited rather than fired-and-forgotten: core's
* own call site awaited `newsGump.syncPost` before this existed, so a save that
* returns 200 still means the mirror was attempted. One subscriber's failure
* must not cost another's, and none of them may cost the save — a sidecar
* hiccup breaking a post edit would be a worse bug than a stale gump.
*/
async function dispatchPostHook(event, payload) {
for (const { owner, [event]: fn } of postHookEntries()) {
if (typeof fn !== 'function') continue
try {
await fn(payload)
} catch (err) {
log.warn('post hook failed', { owner, event, message: err.message })
}
}
}
// ── Announce legs (§1.8) ───────────────────────────────────────────────────
/** Every registered leg, in registration order. */
const announceLegs = () => [...legs.values()]
/** Just the ids — the enqueue order and the retry endpoint's allowlist. */
const announceLegIds = () => [...legs.keys()]
/** One leg, or null. */
const announceLeg = (leg) => legs.get(leg) || null
// ── Shape checks, run the moment a registrant calls ────────────────────────
//
// Split from the collision checks below on the same line PR 3 drew through
// schema-fragment validation: what can be decided from the argument alone is
// decided AT THE CALL, so the error carries the registrant's own stack. What
// depends on other registrants has to wait for the batch to be complete.
function checkStreamShape(entry) {
if (!entry || !STREAM_ID.test(entry.id || '')) {
throw new Error(`registerNotificationStreams: bad stream id "${entry && entry.id}"`)
}
if (!entry.label) throw new Error(`registerNotificationStreams: stream "${entry.id}" has no label`)
return {
id: entry.id,
label: entry.label,
description: entry.description || '',
personal: Boolean(entry.personal),
requiresLinkedAccount: Boolean(entry.requiresLinkedAccount),
}
}
function checkLegShape(entry) {
const { leg, label, dispatch, classify } = entry || {}
if (!LEG_ID.test(leg || '')) throw new Error(`registerAnnounceLeg: bad leg id "${leg}"`)
if (typeof dispatch !== 'function') throw new Error(`announce leg "${leg}" has no dispatch()`)
if (typeof classify !== 'function') throw new Error(`announce leg "${leg}" has no classify()`)
return { leg, label: label || leg, dispatch, classify }
}
/**
* `registerPostHook({ onSaved, onDeleted })` — both optional, at least one
* required. A registration with neither is a subscription that can never fire,
* which is a typo rather than an intention.
*/
function checkPostHookShape(entry) {
const { onSaved, onDeleted } = entry || {}
for (const [name, fn] of [['onSaved', onSaved], ['onDeleted', onDeleted]]) {
if (fn !== undefined && typeof fn !== 'function') {
throw new Error(`registerPostHook: ${name} must be a function`)
}
}
if (!onSaved && !onDeleted) throw new Error('registerPostHook: needs onSaved or onDeleted')
return { onSaved, onDeleted }
}
// `specFile` is CORE-ONLY and is not on the module-facing signature. A slot's
// router reaches the app through declareSlot(), which no static parse of app.js
// can follow, so swagger-autogen would silently drop every route in it — the
// spike's exact failure (MODULE_API.md §7.4). Core names the file so
// `npm run swagger` can generate a fragment from it and merge it into the
// committed spec. A MODULE has no equivalent need: it ships a prebuilt
// `swagger-fragment.json` in its bundle (§6.1a), because core never has its
// sources to analyse.
function checkExtensionShape(slot, router, specFile) {
if (!slots.has(slot)) throw new Error(`unknown extension slot "${slot}"`)
if (typeof router !== 'function') throw new Error(`registerExtension: ${slot} is not a router`)
return { slot, router, specFile: specFile || null }
}
// ── Staging + commit ───────────────────────────────────────────────────────
/**
* A registrant's staging area: shape-checked claims, not yet visible to anyone.
*
* The loader hands one of these to a module through `api`, and `registerCore()`
* builds one for core. Nothing a registrant says is readable through
* `allStreams()` / `announceLeg()` / the slot routers until `apply()`.
*/
function stage(owner) {
const staged = { owner, streams: [], legs: [], extensions: [], postHooks: [] }
return {
staged,
registerNotificationStreams(entries) {
if (!Array.isArray(entries)) throw new Error('registerNotificationStreams: expected an array')
for (const e of entries) staged.streams.push(checkStreamShape(e))
},
registerAnnounceLeg(entry) {
staged.legs.push(checkLegShape(entry))
},
registerExtension(slot, router, specFile) {
staged.extensions.push(checkExtensionShape(slot, router, specFile))
},
registerPostHook(entry) {
staged.postHooks.push(checkPostHookShape(entry))
},
}
}
/**
* Validate a staged batch against everything already registered, then commit it.
*
* Validation is TOTAL before the first write, so this either takes all of a
* registrant's claims or none of them. Throws on the first collision, naming who
* holds the thing already — which is the message an operator needs and the one
* PR 2 learned to protect (mounting inside the scan loop made every collision
* look like it was with core).
*/
function apply({ owner, streams: newStreams, legs: newLegs, extensions: newExtensions, postHooks: newPostHooks = [] }) {
// ── validate ──
const seenStreams = new Set()
for (const s of newStreams) {
const held = streamOwners.get(s.id)
if (held) throw new Error(`stream "${s.id}" is already registered by "${held}"`)
if (seenStreams.has(s.id)) throw new Error(`stream "${s.id}" registered twice`)
if (!namespaced(owner, s.id, LEGACY_STREAM_IDS)) {
throw new Error(`stream "${s.id}" is not namespaced "${owner}."`)
}
seenStreams.add(s.id)
}
const seenLegs = new Set()
for (const l of newLegs) {
const held = legs.get(l.leg)
if (held) throw new Error(`announce leg "${l.leg}" is already registered by "${held.owner}"`)
if (seenLegs.has(l.leg)) throw new Error(`announce leg "${l.leg}" registered twice`)
if (!namespaced(owner, l.leg, LEGACY_LEGS)) {
throw new Error(`announce leg "${l.leg}" is not namespaced "${owner}."`)
}
seenLegs.add(l.leg)
}
const seenSlots = new Set()
for (const x of newExtensions) {
const entry = slots.get(x.slot)
if (entry.filledBy) {
throw new Error(`extension slot "${x.slot}" is already filled by "${entry.filledBy}"`)
}
if (seenSlots.has(x.slot)) throw new Error(`extension slot "${x.slot}" filled twice`)
seenSlots.add(x.slot)
}
if (newPostHooks.length > 1) throw new Error(`"${owner}" registered more than one post hook`)
if (newPostHooks.length && postHooks.has(owner)) {
throw new Error(`"${owner}" already registered a post hook`)
}
// ── commit — nothing below can fail ──
for (const s of newStreams) {
streamOwners.set(s.id, owner)
streams.push(s)
}
for (const l of newLegs) legs.set(l.leg, { owner, ...l })
for (const x of newExtensions) {
const entry = slots.get(x.slot)
entry.filledBy = owner
entry.specFile = x.specFile
entry.router.use(x.router)
}
for (const h of newPostHooks) postHooks.set(owner, h)
}
// ── Core's own registrations ───────────────────────────────────────────────
/**
* Register everything CORE owns, through the same staging area a module uses.
*
* Called once from app.js, before `modules.load()` — before, because a module's
* collision checks are asked against what is already registered, and core's
* claims must be the ones already there.
*
* What is here is what survives Phase 3. Everything after the boundary comment is
* shard content and leaves with module-uo, registered rather than hardcoded so
* the seam is exercised on every boot long before a module first uses it.
*/
function registerCore() {
if (coreRegistered) return
/* eslint-disable global-require */
const coreStreams = require('../config/coreStreams')
const discordLeg = require('../utils/discordAnnounce')
/* eslint-enable global-require */
const api = stage('core')
api.registerNotificationStreams(coreStreams.STREAMS)
api.registerAnnounceLeg(discordLeg.leg)
// The three lines that used to follow — the shard stream catalog, the town
// crier leg and the `admin.users.detail` filling — were shard CONTENT held
// here so the seam would be exercised on every boot before a module first used
// it. Phase 3 moved them into module-uo's `register()` verbatim, with 'core'
// becoming 'uo', and nothing else in core changed. That was the claim PR 4
// made, and this deletion is it being collected.
apply(api.staged)
coreRegistered = true
log.info('core registrations complete', {
streams: streams.length,
announceLegs: legs.size,
extensions: [...slots.keys()].filter(slotFilledBy),
})
}
/** Has registerCore() run? Read by tests, and by the loader's ordering assertion. */
const isCoreRegistered = () => coreRegistered
// Test-only: hand the process back. Registries are process-global by design
// (there is one core), so a test that registers has to be able to undo it.
//
// Slot DECLARATIONS survive, and only their fills are cleared: a slot is declared
// at require time by the router that owns the resource, and that require has
// already happened and will not happen again in this process. Clearing the map
// would leave a slot that nothing can re-declare. The cost is that a test filling
// the same slot twice stacks two routers inside it; no test reads through a slot
// router, so that is left rather than papered over with a rebuilt router that
// would no longer be the object users.router.js mounted.
function _reset() {
for (const entry of slots.values()) {
entry.filledBy = null
entry.specFile = null
}
streams.length = 0
streamOwners.clear()
legs.clear()
postHooks.clear()
coreRegistered = false
}
module.exports = {
declareSlot,
hasSlot,
slotFilledBy,
filledSlots,
declaredSlotRouter,
allStreams,
isValidStream,
personalStreams,
announceLegs,
announceLegIds,
announceLeg,
postHookEntries,
dispatchPostHook,
stage,
apply,
registerCore,
isCoreRegistered,
_reset,
}

84
server/src/modules/schema.js Normal file
View File

@@ -0,0 +1,84 @@
// ── Module schema fragment replay ──────────────────────────────────────────
//
// Phase 2, PR 3 of docs/website/MODULE_SYSTEM.md §2.7. Normative contract:
// docs/website/MODULE_API.md §2.6 (fragments) and §4.4 (failure is a state).
//
// utils/db.js calls replayFragments() once, immediately after core's schema.sql
// is in place and before seedDefaults(), so that by the time a module's onBoot
// runs (PR 5) its tables exist.
//
// The split of responsibility with loader.js is worth stating, because it is the
// reason there are two files:
//
// loader.js VALIDATES a fragment — at load time, with no database, before
// anything is mounted. Every rule §2.6 states about the SQL is
// knowable by reading it, so a fragment that breaks one costs the
// module its mount entirely (§4.4, first column).
// schema.js EXECUTES it. Only reachable failures live here: the database
// rejecting a statement it could not have known was bad. Those are
// post-mount, so they 503 (§4.4, second column).
//
// The property this file exists to keep: **a fragment that fails takes down its
// own module and nothing else.** Not core's boot, not another module's tables.
const fs = require('fs')
const { splitStatements } = require('../utils/sqlStatements')
const log = require('../utils/logger')('modules')
/**
* Replay every installed module's schema fragment, in scan order.
*
* Never throws. A module whose fragment fails is moved to `startup_failed` with
* the database's own message as the reason, its routes answer 503 through the
* dispatch guard the loader already mounted, and the next module is replayed as
* if nothing happened.
*
* Partial application is accepted rather than compensated for: MariaDB commits
* each DDL statement implicitly, so a fragment failing at statement three has
* already created the first two tables and no wrapping transaction could undo
* them. Since every statement is required to be idempotent (§2.6), the fix is
* for the operator to correct the fragment and reboot — the surviving tables are
* re-CREATE-IF-NOT-EXISTSed harmlessly and the replay carries on past them.
*
* @param {object} [deps] injection seam for tests — the whole point of this
* function taking arguments at all, since the server suite runs with the pool
* pointed at a dead port.
* @param {(sql: string) => Promise<any>} [deps.query]
* @param {object} [deps.modules] the loader
*/
async function replayFragments({ query, modules } = {}) {
/* eslint-disable global-require */
const run = query || require('../utils/db').query
const loader = modules || require('./loader')
/* eslint-enable global-require */
// Not an error: `npm run seed` calls ensureSchema() without ever requiring
// app.js, so no scan has happened and there is genuinely nothing to replay.
// Logged rather than silently skipped — the one thing that must not happen is
// a booting server quietly getting no module tables (§7.6).
if (!loader.isLoaded()) {
log.info('no module scan in this process — skipping schema fragment replay')
return
}
for (const { id, file } of loader.fragments()) {
try {
const statements = splitStatements(fs.readFileSync(file, 'utf8'))
for (const statement of statements) {
// Serially, and awaited: a fragment's ALTER TABLE routinely depends on
// the CREATE TABLE above it.
await run(statement)
}
log.info(`schema ensured for module "${id}"`, { statements: statements.length })
} catch (err) {
loader.setState(id, 'startup_failed', { stage: 'schema', reason: err.message })
log.error(`module "${id}" schema fragment failed — its routes will answer 503`, {
reason: err.message,
})
}
}
}
module.exports = { replayFragments }

47
server/src/modules/semver.js Normal file
View File

@@ -0,0 +1,47 @@
// A deliberately tiny semver range check — enough for `coreApi` and no more.
//
// Supports `*`, an exact `x.y.z`, `^x.y.z` and `~x.y.z`. That is the whole
// grammar a module manifest is allowed to use (MODULE_API.md §1.1), so pulling
// in the `semver` package for it would add a dependency to the server for a
// twenty-line job. A range this parser does not understand is REJECTED rather
// than assumed to match — an unparseable range must not silently load a module
// against an API it was never tested on.
const PARTS = /^(\d+)\.(\d+)\.(\d+)$/
function parse(version) {
const m = PARTS.exec(String(version).trim())
if (!m) return null
return { major: Number(m[1]), minor: Number(m[2]), patch: Number(m[3]) }
}
const gte = (a, b) => {
if (a.major !== b.major) return a.major > b.major
if (a.minor !== b.minor) return a.minor > b.minor
return a.patch >= b.patch
}
/**
* Does `version` satisfy `range`?
* @param {string} version an exact x.y.z
* @param {string} range `*` | `x.y.z` | `^x.y.z` | `~x.y.z`
* @returns {boolean} false for anything unparseable, on either side
*/
function satisfies(version, range) {
const v = parse(version)
if (!v) return false
const raw = String(range).trim()
if (raw === '*') return true
const op = raw[0] === '^' || raw[0] === '~' ? raw[0] : ''
const b = parse(op ? raw.slice(1) : raw)
if (!b) return false
if (op === '') return v.major === b.major && v.minor === b.minor && v.patch === b.patch
if (!gte(v, b)) return false
// ^ allows minor+patch within the same major; ~ allows patch within the same minor.
if (op === '^') return v.major === b.major
return v.major === b.major && v.minor === b.minor
}
module.exports = { satisfies, parse }

26
server/src/modules/version.js Normal file
View File

@@ -0,0 +1,26 @@
// The module API version — the single number a module's `coreApi` range is
// checked against (docs/website/MODULE_API.md §1.1).
//
// Bump minor when a member is ADDED to ctx or a new register* call appears;
// major when one is removed, its signature changes, or its behaviour changes
// without a signature change. A core-internal refactor behind an unchanged
// member is not a bump.
//
// Deliberately separate from PROTOCOL_VERSION (which versions the shard wire and
// has nothing to say about a website module) and from any module's own version.
// 1.2.0 — the CLIENT registry gained `registerExtension` and core gained client
// extension slots (MODULE_API.md §3.7): the twin of this half's declareSlot /
// registerExtension, for module content inside a core *page* rather than under a
// core route prefix. Nothing on the server changed, and this file bumps anyway —
// the two halves state ONE version, because a module declares a single `coreApi`
// range and is served one chunk (client/src/modules/version.js).
//
// 1.1.0 — `ctx` gained `activity.log`, `users.getById` and `site.baseUrl`, each
// because module-uo's extraction needed it and none of them could be vendored:
// an admin action a module performs belongs in core's one audit log, the
// extension slot needs the user its prefix names, and §2.7 forbids a module
// reading core's `APP_BASE_URL` for itself. Additions only, so minor.
const MODULE_API_VERSION = '1.2.0'
module.exports = { MODULE_API_VERSION }

39
server/src/router/v1/admin/admin.controller.js
View File

@@ -7,8 +7,8 @@ const users = require('../../../model/users/users.model')
const activity = require('../../../model/activity/activity.model')
const trustedDevices = require('../../../model/trustedDevices/trustedDevices.model')
const recoveryCodes = require('../../../model/recoveryCodes/recoveryCodes.model')
const registries = require('../../../modules/registries')
const announceJobs = require('../../../model/announceJobs/announceJobs.model')
const newsGump = require('../../../utils/newsGump')
const pushDispatch = require('../../../utils/pushDispatch')
const { cleanBody } = require('../../../utils/sanitizeHtml')
const { parseJsonSetting } = require('../../../utils/settingsJson')
@@ -37,11 +37,13 @@ async function announceIfNewlyPublished(post, transition) {
// the single "newly published news" signal for the push too, so we never
// double-fire on edits or replicate the transition logic.
const jobId = await announceJobs.enqueueIfNeeded(post, transition)
// Keep the in-game Town Cryer News gump in sync with the same transition: push
// the article when it becomes published news, refresh it silently on an edit,
// and pull it when it leaves published-news. Best-effort (never throws), so a
// sidecar hiccup never breaks saving a post — same guarantee as the enqueue.
await newsGump.syncPost(post, transition)
// Tell whoever is listening that a post was saved, and what the transition
// was. Core's CMS is the only writer of posts, and a module may mirror one
// somewhere core knows nothing about — module-uo keeps UO's in-game Town Cryer
// News gump in step this way. Awaited but never throwing, so a subscriber's
// sidecar hiccup cannot break saving a post: the same guarantee the enqueue
// above gives.
await registries.dispatchPostHook('onSaved', { post, transition })
// Opt-in push tickle to news.post subscribers, on the same transition.
// Fire-and-forget + self-guarding, so a dead ntfy relay never breaks saving.
if (jobId) {
@@ -203,8 +205,11 @@ async function deletePost(req, res) {
const current = await posts.getById(id)
await posts.remove(id)
await activity.log({ req, action: 'post.delete', detail: { id } })
// If it was live in the News gump, pull it (best-effort).
if (newsGump.inGump(current)) await newsGump.removePost(id)
// And that it is gone. A subscriber decides for itself whether it was
// mirroring this one — core does not know, and asking would mean core
// holding a predicate that belongs to the subscriber (`inGump` used to live
// right here, and it was UO's question, not the CMS's).
await registries.dispatchPostHook('onDeleted', { post: current, id })
return res.json({ id })
} catch (err) {
return res.status(500).json({ message: 'Internal Server Error' })
@@ -728,6 +733,23 @@ async function listUsers(req, res) {
}
}
// GET /admin/users/:id — the sanitized user (so the detail page is refresh-safe).
//
// Lived in usersShard.controller.js until PR 4, purely because the detail page it
// backs is mostly shard panels — MODULE_SYSTEM.md §1.9 called that out as core
// semantics that ended up in the UO controller by proximity. Reading a user is
// core's, and it stays here when the shard panels leave.
async function getUser(req, res) {
try {
const user = await users.getById(Number(req.params.id))
if (!user) return res.status(404).json({ message: 'Not found' })
return res.json(user)
} catch (err) {
log.error('getUser', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
async function createUser(req, res) {
try {
if (await users.getRawByUsername(req.body.username)) {
@@ -938,6 +960,7 @@ module.exports = {
ASSET_RULES,
listActivity,
listUsers,
getUser,
createUser,
updateUser,
deleteUser,

16
server/src/router/v1/admin/index.js
View File

@@ -27,8 +27,6 @@ const postsRouter = require('./posts.router')
const uploadsRouter = require('./uploads.router')
const wikiRouter = require('./wiki.router')
const pagesRouter = require('./pages.router')
const shardRouter = require('./shard.router')
const uoLinkRouter = require('./uoLink.router')
const emailRouter = require('./email.router')
const discordBotRouter = require('./discordBot.router')
const settingsRouter = require('./settings.router')
@@ -64,12 +62,14 @@ adminRouter.use('/posts', postsRouter)
adminRouter.use('/uploads', uploadsRouter)
adminRouter.use('/wiki', wikiRouter)
adminRouter.use('/pages', pagesRouter)
// Ops and configuration. /shard mixes tiers on one prefix — self-service game
// account linking (no extra gate) alongside modAccess in-game staff ops — so
// one router owns the prefix and gates per route. The rest are admin-only.
// /admin/shard/pages is the in-game help-page queue, unrelated to /admin/pages.
adminRouter.use('/shard', shardRouter)
adminRouter.use('/uo-link', uoLinkRouter)
// Ops and configuration.
//
// `/shard` and `/uo-link` are absent here and are still served: they are
// module-uo's, mounted onto this same router by the loader after every core
// mount above (MODULE_API.md §2.4). The URLs did not move — the code did. That
// ordering is also what makes the prefixes unclaimable by anyone else: the
// loader asks this live router what core owns, so a second module claiming
// `/shard` is rejected against the mounts actually present, not against a list.
adminRouter.use('/email', emailRouter)
adminRouter.use('/discord-bot', discordBotRouter)
adminRouter.use('/settings', settingsRouter)

12
server/src/router/v1/admin/posts.router.js
View File

@@ -1,5 +1,5 @@
// Admin · Posts — news, five-on-friday, newsletter and screenshot posts, plus
// the announcement pipeline (town crier + Discord) status and retry.
// the announcement pipeline status and retry.
//
// Mounted at /api/v1/admin/posts by admin/index.js, which already applied
// `noindex, isLoggedIn, staffOnly`. No extra gate: managing content is the
@@ -13,6 +13,7 @@ const { body, param } = require('express-validator')
const ctrl = require('./admin.controller')
const { upload } = require('./imageUpload')
const validate = require('../../../middleware/validate')
const registries = require('../../../modules/registries')
const postsRouter = express.Router()
@@ -124,14 +125,17 @@ postsRouter.get(
postsRouter.post(
'/:id/announce/retry',
// #swagger.tags = ['Admin · Posts']
// #swagger.summary = 'Retry one announcement delivery leg (town crier or Discord)'
// #swagger.summary = 'Retry one announcement delivery leg'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'integer' }, description: 'Post id.' }
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", properties: { leg: { type: "string", enum: ["towncrier", "discord"] } }, required: ["leg"] } } } } */
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", properties: { leg: { type: "string", description: "A registered delivery leg id, as returned by GET /announce." } }, required: ["leg"] } } } } */
/* #swagger.responses[200] = { description: 'Updated announce job', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[404] = { description: 'No announcement job for this post', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
param('id').isInt(),
body('leg').isIn(['towncrier', 'discord']),
// The allowlist is the REGISTERED leg set, read per request rather than
// captured at require time: this file is required while app.js is being built,
// before registerCore() and modules.load() have run (MODULE_SYSTEM.md §1.8).
body('leg').custom((leg) => registries.announceLeg(leg) != null).withMessage('unknown announce leg'),
validate,
ctrl.retryAnnounceLeg,
)

386
server/src/router/v1/admin/shard.router.js
View File

@@ -1,386 +0,0 @@
// Admin · Shard — everything under /api/v1/admin/shard, in two tiers.
//
// Mounted at /api/v1/admin/shard by admin/index.js, which already applied
// `noindex, isLoggedIn, staffOnly`. Two capabilities share this prefix, and
// prefix ownership is the invariant the split preserves — so they share a file:
//
// 1. Self-service game-account linking (no extra gate). A staff member links
// and inspects their OWN in-game account exactly as a player does under
// /player/shard; the handlers are the very same `player/shard.controller`
// ones, keyed off req.user.id. These keep their `Admin · Account` swagger
// tag, which is why the tag disagrees with this filename.
// 2. Privileged live-shard operations and the help-page queue (`modAccess` —
// admin or moderator). `actor` is stamped server-side from the session in
// shardOps.controller.js; the request body never carries it.
//
// `modAccess` stays a per-route gate rather than a router-level `use`: it was
// per-route in admin.routes.js, and half the routes here must NOT have it.
//
// NOTE: /admin/shard/pages is the in-game help-page (support) queue. It is
// unrelated to /admin/pages, the CMS page builder.
const express = require('express')
const { body, param } = require('express-validator')
const shardOps = require('./shardOps.controller')
const shardVisibility = require('./shardVisibility.controller')
const shardAtlas = require('./shardAtlas.controller')
const shardClilocs = require('./shardClilocs.controller')
const selfShard = require('../player/shard.controller')
const { requireRole } = require('../../../utils/auth')
const validate = require('../../../middleware/validate')
const shardRouter = express.Router()
// Moderator gate. Admins can do everything a moderator can.
const modAccess = requireRole('admin', 'moderator')
// Admin-only gate, for settings that decide what the PUBLIC sees.
const adminOnly = requireRole('admin')
// ── Game account linking (self-service, any staff role) ───────────────
// Staff link their OWN in-game account here, exactly like players do under
// /player/shard. The controller keys off req.user.id, so the same handlers work.
const SHARD_ACCOUNT_RE = /^[A-Za-z0-9_.-]{1,120}$/
shardRouter.post(
'/link',
// #swagger.tags = ['Admin · Account']
// #swagger.summary = 'Link an in-game account with a one-time code (self)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { $ref: "#/components/schemas/ShardLinkRequest" } } } } */
/* #swagger.responses[200] = { description: 'Linked', content: { "application/json": { schema: { $ref: "#/components/schemas/ShardLinkResult" } } } } */
/* #swagger.responses[400] = { description: 'Unknown or expired code', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
body('code').isString().trim().isLength({ min: 4, max: 32 }),
validate,
selfShard.link,
)
shardRouter.get(
'/accounts',
// #swagger.tags = ['Admin · Account']
// #swagger.summary = 'List the caller’s linked game accounts (self)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Linked accounts', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/ShardLink" } } } } } */
selfShard.listAccounts,
)
shardRouter.get(
'/roster/:account',
// #swagger.tags = ['Admin · Account']
// #swagger.summary = 'Character roster for an account (self; admins: any account)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['account'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'A game account linked to the caller.' }
/* #swagger.responses[200] = { description: 'Account roster', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[403] = { description: 'Account not linked to the caller', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
param('account').matches(SHARD_ACCOUNT_RE),
validate,
selfShard.roster,
)
shardRouter.get(
'/vendors/:account',
// #swagger.tags = ['Admin · Account']
// #swagger.summary = 'Player vendors for an account (self; admins: any account)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['account'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'A game account linked to the caller.' }
/* #swagger.responses[200] = { description: 'Vendor snapshot', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[403] = { description: 'Account not linked to the caller', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
param('account').matches(SHARD_ACCOUNT_RE),
validate,
selfShard.vendors,
)
shardRouter.get(
'/char/:serial',
// #swagger.tags = ['Admin · Account']
// #swagger.summary = 'Character sheet (self-linked characters; admins: any character)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['serial'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Mobile serial, e.g. 0x24C.' }
/* #swagger.responses[200] = { description: 'Character profile', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[403] = { description: 'Character not on an account linked to the caller', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
param('serial').matches(/^0x[0-9a-fA-F]+$/),
validate,
selfShard.getChar,
)
shardRouter.get(
'/sales',
// #swagger.tags = ['Admin · Account']
// #swagger.summary = 'Recent player-vendor sales for the caller’s linked accounts (self)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Vendor sales', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/ShardVendorSale" } } } } } */
selfShard.getSales,
)
shardRouter.post(
'/account',
// #swagger.tags = ['Admin · Account']
// #swagger.summary = 'Create a game account and link it to the caller (staff self-service)'
// #swagger.description = 'Same as POST /player/shard/account but for a signed-in staff user — provisions a game account (own username + password) and links it. Gated by game_account_signup + the shard’s mode; the password is never stored or logged.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["account","password"], properties: { account: { type: "string" }, password: { type: "string" } } } } } */
/* #swagger.responses[201] = { description: 'Account created and linked', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[403] = { description: 'Game-account signup unavailable (site or shard)', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[409] = { description: 'Account name already taken', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
body('account').matches(/^[A-Za-z0-9][A-Za-z0-9_.-]{2,29}$/),
body('password').isString().isLength({ min: 8, max: 64 }),
validate,
selfShard.createGameAccount,
)
// ── In-game staff operations (uo-link write plane + support queue) ─────
// Privileged live-shard actions and the help-page queue, open to moderators as
// well as admins (modAccess). `actor` is stamped server-side from the session in
// the controller — the body never carries it. See shardOps.controller.js.
shardRouter.post(
'/kick',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Kick every live session of an account (admin/moderator)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", properties: { account: { type: "string" }, serial: { type: "string" } } } } } } */
/* #swagger.responses[200] = { description: 'Kicked', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[403] = { description: 'Protected target or write plane disabled', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
modAccess,
body('account').optional({ values: 'falsy' }).matches(SHARD_ACCOUNT_RE),
body('serial').optional({ values: 'falsy' }).matches(/^0x[0-9a-fA-F]+$/),
validate,
shardOps.kick,
)
shardRouter.post(
'/ban',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Ban an account, timed or indefinite (admin/moderator)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", properties: { account: { type: "string" }, serial: { type: "string" }, durationSec: { type: "integer" }, reason: { type: "string" } } } } } } */
/* #swagger.responses[200] = { description: 'Banned', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[403] = { description: 'Protected target or write plane disabled', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
modAccess,
body('account').optional({ values: 'falsy' }).matches(SHARD_ACCOUNT_RE),
body('serial').optional({ values: 'falsy' }).matches(/^0x[0-9a-fA-F]+$/),
body('durationSec').optional().isInt({ min: 0, max: 315360000 }),
body('reason').optional({ values: 'falsy' }).isString().trim().isLength({ max: 500 }),
validate,
shardOps.ban,
)
shardRouter.post(
'/unban',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Clear an account ban (admin/moderator)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", properties: { account: { type: "string" } }, required: ["account"] } } } } */
/* #swagger.responses[200] = { description: 'Unbanned', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
modAccess,
body('account').matches(SHARD_ACCOUNT_RE),
validate,
shardOps.unban,
)
shardRouter.post(
'/broadcast',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Broadcast a system message to everyone online (admin/moderator)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", properties: { text: { type: "string" }, hue: { type: "integer" } }, required: ["text"] } } } } */
/* #swagger.responses[200] = { description: 'Broadcast', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
modAccess,
body('text').isString().trim().isLength({ min: 1, max: 300 }),
body('hue').optional().isInt({ min: 0, max: 3000 }),
validate,
shardOps.broadcast,
)
shardRouter.get(
'/pages',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Open help-page (support) queue (admin/moderator)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Open pages', content: { "application/json": { schema: { type: "array", items: { type: "object", additionalProperties: true } } } } } */
modAccess,
shardOps.listPages,
)
shardRouter.post(
'/pages/:id/respond',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Reply to a help page, optionally closing it (admin/moderator)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Page id (sender serial).' }
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", properties: { message: { type: "string" }, close: { type: "boolean" } }, required: ["message"] } } } } */
/* #swagger.responses[200] = { description: 'Responded', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[404] = { description: 'Unknown page', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
modAccess,
param('id').matches(/^0x[0-9a-fA-F]+$/),
body('message').isString().trim().isLength({ min: 1, max: 500 }),
body('close').optional().isBoolean(),
validate,
shardOps.respondPage,
)
shardRouter.post(
'/pages/:id/close',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Resolve a help page without a reply (admin/moderator)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Page id (sender serial).' }
/* #swagger.responses[200] = { description: 'Closed', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
modAccess,
param('id').matches(/^0x[0-9a-fA-F]+$/),
validate,
shardOps.closePage,
)
shardRouter.get(
'/audit',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Recent in-game moderation audit events (admin/moderator)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'admin.audit events, newest first', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/ShardEvent" } } } } } */
modAccess,
shardOps.listAudit,
)
shardRouter.get(
'/houses',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Full house registry — owner, price, decay (admin/moderator)'
// #swagger.description = 'The complete house registry. The public endpoint shows only IDOC houses with location; this staff view carries owner/price/co-owner/decay detail.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Houses, ordered by name', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/ShardHouse" } } } } } */
modAccess,
shardOps.listHouses,
)
// ── Spawn atlas (admin only) ──────────────────────────────────────────
// Operating the atlas import. Admin-only rather than moderator: it reads a path
// on the server's filesystem and replaces every atlas table, which is closer to
// a deploy action than to moderation.
//
// These routes sit under /admin/shard even though the public ones deliberately
// do NOT sit under /public/shard. That is not an inconsistency: the public split
// says "this data does not come from the sidecar", while the admin panel is
// simply part of shard administration and belongs beside the rest of it.
shardRouter.get(
'/atlas',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Spawn atlas status: path, drift, counts, pending review (admin only)'
// #swagger.description = 'Where the ServUO tree is, whether it can be read, whether its source files have drifted from the loaded atlas, and any refresh staged for approval. The public /atlas/meta route reports the game world only; the filesystem detail is here.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Atlas status', content: { "application/json": { schema: { $ref: "#/components/schemas/AtlasStatus" } } } } */
/* #swagger.responses[403] = { description: 'Admin role required', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
shardAtlas.getStatus,
)
shardRouter.post(
'/atlas/import',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Re-import the spawn atlas from the ServUO tree (admin only)'
// #swagger.description = 'Applies a map change without a restart. `force` reimports even when the source hashes match what is loaded. A refresh that would REMOVE a facet is still staged for approval rather than applied — that decision is never taken implicitly. An unreadable tree answers 200 with status "unavailable" rather than 500: the refresh contract reports outcomes instead of throwing, and the admin needs to be told what is wrong with the path.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: false, content: { "application/json": { schema: { type: "object", properties: { force: { type: "boolean", description: "Reimport even if the tree is unchanged." } } } } } } */
/* #swagger.responses[200] = { description: 'What happened', content: { "application/json": { schema: { $ref: "#/components/schemas/AtlasRefreshResult" } } } } */
adminOnly,
body('force').optional().isBoolean(),
validate,
shardAtlas.importAtlas,
)
shardRouter.post(
'/atlas/approve',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Approve a staged atlas refresh that removes a facet (admin only)'
// #swagger.description = 'Re-parses the tree and applies it, facet loss included. Only the decision was stored, never the parsed world, so what lands matches the tree at approval time — an operator who has since fixed a half-copied mount gets the corrected import.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'What happened', content: { "application/json": { schema: { $ref: "#/components/schemas/AtlasRefreshResult" } } } } */
adminOnly,
shardAtlas.approve,
)
shardRouter.post(
'/atlas/reject',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Reject a staged atlas refresh (admin only)'
// #swagger.description = 'Keeps the current atlas and remembers the decision against those exact source hashes, so a declined refresh does not re-prompt on every restart. Changing the tree asks again.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Rejected', content: { "application/json": { schema: { $ref: "#/components/schemas/AtlasRefreshResult" } } } } */
/* #swagger.responses[404] = { description: 'Nothing is awaiting review', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
shardAtlas.reject,
)
shardRouter.put(
'/atlas/path',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Set the ServUO tree the atlas reads from (admin only)'
// #swagger.description = 'Persisted as a setting, which wins over the SERVUO_PATH deploy default so the mount can move without a redeploy. Blank clears it and the atlas is simply skipped on the next boot. Deliberately does not import as a side effect — the response carries the refreshed status so the panel can offer that as the next step.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["path"], properties: { path: { type: "string", description: "Absolute path to the ServUO server root. Blank disables the atlas." } } } } } } */
/* #swagger.responses[200] = { description: 'Atlas status after the change', content: { "application/json": { schema: { $ref: "#/components/schemas/AtlasStatus" } } } } */
adminOnly,
body('path').isString().isLength({ max: 512 }),
validate,
shardAtlas.setPath,
)
// ── Cliloc table (admin only) ─────────────────────────────────────────────
// UO's id → display-string map, converted once by the operator from their own
// client (docs/website/CLILOCS.md). Sits beside the atlas for the same reason:
// it is static content derived from operator-supplied files rather than anything
// the sidecar sends, and operating it is shard administration.
//
// There is deliberately NO public counterpart. The table is never served as a
// table — 123k rows would dwarf any page that used it, and the Android client
// consumes the same already-resolved JSON. Names are applied server-side to the
// responses that need them.
shardRouter.get(
'/clilocs',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Cliloc table status: sources, drift, entry count (admin only)'
// #swagger.description = 'Where the cliloc sources are, whether they can be read, how many entries are loaded, and whether the files on disk have drifted from them. The table is built from a SET of sources — the converted client table plus every operator-maintained overlay under `custom/`, which is how shard-added and shard-edited items get names. `missingSources` lists any source that was loaded before and is now gone; an import refuses that without `approve`. A shard with nothing configured is a supported state — item names simply render as ids.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Cliloc status', content: { "application/json": { schema: { $ref: "#/components/schemas/ClilocStatus" } } } } */
/* #swagger.responses[403] = { description: 'Admin role required', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
shardClilocs.getStatus,
)
shardRouter.post(
'/clilocs/import',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Re-import the cliloc table from its source files (admin only)'
// #swagger.description = 'Applies a client patch, or a change to the shard\'s own overlay files, without a restart. `force` reimports even when the source hashes match what is loaded. `approve` accepts a refresh in which a previously-loaded source has VANISHED — refused by default, because an unmounted volume and a deliberate deletion are indistinguishable from the server, and the wrong guess silently drops every name that file contributed. A missing path — or the common mistake of pointing at the client\'s own COMPRESSED Cliloc.enu — answers 200 with status "unavailable" and the reason, rather than 500: the refresh contract reports outcomes instead of throwing, and the admin needs to be told which file to convert.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: false, content: { "application/json": { schema: { type: "object", properties: { force: { type: "boolean", description: "Reimport even if the sources are unchanged." }, approve: { type: "boolean", description: "Accept a refresh in which a previously-loaded source has vanished." } } } } } } */
/* #swagger.responses[200] = { description: 'What happened', content: { "application/json": { schema: { $ref: "#/components/schemas/ClilocRefreshResult" } } } } */
adminOnly,
body('force').optional().isBoolean(),
body('approve').optional().isBoolean(),
validate,
shardClilocs.importClilocs,
)
shardRouter.put(
'/clilocs/path',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Set the cliloc source the site reads from (admin only)'
// #swagger.description = 'Accepts either the converted base file itself or a directory to search. Overlays are read from a `custom/` directory beside it either way — pointing at a file does not forfeit them. Persisted as a setting, which wins over the UO_CLIENT_PATH deploy default so the mount can move without a redeploy. Blank clears it and resolution is skipped on the next boot. Deliberately does not import as a side effect — the response carries the refreshed status so the panel can offer that as the next step.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["path"], properties: { path: { type: "string", description: "Path to the converted cliloc file, or a directory containing one. Blank disables resolution." } } } } } } */
/* #swagger.responses[200] = { description: 'Cliloc status after the change', content: { "application/json": { schema: { $ref: "#/components/schemas/ClilocStatus" } } } } */
adminOnly,
body('path').isString().isLength({ max: 512 }),
validate,
shardClilocs.setPath,
)
// ── Feature visibility (admin only) ───────────────────────────────────
// Who can see which shard surface, and which sensitive fields within it. This
// decides what ANONYMOUS visitors get, so it sits above the moderator tier.
shardRouter.get(
'/visibility',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Get per-feature shard visibility config (admin only)'
// #swagger.description = 'The effective config (compiled defaults merged with stored overrides) plus the vocabulary the admin UI renders from: the audience ladder and the always-locked fields. Defaults reproduce pre-v3 behavior.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Visibility config', content: { "application/json": { schema: { $ref: "#/components/schemas/ShardVisibilityConfig" } } } } */
/* #swagger.responses[403] = { description: 'Admin role required', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
shardVisibility.getVisibility,
)
shardRouter.put(
'/visibility',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Update per-feature shard visibility config (admin only)'
// #swagger.description = 'Patch one or more features. Unknown feature names, unknown rungs, and any attempt to configure a locked field (acct / webId — admin-only always) are rejected with 400 rather than silently dropped.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { $ref: "#/components/schemas/ShardVisibilityUpdate" } } } } */
/* #swagger.responses[200] = { description: 'Updated config', content: { "application/json": { schema: { $ref: "#/components/schemas/ShardVisibilityConfig" } } } } */
/* #swagger.responses[400] = { description: 'Unknown feature, rung, or a locked field', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
body('features').isObject(),
validate,
shardVisibility.putVisibility,
)
module.exports = shardRouter

117
server/src/router/v1/admin/shardAtlas.controller.js
View File

@@ -1,117 +0,0 @@
// ── Admin · Spawn atlas ────────────────────────────────────────────────────
//
// Operating the atlas import: where the ServUO tree is, whether it has drifted
// from what is loaded, and the approve/reject decision for a refresh that would
// remove a facet (docs/website/SPAWN_ATLAS.md).
//
// The policy lives in the model. This controller does three things and no more:
// it validates input, it maps a refresh RESULT onto an HTTP status, and it
// records the action in the admin activity log.
//
// **A refresh result is not an exception.** `shardAtlas.refresh()` reports
// `unavailable` / `failed` / `needsReview` rather than throwing, because the boot
// path must never be stopped by a bad tree. That contract is preserved here: an
// unreadable mount is a 200 carrying `status: 'unavailable'`, not a 500. The
// admin needs to be told what is wrong with their path, and a 500 says only
// "something broke".
const atlas = require('../../../model/shardAtlas/shardAtlas.model')
const activity = require('../../../model/activity/activity.model')
const log = require('../../../utils/logger')('admin-shard-atlas')
// GET /admin/shard/atlas — what is loaded, what the tree looks like, what is
// staged. Unlike the public /atlas/meta route this DOES carry the filesystem
// path and the drift flag: that is the whole point of the panel.
async function getStatus(req, res) {
try {
return res.json(await atlas.status())
} catch (err) {
log.error('getStatus', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/shard/atlas/import — apply a map change without a restart.
//
// `force` reimports even when the source hashes match what is loaded (the escape
// hatch for "the database is wrong but the tree is not"). Facet loss is still
// staged rather than applied — approving is a separate, explicit act.
async function importAtlas(req, res) {
try {
const force = !!req.body?.force
const result = await atlas.refresh({ force })
await activity.log({
req,
action: 'shard.atlas.import',
detail: { force, status: result.status, counts: result.counts ?? null },
})
return res.json(result)
} catch (err) {
log.error('importAtlas', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/shard/atlas/approve — apply a staged refresh, facet loss and all.
//
// Re-parses the tree rather than applying something captured at boot: only the
// DECISION was stored, so what lands matches the tree as it is now. If the
// operator has since fixed a half-copied mount, the approved import is simply
// the corrected one — which is the desired outcome, not a surprise.
async function approve(req, res) {
try {
const result = await atlas.approvePending()
await activity.log({
req,
action: 'shard.atlas.approve',
detail: { status: result.status, removed: result.removedFacets ?? null },
})
return res.json(result)
} catch (err) {
log.error('approveAtlas', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/shard/atlas/reject — keep the current atlas and remember the
// decision against those exact source hashes, so a declined refresh does not
// re-prompt on every restart. Changing the tree asks again.
async function reject(req, res) {
try {
const result = await atlas.rejectPending()
if (result.status === 'none') {
return res.status(404).json({ message: 'No refresh is awaiting review.' })
}
await activity.log({ req, action: 'shard.atlas.reject', detail: {} })
return res.json(result)
} catch (err) {
log.error('rejectAtlas', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// PUT /admin/shard/atlas/path — point the atlas at a different ServUO tree.
//
// Persisted as a setting, which wins over the SERVUO_PATH env default so an
// operator can move the mount without a redeploy. Blank clears it, which turns
// the atlas off (boot skips, the loaded atlas keeps serving) — that is a
// legitimate thing to want, so it is allowed rather than validated away.
//
// Deliberately does NOT import as a side effect: changing where the atlas reads
// from and reloading it are separate decisions, and an operator fixing a typo
// should not have a multi-thousand-row replace happen under them. The response
// carries the refreshed status so the panel can offer the import immediately.
async function setPath(req, res) {
try {
const value = String(req.body?.path ?? '').trim()
await atlas.setServuoPath(value, req.user?.id ?? null)
await activity.log({ req, action: 'shard.atlas.path', detail: { path: value } })
return res.json(await atlas.status())
} catch (err) {
log.error('setAtlasPath', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
module.exports = { getStatus, importAtlas, approve, reject, setPath }

106
server/src/router/v1/admin/shardClilocs.controller.js
View File

@@ -1,106 +0,0 @@
// ── Admin · Cliloc table ───────────────────────────────────────────────────
//
// Operating the cliloc import: where the converted cliloc file is, whether it
// has drifted from what is loaded, and a forced reimport after a client patch
// (docs/website/CLILOCS.md).
//
// The policy lives in the model. This controller does three things and no more:
// it validates input, it maps a refresh RESULT onto an HTTP status, and it
// records the action in the admin activity log.
//
// **A refresh result is not an exception.** `shardClilocs.refresh()` reports
// `unavailable` / `failed` rather than throwing, because the boot path must never
// be stopped by a bad file. That contract is preserved here: a missing file, or
// the single most likely operator mistake — pointing at the client's own
// COMPRESSED `Cliloc.enu` — is a 200 carrying `status: 'unavailable'` and the
// reason, not a 500. A 500 would say only "something broke"; the operator needs
// to be told which file to convert.
const clilocs = require('../../../model/shardClilocs/shardClilocs.model')
const market = require('../../../model/shardMarket/shardMarket.model')
const activity = require('../../../model/activity/activity.model')
const log = require('../../../utils/logger')('admin-shard-clilocs')
// GET /admin/shard/clilocs — what is loaded, what the file looks like, whether
// they disagree. There is no public counterpart: the cliloc table is never
// served as a table, only applied to names the site already returns.
async function getStatus(req, res) {
try {
return res.json(await clilocs.status())
} catch (err) {
log.error('getStatus', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/shard/clilocs/import — reload after a client patch or a change to
// the shard's own overlay files, without a restart.
//
// `force` reimports even when the source hashes match what is loaded (the escape
// hatch for "the database is wrong but the files are not").
//
// `approve` accepts a refresh in which a previously-loaded source has VANISHED.
// That is refused by default because an unmounted volume and a deliberate
// deletion look identical from the server — the lighter cousin of the atlas's
// approve/reject flow, and the reason it can be a flag here rather than a
// pending table is that nothing is stored to approve: the import re-reads the
// files at approval time by construction.
async function importClilocs(req, res) {
try {
const force = !!req.body?.force
const approve = !!req.body?.approve
const result = await clilocs.refresh({ force, approve })
// The marketplace denormalizes resolved item names into
// shard_vendor_items.display_name, and the shard's market sweep will NOT
// re-send an unchanged shop just because the site learned what its items are
// called — so without this pass, an operator who imports clilocs after the
// first sweep keeps seeing item ids until every shop happens to change.
// Awaited (rather than fired and forgotten) so the panel's "imported" is
// honest about the names being live; the pass is a bounded walk of one table
// and never throws.
if (result.status === 'imported') await market.refreshDisplayNames()
await activity.log({
req,
action: 'shard.clilocs.import',
detail: {
force,
approve,
status: result.status,
count: result.count ?? null,
missingSources: result.missingSources ?? result.acceptedMissing ?? null,
},
})
return res.json(result)
} catch (err) {
log.error('importClilocs', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// PUT /admin/shard/clilocs/path — point the site at a different cliloc file.
//
// Persisted as a setting, which wins over the UO_CLIENT_PATH env default so an
// operator can move the mount without a redeploy. Blank clears it, which turns
// resolution off (boot skips, the loaded table keeps serving) — a legitimate
// thing to want, so it is allowed rather than validated away.
//
// Deliberately does NOT import as a side effect, for the same reason the atlas
// path does not: changing where the table reads from and reloading it are
// separate decisions. The response carries the refreshed status so the panel can
// offer the import immediately.
async function setPath(req, res) {
try {
const value = String(req.body?.path ?? '').trim()
await clilocs.setClientPath(value, req.user?.id ?? null)
await activity.log({ req, action: 'shard.clilocs.path', detail: { path: value } })
return res.json(await clilocs.status())
} catch (err) {
log.error('setClilocPath', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
module.exports = { getStatus, importClilocs, setPath }

171
server/src/router/v1/admin/shardOps.controller.js
View File

@@ -1,171 +0,0 @@
// ── Admin: in-game staff operations (uo-link write plane + support queue) ────
//
// The privileged "write plane" (§6 of the sidecar guide): kick / ban / unban /
// broadcast against the live shard, plus the help-page (support ticket) queue.
// Gated admin+moderator at the route (modAccess) — the sidecar trusts the
// loopback socket, so authorization is entirely the site's responsibility.
//
// SECURITY: `actor` (who is taking the action) is ALWAYS set here from the
// authenticated session (req.user.username), never from the request body, so an
// action can't be attributed to someone else. The shard records it in its console
// log, the ban's BanDealer tag, and the admin.audit event it echoes back.
const uoLinkClient = require('../../../utils/uoLinkClient')
const shardState = require('../../../model/shardState/shardState.model')
const shardEvents = require('../../../model/shardEvents/shardEvents.model')
const activity = require('../../../model/activity/activity.model')
const log = require('../../../utils/logger')('admin-shard-ops')
// Map a never-throw uoLinkClient result onto an HTTP response. `okData` shapes the
// success body. Mirrors the sidecar's documented status codes so the UI can tell a
// transient outage (503/504 — retry) from a real rejection (403/404).
function relay(res, result, okData) {
if (result.ok) return res.json(okData(result.data))
switch (result.status) {
case 400:
return res.status(400).json({ message: (result.data && result.data.error) || 'The shard rejected that request.' })
case 403:
return res.status(403).json({
message:
(result.data && result.data.error) ||
'That action was refused — the target is protected, or the write plane is disabled on the shard.',
})
case 404:
return res.status(404).json({ message: 'No such account or target on the shard.' })
case 503:
case 504:
case 0:
return res.status(503).json({ message: 'The shard is unavailable right now — try again shortly.' })
default:
return res.status(502).json({ message: 'Could not reach the shard.' })
}
}
// POST /admin/shard/kick — disconnect every live session of an account (or serial).
async function kick(req, res) {
const { account, serial } = req.body
const actor = req.user.username
try {
const result = await uoLinkClient.adminKick({ actor, account, serial })
if (result.ok) await activity.log({ req, action: 'shard.kick', detail: { account, serial } })
return relay(res, result, (d) => d || { ok: true })
} catch (err) {
log.error('shardOps.kick', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/shard/ban — ban an account (works offline); durationSec 0/absent = indefinite.
async function ban(req, res) {
const { account, serial, durationSec, reason } = req.body
const actor = req.user.username
try {
const result = await uoLinkClient.adminBan({ actor, account, serial, durationSec, reason })
if (result.ok) await activity.log({ req, action: 'shard.ban', detail: { account, serial, durationSec, reason } })
return relay(res, result, (d) => d || { ok: true })
} catch (err) {
log.error('shardOps.ban', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/shard/unban — clear an account's ban.
async function unban(req, res) {
const { account } = req.body
const actor = req.user.username
try {
const result = await uoLinkClient.adminUnban({ actor, account })
if (result.ok) await activity.log({ req, action: 'shard.unban', detail: { account } })
return relay(res, result, (d) => d || { ok: true })
} catch (err) {
log.error('shardOps.unban', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/shard/broadcast — a system message to everyone online.
async function broadcast(req, res) {
const { text, hue } = req.body
const actor = req.user.username
try {
const result = await uoLinkClient.adminBroadcast({ actor, text, hue })
if (result.ok) await activity.log({ req, action: 'shard.broadcast', detail: { text } })
return relay(res, result, (d) => d || { ok: true })
} catch (err) {
log.error('shardOps.broadcast', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /admin/shard/pages — the open help-page (support) queue, from our store.
async function listPages(req, res) {
try {
return res.json(await shardState.listPages())
} catch (err) {
log.error('shardOps.listPages', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/shard/pages/:id/respond — reply to a player (optionally close).
async function respondPage(req, res) {
const { id } = req.params
const { message, close } = req.body
try {
const result = await uoLinkClient.respondPage(id, { message, close: Boolean(close) })
if (result.ok) {
await activity.log({ req, action: 'shard.page.respond', detail: { pageId: id, close: Boolean(close) } })
// Close removes the page from the queue; reflect it locally at once (the
// page.closed event will confirm it, but the UI shouldn't wait a poll cycle).
if (close) await shardState.removePage(id).catch(() => {})
}
return relay(res, result, (d) => d || { ok: true })
} catch (err) {
log.error('shardOps.respondPage', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/shard/pages/:id/close — resolve a page without a reply.
async function closePage(req, res) {
const { id } = req.params
try {
const result = await uoLinkClient.closePage(id)
if (result.ok) {
await activity.log({ req, action: 'shard.page.close', detail: { pageId: id } })
await shardState.removePage(id).catch(() => {})
}
return relay(res, result, (d) => d || { ok: true })
} catch (err) {
log.error('shardOps.closePage', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /admin/shard/audit — recent moderation audit events (admin.audit), from the
// ingested event log. Seeds the live audit log the panel keeps current over SSE.
async function listAudit(req, res) {
try {
const limit = req.query.limit
return res.json(await shardEvents.list({ kind: 'admin.audit', limit }))
} catch (err) {
log.error('shardOps.listAudit', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /admin/shard/houses — the FULL house registry (owner, price, co-owners,
// decay), staff-only (modAccess). The public /public/shard/houses shows only IDOC
// houses with location; this is the complete board, kept live for staff on the
// admin SSE channel (house.update / house.remove).
async function listHouses(req, res) {
try {
return res.json(await shardState.listHouses())
} catch (err) {
log.error('shardOps.listHouses', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
module.exports = { kick, ban, unban, broadcast, listPages, respondPage, closePage, listAudit, listHouses }

98
server/src/router/v1/admin/shardVisibility.controller.js
View File

@@ -1,98 +0,0 @@
// ── Admin · Shard visibility ───────────────────────────────────────────────
//
// Read/write the per-feature audience config that gates every shard-derived
// surface. Admin-only: this decides what anonymous visitors can see, so it is
// not part of the moderator tier.
//
// The policy itself (the ladder, the feature catalog, which fields are locked)
// lives in utils/shardVisibility.js. This controller only validates input
// against that policy and persists it.
const model = require('../../../model/shardVisibility/shardVisibility.model')
const visibility = require('../../../utils/shardVisibility')
const log = require('../../../utils/logger')('admin-shard-visibility')
// GET /admin/shard/visibility — the effective config (defaults merged with any
// stored overrides), plus the vocabulary the admin UI needs to render itself:
// the ladder, and which fields each feature exposes as configurable.
async function getVisibility(req, res) {
try {
const config = await visibility.getConfig()
return res.json({
ladder: visibility.LADDER,
lockedFields: Object.keys(visibility.LOCKED_FIELDS),
defaults: visibility.compileDefaults(),
features: config,
})
} catch (err) {
log.error('getVisibility', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// PUT /admin/shard/visibility — replace the settings for one or more features.
// Body: { features: { <name>: { enabled, audience, stream, fieldRules } } }
//
// Rejects unknown feature names, unknown rungs, and any attempt to configure a
// locked field — a 400 rather than a silent drop, so an admin who tries to make
// `acct` public learns that it is not negotiable.
async function putVisibility(req, res) {
try {
const incoming = req.body?.features
if (!incoming || typeof incoming !== 'object' || Array.isArray(incoming)) {
return res.status(400).json({ message: 'features object required' })
}
const entries = []
for (const [name, patch] of Object.entries(incoming)) {
if (!visibility.isFeature(name)) {
return res.status(400).json({ message: `Unknown feature: ${name}` })
}
if (!patch || typeof patch !== 'object' || Array.isArray(patch)) {
return res.status(400).json({ message: `Invalid settings for ${name}` })
}
if (patch.audience != null && !visibility.isLevel(patch.audience)) {
return res.status(400).json({ message: `Unknown audience for ${name}: ${patch.audience}` })
}
const fieldRules = {}
for (const [field, level] of Object.entries(patch.fieldRules || {})) {
// Matches flattened spellings too (`ownerAcct`, `leaderWebId`), so the
// rejection covers every way the field can be named rather than the two
// canonical keys.
if (visibility.isLockedField(field)) {
return res.status(400).json({ message: `Field '${field}' is admin-only and cannot be configured` })
}
if (!visibility.isLevel(level)) {
return res.status(400).json({ message: `Unknown rung for ${name}.${field}: ${level}` })
}
fieldRules[field] = level
}
const current = (await visibility.getConfig())[name]
entries.push({
feature: name,
enabled: patch.enabled == null ? current.enabled : !!patch.enabled,
audience: patch.audience ?? current.audience,
stream: patch.stream == null ? current.stream : !!patch.stream,
fieldRules,
updatedBy: req.user?.id ?? null,
})
}
for (const entry of entries) await model.upsert(entry)
visibility.invalidate()
log.info('shard visibility updated', {
by: req.user?.id,
features: entries.map((e) => e.feature),
})
return res.json({ features: await visibility.getConfig() })
} catch (err) {
log.error('putVisibility', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
module.exports = { getVisibility, putVisibility }

123
server/src/router/v1/admin/uoLink.controller.js
View File

@@ -1,123 +0,0 @@
// ── Admin: uo-link sidecar control ─────────────────────────────────────────
//
// Configure the connection to the uo-link sidecar (base/ws URL, shared-secret
// token, protocol pin, enabled) and drive the town crier. SECURITY: the token
// is write-only over this API — stored encrypted, NEVER returned; responses
// expose only `hasToken` (same convention as the Discord bot token). Saving
// (re)starts the WS ingest client so a change takes effect with no redeploy.
const uoLinkConfig = require('../../../model/uoLinkConfig/uoLinkConfig.model')
const uoLinkClient = require('../../../utils/uoLinkClient')
const uoLinkSocket = require('../../../utils/uoLinkSocket')
const shardBroadcast = require('../../../utils/shardBroadcast')
const activity = require('../../../model/activity/activity.model')
const log = require('../../../utils/logger')('admin-uolink')
// Assemble the masked config + live health + ingestion stats for the panel.
async function buildStatus() {
const config = await uoLinkConfig.getSafe()
const health = await uoLinkClient.health()
return {
...config,
health: health.ok ? health.data : { ok: false, error: health.error || `status ${health.status}` },
ingest: uoLinkSocket.getState(),
sse: shardBroadcast.stats(),
}
}
// GET /admin/uo-link/config — masked config + live status + ingestion stats.
async function getConfig(req, res) {
try {
return res.json(await buildStatus())
} catch (err) {
log.error('uoLink.getConfig', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// PUT /admin/uo-link/config — save connection settings + (re)start the socket.
async function saveConfig(req, res) {
const { baseUrl, wsUrl, token, protocol, enabled } = req.body
try {
const current = await uoLinkConfig.getSafe()
const willHaveToken = Boolean(token) || current.hasToken
if (enabled && !willHaveToken) {
return res.status(400).json({ message: 'An auth token is required before enabling.' })
}
await uoLinkConfig.save({
baseUrl,
wsUrl,
token,
protocol: protocol !== undefined ? Number(protocol) : undefined,
enabled,
updatedBy: req.user.id,
})
// Drop the client's cached config so the health check below uses the new values.
uoLinkClient.invalidateConfig()
// (Re)start or stop the ingest socket to match the new enabled/URL/token.
const saved = await uoLinkConfig.getSafe()
if (saved.enabled && saved.hasToken) {
await uoLinkSocket.start()
} else {
uoLinkSocket.stop()
await uoLinkConfig.recordStatus({ status: 'disconnected', pluginConnected: false })
}
await activity.log({ req, action: 'uoLink.config.update', detail: { baseUrl: saved.baseUrl, enabled: saved.enabled } })
log.info('uo-link config updated', { by: req.user.username, enabled: saved.enabled })
return res.json(await buildStatus())
} catch (err) {
log.error('uoLink.saveConfig', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/uo-link/towncrier — publish/replace a town-crier message.
async function postTownCrier(req, res) {
const { id, lines, durationSec } = req.body
try {
const result = await uoLinkClient.postTownCrier({ id, lines, durationSec })
if (result.ok) {
await activity.log({ req, action: 'uoLink.towncrier.post', detail: { id } })
return res.json(result.data || { ok: true, id })
}
if (result.status === 400) return res.status(400).json({ message: 'The shard rejected that message (over the line/duration caps?).' })
if (result.status === 503 || result.status === 0) {
return res.status(503).json({ message: 'The shard is unavailable right now.' })
}
return res.status(502).json({ message: 'Could not reach the shard.' })
} catch (err) {
log.error('uoLink.postTownCrier', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// DELETE /admin/uo-link/towncrier/:id — remove a town-crier message.
async function deleteTownCrier(req, res) {
const { id } = req.params
try {
const result = await uoLinkClient.deleteTownCrier(id)
if (result.ok) {
await activity.log({ req, action: 'uoLink.towncrier.delete', detail: { id } })
return res.json(result.data || { ok: true, id })
}
if (result.status === 404) return res.status(404).json({ message: 'No town-crier message with that id.' })
if (result.status === 503 || result.status === 0) {
return res.status(503).json({ message: 'The shard is unavailable right now.' })
}
return res.status(502).json({ message: 'Could not reach the shard.' })
} catch (err) {
log.error('uoLink.deleteTownCrier', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /admin/uo-link/stream — the full live feed (incl. audit/cheat), staff only.
function stream(req, res) {
shardBroadcast.subscribe(req, res, 'admin')
}
module.exports = { getConfig, saveConfig, postTownCrier, deleteTownCrier, stream }

99
server/src/router/v1/admin/uoLink.router.js
View File

@@ -1,99 +0,0 @@
// Admin · uo-link — the sidecar connection config, the town crier, and the
// staff SSE stream.
//
// Mounted at /api/v1/admin/uo-link by admin/index.js, which already applied
// `noindex, isLoggedIn, staffOnly`. This is where shard integration is
// configured: base/ws URL, bearer token, protocol version and the enabled
// toggle all live in the DB (uoLinkConfig), never in env. The token is
// write-only over this API (SECURITY note in uoLink.controller.js).
//
// /stream is the ADMIN SSE channel — it carries staff audit, cheat detection
// and login attempts on top of the public event kinds. The public/admin
// allowlist split in utils/shardIngest.js is a security boundary; the adminOnly
// gate below is its other half.
//
// The routes keep their `Admin · Shard` swagger tag: retagging is a real
// OpenAPI diff and does not belong in a route-move PR.
//
// Admin-only, and kept as a per-route gate rather than a router-level `use` so
// the middleware chain each route carries is unchanged by the move.
const express = require('express')
const { body, param } = require('express-validator')
const uoLink = require('./uoLink.controller')
const { requireRole } = require('../../../utils/auth')
const validate = require('../../../middleware/validate')
const uoLinkRouter = express.Router()
const adminOnly = requireRole('admin')
uoLinkRouter.get(
'/config',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Get uo-link config + live status + ingestion stats (admin only)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Masked config, health and ingestion stats', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[403] = { description: 'Admin role required', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
uoLink.getConfig,
)
uoLinkRouter.put(
'/config',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Save uo-link connection config (admin only)'
// #swagger.description = 'token is write-only — omit/blank it to keep the existing one. Saving (re)starts the WS ingest client.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", properties: { baseUrl: { type: "string" }, wsUrl: { type: "string" }, token: { type: "string" }, protocol: { type: "integer" }, enabled: { type: "boolean" } } } } } } */
/* #swagger.responses[200] = { description: 'Updated config + live status', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[400] = { description: 'Validation error, or missing token while enabling', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[403] = { description: 'Admin role required', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
body('baseUrl').optional({ values: 'falsy' }).isString().trim().isURL({ require_tld: false, protocols: ['http', 'https'] }),
body('wsUrl').optional({ values: 'falsy' }).isString().trim().isURL({ require_tld: false, protocols: ['ws', 'wss'] }),
body('token').optional({ values: 'falsy' }).isString().trim(),
body('protocol').optional().isInt({ min: 1, max: 99 }),
body('enabled').optional().isBoolean(),
validate,
uoLink.saveConfig,
)
uoLinkRouter.post(
'/towncrier',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Publish / replace a town-crier message (admin only)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { $ref: "#/components/schemas/TownCrierRequest" } } } } */
/* #swagger.responses[200] = { description: 'Posted', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[400] = { description: 'Rejected (over caps)', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[503] = { description: 'Shard unavailable', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
body('id').isString().trim().isLength({ min: 1, max: 64 }),
body('lines').isArray({ min: 1, max: 8 }),
body('lines.*').isString().isLength({ max: 200 }),
body('durationSec').optional().isInt({ min: 1, max: 86400 }),
validate,
uoLink.postTownCrier,
)
uoLinkRouter.delete(
'/towncrier/:id',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Remove a town-crier message (admin only)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Town-crier message id.' }
/* #swagger.responses[200] = { description: 'Removed', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[404] = { description: 'Unknown id', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
param('id').isString().trim().isLength({ min: 1, max: 64 }),
validate,
uoLink.deleteTownCrier,
)
uoLinkRouter.get(
'/stream',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Full live shard event stream incl. audit/cheat (SSE, admin only)'
/* #swagger.responses[200] = { description: 'An SSE stream (Content-Type: text/event-stream).' } */
adminOnly,
uoLink.stream,
)
module.exports = uoLinkRouter

108
server/src/router/v1/admin/users.router.js
View File

@@ -4,20 +4,18 @@
// `noindex, isLoggedIn, staffOnly`. The whole capability is admin-only: editors
// and moderators manage content and reports, never accounts.
//
// Handlers still live in admin.controller.js (users) and usersShard.controller.js
// (uo-link footprint); this PR re-wires routes, not logic.
// Handlers live in admin.controller.js. The shard footprint that used to be
// wired here is now an EXTENSION SLOT (MODULE_SYSTEM.md §1.9) — see the bottom of
// this file.
const express = require('express')
const { body, param } = require('express-validator')
const ctrl = require('./admin.controller')
const usersShard = require('./usersShard.controller')
const registries = require('../../../modules/registries')
const { requireRole } = require('../../../utils/auth')
const validate = require('../../../middleware/validate')
// Same shape the shard routes validate account names with.
const SHARD_ACCOUNT_RE = /^[A-Za-z0-9_.-]{1,120}$/
const usersRouter = express.Router()
const adminOnly = requireRole('admin')
@@ -151,11 +149,6 @@ usersRouter.post(
ctrl.resetUserMfa,
)
// ── User → shard (uo-link) footprint (admin only) ─────────────────────
// Backs the /admin/users/:id detail page: a user's linked game accounts and,
// scoped to those accounts, their vendor sales / houses / online characters.
// Live character rosters are fetched by the client through /admin/shard/* (which
// already grants admins a bypass to any account), so no routes for them here.
usersRouter.get(
'/:id',
// #swagger.tags = ['Admin · Users']
@@ -166,84 +159,21 @@ usersRouter.get(
/* #swagger.responses[404] = { description: 'Not found', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
param('id').isInt(),
validate,
usersShard.getUser,
)
usersRouter.get(
'/:id/shard/accounts',
// #swagger.tags = ['Admin · Users']
// #swagger.summary = 'A user’s linked game accounts (admin only)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'integer' }, description: 'User id.' }
/* #swagger.responses[200] = { description: 'Linked accounts', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/ShardLink" } } } } } */
/* #swagger.responses[404] = { description: 'Not found', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
param('id').isInt(),
validate,
usersShard.listAccounts,
)
usersRouter.get(
'/:id/shard/sales',
// #swagger.tags = ['Admin · Users']
// #swagger.summary = 'Recent vendor sales on a user’s accounts (admin only)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'integer' }, description: 'User id.' }
/* #swagger.responses[200] = { description: 'Vendor sales', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/ShardVendorSale" } } } } } */
/* #swagger.responses[404] = { description: 'Not found', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
param('id').isInt(),
validate,
usersShard.getSales,
)
usersRouter.get(
'/:id/shard/houses',
// #swagger.tags = ['Admin · Users']
// #swagger.summary = 'Houses owned by a user’s accounts (admin only)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'integer' }, description: 'User id.' }
/* #swagger.responses[200] = { description: 'Houses (IDOC first)', content: { "application/json": { schema: { type: "array", items: { type: "object", additionalProperties: true } } } } } */
/* #swagger.responses[404] = { description: 'Not found', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
param('id').isInt(),
validate,
usersShard.getHouses,
)
usersRouter.get(
'/:id/shard/online',
// #swagger.tags = ['Admin · Users']
// #swagger.summary = 'A user’s characters currently online (admin only)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'integer' }, description: 'User id.' }
/* #swagger.responses[200] = { description: 'Online characters', content: { "application/json": { schema: { type: "array", items: { type: "object", additionalProperties: true } } } } } */
/* #swagger.responses[404] = { description: 'Not found', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
param('id').isInt(),
validate,
usersShard.getOnline,
)
usersRouter.get(
'/:id/shard/standing',
// #swagger.tags = ['Admin · Users']
// #swagger.summary = 'A user’s shard standing — governorships held and guilds led (admin only)'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'integer' }, description: 'User id.' }
/* #swagger.responses[200] = { description: 'Standing { governorOf, guildsLed }', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[404] = { description: 'Not found', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
param('id').isInt(),
validate,
usersShard.getStanding,
)
usersRouter.delete(
'/:id/shard/link/:account',
// #swagger.tags = ['Admin · Users']
// #swagger.summary = 'Unlink a game account from this user (admin only)'
// #swagger.description = 'Severs a game account’s tie to the website user from the site side (sidecar DELETE /link/{account}) and drops the local mirror. actor is stamped from the session.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['id'] = { in: 'path', required: true, schema: { type: 'integer' }, description: 'User id.' }
// #swagger.parameters['account'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Game account to unlink.' }
/* #swagger.responses[200] = { description: 'Unlinked', content: { "application/json": { schema: { type: "object", properties: { account: { type: "string" }, unlinked: { type: "boolean" } } } } } } */
/* #swagger.responses[403] = { description: 'Protected staff account (refused by shard)', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[404] = { description: 'Not linked', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
param('id').isInt(),
param('account').matches(SHARD_ACCOUNT_RE),
validate,
usersShard.unlinkAccount,
ctrl.getUser,
)
// ── The `admin.users.detail` extension slot (MODULE_SYSTEM.md §1.9) ────────
//
// A module may hang routes off this core resource. Core DECLARES the slot; only
// core may, and a module may only fill one (MODULE_API.md §2.4). What fills it
// today is core's own usersShard.router.js, registered in registries.js's
// registerCore() — the shard footprint that used to be wired inline right here.
// Phase 3 changes the registrant, not this line.
//
// LAST, deliberately: every core route on the resource is already declared, so
// first-match-wins means core owns any path conflict. The router is created at
// declare time and filled later, because this file is required while app.js is
// still being built — long before a module has been scanned.
usersRouter.use('/:id', registries.declareSlot('admin.users.detail'))
module.exports = usersRouter

142
server/src/router/v1/admin/usersShard.controller.js
View File

@@ -1,142 +0,0 @@
// ── Admin: a single user's shard (uo-link) footprint ──────────────────────────
//
// Backs the /admin/users/:id detail page. Every read is scoped to the target
// user's linked game accounts (from the local shard_account_links mirror): their
// vendor sales, houses, and currently-online characters. The live character
// rosters are fetched separately by the client through the existing admin-bypass
// /admin/shard/* endpoints, so nothing here round-trips the sidecar — these are
// fast, DB-backed reads. Admin-only (registered under adminOnly in the router).
const users = require('../../../model/users/users.model')
const shardLinks = require('../../../model/shardLinks/shardLinks.model')
const shardState = require('../../../model/shardState/shardState.model')
const uoLinkClient = require('../../../utils/uoLinkClient')
const activity = require('../../../model/activity/activity.model')
const { salesForAccounts } = require('../../../utils/shardSales')
const log = require('../../../utils/logger')('admin-user-shard')
// Resolve the target user's linked game accounts, or null if the user id is
// unknown (so the handler can 404 rather than silently returning an empty set).
async function accountsForUser(id) {
const user = await users.getById(id)
if (!user) return null
const links = await shardLinks.listForUser(id)
return { user, links, accounts: links.map((l) => l.account) }
}
// GET /admin/users/:id — the sanitized user (so the detail page is refresh-safe).
async function getUser(req, res) {
try {
const user = await users.getById(Number(req.params.id))
if (!user) return res.status(404).json({ message: 'Not found' })
return res.json(user)
} catch (err) {
log.error('getUser', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /admin/users/:id/shard/accounts — the user's linked game accounts.
async function listAccounts(req, res) {
try {
const ctx = await accountsForUser(Number(req.params.id))
if (!ctx) return res.status(404).json({ message: 'Not found' })
return res.json(ctx.links)
} catch (err) {
log.error('listAccounts', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /admin/users/:id/shard/sales — recent vendor sales on the user's accounts.
async function getSales(req, res) {
try {
const ctx = await accountsForUser(Number(req.params.id))
if (!ctx) return res.status(404).json({ message: 'Not found' })
return res.json(await salesForAccounts(ctx.accounts))
} catch (err) {
log.error('getSales', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /admin/users/:id/shard/houses — houses owned by the user's accounts.
async function getHouses(req, res) {
try {
const ctx = await accountsForUser(Number(req.params.id))
if (!ctx) return res.status(404).json({ message: 'Not found' })
return res.json(await shardState.listHousesForAccounts(ctx.accounts))
} catch (err) {
log.error('getHouses', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /admin/users/:id/shard/online — the user's characters currently online.
async function getOnline(req, res) {
try {
const ctx = await accountsForUser(Number(req.params.id))
if (!ctx) return res.status(404).json({ message: 'Not found' })
return res.json(await shardState.listOnlineForAccounts(ctx.accounts))
} catch (err) {
log.error('getOnline', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /admin/users/:id/shard/standing — the user's shard "standing" cross-links:
// city governorships they currently hold and guilds they lead. Both are reliable
// current-state lookups on the user's linked accounts.
async function getStanding(req, res) {
try {
const ctx = await accountsForUser(Number(req.params.id))
if (!ctx) return res.status(404).json({ message: 'Not found' })
const [governorOf, guildsLed] = await Promise.all([
shardState.listGovernorshipsForAccounts(ctx.accounts),
shardState.listGuildsLedForAccounts(ctx.accounts),
])
return res.json({ governorOf, guildsLed })
} catch (err) {
log.error('getStanding', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// DELETE /admin/users/:id/shard/link/:account — unlink a game account from this
// user, site-side. `actor` is stamped from the session (never the browser). On
// success the sidecar clears the WebsiteUserId tag on the shard and we drop the
// local mirror so attribution stops immediately.
async function unlinkAccount(req, res) {
const { account } = req.params
try {
const ctx = await accountsForUser(Number(req.params.id))
if (!ctx) return res.status(404).json({ message: 'Not found' })
// Only unlink an account actually linked to THIS user (avoid cross-user unlink).
if (!ctx.accounts.includes(account)) {
return res.status(404).json({ message: 'That account is not linked to this user.' })
}
const result = await uoLinkClient.unlinkAccount({ actor: req.user.username, account })
if (result.ok) {
await shardLinks.removeByAccount(account)
await activity.log({ req, userId: ctx.user.id, action: 'shard.account.unlink', detail: { account } })
log.info('game account unlinked', { account, userId: ctx.user.id, actor: req.user.username })
return res.json({ account, unlinked: true })
}
if (result.status === 403) return res.status(403).json({ message: 'That account is protected and cannot be unlinked.' })
if (result.status === 404) {
// Not linked on the shard — reconcile our mirror anyway so the two agree.
await shardLinks.removeByAccount(account)
return res.status(404).json({ message: 'That account is not linked.' })
}
if (result.status === 503 || result.status === 0) {
return res.status(503).json({ message: 'The game server is unavailable — try again shortly.' })
}
return res.status(502).json({ message: 'Could not reach the shard to unlink the account.' })
} catch (err) {
log.error('unlinkAccount', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
module.exports = { getUser, listAccounts, getSales, getHouses, getOnline, getStanding, unlinkAccount }

8
server/src/router/v1/auth/notifications.controller.js
View File

@@ -5,7 +5,7 @@
const pushDevices = require('../../../model/pushDevices/pushDevices.model')
const notificationSubs = require('../../../model/notificationSubs/notificationSubs.model')
const { STREAMS } = require('../../../config/notificationStreams')
const registries = require('../../../modules/registries')
const { isAllowedEndpoint } = require('../../../utils/pushDispatch')
const log = require('../../../utils/logger')('notifications')
@@ -49,9 +49,11 @@ async function removeDevice(req, res) {
}
}
// GET /auth/me/notifications/streams — the subscribable catalog (static).
// GET /auth/me/notifications/streams — the subscribable catalog: core's streams
// plus every installed module's, in registration order. Fixed for the lifetime of
// a process (registration is boot-time), not a static constant.
function getStreams(req, res) {
return res.json({ streams: STREAMS })
return res.json({ streams: registries.allStreams() })
}
// GET /auth/me/notifications/subscriptions — the caller's opted-in stream ids.

6
server/src/router/v1/player/index.js
View File

@@ -13,7 +13,9 @@
// Adding a requireRole('player') here would 403 an admin off their own characters
// (it happened once — see docs/website/BACKEND_DESIGN.md). Staff also reach the
// identical self-scoped handlers under /admin/shard and /auth/me/account; those
// are alternative URLs onto the same controllers, not duplicated logic.
// are alternative URLs onto the same controllers, not duplicated logic — and
// both of those live in module-uo now, which changes where they are defined and
// nothing about which URLs answer.
//
// See docs/website/API_V2_PLAN.md § Phase 2 for the split.
@@ -23,7 +25,6 @@ const { requireAuth } = require('../../../auth/session.middleware')
const noindex = require('../../../middleware/noindex')
const accountRouter = require('./account.router')
const shardRouter = require('./shard.router')
const appealsRouter = require('./appeals.router')
const playerRouter = express.Router()
@@ -37,7 +38,6 @@ const playerRouter = express.Router()
playerRouter.use(noindex, requireAuth)
playerRouter.use('/account', accountRouter)
playerRouter.use('/shard', shardRouter)
playerRouter.use('/appeals', appealsRouter)
module.exports = playerRouter

274
server/src/router/v1/player/shard.controller.js
View File

@@ -1,274 +0,0 @@
// ── Player: game-account linking + reads ───────────────────────────────────
//
// The player-facing surface for the uo-link integration. A logged-in player
// runs [link in game, gets a one-time code, and enters it here — the server
// confirms it with the sidecar (which permanently tags the game account with the
// website user id) and mirrors the link locally. Roster/vendor reads are
// ownership-checked against that mirror so a player can only see accounts they
// have linked. The sidecar token stays server-side throughout.
const uoLinkClient = require('../../../utils/uoLinkClient')
const shardLinks = require('../../../model/shardLinks/shardLinks.model')
const shardState = require('../../../model/shardState/shardState.model')
const shardClilocs = require('../../../model/shardClilocs/shardClilocs.model')
const settings = require('../../../model/settings/settings.model')
const { salesForAccounts } = require('../../../utils/shardSales')
const activity = require('../../../model/activity/activity.model')
const log = require('../../../utils/logger')('player-shard')
const SERIAL_RE = /^0x[0-9a-fA-F]+$/
/**
* Resolve the cliloc ids on a profile into display names.
*
* Items on the wire carry a `LabelNumber`, not a name — `BridgeProfile.WriteItem`
* sends `cliloc` on every equipment entry and `name` only for the minority of
* items a player has renamed. Reward titles are the same shape: the shard sends
* a cliloc number as a string, which the sheet previously had to SKIP because it
* had no way to turn it into words.
*
* Resolution happens here rather than in the browser because the table is ~123k
* rows: shipping it to render a dozen names would dwarf the page, and the
* Android client consumes this same JSON and would otherwise need its own copy.
*
* A shard with no cliloc table configured resolves nothing and the sheet renders
* ids exactly as it did before — this is decoration, and it is applied in the
* same best-effort block as the guild/governor cross-links.
*/
async function resolveProfileClilocs(profile) {
const wanted = []
const equipment = Array.isArray(profile.equipment) ? profile.equipment : []
for (const item of equipment) {
if (Number.isInteger(item?.cliloc)) wanted.push(item.cliloc)
}
// Reward titles arrive as strings that may be either a literal ("Knight of
// Trinsic") or a cliloc number in string form. Only the numeric ones need us.
const reward = Array.isArray(profile.titles?.reward) ? profile.titles.reward : []
const rewardNumbers = reward.map((r) => (/^\d+$/.test(String(r)) ? Number(r) : null))
for (const n of rewardNumbers) if (n !== null) wanted.push(n)
if (wanted.length === 0) return
const names = await shardClilocs.resolveMany(wanted)
if (names.size === 0) return
for (const item of equipment) {
// A player-given name always wins over the type name: an item called "Bob's
// lucky axe" should not be relabelled "hatchet".
if (item?.name) continue
const resolved = names.get(item?.cliloc)
if (resolved) item.clilocName = resolved
}
if (rewardNumbers.some((n) => n !== null)) {
profile.titles.rewardResolved = reward.map((raw, i) => {
const n = rewardNumbers[i]
return n === null ? String(raw) : names.get(n) ?? null
})
}
}
// Decorate a char.profile with cross-links from our own board data: the guild the
// character leads and any city governorship on its account, plus resolved cliloc
// names. Best-effort — a failure here never fails the profile (it's a nicety,
// not the sheet).
async function enrichCharProfile(profile) {
if (!profile) return profile
try {
const guild = await shardState.findGuildForActor({ serial: profile.serial, acct: profile.acct })
if (guild) profile.guild = guild
if (profile.acct) {
const govs = await shardState.listGovernorshipsForAccounts([profile.acct])
if (govs.length) profile.governorOf = govs.map((g) => g.city)
}
await resolveProfileClilocs(profile)
} catch (err) {
log.warn('enrichCharProfile failed', { serial: profile.serial, message: err.message })
}
return profile
}
// POST /player/shard/link — confirm an in-game link code.
async function link(req, res) {
const { code } = req.body
try {
const result = await uoLinkClient.confirmLink(code, req.user.id)
if (result.ok && result.data && result.data.kind === 'link.ok') {
const account = result.data.account
await shardLinks.link({ account, userId: req.user.id, charName: result.data.char || null })
await activity.log({ req, action: 'uoLink.account.link', detail: { account } })
log.info('player linked game account', { user: req.user.username, account })
return res.json({ linked: true, account })
}
// Sidecar reports bad/expired codes as 400 link.error or 404.
if (result.status === 400 || result.status === 404) {
return res.status(400).json({ message: 'That code is unknown or has expired. Run [link in game for a new one.' })
}
if (result.status === 503 || result.status === 0) {
return res.status(503).json({ message: 'The shard is unavailable right now — try again shortly.' })
}
return res.status(502).json({ message: 'Could not confirm the link with the shard.' })
} catch (err) {
log.error('player.shard.link', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /player/shard/accounts — the caller's linked game accounts.
async function listAccounts(req, res) {
try {
return res.json(await shardLinks.listForUser(req.user.id))
} catch (err) {
log.error('player.shard.listAccounts', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// Admins may view any character's data; everyone else is limited to accounts
// they have personally linked. The same handlers back /player/shard (role
// `player`, never admin) and /admin/shard (staff), so this bypass only ever
// widens access for genuine admins.
const isAdmin = (req) => req.user && req.user.role === 'admin'
// Shared ownership gate + live round-trip for roster/vendors. `fetcher` is the
// uoLinkClient method to call with the account.
async function ownedRoundTrip(req, res, fetcher, label) {
const { account } = req.params
try {
const owns = isAdmin(req) || (await shardLinks.ownsAccount(account, req.user.id))
if (!owns) return res.status(403).json({ message: 'That account is not linked to your profile.' })
const result = await fetcher(account)
if (result.ok) return res.json(result.data)
if (result.status === 404) return res.status(404).json({ message: 'Not found.' })
if (result.status === 503 || result.status === 0) {
return res.status(503).json({ message: 'The shard is unavailable right now — try again shortly.' })
}
return res.status(502).json({ message: 'Could not reach the shard.' })
} catch (err) {
log.error(`player.shard.${label}`, err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /player/shard/roster/:account — characters on a linked account.
const roster = (req, res) => ownedRoundTrip(req, res, uoLinkClient.getRoster, 'roster')
// GET /player/shard/vendors/:account — player vendors on a linked account.
const vendors = (req, res) => ownedRoundTrip(req, res, uoLinkClient.getVendors, 'vendors')
// GET /player/shard/char/:serial — a character sheet, but ONLY if the character's
// account is linked to the caller. The sidecar returns the owning account in the
// profile, which we check against the caller's links before returning anything.
async function getChar(req, res) {
const { serial } = req.params
if (!SERIAL_RE.test(serial)) return res.status(400).json({ message: 'Invalid serial.' })
try {
const result = await uoLinkClient.getCharBySerial(serial)
if (result.ok) {
// Admins see any character; others only characters on an account they linked.
if (!isAdmin(req)) {
const acct = result.data && result.data.acct
const owns = acct ? await shardLinks.ownsAccount(acct, req.user.id) : false
if (!owns) return res.status(403).json({ message: 'That character is not on an account linked to you.' })
}
return res.json(await enrichCharProfile(result.data))
}
if (result.status === 404) return res.status(404).json({ message: 'Character not found.' })
if (result.status === 503 || result.status === 0) {
return res.status(503).json({ message: 'The game server is restarting — try again shortly.' })
}
return res.status(502).json({ message: 'Could not reach the shard.' })
} catch (err) {
log.error('player.shard.getChar', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /player/shard/sales — recent player-vendor sales for the caller's linked
// accounts only (as seller/owner). Read from the site's own event log.
async function getSales(req, res) {
try {
const links = await shardLinks.listForUser(req.user.id)
const accounts = links.map((l) => l.account)
return res.json(await salesForAccounts(accounts))
} catch (err) {
log.error('player.shard.getSales', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /player/shard/houses — the caller's OWN houses (home status), scoped to
// their linked accounts. A player sees their own decay/IDOC standing; never
// anyone else's. Full detail is fine here — it's their property.
async function getHouses(req, res) {
try {
const links = await shardLinks.listForUser(req.user.id)
const accounts = links.map((l) => l.account)
return res.json(await shardState.listHousesForAccounts(accounts))
} catch (err) {
log.error('player.shard.getHouses', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// Map a failed uoLinkClient.createAccount result to a user-facing HTTP response.
// The password is never echoed anywhere; only the mapped reason is returned.
function mapCreateAccountError(res, result) {
const reason = (result.data && result.data.reason) || ''
switch (result.status) {
case 409:
return res.status(409).json({ message: 'That account name is already taken.' })
case 429:
return res.status(429).json({ message: 'The account limit for your network has been reached.' })
case 403:
return res.status(403).json({ message: 'Game-account signups are not available on this shard right now.' })
case 400:
return res.status(400).json({ message: reason || 'The account name or password was not accepted.' })
case 503:
case 0:
return res.status(503).json({ message: 'The game server is unavailable — try again shortly.' })
default:
return res.status(502).json({ message: 'Could not reach the shard to create the account.' })
}
}
// POST /player/shard/account — provision a GAME account for the signed-in website
// user and auto-link it (Protocol 2.0 hybrid). Used by self-serve signup and the
// invite-accept "create game account" step alike (both act as the signed-in user).
// actor + websiteUserId are stamped from the session; the browser IP (req.ip,
// trust-proxy configured) is forwarded for the shard's per-IP cap; the password is
// never logged. Gated by the game_account_signup setting AND the shard's own mode.
async function createGameAccount(req, res) {
const { account, password } = req.body
try {
if (!(await settings.isGameAccountSignupEnabled())) {
return res.status(403).json({ message: 'Game-account signup is not available right now.' })
}
const result = await uoLinkClient.createAccount({
actor: req.user.username,
account,
password,
websiteUserId: req.user.id,
ip: req.ip,
})
if (result.ok) {
// Mirror the link locally so the portal lists the account immediately.
await shardLinks.link({ account, userId: req.user.id })
await activity.log({ req, userId: req.user.id, action: 'shard.account.create', detail: { account } })
log.info('game account created', { account, userId: req.user.id, ip: req.ip })
return res.status(201).json({ account, linked: true })
}
return mapCreateAccountError(res, result)
} catch (err) {
log.error('player.shard.createGameAccount', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
module.exports = { link, listAccounts, roster, vendors, getChar, getSales, getHouses, createGameAccount }

124
server/src/router/v1/player/shard.router.js
View File

@@ -1,124 +0,0 @@
// Player · Shard — game-account linking and the caller's own roster / vendors /
// characters / sales / houses, ownership-checked against the local link mirror.
//
// Mounted at /api/v1/player/shard by player/index.js, which already applied
// `noindex, requireAuth`. No extra gate: every handler is self-scoped to
// req.user.id.
//
// These are the *same* handlers (player/shard.controller) that admin/shard.router.js
// serves under /admin/shard for the seven self-service routes — staff are a
// superset of players, and the controller keys off req.user.id either way. Two
// URL surfaces, one implementation.
const express = require('express')
const { body, param } = require('express-validator')
const shard = require('./shard.controller')
const validate = require('../../../middleware/validate')
const { accountChangeLimiter } = require('../../../middleware/rateLimit')
const shardRouter = express.Router()
// Link an in-game account with a one-time code from [link, then read the
// account's roster / vendors (ownership-checked against the local link mirror).
const ACCOUNT_RE = /^[A-Za-z0-9_.-]{1,120}$/
shardRouter.post(
'/link',
// #swagger.tags = ['Player · Shard']
// #swagger.summary = 'Link an in-game account with a one-time code'
// #swagger.description = 'The player runs [link in game to get a code, then submits it here. The server confirms it with the sidecar and mirrors the link.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { $ref: "#/components/schemas/ShardLinkRequest" } } } } */
/* #swagger.responses[200] = { description: 'Linked', content: { "application/json": { schema: { $ref: "#/components/schemas/ShardLinkResult" } } } } */
/* #swagger.responses[400] = { description: 'Unknown or expired code', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[503] = { description: 'Shard unavailable — retry', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
body('code').isString().trim().isLength({ min: 4, max: 32 }),
validate,
shard.link,
)
shardRouter.post(
'/account',
// #swagger.tags = ['Player · Shard']
// #swagger.summary = 'Create a game account (hybrid signup) and link it to the caller'
// #swagger.description = 'Provisions a new game account with its own username + password and auto-links it to the signed-in website user. Available only when game_account_signup is enabled and the shard accepts website signups. The password is hashed on the shard and never stored or logged by the site.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["account","password"], properties: { account: { type: "string" }, password: { type: "string" } } } } } */
/* #swagger.responses[201] = { description: 'Account created and linked', content: { "application/json": { schema: { type: "object", properties: { account: { type: "string" }, linked: { type: "boolean" } } } } } } */
/* #swagger.responses[400] = { description: 'Validation error or rejected name/password', content: { "application/json": { schema: { $ref: "#/components/schemas/ValidationError" } } } } */
/* #swagger.responses[403] = { description: 'Game-account signup unavailable (site or shard)', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[409] = { description: 'Account name already taken', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[429] = { description: 'Per-IP account cap reached', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[503] = { description: 'Shard unavailable — retry', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
accountChangeLimiter,
body('account').matches(/^[A-Za-z0-9][A-Za-z0-9_.-]{2,29}$/),
body('password').isString().isLength({ min: 8, max: 64 }),
validate,
shard.createGameAccount,
)
shardRouter.get(
'/accounts',
// #swagger.tags = ['Player · Shard']
// #swagger.summary = 'List the caller’s linked game accounts'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Linked accounts', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/ShardLink" } } } } } */
shard.listAccounts,
)
shardRouter.get(
'/roster/:account',
// #swagger.tags = ['Player · Shard']
// #swagger.summary = 'Character roster for a linked account'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['account'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'A game account linked to the caller.' }
/* #swagger.responses[200] = { description: 'Account roster', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[403] = { description: 'Account not linked to the caller', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[503] = { description: 'Shard unavailable — retry', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
param('account').matches(ACCOUNT_RE),
validate,
shard.roster,
)
shardRouter.get(
'/vendors/:account',
// #swagger.tags = ['Player · Shard']
// #swagger.summary = 'Player vendors for a linked account'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['account'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'A game account linked to the caller.' }
/* #swagger.responses[200] = { description: 'Vendor snapshot', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[403] = { description: 'Account not linked to the caller', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[503] = { description: 'Shard unavailable — retry', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
param('account').matches(ACCOUNT_RE),
validate,
shard.vendors,
)
shardRouter.get(
'/char/:serial',
// #swagger.tags = ['Player · Shard']
// #swagger.summary = 'Character sheet — only for a character on the caller’s linked account'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
// #swagger.parameters['serial'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Mobile serial, e.g. 0x24C.' }
/* #swagger.responses[200] = { description: 'Character profile', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
/* #swagger.responses[403] = { description: 'Character not on an account linked to the caller', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[503] = { description: 'Shard unavailable — retry', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
param('serial').matches(/^0x[0-9a-fA-F]+$/),
validate,
shard.getChar,
)
shardRouter.get(
'/sales',
// #swagger.tags = ['Player · Shard']
// #swagger.summary = 'Recent player-vendor sales for the caller’s linked accounts'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Vendor sales', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/ShardVendorSale" } } } } } */
shard.getSales,
)
shardRouter.get(
'/houses',
// #swagger.tags = ['Player · Shard']
// #swagger.summary = 'The caller’s own houses (home status)'
// #swagger.description = 'Houses owned by the caller’s linked accounts, with decay/IDOC status. Only the caller’s own houses — never anyone else’s.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'The caller’s houses', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/ShardHouse" } } } } } */
shard.getHouses,
)
module.exports = shardRouter

134
server/src/router/v1/public/atlas.controller.js
View File

@@ -1,134 +0,0 @@
// ── Public: the spawn atlas ────────────────────────────────────────────────
//
// A browsable catalogue of what the shard CONTAINS — which creatures spawn,
// where, how many, and which champion altars are configured. Everything here is
// a plain indexed read of the tables the boot-time import fills from the shard's
// own ServUO tree (docs/website/SPAWN_ATLAS.md).
//
// Two properties separate this from /public/shard/*:
//
// • **Nothing touches the sidecar.** The atlas is static shard content, not
// live shard state, so these pages stay fully populated while the shard is
// down. That is why the routes are mounted at /public/atlas and are
// siteMode-gated like /posts and /wiki, rather than under /shard.
// • **The live champion feed is a different thing.** `/atlas/champions` is the
// configured roster ("there is an Unholy Terror altar in Deceit");
// `/shard/champs` is the running state ("it is on level 3 right now").
//
// Every response is still passed through `projectFeature` for the `atlas`
// feature. It declares no sensitive fields today, so the projection is a
// no-op — but v3.md §3.6.1's rule is that a read path returning shard data and
// not projecting is a bug, and the cost of honouring it is one call per handler
// rather than a retrofit the first time a field needs gating.
const atlas = require('../../../model/shardAtlas/shardAtlas.model')
const visibility = require('../../../utils/shardVisibility')
const log = require('../../../utils/logger')('public-atlas')
const FEATURE = 'atlas'
// Query params arrive as strings; express-validator has already bounded them.
const int = (value, fallback) => {
const n = Number.parseInt(value, 10)
return Number.isFinite(n) ? n : fallback
}
const str = (value) => (typeof value === 'string' ? value.trim() : '')
// GET /public/atlas/creatures?q=&facet=&limit=&offset=
async function getCreatures(req, res) {
try {
const page = await atlas.searchCreatures({
q: str(req.query.q),
facet: str(req.query.facet),
limit: int(req.query.limit, 50),
offset: int(req.query.offset, 0),
})
return res.json(await visibility.project(FEATURE, page, req))
} catch (err) {
log.error('atlas.getCreatures', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/atlas/creatures/:slug — one creature, with the places it spawns.
//
// 404 means "no such creature in this atlas", which also covers "the atlas has
// never been imported" — an empty atlas has no slugs, and there is nothing more
// specific to say to an anonymous caller.
async function getCreature(req, res) {
try {
const creature = await atlas.getCreature(req.params.slug, {
facet: str(req.query.facet),
points: int(req.query.points, 200),
})
if (!creature) return res.status(404).json({ message: 'Not Found' })
return res.json(await visibility.project(FEATURE, creature, req))
} catch (err) {
log.error('atlas.getCreature', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/atlas/regions?facet=&q=
async function getRegions(req, res) {
try {
const regions = await atlas.listRegions({
facet: str(req.query.facet),
q: str(req.query.q),
})
return res.json(await visibility.project(FEATURE, regions, req))
} catch (err) {
log.error('atlas.getRegions', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/atlas/landmarks?facet=&q=
async function getLandmarks(req, res) {
try {
const landmarks = await atlas.listLandmarks({
facet: str(req.query.facet),
q: str(req.query.q),
})
return res.json(await visibility.project(FEATURE, landmarks, req))
} catch (err) {
log.error('atlas.getLandmarks', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/atlas/champions?facet= — the CONFIGURED altar roster.
async function getChampions(req, res) {
try {
const champions = await atlas.listChampions({ facet: str(req.query.facet) })
return res.json(await visibility.project(FEATURE, champions, req))
} catch (err) {
log.error('atlas.getChampions', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/atlas/meta — what is loaded: facets, counts, when it was imported.
//
// Public-safe by construction: the model omits the ServUO path, the per-file
// hashes and the pending-refresh state, all of which describe the operator's
// filesystem rather than the game world. The admin status route carries those.
async function getMeta(req, res) {
try {
return res.json(await visibility.project(FEATURE, await atlas.publicMeta(), req))
} catch (err) {
log.error('atlas.getMeta', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
module.exports = {
getCreatures,
getCreature,
getRegions,
getLandmarks,
getChampions,
getMeta,
}

128
server/src/router/v1/public/atlas.router.js
View File

@@ -1,128 +0,0 @@
// Public · Atlas — the spawn atlas / bestiary. Static shard CONTENT derived from
// the shard's own ServUO tree, not live shard state.
//
// Mounted at /api/v1/public/atlas by public/index.js. Two deliberate differences
// from the /public/shard routes next door (docs/link/v3.md §6):
//
// • **Not under /shard.** Nothing here round-trips the sidecar, and the pages
// stay fully populated while the shard is down. Mounting it under /shard
// would imply a dependency it does not have.
// • **siteMode-gated, like /posts and /wiki.** The shard routes are exempt
// because shard status is wanted *during* maintenance; a bestiary is site
// content and follows site content's rules.
//
// Every route also carries `requireFeature('atlas')` — 404 when an admin has
// disabled the feature, 403 when the caller sits below its configured audience.
// The default audience is `anonymous`, so these gates are inert until an admin
// changes something.
const express = require('express')
const { param, query } = require('express-validator')
const atlas = require('./atlas.controller')
const siteMode = require('../../../middleware/siteMode')
const validate = require('../../../middleware/validate')
const { requireFeature } = require('../../../utils/shardVisibility')
const atlasRouter = express.Router()
// Facet names come from the shard's own files and are never validated against a
// list — nothing in the codebase names a facet (§6.1 R2). Only the length is
// bounded, and the query matches exactly, so an unknown name returns an empty
// result rather than an error.
const facetParam = query('facet').optional({ values: 'falsy' }).isString().isLength({ max: 40 })
atlasRouter.get(
'/creatures',
requireFeature('atlas'),
// #swagger.tags = ['Public · Atlas']
// #swagger.summary = 'Search the bestiary (paginated)'
// #swagger.description = 'Every creature the shard spawns, most numerous first. `total` is how many can be alive at once across all spawners; `points` is how many spawners mention it; `facets` maps facet name to that creature\'s share on it. Static content parsed from the shard\'s ServUO tree — unaffected by the shard being offline.'
// #swagger.parameters['q'] = { in: 'query', required: false, schema: { type: 'string' }, description: 'Substring match on the creature name (max 60 chars).' }
// #swagger.parameters['facet'] = { in: 'query', required: false, schema: { type: 'string' }, description: 'Limit to creatures spawning on this facet. Facet names come from the shard\'s own files; an unknown one returns an empty page.' }
// #swagger.parameters['limit'] = { in: 'query', required: false, schema: { type: 'integer' }, description: 'Page size, 1..100 (default 50).' }
// #swagger.parameters['offset'] = { in: 'query', required: false, schema: { type: 'integer' }, description: 'Rows to skip (default 0).' }
/* #swagger.responses[200] = { description: 'A page of creatures plus the unpaginated total', content: { "application/json": { schema: { $ref: "#/components/schemas/AtlasCreaturePage" } } } } */
/* #swagger.responses[403] = { description: 'The atlas feature is gated above this caller', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[404] = { description: 'The atlas feature is disabled', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
query('q').optional({ values: 'falsy' }).isString().isLength({ max: 60 }),
facetParam,
query('limit').optional().isInt({ min: 1, max: 100 }),
query('offset').optional().isInt({ min: 0, max: 100000 }),
validate,
siteMode,
atlas.getCreatures,
)
atlasRouter.get(
'/creatures/:slug',
requireFeature('atlas'),
// #swagger.tags = ['Public · Atlas']
// #swagger.summary = 'One creature: where it spawns, and what spawns with it'
// #swagger.description = 'The answer the atlas exists to give. `places` is the aggregate — "lizardman → Shrines, Isamu-Jima, Yew" — resolved by point-in-rect against the shard\'s own region rectangles, falling back to the nearest landmark, else "Wilderness". `spawners` lists the individual spawn points (bounded; `spawnersTruncated` says when the list was cut), and `alsoHere` is what shares those spawners.'
// #swagger.parameters['slug'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'Creature slug, e.g. lizardman.' }
// #swagger.parameters['facet'] = { in: 'query', required: false, schema: { type: 'string' }, description: 'Restrict places and spawners to one facet.' }
// #swagger.parameters['points'] = { in: 'query', required: false, schema: { type: 'integer' }, description: 'Max spawners to return, 1..1000 (default 200).' }
/* #swagger.responses[200] = { description: 'The creature', content: { "application/json": { schema: { $ref: "#/components/schemas/AtlasCreature" } } } } */
/* #swagger.responses[404] = { description: 'No such creature in this atlas (or the feature is disabled)', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
param('slug').isString().isLength({ min: 1, max: 120 }),
facetParam,
query('points').optional().isInt({ min: 1, max: 1000 }),
validate,
siteMode,
atlas.getCreature,
)
atlasRouter.get(
'/regions',
requireFeature('atlas'),
// #swagger.tags = ['Public · Atlas']
// #swagger.summary = 'Named regions and their rectangles'
// #swagger.description = 'Flattened out of the shard\'s nested Regions.xml. `priority` and the rectangles are what placed each spawn point, kept so the placement can be re-derived rather than taken on trust.'
// #swagger.parameters['facet'] = { in: 'query', required: false, schema: { type: 'string' }, description: 'Limit to one facet.' }
// #swagger.parameters['q'] = { in: 'query', required: false, schema: { type: 'string' }, description: 'Substring match on the region name.' }
/* #swagger.responses[200] = { description: 'Regions, by facet then name', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/AtlasRegion" } } } } } */
facetParam,
query('q').optional({ values: 'falsy' }).isString().isLength({ max: 60 }),
validate,
siteMode,
atlas.getRegions,
)
atlasRouter.get(
'/landmarks',
requireFeature('atlas'),
// #swagger.tags = ['Public · Atlas']
// #swagger.summary = 'Points of interest (dungeon levels, town markers)'
// #swagger.description = 'From the shard\'s Data/Locations files. `group` is the innermost enclosing parent ("Covetous"), which is the label worth showing over the individual marker ("Level 1").'
// #swagger.parameters['facet'] = { in: 'query', required: false, schema: { type: 'string' }, description: 'Limit to one facet.' }
// #swagger.parameters['q'] = { in: 'query', required: false, schema: { type: 'string' }, description: 'Substring match on the landmark name or its group.' }
/* #swagger.responses[200] = { description: 'Landmarks, by facet then group', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/AtlasLandmark" } } } } } */
facetParam,
query('q').optional({ values: 'falsy' }).isString().isLength({ max: 60 }),
validate,
siteMode,
atlas.getLandmarks,
)
atlasRouter.get(
'/champions',
requireFeature('atlas'),
// #swagger.tags = ['Public · Atlas']
// #swagger.summary = 'Configured champion altars (the roster, not the live board)'
// #swagger.description = 'Where the altars are and what each one summons — "there is an Unholy Terror altar in Deceit". `randomType` marks altars whose champion is drawn at activation. Do not conflate this with GET /public/shard/champs, which is the live sidecar-fed board ("it is on level 3 right now").'
// #swagger.parameters['facet'] = { in: 'query', required: false, schema: { type: 'string' }, description: 'Limit to one facet.' }
/* #swagger.responses[200] = { description: 'Altars, by facet then name', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/AtlasChampion" } } } } } */
facetParam,
validate,
siteMode,
atlas.getChampions,
)
atlasRouter.get(
'/meta',
requireFeature('atlas'),
// #swagger.tags = ['Public · Atlas']
// #swagger.summary = 'What atlas is loaded: facets, counts, when it was imported'
// #swagger.description = 'Drives the facet filter and the "parsed from the shard\'s own files on <date>" line. Reports the game world only — the ServUO path, the per-file hashes and any pending refresh are operator detail and live on the admin status route.'
/* #swagger.responses[200] = { description: 'Atlas metadata', content: { "application/json": { schema: { $ref: "#/components/schemas/AtlasMeta" } } } } */
siteMode,
atlas.getMeta,
)
module.exports = atlasRouter

16
server/src/router/v1/public/index.js
View File

@@ -20,8 +20,7 @@ const express = require('express')
const postsRouter = require('./posts.router')
const wikiRouter = require('./wiki.router')
const pagesRouter = require('./pages.router')
const shardRouter = require('./shard.router')
const atlasRouter = require('./atlas.router')
const modulesRouter = require('./modules.router')
const siteRouter = require('./site.router')
const publicRouter = express.Router()
@@ -31,13 +30,12 @@ const publicRouter = express.Router()
publicRouter.use('/posts', postsRouter)
publicRouter.use('/wiki', wikiRouter)
publicRouter.use('/pages', pagesRouter)
// Live shard data, never site-mode gated.
publicRouter.use('/shard', shardRouter)
// The spawn atlas: static shard CONTENT, parsed from the shard's ServUO tree
// rather than fetched from the sidecar. Deliberately not under /shard — nothing
// here depends on the bridge — and site-mode gated per route like the content
// routers above, which is the other half of that distinction.
publicRouter.use('/atlas', atlasRouter)
// What this backend serves beyond core. A real prefix layer rather than a fifth
// singleton in site.router.js, because the loader's prefix-collision probe reads
// the live tier stack and skips root-mounted layers — this mount is what makes
// /modules unclaimable by a module. Never site-mode gated: a client must be able
// to feature-detect while the site is in maintenance.
publicRouter.use('/modules', modulesRouter)
// The four singletons that own no path segment of their own: /settings, /status,
// /version and /contact. Mounted at the group root, last — safe only because

60
server/src/router/v1/public/modules.controller.js Normal file
View File

@@ -0,0 +1,60 @@
// Public · Modules — what this backend is currently serving beyond core.
//
// Phase 2, PR 6 of docs/website/MODULE_SYSTEM.md §2.7. The published shape is
// settled in MODULE_API.md §2.1 (`capabilities` are opaque strings, published
// here, for clients to feature-detect against).
//
// Two decisions are visible in the ten lines below and are the whole of this
// file's design:
//
// • **`started` only.** The public surface answers "what is serving", and
// nothing else. A module that failed to load, or that an operator disabled,
// is simply ABSENT — the same treatment §4.4 already gives its routes and
// its nav, so an anonymous visitor sees a site without that capability
// rather than a site advertising a capability that 503s. `state`, the
// failure stage and the failure reason are core's business and belong to the
// admin Modules screen; none of the three is published here.
// • **No database, and no siteMode gate.** The answer comes from the loader's
// in-memory records, so this endpoint keeps working with the database down —
// the same class as /public/version and /public/status, both of which must
// answer during maintenance so a client can bootstrap and render the
// maintenance page. A client that could not feature-detect while the site
// was in maintenance would render its maintenance page as though no module
// existed.
//
// This endpoint is deliberately NOT how a module's client chunk gets loaded.
// `utils/htmlShell.js` injects a `<script type="module">` per started module
// (MODULE_API.md §3.1.3), so the browser is handed the tag rather than a URL to
// go and fetch; there is no `client` field here for the same reason there is no
// second copy of any other fact. See MODULE_SYSTEM.md §2.6, amended to match.
const loader = require('../../../modules/loader')
const log = require('../../../utils/logger')('public:modules')
/** id, name, version and capabilities — everything else the loader knows is internal. */
const publish = (m) => ({
id: m.id,
name: m.name,
version: m.version,
capabilities: m.capabilities,
})
function getModules(req, res) {
try {
// Scan order (alphabetical by id) comes from the loader and is preserved:
// there is no dependency resolution, so any other order would imply a
// precedence nothing computes (MODULE_API.md §4.2).
return res.json({ modules: loader.list().filter((m) => m.state === 'started').map(publish) })
} catch (err) {
// The only reachable throw is §7.6's guard — the module list read before
// modules.load() ran. That is a mis-ordered boot, not a bad request, so it
// is logged rather than answered with an empty list: `{ modules: [] }` is a
// true answer for a core with no modules installed and a caller cannot tell
// the two apart.
log.error('module list unavailable', { message: err.message })
return res.status(500).json({ message: 'Internal Server Error' })
}
}
module.exports = { getModules }

30
server/src/router/v1/public/modules.router.js Normal file
View File

@@ -0,0 +1,30 @@
// Public · Modules — the installed-module list a client feature-detects against.
//
// Mounted at /api/v1/public/modules by public/index.js. One route, and it owns a
// prefix rather than sitting beside /settings and /version in site.router.js —
// which is the point of the file existing at all. The loader asks the LIVE
// public tier router whether a prefix is already core's (`ownedByCore`,
// modules/loader.js), and it skips root-mounted layers because a `use('/', …)`
// matches every path. A route declared inside the root-mounted site router is
// therefore invisible to that probe; a real `use('/modules', …)` layer is not.
// So mounting it here is what makes "no module may ever claim /modules" an
// enforced rule instead of a convention.
//
// No siteMode gate and no database — see modules.controller.js for why.
const express = require('express')
const ctrl = require('./modules.controller')
const modulesRouter = express.Router()
modulesRouter.get(
'/',
// #swagger.tags = ['Public']
// #swagger.summary = 'Installed modules (id, version, capabilities)'
// #swagger.description = 'The modules this backend is currently SERVING, in scan order. A module that is disabled or failed to load is absent rather than listed with a state — its routes and nav are absent too, so the client renders a site without that capability. `capabilities` are opaque strings declared by the module for clients (the SPA, the Android app) to feature-detect against; treat an unknown one as absent. Database-free and never gated by site mode, so a client can feature-detect during maintenance.'
/* #swagger.responses[200] = { description: 'The started modules', content: { "application/json": { schema: { $ref: "#/components/schemas/PublicModules" } } } } */
ctrl.getModules,
)
module.exports = modulesRouter

422
server/src/router/v1/public/shard.controller.js
View File

@@ -1,422 +0,0 @@
// ── Public: shard live data ────────────────────────────────────────────────
//
// Same-origin, token-free read endpoints backed by the data the WS ingest
// pipeline persists (shard_online / shard_events / shard_economy / shard_houses)
// plus a live character round-trip to the sidecar. The browser never sees the
// sidecar URL or token — every sidecar call is server-side (uoLinkClient).
//
// The stored-data endpoints are cheap DB reads. The live /char endpoint hits the
// running shard, so it is briefly cached and degrades gracefully: a 503 (shard
// restarting) surfaces as a retry-able banner rather than an error.
const shardEvents = require('../../../model/shardEvents/shardEvents.model')
const shardState = require('../../../model/shardState/shardState.model')
const shardMarket = require('../../../model/shardMarket/shardMarket.model')
const uoLinkConfig = require('../../../model/uoLinkConfig/uoLinkConfig.model')
const broadcast = require('../../../utils/shardBroadcast')
const visibility = require('../../../utils/shardVisibility')
const log = require('../../../utils/logger')('public-shard')
// GET /public/shard/status — connection state + online count + latest economy.
async function getStatus(req, res) {
try {
const config = await uoLinkConfig.getSafe()
const [online, economy] = await Promise.all([
shardState.onlineCount(),
shardState.latestEconomy(),
])
return res.json({
enabled: config.enabled,
status: config.status,
pluginConnected: config.pluginConnected,
lastEventAt: config.lastEventAt,
onlineCount: online,
economy,
})
} catch (err) {
log.error('shard.getStatus', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/feed?kind=&limit= — recent notable events from the log.
//
// This is the stored-history twin of the SSE stream, and it must reach the same
// verdict the stream does about the same event. Two things are therefore resolved
// against the LIVE config rather than the compiled defaults:
//
// • which kinds this viewer may read at all — `visibleKinds`, not the static
// PUBLIC_KINDS set (which is fixed at module load, so an admin moving
// `guilds` to `staff` would gate /guilds while /feed kept serving
// guild.join to anonymous callers), and
// • the payload itself, projected per event against ITS OWN kind's feature —
// the rows are a mix of features, and without this the stored frames were
// returned verbatim, `acct`/`webId` and all, on an anonymous endpoint.
async function getFeed(req, res) {
try {
const config = await visibility.getConfig()
const level = req.viewerLevel || (await visibility.viewerLevel(req))
const allowed = new Set(visibility.visibleKinds(level, config))
const { kind, limit } = req.query
// No readable kinds ⇒ nothing to serve. Returning early also keeps us clear
// of `list({ kinds: [] })`, which means "no filter", not "match nothing".
if (allowed.size === 0) return res.json([])
let events
if (kind) {
if (!allowed.has(kind)) return res.json([])
events = await shardEvents.list({ kind, limit })
} else {
events = await shardEvents.list({ kinds: [...allowed], limit })
}
return res.json(
events.map((ev) => ({
...ev,
payload: visibility.projectFeature(
visibility.KIND_FEATURE.get(ev.kind),
ev.payload,
level,
config,
),
})),
)
} catch (err) {
log.error('shard.getFeed', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/economy — gold-supply series, oldest → newest.
async function getEconomy(req, res) {
try {
return res.json(await shardState.listEconomy(req.query.limit))
} catch (err) {
log.error('shard.getEconomy', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/online — players online now whose account is linked to a
// STAFF website user (admin/editor/moderator). Everyone sees that a staff member
// is online (name + serial); their in-game location (map + coordinates) is gated
// on the `presence` feature's `location` field rule, which defaults to `staff`
// — the same admin/moderator set this used to hardcode. Non-staff players are
// never listed.
async function canSeeStaffLocation(req) {
const config = await visibility.getConfig()
const required = config.presence?.fields?.location || 'staff'
const level = req.viewerLevel || (await visibility.viewerLevel(req))
return visibility.meets(level, required)
}
async function getOnline(req, res) {
try {
const rows = await shardState.listOnlineLinked()
const showLocation = await canSeeStaffLocation(req)
return res.json(
rows.map((r) => {
const entry = { serial: r.serial, name: r.name }
if (showLocation) {
entry.map = r.map
entry.x = r.x
entry.y = r.y
entry.z = r.z
}
return entry
}),
)
} catch (err) {
log.error('shard.getOnline', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/idoc — houses currently in danger (stage IDOC).
//
// Projected: shapeHouse flattens the owner actor into `ownerSerial`/`ownerAcct`/
// `ownerName`, so this endpoint used to hand an anonymous caller the house
// owner's GAME ACCOUNT NAME. The public IDOC board only ever needed name, region
// and location — which is all that survives projection below `staff`.
async function getIdoc(req, res) {
try {
return res.json(await visibility.project('houses', await shardState.listIdoc(), req))
} catch (err) {
log.error('shard.getIdoc', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/champs — the current champion-spawn board (all categories).
// Served from our own store; live deltas (champ.update / champ.remove) arrive on
// the public SSE stream so the page can update in place.
async function getChamps(req, res) {
try {
return res.json(await visibility.project('champs', await shardState.listChamps(), req))
} catch (err) {
log.error('shard.getChamps', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/guilds — the current guild board. Served from our store;
// live via guild.update / guild.remove / guild.join on the public SSE stream.
//
// Projected: the stored payload is the raw guild.update frame, whose `leader`
// actor carries `acct` and `webId`. Those are admin-only and were previously
// returned verbatim to anonymous callers.
async function getGuilds(req, res) {
try {
return res.json(await visibility.project('guilds', await shardState.listGuilds(), req))
} catch (err) {
log.error('shard.getGuilds', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/governors — the current town-governor board (empty on shards
// without City Loyalty). Live via city.update on the public SSE stream. Projected
// for the same reason as getGuilds: `governor` / `governorElect` are actors.
async function getGovernors(req, res) {
try {
return res.json(await visibility.project('governors', await shardState.listGovernors(), req))
} catch (err) {
log.error('shard.getGovernors', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/governors/:city/history — the term ledger for one city
// (look-back: "who were all the governors of Britain?"), newest first.
async function getGovernorHistory(req, res) {
try {
const terms = await shardState.listGovernorHistory(req.params.city, req.query.limit)
return res.json(await visibility.project('governors', terms, req))
} catch (err) {
log.error('shard.getGovernorHistory', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/presence — the online-population aggregate (count + per-facet
// + per-region). Live via presence.online on the public SSE stream.
async function getPresence(req, res) {
try {
return res.json(await visibility.project('presence', await shardState.latestPresence(), req))
} catch (err) {
log.error('shard.getPresence', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/houses — PUBLIC view: only houses in danger (IDOC), and only
// their location (name + region + map/coords). Owner, price, co-owners and decay
// detail are staff-only (see admin GET /admin/shard/houses). Live via house.decay
// on the public SSE stream. This is the "where are the falling houses" board.
async function getHouses(req, res) {
try {
const idoc = await shardState.listIdoc()
const publicHouses = idoc.map((h) => ({
serial: h.serial,
name: h.name,
region: h.region,
map: h.map,
x: h.x,
y: h.y,
z: h.z,
isIdoc: true,
}))
// Already a hand-picked safe subset; projected anyway so an admin who
// tightens a `houses` field rule sees it honoured on every houses surface
// rather than on some of them.
return res.json(await visibility.project('houses', publicHouses, req))
} catch (err) {
log.error('shard.getHouses', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/ruleset — the shard's published ruleset (Protocol 3.0):
// expansion, which optional systems are on, skill/stat caps, account and house
// limits, champion scroll rules, the save/restart schedule. Served from our own
// store, so it renders while the shard is down; live via world.ruleset on the
// public SSE stream.
//
// `null` means the shard has never published one (an old plugin, or
// Bridge.RulesetEnabled=false) — a real answer, distinct from a published
// ruleset, and the page says so rather than rendering an empty one.
//
// Projected like every other shard read (§3.6.1's rule: a read path that returns
// shard data and does not call projectFeature is a bug). The `connect` string is
// the one configurable field — an operator who published a connect address may
// still want it behind a login.
async function getRuleset(req, res) {
try {
const ruleset = await shardState.getRuleset()
if (!ruleset) return res.json(null)
return res.json(await visibility.project('ruleset', ruleset, req))
} catch (err) {
log.error('shard.getRuleset', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// The shard keys boards by its own PointsType enum name (QueensLoyalty,
// CleanUpBritannia, …). Constrain the path param to that shape before it reaches
// the model: the column is VARCHAR(48), and an unbounded string here is a needless
// query on a value that can only ever be an identifier.
const SYSTEM_RE = /^[A-Za-z][A-Za-z0-9_]{0,47}$/
// GET /public/shard/points — every points/loyalty leaderboard the shard publishes.
// Served from our own store, so the page renders while the shard is down — which
// matters more here than for live state: these are standings accumulated over
// months, and blanking them during a restart would look like a data loss.
async function getPointsBoards(req, res) {
try {
const boards = await shardState.listPointsBoards()
return res.json(await visibility.project('leaderboards', boards, req))
} catch (err) {
log.error('shard.getPointsBoards', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/points/:system — one system's board.
//
// 404 for a system the shard has never published, matching the sidecar: "no such
// board" and "a board nobody is on yet" are different answers.
async function getPointsBoard(req, res) {
const { system } = req.params
if (!SYSTEM_RE.test(system)) return res.status(400).json({ message: 'Invalid points system.' })
try {
const board = await shardState.getPointsBoard(system)
if (!board) return res.status(404).json({ message: 'Unknown points system.' })
return res.json(await visibility.project('leaderboards', board, req))
} catch (err) {
log.error('shard.getPointsBoard', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// ── Marketplace (Protocol 3.0 vendor.listing) ──────────────────────────────
//
// The shard-wide player-vendor index. Served entirely from our own tables — the
// sidecar is never touched on this path — so shops stay browsable while the shard
// is down, labelled with how stale they may be.
//
// The staleness label is not decoration. The shard sweeps vendors round-robin, so
// a shop can legitimately be a full cycle behind; a page that implied live prices
// would send people to a vendor whose item sold twenty minutes ago.
// The serial spelling the bridge uses everywhere: "0x" and hex. Constrained
// before it reaches the model, like SYSTEM_RE above.
const SERIAL_RE = /^0x[0-9A-Fa-f]{1,16}$/
const intParam = (value) => {
const n = Number.parseInt(value, 10)
return Number.isFinite(n) ? n : undefined
}
// GET /public/shard/market — search the index.
//
// Returns LISTINGS, not vendors: "who sells a vanquishing kryss and for how much"
// is the question, and a vendor-shaped result would make every caller flatten the
// shops back out.
async function getMarket(req, res) {
try {
const page = await shardMarket.search({
q: typeof req.query.q === 'string' ? req.query.q : '',
minPrice: intParam(req.query.minPrice),
maxPrice: intParam(req.query.maxPrice),
itemId: intParam(req.query.itemId),
map: typeof req.query.map === 'string' ? req.query.map : '',
region: typeof req.query.region === 'string' ? req.query.region : '',
sort: typeof req.query.sort === 'string' ? req.query.sort : 'price_asc',
limit: intParam(req.query.limit) ?? 50,
offset: intParam(req.query.offset) ?? 0,
})
return res.json(await visibility.project('market', page, req))
} catch (err) {
log.error('shard.getMarket', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/market/meta — index size, staleness, and the filter options
// (which facets and regions actually hold vendors). Separate from the search so
// the page can build its filters without running a query it will throw away.
async function getMarketMeta(req, res) {
try {
return res.json(await visibility.project('market', await shardMarket.meta(), req))
} catch (err) {
log.error('shard.getMarketMeta', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/market/vendors/:serial — one shop and its listings.
//
// 404 for a serial the index has never seen, which also covers a vendor that has
// since been dismissed or hidden: to an anonymous caller "no such shop" is the
// only honest answer, and distinguishing the two would leak that a vendor exists
// but was hidden.
async function getMarketVendor(req, res) {
const { serial } = req.params
if (!SERIAL_RE.test(serial)) return res.status(400).json({ message: 'Invalid vendor serial.' })
try {
const vendor = await shardMarket.getVendor(serial, {
limit: intParam(req.query.limit) ?? 250,
offset: intParam(req.query.offset) ?? 0,
})
if (!vendor) return res.status(404).json({ message: 'Unknown vendor.' })
return res.json(await visibility.project('market', vendor, req))
} catch (err) {
log.error('shard.getMarketVendor', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/features — the shard features THIS caller can actually see,
// so the SPA (and the Android client) can hide nav entries instead of rendering
// links that 403. Deliberately reports only what the viewer may reach: the list
// itself must not disclose the existence of a feature they're gated out of.
async function getFeatures(req, res) {
try {
const config = await visibility.getConfig()
const level = await visibility.viewerLevel(req)
return res.json({ level, features: visibility.visibleFeatures(level, config) })
} catch (err) {
log.error('shard.getFeatures', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/stream — live-event SSE channel. What arrives depends on the
// caller's audience rung, resolved once at subscribe time; see shardBroadcast.js.
function stream(req, res) {
return broadcast.subscribe(req, res, 'public')
}
module.exports = {
getStatus,
getFeed,
getEconomy,
getOnline,
getIdoc,
getChamps,
getGuilds,
getGovernors,
getGovernorHistory,
getPresence,
getHouses,
getRuleset,
getPointsBoards,
getPointsBoard,
getMarket,
getMarketMeta,
getMarketVendor,
getFeatures,
stream,
}

253
server/src/router/v1/public/shard.router.js
View File

@@ -1,253 +0,0 @@
// Public · Shard — token-free, same-origin reads of the live shard. The
// status/feed/economy/idoc/champs/guilds/governors/presence/houses endpoints read
// the site's own ingested data; nothing here round-trips the sidecar per request.
//
// Mounted at /api/v1/public/shard by public/index.js. Deliberately NOT site-mode
// gated — shard status is useful (and wanted) while the site itself is in
// maintenance.
//
// **GET /shard/stream stays anonymous.** It is consumed by logged-out browser
// visitors *and* by the Android ShardStreamClient, neither of which sends an
// Authorization header; adding requireAuth here blacks out the public live boards
// on web and mobile. The sensitive kinds (staff audit, cheat detection, login
// attempts, IPs) are withheld by utils/shardBroadcast.js, not by a route gate —
// that per-frame filtering is the security boundary, not this file. /stream is
// deliberately NOT wrapped in requireFeature either: it spans every feature, and
// each frame is gated individually against the subscriber's rung.
//
// Every other route carries `requireFeature(<name>)` (utils/shardVisibility.js),
// which 404s when an admin has disabled the feature and 403s when the caller sits
// below its configured audience. Defaults reproduce pre-v3 behavior exactly, so
// these gates are inert until an admin changes something.
const express = require('express')
const { param, query } = require('express-validator')
const shard = require('./shard.controller')
const validate = require('../../../middleware/validate')
const { marketLimiter } = require('../../../middleware/rateLimit')
const { requireFeature } = require('../../../utils/shardVisibility')
const shardRouter = express.Router()
shardRouter.get(
'/status',
requireFeature('status'),
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'Shard connection state, online count and latest economy'
/* #swagger.responses[200] = { description: 'Shard status', content: { "application/json": { schema: { $ref: "#/components/schemas/ShardStatus" } } } } */
shard.getStatus,
)
shardRouter.get(
'/feed',
requireFeature('activity'),
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'Recent notable shard events (from the ingested log)'
// #swagger.description = 'The stored-history twin of /shard/stream, and it reaches the same verdict: which kinds are returned is resolved against the caller\'s audience rung under the live visibility config, and each event\'s payload is field-projected against its own kind\'s feature. Kinds the caller may not read are omitted (an explicit ?kind= for one of them returns []), and acct/webId never appear below admin.'
// #swagger.parameters['kind'] = { in: 'query', required: false, schema: { type: 'string' }, description: 'Filter to a single event kind, e.g. vendor.sale. Returns [] if the caller may not read that kind.' }
// #swagger.parameters['limit'] = { in: 'query', required: false, schema: { type: 'integer' }, description: 'Max rows (default 100, max 1000).' }
/* #swagger.responses[200] = { description: 'Events, newest first', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/ShardEvent" } } } } } */
query('kind').optional({ values: 'falsy' }).isString().isLength({ max: 48 }),
query('limit').optional().isInt({ min: 1, max: 1000 }),
validate,
shard.getFeed,
)
shardRouter.get(
'/economy',
requireFeature('status'),
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'Gold-supply time series (oldest → newest)'
// #swagger.parameters['limit'] = { in: 'query', required: false, schema: { type: 'integer' }, description: 'Max samples (default 100, max 1000).' }
/* #swagger.responses[200] = { description: 'Economy samples', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/ShardEconomyPoint" } } } } } */
query('limit').optional().isInt({ min: 1, max: 1000 }),
validate,
shard.getEconomy,
)
shardRouter.get(
'/online',
requireFeature('presence'),
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'Staff online now (linked staff accounts; location is admin/moderator-only)'
/* #swagger.responses[200] = { description: 'Online players', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/ShardOnlinePlayer" } } } } } */
shard.getOnline,
)
shardRouter.get(
'/idoc',
requireFeature('houses'),
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'Houses currently in danger (IDOC)'
// #swagger.description = 'Location-level board of the houses about to collapse. Owner identity and price are gated by the `houses` feature\'s field rules (default `staff`), and the owner\'s game account is admin-only always — so an anonymous caller sees name, region and coordinates only.'
/* #swagger.responses[200] = { description: 'IDOC houses', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/ShardHouse" } } } } } */
shard.getIdoc,
)
shardRouter.get(
'/champs',
requireFeature('champs'),
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'Current champion-spawn board (all categories)'
// #swagger.description = 'The live board of every champion / mini-champ / sea-boss spawn. Update in place via the champ.update / champ.remove frames on /shard/stream.'
/* #swagger.responses[200] = { description: 'Champion spawns, ordered by name', content: { "application/json": { schema: { type: "array", items: { type: "object", additionalProperties: true } } } } } */
shard.getChamps,
)
shardRouter.get(
'/guilds',
requireFeature('guilds'),
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'Current guild board (rosters, alliances, leaders)'
// #swagger.description = 'The live board of every guild. Update in place via the guild.update / guild.remove / guild.join frames on /shard/stream.'
/* #swagger.responses[200] = { description: 'Guilds, ordered by name', content: { "application/json": { schema: { type: "array", items: { type: "object", additionalProperties: true } } } } } */
shard.getGuilds,
)
shardRouter.get(
'/governors',
requireFeature('governors'),
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'Current town-governor board (City Loyalty)'
// #swagger.description = 'One entry per city with its governor and election phase. Empty if the shard does not run the City Loyalty system. Live via city.update on /shard/stream.'
/* #swagger.responses[200] = { description: 'Cities, ordered by name', content: { "application/json": { schema: { type: "array", items: { type: "object", additionalProperties: true } } } } } */
shard.getGovernors,
)
shardRouter.get(
'/governors/:city/history',
requireFeature('governors'),
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'Governor term history for a city'
// #swagger.parameters['city'] = { in: 'path', required: true, schema: { type: 'string' }, description: 'City name, e.g. Britain.' }
// #swagger.parameters['limit'] = { in: 'query', required: false, schema: { type: 'integer' }, description: 'Max terms (default 100, max 500).' }
/* #swagger.responses[200] = { description: 'Terms, newest first', content: { "application/json": { schema: { type: "array", items: { type: "object", additionalProperties: true } } } } } */
param('city').isString().isLength({ min: 1, max: 40 }),
query('limit').optional().isInt({ min: 1, max: 500 }),
validate,
shard.getGovernorHistory,
)
shardRouter.get(
'/presence',
requireFeature('presence'),
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'Online population aggregate (count + per-facet + per-region)'
// #swagger.description = 'The latest presence.online snapshot powering the "Players Online" widget. Live via presence.online on /shard/stream.'
/* #swagger.responses[200] = { description: 'Population snapshot', content: { "application/json": { schema: { type: "object", additionalProperties: true } } } } */
shard.getPresence,
)
shardRouter.get(
'/houses',
requireFeature('houses'),
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'House registry (owner, co-owners, price, decay)'
// #swagger.description = 'Every house seen via the house.update registry feed. `price` is the placement value, not a for-sale flag. Live via house.update / house.remove on /shard/stream.'
/* #swagger.responses[200] = { description: 'Houses, ordered by name', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/ShardHouse" } } } } } */
shard.getHouses,
)
shardRouter.get(
'/ruleset',
requireFeature('ruleset'),
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'The shard\'s published ruleset (expansion, systems, caps, limits)'
// #swagger.description = 'How this shard is actually configured, published by the shard itself as one world.ruleset frame: expansion, which optional systems are on, skill/stat caps, account and house limits, champion scroll rules and the save/restart schedule. Served from our own store, so it renders while the shard is down; live via world.ruleset on /shard/stream. Returns `null` if the shard has never published one (an older plugin, or Bridge.RulesetEnabled=false) — distinct from a published ruleset, and the page renders it differently.'
/* #swagger.responses[200] = { description: 'The ruleset, or null if never published', content: { "application/json": { schema: { type: "object", nullable: true, additionalProperties: true } } } } */
shard.getRuleset,
)
shardRouter.get(
'/points',
requireFeature('leaderboards'),
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'Points / loyalty leaderboards, one board per point system'
// #swagger.description = 'Every points/loyalty leaderboard the shard publishes (Queen\'s Loyalty, Void Pool, the nine city loyalties, Clean Up Britannia, …), each with its display name, max points, participant count and top N. Served from our own store, so it renders while the shard is down; live via points.board on /shard/stream. A board\'s display name may arrive as a literal (`nameString`) or a cliloc id (`nameNumber`) — resolve clilocs client-side.'
/* #swagger.responses[200] = { description: 'Boards, ordered by display name', content: { "application/json": { schema: { type: "array", items: { $ref: "#/components/schemas/ShardPointsBoard" } } } } } */
shard.getPointsBoards,
)
shardRouter.get(
'/points/:system',
requireFeature('leaderboards'),
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'One points system\'s leaderboard'
// #swagger.description = 'A single board by the shard\'s own PointsType name (e.g. `QueensLoyalty`, `CleanUpBritannia`). Returns 404 when the shard has never published that system — distinct from a published board that nobody has scored in yet, which returns 200 with an empty `top`.'
/* #swagger.parameters['system'] = { in: 'path', required: true, description: 'PointsType name, e.g. QueensLoyalty', schema: { type: 'string' } } */
/* #swagger.responses[200] = { description: 'The board', content: { "application/json": { schema: { $ref: "#/components/schemas/ShardPointsBoard" } } } } */
/* #swagger.responses[400] = { description: 'Malformed system name' } */
/* #swagger.responses[404] = { description: 'The shard has never published that system' } */
shard.getPointsBoard,
)
// ── Marketplace ────────────────────────────────────────────────────────────
//
// Rate-limited, unlike every other route in this file. These are the first
// genuinely expensive PUBLIC reads on the site — a LIKE scan plus a COUNT over
// what is typically the largest shard_* table, reachable with no session.
shardRouter.get(
'/market',
requireFeature('market'),
marketLimiter,
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'Search the player-vendor marketplace'
// #swagger.description = 'Every priced listing on every player vendor the shard publishes — the same index the in-game Vendor Search gump reads, and it honours the same per-vendor opt-out, so a player who hid their shop in game is hidden here too. Results are LISTINGS, each carrying enough of its shop to be actionable. Served from the site\'s own tables (the sidecar is not touched), so it renders while the shard is down; `staleAt` is the oldest vendor row and the page must say how far behind the index can be — the shard sweeps vendors round-robin, so prices are inherently up to one full cycle old. Item names are resolved server-side against the cliloc table (docs/website/CLILOCS.md); on a shard that has not configured one, `displayName` is null and clients render the item id.'
// #swagger.parameters['q'] = { in: 'query', required: false, schema: { type: 'string' }, description: 'Substring match on the resolved item name or the item\'s own literal name (max 60 chars).' }
// #swagger.parameters['minPrice'] = { in: 'query', required: false, schema: { type: 'integer' }, description: 'Lowest price to include.' }
// #swagger.parameters['maxPrice'] = { in: 'query', required: false, schema: { type: 'integer' }, description: 'Highest price to include.' }
// #swagger.parameters['itemId'] = { in: 'query', required: false, schema: { type: 'integer' }, description: 'Exact ItemID (art id) match, for "more like this".' }
// #swagger.parameters['map'] = { in: 'query', required: false, schema: { type: 'string' }, description: 'Limit to one facet. Facet names come from the shard\'s own data; an unknown one returns an empty page.' }
// #swagger.parameters['region'] = { in: 'query', required: false, schema: { type: 'string' }, description: 'Limit to one named region.' }
// #swagger.parameters['sort'] = { in: 'query', required: false, schema: { type: 'string', enum: ['price_asc','price_desc','recent'] }, description: 'Default price_asc. `recent` orders by when the shop was last seen.' }
// #swagger.parameters['limit'] = { in: 'query', required: false, schema: { type: 'integer' }, description: 'Page size, 1..100 (default 50).' }
// #swagger.parameters['offset'] = { in: 'query', required: false, schema: { type: 'integer' }, description: 'Rows to skip (default 0).' }
/* #swagger.responses[200] = { description: 'A page of listings plus the unpaginated total and the staleness stamp', content: { "application/json": { schema: { $ref: "#/components/schemas/ShardMarketPage" } } } } */
/* #swagger.responses[403] = { description: 'The market feature is gated above this caller', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[404] = { description: 'The market feature is disabled', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
/* #swagger.responses[429] = { description: 'Rate limited', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
query('q').optional({ values: 'falsy' }).isString().isLength({ max: 60 }),
query('minPrice').optional({ values: 'falsy' }).isInt({ min: 0, max: 999999999 }),
query('maxPrice').optional({ values: 'falsy' }).isInt({ min: 0, max: 999999999 }),
query('itemId').optional({ values: 'falsy' }).isInt({ min: 0, max: 65535 }),
query('map').optional({ values: 'falsy' }).isString().isLength({ max: 40 }),
query('region').optional({ values: 'falsy' }).isString().isLength({ max: 80 }),
query('sort').optional({ values: 'falsy' }).isIn(['price_asc', 'price_desc', 'recent']),
query('limit').optional().isInt({ min: 1, max: 100 }),
query('offset').optional().isInt({ min: 0, max: 100000 }),
validate,
shard.getMarket,
)
shardRouter.get(
'/market/meta',
requireFeature('market'),
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'Marketplace size, staleness and filter options'
// #swagger.description = 'How many vendors and listings the index holds, how stale it may be (`staleAt` = the oldest vendor row, `freshAt` = the newest), and which facets and regions actually hold vendors — so a client can build its filters without running a search it will discard.'
/* #swagger.responses[200] = { description: 'Marketplace metadata', content: { "application/json": { schema: { $ref: "#/components/schemas/ShardMarketMeta" } } } } */
shard.getMarketMeta,
)
shardRouter.get(
'/market/vendors/:serial',
requireFeature('market'),
marketLimiter,
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'One player vendor and everything it is selling'
// #swagger.description = 'A single shop by its vendor serial, with its listings. `truncated` (and `total` exceeding `count`) means the shop holds more than the shard publishes per frame — a commodity reseller with thousands of stacks is a real thing, and the page says so rather than presenting a partial shop as complete. Returns 404 for a serial the index has never seen, which also covers a vendor since dismissed or hidden.'
/* #swagger.parameters['serial'] = { in: 'path', required: true, description: 'Vendor serial, e.g. 0x40001234', schema: { type: 'string' } } */
// #swagger.parameters['limit'] = { in: 'query', required: false, schema: { type: 'integer' }, description: 'Listings to return, 1..500 (default 250).' }
// #swagger.parameters['offset'] = { in: 'query', required: false, schema: { type: 'integer' }, description: 'Listings to skip (default 0).' }
/* #swagger.responses[200] = { description: 'The vendor', content: { "application/json": { schema: { $ref: "#/components/schemas/ShardMarketVendor" } } } } */
/* #swagger.responses[400] = { description: 'Malformed vendor serial' } */
/* #swagger.responses[404] = { description: 'No such vendor in the index' } */
param('serial').isString().isLength({ max: 20 }),
query('limit').optional().isInt({ min: 1, max: 500 }),
query('offset').optional().isInt({ min: 0, max: 100000 }),
validate,
shard.getMarketVendor,
)
shardRouter.get(
'/features',
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'Shard features visible to the caller (drives client nav)'
// #swagger.description = 'The caller\'s audience rung plus the shard features they may reach, so a client can hide nav entries instead of rendering links that 403. Reports only what the caller can see — the list itself does not disclose gated features.'
/* #swagger.responses[200] = { description: 'Visible features', content: { "application/json": { schema: { $ref: "#/components/schemas/ShardFeatures" } } } } */
shard.getFeatures,
)
shardRouter.get(
'/stream',
// #swagger.tags = ['Public · Shard']
// #swagger.summary = 'Live shard event stream (Server-Sent Events, filtered by audience)'
// #swagger.description = 'text/event-stream of live events. The caller\'s audience rung is resolved once at subscribe time and frozen for the connection; each frame is then gated on its feature and field-projected, so sensitive kinds and fields (staff audit, cheat detection, login attempts, IPs, acct/webId) never reach a caller below their configured rung.'
/* #swagger.responses[200] = { description: 'An SSE stream (Content-Type: text/event-stream).' } */
shard.stream,
)
module.exports = shardRouter

96
server/src/server.js
View File

@@ -4,19 +4,13 @@ const http = require('http')
const app = require('./app')
const internalApp = require('./internalApp')
const botScore = require('./middleware/botScore')
const uoLinkSocket = require('./utils/uoLinkSocket')
const uoLinkClient = require('./utils/uoLinkClient')
const uoLinkConfig = require('./model/uoLinkConfig/uoLinkConfig.model')
const shardBroadcast = require('./utils/shardBroadcast')
const announceWorker = require('./utils/announceWorker')
const { ensureSchema, close } = require('./utils/db')
const { seedDefaults, createInitialAdminFromEnv } = require('../db/seed')
const settings = require('./model/settings/settings.model')
const revokedSessions = require('./model/revokedSessions/revokedSessions.model')
const mobileAuthBridge = require('./model/mobileAuthBridge/mobileAuthBridge.model')
const shardAtlas = require('./model/shardAtlas/shardAtlas.model')
const shardClilocs = require('./model/shardClilocs/shardClilocs.model')
const shardMarket = require('./model/shardMarket/shardMarket.model')
const moduleLifecycle = require('./modules/lifecycle')
const createLogger = require('./utils/logger')
const { evaluateBotInternalKey } = require('./utils/botInternalKey')
const brand = require('./config/brand')
@@ -80,35 +74,25 @@ async function start() {
log.warn('mobile-auth-bridge prune failed', { error: err.message })
}
// Re-derive the spawn atlas from the shard's own ServUO tree. The shard's maps
// change over its lifetime — facets get added, replaced or renamed — so the
// atlas is rebuilt on every boot rather than shipped as a snapshot that would
// silently go stale. Hash-gated, so an unchanged tree costs one read pass and
// no database write.
//
// Best-effort by contract: no configured path, an unreadable mount or a
// malformed file must never stop the site coming up. A refresh that would
// REMOVE a facet is staged for admin approval instead of being applied.
await shardAtlas.refreshOnBoot()
// Refresh the cliloc table (UO's id → display-string map) from the file the
// operator converted out of their own client. Same contract as the atlas:
// hash-gated so an unchanged file costs one read, and best-effort so a missing
// or wrong-format file never stops the site coming up — it just means item
// names render as ids, which is what they did before the table existed.
const clilocResult = await shardClilocs.refreshOnBoot()
// A cliloc import changes what item names RESOLVE to, and the marketplace
// stores those names denormalized (shard_vendor_items.display_name) so it can
// index and search them. The shard's market sweep will not re-send an unchanged
// shop just because the site learned what its items are called, so the backfill
// has to be pulled rather than waited for. Only after an actual import — the
// common boot is hash-gated to a no-op and must stay one.
if (clilocResult && clilocResult.status === 'imported') await shardMarket.refreshDisplayNames()
const mode = await settings.get('site_mode')
log.info(`site mode: ${String(mode || 'live').toUpperCase()}`)
// Reconcile installed_modules with what the loader found on the volume at
// require time, then run each module's onBoot (MODULE_API.md §2.5).
//
// This is where the shard now warms up. Core used to do it inline just above —
// rebuild the spawn atlas from the ServUO tree, refresh the cliloc table, open
// the uo-link WebSocket — and all of it is module-uo's `onBoot` since Phase 3.
// The ordering guarantee is unchanged and is why it belongs here rather than
// after the listener: the tables exist by now, and nothing is served until the
// warm-up finishes. Placed
// after core's own boot work and BEFORE the listener binds, for both reasons
// the contract gives: a module's warm-up may depend on core being up, and a
// module that must not serve traffic until it has warmed a cache gets that
// guarantee only if nothing is listening yet. Never throws — a module that
// fails here keeps its URLs and answers 503.
await moduleLifecycle.boot()
const server = http.createServer(app)
server.listen(PORT, HOST, () => {
log.info(`listening on http://${HOST}:${PORT} (API at /api/v1, health at /api/health)`)
@@ -122,17 +106,6 @@ async function start() {
log.info(`internal API listening on http://${HOST}:${INTERNAL_PORT} (server<->bot only — do NOT proxy)`)
})
// Start the uo-link WebSocket ingest client. Self-guards: it only actually
// connects when the admin has enabled the integration and saved a token, so
// this is a no-op on shards that haven't configured the sidecar. Never let a
// sidecar problem block server startup.
try {
await uoLinkSocket.start()
await checkUoLink()
} catch (err) {
log.warn('uo-link socket failed to start (continuing)', { error: err.message })
}
// Start the news-announcement dispatcher: a light in-process poller that pushes
// published news posts to the in-game town crier + Discord with independent
// retry per leg. No-op until a news post is actually published.
@@ -141,43 +114,20 @@ async function start() {
setupShutdown(server, internalServer)
}
// Best-effort startup probe of the uo-link sidecar: if the integration is
// enabled, log whether it is reachable and warn loudly on a protocol mismatch
// (fail-fast visibility rather than silently mis-parsing a newer wire format).
async function checkUoLink() {
const config = await uoLinkConfig.getSafe()
if (!config.enabled) return
const health = await uoLinkClient.health()
if (!health.ok) {
log.warn('uo-link is enabled but the sidecar is unreachable at startup', {
baseUrl: config.baseUrl,
error: health.error || `status ${health.status}`,
})
return
}
if (health.data && health.data.protocol && health.data.protocol !== config.protocol) {
log.error('uo-link PROTOCOL MISMATCH — pinned vs sidecar', {
pinned: config.protocol,
sidecar: health.data.protocol,
})
} else {
log.info('uo-link sidecar reachable', {
pluginConnected: health.data && health.data.plugin_connected,
protocol: health.data && health.data.protocol,
})
}
}
function setupShutdown(server, internalServer) {
let closing = false
const shutdown = async (signal) => {
if (closing) return
closing = true
log.warn(`${signal} received — shutting down gracefully`)
// Modules first, while everything they were handed still works: the database
// pool, the push dispatcher and the SSE fan-out are all still open here, and
// a module's onShutdown is the only chance it gets to flush through them
// (MODULE_API.md §2.5). Each hook is budgeted, so one that will not let go
// costs five seconds rather than the whole shutdown.
await moduleLifecycle.shutdown()
botScore.stopSweeper() // stop the bot-store cleanup interval
announceWorker.stop() // stop the news-announcement dispatcher poller
uoLinkSocket.stop() // close the uo-link WS ingest client
shardBroadcast.closeAll() // end any open shard live-feed SSE streams
server.close(() => log.info('http server closed'))
if (internalServer) internalServer.close(() => log.info('internal http server closed'))
try {

96
server/src/utils/announceWorker.js
View File

@@ -2,59 +2,35 @@
//
// A lightweight, in-process table poller (no Redis/BullMQ in the stack). Every
// ANNOUNCE_POLL_MS it sweeps announce_jobs for legs that are due — freshly
// enqueued or past their backoff — and dispatches each one:
// • town crier → uoLinkClient.postTownCrier (sidecar → in-game)
// • discord → botInternalClient.announce (bot → #news channel)
// Both clients never throw (they return { ok, status, error }); the model turns
// each result into done / retry / terminal and owns the backoff + rollup. One
// leg failing never touches the other. Same setInterval + unref + stop() shape
// as middleware/botScore's sweeper, wired into server.js start/shutdown.
// enqueued or past their backoff — and dispatches each one through the leg that
// registered itself for that id (modules/registries.js). Core registers
// `discord`; module-uo registers `towncrier`; another game's module registers its
// own, and nothing in this file changes.
//
// A leg's client never throws (they return { ok, status, error }) and its
// classify() turns that into done / retry / terminal, which the model converts to
// backoff + rollup. One leg failing never touches another. Same setInterval +
// unref + stop() shape as middleware/botScore's sweeper, wired into server.js
// start/shutdown.
const announceJobs = require('../model/announceJobs/announceJobs.model')
const announceJobsDb = require('../model/announceJobs/announceJobs.db')
const logic = require('../model/announceJobs/announceJobs.logic')
const posts = require('../model/posts/posts.model')
const uoLinkClient = require('./uoLinkClient')
const botInternalClient = require('./botInternalClient')
const registries = require('../modules/registries')
const log = require('./logger')('announce-worker')
const POLL_MS = Number(process.env.ANNOUNCE_POLL_MS) || 15_000
const TOWNCRIER_DURATION_SEC = Number(process.env.TOWNCRIER_DURATION_SEC) || 3600
function baseUrl() {
return (process.env.APP_BASE_URL || 'http://localhost:5173').replace(/\/+$/, '')
}
// ── Leg dispatchers ─────────────────────────────────────────────────────────
// Return the raw client result ({ ok, status, data, error }); classification is
// the model/logic's job.
async function dispatchTownCrier(post) {
const lines = logic.buildTownCrierText(post, { baseUrl: baseUrl() })
// Stable id: re-posting `post-<id>` REPLACES the prior town-crier entry rather
// than stacking a duplicate, so a retry after a partial failure is safe.
return uoLinkClient.postTownCrier({
id: `post-${post.id}`,
lines,
durationSec: TOWNCRIER_DURATION_SEC,
})
}
async function dispatchDiscord(post) {
const base = baseUrl()
// Stored image paths are relative ("/uploads/x.png"); Discord embeds need an
// absolute URL.
const imageUrl = post.image_url ? new URL(post.image_url, base).toString() : null
return botInternalClient.announce({
title: post.title,
excerpt: post.excerpt,
url: `${base}/site/news`,
imageUrl,
})
}
// Process a single due leg of a job: fetch the post, dispatch, classify, record.
async function processLeg(job, leg) {
const registered = registries.announceLeg(leg)
if (!registered) {
// A row for a leg nobody registers any more (its module was removed). Leave
// it alone: failing it would make the job roll up terminal on the strength of
// a leg that no longer exists, and reinstalling the module should resume it.
return
}
const post = await posts.getById(job.post_id)
if (!post) {
// Post was deleted between enqueue and dispatch (the CASCADE usually reaps
@@ -63,16 +39,9 @@ async function processLeg(job, leg) {
return
}
let result
let classification
try {
if (leg === 'towncrier') {
result = await dispatchTownCrier(post)
classification = logic.classifyTownCrier(result)
} else {
result = await dispatchDiscord(post)
classification = logic.classifyDiscord(result)
}
classification = registered.classify(await registered.dispatch(post))
} catch (err) {
// Clients shouldn't throw, but if one does, treat it as a transient failure
// rather than crashing the tick.
@@ -83,11 +52,10 @@ async function processLeg(job, leg) {
await announceJobs.recordOutcome(job, leg, classification)
}
// One sweep: find due jobs and process each due leg. A job may have both legs due
// (a fresh enqueue) — process the ones that are actually pending. `job` is a
// snapshot from the SELECT; recordOutcome re-reads for the rollup, so processing
// the two legs sequentially off the same snapshot is fine (each leg only writes
// its own columns).
// One sweep: find due jobs and process each due leg. A job may have several legs
// due at once (a fresh enqueue). `job` is a snapshot from the SELECT;
// recordOutcome re-reads for the rollup, so processing the legs sequentially off
// the same snapshot is fine (each leg only writes its own row).
async function tick(now = new Date()) {
let jobs
try {
@@ -99,15 +67,15 @@ async function tick(now = new Date()) {
if (!jobs || jobs.length === 0) return
for (const job of jobs) {
if (isLegDue(job, 'towncrier', now)) await processLeg(job, 'towncrier')
if (isLegDue(job, 'discord', now)) await processLeg(job, 'discord')
for (const row of job.legs || []) {
if (isLegDue(row, now)) await processLeg(job, row.leg)
}
}
}
function isLegDue(job, leg, now) {
if (job[`${leg}_status`] !== 'pending') return false
const next = job[`${leg}_next_attempt_at`]
return next == null || new Date(next) <= now
function isLegDue(row, now) {
if (!row || row.status !== 'pending') return false
return row.next_attempt_at == null || new Date(row.next_attempt_at) <= now
}
let timer = null
@@ -118,7 +86,7 @@ function start() {
tick().catch((err) => log.error('announce tick failed', { message: err.message }))
}, POLL_MS)
if (timer.unref) timer.unref() // don't keep the event loop alive (tests, shutdown)
log.info('announcement dispatcher started', { pollMs: POLL_MS })
log.info('announcement dispatcher started', { pollMs: POLL_MS, legs: registries.announceLegIds() })
return timer
}
@@ -129,4 +97,4 @@ function stop() {
}
}
module.exports = { start, stop, tick, processLeg, dispatchTownCrier, dispatchDiscord }
module.exports = { start, stop, tick, processLeg, isLegDue }

287
server/src/utils/clilocParse.js
View File

@@ -1,287 +0,0 @@
// Cliloc parsing — the pure half.
//
// A "cliloc" is UO's localization table: an integer id mapped to a display
// string. Items carry a `LabelNumber` rather than a name, so without this table
// the site can only render `id 1023721` where the game shows "quarter staff".
// The shard already sends the id on every equipment entry (`char.profile`'s
// `cliloc` field) and will send one per marketplace listing — the *number* was
// never the missing piece, the *table* was.
//
// This module is fs-free on purpose, exactly like `spawnAtlasParse.js`: the
// suite runs in CI where there is no UO client, so every parser here is driven
// from inline fixtures. `clilocSource.js` is the only thing that touches disk.
//
// ── Two input formats, and why ─────────────────────────────────────────────
//
// The client's own `Cliloc.enu` is COMPRESSED (Mythic format) on any modern
// client, and decompressing it is a bit-level port of an inverse-BWT coder that
// nothing in this stack needs at runtime. ServUO's own bundled `Ultima.StringList`
// cannot read it either — which is why `VendorSearch.GetItemName` is already inert
// on such a shard and the plugin could not supply names even if we asked it to.
//
// So the operator converts once, from their own client, and points the site at
// the result (see docs/website/CLILOCS.md). Two shapes are accepted because
// different tools produce different things:
//
// • PLAIN BINARY — the pre-compression cliloc layout: a 6-byte header, then
// records of {int32 number, byte flag, uint16 length, UTF-8 bytes}.
// • DELIMITED TEXT — `number<TAB|,|;>text` per line, which is what the common
// GUI exports emit. Quoted CSV fields and a header row are tolerated.
//
// Nothing derived from the client is ever committed: the converted file lives at
// an operator-supplied path and is gitignored, the same rule the spawn atlas art
// map already follows.
/** Raised for a file we can identify but deliberately refuse to guess at. */
class ClilocFormatError extends Error {
constructor(message, code) {
super(message)
this.name = 'ClilocFormatError'
this.code = code
}
}
/**
* Bumped when this parser produces DIFFERENT data from an IDENTICAL source file.
*
* Stored beside the source hash so the boot path can tell "same file, but the
* parser moved on" from "same file, nothing to do". Without it a corrected parse
* would ship and never reach an install whose cliloc file never changes — the
* trap `spawnAtlasSource.PARSER_VERSION` documents.
*/
const PARSER_VERSION = 1
// The plain layout's header is `02 00 00 00 01 00` — a 4-byte version and a
// 2-byte language marker. Only the size matters for parsing; the values are
// checked to sniff the format, not to validate it.
const HEADER_BYTES = 6
const RECORD_HEADER_BYTES = 7 // int32 number + byte flag + uint16 length
// Every compressed cliloc file the client ships begins with a DWORD whose high
// byte is 0x8E (the XOR key UOFiddler calls `HeaderXorKey`, 0x8E2C9A3D). That is
// the single cheapest way to tell an operator they exported the wrong file —
// without it, the plain parser happily reads compressed bytes as ~19k records of
// negative ids and 60 KB "strings" before dying somewhere in the middle, and the
// resulting error names the wrong problem.
const MYTHIC_HIGH_BYTE = 0x8e
/** True when `buffer` is a Mythic-compressed cliloc rather than the plain layout. */
function isCompressedCliloc(buffer) {
return buffer.length >= 4 && buffer[3] === MYTHIC_HIGH_BYTE
}
/**
* Parse the plain binary cliloc layout.
*
* Strict about truncation, and that strictness is load-bearing: a half-copied or
* partly-written file is the realistic failure here, and it must fail loudly
* rather than import a silently short table that then renders half the world as
* `id 1023721`. A record that runs past the end of the buffer throws.
*/
function parseClilocBinary(buffer) {
if (!Buffer.isBuffer(buffer)) throw new ClilocFormatError('Not a buffer', 'NOT_BUFFER')
if (isCompressedCliloc(buffer)) {
throw new ClilocFormatError(
'This is a compressed (Mythic-format) cliloc file, which the site cannot read. ' +
'Convert it to the plain format first — see docs/website/CLILOCS.md.',
'COMPRESSED',
)
}
if (buffer.length < HEADER_BYTES) {
throw new ClilocFormatError('File is shorter than a cliloc header', 'TRUNCATED')
}
const entries = []
let offset = HEADER_BYTES
while (offset < buffer.length) {
if (offset + RECORD_HEADER_BYTES > buffer.length) {
throw new ClilocFormatError(
`Truncated record header at byte ${offset} (${entries.length} entries read)`,
'TRUNCATED',
)
}
const number = buffer.readInt32LE(offset)
const flag = buffer.readUInt8(offset + 4)
// The length is written by the client as an unsigned 16-bit value. Reading it
// signed (as ServUO's own SDK does) turns any string over 32 KB into a
// negative length; real tables top out around 12 KB, so this has no effect on
// current data and costs nothing to get right.
const length = buffer.readUInt16LE(offset + 5)
offset += RECORD_HEADER_BYTES
if (offset + length > buffer.length) {
throw new ClilocFormatError(
`Truncated record body at byte ${offset} (${entries.length} entries read)`,
'TRUNCATED',
)
}
entries.push({ number, flag, text: buffer.toString('utf8', offset, offset + length) })
offset += length
}
return entries
}
// A delimited line splits on the FIRST separator only: cliloc text is full of
// commas ("a scroll of magery, unfinished") and splitting on all of them would
// truncate every such entry at its first comma.
const TEXT_SEPARATORS = ['\t', ',', ';']
/** Unwrap one CSV field: strip surrounding quotes and unescape doubled quotes. */
function unquote(value) {
const trimmed = value.trim()
if (trimmed.length >= 2 && trimmed.startsWith('"') && trimmed.endsWith('"')) {
return trimmed.slice(1, -1).replace(/""/g, '"')
}
return trimmed
}
/**
* Parse a delimited text export: `number<sep>text` per line.
*
* Tolerant by design — this is whatever an operator's GUI tool produced, not a
* format we control. A header row, blank lines, `#` comments and a trailing
* flags column are all ignored. A line whose first field is not an integer is
* skipped rather than fatal, because that is exactly what a header row is.
*
* The one thing it will NOT do is return an empty table quietly: a file that
* yields no entries at all is a wrong file, not an empty one.
*/
function parseClilocText(text) {
const entries = []
for (const line of String(text).split(/\r?\n/)) {
// The line is deliberately NOT trimmed before the separator search. Roughly
// half of a real cliloc table is empty strings (unused ids), which export as
// `1005008<TAB>` — and trimming eats that trailing separator, leaving a bare
// number that then looks like a header row and is skipped. That silently
// dropped 55,994 of 123,490 entries. Individual FIELDS are trimmed instead,
// by `unquote`.
if (line.trim() === '' || line.trimStart().startsWith('#')) continue
// Pick the separator that actually appears first, so a tab-delimited line
// whose text contains a comma still splits on the tab.
let cut = -1
for (const sep of TEXT_SEPARATORS) {
const at = line.indexOf(sep)
if (at !== -1 && (cut === -1 || at < cut)) cut = at
}
if (cut === -1) continue
// An EMPTY first field must not become id 0: `Number('')` is 0, not NaN, so
// a line that merely starts with a separator would otherwise import as a
// bogus cliloc 0 instead of being skipped.
const head = unquote(line.slice(0, cut))
if (head === '') continue
const number = Number(head)
if (!Number.isInteger(number)) continue // header row, or a wrapped line
let rest = line.slice(cut + 1)
// Some exports carry `number,flag,text`. A bare integer in the second field
// is a flag; anything else is the text itself (and a text field that IS just
// a number is indistinguishable, so it stays as the text — the safer miss).
let flag = 0
for (const sep of TEXT_SEPARATORS) {
const at = rest.indexOf(sep)
if (at === -1) continue
const head = unquote(rest.slice(0, at))
if (/^\d{1,3}$/.test(head) && rest.slice(at + 1).trim() !== '') {
flag = Number(head)
rest = rest.slice(at + 1)
}
break
}
entries.push({ number, flag, text: unquote(rest) })
}
if (entries.length === 0) {
throw new ClilocFormatError('No cliloc entries found in the text export', 'EMPTY')
}
return entries
}
/**
* Parse either supported shape, sniffing which one this is.
*
* The sniff is on the binary header rather than the file extension: operators
* name these things whatever they like, and an `.enu` that is really a TSV (or a
* `.txt` that is really binary) should still import.
*/
function parseCliloc(buffer) {
const buf = Buffer.isBuffer(buffer) ? buffer : Buffer.from(buffer)
if (isCompressedCliloc(buf)) {
throw new ClilocFormatError(
'This is a compressed (Mythic-format) cliloc file, which the site cannot read. ' +
'Convert it to the plain format first — see docs/website/CLILOCS.md.',
'COMPRESSED',
)
}
// The plain layout always opens with version 2 / language 1. Anything else is
// treated as text, which is the recoverable guess: a mis-sniffed text file
// yields "no entries found", while a mis-sniffed binary yields nonsense.
if (buf.length >= HEADER_BYTES && buf.readInt32LE(0) === 2 && buf.readUInt16LE(4) === 1) {
return parseClilocBinary(buf)
}
return parseClilocText(buf.toString('utf8'))
}
// ── Display ────────────────────────────────────────────────────────────────
// Cliloc strings interpolate arguments the client supplies out of an item's
// property list: `~1_val~`, `~2_NAME~`, `~1_ITEM~`. We never have those — the
// bridge sends the id, not the packet — so a name carrying them must be reduced
// to what is actually knowable rather than shown with the raw tokens in it.
const PLACEHOLDER_RE = /~\d+_[^~]*~/g
/**
* Reduce a raw cliloc string to something displayable.
*
* Placeholders are dropped and the leftover punctuation tidied, so
* `"[~1_stuff~]"` becomes `""` (correctly nothing — the whole string was the
* argument) and `"cold damage ~1_val~%"` becomes `"cold damage"`.
*
* **Punctuation is only tidied when a placeholder was actually removed.** The
* trailing `%` above is the unit belonging to the number we never had, and the
* brackets in `[~1_stuff~]` only ever wrapped the argument — but a string with
* no placeholder has no such debris, and trimming it anyway corrupts real names.
* A shard's `"Runic Gateway Sigil (v2)"` came back as `"(v2"` while this was
* unconditional.
*
* Returns `''` when nothing survives, which callers treat as "no name" and fall
* back to the item id — better than showing a bracket.
*/
const DEBRIS = /^[\s\-–—,.;:%[\]()]+|[\s\-–—,.;:%[\]()]+$/g
function displayText(raw) {
if (raw == null) return ''
const source = String(raw)
const hadPlaceholder = PLACEHOLDER_RE.test(source)
PLACEHOLDER_RE.lastIndex = 0 // the regex is global; `test` advances it
if (!hadPlaceholder) return source.replace(/\s+/g, ' ').trim()
return source
.replace(PLACEHOLDER_RE, ' ')
.replace(/\s+/g, ' ')
.replace(/\s+([,.;:!?])/g, '$1')
.replace(DEBRIS, '')
.trim()
}
/** True when a raw cliloc string is nothing but interpolated arguments. */
const isPlaceholderOnly = (raw) => raw != null && String(raw).trim() !== '' && displayText(raw) === ''
module.exports = {
ClilocFormatError,
PARSER_VERSION,
HEADER_BYTES,
isCompressedCliloc,
parseCliloc,
parseClilocBinary,
parseClilocText,
displayText,
isPlaceholderOnly,
}

316
server/src/utils/clilocSource.js
View File

@@ -1,316 +0,0 @@
// Cliloc table — the filesystem layer.
//
// `clilocParse.js` holds the pure parsers; this module is the only thing that
// touches cliloc files on disk, and it is shared by both callers:
//
// - the server, which refreshes the table on boot (`shardClilocs.model.js`)
// - the admin panel, which can force a reimport without a restart
//
// The files are the OPERATOR'S (see docs/website/CLILOCS.md). Nothing derived
// from them is committed: the repo holds no string table, exactly as it holds no
// map snapshot and no artwork. That rule is why this module reads a configured
// path instead of a path inside the repo.
//
// ── Why this reads a SET of files, not one ────────────────────────────────
//
// Shards edit items and add new ones. Those carry cliloc ids that a stock client
// table does not have — and forcing a 5 MB client re-export every time an
// operator adds one item would be miserable enough that the table would simply
// go stale, which is the exact failure the spawn atlas was redesigned to avoid.
//
// So this mirrors `spawnAtlasSource.readSources()`: a BASE table (the converted
// client file) plus every operator-maintained OVERLAY beside it, all re-read on
// every boot and hash-gated as a SET. Adding, editing or removing any overlay
// counts as drift and re-imports. Later sources win, so an overlay both adds new
// ids and overrides stock ones.
//
// Measured on a real shard: the script tree references 16,434 cliloc ids and only
// 37 are absent from the stock client table. Tens of entries against a 67k base
// is what makes the overlay the right shape rather than a second full table.
//
// Reading and hashing ~5 MB costs a few milliseconds and a full parse ~50 ms, so
// the boot path hashes first and only parses when something actually changed.
const crypto = require('crypto')
const fs = require('fs')
const path = require('path')
const { ClilocFormatError, PARSER_VERSION, parseCliloc, isCompressedCliloc } = require('./clilocParse')
/**
* Filenames looked for as the BASE table when the configured path is a directory.
*
* Ordered by how specific they are: an explicitly converted file wins over
* something that merely sits in a client folder, so an operator who dropped a
* `cliloc.plain.enu` next to the original compressed `cliloc.enu` gets the one
* they made rather than the one that will be rejected.
*
* Matching is case-insensitive against the real directory listing, because the
* client ships `Cliloc.enu` on Windows and the site usually runs on Linux, where
* a hardcoded lowercase open would simply miss.
*/
const CANDIDATE_NAMES = [
'clilocs.tsv',
'clilocs.csv',
'clilocs.plain',
'cliloc.plain',
'cliloc.plain.enu',
'cliloc.enu.plain',
'clilocs.txt',
'cliloc.enu',
]
/**
* Where shard-specific additions and overrides live: a `custom/` directory
* beside the base table.
*
* ServUO has **no server-side convention** for custom clilocs — they live in the
* patched client file a shard distributes to its players, and nothing in the
* tree declares them. There is therefore nothing to discover, and this is the
* one place in the cliloc pipeline that is a convention we chose rather than one
* the shard already has. It is a directory rather than a single file so an
* operator can keep additions grouped however they like (per system, per patch)
* without the site caring.
*/
const CUSTOM_DIR = 'custom'
const CUSTOM_EXTENSIONS = ['.tsv', '.csv', '.txt', '.enu', '.plain']
class ClilocSourceError extends Error {
constructor(message, code) {
super(message)
this.name = 'ClilocSourceError'
this.code = code
}
}
function sha256(buffer) {
return crypto.createHash('sha256').update(buffer).digest('hex')
}
/**
* Resolve the configured path to `{ root, base }`.
*
* Accepts either a direct file path or a directory to search, because operators
* reasonably supply both — "here is the file" and "here is the folder I put it
* in" are equally natural answers to the admin panel's prompt. When it is a
* file, `root` is the directory CONTAINING it, so overlays work either way: an
* operator who pointed at a file should not have to re-point at its folder just
* to add a `custom/` directory next to it.
*/
function resolveBase(configured) {
if (!configured || String(configured).trim() === '') {
throw new ClilocSourceError('No cliloc path configured', 'NO_PATH')
}
const target = String(configured).trim()
let stat
try {
stat = fs.statSync(target)
} catch {
throw new ClilocSourceError(`Cliloc path does not exist: ${target}`, 'NOT_FOUND')
}
if (stat.isFile()) return { root: path.dirname(target), base: target }
if (!stat.isDirectory()) {
throw new ClilocSourceError(`Cliloc path is neither a file nor a directory: ${target}`, 'NOT_FOUND')
}
let listing
try {
listing = fs.readdirSync(target)
} catch {
throw new ClilocSourceError(`Cliloc directory is not readable: ${target}`, 'NOT_FOUND')
}
const byLower = new Map(listing.map((name) => [name.toLowerCase(), name]))
for (const candidate of CANDIDATE_NAMES) {
const actual = byLower.get(candidate)
if (actual) return { root: target, base: path.join(target, actual) }
}
throw new ClilocSourceError(
`No cliloc file found in ${target} (looked for ${CANDIDATE_NAMES.join(', ')})`,
'NO_FILE',
)
}
/** Overlay files under `<root>/custom/`, sorted so precedence is deterministic. */
function listCustom(root) {
const dir = path.join(root, CUSTOM_DIR)
let listing
try {
listing = fs.readdirSync(dir, { withFileTypes: true })
} catch (err) {
// No overlay directory is the normal case, not an error.
if (err.code === 'ENOENT' || err.code === 'ENOTDIR') return []
throw new ClilocSourceError(`Cliloc overlay directory is not readable: ${dir}`, 'UNREADABLE')
}
return listing
.filter((e) => e.isFile() && CUSTOM_EXTENSIONS.includes(path.extname(e.name).toLowerCase()))
.map((e) => e.name)
.sort()
.map((name) => path.join(dir, name))
}
function readFileOrThrow(file) {
try {
return fs.readFileSync(file)
} catch {
throw new ClilocSourceError(`Cliloc file is not readable: ${file}`, 'UNREADABLE')
}
}
/**
* Read every cliloc source under the configured path.
*
* Returns `{ root, files: [{ label, kind, file, buffer, sha256, bytes, compressed }] }`
* with the base first and overlays after, in the order they must be merged.
*
* Labels are root-relative and forward-slashed so a hash map compares equal
* across platforms — the same directory read on Windows and Linux must produce
* the same fingerprint, or every boot would look like a change. (The same
* reasoning, and the same bug, as `spawnAtlasSource.readSources`.)
*/
function readSources(configured) {
const { root, base } = resolveBase(configured)
const describe = (file, kind) => {
const buffer = readFileOrThrow(file)
return {
label: path.relative(root, file).split(path.sep).join('/'),
kind,
file,
buffer,
sha256: sha256(buffer),
bytes: buffer.length,
compressed: isCompressedCliloc(buffer),
}
}
const files = [describe(base, 'base')]
for (const overlay of listCustom(root)) files.push(describe(overlay, 'custom'))
return { root, files }
}
/**
* A fingerprint of every source: `{ "<label>": "<sha256>" }`, plus the base's
* details for the admin panel.
*
* `compressed` is reported here rather than left to the parse because the admin
* panel calls this and NOT `readCliloc` (parsing 5 MB on every status poll would
* be wasteful). Without it, pointing the setting at an unconverted client
* directory reports a perfectly readable file with pending drift — "ready to
* import" — and the operator only learns otherwise when the import fails. The
* check is four bytes of a buffer already in hand.
*/
function hashSources(configured) {
const { root, files } = readSources(configured)
const hashes = {}
for (const file of files) hashes[file.label] = file.sha256
const base = files[0]
return {
root,
hashes,
file: base.file,
bytes: base.bytes,
compressed: files.some((f) => f.compressed),
customCount: files.length - 1,
}
}
/** True when two source fingerprints describe the same set of files. */
function sameSources(a, b) {
if (!a || !b) return false
const aKeys = Object.keys(a).sort()
const bKeys = Object.keys(b).sort()
if (aKeys.length !== bKeys.length) return false
return aKeys.every((key, i) => key === bKeys[i] && a[key] === b[key])
}
/**
* Labels present in `loaded` that are absent from `current`.
*
* This is the multi-source hazard that a single file did not have. One corrupt
* file fails the parse loudly, but a source that has simply VANISHED — an
* unmounted volume, a half-copied deploy — parses perfectly and imports a table
* quietly missing everything that file contributed. That is the same ambiguity
* the spawn atlas escalates for a disappearing facet, so it is escalated here
* too rather than applied.
*/
function missingSources(current, loaded) {
if (!loaded) return []
return Object.keys(loaded).filter((label) => !Object.hasOwn(current, label))
}
/**
* Read and parse every source, merged into one entry list.
*
* Later sources win: the base client table first, then each overlay in sorted
* order, so an overlay both ADDS ids the client never had and OVERRIDES stock
* ones the shard has re-purposed.
*
* Returns `{ entries, source }`. Throws `ClilocSourceError` for anything about
* the paths and `ClilocFormatError` for anything about the contents — different
* problems for an operator (wrong place vs wrong file), and the admin panel says
* which. A format error names the file it came from, because "which of my six
* overlay files is malformed" is otherwise a guessing game.
*/
function readCliloc(configured) {
const { root, files } = readSources(configured)
const merged = new Map()
const perSource = []
for (const file of files) {
let entries
try {
entries = parseCliloc(file.buffer)
} catch (err) {
if (err instanceof ClilocFormatError) {
throw new ClilocFormatError(`${file.label}: ${err.message}`, err.code)
}
throw err
}
let added = 0
let overrode = 0
for (const entry of entries) {
if (!Number.isInteger(entry.number)) continue
if (merged.has(entry.number)) overrode++
else added++
merged.set(entry.number, entry)
}
perSource.push({ label: file.label, kind: file.kind, entries: entries.length, added, overrode })
}
return {
entries: [...merged.values()],
source: {
root,
file: files[0].file,
sha256: files[0].sha256,
bytes: files[0].bytes,
hashes: Object.fromEntries(files.map((f) => [f.label, f.sha256])),
parserVersion: PARSER_VERSION,
sources: perSource,
},
}
}
module.exports = {
ClilocFormatError,
ClilocSourceError,
PARSER_VERSION,
CANDIDATE_NAMES,
CUSTOM_DIR,
CUSTOM_EXTENSIONS,
resolveBase,
listCustom,
readSources,
hashSources,
sameSources,
missingSources,
readCliloc,
}

42
server/src/utils/db.js
View File

@@ -4,6 +4,7 @@ const mariadb = require('mariadb')
require('dotenv').config()
const log = require('./logger')('db')
const { splitStatements } = require('./sqlStatements')
const pool = mariadb.createPool({
host: process.env.DB_HOST || '127.0.0.1',
@@ -44,28 +45,31 @@ const SCHEMA_PATH = path.join(__dirname, '..', '..', 'db', 'schema.sql')
/**
* Create tables if they do not exist. Idempotent. Retries while the DB is still
* coming up (important under docker-compose even with a healthcheck).
*
* Once core's schema is in place, every installed module's schema fragment is
* replayed after it (MODULE_API.md §2.6). That step is deliberately OUTSIDE the
* retry loop: a fragment that throws is that module's failure, not a signal the
* database is still coming up, and retrying core's whole schema nine more times
* because one module shipped bad SQL would turn a 503'd module into a two-minute
* boot. It is also why this file knows nothing about modules beyond the one call
* below — the discovery, splitting and per-module failure handling all live in
* modules/schema.js, required lazily so that requiring the pool never drags the
* loader in with it.
*/
async function ensureSchema({ retries = 10, delayMs = 2000 } = {}) {
await ensureCoreSchema({ retries, delayMs })
// eslint-disable-next-line global-require
await require('../modules/schema').replayFragments()
}
/** Core's own schema.sql, with the wait-for-the-database retry. */
async function ensureCoreSchema({ retries, delayMs }) {
for (let attempt = 1; attempt <= retries; attempt++) {
try {
const conn = await pool.getConnection()
try {
const sql = fs.readFileSync(SCHEMA_PATH, 'utf8')
// Strip `--` comments (full-line AND trailing) before splitting — so a
// leading comment block doesn't get glued onto the statement that follows
// it, and a `;` inside a trailing comment can't chop a statement in half.
// Safe because the schema never puts `--` inside a string literal.
const statements = sql
.split('\n')
.map((line) => {
const i = line.indexOf('--')
return i === -1 ? line : line.slice(0, i)
})
.join('\n')
.split(';')
.map((s) => s.trim())
.filter((s) => s.length > 0)
for (const statement of statements) {
for (const statement of splitStatements(sql)) {
await conn.query(statement)
}
log.info('schema ensured')
@@ -83,7 +87,15 @@ async function ensureSchema({ retries = 10, delayMs = 2000 } = {}) {
}
}
// Idempotent: `pool.end()` throws "pool is already closed" on a second call, and
// closing twice is normal rather than exceptional — a SIGINT followed by a
// SIGTERM reaches the shutdown handler twice, and the test harness closes the
// pool for every file on top of the suites that close it themselves. A teardown
// that fails because it had already succeeded is noise.
let closed = false
async function close() {
if (closed) return
closed = true
await pool.end()
}

Some files were not shown because too many files have changed in this diff Show More