Protocol 3.0 order 3 (Part C), first of two website PRs. This half is the data
pipeline only — parsers, the build/import CLI, and the tables. No routes and no
client, so nothing is user-visible yet; the API and pages follow in PR 2.
Part C is website-only: no plugin, no sidecar, no new event kinds, no wire
change.
## Parsing
`src/utils/spawnAtlasParse.js` is pure and fs-free so CI covers it with no
ServUO tree. Zero new dependencies — `Regions.xml` genuinely nests, so it gets a
small hand-rolled subset tokenizer rather than a new XML package. The 10.5 MB of
`Spawns/*.xml` never touches it: those records are flat and get a streaming
regex sweep instead.
The high-value transform is point-in-rect placement — highest region priority
wins, ties break to the smaller rect, then a nearest-landmark fallback within
200 tiles, else "Wilderness". That is what turns "lizardman at 5411,1234" into
"Despise, Felucca", and it resolves 83.2% of points (5,369 of 6,455).
Three things the real data forced, none of which were in the design:
- **Only 6 facets, not 13.** `Eodon.xml`, `GravewaterLake.xml` and the other
named-area files carry TerMur/Trammel points, so the facet comes from each
record's own `<Map>` and the artifact shards 6 ways.
- **Facet names disagree across sources.** `Data/Locations/*.xml` spells them
`Ter Mur` and `Tokuno Islands`; `<Map>` and `<Facet name>` say `TerMur` and
`Tokuno`. Unreconciled this is silent — the landmark fallback simply never
fires on those facets and every unregioned spawn there reads "Wilderness".
- **Spawn type tokens carry XmlSpawner directives**: `Fairy,{RND,4,8}`,
`alchemist/z/-50`, `Agralem/Name/Agralem`. Taken literally these invent
creatures that do not exist AND split real ones in two, since `Fairy` and
`Fairy,{RND,4,8}` slug apart. 71 of 845 entries were affected; stripping at
the first `/` or `,` leaves 800 clean ones.
## Artifact
`npm run atlas:build -- --servuo <path>` writes `db/data/spawnAtlas.*.json`:
6 facet shards + a compact index + a small indented `meta`. 1.41 MB committed,
down from 4.40 MB by dropping `facet` per record, omitting defaulted fields, and
tuple-encoding the ~24,000 type entries. `encodePoint()` and the importer's
`readPoint()` are exact inverses and are round-tripped in tests.
Display spelling is chosen deterministically (most common, ties to the
capitalised form) because the spawn files are inconsistent about case and the
name would otherwise depend on file read order — a spurious diff on every
unrelated rebuild.
## Import
`npm run atlas:import` needs no ServUO tree, which is the whole reason build and
import are separate: the container has the artifact but not the tree. It
reloads all six tables in one transaction (DELETE, not TRUNCATE, which is DDL
and would implicitly commit), so a failed import leaves the previous atlas
intact.
## No artwork, by design
The repo ships no creature art and no extraction tooling. Sprites live in the
operator's own client `.mul`/`.uop` files and are theirs, not ours to
redistribute. `shard_spawn_creatures.art` is nullable and NULL on every fresh
import; an operator who wants art extracts it themselves, drops it under
`server/uploads/atlas/` (already gitignored) and maps slugs in a gitignored
`spawnAtlas.art.json`. Text-only is the normal, fully supported state.
## Verification
- **544 server tests pass**, 57 new across `spawnAtlas.parse.test.js` (the
`:OBJ=` split, directive stripping, nested-region priority inheritance,
half-open rects, the facet reconciliation, tokenizer edge cases) and
`spawnAtlas.build.test.js` (aggregation, deterministic naming, and the
encode/decode round trip).
- Built and imported for real against the local MariaDB and the ServUO tree at
`C:\Users\colby\Desktop\ServUO`: 6,455 points, 800 creatures, 23,927
point/type rows, 387 regions, 558 landmarks, 25 champion altars.
- "Where does a lizardman spawn?" answers Shrines / Isamu-Jima / Yew across
Felucca, Trammel and Tokuno.
No routes changed, so the OpenAPI spec and route manifest are untouched.
---
- [x] AI-assisted: written with **Claude Code** (Claude Opus 5), reviewed before opening.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U7CBg11prhLimL9iHSX1bP
Protocol 3.0 Part A follow-up, found by the live five-rung smoke test.
Part A implemented the visibility framework correctly on the SSE path
and on /guilds + /governors, but the remaining public REST reads never
called into it. The result was that one event was projected live and
served verbatim from history:
* GET /public/shard/feed returned the stored payload as-is, so
actor.acct and actor.webId were readable ANONYMOUSLY for every
logged kind - player.death, player.murdered, mob.killed,
quest.complete, skill.gain, fame/karma.change, mob.login/logout,
guild.join. Broader than the guild-leader leak Part A set out to
close, since it covers every player rather than board holders.
* GET /public/shard/idoc returned ownerAcct - the house owner's game
account - to anonymous callers.
* The `houses` field rules (owner/price -> staff) were dead config:
neither getIdoc nor getHouses projected, so an admin could set them
in the panel and nothing happened.
* /feed filtered on PUBLIC_KINDS, a module-load constant derived from
the compiled DEFAULTS, so live audience changes did not reach it.
With `guilds` moved to staff, /guilds 403'd while /feed happily
served guild.join to anonymous.
Four fixes, all at the root rather than per-route:
1. Rule 1 now matches a field's MEANING, not one spelling. The wire
nests actors (leader.acct) but the read models flatten them
(shapeHouse -> ownerAcct, shapeGuild -> leaderWebId), and an
exact-key check missed every flattened one. isLockedField() locks a
key that is or ends in acct/webId, case-insensitively, so it fails
closed for shapes not yet written. The admin PUT rejects those
spellings too - `ownerAcct` is no longer configurable.
2. visibleKinds(level, config) resolves readable kinds from the LIVE
config; getFeed uses it and projects each row against its own kind's
feature. Deliberately independent of the `stream` flag, which governs
SSE fan-out only - so market history stays readable with its firehose
off. This makes the set a superset of PUBLIC_KINDS by exactly the two
vendor kinds.
3. getIdoc/getHouses/getChamps/getPresence project, so every shard
surface honours the same config.
4. shardEvents.db.list treats an EMPTY kinds array as "serve nothing".
It previously fell through to the unfiltered query, so a fully-gated
config would have dumped the whole event log, staff audit included.
Also fixes a bug introduced while wiring this up: projectValue recursed
into any object, so a Date column came back as {}. It now walks arrays
and plain objects only. The unit tests used JSON fixtures and could not
have caught it - the live /idoc read did.
Verified live against MariaDB + a stub sidecar, all five rungs: 13
routes x 5 rungs, defaults reproducing pre-v3 access exactly, zero
acct/webId below admin on any read, unmapped kinds (staff.command,
cheat.detect, login.attempt) reaching only admin on SSE, and audience /
enabled / stream changes taking effect live on an already-open stream.
Tests: 487 server (+9). Swagger regenerated; route manifest unchanged.
Co-Authored-By: Claude <noreply@anthropic.com>
Protocol 3.0 Part A. Replaces the static PUBLIC_KINDS allowlist - which
was the entire public/admin boundary - with per-feature, per-field
audience control an admin owns from Admin -> Shard Visibility.
Closes a live leak. BridgeJson.Actor() writes acct and webId;
shapeGuild() returned the stored payload verbatim; GET
/api/v1/public/shard/guilds is anonymous. Guild leaders' game account
names and website user ids were readable by anyone, and the same path
existed for governors. Both are now projected.
The ladder is anonymous < logged_in < player < staff < admin, each rung
implying the ones below. Staff satisfy `player` without a linked account
(as /player/* already does); `editor` is a content role and gets no
shard privilege, since mapping it to staff would silently widen what
editors see.
Two invariants are code, not configuration, and both reject rather than
silently ignore:
1. acct/webId are admin-only always - not configurable, discarded on
read as well as rejected on write.
2. A kind absent from KIND_FEATURE never reaches anyone below admin.
Fail closed, so a shard emitting a new event degrades to staff-only
rather than to public.
Enforcement is three points over one config: requireFeature() on routes
(404 disabled, 403 out-of-rung) plus field projection; per-connection
filtering on SSE, where a subscriber's rung is resolved once at subscribe
time and frozen so a long-open stream cannot gain privilege; and
/public/shard/features so the SPA hides links it cannot follow.
PUBLIC_KINDS still exists and is still exported (/feed filtering,
notificationStreams) but is now derived from the kind map, so the two
can no longer drift. Defaults reproduce pre-3.0 behavior exactly - a
test pins the derived set against the old allowlist.
Also fixes an SSE resource leak found while testing: a client dropped
because its write threw was removed from the bucket but its keepalive
interval was never cleared, firing forever on a dead socket. Both paths
now go through one drop().
Tests: 478 server (33 new across shardVisibility + shardBroadcast),
43 client. Route manifest and OpenAPI spec regenerated.
Co-Authored-By: Claude <noreply@anthropic.com>
`uoLinkClient.call()` resolved the uo-link config OUTSIDE its try/catch.
resolveConfig() decrypts the stored auth token, and secretBox.decrypt throws
when the ciphertext can't be authenticated — SECRET_ENC_KEY rotated, or a DB
dump restored into an environment keyed differently. That throw escaped the
client entirely, breaking its documented "never throws / always returns
{ ok, data, status }" contract and turning a misconfiguration into a 500 on
every route that does a live sidecar round-trip:
GET /admin/uo-link/config
GET /{admin,player}/shard/char/:serial
GET /{admin,player}/shard/roster/:account
GET /{admin,player}/shard/vendors/:account
Found by a live smoke test of all 200 routes at every access level. Public
shard routes were unaffected because they read the DB via getSafe(), which
never decrypts.
Move resolveConfig() inside the try so the failure returns the standard
{ ok: false } shape, and log it at ERROR with a distinct message: a wrong key
previously looked identical to "the shard is offline", with no clue why.
Those routes now degrade to 503, and GET /admin/uo-link/config returns 200
again — it is the screen an admin needs to re-enter the token and recover, so
having it 500 locked them out of the fix.
Also gate the admin Dashboard's site-mode toggle. PUT /admin/site-mode is
adminOnly, but the button rendered for every staff role, and toggle() had a
try/finally with no catch — so an editor clicking it got an unhandled promise
rejection and zero UI feedback. Gate the control on role === 'admin' (the rule
AdminLayout already documents: never show a non-admin a control that would 403)
and surface a message if the call is refused anyway.
Adds server/test/uoLinkClient.test.js, which fails against the unfixed client.
Co-Authored-By: Claude <noreply@anthropic.com>
The last split PR of docs/website/API_V2_PLAN.md § Phase 2. public.routes.js,
player.routes.js and auth.routes.js are deleted; each group is now a directory
whose index.js owns the group gate and the mount table and declares no routes.
Every one of the 200 manifest routes is now in a capability router.
public/ posts (2) wiki (4) pages (2) shard (12) site (4, group root)
player/ account (8) shard (8) appeals (4), behind noindex + requireAuth
auth/ login (2) register (1) invite (2) password (3) session (2, root)
No URL moves. All four gates zero-diff: routes.manifest.json (200 public + 2
internal), routes.guards.json, swagger-output.json (198 operations), and
docs/website/api-route-inventory.json was already in sync. 434 tests green.
Notes on the non-mechanical parts:
- public/index.js and auth/index.js carry no group gate, deliberately, and say
so. The public surface is anonymous by contract (logged-out SPA, Discord bot,
Android ShardStreamClient on /public/shard/stream); /auth is where a caller
becomes authenticated. player/index.js gates on requireAuth only, never
requireRole('player') — staff are a superset of players.
- GET /auth/me has a mount-order dependency: use('/me', meRouter) matches the
bare /me, so the request runs meRouter's noindex + requireAuth and falls
through. session.router.js must stay mounted last. Verified by the
counterfactual — mounting it first still 401s but drops X-Robots-Tag, which
no manifest or guards file can see.
- loginGuards moved to auth/loginGuards.js (frozen) rather than being copied
into the three routers that spread it; sso.routes.js drops its duplicate.
- The :param shadowing check was re-run in dispatch order against the built
stack: 86 routes, 64 literal, none shadowed. /public/wiki/{categories,tags}
ahead of /:slug is the only ordering-sensitive pair.
Co-Authored-By: Claude <noreply@anthropic.com>
PR 4 of the in-place admin router split (docs/website/API_V2_PLAN.md § Phase 2),
and the last admin one: it moves the entire residual 33 and DELETES
admin.routes.js. Every one of the 110 admin routes is now declared in a
capability router. No URL, gate or handler changes.
shard.router.js (16) /admin/shard
uoLink.router.js ( 5) /admin/uo-link
email.router.js ( 6) /admin/email
discordBot.router.js ( 2) /admin/discord-bot
settings.router.js ( 2) /admin/settings
dashboard.router.js ( 2) GET /dashboard + PUT /site-mode, at the group root
admin.routes.js deleted, was 33
No gate moved to router level. Every adminOnly in the residual file was
per-route, and modAccess on /shard must stay per-route because half that router
must not have it — which keeps the per-route handler count intact, the one
number routes.guards.json can actually check.
/shard is the first prefix where two tiers share one router: 7 self-service
account-linking routes (no extra gate, served by the same player/shard
controller handlers, tagged `Admin · Account`) alongside 9 in-game staff ops on
modAccess. Prefix ownership beats tag grouping — splitting by tag would put two
routers under one prefix for no gain. The tag mismatch stays; retagging is a
real spec diff and belongs in a PR about tags.
dashboard.router.js is the one router mounted at the group root rather than a
prefix: GET /dashboard and PUT /site-mode share no path segment. That is safe
only because the file declares no router-level middleware — a bare use(gate) in
a root-mounted router would run for every request passing through toward
another mount. The file carries a comment saying so.
Acceptance — all four gates zero-diff:
routes.manifest.json unchanged (200 public + 2 internal)
routes.guards.json unchanged (no route lost or gained a gate)
swagger-output.json unchanged (198 operations)
api-route-inventory.json already in sync
plus 434 server tests green.
Verified separately, because no gate can catch it: introspecting the built
stack, all 59 literal admin paths still dispatch to their own layer — nothing
is captured first by a /:param sibling. The manifest sorts its entries, so
declaration order is invisible to it.
Also repoints the comments that referenced admin.routes.js by name
(botActivity/moderation controllers, the town-crier cap mirror in
announceJobs.logic.js) and generalizes the "the path is on the line after
router.get(" rationale in routeManifest.js, README.md and pr-checks.yml, which
was never about that one file.
Co-Authored-By: Claude <noreply@anthropic.com>
Clears the 124 CODE_SMELL findings from the SonarQube scan (server, client,
and bot). All changes are behaviour-preserving refactors — no route, protocol,
schema, or config changes — verified against the full server (381) and client
(43) test suites plus a clean client build.
By rule:
- S3776 (20, cognitive complexity): extract helpers/handlers so each function
drops under the threshold — shard model upsert builders, page/wiki update,
block validation, notification stream mapping (dispatch table), SSO mobile
login, shard ingest deps, uo-link socket backfill/connect, the bot slash-
command dispatchers + discord manager, and the Shard/UserDetail/HeroEditor/
CharacterStats React components.
- S4624 (34, nested template literals): pull inner templates into locals /
a withQs() helper; rewrite shardEvents.describe() as a formatter table.
- S3358 (35, nested ternaries): lift to if/else vars, lookup maps, small
components, or guarded JSX expressions.
- S6479 (12, array-index React keys): key by stable content instead of index
(two in-editor lists left as-is; index matches their by-index edit model).
- S6353 (6): [0-9]/[^0-9] -> \d/\D. S125 (5): reword state-shape comments that
parsed as code. S3800/S3782 (botScore): JSDoc-type PATH_WEIGHTS tuples.
- S6481 (2): memoize Auth/Site context values (and SiteContext brand).
- S4144: dedupe HeroEditor upload handler into useImageUpload().
- S1126 (2), S6035, S5869 (redundant A-Z under /i), S5843 (town-name regex ->
prefix list): assorted one-liners.
Co-Authored-By: Claude <noreply@anthropic.com>
The schema loader stripped only full-line -- comments, then split the
file on ';'. A trailing comment containing a semicolon (e.g. the
mobile_auth_sessions.session_id column: `-- uuid; carried inside...`)
chopped the CREATE TABLE in half, so MariaDB got the fragment and failed
with `error ... near '' at line 3`, crash-looping the server on boot.
Strip -- comments on every line (full-line and trailing) before the ';'
split. Safe because the schema never places -- inside a string literal.
Verified by running ensureSchema() against a fresh MariaDB: all 49 tables
create cleanly and mobile_auth_sessions has all 11 columns.
Co-Authored-By: Claude <noreply@anthropic.com>
Additive, v1-only backend contract for the Android app's opt-in push (Part 1 of
M7; docs/android/PLAN.md §11). The app is a pure consumer — this lands the
endpoints, fan-out, and relay it needs.
- Schema: push_devices (per-device endpoint) + notification_subscriptions
(per-user opted-in streams), FK→users ON DELETE CASCADE.
- Stream catalog + event→stream mapping (config/notificationStreams.js): public
streams (news.post, server.status, idoc.warning, champ.start, governor.election)
drawn ONLY from the SSE PUBLIC_KINDS allowlist; personal owner-keyed streams
(vendor.sale, house.idoc, account.login). Full-state upserts (champ/city) fire
only on a real transition via an injectable tracker.
- Fan-out (utils/pushDispatch.js): content-free tickles ({ stream, ref }) POSTed
to each subscribed device; never throws. Two producers — shardIngest.ingest
(beside the SSE broadcast) and the create/publish-post path (news.post).
Personal events resolve to the owner via shardLinks. SSRF guard: endpoints must
be HTTPS, non-private, and on the NTFY_BASE_URL/NTFY_ALLOWED_ORIGINS allow-set —
enforced at registration and every publish.
- Routes under the role-agnostic self surface (never /admin): POST|GET
/auth/me/devices, DELETE /auth/me/devices/:id, GET
/auth/me/notifications/streams, GET|PUT /auth/me/notifications/subscriptions.
Swagger regenerated (4 paths, PushDevice/NotificationStreams/etc. schemas).
- ntfy service in docker-compose.yml: pinned image, declarative ./ntfy/server.yml,
no published host port, anonymous unguessable topics (no accounts) — zero
interactive setup. No publish token required (content-free design); optional
NTFY_PUBLISH_TOKEN honored.
- Tests: pushDispatch (mapping, PUBLIC_KINDS gate, owner-keying, SSRF guard,
content-free payload) + notifications route auth gate. Full suite green (247).
Co-Authored-By: Claude <noreply@anthropic.com>
Add a full password-reset flow — the prerequisite for the Android app
(docs/android/PLAN.md §8.2), which hands off to the website for reset
rather than shipping a native screen.
Backend:
- password_resets table: stores only the sha256 hash of an opaque 32-byte
token (mirrors user_invites / mobile_refresh_tokens), single-use, ~1h TTL.
- model/passwordResets + users.getActiveByEmail (email is non-unique, so a
request can match several accounts, each emailed its own link).
- mailer.sendPasswordReset (fails soft when email is unconfigured).
- Endpoints: POST /auth/password/forgot (always a generic 200 — no account
enumeration), GET|POST /auth/password/reset/:token. Confirming rotates the
hash and revokes every session (web cutoff + mobile refresh tokens); it does
not auto-login, so a 2FA account still passes TOTP next sign-in. Also serves
SSO-only accounts (null hash) as their set-initial-password path.
- Dedicated request/confirm rate limiters. Swagger regenerated.
Web:
- ForgotPassword + ResetPassword pages, routes /account/forgot and
/account/reset/:token, and a "Forgot your password?" link on the login page.
Tests: test/passwordResets.test.js (5). All server tests pass; client builds;
end-to-end smoketest against MariaDB passes (no-enumeration, single-use, hash
rotation, session revoke, login with the new password).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NgyHnrNa8WwG3doxvxjuCr
Players whose linked Discord identity was banned or muted can now submit
an appeal from the portal and track it; staff get a queue in the admin
moderation section to claim and resolve (approve/deny) appeals. Approving
a ban/mute appeal best-effort asks the Discord bot to reverse the action
(unban / clear timeout) via the internal API and posts a mod-log embed; a
down bot never fails the resolution (reversal_status is recorded).
- Schema: new server-owned `appeals` table (no cross-owner FK to
mod_actions; existence validated in app code).
- Server: model/appeals/* + player appeals controller (submit/mine/
eligible/withdraw) and admin queue handlers (list/claim/resolve/
per-user) under the existing admin+moderator gate; one-active-appeal
enforced app-side; eligibility keyed on the caller's linked Discord id.
- 6d: bot POST /internal/mod-reverse (+ modLog.postReversal) and
server botInternalClient.reverseModAction, wired into resolve().
- Client: admin Appeals queue + resolve modal, ModerationUser appeals
tab, player Appeals page (submit/withdraw), nav + routes + api methods.
- Docs: swagger annotations + component schemas, regenerated output.
- Tests: appeals controller + pure suites (server npm test 224 green).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XmHdsbnLzDMAVQkAoTQSBe
Replace baked-in UOM/MysticMoon/UOMysticmoon branding with a BRAND_* env
scheme so one prebuilt image runs as any shard; UOMysticmoon becomes the
first tenant that sets these vars rather than a special case in the code.
Architecture (chosen because the app ships as a prebuilt image):
- server/src/config/brand.js + bot/src/brand.js read BRAND_* once at boot,
with Runic Gateway defaults.
- Text/colors reach the SPA at RUNTIME through the existing public settings
API (settings.model.getPublic -> SiteContext), so no client rebuild. The
admin-editable site title + contact email still override BRAND_NAME/email.
- SiteContext applies BRAND_ACCENT_COLOR to the --accent CSS var at runtime.
- Express templates the built index.html <title>/description/OG/favicon at
serve time from BRAND_* (renderIndexHtml in app.js).
- Server-side consumers read brand directly: emails, TOTP issuer, API docs,
boot logs, HTML error page. Bot uses it for embed color + logs.
Assets: logo/hero/favicon delivered from a ./brand:/app/brand bind-mount
(BRAND_LOGO/HERO/FAVICON), with neutral defaults baked in; hero falls back
to a built-in image when unset.
Scope: also genericized package.json names (uomysticmoon-* -> runic-gateway-*)
and the DB_NAME/DB_USER/COOKIE_NAME code defaults (runic_gateway/runic/
rg_token). Production keeps its real values by pinning them in .env — see
.env.uomysticmoon.example, which reproduces the exact UOMysticmoon identity
(proof the substitution works). Changing a deployed COOKIE_NAME invalidates
existing sessions, so UOMysticmoon pins uomm_token.
Verified: 193 server tests pass, client builds, app.js loads + templates the
built index.html, brand transform injects title/description/OG/favicon.
Per request, split the single public house registry into three role-scoped views:
- Public /site/houses → only houses in DANGER (IDOC), by LOCATION (region + map/
coords). No owner, price, co-owners or decay detail. Renamed "Houses in danger";
kept live via the public house.decay feed. The full-registry deltas
(house.update / house.remove — which carry owner/price) are REMOVED from the
public SSE allowlist so they never reach the public channel.
- Staff full registry → new /admin/houses (admin + moderator, RoleGate + MOD_PATHS)
backed by GET /admin/shard/houses (modAccess), with owner/price/co-owners/decay
and search, kept live on the admin SSE channel.
- Player portal → "My houses" home-status section (own houses only, with decay/
IDOC status) via GET /player/shard/houses, scoped to the caller's linked accounts.
Server tests green, client build clean, swagger regenerated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 5: the account-provisioning backend — link-only stays, plus hybrid
self-signup, an admin email-invite tool, and site-side unlink.
- uoLinkClient.createAccount / unlinkAccount (v2). Password is forwarded to the
shard (hashed there) and never stored/logged; the end-user browser IP is passed
for the shard's per-IP cap; actor is stamped server-side.
- Hybrid signup: POST /player/shard/account provisions a game account (its own
username + password) for the signed-in user and mirrors the link locally. Gated
by the new game_account_signup setting AND the shard's own mode (mapped 403/409/
429/400/503). Serves both self-serve signup and the invite-accept game step.
- Email invites: user_invites table (sha256 token hash, single-use, expiring);
invites model + admin CRUD (POST/GET/DELETE /admin/invites, admin-only) +
mailer.sendInvite (falls back to returning the accept link if email is off);
public token-gated accept (GET /auth/invite/:token, POST .../accept) creates the
user at the invite's preset role and logs them in, bypassing the registration
gate. Accept is race-safe (atomic single-use; rolls back the user if it loses).
- Admin unlink: DELETE /admin/users/:id/shard/link/:account (admin-only) + local
mirror drop; account.unlinked ingest reconciles the mirror when a player runs
[unlink in game. account.audit / account.unlinked are logged (admin channel
only — never on the public SSE allowlist).
Tests: invites model (hashing, single-use, expiry, revoke) + account.* ingest
reconcile/visibility. Full suite 193/193; swagger regenerated.
Refs .plans/protocol2-integration.md (Phase 5).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 4: sync the site's published news posts into the Protocol 2.1 News gump.
- uoLinkClient.postNews / deleteNews.
- utils/newsGump.js — a STATE SYNC (not a one-shot announce leg): an article
stays in the gump while its post is published news and is pulled when it leaves
that state. buildArticle renders a compact gump-HTML block (centred title +
plain-text excerpt — the gump supports only a small HTML subset) with a
"more info" link to /site/news and an optional gump image from the
`news_gump_image` setting. Every call is best-effort / never-throws.
- Hooked into the posts pipeline alongside the existing announce enqueue:
syncPost on create/update/publish (fresh publish announces; edits refresh
silently; leaving published-news pulls the article), removePost on delete.
- reassertAll() runs in uoLinkSocket.backfill on every WS (re)connect —
reconciles the gump to our source of truth and recovers any article whose
original live push failed (silent, so a reconnect never re-proclaims old news).
Server 185/185, swagger regenerated. Refs .plans/protocol2-integration.md (Phase 4).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 1 of the Protocol 2.0/2.1 integration: the read/ingest backend for the four
new uo-link boards, following the established champs/pages pattern (ingest → our
MariaDB + snapshot-on-reconnect + public SSE + token-free public endpoint).
- Schema: shard_guilds, shard_governors, shard_governor_terms, shard_presence;
extend shard_houses with the house.update registry columns (owner_name,
co_owners, friends, price, decay, in_registry) so the decay-transition and
registry feeds share one house row without clobbering each other.
- Ingest: route guild.update/remove, city.update, presence.online,
house.update/remove; log guild.join (real-time joins feed); region.enter is
broadcast-only. All new public kinds added to the SSE allowlist.
- Governor term history captured from day one: on every observed governor CHANGE
the open term is closed and a new one opened, idempotent so backfill/duplicate
city.update never spawn spurious terms. votes stays NULL (the feed carries only
candidate count, not tallies) — we never fabricate vote numbers.
- Client + backfill: getGuilds/getGovernors/getHouses/getPresence; snapshot each
board on every WS (re)connect, independently guarded so an empty/failed board
(e.g. no City Loyalty) never wipes another.
- Public endpoints: /shard/{guilds,governors,governors/:city/history,presence,houses}.
- Tests: ingest routing for all new kinds + governor term-capture idempotency
(15 new; full suite 179/179). Swagger regenerated.
Refs .plans/protocol2-integration.md (Phase 1).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire up the three uo-link sidecar surfaces that weren't integrated yet.
Champion spawns
- Ingest champ.update/champ.remove into a new shard_champs table (served from
our own store, like online/houses); public /site/champs board with a nav link,
live via the existing SSE feed (champ.* added to the public allowlist).
Staff write plane (admin + moderator)
- kick / ban / unban / broadcast via /admin/shard/*; actor is stamped server-side
from the session, never the browser. Sidecar status codes mapped (403 disabled/
protected, 404 unknown, 503/504 transient). admin.audit events are logged and
surfaced at /admin/shard/audit.
- New admin "In-Game Ops" view (/admin/shard-ops), plus per-account Kick/Ban/Unban
on the user-detail and character views (ShardAccountActions, self-gated to staff).
Help-page (support) queue
- Ingest page.new/updated/closed into a new shard_pages table; respond/close via
/admin/shard/pages/*. Champ board and page queue are snapshotted from the
sidecar's /champs and /pages on every WS (re)connect (guarded so a failed call
never wipes local state).
Verified live end-to-end against MariaDB + the Rust sidecar + ServUO; unit tests
cover ingest routing (shardIngest.champsPages.test.js). Swagger regenerated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0114TpmrNW4wNXsHq5CR72jQ
Add a "View" action beside Edit in the users table that opens a dedicated,
read-only page showing everything the uo-link shard knows about a user,
scoped to their linked game accounts: character rosters, currently-online
characters, houses (IDOC-first), and recent vendor sales.
Backend (admin-only, under the existing /users adminOnly gate):
- GET /admin/users/:id — single sanitized user (page is deep-linkable)
- GET /admin/users/:id/shard/{accounts,sales,houses,online}
- shardState: listHousesByAccounts / listOnlineByAccounts (+ model shapers)
- Extract salesForAccounts into utils/shardSales; reuse in player getSales
- Live rosters reuse the existing admin-bypass /admin/shard/* endpoints,
so no new routes for roster/vendors/char
Frontend:
- UserDetail page reusing CharacterStats / GameAccounts / VendorSales
- GameAccounts gains a readOnly prop (drops link form + self-voice copy)
- api.admin.getUser + api.admin.userShard(id) scope; route + layout title
Tests: adminUserShard.test.js (404, account scoping, empty accounts,
salesForAccounts cap/filter). Full server suite 164 pass; client builds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0114TpmrNW4wNXsHq5CR72jQ
Replace the fire-and-forget Discord-only announce on publish with a
retry-safe, two-leg pipeline. When a post transitions into published-news
(false→true publish while in news, or category→news while published), an
announce_jobs row is enqueued with two INDEPENDENT delivery legs:
• town crier — sidecar POST /towncrier via uoLinkClient (stable id
`post-<id>` so a retry replaces rather than duplicates)
• discord — bot POST /internal/announce via botInternalClient
(single source of truth for the #news channel stays in the bot)
An in-process poller (utils/announceWorker) sweeps the table every
ANNOUNCE_POLL_MS and dispatches each due leg with its own exponential
backoff (30s→2h, 6 attempts). A leg is retried on transient failures
(503/504/network) and failed fast on data/config errors (400 over-cap,
401/409). Publishing never blocks on the sidecar or Discord — enqueue is
local DB only. Parent `status` is a done/partial/failed rollup of the two
legs; posts.announced_at is stamped once both deliver.
Admin visibility: GET /admin/posts/:id/announce + a per-leg Retry
(POST .../announce/retry) surfaced in the PostEditor for news posts.
Pure decisions (text build/caps, classification, backoff, rollup) live in
announceJobs.logic and are unit-tested (server/test/announceJobs.test.js,
10 tests). The old manual /admin/uo-link/towncrier form is untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0114TpmrNW4wNXsHq5CR72jQ
Public "Online now" now lists only players whose game account is linked
to a STAFF website user (admin/editor/moderator) — linked players are no
longer exposed publicly with their name and location. listOnlineLinked
joins through to users and filters on role; the section is relabeled
"Staff online".
Character/roster/vendor reads gain an admin bypass: admins may view any
character's data, while players (and editor/moderator staff) stay limited
to accounts they have personally linked. The bypass lives in the shared
player controller and only ever widens access for genuine admins.
Also finalizes the uo-link character/vendor front end (player + admin
character sheets, VendorSales component, ShardChar removed) and
regenerates swagger-output.json.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018kj5s1QCKobuFPYmqxjy1q
The site now ingests the sidecar's live WebSocket feed and persists it to its
own MariaDB, and re-broadcasts curated events to browsers over SSE.
- schema: shard_events (append-only notable-kind log, sha1 dedupe_key +
INSERT IGNORE for idempotent reconnect backfill), shard_online (current
players, upsert/refresh/remove), shard_economy (gold-supply series),
shard_houses (per-house decay stage + derived is_idoc).
- model/shardEvents + model/shardState: the .db.js/.model.js split; writes
take camelCase event data, reads are shaped; online upsert uses COALESCE so
a partial char.vitals refresh never blanks login fields.
- utils/shardIngest: single dispatcher routing each kind to state writes
and/or the event log, then the broadcaster. High-frequency kinds
(char.vitals, economy.supply) update state only. A changed server.hello
bootId clears the stale online roster. Deps are injected for unit testing.
- utils/uoLinkSocket: the server's first outbound WS client (ws dep). Verifies
the ws.hello protocol, backfills via /history + /economy on every
(re)connect (dedupe handles overlap), reconnects with capped backoff, and
mirrors connection state into uo_link_config. Self-guards: only connects when
the integration is enabled with a token.
- utils/shardBroadcast: SSE fan-out with public (safe kinds only) and admin
(all) channels, keepalive pings, per-client cleanup.
- server.js: start the ingest socket on boot (no-op until configured) and stop
it + close SSE streams on graceful shutdown.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qPmpmVH1xGCiZoz9m9vW3
Introduces the DB-backed connection config for the uo-link sidecar (the
HTTP + WebSocket bridge to the ServUO shard) and a never-throw REST client,
mirroring the existing Discord-bot integration:
- uo_link_config singleton table (base/ws URL, AES-256-GCM-encrypted shared
token, protocol pin, enabled, and last-known status/plugin_connected/
last_event_at/boot_id mirrors for the admin panel).
- model/uoLinkConfig: getSafe (never returns the token — only hasToken),
getWithToken (server-side decrypt), save (blank token = unchanged),
recordStatus (mirror the sidecar's reported state).
- utils/uoLinkClient: never-throw fetch client returning {ok,data,status,
error}; Bearer token + X-UOLink-Version on every call; brief config cache;
helpers for health/char/roster/vendors/history/economy/link/towncrier.
- .env.example: UOLINK_BASE_URL/WS_URL/PROTOCOL defaults (token stays
admin-managed in the DB, never an env var).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qPmpmVH1xGCiZoz9m9vW3
Shared RichTextEditor gains @tiptap/extension-text-align for heading and
paragraph nodes, serializing alignment as inline text-align on the block
node so it round-trips through save/reload. Fixed once at the shared
component so it also flows into the upcoming rich_text and two_column
page blocks.
Server sanitize allowlist now permits `style` on p/h1-h6, constrained by
allowedStyles to text-align (left/right/center/justify) only; all other
CSS properties and values are stripped.
Step 1 of the CMS Page Builder spec.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Retire env-var SMTP basic-auth and send the contact form through Gmail over
OAuth2 (SMTP XOAUTH2), configured in Admin -> Settings -> Email via an in-app
"Connect Gmail" consent flow. Reuses the existing google SSO OAuth client; the
captured refresh token is stored AES-GCM-encrypted (write-only over the API,
never returned), mirroring the auth-provider and Discord-bot secret patterns.
- schema: new email_config singleton table (mirrors bot_config)
- model: emailConfig.{db,model} with encrypted refresh token + getSafe/getWithSecret
- mailer: nodemailer OAuth2 transport (client id/secret from the google provider
row), contact recipient = contact_email setting, mailto: fallback preserved,
plus sendTest()
- routes/controller: /admin/email config, connect start+callback (ssoState CSRF
+ PKCE), test, disconnect
- client: EmailDelivery section on the Settings page + api methods; Settings copy
now spells out that contact_email is the delivery recipient
- docs/env: drop SMTP_*/CONTACT_TO from env examples; update README/BACKEND_DESIGN
- tests: emailConfig.model + mailer suites (8 new; full suite 142 pass)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XKeCQEJZr1AFJN4Bgcmvh3
The GET /internal/bot-config route returns the DECRYPTED Discord bot
token and was mounted on the same Express app / port 3000 that Pangolin
proxies publicly. Its only guard was the BOT_INTERNAL_KEY shared secret,
and .env.example shipped a placeholder default — so a forwarded path or a
weak/unrotated key would expose the plaintext token to the internet.
Move server<->bot internal traffic onto its own listener and fail fast on
a weak key:
- Add server/src/internalApp.js: a standalone Express app mounting
requireInternalKey + /internal (and a no-secret /health), mirroring the
bot's unpublished port-4100 pattern.
- server.js starts a second listener on INTERNAL_PORT (default 3001),
closed on graceful shutdown.
- Remove the /internal mount from the public v1.router; the public app now
404s /api/v1/internal/bot-config even with a valid key.
- Fail fast: new utils/botInternalKey.js rejects an empty, placeholder, or
<16-char BOT_INTERNAL_KEY — fatal in production (exit 1), warning in dev.
- docker-compose: bot SITE_INTERNAL_URL -> app:3001/internal/bot-config;
document that INTERNAL_PORT stays unpublished.
- .env.example (root/server/bot): document INTERNAL_PORT, the fail-fast
behavior, and a defense-in-depth Pangolin deny rule for /api/v1/internal.
Tests: add requireInternalKey.test.js and botInternalKey.test.js
(node --test: 93 pass).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rao86n5cXpwAyjdBFEshV
Standalone bot/ service (its own package.json/Dockerfile) managed entirely
through a new admin-only Discord Bot panel — token stored encrypted in the
DB and pushed to the bot process in-memory, never an env var. Built in
phases, each independently verified against a live Discord guild:
- Bot skeleton: gateway connection, internal shared-secret API, self-heals
on its own restart by pulling config from the site
- Moderation core: /ban /kick /mute /warn /warnings + mod-log channel
- Word/invite/spam filtering with leetspeak-resistant normalization and a
staff role/channel allowlist
- Scheduled messages: recurring (cron) and one-off channel posts
- Role assignment: button role menus, auto-role on join, temp roles,
bulk role ops
- Auto-rotating primary invite with an audit log
- Site integration: news-publish -> Discord announce webhook, manual
/announce, read-only /wiki search
Also fixes a pre-existing bug in both DB pools (server + bot): the mariadb
driver defaulted to timezone 'local', silently mis-serializing bound Date
params by the host's local offset instead of the DB's UTC session.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Refactor authentication into a provider-agnostic session layer and build
two new auth surfaces on top of it, without changing local password/TOTP
behavior. Every flow now issues sessions through
sessionService.createSession(user, authMethod).
Part 1 — Session abstraction (backward-compatible refactor):
- New server/src/auth/: token.js (JWT/cookie primitives), session.service.js
(create/validate/partial-TOTP/revoke), session.middleware.js
(attachSession/requireAuth/requireRole). utils/auth.js is now a thin
compat facade so existing imports are unchanged.
Part 2 — Mobile bearer auth (additive):
- /api/v1/auth/mobile/{login,refresh,logout}: short-lived access JWT +
long-lived refresh token, stored hashed and rotated on use, in a new
mobile_refresh_tokens table. Reuses web bot-scoring/backoff; single-request
TOTP. token.signToken gains a backward-compatible expiresIn option.
Part 3 — Pluggable SSO (Google, Discord, generic OIDC):
- OAuth2Provider base + built-in Google/Discord (fixed endpoints) + generic
OIDC, a registry with health/validation, PKCE+CSRF transaction state, and
discovery (GET /auth/providers), start/link/callback routes.
- Link-only policy: SSO signs in only to an already-linked account; external
identities are never auto-provisioned. Client secrets encrypted at rest
(AES-256-GCM, utils/secretBox.js). Admin CRUD (/admin/auth/providers) and
account linking (/admin/account/identities). New auth_providers +
user_identities tables.
Frontend:
- Login page renders provider buttons from /auth/providers (inline SVG icons,
graceful with zero providers). New Authentication admin view
(Local/Google/Discord/Custom). Account page linked-accounts section.
Tests: 83 passing (session, mobile, providers, registry, secretBox, ssoState,
ssoCallback) — all DB-free via fetch mocks + model stubs. README + .env.example
updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
auth.js previously only logged a warning when JWT_SECRET was unset and
then continued to boot. With no secret, jwt.sign/jwt.verify cannot
produce or validate a usable token, so every login silently fails while
the server appears healthy — and booting a production instance without a
configured secret is a safety hazard.
Resolve the secret through resolveJwtSecret():
- production (NODE_ENV=production): throw, so the process refuses to
start without a real secret instead of running unusable.
- dev/other: fall back to a known insecure secret so local login keeps
working, with a loud warning to set JWT_SECRET before deploying.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a layered set of protections around the admin login and the app edge.
Trust proxy (server/src/utils/trustProxy.js)
- Configurable via TRUST_PROXY; pin to the newt agent ("ptero") LAN IP so
X-Forwarded-For is trusted ONLY from that peer. A blanket "true" is
rejected (coerced to 1) to prevent XFF spoofing that would dodge every
IP-based control. DEBUG_TRUST_PROXY logs peer/XFF/req.ip to re-verify the
proxy IP without a redeploy. Documents the Omada static-reservation
assumption.
Login throttling (server/src/middleware/loginProtection.js, rateLimit.js)
- express-slow-down progressive delay + the existing hard rate cap + a
separate per-IP exponential backoff that persists across the rate window.
All failures return one generic message (no user/pass disclosure).
Honeypot (login form + auth.controller)
- Hidden, plausibly-named field ("company"); a filled value fails
generically and is scored as an unambiguous bot.
Optional per-user TOTP 2FA (speakeasy/qrcode)
- totp_secret/totp_enabled columns (+ idempotent migration). Self-service
Account page: enroll via QR, confirm a code to enable, code-gated disable.
- Login is two-step for enrolled users: after the password, a short-lived
signed challenge (stage:'totp', not a session) is required before the
real session is issued.
Bot / scanner scoring + IP ban (server/src/middleware/botScore.js)
- Weighted CMS-scanner paths (this app uses none). Junk paths 404 FIRST,
unconditionally — independent of score/ban state, so a scanner rotating
through fresh Cloudflare IPs gets no free pass. /wp-admin/install.php is
the top-weighted near-1-hit ban (worst offender in prod logs). Per-IP
score with quiet-period decay temp-bans an IP from ALL routes once past a
(deliberately low) threshold, to protect /admin from credential stuffing.
Failed logins and honeypot hits feed the same score.
- Periodic sweep evicts stale, unbanned, quiet entries so the in-memory
store can't grow unbounded; the interval is unref'd and cleared on
graceful shutdown.
Tests: node --test suite (40) covering trust-proxy parsing + live req.ip
(incl. pinned-IP), rate limiter + exponential backoff, honeypot rejection,
TOTP verify (enabled/disabled) + challenge-isn't-a-session, bot-score
threshold/decay/ban + junk-404-independence + install.php + store sweep.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
isLoggedIn trusted id and role straight from the JWT and never
re-checked the database, so a demoted admin kept their old role and a
deleted user kept a working session until the token expired (up to
JWT_EXPIRES_IN). This also undercut the "last admin" guards.
isLoggedIn now loads the user from the DB by the token's id on every
request: a missing user returns 401 (deleted), and req.user carries the
fresh DB row so the current role is always used downstream.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
isLoggedIn only verified a valid JWT, so an authenticated editor could
call any admin endpoint (create/promote/delete users, flip site mode,
change settings). Add a requireRole middleware factory and gate the
sensitive routes with admin-only:
- PUT /site-mode
- GET/PUT /settings
- all /users/* (list/create/update/delete)
Content routes (posts, wiki, categories, tags, uploads, dashboard,
activity) remain available to editors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extend the wiki's RichTextEditor to the Posts editor and close the
stored-XSS gap on public post bodies.
- RichTextEditor: add `variant` prop — `full` (wiki), `post` (no
internal wiki-page link picker), `minimal` (image-only, for
Screenshots captions). Toolbar sections rendered conditionally.
- PostEditor: replace the body textarea with a lazy-loaded
RichTextEditor in Suspense; variant chosen by category
(minimal for screenshots, post otherwise).
- posts.model: sanitize body via shared cleanBody on create/update,
treat an empty TipTap `<p></p>` as null, and auto-derive the
excerpt from the body (max 280 chars) when left blank.
- sanitizeHtml util: add deriveExcerpt() helper.
- FiveOnFriday / NewsletterIssue: wrap dangerouslySetInnerHTML with
DOMPurify.sanitize() as defense-in-depth on render.
No schema or dependency changes. Verified end-to-end against the
local stack: 24/24 API assertions and a full UI round-trip across
all four post categories.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Foundation & safety phase of the wiki upgrade (see WIKI_UPGRADE.md).
Schema (additive, idempotent via ensureSchema):
- new wiki_categories table; wiki_pages gains category_id, excerpt,
published, published_at, sort_order, and a FULLTEXT index
- migration ALTERs guarded with IF NOT EXISTS for existing databases
- seed reworked into 4 sections with the 8 starter pages assigned
Security:
- new utils/sanitizeHtml.js (sanitize-html allowlist); wiki bodies are
sanitized on every save, and the article renders through DOMPurify
- strips <script>, event handlers (onerror), and javascript: URLs
Backend:
- public: published-only list with ?category filter + /wiki/categories
- admin: extended page CRUD, PATCH publish toggle, category CRUD;
drafts visible to admin, hidden from public
- all writes logged to activity_log
Frontend:
- data-driven public wiki index (sections + real descriptions; removed
hardcoded blurbs/Roman numerals) with ?category filtering
- article: category breadcrumb + sanitized render
- admin: Section/Status columns, draft/publish + section + excerpt in the
editor, and a Manage sections modal
Verified end-to-end against MariaDB 11: migration clean, XSS neutralized,
drafts hidden, client builds, server boots.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
