61 Commits

Author	SHA1	Message	Date
wtclaude	12ff201ed5	feat(engagement): templates — the email block family, renderer and seeded set (engagement Phase 5a) ... Every subject and body moves out of `mailer.js` into `engagement_templates` rows an operator can edit. A relocation, not a regression: nothing that sends mail today starts depending on an operator authoring something first. - `email.*` block family in its own registry, sharing the page family's envelope walk and validate-then-sanitize order by binding rather than by copy. - A server-side renderer producing both parts of a multipart message; the text part is byte-identical to the literals this commit deletes. - Nine seeded templates, six of them wired now; the seeder's `customized = 0` guard lives in the UPDATE's own WHERE. - `renderByKey` falls back to the shipped seed when a row is missing or unusable, so no failure of the table can stop a password reset. Also fixes `check:hosts` reading the template key `auth.email-verify` as the hostname `auth.email`. Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-29 13:07:39 -05:00
wtclaude	2079aaf667	feat(engagement): the rules engine, cooldowns and outbox (engagement Phase 4a) ... Phase 4 of docs/website/ENGAGEMENT.md, split 4a/4b at the org lead's direction. This is 4a: the engine, server only, with no HTTP surface at all. A fired trigger now produces outbox rows and send-log entries; Admin - Engagement - Rules and the segment composition UI are 4b. Five tables (rules, audience segments, cooldowns, outbox, sends), the sweep worker, audience resolution, condition evaluation, the grace window and its cancellation, and the save-path validation 4b's form will call. engagementEmit's Phase 2 log line becomes the engine call. Two settled questions this phase was blocked on: Q2 (multi-instance) - neither SKIP LOCKED nor documented single-instance: the outbox claims each row with a compare-and-set into the 'sending' state the ENUM already carried. It makes the outbox safe for two instances, not the deployment. Q4 (admin surface) - its own top-level nav group, built in 4b. Two defects in the plan's own section 4, both found by building it: The global UNIQUE(dedupe_key) was data loss. A dedupe key names the EVENT, and one event is one row per (rule, user, channel) - so a fifty-person audience would have had one row admitted and forty-nine silently ignored. Scoped. Section 4.1's single INSERT ... ON DUPLICATE KEY UPDATE cooldown claim always passes against this codebase's pool: the mariadb connector defaults foundRows:true, so a no-op update reports affectedRows 1 rather than 0. It is two statements now, with the interval guard in a WHERE clause. The second defect is why there is a second test file. The stubbed suite was green against the broken claim, because a stub can only agree with whoever wrote it; engagementEngineSql.test.js runs the raw statements against a real MariaDB and skips when there is none. Verification: 43 new tests green in engagementEngine.test.js, 12 more against MariaDB 11.8, and the whole path exercised end to end against a live database - per-subject cooldowns, conditions, the CAS claim, the send log's honest failure detail, and dormancy on uninstall. The three pre-existing Windows-only CRLF failures in the generated-artifact tests are unchanged from clean edge. Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-29 08:07:27 -05:00
wtclaude	b13ffd584f	feat(notifications): per-channel preferences and the delivery-channel registry (engagement Phase 3) ... `notification_subscriptions` answers one question — which streams a user wants PUSHED — because that is the only question the shipped Android client can ask. This adds the general one: which subscribable ids, on which channel, in which mode. The old table becomes the push projection of the new one and keeps its exact wire shape, so the shipped APK needs no update and no delivery path is touched. What lands: - `engagement/channels.js` — `registerDeliveryChannel` (ENGAGEMENT.md §3.1), the declarative half only: id, label, `carriesContent`, `defaultMode`, `supportsDigest`. `addressFor`/`render`/`deliver` wait for Phases 6 and 7, for the reason `transports/index.js` deferred this file at all. `coreChannels.js` declares push / email / inapp through the subsystem's one door. - `notification_channel_prefs` + a replay-safe `INSERT IGNORE … SELECT` backfill, copying the `announce_jobs → announce_job_legs` precedent. - `GET · PUT /auth/me/notifications/channels`. The PUT is SPARSE — only the `(id, channel)` pairs named are written — deliberately unlike the two whole-set PUTs beside it. `off` is a mode rather than an omission, so this endpoint has no empty-array case and the kotlinx DTO gotcha cannot arise here. Three decisions the org lead settled before any code, and one corrects the phase's own acceptance criterion: push's `defaultMode` is `off`, not `instant`. The plan borrowed "push is opt-OUT" from `team_notification_prefs`, where no row does mean notified — but stream subscriptions have never worked that way, so `instant` would have projected the whole catalog into the legacy GET for every existing user and switched every toggle on in the shipped app after an upgrade nobody asked for. A test pins the legacy GET at `{streams:[]}` for a fresh user. One thing not named by the phase, and it is a G24 consequence rather than scope creep: a trigger ceilinged at `staff` can never reach a non-staff user, so offering the toggle would be offering a dead control AND disclosing the event exists — `uo.cheat.detected` would otherwise appear in every player's screen the moment Phase 11 declared it. Filtered from the catalog and gated on write. That gave the `staff` label its first consumer, now written down as `ceilings.STAFF_CEILING_ROLES` (the admin tier's three, deliberately not `teamGrants.STAFF_ROLES`, which answers a different question). 15 new tests; swagger, route manifest and guards regenerated. No web or app surface — those are Phases 7 and 8, where a preference governs something visible. Refs: docs/website/ENGAGEMENT.md Phase 3, §3.1, §4.5 Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-29 07:08:17 -05:00
wtclaude	fbb4b0bd91	feat(auth): unique, changeable, verifiable email addresses (engagement Phase 1b) ... Makes `users.email` unique, de-duplicates the addresses an upgrade will find, and builds the self-service change-and-verify flow that did not exist. The uniqueness index is on a generated `email_norm AS (LOWER(email)) STORED` column under `utf8mb4_bin`, NOT on `email` under a `_ci` collation as the plan specified. Every case-insensitive collation this server offers is also accent-insensitive: `josé@x.com` and `jose@x.com` compare equal, and those are two different mailboxes. The plan's index would have refused the second address forever and the de-duplication would have nulled a legitimate account's. A requested address is STAGED in `email_pending` and only a tokened link installs it, so a typo cannot silently redirect account-recovery mail. `isDuplicateUsername()` now distinguishes the two indexes. All five call sites branch on it; each answers differently on purpose, because a public form, an IdP callback, a half-completed invite and an admin screen do not owe the same person the same amount of truth. SSO reads the IdP's actual `email_verified`/`verified` claim instead of inferring verification from an address merely being present. Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-29 01:53:50 -05:00
Claude	47c8b37d45	feat(email): remove Gmail OAuth2, put SMTP behind a transport registry ... Engagement Phase 1 (docs/website/ENGAGEMENT.md §1.2a, §3.1, §3.2). A subtraction and a replacement in one commit, because leaving the OAuth2 flow half-wired across a release is worse than either end state. Deleted, per the §1.2a inventory: GET /admin/email/connect/start and /connect/callback, the connectStart/connectCallback controllers with the email_oauth_tx signed cookie, the PKCE verifier and CSRF nonce plumbing, the https://mail.google.com/ scope, the borrowed `google` auth-providers client, the OAuth2 nodemailer transport with its smtp.gmail.com:465 literals, the refresh-token decrypt in the model, and the client's Connect Gmail button, redirect banner and six Gmail error strings. `provider` and `refresh_token_enc` stay as columns under the additive-only discipline, unread. Added: a mail transport registry (server/src/engagement/transports) with `smtp` as the sole registration. `credentialFields` is the single declaration the admin form renders, the sanitizer filters against, and the "is it secret" answer comes from, so adding a transport is a registration rather than four edits. email_config gains transport / credential_enc (one encrypted JSON blob, since the field list is the transport's to declare) / reply_to. All six call sites keep their exact failure contracts: the contact form's mailto fallback, the invite's copyable link, the reset's generic 200, and sendTeamNotification's never-throws. One deliberate behaviour change: `enabled` now gates every sender rather than only isConfigured() — the connect flow used to set it as a side effect, and with a credential form the toggle has to mean what it says. Send-test becomes the real verification. Under OAuth2 the sender came back from Google and was guaranteed to belong to the credential; operator-typed, it can be refused, so failures name the sender and the SPF/DMARC reason (§1.2a consequence 2). G22, the silent degradation: an upgraded deployment backfills to smtp with no credentials and every sink politely does nothing. The admin dashboard now warns when the deprecated Gmail token is present and no replacement credential is, so the one deployment this happens to is told. A fresh install has never had mail and is not nagged. Guardrails: new `npm run check:hosts` (§3.2 rule 4) with its own self-test, wired into pr-checks before the install; routes.manifest and routes.guards regenerated (-2 routes). Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-28 20:52:55 -05:00
wtclaude	61abb3ec89	feat(teams): phase 9 — one voice channel per Team, granted by a role ... TEAMS.md §7.3. Each qualifying Team gets a Discord voice channel of its own and a role that opens it, kept in step by a reconciler that rides the Team reconcile it already depends on. Access is a per-Team ROLE, always. §7.3 designed per-member overwrites with escalation to a role above ~90 members; the org lead settled on roles always (2026-08-18), which deletes `voice_overwrite_max`, the escalation and the `mode` column — and moves the ceiling. Overwrites are capped per channel, so the old shape's limit was "how big can one Team be"; roles are capped per guild at 250, so the new one is "how many Teams can have voice at all". That is a limit an operator must be told about before they hit it, so the panel reports it and the pass refuses the create rather than letting Discord do it. Three things §7.3 named that this codebase does not have, all settled by asking the operator because nothing in the data model can answer: - "the staff role" — there is no staff-role concept anywhere. Now a list of role ids the admin designates; empty is a normal answer, since guild administrators bypass overwrites and what is really missing is a way to let NON-admin staff in. - the parent category — §7.3 said the bot creates it and gave the id nowhere to live (`team_integrations.team_id` is NOT NULL). The bot creates it and the server stores the id in settings. - whether the bot can act at all — nothing has ever checked. The operator invites the bot by hand and no invite URL with a permission integer exists in the tree, so a deployment can be one unticked box from every call failing. A preflight is now a PRECONDITION to enabling (422), not a per-Team error discovered afterwards. Two more, decided rather than asked: - the threshold counts every active member, not linked ones. §7.3 wrote `voice_min_linked_members`; the operator is judging whether a Team is real, and link state answers a different question. - hidden Teams are never provisioned. A channel name is a game-sourced string published outside the site, which is exactly §2.8's concern — reservedNames.js already names "and eventually a Discord channel name" as a surface it protects — so the screen that suppresses a Team's page suppresses its channel, and a Team that becomes hidden takes the grace window. Turning voice OFF tears nothing down: the pass suspends in both directions and the panel offers per-row removal. A checkbox must not delete structure in somebody's guild. Fixes a phase 8 defect that blocks this phase's own artifact: `npm run swagger` has been unable to run on `edge` at all. `param('teamId').custom((v) => ... || /^[0-9]+$/.test(v))` makes swagger-autogen's parser run away — a regex literal followed directly by `.test(`. Hoisted to a const, as modules.router.js already does. Underneath it, `teams.router.js` sits exactly at that parser's per-file limit: at twenty `teamsRouter.*` statements it dies, at nineteen it generates, and one more statement of ANY shape tips it — an unannotated route does, and so does a bare `use`. So the voice routes are their own router file mounted from `admin/index.js`, and teams.router.js keeps its nineteen. Also breaks a require cycle this phase would have introduced: teamSync -> teamVoiceSync -> teams.model -> teamSync left `teams.model` holding the reconciler's exports object as it stood mid-load — the empty one, since `module.exports = {…}` replaces rather than fills. The symptom is not in the new code: it is `teamSync.intervalSeconds is not a function` thrown out of `syncStatus()`, the freshness banner on every public Team page. Tests: 1160 server (+40), 53 bot (+21), 284 client (+21). Swagger, routes manifest and guards regenerated; the guard shape of the four new routes is byte-identical to the existing admin-only ones. Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-18 23:49:28 -05:00
wtclaude	11b4368b57	feat(teams): phase 8 — the notifications bridge, and the gate §7.2 could not check ... The same Team event as §6, delivered a third time: push, email, and now a Discord channel the operator configured. Not a second pipeline — teamNotify.js already computed the recipient set once, so the bridge is a sink beside the two that were there. The design's gate has no data source. §7.2 bridges an event only if "its visibility is public, or its destination channel is configured for a members-only Team context". The four team.* streams carry no visibility; forum threads have no public/members column because a forum is members-only by construction; and core cannot see a Discord channel's permissions. So §7.2's own example config names exactly the two events that are never public. The gate is therefore an attributed operator acknowledgement, in the shape teams_forum_uploads_ack already uses. It is a precondition — 422, not a quiet drop at delivery — it is re-asked at delivery as well as at the save, and changing the channel clears it, because an acknowledgement is about a destination and cannot survive the destination changing underneath it. The design's DDL cannot hold its own default row: MariaDB coerces every PRIMARY KEY column to NOT NULL, so `team_id NULL` — the deployment-wide default every override overrides — is unrepresentable. Proved on a real MariaDB (error 1048). Replaced with a surrogate id, a generated team_key AS IFNULL(team_id, 0) in the unique key, and the foreign key the original had no room for. One-shot, not queued: "identical to announce and mod-reverse" names two different reliability models, and a Team notification is the moment it describes. Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-18 20:25:30 -05:00
wtclaude	26c23bd603	feat(teams): the notification core — four streams, and a recipient set ... Phase 6's foundation: the fan-out shape the existing pipeline could not express. `pushDispatch.publish` answers "everyone subscribed to a stream" and "this one owner". Team notifications need "these N users", because Team scoping cannot live in a stream id: the catalog is a static registration validated at boot against a namespaced pattern, so a stream per Team is unexpressible, and stream ids are stored in `notification_subscriptions` rows that would need collecting every time a Team archived. So there are FOUR fixed core streams and the Team lives entirely in the recipient set. `team_notification_prefs` is opt-out for push and opt-IN for email — the two sinks default opposite ways, and the asymmetry lives in the column defaults so no condition anywhere has to remember it. One recipient query serves all four streams, because §6.2's two populations are the same set written twice: "active members with a user_id plus active grants" IS "everyone with resolved forum access". Mutes are subtracted in SQL rather than by the caller — there is no function here that returns an unfiltered set. Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-18 14:34:24 -05:00
wtclaude	fff14848f1	feat(moderation): member-raised abuse reports, to site staff only ... TEAMS.md §5.6. **Core has had no user-facing report flow of any kind** — the `moderation`, `mod_notes` and `appeals` tables are all either staff-initiated or Discord-sanction-shaped, and nothing anywhere let a member say "this is a problem". That was survivable while every piece of content on the site came from staff; phase 5 lets players write to each other, so it stops being. The gap has a specific shape: leaders moderate their own Team's forum, and a Team's leaders are exactly the people who will not report their own Team. So the whole point of this queue is a path that routes AROUND a Team's own leadership. Org lead settled it on 2026-08-18: **reports are site administration only** — there is no leader-facing view of this queue, not even a read-only one scoped to their own Team. §5.6's "a leader may also see and act on reports for their own Team" is not implemented and is not deferred. `content_reports` is deliberately generic — `target_type` is a VARCHAR so a wiki page or a news comment becomes a value rather than a table — and the queue is mounted beside appeals under /admin/moderation rather than under Teams, because a staffer working a queue should have one place to work. **§5.6's literal unique key has a defect and this does not copy it.** Written as (target_type, target_id, reporter_user_id, status) it makes CLOSED rows collide with each other too: reporter reports a post, staff dismiss it, the behaviour recurs, they report again — and the second dismissal is an UPDATE into a tuple that already exists, so working the queue starts throwing duplicate-key errors on the first repeat reporter. The key is on a generated `open_marker` instead, the same trick `team_forum_grants.active_marker` uses: 1 while open, NULL once closed, and NULLs are distinct — which is what §5.6's prose asks for, "one open report per (target, reporter)". Two other departures from the doc, both small and both flagged in the docs PR: `handled_note`, because a queue whose resolution reason lives only in an activity_log line is one where the next staffer to see a repeat report cannot find out why the last was dismissed; and a CASCADE on `team_id`, so a deleted Team does not leave a queue full of reports about content that no longer exists. Also here: a report is filed against a target the model verifies really belongs to the Team the request came through, or the queue's per-Team filter would quietly be lying; the queue resolves every row's target in three batched reads rather than N+1, which is §5.6's fourth rule (uploader, size and sniffed type without hunting) actually paying for §5.5.4's attribution table; a target that has since been hard-deleted comes back null and the report still lists, because "somebody reported this and by the time we looked it was gone" is a fact a moderator needs; and every transition writes activity_log, `dismissed` included — a queue where acting is audited and declining to act is not is one where the cheapest way to make a report vanish leaves no trace. `teams_forum_edit_window_minutes` gains its range validation on the admin settings PUT and is seeded at 15, so the value on the settings screen is the value in force. Route manifest and OpenAPI regenerated: 6 operations added, 0 lost. Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-18 12:51:42 -05:00
wtclaude	11fd9821bf	feat(teams): the forum schema, the operator's two switches, and the ack gate ... The whole forum schema lands at once — threads, posts, the moderation ledger and upload attribution — including the columns only phase 5's discussion threads use. That is TEAMS.md 5.1's split BY LAYER rather than by feature: phase 5 opens paths instead of migrating data. Three settings keys, and only one of them is ordinary. `teams_forums_enabled` and `teams_forum_images` are enum keys on the existing admin settings endpoint; `teams_forum_images` also carries a server-side PRECONDITION, which is why the three live in their own model rather than in the generic setMany() loop where a reader would never find it. The gate is the server's. `PUT teams_forum_images = 'uploads'` is rejected 400 unless the same request carries the acknowledgement version — the admin checkbox is how the gate is presented, never the gate. What is stored is the TEXT VERSION, so "which wording did they agree to" is answerable later; settings already record updated_by/updated_at, and an activity_log row puts it in the staff audit trail. A reworded notice makes a stored acknowledgement stale, and neither obvious answer is right: uploads KEEP WORKING, and no other forum setting may be saved until it is re-given. Non-destructive, and impossible to ignore. Both reads fail closed. A DB fault reports the forum off and images disabled — a forum that 404s for a minute is the cheap failure; a policy that is not a policy is not. Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-18 07:23:15 -05:00
wtclaude	aa332eda82	feat(teams): the activity feed, its two writers and its retention ... TEAMS.md Part 4. `team_activity` takes items from two sources and treats them identically on the read path: core writes its own membership and rename items with source='core', and a module pushes game items through `ctx.teams.activity.push`, which stops throwing and starts working. Core writing here too is deliberate — the rendering path is exercised by core's own content from day one, so the feed is never empty on a deployment whose module pushes nothing. Three rules shape the model: - core never composes a summary. It arrives already rendered and is stored verbatim; core cannot phrase "gained 15,000 gold" for a game whose vocabulary it does not know. - visibility fails closed. An item with no stated visibility is `members`. - a push never throws at its call site. It is called from inside a game-event handler, and a storage problem of core's must not become the module's control flow. Core emits four of the five kinds §4.2 names — `core.forum.thread` has nothing to emit it until the forum lands in phase 4 — and emits none of them for a Team's FIRST roster: importing a 155-member guild is one Team arriving, not 155 people joining, and a join per member would bury every real event under the import and reach the row cap on day one. Retention ships with the feed rather than after someone notices. A nightly worker applies an age horizon and a per-Team row cap, both settings; either alone has a hole, since age lets one busy guild write a million rows inside the window and a cap keeps a dead Team's feed forever. The sync now reads member ROWS rather than keys, replacing the `memberKeys` call rather than adding to it: the feed needs each changing member's display name and prior `is_leader`, and the upsert is about to overwrite both. Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-17 20:15:18 -05:00
wtclaude	92631347f9	feat(teams): the reconciler, its four refusal gates, and ctx.teams (API 1.6.0) ... Core's projection of the module's Teams, kept in step (docs/website/TEAMS.md §2.4), plus the two ctx members a module pushes through. The four gates are the file, and each is invariant 1 in a different costume -- module unavailability is staleness, never emptiness: 1. getTeams() not ok -> record the failure, touch NOTHING, return. 2. ok but empty, core holds >=1 -> quarantine; apply only if the NEXT authoritative answer, an interval later, agrees. 3. getTeamMembers() not ok -> that Team's roster untouched and stale; the other Teams sync normally. 4. ok but zero members, had some -> the same two-strikes quarantine, per Team. Gates 2 and 4 exist because an authoritative-looking empty answer during a cold start is the one failure indistinguishable from a real wipe. "Every Team on the shard disbanded at once" costs one interval to confirm; getting it wrong empties every roster on the site. Events are an optimisation, never the source of truth. Member and leadership deltas apply at once for a Team core already knows; team.created and team.disbanded only ask for a run. §2.2 scopes archival to an authoritative full list, so a repeated or spurious disband event costs a reconcile rather than a Team -- and a Team invented from a delta would have no name, no roster and no leaders anyway. Two columns TEAMS.md did not contemplate, both on `teams`: - roster_synced_at, because team_sync_state holds one row per MODULE and gate 3 leaves ONE Team behind while the others sync. Without a per-Team stamp that Team's page would report the module's last success as its own -- exactly the staleness the gate exists to surface. - members_empty_since, gate 4's per-Team quarantine. The twin of team_sync_state.pending_empty_since, which is per module and cannot express it. One real bug found by its own test. The roster upsert was writing is_leader, so a refused getTeamLeaders() left every member demoted -- the roster had already written `leader: false` before the authoritative call was even made. §2.5 is explicit that path 2 is answered by getTeamLeaders(), so is_leader is now set on INSERT only (seeding a Team so it is not leaderless while that call fails) and moved afterwards by setLeaders() alone. Two writers for one column was the whole defect. MODULE_API_VERSION 1.6.0 on both halves -- they state one contract and a module declares one coreApi range. The number covers the whole Team surface per Part 11; the members arrive by phase. registerTeamProvider, ctx.teams.publish and ctx.teams.reconcile are live. ctx.teams.activity.push (§4, phase 3) and api.registerSlashCommands (§7.1, phase 7) are present and THROW with a sentence naming their phase, rather than being absent or silently accepting data into tables that do not exist yet. 39 tests here, and the ctx surface guard in moduleLoader.test.js updated -- it caught the addition, which is what it is for. Server 809 passed, client 192 passed, 0 failed. Refs docs/website/TEAMS.md §2.2, §2.3, §2.4, Part 11, Part 12 phase 2 Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-17 14:53:52 -05:00
wtclaude	225663d62e	feat(teams): core schema for Teams, membership, sync state and moderation ... The six core tables Team core is built on (docs/website/TEAMS.md §2.1, §2.5, §2.5.1, §2.9), plus the §2.10 account-deletion decisions expressed as foreign keys rather than left to whatever the defaults happened to be. Every table is core-internal (§10.3): a module populates them through the team provider and must never read or write one directly. They carry no <moduleId>_ prefix, correctly -- MODULE_API.md §2.6's prefix rule binds modules, and these are core's. team_forum_grants lands in this phase rather than in phase 4, so the four-path resolver is written once and its non-contamination tests are real. Nothing writes it yet; the grant/revoke flow, the per-Team cap and the leader UI are phase 4's. Two departures from the SQL as TEAMS.md sketched it, both recorded in the file: - team_forum_grants.user_id is nullable with ON DELETE SET NULL, following §2.10 (the audit trail of who granted whom must survive the account) rather than §2.5's CASCADE. - its uniqueness marker is derived from revoked_at alone, with user_id moved into the unique KEY. §2.5's `active_user AS (IF(revoked_at IS NULL, user_id, NULL))` cannot coexist with the line above: MariaDB refuses ON DELETE SET NULL on a foreign key whose column is a base column of a STORED generated column (error 1901). The semantics are identical -- at most one active grant per (team, user), unlimited revoked rows. Verified by running ensureSchema() against MariaDB 11: all six tables create, both generated columns materialise, and every foreign key's delete rule matches §2.10's table. The uniqueness encoding was checked directly -- a second active grant for the same (team, user) is rejected 1062 while revoked rows accumulate freely. Refs docs/website/TEAMS.md Part 12 phase 2 Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-17 14:40:26 -05:00
wtclaude	b30e82cde2	feat(modules): install, uninstall, purge and restart (phase 4, slice 1) ... The consumer half of a release module-uo's CI has been publishing since phase 3 closed. Before this, core had the installed_modules provenance columns and no code that could ever fill them: nothing fetched, verified, unpacked, removed or purged anything, and there was no admin route at all. Adds modules/archive.js, modules/install.js, schema.runPurge(), lifecycle.stop(), loader.stopHook(), and /api/v1/admin/modules with eight routes. 797 server tests (+76), manifest 158 -> 166 + 2 internal, OpenAPI gains 8 operations and loses nothing. Reject, never sanitise ---------------------- The download is the easy part: an https-only allowlist re-checked on every redirect hop, a declared sha256 compared against the bytes that arrived, and a byte cap. Unpacking is where the archive chooses the filenames, and core writes into a directory bind-mounted from the host, so an escape is not confined to the container. archive.js inspects the whole archive before a byte is unpacked and refuses absolute and drive-absolute paths, `..` segments, NUL bytes, backslashes, anything that is not a regular file or a directory, more than one top-level entry, and anything over the entry or byte caps. Refusing symlinks and hardlinks outright is what keeps this off the majority of node-tar's published advisories rather than depending on the library to contain them. That two-pass shape is load-bearing, and it was measured rather than assumed: extracting an archive whose fourth member escapes upward throws under node-tar 7.5.22 -- and leaves the first three members on disk. The loader scans that directory at require time on the next boot, so a half-unpacked module is a module. Everything therefore happens in a scratch directory that is removed on any failure, and the move into place is the last step. `tar` is pinned to ^7.5.22 rather than the ^6 that installs by default: 6.x is flagged critical, and reading the advisory list is what the file's header now says out loud -- almost all of it is hardlink or symlink traversal and PAX header interpretation differentials, which is exactly this feature's threat model. Two things the plan had wrong ----------------------------- The bundle's top-level directory is `module-uo-<version>`, not the module id -- so "the top-level name must equal the id" was checked against nothing real. The extractor strips that level instead, because its name belongs to whoever published the bundle and the directory it lands in is core's. What is checked instead is the unpacked module.json: a manifest promising `uo` and delivering something else is refused rather than installed under the name it promised. And purge cannot be a follow-up action (decision 5): purge.sql lives inside the directory uninstall deletes. It is offered in the uninstall flow and as a standalone action on a still-installed module, and the standalone one refuses unless the module is already disabled -- dropping tables under something that is still serving leaves it answering out of a world that no longer exists. Disable now means stopped ------------------------- lifecycle.stop() dispatches that one module's onShutdown before flipping the guard, so a module an operator switches off actually releases its sockets and closes its streams instead of merely becoming unreachable. The hook runs first and the state moves after it, because while onShutdown runs the module is still `started` and that is the only state in which its routes and the world it is tearing down agree. A hook that throws does not stop the disable -- the opposite of the boot path's rule, and deliberately. Enable is not its mirror and there is no start(id) beside it. There is no onBoot re-dispatch and the hooks were never promised re-entrant, so enable moves the row and the restart route starts it. A test pins that enable does not touch the loader, because "fixing" it is a one-line change that would put a module with closed sockets back on the nav. Restart raises SIGTERM against its own process rather than calling the shutdown path directly, so server.js's handler stays the one graceful-shutdown path and this route cannot drift from it. The allowlist bootstraps from MODULE_SOURCE_HOSTS into a settings row and is admin-managed after that (decision 6); seedDefault is INSERT IGNORE, so changing the variable on an existing deployment is a no-op by design. An empty list forbids every install rather than allowing every host -- the safe direction for a value someone might blank by accident. Verified against the real v0.3.0 release ---------------------------------------- Not a fixture: fetched the published install manifest over the real Gitea host and its redirect chain, verified the sha256, inspected and unpacked the 252,517-byte artifact to 82 files, and then booted core against the result -- the module registered its five mounts, seven streams and eight capabilities and resolved its client chunk, with no scratch directory left behind. Two defects this slice's own tooling caught, both of which had already been written down as classes: - the controller destructured runPurge at require time, capturing the function rather than the module, which made the one dependency whose ORDER matters the one that could not be substituted; - two swagger annotations carried an apostrophe inside a quoted string, dropped silently by swagger-autogen before slice 5 taught it to fail loudly. Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-12 03:09:45 -05:00
wtclaude	0c4eacfa4a	refactor(modules)!: de-UO core's copy, and enforce it (phase 3, slice 4) ... Phase 3's acceptance criterion 1, made real. Three things, one review: **The dead bindings.** `client/src/api/client.js` still carried ~190 lines of UO namespaces — `shard`, `atlas`, the two SSE URLs, `admin.shard/shardOps/atlas/ userShard`, the uo-link and town-crier calls, `player.shard` — with zero core consumers since slice 3 deleted the views. module-uo vendors its own bindings. The five assertions core's `apiClient.test.js` made about those URLs moved with them (Module-uo#5); the encoding test that used `governorHistory` now uses a core route. **The copy.** Core is the platform, not one game's site, so its words are game-neutral now: `About`, `Screenshots`, `Website`'s cards, `Status` (which was never about a game server at all — it reports site mode), `Wiki`, `SiteFooter`, the default hero, `brand.js`'s tagline and description, the seeded wiki categories, and two user-visible NavEditor strings that named a module's admin screen by its proper name. Which game an instance is for is the operator's to say — BRAND_* vars, the hero editor, CMS pages — and every real instance already does: `.env.uomysticmoon.example` sets both brand strings explicitly, so nothing live changes wording. Wiki page SLUGS are untouched: `seedDefault*` only inserts what is absent, so renaming one adds a duplicate page to every install. Also gone: an orphan comment block in `schema.sql` describing the spawn-atlas tables slice 1 took away, and the two settings rows core seeded for a module (`game_account_signup`, `uo_link_protocol_3_migrated`). The second was a live defect — see Module-uo#5, which takes ownership of both and repairs the one-shot migration core's ordering had disabled. **The check.** `scripts/checkModuleIdentifiers.js` + `npm run check:modules`, first step of the server-tests job because it needs no dependencies. It reads CODE, not prose — file names, import specifiers, route path literals, declared identifiers and property names — per §5.2, so core's English may still say "shard" where saying it is worth more than the word costs. Two things it gets right only because getting them wrong was tried first: it matches WHOLE WORDS (a substring pass flags `defaultImage`, which contains "ultIma", four times in this repo), and it strips comments and string bodies in one character walk (a comment contains quotes, a string contains `//`) — the `checkImports.js` lesson. It has its own 17-test suite, because a boundary check that silently stops checking is worse than none. The three §6.5 grandfathering allowlists are exempt by name, and an exemption that stops matching fails the build rather than lingering. BREAKING CHANGE: core no longer seeds `game_account_signup` or `uo_link_protocol_3_migrated`; module-uo's schema fragment does. An install running core without module-uo keeps whatever rows it already has and gains no new ones — nothing in core reads either key. Deferred to slice 5, deliberately: README.md's 48 UO mentions, including a `## Shard integration (uo-link)` section and the architecture diagram. That is documentation, which §5.2 does not cover, and it belongs with the phase-closing docs pass rather than half-done here. Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-11 21:41:18 -05:00
wtclaude	39d731d87a	refactor(modules)!: move the UO server half out to module-uo ... 40 files, ~9,674 lines, 27 of 68 tables. Core no longer contains anything that knows what a shard is. BREAKING for a deployment only in the sense that the module must be installed for these URLs to answer -- no URL moved. routes.manifest.json goes 228 -> 158 public routes here, and the 70 that left reappear byte-identical when the module is loaded: verified by generating the manifest against core+module and diffing it against the pre-extraction file. Zero missing, zero added, and routes.guards identical across all 228, so no auth gate moved either. The five tier mounts are gone from public/admin/player index.js and are still served: the loader mounts them onto the same routers after every core mount. That ordering is also what keeps the prefixes unclaimable -- the collision check asks the live router what core owns, so a second module claiming /shard is rejected against the mounts actually present rather than against a list. server.js loses its eight UO call sites to the module's onBoot/onShutdown. schema.sql loses its 27 shard_*/uo_link_* statements; the two that FK into users are why the fragment replays AFTER core's schema, and no core table ever referenced a module table, which is what makes core still able to boot alone. Verified against a running server with the module installed: it loads, mounts five prefixes, replays 35 statements, warms up and reaches `started`; public shard and atlas routes answer 200 with real data (800 creatures, 6,455 spawners), admin and player answer 401 from core's tier gates, and the extension slot answers at /admin/users/:id/shard/*. Core's own SPA renders the shard and atlas pages unchanged against the module-served API, with no console errors and no CSP reports. Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-11 12:08:03 -05:00
wtclaude	f50541f374	feat(modules): ctx additions and the post-hook registry (API 1.1.0) ... Everything the extraction needed from core that ctx did not already offer. Additions only, so minor. ctx.activity.log, because an admin action a module performs has to land in core's one audit log or the trail has a hole exactly where a module operates the game -- a module keeping its own log would be a second place to look, which in practice means a place nobody looks. Write-only; reading the log is the admin panel's job and it spans every actor. ctx.users.getById, one function for one caller: the admin.users.detail slot router needs the user its prefix names. ctx.site.baseUrl, because a module has to build absolute links and §2.7 forbids it reading core's APP_BASE_URL -- a getter, not a captured string, so it cannot go stale against the env. ctx.middleware.rateLimit is core's makeLimiter, plus accountChangeLimiter handed over whole. The split is deliberate: a module states its own window and cap because it knows what its endpoints cost, and takes the plumbing from core so there is one express-rate-limit in the process and one place a breach is logged. accountChangeLimiter is shared policy -- core's /auth/me and /player/account sit behind the same counter -- so a module's account-change route has to land IN it rather than beside it. marketLimiter was UO policy living in core's file and leaves with the route it guards. registerPostHook is the fourth registry, and the last thing binding core to the module. Core's post controller called newsGump.syncPost directly: core's CMS naming a UO file. It now publishes what it already knows and a subscriber decides what to do with it. Not folded into registerAnnounceLeg, which fires on the same transition, because a leg is a one-shot DELIVERY with retry and classification while a post hook maintains idempotent STATE, runs on delete as well as save, and refreshes silently on an edit. Also fixes a real loader defect the extraction exposed: schema table names were matched against the RAW file, so a fragment whose header says "every CREATE TABLE carries IF NOT EXISTS" was rejected for a prefix violation on a table called `carries`. module-uo's fragment hit exactly that. Both scans now read split statements, which strip comments -- the same class of bug as a boundary check failing on its own documentation. Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-11 12:07:45 -05:00
wtclaude	6195c76d61	feat(modules): the three de-entanglement registries, with core as the registrant ... Phase 2 PR 4 of docs/website/MODULE_SYSTEM.md §2.7. Adds server/src/modules/registries.js and moves core's own notification streams, announce leg and users-detail routes behind it, so the three seams §1.8 and §1.9 named are exercised on every boot before any module depends on them. Registering is validate-then-commit per registrant: the loader stages what a module claims and the second pass commits it, so a module that throws halfway through register() — or fails checkDeclared after it — leaves nothing behind. That is the registry-side twin of PR 2's second-pass mount rule. Four decisions, all the recommended option: - announce legs became a child table. `announce_job_legs` replaces the towncrier_*/discord_* column groups, so the leg set is data: core registers `discord`, module-uo will register `towncrier`, and a module cannot ALTER a core table to add its own. Backfill is guarded on information_schema (a SELECT of a dropped column is a parse error, not a runtime one) and the columns go with DROP COLUMN IF EXISTS. Verified against the live dev DB: three legacy jobs migrated faithfully, three replays, no duplicates. - `mapEvent` dropped from registerNotificationStreams. §1.8 already inverts the push path so a module owns fromShardEvent and calls core's publish() with a stream id it resolved; a second mapping mechanism was a leftover. The public safety filter, the kinds it reads and the streams it protects now live in one file and move together. - core registers through the same staging area a module uses, via an explicit registries.registerCore() in app.js before modules.load(). - core's six /admin/users/:id/shard/* paths now go through the `admin.users.detail` slot, and getUser moved back to admin.controller.js. Found on the way, and the reason two build tools changed: - scripts/routeManifest.js could not decode a parameterised mount. Its unwinder expected `(?:([^\/]+?))`; express 4.22 emits `(?:\/([^/]+?))` with the separator inside the group. The branch had never run. It threw rather than guessing, which is what it is for. - swagger-autogen cannot follow a route into an extension slot — the slot's router is created by declareSlot() and filled later, so there is no literal mount for a static parse. Regenerating deleted 407 lines and printed `Swagger-autogen: Success`, the spike's exact failure (MODULE_API.md §7.4). swagger/slotSpecs.js generates a fragment per filled slot and re-roots it at the prefix the router actually hangs at in the live app — read from the express stack via routeManifest's own mountPath, so the manifest and the spec cannot disagree. swagger/mergeSpec.js is the merge helper core owes for module fragments anyway (§6.1a), proved here against core's own slot first. 884 tests pass (856 before). routes.manifest.json is unchanged at 229 routes. The OpenAPI spec diff is two lines of intent: the retry endpoint's summary, and its `leg` no longer being a fixed enum. Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-10 17:47:59 -05:00
wtclaude	3add0063bf	feat(modules): installed_modules and the module state machine ... Phase 2 PR 1 of the module system (docs/website/MODULE_SYSTEM.md 2.7). The table and the state machine only: no loader, no routes, no boot wiring, so nothing an operator or a client can see changes and the route manifest diff is zero lines. The five states of 2.4 live in one `state` column: installed -> enabled -> started, with disabled and startup_failed as the recoverable ones. The row is a record of what happened, never the source of truth for what is mounted -- the loader scans the filesystem at require time, before the database is reachable (MODULE_API.md 4.1), which is what keeps routes.manifest.json generatable against a dead database. Two rules the model owns and the boot path will lean on: - Every boot resets each non-disabled row to `enabled` and clears its recorded failure, so a startup_failed module is retried on the next restart and a fixed one recovers with no admin-panel visit. `disabled` is the one operator decision rather than outcome, so it survives untouched -- and a disabled module's failure is a no-op, never a re-enable. - A failure carries the stage it happened at, and every non-failing transition clears it, so a running module can never show a stale reason. An illegal move throws instead of writing a row that misrepresents the state, except on the two boot-path softenings noted above, because one module's failure must never become everybody's. 22 model tests over an in-memory fake; the SQL and the DDL were round-tripped against a real MariaDB separately. Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-10 06:35:57 -05:00
wtclaude	779a304173	feat(shard)!: declare wire protocol 3 ... The site's declared version is the admin-set uo_link_config.protocol column, so the sidecar's PROTOCOL_VERSION 2 -> 3 bump has to be matched here or every REST call 409s and uoLinkSocket closes the WS on the ws.hello mismatch. Five places carry the number and all five move together: the column default, the model's DEFAULT_PROTOCOL (what a site with nothing saved yet declares), the two `config.protocol || 1` fallbacks in uoLinkClient/uoLinkSocket -- unreachable today, but an unset value quietly sending 1 is exactly the confusing 409 the version check exists to prevent -- the admin form's initial value, and the documented env default. The boot migration is the only subtle part. schema.sql is re-run on EVERY boot, and `protocol` is admin-editable, so a bare UPDATE would silently un-pin an operator who had deliberately pinned an older sidecar in Admin -> Shard. It is therefore gated on a marker row in `settings`, written after the UPDATE: the first boot on this build migrates, every later boot is a no-op. `protocol < 3` rather than `= 2` picks up an install still on the old default of 1, which could not have been talking to a v2 sidecar anyway. A fresh install has no row to update and just gets the marker plus the new column default. Verified against the local MariaDB through ensureSchema (the production path): 2 -> 3 with the marker written and the column default now 3; pinned back to 2 by hand, re-ran, and it STAYED 2 -- the one-shot property holds. 673 server tests, 47 client tests, client build green. Co-Authored-By: Claude <noreply@anthropic.com>	2026-07-29 18:03:48 -05:00
wtclaude	8771a1cf6c	feat(shard): the player-vendor marketplace ... Protocol 3.0 §8, the website half. Ingests vendor.listing / vendor.listing.remove into shard_vendors + shard_vendor_items, serves a searchable public API over them, and ships /site/market and /site/market/vendors/:serial. Three things the pages have to say out loud, all consequences of how the data is gathered: - The prices are NOT live. The shard sweeps vendors round-robin, so a shop can be a full cycle behind. The banner is driven by the OLDEST vendor row, not the newest — the one stale shop is the one that wastes somebody's trip. - A shop can be truncated. `total` exceeding `count` means the shop holds more than the shard publishes per frame; the vendor page says "showing 250 of 3,104" rather than presenting a partial shop as complete. - An item may have no name. On a shard with no cliloc table the honest render is the item id, never an invented label. ## The pre-wired visibility rules, re-checked Part A pre-wired market.ownerName and market.location before the frame existed, and the sibling rule it pre-wired for leaderboards (`characterName`) turned out to be INERT because projectValue matches literal JSON keys. Both market rules were checked against the real frame this time: - `ownerName` is a real key. Kept. - `location` is a real key ONLY because the frame nests it. Flat map/x/y/region would have made the rule match nothing — the same failure, one part later. It is nested on the wire and on the read model so one rule hides the facet, the coordinates, the region and the house together; five flat keys would be five rules that drift apart. - `ownerSerial` was ADDED. An admin who hides the owner's name and leaves a serial that the leaderboards and guild boards resolve back to that same name has not hidden anything. Tests assert all three bite, on the stored read model AND on the raw frame — the market's SSE stream is off by default but an admin can turn it on, and a rule that worked on only one path is exactly the leak §3.6.1 records. ## Notable - **No payload column on shard_vendors**, unlike shard_points_boards next door. The board's top-N is a fixed-size list read whole; here the items ARE the searchable rows, so they are normalized and nothing is left worth duplicating. - **display_name is denormalized at ingest** (literal name preferred over the cliloc — a player set it, so it is more specific). Resolving at query time would put the cliloc table on the hot path and make search-by-name impossible. Because the shard's diff sweep will not re-send an unchanged shop just because the site learned what its items are called, a cliloc import now triggers a bulk re-resolution — 50 ms per thousand rows, never throws. - **updated_at is written explicitly** on every upsert. MariaDB does not fire ON UPDATE CURRENT_TIMESTAMP when every column is written back unchanged, and a shop re-published identically is still freshly confirmed — without this the staleness banner would age a perfectly current shop forever. - **LIKE wildcards in `q` are escaped.** `%` and `_` are LIKE metacharacters, not SQL ones, so parameterization does not neutralize them: `?q=%` would otherwise match every listing on the shard. - **Rate-limited** (60/min/IP), the only limited public read. Every other public GET is an indexed lookup of bounded size; this is a LIKE scan plus a COUNT over the largest shard_* table, anonymous by default. - Reconnect backfill pages /market, bounded by MARKET_SNAPSHOT_MAX = 5000 and stopping on a short page as well as on `total`, so a concurrent sweep shrinking the index cannot spin the walk. ## How it was tested 673 server tests pass (27 new). Client builds clean; swagger-output.json, routes.manifest.json and routes.guards.json regenerated. Verified full-stack against the live MariaDB and a real shard, not only units: - 27 real vendors / 1,040 listings swept off the ServUO tree, through the Rust sidecar, into the site — names resolving through the cliloc table ("longsword", "katana"), real facets and regions in the filters. - `?q=sword` 682, `?q=%` and `?q=_` **0** (the escape), map/region/price/sort filters, paging, and the vendor detail route. - Visibility live: fields gated to staff vanish for an anonymous caller while shopName and price survive; audience=player 403s; enabled=0 404s; and /shard/features correctly drops `market` so the nav hides it. - Re-publishing a shop smaller leaves no orphan items; an identical re-publish moves updated_at. - The limiter fires (38x200 then 32x429 on a 70-request burst). Not covered by an automated test: the two React pages are presentational and this repo's client suite covers pure-logic modules only. They were driven against the live API above, but not rendered in a DOM harness. Co-Authored-By: Claude <noreply@anthropic.com>	2026-07-29 09:51:50 -05:00
wtclaude	b61a4d6721	feat(shard): resolve cliloc names for items and reward titles ... Protocol 3.0 §8.6 (docs/link/v3.md), the dependency order 5 was sequenced behind. Items on the wire carry a LabelNumber, not a name — the bridge has always sent it (char.profile.equipment.cliloc, reward titles as a cliloc number in string form, and one per marketplace listing) but the site had no table to resolve it against, so a character sheet could only render `id 1023721` where the game renders "quarter staff". The number was never the missing piece. The table was. Sourced from a file the operator converts once from their own client, at a path from the `cliloc_client_path` setting falling back to UO_CLIENT_PATH. Nothing client-derived is committed: UO's strings are EA's, exactly as the creature sprites are. A shard with nothing configured is fully supported — names render as ids, as they did before. The conversion step is not avoidable, and that is the substantive finding here: every current client ships its cliloc files COMPRESSED (first DWORD's high byte 0x8E, the Mythic container), and ServUO's own bundled Ultima.StringList cannot read that either — so VendorSearch.GetItemName is already inert on such a shard and the plugin could not supply names instead. v3.md's original "read the client's Cliloc.enu" recommendation was therefore not implementable as written, and its committed db/data/clilocs.json artifact also predates the Part C corrections (no committed derived snapshots, nothing EA-derived shipped). Replaced with the spawn-atlas pattern: parse on boot from an operator-configured path, hash-gated, output gitignored. - utils/clilocParse.js — pure parsers, fs-free so the suite runs in CI. Accepts the plain binary layout and delimited text, sniffed by header rather than extension. Rejects a compressed file BY NAME: without that check the plain parser reads it as ~19k records of negative ids and 60 KB "strings" before dying mid-file, and the resulting error names the wrong problem. displayText() drops the ~1_val~ arguments the bridge never sends. - utils/clilocSource.js — the fs layer. hashSource reports `compressed` so the admin panel can flag an unconverted file WITHOUT parsing 5 MB per poll; otherwise pointing at a client directory reports a healthy file with pending drift ("ready to import") and the operator only finds out on failure. - model/shardClilocs — refresh/status/lookup. All-or-nothing replace (DELETE, not TRUNCATE — TRUNCATE is DDL in MariaDB and implicitly commits). Batched server-side resolution behind a capped cache; never throws, because a cliloc lookup is decoration on a character sheet. - Deliberately NO staged-approval flow, unlike the atlas: the atlas escalates facet loss because a half-copied tree and a real map change are indistinguishable from inside the process, whereas a partial cliloc copy makes the parser fail on a truncated record. The ambiguity the atlas must escalate is one this parser simply detects. - No public route. The table is never served AS a table: 67k rows would dwarf any page using them, and the Android client consumes the same resolved JSON. Two parser bugs found by building it, both now covered by tests: trimming a text line before splitting ate the trailing separator on empty-text entries and silently dropped 55,994 of 123,490 while still reporting success; and Number('') is 0, not NaN, so a line starting with a separator imported as a bogus cliloc 0. Verified against the real client table (123,490 entries) and the live MariaDB: import 663 ms, hash-gated boot no-op 14 ms, cold resolve 4.2 ms / warm 0.015 ms. Binary and TSV imports converge on the same 67,496 rows with identical keys (blank entries — half the table — are dropped at import). A file truncated to half its length is refused with TRUNCATED and leaves the previous table serving. Boot logs verified for both the import and the compressed-file warning; neither blocks startup. All three admin routes exercised over HTTP with a real session. 629 server tests pass; client builds clean; swagger, routes.manifest.json and routes.guards.json regenerated. Not covered by an automated test: the character sheet renders resolved names in presentational React with no DOM test harness in this repo, and was not rendered against a live linked-player profile — that needs a logged-in player with a linked game account and a shard answering a profile RPC. Co-Authored-By: Claude <noreply@anthropic.com>	2026-07-29 04:21:38 -05:00
wtclaude	26094459ae	feat(shard): ingest points.board and publish the leaderboards ... Protocol 3.0 §7 (docs/link/v3.md). The shard publishes ~25 points/loyalty leaderboards — Queen's Loyalty, Void Pool, the nine city loyalties, Clean Up Britannia — and the site renders them, plus each character's own standings on their sheet. Server - shard_points_boards: one row per system, keyed by the shard's PointsType name. The top-N list stays inside `payload` — a fixed-size list read whole, exactly like shard_governors.candidates. Normalizing into an entries table buys nothing until something needs a per-character reverse lookup, and a character's own standings already ride inside char.profile. - shardIngest routes points.board to upsertPointsBoard and deliberately does NOT log it: this is board state like guild.update, and the shard emits a frame every time anyone's score moves a top ten. - uoLinkSocket backfills /points through snapshot() with ingestEach rather than a replace*: there is no points.remove and the system set is fixed, so upserting IS the reconciliation, and a system the operator later excludes keeps its last-known board rather than vanishing. - GET /public/shard/points and /points/:system behind requireFeature('leaderboards'), both projected per §3.6.1. :system is constrained to an identifier before any query runs; 404 for a system never published, distinct from a published board nobody has scored in (200, empty top). The leaderboards field rule now keys on `name`, not `characterName` Part A pre-wired FEATURES.leaderboards.fields = { characterName: ... }, but projectValue matches on the LITERAL JSON key and the wire key is `name`. As written the rule was inert: an admin tightening character names would have got no enforcement and no error — precisely the failure §3.6.1 records for the flattened `ownerAcct` spelling. Fixed, with a test that fails if it is renamed back, and the admin panel's FIELD_LABEL carries the meaning instead. Client - routes/public/Leaderboards.jsx at /site/leaderboards. A points.board frame describes ONE system, so live frames merge over the fetched set by system key rather than replacing it wholesale the way the ruleset does. Filter matches board name, system key, or any ranked player — the last is what makes it useful ("where do I appear?"). - A "Loyalty & Points" section in CharacterSheet.jsx, one edit serving both PlayerCharacter and AdminCharacter. - Both treat maxPoints: 0 as UNCAPPED and both fall back to humanising the system key when nameString is null. Neither is defensive padding: on a real shard uncapped and cliloc-only names are the majority case. Verified end to end against the local MariaDB, the Rust sidecar, and the real ServUO shard: backfill from /points, live SSE delivery (a board absent from the initial fetch appearing without a reload, and an existing one updating in place), REST reflecting the overwrite, and the gate at every rung — 200 by default with names, names stripped but points kept at fieldRules name=staff, 403 plus dropped from /features at audience=staff, 404 when disabled. Page rendered clean, no console errors beyond the pre-existing React Router v7 warnings. 605 server tests pass; routes.manifest.json, routes.guards.json and the OpenAPI spec regenerated. Co-Authored-By: Claude <noreply@anthropic.com>	2026-07-28 21:04:44 -05:00
Claude	a4ef9d676d	Merge remote-tracking branch 'origin/edge' into feat/spawn-atlas-parse	2026-07-28 16:45:56 -05:00
wtclaude	2801ec8f4d	refactor(atlas): derive the atlas from the shard's tree on every boot ... Replaces the committed-artifact design from the first commit. Two problems with it, both raised in review: **Facets are not a fixed list.** The first pass carried a hardcoded table of the six stock UO facets to reconcile the spelling drift between sources. That is wrong: a shard may add facets, replace them outright, or rename them when its maps are updated, and a built-in list quietly mishandles all three. Nothing in the atlas names a facet any more. The facet set is discovered from the tree — spawn records and region definitions are the authority — and the loose spellings in Data/Locations are matched against it by key and prefix. Custom facets get identical treatment; the tests use `Sosaria` and `Underdark` precisely so a stock-facet assumption cannot creep back in. **A snapshot goes stale.** Maps change over a server's life, so a build-once artifact silently drifts from the world players actually see. The tree is now the single source of truth and the atlas is re-derived on every boot. ## What that changed - **The committed artifact is gone** — 1.41 MB of generated JSON removed, along with `scripts/buildSpawnAtlas.js` and the whole encode/decode seam it needed (`encodePoint`/`readPoint`, the tuple encoding, the omitted-defaults scheme and their round-trip tests). Nothing to keep in sync, nothing to go stale. - **NEW `src/utils/spawnAtlasSource.js`** — the only thing that touches a ServUO tree; shared by the boot path and the CLI. Parsers stay pure and fs-free. - **NEW `src/model/shardAtlas/`** — `.db.js` (the one-transaction replace) and `.model.js` (the refresh decision). - **`scripts/importSpawnAtlas.js`** is now a thin CLI over the model: `--servuo`, `--force`, `--approve`, `--reject`, `--status`. `atlas:build` is gone; `atlas:import` remains. - Path comes from the `spawn_atlas_servuo_path` admin setting, falling back to `SERVUO_PATH`. The setting wins, matching how the rest of the shard integration is admin-managed rather than env-configured. ## Two contracts on the boot path **It never blocks startup.** No path, an unreadable mount, a malformed file, a database error — every one is caught and logged, and the site comes up serving whatever atlas it already had. Verified by booting the real server with no path, a broken path, and a good path. **A facet disappearing is never applied automatically.** Losing a facet is the signature of a half-copied or mid-update tree as much as of a real map change, and boot cannot tell them apart. The refresh is staged in `shard_atlas_pending` for an admin to approve or reject, and startup continues regardless. Additions and every other change apply immediately, since none of them can destroy something an operator would miss. Only the decision is stored, not the parsed world: a few KB of source hashes and the facet diff. Approving re-parses, so what gets applied matches the tree at approval time rather than at boot. A rejection is remembered against those exact hashes, so a declined refresh does not re-prompt on every restart — changing the tree changes the hashes and asks again. Hash-gated, so the common case (restart, maps unchanged) reads and hashes the tree (~120 ms) and writes nothing. A real change costs a ~400 ms parse. The admin approve/reject UI is part of the second PR, with the rest of the routes and pages. Until then the CLI covers it. ## Verification - **564 server tests pass**, 28 new in `spawnAtlas.source.test.js` covering the custom-facet build, the spelling reconciliation, hash gating, and every branch of the refresh decision — including that `refreshOnBoot` survives a database that throws on every call. - End-to-end against the local MariaDB and the real ServUO tree: 6,455 points, 800 creatures, 23,927 point/type rows, 387 regions, 558 landmarks, 25 altars, 83.2% of points resolved to a place name. - The facet gate exercised against a real tree copy with `malas.xml` removed: staged rather than applied, atlas untouched with all 293 Malas points intact, reject then stays quiet on re-run, approve applies and drops the facet. - Booted the real server under all three source conditions; none blocked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U7CBg11prhLimL9iHSX1bP	2026-07-28 16:41:33 -05:00
wtclaude	353cce9f26	feat(atlas): parse a ServUO tree into a committed spawn atlas artifact ... Protocol 3.0 order 3 (Part C), first of two website PRs. This half is the data pipeline only — parsers, the build/import CLI, and the tables. No routes and no client, so nothing is user-visible yet; the API and pages follow in PR 2. Part C is website-only: no plugin, no sidecar, no new event kinds, no wire change. ## Parsing `src/utils/spawnAtlasParse.js` is pure and fs-free so CI covers it with no ServUO tree. Zero new dependencies — `Regions.xml` genuinely nests, so it gets a small hand-rolled subset tokenizer rather than a new XML package. The 10.5 MB of `Spawns/*.xml` never touches it: those records are flat and get a streaming regex sweep instead. The high-value transform is point-in-rect placement — highest region priority wins, ties break to the smaller rect, then a nearest-landmark fallback within 200 tiles, else "Wilderness". That is what turns "lizardman at 5411,1234" into "Despise, Felucca", and it resolves 83.2% of points (5,369 of 6,455). Three things the real data forced, none of which were in the design: - **Only 6 facets, not 13.** `Eodon.xml`, `GravewaterLake.xml` and the other named-area files carry TerMur/Trammel points, so the facet comes from each record's own `<Map>` and the artifact shards 6 ways. - **Facet names disagree across sources.** `Data/Locations/*.xml` spells them `Ter Mur` and `Tokuno Islands`; `<Map>` and `<Facet name>` say `TerMur` and `Tokuno`. Unreconciled this is silent — the landmark fallback simply never fires on those facets and every unregioned spawn there reads "Wilderness". - **Spawn type tokens carry XmlSpawner directives**: `Fairy,{RND,4,8}`, `alchemist/z/-50`, `Agralem/Name/Agralem`. Taken literally these invent creatures that do not exist AND split real ones in two, since `Fairy` and `Fairy,{RND,4,8}` slug apart. 71 of 845 entries were affected; stripping at the first `/` or `,` leaves 800 clean ones. ## Artifact `npm run atlas:build -- --servuo <path>` writes `db/data/spawnAtlas.*.json`: 6 facet shards + a compact index + a small indented `meta`. 1.41 MB committed, down from 4.40 MB by dropping `facet` per record, omitting defaulted fields, and tuple-encoding the ~24,000 type entries. `encodePoint()` and the importer's `readPoint()` are exact inverses and are round-tripped in tests. Display spelling is chosen deterministically (most common, ties to the capitalised form) because the spawn files are inconsistent about case and the name would otherwise depend on file read order — a spurious diff on every unrelated rebuild. ## Import `npm run atlas:import` needs no ServUO tree, which is the whole reason build and import are separate: the container has the artifact but not the tree. It reloads all six tables in one transaction (DELETE, not TRUNCATE, which is DDL and would implicitly commit), so a failed import leaves the previous atlas intact. ## No artwork, by design The repo ships no creature art and no extraction tooling. Sprites live in the operator's own client `.mul`/`.uop` files and are theirs, not ours to redistribute. `shard_spawn_creatures.art` is nullable and NULL on every fresh import; an operator who wants art extracts it themselves, drops it under `server/uploads/atlas/` (already gitignored) and maps slugs in a gitignored `spawnAtlas.art.json`. Text-only is the normal, fully supported state. ## Verification - **544 server tests pass**, 57 new across `spawnAtlas.parse.test.js` (the `:OBJ=` split, directive stripping, nested-region priority inheritance, half-open rects, the facet reconciliation, tokenizer edge cases) and `spawnAtlas.build.test.js` (aggregation, deterministic naming, and the encode/decode round trip). - Built and imported for real against the local MariaDB and the ServUO tree at `C:\Users\colby\Desktop\ServUO`: 6,455 points, 800 creatures, 23,927 point/type rows, 387 regions, 558 landmarks, 25 champion altars. - "Where does a lizardman spawn?" answers Shrines / Isamu-Jima / Yew across Felucca, Trammel and Tokuno. No routes changed, so the OpenAPI spec and route manifest are untouched. --- - [x] AI-assisted: written with **Claude Code** (Claude Opus 5), reviewed before opening. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U7CBg11prhLimL9iHSX1bP	2026-07-28 16:07:30 -05:00
wtclaude	61d6bfaca2	feat(shard): ingest world.ruleset and publish it at /site/rules ... Protocol 3.0 §5 (docs/link/v3.md). The shard publishes its own ruleset — expansion, which optional systems are on, skill/stat caps, account and house limits, champion scroll rules, the save/restart schedule — and the site renders it, so the rules page cannot drift from how the shard actually plays. Server - shard_ruleset: a singleton table (id = 1) holding the whole frame in `payload`, with `rev` and `expansion` hoisted. Nothing is normalized out: the frame is a flat description of config read as one page, and splitting it into columns would mean a schema change every time the shard grows a block. - shardIngest routes world.ruleset to setRuleset and deliberately does NOT log it — the shard re-emits the whole ruleset on every sidecar connect, so logging would append a duplicate row per reconnect, and server.hello already marks each of those. - uoLinkSocket backfills GET /ruleset explicitly rather than via snapshot(), which asserts an array; this covers the order where the sidecar was already up and holding the ruleset when we reconnected. - GET /public/shard/ruleset behind requireFeature('ruleset') and projected, per §3.6.1's rule that a shard read which doesn't project is a bug. `null` means the shard has never published one — a real answer, distinct from a published ruleset, and the page says so. Client - routes/public/Rules.jsx at /site/rules, live via world.ruleset (a frame is a complete ruleset, not a delta, so the newest one wins outright). Caps are rendered from tenths — 7000 is 700.0, and showing the raw number would mislead. A systems key this build doesn't know still renders, humanised, so a newer plugin can't go invisible against an older client. - Nav entry gated on the `ruleset` feature, so it hides rather than 403s. Verified end to end against the local MariaDB and a sidecar fed by a fake shard: backfill snapshot, live SSE delivery of a changed ruleset, REST reflecting the overwrite, an empty /feed (not logged), and the gate — 200 by default, 403 at audience=staff (and dropped from /features so nav hides it), 404 when disabled. Page rendered clean at all breakpoints checked, no console errors. 497 server tests pass; routes.manifest.json, routes.guards.json and the OpenAPI spec regenerated. Co-Authored-By: Claude <noreply@anthropic.com>	2026-07-28 14:35:24 -05:00
wtclaude	f3450686e0	feat(shard): admin-configurable visibility for every shard surface ... Protocol 3.0 Part A. Replaces the static PUBLIC_KINDS allowlist - which was the entire public/admin boundary - with per-feature, per-field audience control an admin owns from Admin -> Shard Visibility. Closes a live leak. BridgeJson.Actor() writes acct and webId; shapeGuild() returned the stored payload verbatim; GET /api/v1/public/shard/guilds is anonymous. Guild leaders' game account names and website user ids were readable by anyone, and the same path existed for governors. Both are now projected. The ladder is anonymous < logged_in < player < staff < admin, each rung implying the ones below. Staff satisfy `player` without a linked account (as /player/* already does); `editor` is a content role and gets no shard privilege, since mapping it to staff would silently widen what editors see. Two invariants are code, not configuration, and both reject rather than silently ignore: 1. acct/webId are admin-only always - not configurable, discarded on read as well as rejected on write. 2. A kind absent from KIND_FEATURE never reaches anyone below admin. Fail closed, so a shard emitting a new event degrades to staff-only rather than to public. Enforcement is three points over one config: requireFeature() on routes (404 disabled, 403 out-of-rung) plus field projection; per-connection filtering on SSE, where a subscriber's rung is resolved once at subscribe time and frozen so a long-open stream cannot gain privilege; and /public/shard/features so the SPA hides links it cannot follow. PUBLIC_KINDS still exists and is still exported (/feed filtering, notificationStreams) but is now derived from the kind map, so the two can no longer drift. Defaults reproduce pre-3.0 behavior exactly - a test pins the derived set against the old allowlist. Also fixes an SSE resource leak found while testing: a client dropped because its write threw was removed from the bucket but its keepalive interval was never cleared, firing forever on a dead socket. Both paths now go through one drop(). Tests: 478 server (33 new across shardVisibility + shardBroadcast), 43 client. Route manifest and OpenAPI spec regenerated. Co-Authored-By: Claude <noreply@anthropic.com>	2026-07-28 10:04:48 -05:00
wtclaude	620781b7bc	feat(auth): honor and establish trusted devices on the SSO login paths ... "Trust this device" did nothing for anyone who signs in with Google or Discord. sso.controller went straight from needsTotp(user) to staging a pending-TOTP challenge and never consulted resolveTrustedDevice, so an SSO user was asked for a code on EVERY sign-in no matter how many times they had ticked the box — and POST /auth/sso/totp accepted only `code`, so that step could not establish a trust either. The password paths (web + native) were unaffected and already worked; this closes the gap for SSO, on the website AND in the Android app. Server: - finishLogin and finishMobileLogin now run the same trusted-device check as auth.controller.login, via one shared helper: honor a trust that belongs to THIS user, stamp last_used_at, log auth.login.trusted_device. A store error falls through to the challenge — fail closed to asking for the code. - POST /auth/sso/totp gains optional trustDevice + deviceName, sets the rg_trust cookie, and mirrors the password path's { trustLimitReached, devices } response at the cap (the sign-in still completes). Recovery codes stay password-only. Android coverage, without leaking a secret into a URL: - The app opens SSO in a Custom Tab, which shares the system browser's cookie jar, so the rg_trust cookie set on that TOTP form is presented back on the next app sign-in. That alone makes native SSO skip the code. Passing the app's token into the start URL was rejected — it would put a 256-bit secret in query strings, Referer headers and access logs. - To also cover the app's NATIVE password login, ticking the box sets mobile_auth_sessions.trust_device (a boolean; never the token), and /auth/mobile/sso/exchange mints a platform:'mobile' trust and returns { trustToken }. Minting there keeps the raw token on an authenticated app→server call, out of the deep link and out of the bridge row. Best-effort: at the cap the response just omits it rather than failing a good sign-in. Client: the trust checkbox is no longer hidden on the SSO second step, on both the admin and player login screens. On the mobile bridge the deep-link redirect takes priority over the cap prompt — the sign-in succeeded and the link is single-use, so stalling there would strand the app. Tests: 8 new cases in server/test/ssoTrustedDevice.test.js (verified to fail against the pre-fix controller). Full suites green — server 445, client 43 — and routes.manifest.json is a zero-line diff: no URL moved, only +2 handlers on /auth/sso/totp in routes.guards.json for the two new validators. Swagger regenerated. Verified live against the running server and real MariaDB: the TOTP step issues rg_trust and persists the row, a subsequent SSO callback carrying it skips the code, and an invalid trust is still challenged. Co-Authored-By: Claude <noreply@anthropic.com>	2026-07-28 01:01:12 -05:00
wtclaude	60ebacff2c	feat(auth): trusted devices, recovery codes, and admin MFA management ... Add opt-in "Trust this device" so a browser/app skips the TOTP step (never the password) for 30 days, single-use bcrypt recovery codes as a 2FA-lockout fallback, and admin trusted-device/MFA-reset management — backend, web UI, OpenAPI spec, and tests. - Schema: trusted_devices (sha256 token hash, looked up by unique index) and recovery_codes (bcrypt, single-use). Both additive/idempotent. - Session service: trust-token mint/hash/resolve + cap helpers; new rg_trust httpOnly cookie (survives logout, revoked on untrust/password change/reset/ TOTP disable). JWTs stay stateless — trust is a server-side row, not a claim. - Web + mobile login accept a trusted-device token / recovery code; login/totp gains trustDevice + recoveryCode. Cap of 10/user with NO silent pruning — an over-cap trust returns 409/trustLimitReached and the client prompts to revoke. - Self-service /auth/me/trusted-devices* + recovery-codes*; admin /admin/users/:id/trusted-devices* + /mfa/reset. All actions audit-logged. - Client: "Trust this device" + recovery-code login options, one-time recovery code display, Trusted Devices + Recovery Codes account panels, a TOTP-styled revoke-to-continue cap modal, and admin per-user security controls. - OpenAPI regenerated; 33 new server tests (all suites green). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-21 23:38:48 -05:00
wtclaude	e1461d9161	fix(security): add SPA CSP, drop x-powered-by, strengthen dedupe hash ... Address SonarQube security hotspots on the website: - server/src/app.js: replace `contentSecurityPolicy: false` with a helmet CSP tuned for the built React SPA (script-src 'self'; style-src adds 'unsafe-inline' for React inline styles + the Google Fonts stylesheet; font-src gstatic; img-src allows data:/https: for uploads, embedded body images and BRAND_* assets; connect-src 'self' for REST+SSE). upgrade-insecure-requests is intentionally omitted (TLS terminates at the proxy; keeps local `npm start` over http working). The /api/docs Swagger UI route gets a scoped looser policy (inline script/style) since swagger-ui-express injects an inline bootstrap. - client/vite.config.js: disable the inline module-preload polyfill so code-split builds keep `script-src 'self'` valid (RichTextEditor is a separate chunk). - bot/src/app.js, server/src/internalApp.js: disable x-powered-by on the two internal-only listeners (the public app already strips it via helmet). - shardEvents dedupe key: SHA-1 -> SHA-256 truncated to 40 hex chars (fits the existing CHAR(40) column, no migration; it is a content fingerprint, not a security value). schema.sql comment updated to match. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NgyHnrNa8WwG3doxvxjuCr	2026-07-20 23:02:11 -05:00
wtclaude	bcc96e7cfb	feat(mobile-sso): serve assetlinks.json + App Links redirect allowlist ... Add the server side of Android App Links (M9 follow-up, docs/android/APP_LINKS.md): - GET /.well-known/assetlinks.json at the web root, gated by the new admin setting `mobile_app_links_enabled` (default off -> 404; on-but-no-fingerprint -> 404). Emits the Digital Asset Links statement for the fixed published package (MOBILE_APP_PACKAGE) + MOBILE_APP_CERT_SHA256 fingerprint(s). - mobileSso `/start` additionally accepts this shard's own self-origin https://<host>/mobile/callback when App Links are enabled — one additive exact-match entry, derived from APP_BASE_URL/request origin, never client input; the custom-scheme allowlist is never narrowed. The settings lookup is short-circuited for non-https redirects so custom-scheme rejections stay fast. - settings.isMobileAppLinksEnabled() (fail-closed) + getPublic().mobileAppLinks; admin updateSettings validates the boolean; seed default off. Tests: test/appLinks.test.js (route gating + allowlist). Full suite 284 pass. Swagger unchanged (web-root verification file is #swagger.ignore'd). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NgyHnrNa8WwG3doxvxjuCr	2026-07-20 18:37:17 -05:00
wtclaude	e3dd5358b6	feat(auth): Active Devices — view/revoke mobile sessions ... Adds the self-service device-session surface the mobile-SSO spec requires, on top of the existing mobile_refresh_tokens store. - Schema: device_name + last_used_at columns on mobile_refresh_tokens (nullable, additive via the ALTER section; seeded to now on insert). With single-use rotation each login/refresh inserts a fresh row, so the active row's timestamp is the session's last activity, and the label is carried forward on refresh. - Model: listActiveForUser (one row per live device, no token hash) + revokeByIdForUser (ownership-scoped, idempotent). - GET /auth/me/sessions + DELETE /auth/me/sessions/:id (role-agnostic, behind requireAuth). Named distinctly from /auth/me/devices (push endpoints). - device_name is an optional field on /auth/mobile/login and /auth/mobile/sso/exchange so the app can label a device. - Client: an "Active Devices" panel on the player account page (list + sign a device out), plus the PlayerLogin change to honor the mobile SSO bridge's { redirect } deep link on a 2FA completion. - Swagger DeviceSession schema + regenerated spec; 3 controller tests. Full server suite green (274); client builds. Co-Authored-By: Claude <noreply@anthropic.com>	2026-07-20 17:01:47 -05:00
wtclaude	61f4591a6b	feat(auth): native SSO authorization bridge for the Android app ... Add a Mobile SSO Authorization Bridge so the native app can "Sign in with Google/Discord" without shipping any OAuth secret. It EXTENDS the existing /auth/sso/* redirect flow (same PKCE-vs-IdP, link-only + opt-in provisioning, TOTP gate) and terminates in the existing mobile bearer tokens — not a parallel auth path. - Schema: mobile_auth_sessions + mobile_auth_codes (short-lived, self-pruning; authorization code stored hash-only, PKCE challenge is a hash by construction). - GET /auth/mobile/sso/start: validate provider enabled + redirect_uri by EXACT allowlist match (never prefix), seed a bridge session, reuse the SSO redirect tagged mode:'mobile' (new redirectToIdp helper extracted from beginFlow). - SSO callback + finishSsoTotp gain a mode:'mobile' branch: mint a single-use, hashed, PKCE-bound code and redirect to the fixed app callback (code + echoed state, never a token) instead of setting a cookie. 2FA keeps full parity via the existing web TOTP form (now carrying the bridge session). - POST /auth/mobile/sso/exchange: verify Layer-B PKCE (before burning the code), single-use consume, then issue the SAME pair as /auth/mobile/login. - Discovery reuses GET /auth/providers; refresh/logout reuse /auth/mobile/*. - Rate limits: /start per-IP+provider, /exchange per-IP. Boot-time + opportunistic prune of both tables (no cron, mirrors revoked_sessions). - Redirect allowlist is MOBILE_AUTH_REDIRECT_URIS (default the one fixed runicgateway://auth/callback); App Link URIs can be appended per shard later. - Swagger regenerated; 39 tests (model single-use/gating + full controller matrix: bad/expired/reused code, PKCE mismatch, disabled provider, redirect allowlist, TOTP-through-bridge). Full suite green (271). Refs docs/website/BACKEND_DESIGN.md, docs/android/PLAN.md §9 (M9). Co-Authored-By: Claude <noreply@anthropic.com>	2026-07-20 16:55:06 -05:00
wtclaude	416761f8f7	feat(push): M7 backend — opt-in push notifications via self-hosted ntfy ... Additive, v1-only backend contract for the Android app's opt-in push (Part 1 of M7; docs/android/PLAN.md §11). The app is a pure consumer — this lands the endpoints, fan-out, and relay it needs. - Schema: push_devices (per-device endpoint) + notification_subscriptions (per-user opted-in streams), FK→users ON DELETE CASCADE. - Stream catalog + event→stream mapping (config/notificationStreams.js): public streams (news.post, server.status, idoc.warning, champ.start, governor.election) drawn ONLY from the SSE PUBLIC_KINDS allowlist; personal owner-keyed streams (vendor.sale, house.idoc, account.login). Full-state upserts (champ/city) fire only on a real transition via an injectable tracker. - Fan-out (utils/pushDispatch.js): content-free tickles ({ stream, ref }) POSTed to each subscribed device; never throws. Two producers — shardIngest.ingest (beside the SSE broadcast) and the create/publish-post path (news.post). Personal events resolve to the owner via shardLinks. SSRF guard: endpoints must be HTTPS, non-private, and on the NTFY_BASE_URL/NTFY_ALLOWED_ORIGINS allow-set — enforced at registration and every publish. - Routes under the role-agnostic self surface (never /admin): POST|GET /auth/me/devices, DELETE /auth/me/devices/:id, GET /auth/me/notifications/streams, GET|PUT /auth/me/notifications/subscriptions. Swagger regenerated (4 paths, PushDevice/NotificationStreams/etc. schemas). - ntfy service in docker-compose.yml: pinned image, declarative ./ntfy/server.yml, no published host port, anonymous unguessable topics (no accounts) — zero interactive setup. No publish token required (content-free design); optional NTFY_PUBLISH_TOKEN honored. - Tests: pushDispatch (mapping, PUBLIC_KINDS gate, owner-keying, SSRF guard, content-free payload) + notifications route auth gate. Full suite green (247). Co-Authored-By: Claude <noreply@anthropic.com>	2026-07-20 05:13:48 -05:00
Colby Whitlock	250cb1e2d3	Merge branch 'main' into feat/password-reset	2026-07-19 09:04:43 +00:00
wtclaude	10aed49bb6	feat(auth): self-service password reset (backend + web) ... Add a full password-reset flow — the prerequisite for the Android app (docs/android/PLAN.md §8.2), which hands off to the website for reset rather than shipping a native screen. Backend: - password_resets table: stores only the sha256 hash of an opaque 32-byte token (mirrors user_invites / mobile_refresh_tokens), single-use, ~1h TTL. - model/passwordResets + users.getActiveByEmail (email is non-unique, so a request can match several accounts, each emailed its own link). - mailer.sendPasswordReset (fails soft when email is unconfigured). - Endpoints: POST /auth/password/forgot (always a generic 200 — no account enumeration), GET|POST /auth/password/reset/:token. Confirming rotates the hash and revokes every session (web cutoff + mobile refresh tokens); it does not auto-login, so a 2FA account still passes TOTP next sign-in. Also serves SSO-only accounts (null hash) as their set-initial-password path. - Dedicated request/confirm rate limiters. Swagger regenerated. Web: - ForgotPassword + ResetPassword pages, routes /account/forgot and /account/reset/:token, and a "Forgot your password?" link on the login page. Tests: test/passwordResets.test.js (5). All server tests pass; client builds; end-to-end smoketest against MariaDB passes (no-enumeration, single-use, hash rotation, session revoke, login with the new password). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NgyHnrNa8WwG3doxvxjuCr	2026-07-19 03:57:13 -05:00
Claude	028ba8c5e4	feat(moderation): appeals (6c) + Discord reversal on approve (6d) ... Players whose linked Discord identity was banned or muted can now submit an appeal from the portal and track it; staff get a queue in the admin moderation section to claim and resolve (approve/deny) appeals. Approving a ban/mute appeal best-effort asks the Discord bot to reverse the action (unban / clear timeout) via the internal API and posts a mod-log embed; a down bot never fails the resolution (reversal_status is recorded). - Schema: new server-owned `appeals` table (no cross-owner FK to mod_actions; existence validated in app code). - Server: model/appeals/* + player appeals controller (submit/mine/ eligible/withdraw) and admin queue handlers (list/claim/resolve/ per-user) under the existing admin+moderator gate; one-active-appeal enforced app-side; eligibility keyed on the caller's linked Discord id. - 6d: bot POST /internal/mod-reverse (+ modLog.postReversal) and server botInternalClient.reverseModAction, wired into resolve(). - Client: admin Appeals queue + resolve modal, ModerationUser appeals tab, player Appeals page (submit/withdraw), nav + routes + api methods. - Docs: swagger annotations + component schemas, regenerated output. - Tests: appeals controller + pure suites (server npm test 224 green). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XmHdsbnLzDMAVQkAoTQSBe	2026-07-18 22:01:06 -05:00
Claude	7a08546da6	feat(brand): BRAND_* env scheme — instance branding without a rebuild ... Replace baked-in UOM/MysticMoon/UOMysticmoon branding with a BRAND_* env scheme so one prebuilt image runs as any shard; UOMysticmoon becomes the first tenant that sets these vars rather than a special case in the code. Architecture (chosen because the app ships as a prebuilt image): - server/src/config/brand.js + bot/src/brand.js read BRAND_* once at boot, with Runic Gateway defaults. - Text/colors reach the SPA at RUNTIME through the existing public settings API (settings.model.getPublic -> SiteContext), so no client rebuild. The admin-editable site title + contact email still override BRAND_NAME/email. - SiteContext applies BRAND_ACCENT_COLOR to the --accent CSS var at runtime. - Express templates the built index.html <title>/description/OG/favicon at serve time from BRAND_* (renderIndexHtml in app.js). - Server-side consumers read brand directly: emails, TOTP issuer, API docs, boot logs, HTML error page. Bot uses it for embed color + logs. Assets: logo/hero/favicon delivered from a ./brand:/app/brand bind-mount (BRAND_LOGO/HERO/FAVICON), with neutral defaults baked in; hero falls back to a built-in image when unset. Scope: also genericized package.json names (uomysticmoon-* -> runic-gateway-*) and the DB_NAME/DB_USER/COOKIE_NAME code defaults (runic_gateway/runic/ rg_token). Production keeps its real values by pinning them in .env — see .env.uomysticmoon.example, which reproduces the exact UOMysticmoon identity (proof the substitution works). Changing a deployed COOKIE_NAME invalidates existing sessions, so UOMysticmoon pins uomm_token. Verified: 193 server tests pass, client builds, app.js loads + templates the built index.html, brand transform injects title/description/OG/favicon.	2026-07-18 02:20:04 -05:00
Claude	a165c90c62	fix(schema): remove semicolons from shard_governor_terms inline comments ... ensureSchema() splits schema.sql on ';' and is not comment-aware, so the inline comments "epoch ms; NULL = current" and "not in the feed; reserved" shattered the CREATE TABLE into invalid fragments (ER_PARSE_ERROR on a fresh boot). Reworded both to drop the semicolons. Caught during live-stack bring-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-17 16:25:14 -05:00
Claude	91c206bf76	feat(provisioning): game-account signup, admin email invites, unlink (2.0) ... Phase 5: the account-provisioning backend — link-only stays, plus hybrid self-signup, an admin email-invite tool, and site-side unlink. - uoLinkClient.createAccount / unlinkAccount (v2). Password is forwarded to the shard (hashed there) and never stored/logged; the end-user browser IP is passed for the shard's per-IP cap; actor is stamped server-side. - Hybrid signup: POST /player/shard/account provisions a game account (its own username + password) for the signed-in user and mirrors the link locally. Gated by the new game_account_signup setting AND the shard's own mode (mapped 403/409/ 429/400/503). Serves both self-serve signup and the invite-accept game step. - Email invites: user_invites table (sha256 token hash, single-use, expiring); invites model + admin CRUD (POST/GET/DELETE /admin/invites, admin-only) + mailer.sendInvite (falls back to returning the accept link if email is off); public token-gated accept (GET /auth/invite/:token, POST .../accept) creates the user at the invite's preset role and logs them in, bypassing the registration gate. Accept is race-safe (atomic single-use; rolls back the user if it loses). - Admin unlink: DELETE /admin/users/:id/shard/link/:account (admin-only) + local mirror drop; account.unlinked ingest reconciles the mirror when a player runs [unlink in game. account.audit / account.unlinked are logged (admin channel only — never on the public SSE allowlist). Tests: invites model (hashing, single-use, expiry, revoke) + account.* ingest reconcile/visibility. Full suite 193/193; swagger regenerated. Refs .plans/protocol2-integration.md (Phase 5). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-17 15:50:49 -05:00
Claude	080478c4a1	feat(shard): ingest Protocol 2.0 boards — guilds, governors, presence, houses ... Phase 1 of the Protocol 2.0/2.1 integration: the read/ingest backend for the four new uo-link boards, following the established champs/pages pattern (ingest → our MariaDB + snapshot-on-reconnect + public SSE + token-free public endpoint). - Schema: shard_guilds, shard_governors, shard_governor_terms, shard_presence; extend shard_houses with the house.update registry columns (owner_name, co_owners, friends, price, decay, in_registry) so the decay-transition and registry feeds share one house row without clobbering each other. - Ingest: route guild.update/remove, city.update, presence.online, house.update/remove; log guild.join (real-time joins feed); region.enter is broadcast-only. All new public kinds added to the SSE allowlist. - Governor term history captured from day one: on every observed governor CHANGE the open term is closed and a new one opened, idempotent so backfill/duplicate city.update never spawn spurious terms. votes stays NULL (the feed carries only candidate count, not tallies) — we never fabricate vote numbers. - Client + backfill: getGuilds/getGovernors/getHouses/getPresence; snapshot each board on every WS (re)connect, independently guarded so an empty/failed board (e.g. no City Loyalty) never wipes another. - Public endpoints: /shard/{guilds,governors,governors/:city/history,presence,houses}. - Tests: ingest routing for all new kinds + governor term-capture idempotency (15 new; full suite 179/179). Swagger regenerated. Refs .plans/protocol2-integration.md (Phase 1). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-17 12:28:54 -05:00
Claude	c31553aeb6	feat(shard): admin write plane, help-page queue, and public champion board ... Wire up the three uo-link sidecar surfaces that weren't integrated yet. Champion spawns - Ingest champ.update/champ.remove into a new shard_champs table (served from our own store, like online/houses); public /site/champs board with a nav link, live via the existing SSE feed (champ.* added to the public allowlist). Staff write plane (admin + moderator) - kick / ban / unban / broadcast via /admin/shard/*; actor is stamped server-side from the session, never the browser. Sidecar status codes mapped (403 disabled/ protected, 404 unknown, 503/504 transient). admin.audit events are logged and surfaced at /admin/shard/audit. - New admin "In-Game Ops" view (/admin/shard-ops), plus per-account Kick/Ban/Unban on the user-detail and character views (ShardAccountActions, self-gated to staff). Help-page (support) queue - Ingest page.new/updated/closed into a new shard_pages table; respond/close via /admin/shard/pages/*. Champ board and page queue are snapshotted from the sidecar's /champs and /pages on every WS (re)connect (guarded so a failed call never wipes local state). Verified live end-to-end against MariaDB + the Rust sidecar + ServUO; unit tests cover ingest routing (shardIngest.champsPages.test.js). Swagger regenerated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0114TpmrNW4wNXsHq5CR72jQ	2026-07-14 13:16:10 -05:00
Claude	986a8d5d86	News post → town crier + Discord announcement pipeline ... Replace the fire-and-forget Discord-only announce on publish with a retry-safe, two-leg pipeline. When a post transitions into published-news (false→true publish while in news, or category→news while published), an announce_jobs row is enqueued with two INDEPENDENT delivery legs: • town crier — sidecar POST /towncrier via uoLinkClient (stable id `post-<id>` so a retry replaces rather than duplicates) • discord — bot POST /internal/announce via botInternalClient (single source of truth for the #news channel stays in the bot) An in-process poller (utils/announceWorker) sweeps the table every ANNOUNCE_POLL_MS and dispatches each due leg with its own exponential backoff (30s→2h, 6 attempts). A leg is retried on transient failures (503/504/network) and failed fast on data/config errors (400 over-cap, 401/409). Publishing never blocks on the sidecar or Discord — enqueue is local DB only. Parent `status` is a done/partial/failed rollup of the two legs; posts.announced_at is stamped once both deliver. Admin visibility: GET /admin/posts/:id/announce + a per-leg Retry (POST .../announce/retry) surfaced in the PostEditor for news posts. Pure decisions (text build/caps, classification, backoff, rollup) live in announceJobs.logic and are unit-tested (server/test/announceJobs.test.js, 10 tests). The old manual /admin/uo-link/towncrier form is untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0114TpmrNW4wNXsHq5CR72jQ	2026-07-11 16:25:25 -05:00
Claude	064f02c4b6	Add player account linking + roster/vendor reads (phase 3) ... Ties an in-game account to a website user and gates reads on ownership. - schema: shard_account_links (account PK → user_id, char_name, linked_at; FK users ON DELETE CASCADE) — the site-side mirror of the sidecar's authoritative link. - model/shardLinks: upsert/list/ownership-check/getByAccount/unlink. - player/shard.controller.js: - POST /player/shard/link — confirm a one-time [link code via uoLinkClient.confirmLink(code, req.user.id); on link.ok mirror the link and activity.log it; bad/expired codes → 400, shard down → 503. - GET /player/shard/accounts — the caller's linked accounts. - GET /player/shard/roster/:account and /vendors/:account — live round-trips, ownership-checked against the mirror (403 otherwise), 503 on shard restart. - player.routes.js: mounted under the existing requireRole('player') gate with express-validator guards + #swagger annotations; new "Player · Shard" tag and ShardLinkRequest/ShardLinkResult/ShardLink schemas; spec regenerated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qPmpmVH1xGCiZoz9m9vW3	2026-07-11 02:13:46 -05:00
Claude	9d9f5aac28	Add uo-link WS ingest, storage tables and SSE broadcaster (phase 1) ... The site now ingests the sidecar's live WebSocket feed and persists it to its own MariaDB, and re-broadcasts curated events to browsers over SSE. - schema: shard_events (append-only notable-kind log, sha1 dedupe_key + INSERT IGNORE for idempotent reconnect backfill), shard_online (current players, upsert/refresh/remove), shard_economy (gold-supply series), shard_houses (per-house decay stage + derived is_idoc). - model/shardEvents + model/shardState: the .db.js/.model.js split; writes take camelCase event data, reads are shaped; online upsert uses COALESCE so a partial char.vitals refresh never blanks login fields. - utils/shardIngest: single dispatcher routing each kind to state writes and/or the event log, then the broadcaster. High-frequency kinds (char.vitals, economy.supply) update state only. A changed server.hello bootId clears the stale online roster. Deps are injected for unit testing. - utils/uoLinkSocket: the server's first outbound WS client (ws dep). Verifies the ws.hello protocol, backfills via /history + /economy on every (re)connect (dedupe handles overlap), reconnects with capped backoff, and mirrors connection state into uo_link_config. Self-guards: only connects when the integration is enabled with a token. - utils/shardBroadcast: SSE fan-out with public (safe kinds only) and admin (all) channels, keepalive pings, per-client cleanup. - server.js: start the ingest socket on boot (no-op until configured) and stop it + close SSE streams on graceful shutdown. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qPmpmVH1xGCiZoz9m9vW3	2026-07-11 02:08:56 -05:00
Claude	ab647756f0	Add uo-link sidecar foundation: config store + REST client (phase 0) ... Introduces the DB-backed connection config for the uo-link sidecar (the HTTP + WebSocket bridge to the ServUO shard) and a never-throw REST client, mirroring the existing Discord-bot integration: - uo_link_config singleton table (base/ws URL, AES-256-GCM-encrypted shared token, protocol pin, enabled, and last-known status/plugin_connected/ last_event_at/boot_id mirrors for the admin panel). - model/uoLinkConfig: getSafe (never returns the token — only hasToken), getWithToken (server-side decrypt), save (blank token = unchanged), recordStatus (mirror the sidecar's reported state). - utils/uoLinkClient: never-throw fetch client returning {ok,data,status, error}; Bearer token + X-UOLink-Version on every call; brief config cache; helpers for health/char/roster/vendors/history/economy/link/towncrier. - .env.example: UOLINK_BASE_URL/WS_URL/PROTOCOL defaults (token stays admin-managed in the DB, never an env var). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011qPmpmVH1xGCiZoz9m9vW3	2026-07-11 02:02:40 -05:00
Claude	fcef08e9b6	Add pages table + block registry scaffold (page builder step 2) ... New `pages` table: slug/title/blocks(JSON-as-text)/status/protected, author FK, grouped SEO metadata + layout/nav settings columns (added up front per spec — cheap now, painful to retrofit), published_at mirroring posts. Block registry scaffold, server and client, defining the pattern without any block types yet (Wave 1 lands in step 3): - server/src/blocks: registry (register/get/list, reserved envelope keys, container metadata) + validateBlocks (authoritative save-time gate: envelope, registered-type, per-block schema, one-level nesting cap) + index entrypoint that will register Wave 1 defs. - client/src/blocks: mirror registry carrying renderer/editor/palette + makeBlockId, plus index entrypoint. Verified: schema applies idempotently against the dev DB (pages table + indexes present); validator exercised for empty/non-array/unknown-type/ bad-envelope/duplicate-id/nested-container cases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-09 20:26:14 -05:00
Claude	f8652c2399	Modernize email: Gmail OAuth2 sending, configured under Settings ... Retire env-var SMTP basic-auth and send the contact form through Gmail over OAuth2 (SMTP XOAUTH2), configured in Admin -> Settings -> Email via an in-app "Connect Gmail" consent flow. Reuses the existing google SSO OAuth client; the captured refresh token is stored AES-GCM-encrypted (write-only over the API, never returned), mirroring the auth-provider and Discord-bot secret patterns. - schema: new email_config singleton table (mirrors bot_config) - model: emailConfig.{db,model} with encrypted refresh token + getSafe/getWithSecret - mailer: nodemailer OAuth2 transport (client id/secret from the google provider row), contact recipient = contact_email setting, mailto: fallback preserved, plus sendTest() - routes/controller: /admin/email config, connect start+callback (ssoState CSRF + PKCE), test, disconnect - client: EmailDelivery section on the Settings page + api methods; Settings copy now spells out that contact_email is the delivery recipient - docs/env: drop SMTP_*/CONTACT_TO from env examples; update README/BACKEND_DESIGN - tests: emailConfig.model + mailer suites (8 new; full suite 142 pass) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XKeCQEJZr1AFJN4Bgcmvh3	2026-07-07 22:29:27 -05:00
Claude	5daf260db9	Player accounts frontend + Swagger + schema comment fix ... - Player portal: RequirePlayer guard, /account routes (login, register, settings) with shared PlayerShell; register reads /public/settings derived flags; AuthContext.register; api.register + api.player.* namespace. - Admin UI: player role + status/email + reset-password hint in UserEditor, status column + badge-player in UsersAdmin, player_registration select in SettingsAdmin; 'disabled' SSO error copy. - Swagger: Player tag + RegisterRequest/ChangeUsername/ChangePassword/ PlayerAccount/OkFlag schemas; regenerated swagger-output.json. - Fix: remove a semicolon from a schema.sql inline comment that broke the statement splitter in ensureSchema. Verified against the live dev DB: schema migrations apply (player enum, nullable password_hash, email/status/last_login_ip, seeded setting); 21-check controller smoke (register gating, dup/reserved, null-hash rules, self change username/password with session re-issue surviving the cutoff, SSO-only initial password, banned-login refusal); case-insensitive uniqueness; public settings expose only derived registration flags. Client builds; 133 server tests green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019rao86n5cXpwAyjdBFEshV	2026-07-06 01:49:12 -05:00