From 76da8ee88d47278088e87e92e32011251bfab94c Mon Sep 17 00:00:00 2001 From: wtclaude Date: Wed, 19 Aug 2026 18:04:41 -0500 Subject: [PATCH] docs: hygiene sweep - the four places the docs describe a site that no longer exists Four documents still describe the pre-module-system website. None of them misconfigures anything, which is why they survived; all four mislead a reader trying to understand how the system is actually put together. ARCHITECTURE.md placed shardIngest.js and uoLinkClient.js inside the website backend. Both live in module-uo/server/utils/ - verified, they are not in website/server/src at all. The document claimed to be "the canonical copy of the diagram; the same diagram is embedded in the website's README", and the two had silently diverged: the live README's diagram has the module subgraph, the loader, and the game behind the module, and this one did not. The diagram is now the live one verbatim, the surrounding prose attributes the shard integration to the module, and the intro no longer frames core as game-aware. The SSE bullet gains the distinction the live README makes: the module declares which kinds are public, core enforces the split. website-README.md had drifted from the live README by 28 lines, all of them the "Three ways in, and none of them is a build" section - the admin panel, the MODULES environment variable, and by hand - which is now the primary module-install story. Re-synced verbatim, since a faithful snapshot is the file's whole purpose. The diff was purely additive; the snapshot contained nothing the live README had dropped. README.md's index was missing thirteen documents, not the four the audit had found: TEAMS.md, ARCHITECTURE.md, TRUSTED_DEVICES_MFA.md and MODERATION_APPEALS.md, and also link/v4.md - the current protocol - android/THEMING_AND_NAV.md, ci/SONARQUBE.md, installer/PROJECT_TREE.md, modules/kit-acceptance.md, modules/uo/API.md, modules/uo/SCHEMA.md, website/test-plan.md and the two API_V2 documents. The layout block already advertised a ci/ directory that had no section. Every markdown file outside the issue templates is now indexed, and every link resolves. API_V2_SKELETON.md is listed as superseded, which is what its own header says. BACKEND_DESIGN.md was titled "UOMysticmoon Website - Backend Design" though it is core's contract and core is game-agnostic. Retitled, with a note that nothing in it is instance-specific. Its hardcoded public contact address is now described as what it is - seeded from BRAND_CONTACT_EMAIL into the contact_email setting, with UOMysticmoon as the example instance. Co-Authored-By: Claude --- README.md | 17 +++++++++ website/ARCHITECTURE.md | 77 +++++++++++++++++++++------------------ website/BACKEND_DESIGN.md | 12 ++++-- website/website-README.md | 28 ++++++++++++++ 4 files changed, 96 insertions(+), 38 deletions(-) diff --git a/README.md b/README.md index 2bb803a..9f95f88 100644 --- a/README.md +++ b/README.md @@ -23,17 +23,24 @@ sidecar as a service, and hands you the values the website needs. | Doc | What it covers | |---|---| | [BACKEND_DESIGN.md](website/BACKEND_DESIGN.md) | API contract, DB schema, security model | +| [ARCHITECTURE.md](website/ARCHITECTURE.md) | The system diagram — how core, an installed module, the sidecar and the clients fit together | +| [TEAMS.md](website/TEAMS.md) | Teams as a platform primitive: roster, forums, notifications, Discord slash commands and voice — design of record | | [HERO_EDITOR.md](website/HERO_EDITOR.md) | Hero canvas editor feature spec | | [THEMING_AND_NAV.md](website/THEMING_AND_NAV.md) | Admin-configurable theme, brand assets and navigation — build contract | | [MODULE_SYSTEM.md](website/MODULE_SYSTEM.md) | Making the site game-agnostic: game logic becomes an installable module — design of record | | [MODULE_API.md](website/MODULE_API.md) | The module ↔ core contract: `ctx`, the `register*` calls, the client registry and the loader's obligations | | [WIKI_UPGRADE.md](website/WIKI_UPGRADE.md) | Wiki subsystem upgrade notes | | [SHARD_VISIBILITY.md](website/SHARD_VISIBILITY.md) | Who sees which shard data — the admin-configurable audience framework | +| [TRUSTED_DEVICES_MFA.md](website/TRUSTED_DEVICES_MFA.md) | TOTP two-factor, trusted devices and recovery codes | +| [MODERATION_APPEALS.md](website/MODERATION_APPEALS.md) | Moderation actions, content reports and the appeals flow | | [SPAWN_ATLAS.md](website/SPAWN_ATLAS.md) | The bestiary / spawn atlas: what the shard contains, parsed from its own ServUO tree | | [CLILOCS.md](website/CLILOCS.md) | UO's id → name table: converting one from your client so items have names | | [UOFIDDLER.md](website/UOFIDDLER.md) | **Operator runbook** — step-by-step extraction from your own UO client (cliloc table, creature art) | | [MARKETPLACE.md](website/MARKETPLACE.md) | The player-vendor index: how it is gathered, what it costs, how to tune it | | [website-README.md](website/website-README.md) | Snapshot of the website repo's README (setup/run reference) | +| [test-plan.md](website/test-plan.md) | The website's test strategy and harness | +| [API_V2_PLAN.md](website/API_V2_PLAN.md) | Router domain split + CSP hardening. The split is complete; only the CSP enforce step remains | +| [API_V2_SKELETON.md](website/API_V2_SKELETON.md) | **Superseded** — the `/api/v2` scaffold that was never built. Kept as the record of why the split was done in place instead | | [PROJECT_TREE.md](website/PROJECT_TREE.md) | Auto-generated snapshot of the repo's tracked file layout | ### `modules/` @@ -45,6 +52,8 @@ particular game; a module is what makes it a site *for* one. | Doc | What it covers | |---|---| | [uo/](modules/uo/README.md) | **module-uo** — the Ultima Online module: what it serves, what it owns, and what an operator needs | +| [uo/API.md](modules/uo/API.md) · [uo/SCHEMA.md](modules/uo/SCHEMA.md) | module-uo's own route surface and the tables it owns | +| [kit-acceptance.md](modules/kit-acceptance.md) | The Integration Kit acceptance run — building a module by following the kit alone, and what it found | | [rust-dryrun.md](modules/rust-dryrun.md) | A written, deliberately unimplemented `module-rust` — the test that the module contract generalises past the game it was extracted from | ### `link/` @@ -53,6 +62,7 @@ particular game; a module is what makes it a site *for* one. | [INTEGRATION.md](link/INTEGRATION.md) | How the website integrates with the uo-link sidecar | | [PROTOCOL_2.md](link/PROTOCOL_2.md) | Protocol 2.0 / 2.1 design | | [v3.md](link/v3.md) | Protocol 3.0 design — shard content/standings streams + the visibility framework | +| [v4.md](link/v4.md) | Protocol 4.0 — guild membership on the wire (`guild.roster`, `guild.leave`). **The current protocol** | | [ADMIN_CONTROLS.md](link/ADMIN_CONTROLS.md) | Staff write-plane (kick/ban/broadcast, page queue) | | [SHARD_PREREQS.md](link/SHARD_PREREQS.md) | Shard-side prerequisites for the bridge | | [PLAN.md](link/PLAN.md) | uo-link build plan | @@ -67,6 +77,7 @@ particular game; a module is what makes it a site *for* one. | [COVERAGE_PLAN.md](android/COVERAGE_PLAN.md) | Test-coverage rollout plan | | [APP_LINKS.md](android/APP_LINKS.md) | Android App Links / deep-link setup | | [theme-plan.md](android/theme-plan.md) | Theming plan | +| [THEMING_AND_NAV.md](android/THEMING_AND_NAV.md) | The app's half of admin-configurable theming and navigation — build contract | | [TRUSTED_DEVICES_APP_HANDOFF.md](android/TRUSTED_DEVICES_APP_HANDOFF.md) | Trusted-devices app handoff notes | | [PROJECT_TREE.md](android/PROJECT_TREE.md) | Auto-generated snapshot of the repo's tracked file layout | @@ -75,6 +86,12 @@ particular game; a module is what makes it a site *for* one. |---|---| | [INSTALL.md](installer/INSTALL.md) | **Start here to set up a shard** — the installer deploys the plugin overlay and the uo-link sidecar, registers the service, and connects it to the website. Appendix A is the same thing by hand, still supported | | [PLAN.md](installer/PLAN.md) | Installer design of record — phases, locked decisions, the bundle/compat-matrix model | +| [PROJECT_TREE.md](installer/PROJECT_TREE.md) | Auto-generated snapshot of the repo's tracked file layout | + +### `ci/` +| Doc | What it covers | +|---|---| +| [SONARQUBE.md](ci/SONARQUBE.md) | The SonarQube setup: project keys, how analysis runs, and how to read a report | ## Provenance diff --git a/website/ARCHITECTURE.md b/website/ARCHITECTURE.md index dfd43ea..358d758 100644 --- a/website/ARCHITECTURE.md +++ b/website/ARCHITECTURE.md @@ -1,9 +1,13 @@ # Website — Architecture How the pieces of `RunicGateway/website` fit together. The React SPA and the native mobile app -talk to one Express backend (`router → controller → model → db`), which persists to MariaDB and -bridges to the live game world **only** through the **uo-link** sidecar. The ServUO shard itself is -never internet-facing — it dials out to the sidecar over loopback, and only the sidecar is exposed. +talk to one Express backend (`router → controller → model → db`), which persists to MariaDB. + +**Core is game-agnostic.** Since the module system shipped on 2026-08-12, everything about a +particular game — its routes, tables, pages and its connection to a game server — lives in an +installed module, not here. For Ultima Online that is [`module-uo`](../modules/uo/README.md), which +bridges to the live world **only** through the **uo-link** sidecar; the ServUO shard itself is never +internet-facing, dialling out to the sidecar over loopback, and only the sidecar is exposed. This is the canonical copy of the diagram; the same diagram is embedded in the website's [`README.md`](https://gitea.whitlocktech.com/RunicGateway/website/src/branch/main/README.md#architecture). @@ -28,32 +32,29 @@ flowchart TB subgraph backend["server/ — Express backend"] direction TB mw["Middleware
helmet · siteMode · noindex
rateLimit · loginProtection · botScore · validate"] - router["Router /api/v1
auth (web · mobile · sso) · public · admin"] + router["Router /api/v1
auth (web · mobile · sso) · public · admin · player"] ctrl["Controllers"] auth["Session layer (auth/)
sessionService · JWT/cookie · bearer · SSO+PKCE"] model["Models (.model + .db)
raw parameterized SQL — no ORM"] sse["SSE fan-out
public stream (allowlist) · admin stream (sensitive)"] - - subgraph shardutil["Shard integration (utils/)"] - ingest["shardIngest.js
WS ingest dispatcher"] - restcli["uoLinkClient.js
REST client (never throws)"] - end - + loader["modules/loader.js
scans the volume · mounts · registries · lifecycle"] secret["secretBox.js
AES-256-GCM secrets at rest"] end bot["bot/
Discord bot"] end - db[("MariaDB
users · posts · wiki · settings · activity
mobileSessions · authProviders · userIdentities
uoLinkConfig · shard_online/economy/houses/events")] + db[("MariaDB
users · posts · wiki · settings · activity
mobileSessions · authProviders · userIdentities
installed_modules · <module>_*")] - %% ---------- Shard side ---------- - subgraph shardside["Game shard (never internet-facing)"] + %% ---------- Module side ---------- + subgraph modside["modules/<id>/  — installed, not built (e.g. Module-uo)"] direction TB - sidecar["uo-link sidecar
(Rust) — the only bridge exposed"] - servuo["ServUO shard
(C# plugin)"] + modsrv["server/ — routers, models, schema fragment
reaches core only through ctx"] + modcli["client/dist/entry.js — prebuilt ESM chunk
React shared via window.__rg"] end + game["The game
whatever the module talks to
(for Module-uo: a ServUO shard,
via the uo-link sidecar)"] + %% ---------- Edges ---------- browser <-->|"same-origin JSON + SSE (cookie)"| mw mobile -->|"REST (bearer access/refresh)"| mw @@ -63,29 +64,27 @@ flowchart TB mw --> router --> ctrl ctrl --> auth ctrl --> model - ctrl --> restcli ctrl --> sse auth --> model model <--> db auth -. reads/writes secrets .-> secret - restcli -. reads config/token .-> secret - ingest --> model - ingest --> sse sse -->|"live events"| browser bot -->|"messages"| discord bot <--> db - restcli -->|"REST: /char /roster /economy /history · /link/confirm · /towncrier"| sidecar - sidecar -->|"WebSocket live event feed (bearer + X-UOLink-Version)"| ingest - servuo -->|"loopback TCP 127.0.0.1:7788
newline-delimited JSON (shard dials out)"| sidecar + loader -->|"mounts under /api/v1/<tier>/<prefix>"| router + loader -->|"require() + register(ctx, api)"| modsrv + modsrv -->|"ctx.db · ctx.push · ctx.activity …"| model + modsrv <--> game + browser -->|"<script type=module> injected by htmlShell"| modcli %% ---------- Styling ---------- classDef ext fill:#2d2233,stroke:#7a5c94,color:#e8dff0; classDef store fill:#1f2d2a,stroke:#4c8c7d,color:#dff0ea; - classDef bridge fill:#2d2620,stroke:#94764c,color:#f0e6d8; - class idp,discord ext; + classDef mod fill:#2d2620,stroke:#94764c,color:#f0e6d8; + class idp,discord,game ext; class db store; - class sidecar,servuo bridge; + class modsrv,modcli mod; ``` ## Notes on the diagram @@ -95,12 +94,19 @@ flowchart TB access tokens plus rotated, hashed, revocable refresh tokens; SSO (Google/Discord/OIDC) is link-only and PKCE-guarded. All three surfaces resolve to the *same* session model via the session layer, and admin routes are re-validated against the DB on every request. +- **Core knows nothing about any game.** Routes, tables, nav entries, SPA pages and push streams for + a specific game arrive from a module the operator installed; core provides the seams and the + module fills them. Everything in the next three bullets is therefore **`module-uo`'s**, not core's + — it is described here because it is the worked example every other module is measured against. + A module that fails does not take the site down: the loader marks it `startup_failed`, the site + comes up with its routes and nav absent, and the admin panel says why. - **The shard is never reachable.** The ServUO shard *dials out* over loopback TCP `127.0.0.1:7788` - (newline-delimited JSON) to the uo-link sidecar; only the sidecar is exposed, and only the backend - talks to it. Every backend→sidecar call carries `Authorization: Bearer ` and an - `X-UOLink-Version` header (a protocol mismatch fails fast with `409`). The REST client - (`uoLinkClient.js`) never throws — every call returns `{ ok, data, status }` — so the site degrades - gracefully when the shard is down. That guarantee covers **reading the config too**: resolving the + (newline-delimited JSON) to the uo-link sidecar; only the sidecar is exposed, and only the + module's server half talks to it. Every module→sidecar call carries `Authorization: Bearer + ` and an `X-UOLink-Version` header (a protocol mismatch fails fast with `409`). The REST + client (`module-uo`'s `server/utils/uoLinkClient.js`) never throws — every call returns + `{ ok, data, status }` — so the site degrades gracefully when the shard is down. That guarantee + covers **reading the config too**: resolving the admin-managed config decrypts the stored auth token, which throws when the ciphertext can't be authenticated (`SECRET_ENC_KEY` rotated, or a DB dump restored under a different key). This is handled inside the client and reported as `{ ok: false, status: 0, error: 'uo-link config @@ -108,13 +114,14 @@ flowchart TB "unavailable" instead of 500ing it — and `GET /admin/uo-link/config` keeps working, which is the screen an admin needs to re-enter the token and recover. - **Two ways in from the sidecar.** Live game events arrive over an outbound **WebSocket** and are - routed by the `shardIngest.js` dispatcher (state-changing kinds update `shard_*` tables, notable - kinds append to `shard_events`, high-frequency kinds only update state). Point-in-time reads and - commands go over **REST** through `uoLinkClient.js`. + routed by the module's `server/utils/shardIngest.js` dispatcher (state-changing kinds update + `shard_*` tables, notable kinds append to `shard_events`, high-frequency kinds only update + state). Point-in-time reads and commands go over **REST** through the same `uoLinkClient.js`. - **Sensitive events stay private.** Ingested events fan out to browsers over two **SSE** channels — a public allowlist stream and an admin-only stream that additionally carries staff audit, cheat - detection, and login-attempt events. The allowlist split is a security boundary; sensitive kinds - can never leak onto the public channel. + detection, and login-attempt events. **Which kinds are public is declared by the module that + publishes them, and core enforces the split** — the boundary is core's even though the catalog is + the module's. Sensitive kinds can never leak onto the public channel. - **Secrets at rest.** OAuth client secrets, the uo-link token, and the Gmail refresh token are AES-256-GCM encrypted via `secretBox.js` (keyed by `SECRET_ENC_KEY`). The uo-link token is write-only in the API — never returned to any client. diff --git a/website/BACKEND_DESIGN.md b/website/BACKEND_DESIGN.md index a5e7668..34c0ee1 100644 --- a/website/BACKEND_DESIGN.md +++ b/website/BACKEND_DESIGN.md @@ -1,9 +1,14 @@ -# UOMysticmoon Website — Backend Design +# Runic Gateway Website — Backend Design > Phase 1 of 3: **backend design** → Claude Design (frontend mockup) → coding. > This document is the contract the later phases build against. -Public contact email: **UOMysticmoon@gmail.com** +**This is core's contract, and core is game-agnostic.** Nothing here is specific to any one game or +instance: the site's name, colours, logo and public contact address are data +(`BRAND_*` / the `settings` table), and everything about a *particular* game arrives from an +installed module — see [MODULE_SYSTEM.md](MODULE_SYSTEM.md) and, for the worked example, +[../modules/uo/](../modules/uo/README.md). **UOMysticmoon** is the first instance, and appears +below only as an example value. --- @@ -272,7 +277,8 @@ Seeded with the 8 spec categories: `new-player-guide, maps-atlas, systems, items Seeded keys: `site_mode` (default `maintenance`), `site_mode_changed_at`, `site_mode_changed_by`, `maintenance_message`, `status_message`, `homepage_teaser`, -`contact_email` (=UOMysticmoon@gmail.com), `site_title`, `player_registration` +`contact_email` (seeded from `BRAND_CONTACT_EMAIL`; e.g. `UOMysticmoon@gmail.com` on the first +instance), `site_title`, `player_registration` (default `disabled`), `mobile_app_links_enabled`, `module_source_hosts`. `module_source_hosts` is the allowlist of hostnames a module may be installed from diff --git a/website/website-README.md b/website/website-README.md index 0d21c09..5b15305 100644 --- a/website/website-README.md +++ b/website/website-README.md @@ -489,6 +489,32 @@ modules/ supported install. The directory is tracked in git (via its README) on purpose: Docker recreates a *missing* bind-mount source as `root:root`, and the container is uid 1000. +### Three ways in, and none of them is a build + +| | How | Where it fits | +|---|---|---| +| **Admin panel** | Admin → Modules, paste the URL of a release's install manifest | The click path. Installs, upgrades, disables, uninstalls and purges, with a restart button — no shell on the box | +| **`MODULES`** | Declare the set in the environment; the container resolves it at every start | The compose-managed host. The running set is a line in a file you version-control, not the residue of past clicks | +| **By hand** | `tar -xf module-uo-0.3.0.tar.gz -C ./modules && mv modules/module-uo-0.3.0 modules/uo`, then restart | Development, and any host where the other two do not fit | + +`MODULES` takes one entry per module, whitespace- or comma-separated: + +``` +MODULES=uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json +``` + +The id and the version are written out rather than discovered inside the manifest so that **the +no-op case needs no network**: a module already unpacked at the declared version is answered by +reading its own `module.json`, so a restart with the internet down brings the site up exactly as it +was. Only a missing or different version is fetched, and it goes through the same +verify-and-unpack path — allowlisted `https` host, sha256 from the manifest, whole-archive +inspection before anything is written — that the admin panel uses. A version that cannot be +resolved is logged and shown on the admin screen; **it never stops the site from starting**. + +The declaration owns what is *on the volume*, never what runs. A module disabled from the admin +panel gets its files back at the next start and stays disabled, because the row and the variable are +answering different questions. + ### What a module gets, and what it may not do At boot, `app.js` scans the volume synchronously, validates each `module.json`, and calls the @@ -534,6 +560,8 @@ Copy `.env.example` (Compose) or `server/.env.example` (local) and fill in. **`. | `PORT` | `3000` | server listens on `0.0.0.0:PORT` | | `UPLOAD_DIR` | `/uploads` | where post images are written (`/app/uploads`, volume-mounted, in Compose) | | `MODULES_DIR` | `/modules` | where installed modules are scanned from (`/app/modules`, bind-mounted, in Compose) | +| `MODULES` | — | the module set this deployment runs, resolved at every start: `@=`, whitespace/comma separated. Already at the declared version = no network. A failure is logged and shown in Admin → Modules, never fatal. See [Modules](#modules) | +| `MODULE_SOURCE_HOSTS` | `gitea.whitlocktech.com` | **bootstrap only** — seeds the `module_source_hosts` setting on first boot; after that the setting is authoritative and is edited in Admin → Modules | | `DB_HOST` / `DB_PORT` | `db` / `3306` | `db` in Compose; `127.0.0.1` for local dev | | `DB_NAME` / `DB_USER` / `DB_PASSWORD` | `runic_gateway` / `runic` / — | app database credentials | | `DB_ROOT_PASSWORD` | — | MariaDB root (Compose only) |