diff --git a/README.md b/README.md index 2bb803a..9f95f88 100644 --- a/README.md +++ b/README.md @@ -23,17 +23,24 @@ sidecar as a service, and hands you the values the website needs. | Doc | What it covers | |---|---| | [BACKEND_DESIGN.md](website/BACKEND_DESIGN.md) | API contract, DB schema, security model | +| [ARCHITECTURE.md](website/ARCHITECTURE.md) | The system diagram — how core, an installed module, the sidecar and the clients fit together | +| [TEAMS.md](website/TEAMS.md) | Teams as a platform primitive: roster, forums, notifications, Discord slash commands and voice — design of record | | [HERO_EDITOR.md](website/HERO_EDITOR.md) | Hero canvas editor feature spec | | [THEMING_AND_NAV.md](website/THEMING_AND_NAV.md) | Admin-configurable theme, brand assets and navigation — build contract | | [MODULE_SYSTEM.md](website/MODULE_SYSTEM.md) | Making the site game-agnostic: game logic becomes an installable module — design of record | | [MODULE_API.md](website/MODULE_API.md) | The module ↔ core contract: `ctx`, the `register*` calls, the client registry and the loader's obligations | | [WIKI_UPGRADE.md](website/WIKI_UPGRADE.md) | Wiki subsystem upgrade notes | | [SHARD_VISIBILITY.md](website/SHARD_VISIBILITY.md) | Who sees which shard data — the admin-configurable audience framework | +| [TRUSTED_DEVICES_MFA.md](website/TRUSTED_DEVICES_MFA.md) | TOTP two-factor, trusted devices and recovery codes | +| [MODERATION_APPEALS.md](website/MODERATION_APPEALS.md) | Moderation actions, content reports and the appeals flow | | [SPAWN_ATLAS.md](website/SPAWN_ATLAS.md) | The bestiary / spawn atlas: what the shard contains, parsed from its own ServUO tree | | [CLILOCS.md](website/CLILOCS.md) | UO's id → name table: converting one from your client so items have names | | [UOFIDDLER.md](website/UOFIDDLER.md) | **Operator runbook** — step-by-step extraction from your own UO client (cliloc table, creature art) | | [MARKETPLACE.md](website/MARKETPLACE.md) | The player-vendor index: how it is gathered, what it costs, how to tune it | | [website-README.md](website/website-README.md) | Snapshot of the website repo's README (setup/run reference) | +| [test-plan.md](website/test-plan.md) | The website's test strategy and harness | +| [API_V2_PLAN.md](website/API_V2_PLAN.md) | Router domain split + CSP hardening. The split is complete; only the CSP enforce step remains | +| [API_V2_SKELETON.md](website/API_V2_SKELETON.md) | **Superseded** — the `/api/v2` scaffold that was never built. Kept as the record of why the split was done in place instead | | [PROJECT_TREE.md](website/PROJECT_TREE.md) | Auto-generated snapshot of the repo's tracked file layout | ### `modules/` @@ -45,6 +52,8 @@ particular game; a module is what makes it a site *for* one. | Doc | What it covers | |---|---| | [uo/](modules/uo/README.md) | **module-uo** — the Ultima Online module: what it serves, what it owns, and what an operator needs | +| [uo/API.md](modules/uo/API.md) · [uo/SCHEMA.md](modules/uo/SCHEMA.md) | module-uo's own route surface and the tables it owns | +| [kit-acceptance.md](modules/kit-acceptance.md) | The Integration Kit acceptance run — building a module by following the kit alone, and what it found | | [rust-dryrun.md](modules/rust-dryrun.md) | A written, deliberately unimplemented `module-rust` — the test that the module contract generalises past the game it was extracted from | ### `link/` @@ -53,6 +62,7 @@ particular game; a module is what makes it a site *for* one. | [INTEGRATION.md](link/INTEGRATION.md) | How the website integrates with the uo-link sidecar | | [PROTOCOL_2.md](link/PROTOCOL_2.md) | Protocol 2.0 / 2.1 design | | [v3.md](link/v3.md) | Protocol 3.0 design — shard content/standings streams + the visibility framework | +| [v4.md](link/v4.md) | Protocol 4.0 — guild membership on the wire (`guild.roster`, `guild.leave`). **The current protocol** | | [ADMIN_CONTROLS.md](link/ADMIN_CONTROLS.md) | Staff write-plane (kick/ban/broadcast, page queue) | | [SHARD_PREREQS.md](link/SHARD_PREREQS.md) | Shard-side prerequisites for the bridge | | [PLAN.md](link/PLAN.md) | uo-link build plan | @@ -67,6 +77,7 @@ particular game; a module is what makes it a site *for* one. | [COVERAGE_PLAN.md](android/COVERAGE_PLAN.md) | Test-coverage rollout plan | | [APP_LINKS.md](android/APP_LINKS.md) | Android App Links / deep-link setup | | [theme-plan.md](android/theme-plan.md) | Theming plan | +| [THEMING_AND_NAV.md](android/THEMING_AND_NAV.md) | The app's half of admin-configurable theming and navigation — build contract | | [TRUSTED_DEVICES_APP_HANDOFF.md](android/TRUSTED_DEVICES_APP_HANDOFF.md) | Trusted-devices app handoff notes | | [PROJECT_TREE.md](android/PROJECT_TREE.md) | Auto-generated snapshot of the repo's tracked file layout | @@ -75,6 +86,12 @@ particular game; a module is what makes it a site *for* one. |---|---| | [INSTALL.md](installer/INSTALL.md) | **Start here to set up a shard** — the installer deploys the plugin overlay and the uo-link sidecar, registers the service, and connects it to the website. Appendix A is the same thing by hand, still supported | | [PLAN.md](installer/PLAN.md) | Installer design of record — phases, locked decisions, the bundle/compat-matrix model | +| [PROJECT_TREE.md](installer/PROJECT_TREE.md) | Auto-generated snapshot of the repo's tracked file layout | + +### `ci/` +| Doc | What it covers | +|---|---| +| [SONARQUBE.md](ci/SONARQUBE.md) | The SonarQube setup: project keys, how analysis runs, and how to read a report | ## Provenance diff --git a/website/ARCHITECTURE.md b/website/ARCHITECTURE.md index dfd43ea..358d758 100644 --- a/website/ARCHITECTURE.md +++ b/website/ARCHITECTURE.md @@ -1,9 +1,13 @@ # Website — Architecture How the pieces of `RunicGateway/website` fit together. The React SPA and the native mobile app -talk to one Express backend (`router → controller → model → db`), which persists to MariaDB and -bridges to the live game world **only** through the **uo-link** sidecar. The ServUO shard itself is -never internet-facing — it dials out to the sidecar over loopback, and only the sidecar is exposed. +talk to one Express backend (`router → controller → model → db`), which persists to MariaDB. + +**Core is game-agnostic.** Since the module system shipped on 2026-08-12, everything about a +particular game — its routes, tables, pages and its connection to a game server — lives in an +installed module, not here. For Ultima Online that is [`module-uo`](../modules/uo/README.md), which +bridges to the live world **only** through the **uo-link** sidecar; the ServUO shard itself is never +internet-facing, dialling out to the sidecar over loopback, and only the sidecar is exposed. This is the canonical copy of the diagram; the same diagram is embedded in the website's [`README.md`](https://gitea.whitlocktech.com/RunicGateway/website/src/branch/main/README.md#architecture). @@ -28,32 +32,29 @@ flowchart TB subgraph backend["server/ — Express backend"] direction TB mw["Middleware
helmet · siteMode · noindex
rateLimit · loginProtection · botScore · validate"] - router["Router /api/v1
auth (web · mobile · sso) · public · admin"] + router["Router /api/v1
auth (web · mobile · sso) · public · admin · player"] ctrl["Controllers"] auth["Session layer (auth/)
sessionService · JWT/cookie · bearer · SSO+PKCE"] model["Models (.model + .db)
raw parameterized SQL — no ORM"] sse["SSE fan-out
public stream (allowlist) · admin stream (sensitive)"] - - subgraph shardutil["Shard integration (utils/)"] - ingest["shardIngest.js
WS ingest dispatcher"] - restcli["uoLinkClient.js
REST client (never throws)"] - end - + loader["modules/loader.js
scans the volume · mounts · registries · lifecycle"] secret["secretBox.js
AES-256-GCM secrets at rest"] end bot["bot/
Discord bot"] end - db[("MariaDB
users · posts · wiki · settings · activity
mobileSessions · authProviders · userIdentities
uoLinkConfig · shard_online/economy/houses/events")] + db[("MariaDB
users · posts · wiki · settings · activity
mobileSessions · authProviders · userIdentities
installed_modules · <module>_*")] - %% ---------- Shard side ---------- - subgraph shardside["Game shard (never internet-facing)"] + %% ---------- Module side ---------- + subgraph modside["modules/<id>/  — installed, not built (e.g. Module-uo)"] direction TB - sidecar["uo-link sidecar
(Rust) — the only bridge exposed"] - servuo["ServUO shard
(C# plugin)"] + modsrv["server/ — routers, models, schema fragment
reaches core only through ctx"] + modcli["client/dist/entry.js — prebuilt ESM chunk
React shared via window.__rg"] end + game["The game
whatever the module talks to
(for Module-uo: a ServUO shard,
via the uo-link sidecar)"] + %% ---------- Edges ---------- browser <-->|"same-origin JSON + SSE (cookie)"| mw mobile -->|"REST (bearer access/refresh)"| mw @@ -63,29 +64,27 @@ flowchart TB mw --> router --> ctrl ctrl --> auth ctrl --> model - ctrl --> restcli ctrl --> sse auth --> model model <--> db auth -. reads/writes secrets .-> secret - restcli -. reads config/token .-> secret - ingest --> model - ingest --> sse sse -->|"live events"| browser bot -->|"messages"| discord bot <--> db - restcli -->|"REST: /char /roster /economy /history · /link/confirm · /towncrier"| sidecar - sidecar -->|"WebSocket live event feed (bearer + X-UOLink-Version)"| ingest - servuo -->|"loopback TCP 127.0.0.1:7788
newline-delimited JSON (shard dials out)"| sidecar + loader -->|"mounts under /api/v1/<tier>/<prefix>"| router + loader -->|"require() + register(ctx, api)"| modsrv + modsrv -->|"ctx.db · ctx.push · ctx.activity …"| model + modsrv <--> game + browser -->|"<script type=module> injected by htmlShell"| modcli %% ---------- Styling ---------- classDef ext fill:#2d2233,stroke:#7a5c94,color:#e8dff0; classDef store fill:#1f2d2a,stroke:#4c8c7d,color:#dff0ea; - classDef bridge fill:#2d2620,stroke:#94764c,color:#f0e6d8; - class idp,discord ext; + classDef mod fill:#2d2620,stroke:#94764c,color:#f0e6d8; + class idp,discord,game ext; class db store; - class sidecar,servuo bridge; + class modsrv,modcli mod; ``` ## Notes on the diagram @@ -95,12 +94,19 @@ flowchart TB access tokens plus rotated, hashed, revocable refresh tokens; SSO (Google/Discord/OIDC) is link-only and PKCE-guarded. All three surfaces resolve to the *same* session model via the session layer, and admin routes are re-validated against the DB on every request. +- **Core knows nothing about any game.** Routes, tables, nav entries, SPA pages and push streams for + a specific game arrive from a module the operator installed; core provides the seams and the + module fills them. Everything in the next three bullets is therefore **`module-uo`'s**, not core's + — it is described here because it is the worked example every other module is measured against. + A module that fails does not take the site down: the loader marks it `startup_failed`, the site + comes up with its routes and nav absent, and the admin panel says why. - **The shard is never reachable.** The ServUO shard *dials out* over loopback TCP `127.0.0.1:7788` - (newline-delimited JSON) to the uo-link sidecar; only the sidecar is exposed, and only the backend - talks to it. Every backend→sidecar call carries `Authorization: Bearer ` and an - `X-UOLink-Version` header (a protocol mismatch fails fast with `409`). The REST client - (`uoLinkClient.js`) never throws — every call returns `{ ok, data, status }` — so the site degrades - gracefully when the shard is down. That guarantee covers **reading the config too**: resolving the + (newline-delimited JSON) to the uo-link sidecar; only the sidecar is exposed, and only the + module's server half talks to it. Every module→sidecar call carries `Authorization: Bearer + ` and an `X-UOLink-Version` header (a protocol mismatch fails fast with `409`). The REST + client (`module-uo`'s `server/utils/uoLinkClient.js`) never throws — every call returns + `{ ok, data, status }` — so the site degrades gracefully when the shard is down. That guarantee + covers **reading the config too**: resolving the admin-managed config decrypts the stored auth token, which throws when the ciphertext can't be authenticated (`SECRET_ENC_KEY` rotated, or a DB dump restored under a different key). This is handled inside the client and reported as `{ ok: false, status: 0, error: 'uo-link config @@ -108,13 +114,14 @@ flowchart TB "unavailable" instead of 500ing it — and `GET /admin/uo-link/config` keeps working, which is the screen an admin needs to re-enter the token and recover. - **Two ways in from the sidecar.** Live game events arrive over an outbound **WebSocket** and are - routed by the `shardIngest.js` dispatcher (state-changing kinds update `shard_*` tables, notable - kinds append to `shard_events`, high-frequency kinds only update state). Point-in-time reads and - commands go over **REST** through `uoLinkClient.js`. + routed by the module's `server/utils/shardIngest.js` dispatcher (state-changing kinds update + `shard_*` tables, notable kinds append to `shard_events`, high-frequency kinds only update + state). Point-in-time reads and commands go over **REST** through the same `uoLinkClient.js`. - **Sensitive events stay private.** Ingested events fan out to browsers over two **SSE** channels — a public allowlist stream and an admin-only stream that additionally carries staff audit, cheat - detection, and login-attempt events. The allowlist split is a security boundary; sensitive kinds - can never leak onto the public channel. + detection, and login-attempt events. **Which kinds are public is declared by the module that + publishes them, and core enforces the split** — the boundary is core's even though the catalog is + the module's. Sensitive kinds can never leak onto the public channel. - **Secrets at rest.** OAuth client secrets, the uo-link token, and the Gmail refresh token are AES-256-GCM encrypted via `secretBox.js` (keyed by `SECRET_ENC_KEY`). The uo-link token is write-only in the API — never returned to any client. diff --git a/website/BACKEND_DESIGN.md b/website/BACKEND_DESIGN.md index a5e7668..34c0ee1 100644 --- a/website/BACKEND_DESIGN.md +++ b/website/BACKEND_DESIGN.md @@ -1,9 +1,14 @@ -# UOMysticmoon Website — Backend Design +# Runic Gateway Website — Backend Design > Phase 1 of 3: **backend design** → Claude Design (frontend mockup) → coding. > This document is the contract the later phases build against. -Public contact email: **UOMysticmoon@gmail.com** +**This is core's contract, and core is game-agnostic.** Nothing here is specific to any one game or +instance: the site's name, colours, logo and public contact address are data +(`BRAND_*` / the `settings` table), and everything about a *particular* game arrives from an +installed module — see [MODULE_SYSTEM.md](MODULE_SYSTEM.md) and, for the worked example, +[../modules/uo/](../modules/uo/README.md). **UOMysticmoon** is the first instance, and appears +below only as an example value. --- @@ -272,7 +277,8 @@ Seeded with the 8 spec categories: `new-player-guide, maps-atlas, systems, items Seeded keys: `site_mode` (default `maintenance`), `site_mode_changed_at`, `site_mode_changed_by`, `maintenance_message`, `status_message`, `homepage_teaser`, -`contact_email` (=UOMysticmoon@gmail.com), `site_title`, `player_registration` +`contact_email` (seeded from `BRAND_CONTACT_EMAIL`; e.g. `UOMysticmoon@gmail.com` on the first +instance), `site_title`, `player_registration` (default `disabled`), `mobile_app_links_enabled`, `module_source_hosts`. `module_source_hosts` is the allowlist of hostnames a module may be installed from diff --git a/website/website-README.md b/website/website-README.md index 0d21c09..5b15305 100644 --- a/website/website-README.md +++ b/website/website-README.md @@ -489,6 +489,32 @@ modules/ supported install. The directory is tracked in git (via its README) on purpose: Docker recreates a *missing* bind-mount source as `root:root`, and the container is uid 1000. +### Three ways in, and none of them is a build + +| | How | Where it fits | +|---|---|---| +| **Admin panel** | Admin → Modules, paste the URL of a release's install manifest | The click path. Installs, upgrades, disables, uninstalls and purges, with a restart button — no shell on the box | +| **`MODULES`** | Declare the set in the environment; the container resolves it at every start | The compose-managed host. The running set is a line in a file you version-control, not the residue of past clicks | +| **By hand** | `tar -xf module-uo-0.3.0.tar.gz -C ./modules && mv modules/module-uo-0.3.0 modules/uo`, then restart | Development, and any host where the other two do not fit | + +`MODULES` takes one entry per module, whitespace- or comma-separated: + +``` +MODULES=uo@0.3.0=https://gitea.whitlocktech.com/RunicGateway/Module-uo/releases/download/v0.3.0/module-uo-0.3.0.json +``` + +The id and the version are written out rather than discovered inside the manifest so that **the +no-op case needs no network**: a module already unpacked at the declared version is answered by +reading its own `module.json`, so a restart with the internet down brings the site up exactly as it +was. Only a missing or different version is fetched, and it goes through the same +verify-and-unpack path — allowlisted `https` host, sha256 from the manifest, whole-archive +inspection before anything is written — that the admin panel uses. A version that cannot be +resolved is logged and shown on the admin screen; **it never stops the site from starting**. + +The declaration owns what is *on the volume*, never what runs. A module disabled from the admin +panel gets its files back at the next start and stays disabled, because the row and the variable are +answering different questions. + ### What a module gets, and what it may not do At boot, `app.js` scans the volume synchronously, validates each `module.json`, and calls the @@ -534,6 +560,8 @@ Copy `.env.example` (Compose) or `server/.env.example` (local) and fill in. **`. | `PORT` | `3000` | server listens on `0.0.0.0:PORT` | | `UPLOAD_DIR` | `/uploads` | where post images are written (`/app/uploads`, volume-mounted, in Compose) | | `MODULES_DIR` | `/modules` | where installed modules are scanned from (`/app/modules`, bind-mounted, in Compose) | +| `MODULES` | — | the module set this deployment runs, resolved at every start: `@=`, whitespace/comma separated. Already at the declared version = no network. A failure is logged and shown in Admin → Modules, never fatal. See [Modules](#modules) | +| `MODULE_SOURCE_HOSTS` | `gitea.whitlocktech.com` | **bootstrap only** — seeds the `module_source_hosts` setting on first boot; after that the setting is authoritative and is edited in Admin → Modules | | `DB_HOST` / `DB_PORT` | `db` / `3306` | `db` in Compose; `127.0.0.1` for local dev | | `DB_NAME` / `DB_USER` / `DB_PASSWORD` | `runic_gateway` / `runic` / — | app database credentials | | `DB_ROOT_PASSWORD` | — | MariaDB root (Compose only) |