diff --git a/.gitea/workflows/pr-checks.yml b/.gitea/workflows/pr-checks.yml index 44e4e6c..bce64bd 100644 --- a/.gitea/workflows/pr-checks.yml +++ b/.gitea/workflows/pr-checks.yml @@ -65,6 +65,19 @@ jobs: # the shorter form would pass locally and break only here. run: npm test + - name: Sidebar + # PLAN.md §12, phase 8. src/config/sidebar.mjs holds two trees — the one Starlight + # renders and the one §10 planned — and they must agree on groups, labels and + # ORDER. Order because the order of "Getting started" IS the installation path. + # + # While pages were being written the planned tree was a checklist; now that every + # page exists it is a hand-maintained second copy, and it had already drifted + # unnoticed (phase 7 added Content under D37 and never updated it). Nothing caught + # that because nothing read it. + # + # No token, no network, no build — so it runs early and fails fast. + run: npm run check:sidebar + - name: Production build run: npm run build @@ -115,3 +128,22 @@ jobs: env: GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }} run: npm run check:quickstart + + - name: Reference enumerations against their sources + # PLAN.md §12, phase 8. The Reference section names things — every environment + # variable, config key, installer command, visibility rung and canonical document. + # §1 forbids re-specifying a contract, and this is what makes writing the NAMES + # down safe anyway: each list is a SET comparison against the repository that owns + # it, in both directions. + # + # The second direction is the one that earns its keep. A reference page does not + # usually rot by describing something that vanished — it rots by quietly not + # mentioning the three things added since it was written. + # + # Descriptions are deliberately NOT checked; nothing here can know whether a + # one-line summary is still true, so it does not pretend to. + # + # Same token, and for the same reason: it reads five other repositories in the org. + env: + GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }} + run: npm run check:reference diff --git a/PLAN.md b/PLAN.md index 15b2137..f76daca 100644 --- a/PLAN.md +++ b/PLAN.md @@ -236,7 +236,7 @@ Taken by the org lead (Colby Whitlock) on 2026-08-19. Recorded so they are not r **Decisions after D13 are recorded where they were taken**, in the section describing the phase that raised them, rather than appended here — a decision is only re-litigated when its reasoning is -somewhere other than the thing it decided. The count of record is **thirty-seven**: +somewhere other than the thing it decided. The count of record is **forty-one**: | # | Where | What it settled | |---|---|---| @@ -246,6 +246,7 @@ somewhere other than the thing it decided. The count of record is **thirty-seven | D26–D29 | §8, "How phase 5 built the app and the beta" | The screenshot slot reserved for phase 9, the demo as the tester target, `/beta` handling its own POST, equal billing for the APK and the beta | | D30–D33 | §9, "How phase 6 built the legal pages" | One logging hop and no edge provider, eighteen or older, no governing-law clause, the Data Safety notes as a generated document | | D34–D37 | §10, "How phase 7 built the documentation journey" | One PR for all twenty pages, a self-contained install quickstart with a drift check, every admin screen walked before it was described, a thirteenth Administration page for content | +| D38–D41 | §10, "How phase 8 built the builder and reference docs" | One PR for all twenty pages again, Reference enumerates names and checks every one of them, the docs section links to the drawn diagrams rather than importing them, `plannedSidebar` becomes a checked invariant | --- @@ -837,7 +838,7 @@ Reference Environment variables · Installer CLI · sidecar.toml · Bridge.cfg · HTTP API · Event catalog · Canonical documents ``` -Roughly 38 pages — 37 planned, plus the Content page D37 added in phase 7. Every Reference page is a **navigable summary plus a link to the canonical +**Forty pages** — thirty-nine planned, plus the Content page D37 added in phase 7. (This said "roughly 38, 37 planned" until phase 8 counted the tree: 7 + 13 + 8 + 5 + 7. `checkSidebar.mjs` now keeps the count honest.) Every Reference page is a **navigable summary plus a link to the canonical document** — never a re-specification, per §1. ### The installation path @@ -938,6 +939,79 @@ and phase 6 (the card void) for the same lesson. --- +### How phase 8 built the builder and reference docs + +Twenty more pages — Modules (8), Architecture (5), Reference (7) — completing the tree §10 +planned. Four decisions, taken by the org lead before anything was written. + +**D38 — one PR for all twenty pages, again.** The alternative on the table was splitting the +prose (Modules + Architecture) from Reference, since only Reference needed new checking +machinery. Rejected for the same reason D34 was: the three sections cross-reference each +other heavily, and a split means either landing pages whose links point at nothing yet or +writing the links twice. + +**D39 — Reference enumerates the NAMES, and checks every one of them.** This is the phase's +central decision, because §1 forbids re-specifying a contract and a Reference section is +exactly where that rule is most tempting to break. + +The line drawn: **names are on the page, semantics are not.** Every environment variable, +config key, installer command, visibility rung and canonical document is listed, with one +terse line saying what it is *for*. Shapes, defaults that matter, interactions and every +"why" stay in the canonical document. + +That is only safe because `scripts/checkReference.mjs` compares each list against the +repository that owns it — six sources, over the Gitea API, never from a working tree — as a +**set comparison in both directions**. The second direction is the one that earns its keep: +a reference page does not usually rot by describing something that vanished, it rots by +quietly not mentioning the three things added since it was written. + +The alternative considered was strict summary-plus-link with nothing enumerated. It needs no +machinery and cannot rot — but a Reference section that cannot answer "what variables are +there?" without a click-through is a link farm, and the checking machinery turned out to be +one script. + +Descriptions are deliberately **not** checked, and the script says so. Nothing can know +whether a one-line summary is still true; keeping them short enough to re-read is the +mitigation, not a check. + +**D40 — the docs link to the drawn diagrams rather than importing them.** `/architecture/`'s +three diagrams are Astro components carrying marketing chrome and depending on +`src/styles/diagram.css`, which Starlight does not load. Reusing them inside the docs would +have coupled the two layouts for one page's benefit. The docs use text diagrams in code +blocks — which are also copy-pasteable into an issue — and link out to the drawn versions. + +**D41 — `plannedSidebar` stops being a checklist and becomes a checked invariant.** It was +written in phase 1 so phases 7 and 8 had their checklist where they would be working. With +every page now written it is a second, hand-maintained copy of the live tree, which is the +exact shape §1 warns about — so `checkSidebar.mjs` asserts the two agree on groups, labels +**and order**. + +Order, because the order of "Getting started" *is* the installation path, and a reordering +nobody noticed would be a worse defect than a missing page. + +**What the checks found, before any of the pages shipped.** + +- **`plannedSidebar` had already drifted.** Phase 7 added the Content page under D37 and + never updated the planned list. Nothing failed, because nothing read it — which is the + whole argument for D41. Reproduced by deleting the entry again and watching the new check + catch it. +- **The page count in this document was wrong**, and had been since §10 was written: it said + "roughly 38 — 37 planned", where the tree it describes is forty. +- **`module.json`'s `mounts` and the SPA's paths are different mechanisms**, which is not + stated plainly in any one place. `module-uo` declares `admin: ["/shard", "/uo-link"]` and + its screen lives at `/admin/uo/link`; API routes are deliberately *not* namespaced while + SPA routes are. That is the distinction the installer got wrong in v0.1.0, and it now has + a named home on *The module system*. + +**The check was verified by breaking it, not by watching it pass.** It went green on its +first run, which is the least trustworthy possible outcome, so seven mutations were fed +through it — a stale name, an omitted name, a renamed key in each of three sources, a +canonical document that moved, and the visibility ladder **reordered with its membership +unchanged**. All seven failed the build. The ladder case is the one worth keeping: it is a +security boundary, and a set comparison alone would have passed it. + +--- + ## 11. Visual direction **"Modern infrastructure software with an arcane identity."** Dark-first. Marketing pages are @@ -1053,10 +1127,10 @@ a mechanism rather than diligence: | **5** | The app and the beta: `/app/`, `/beta/`, the signup handler, the SQLite store, rate limiting, the export CLI (§8). **Also the repository's first `node --test` suite**, and phase 9 inherits an emulator pass (D26) | | **6** | Legal: `/privacy/`, `/terms/`, footer links, and the Play Data Safety notes (§9) | | **7** | Docs — the journey: Getting started (7) + Administration (**13**, per D37) — twenty pages in one PR (D34), with the install page self-contained and drift-checked (D35) and every admin screen walked before it was described (D36). **The installation path is the priority of the whole project** | -| **8** | Docs — builder and reference: Modules (8) + Architecture (5) + Reference (7) | +| **8** | Docs — builder and reference: Modules (8) + Architecture (5) + Reference (7) — twenty pages in one PR (D38), with Reference enumerating names and **checking every one of them** against its source (D39), and `plannedSidebar` becoming a checked invariant (D41) | | **9** | Screenshots (D4): stand up the local review stack, seed presentable content, capture the admin panel, Teams, forums, marketplace, spawn atlas and shard console; build the screenshot components. **Plus an emulator pass against the same seeded stack** to fill `/app/`'s reserved slot (D26) | | **10** | Polish: responsive, accessibility, SEO/OpenGraph/sitemap/robots, full-text search, CSP headers | -| **11** | Validation: `astro check`, production build, **all six check scripts** (tokens, brand, links, facts, quickstart, data safety), mobile layout verified in a real browser, a signup walked end to end | +| **11** | Validation: `astro check`, production build, **all eight check scripts** (tokens, brand, links, facts, quickstart, data safety, reference, sidebar), mobile layout verified in a real browser, a signup walked end to end | | **12** | Delivery: Dockerfile, `docker-compose.yml` with both bind mounts documented, Gitea Actions workflow publishing to the registry, README, CONTRIBUTING with the AI-disclosure requirement, and an operator note covering DNS, TLS and the reverse proxy (D6) | Phases 5 and 6 are deliberately adjacent and early: the beta cannot start without `/privacy`, and diff --git a/package.json b/package.json index b38a2f1..0d46eae 100644 --- a/package.json +++ b/package.json @@ -20,11 +20,13 @@ "check:links": "node scripts/checkLinks.mjs", "check:datasafety": "node scripts/playDataSafety.mjs --check", "check:quickstart": "node scripts/checkQuickstart.mjs", + "check:reference": "node scripts/checkReference.mjs", + "check:sidebar": "node scripts/checkSidebar.mjs", "play:datasafety": "node scripts/playDataSafety.mjs", "beta": "node scripts/beta.mjs", "test": "node --test test/beta.test.mjs test/legal.test.mjs", "brand:assets": "node scripts/buildBrandAssets.mjs", - "verify": "npm run check:tokens && npm run check:brand && npm run check:datasafety && npm run check && npm test && npm run build && npm run check:links && npm run check:facts && npm run check:quickstart" + "verify": "npm run check:sidebar && npm run check:tokens && npm run check:brand && npm run check:datasafety && npm run check && npm test && npm run build && npm run check:links && npm run check:facts && npm run check:quickstart && npm run check:reference" }, "dependencies": { "@astrojs/node": "^11.1.4", diff --git a/scripts/checkReference.mjs b/scripts/checkReference.mjs new file mode 100644 index 0000000..e297ce0 --- /dev/null +++ b/scripts/checkReference.mjs @@ -0,0 +1,191 @@ +#!/usr/bin/env node +/** + * checkReference.mjs — PLAN.md §12, added in phase 8. + * + * The Reference section names things: every environment variable, every config key, every + * installer command, every canonical document. §1 forbids re-specifying a contract, and + * this is the machinery that makes writing the NAMES down safe anyway — the same bargain + * checkQuickstart.mjs struck for the quickstart, applied to six more sources. + * + * Each enumeration in `src/data/reference.mjs` is compared against its authority, read from + * the repository that owns it over the Gitea API — never from a working tree, per §1's + * process rule. Every comparison is a SET comparison in both directions: + * + * - a name this site lists that the source no longer has fails (the reference is stale); + * - a name the source has that this site does not list fails (the reference is + * incomplete, which is the failure mode a hand-maintained list actually has). + * + * The second direction is the one that earns its keep. A reference page does not usually + * rot by describing something that vanished — it rots by quietly not mentioning the three + * things added since it was written. + * + * Descriptions are deliberately NOT checked. Nothing here can know whether a one-line + * summary is still true, so it does not pretend to; keeping them terse is the mitigation. + * + * GITEA_TOKEN= node scripts/checkReference.mjs + * + * Anonymous raw fetches fail on this instance, so the token is required. A check that + * silently skips itself is worse than no check. + */ + +import { readFileSync } from 'node:fs'; +import { fileURLToPath } from 'node:url'; +import path from 'node:path'; + +import { + envVars, + sidecarConfig, + installerCommands, + bridgeCfg, + visibilityLadder, + canonicalDocs, +} from '../src/data/reference.mjs'; + +const ROOT = fileURLToPath(new URL('..', import.meta.url)); +const platform = JSON.parse(readFileSync(path.join(ROOT, 'src/data/platform.json'), 'utf8')); + +const BASE = platform.gitea.base; +const ORG = platform.gitea.org; +const TOKEN = process.env.GITEA_TOKEN?.trim(); + +const failures = []; +const checked = []; +const ok = (what) => checked.push(what); +const fail = (what, detail) => failures.push({ what, detail }); + +/** Same raw-file accessor checkFacts.mjs and checkQuickstart.mjs use. */ +async function raw(repo, filePath, ref = 'main') { + const url = `${BASE}/api/v1/repos/${ORG}/${repo}/raw/${filePath}?ref=${encodeURIComponent(ref)}`; + const res = await fetch(url, { headers: { Authorization: `token ${TOKEN}` } }); + if (!res.ok) throw new Error(`${res.status} ${res.statusText} for ${url}`); + return res.text(); +} + +/** + * The one comparison this whole script performs, so the failure messages are identical + * everywhere and say which direction broke. + */ +function compareSets(label, mine, theirs, hint) { + const mineSet = new Set(mine); + const theirsSet = new Set(theirs); + + const stale = [...mineSet].filter((k) => !theirsSet.has(k)); + const missing = [...theirsSet].filter((k) => !mineSet.has(k)); + + for (const k of stale) { + fail(`${label}: ${k}`, `listed here, but ${hint} no longer has it — remove it, and re-read the prose around it`); + } + for (const k of missing) { + fail(`${label}: ${k}`, `is in ${hint} and NOT listed here — add it, or the reference is lying by omission`); + } + if (!stale.length && !missing.length) ok(`${label} (${mineSet.size})`); +} + +/** `KEY=value` lines. Commented-out suggestions are prose about a variable, not a key. */ +const envKeysOf = (text) => + text + .split(/\r?\n/) + .map((l) => l.match(/^([A-Z][A-Z0-9_]*)=/)) + .filter(Boolean) + .map((m) => m[1]); + +/** `Key=value` lines from the plugin's config, same rule about comments. */ +const cfgKeysOf = (text) => + text + .split(/\r?\n/) + .map((l) => l.match(/^([A-Za-z][A-Za-z0-9]*)=/)) + .filter(Boolean) + .map((m) => m[1]); + +async function run() { + if (!TOKEN) { + console.error('checkReference: GITEA_TOKEN is not set. This check cannot run anonymously.'); + process.exit(2); + } + + // ── 1. Environment variables ────────────────────────────────────────────── + compareSets( + 'env', + Object.keys(envVars), + envKeysOf(await raw('website', '.env.example')), + 'website main:.env.example', + ); + + // ── 2. sidecar.toml ─────────────────────────────────────────────────────── + // + // Parsed from the serde structs rather than from a sample file, because the sample is + // GENERATED by the binary on first run and no committed copy is authoritative. Each + // `pub name: T` inside a `struct XCfg` is one key, and the struct name gives the section. + const configRs = await raw('link', 'sidecar/src/config.rs'); + const sidecarKeys = []; + for (const m of configRs.matchAll(/struct\s+(\w+)Cfg\s*\{([\s\S]*?)\n\}/g)) { + const section = m[1].toLowerCase(); + for (const f of m[2].matchAll(/pub\s+(\w+)\s*:/g)) sidecarKeys.push(`${section}.${f[1]}`); + } + compareSets('sidecar.toml', Object.keys(sidecarConfig), sidecarKeys, 'link main:sidecar/src/config.rs'); + + // ── 3. Installer commands ───────────────────────────────────────────────── + const cliRs = await raw('installer', 'src/cli.rs'); + const cmdBlock = cliRs.match(/enum\s+Command\s*\{([\s\S]*?)\n\}/); + const cmds = cmdBlock ? [...cmdBlock[1].matchAll(/^\s*([A-Z]\w*)\s*[,{]/gm)].map((m) => m[1]) : []; + compareSets('installer command', Object.keys(installerCommands), cmds, 'installer main:src/cli.rs'); + + // ── 4. Bridge.cfg ───────────────────────────────────────────────────────── + const bridgeKeys = Object.values(bridgeCfg).flatMap((group) => Object.keys(group)); + compareSets( + 'Bridge.cfg', + bridgeKeys, + cfgKeysOf(await raw('servuo-plugins', 'overlay/Config/Bridge.cfg')), + 'servuo-plugins main:overlay/Config/Bridge.cfg', + ); + + // ── 5. The visibility ladder ────────────────────────────────────────────── + // + // A security boundary, so it is checked against the module that enforces it rather than + // against prose. The order matters as much as the membership: it is a ladder, and a + // reader reasoning about "staff and above" needs the rungs in the right sequence. + const vis = await raw('Module-uo', 'server/utils/shardVisibility.js'); + const ladderMatch = vis.match(/const\s+LADDER\s*=\s*\[([\s\S]*?)\]/); + const ladder = ladderMatch + ? [...ladderMatch[1].matchAll(/'([a-z_]+)'/g)].map((m) => m[1]) + : []; + if (ladder.length === 0) { + fail('visibility ladder', 'could not find LADDER in Module-uo main:server/utils/shardVisibility.js'); + } else if (ladder.join(' ') !== visibilityLadder.join(' ')) { + fail( + 'visibility ladder', + `order or membership differs — here "${visibilityLadder.join(' → ')}", upstream "${ladder.join(' → ')}"`, + ); + } else ok(`visibility ladder (${ladder.length} rungs, in order)`); + + // ── 6. Canonical documents ──────────────────────────────────────────────── + // + // Existence only. A link to a document that moved is the single most likely way this + // section breaks, and it is exactly what a build can answer. + for (const docPath of Object.keys(canonicalDocs)) { + const url = `${BASE}/api/v1/repos/${ORG}/docs/contents/${docPath}?ref=main`; + const res = await fetch(url, { headers: { Authorization: `token ${TOKEN}` } }); + if (res.ok) ok(`canonical doc ${docPath}`); + else fail(`canonical doc ${docPath}`, `not found in docs main (HTTP ${res.status})`); + } + + // ── Report ──────────────────────────────────────────────────────────────── + if (failures.length === 0) { + console.log(`checkReference: ${checked.length} enumeration check(s) passed against their sources.`); + return; + } + + console.error(`\ncheckReference: ${failures.length} disagreement(s) with the platform:\n`); + for (const f of failures) console.error(` ✗ ${f.what}\n ${f.detail}`); + console.error(` +The Reference section names things, which is only safe while the names are checked +(§1, and the same bargain checkQuickstart.mjs struck). Update src/data/reference.mjs +to match the source. Do not "fix" the check. +`); + process.exit(1); +} + +run().catch((err) => { + console.error(`checkReference: ${err.message}`); + process.exit(2); +}); diff --git a/scripts/checkSidebar.mjs b/scripts/checkSidebar.mjs new file mode 100644 index 0000000..e650f0d --- /dev/null +++ b/scripts/checkSidebar.mjs @@ -0,0 +1,77 @@ +#!/usr/bin/env node +/** + * checkSidebar.mjs — PLAN.md §12, added in phase 8. + * + * `src/config/sidebar.mjs` holds two trees: `docsSidebar`, which Starlight renders, and + * `plannedSidebar`, the tree §10 planned. While pages were still being written the second + * was a checklist. Now that every page exists it is a second copy of the first, maintained + * by hand — and a hand-maintained copy with nothing reading it is exactly the shape of + * thing §1 is about. + * + * It had already drifted, silently: phase 7 added the `Content` page under D37 and this + * list was never updated. Nothing failed, because nothing read it. That is the whole + * argument for this check. + * + * So the two must agree on groups, labels AND order. Order is checked because the order of + * "Getting started" IS the installation path — §10 calls it the priority of the whole + * project — and a reordering that nobody noticed would be a worse defect than a missing + * page. + * + * node scripts/checkSidebar.mjs + * + * No token and no network: both trees are in this repository. + */ + +import { docsSidebar, plannedSidebar } from '../src/config/sidebar.mjs'; + +const failures = []; +const fail = (what, detail) => failures.push({ what, detail }); + +const live = new Map(docsSidebar.map((g) => [g.label, g.items.map((i) => i.label)])); +const planned = new Map(Object.entries(plannedSidebar)); + +// ── Groups ────────────────────────────────────────────────────────────────── +for (const label of live.keys()) { + if (!planned.has(label)) fail(`group ${label}`, 'is in the live sidebar and not in plannedSidebar'); +} +for (const label of planned.keys()) { + if (!live.has(label)) fail(`group ${label}`, 'is in plannedSidebar and not in the live sidebar'); +} + +// ── Pages, in order ───────────────────────────────────────────────────────── +for (const [label, liveItems] of live) { + const plannedItems = planned.get(label); + if (!plannedItems) continue; + + for (const page of liveItems) { + if (!plannedItems.includes(page)) fail(`${label} → ${page}`, 'is live but not in plannedSidebar'); + } + for (const page of plannedItems) { + if (!liveItems.includes(page)) fail(`${label} → ${page}`, 'is planned but has no live sidebar entry'); + } + + // Only meaningful once membership matches; otherwise it just repeats the above. + if (liveItems.length === plannedItems.length && liveItems.every((p) => plannedItems.includes(p))) { + if (liveItems.join(' | ') !== plannedItems.join(' | ')) { + fail( + `${label} order`, + `live "${liveItems.join(' → ')}" vs planned "${plannedItems.join(' → ')}"`, + ); + } + } +} + +// ── Report ────────────────────────────────────────────────────────────────── +if (failures.length === 0) { + const pages = [...live.values()].reduce((n, items) => n + items.length, 0); + console.log(`checkSidebar: ${live.size} groups and ${pages} pages agree with plannedSidebar.`); +} else { + console.error(`\ncheckSidebar: ${failures.length} disagreement(s) between the two trees:\n`); + for (const f of failures) console.error(` ✗ ${f.what}\n ${f.detail}`); + console.error(` +Both trees are in src/config/sidebar.mjs. Decide which one is right — if a page was +deliberately added, renamed or reordered, plannedSidebar records that decision and +should move with it. +`); + process.exit(1); +} diff --git a/src/config/sidebar.mjs b/src/config/sidebar.mjs index 7d5ec67..08a06cc 100644 --- a/src/config/sidebar.mjs +++ b/src/config/sidebar.mjs @@ -44,12 +44,54 @@ export const docsSidebar = [ { label: 'Troubleshooting', slug: 'docs/administration/troubleshooting' }, ], }, + { + label: 'Modules', + items: [ + { label: 'The module system', slug: 'docs/modules/the-module-system' }, + { label: 'Installing modules', slug: 'docs/modules/installing-modules' }, + { label: 'Module lifecycle', slug: 'docs/modules/module-lifecycle' }, + { label: 'The module manifest', slug: 'docs/modules/the-module-manifest' }, + { label: 'The module API', slug: 'docs/modules/the-module-api' }, + { label: 'Building a module', slug: 'docs/modules/building-a-module' }, + { label: 'The Integration Kit', slug: 'docs/modules/the-integration-kit' }, + { label: 'Testing and release', slug: 'docs/modules/testing-and-release' }, + ], + }, + { + label: 'Architecture', + items: [ + { label: 'System architecture', slug: 'docs/architecture/system-architecture' }, + { label: 'The bridge', slug: 'docs/architecture/the-bridge' }, + { label: 'Authentication architecture', slug: 'docs/architecture/authentication-architecture' }, + { label: 'Teams architecture', slug: 'docs/architecture/teams-architecture' }, + { label: 'Protocol versions', slug: 'docs/architecture/protocol-versions' }, + ], + }, + { + label: 'Reference', + items: [ + { label: 'Environment variables', slug: 'docs/reference/environment-variables' }, + { label: 'Installer CLI', slug: 'docs/reference/installer-cli' }, + { label: 'sidecar.toml', slug: 'docs/reference/sidecar-toml' }, + { label: 'Bridge.cfg', slug: 'docs/reference/bridge-cfg' }, + { label: 'HTTP API', slug: 'docs/reference/http-api' }, + { label: 'Event catalog', slug: 'docs/reference/event-catalog' }, + { label: 'Canonical documents', slug: 'docs/reference/canonical-documents' }, + ], + }, ]; /** - * The full planned tree, kept next to the live sidebar so phases 7 and 8 have their - * checklist in the place they will be working. Not exported into the Starlight config — - * it names pages that do not exist yet. + * The tree §10 planned, kept as the record of what was intended — every page it names now + * exists, as of phase 8. + * + * It was the phases 7/8 checklist, and a checklist with nothing left on it is no longer + * pulling its weight: it is a second copy of the tree above, maintained by hand, and it had + * already drifted once (phase 7 added `Content` under D37 and this list was not updated, + * which nothing caught because nothing reads it). `checkSidebar.mjs` now asserts the two + * agree, which is what makes keeping it safe. + * + * Not exported into the Starlight config. */ export const plannedSidebar = { 'Getting started': [ @@ -65,6 +107,7 @@ export const plannedSidebar = { 'Configuration', 'Branding and theming', 'Navigation and pages', + 'Content', 'Users and roles', 'Authentication', 'Teams', diff --git a/src/content/docs/docs/architecture/authentication-architecture.mdx b/src/content/docs/docs/architecture/authentication-architecture.mdx new file mode 100644 index 0000000..13eb968 --- /dev/null +++ b/src/content/docs/docs/architecture/authentication-architecture.mdx @@ -0,0 +1,123 @@ +--- +title: Authentication architecture +description: One session model behind three very different front doors — cookies, bearer tokens and SSO — and where the boundaries actually are. +--- + +import { Aside } from '@astrojs/starlight/components'; + +The administrator's view of this is +[Authentication](/docs/administration/authentication/). This is how it is built. + +## One session service, three surfaces + +The governing decision: **there is a single source of truth for sessions**, and every +authentication surface produces the *same* session model. + +``` + browser native app SSO provider + (httpOnly JWT) (bearer + refresh) (OAuth2 / OIDC + PKCE) + │ │ │ + └───────────────────┼────────────────────────┘ + ▼ + sessionService + createSession(user, authMethod) + validateSession() +``` + +Controllers call `createSession`; middleware calls `validateSession`. Nothing invents its +own notion of "logged in". + +That matters more than it sounds. Three front doors with three session implementations is +three places for an authorization bug to hide, and the one that gets least attention is the +one that gets exploited. + + + +## The three surfaces + +**Web** — a JWT signed with `JWT_SECRET`, carried in an `httpOnly`, `sameSite=Lax` cookie. +`secure` is decided **per request** (`COOKIE_SECURE=auto` → `secure: req.secure`), which is +what lets one deployment work both over HTTPS through a proxy and over plain HTTP on a LAN +address. + +**Mobile** — short-lived bearer access tokens plus **rotated, hashed, revocable** refresh +tokens. Hashed server-side, so a database disclosure does not hand over live sessions. + +**SSO** — Google, Discord or a custom OIDC provider, PKCE-guarded. + +## SSO is link-only, by policy + +**An external identity must already be linked to an existing account.** Identities are +never auto-provisioned. + +This is a deliberate policy rather than an unimplemented feature. Auto-provisioning turns +"anyone with a Google account" into "anyone with an account here", which is not a decision +a site operator should make by installing an OAuth client. + +## Admin is re-validated every request + +Roles are **re-checked against the database on every admin request**, not trusted from the +token. + +The consequence is the point: a demoted user loses access **at once**, rather than when +their token happens to expire. A stateless JWT that carried the role would keep asserting it +for up to a day. + +## Trusted devices gate the second factor only + +A second, separate httpOnly cookie (`rg_trust`, 30 days by default) lets a browser or app +**skip the TOTP step** on future logins — **never the password**. + +Four properties, each chosen: + +- It is **opaque and sha256-hashed server-side**, stored in a table. It is not a JWT claim, + so the stateless session token is unchanged. +- It is **per-row revocable**, from the admin panel or by the user. +- It **deliberately outlives logout.** Logging out ends a session; it does not make the + device untrusted, because the device is still the same device. +- It is **cleared** on untrust, password change, password reset, or disabling TOTP. + +**Recovery codes** (bcrypt, single-use) are the lockout fallback. Every trusted-device and +MFA action is audit-logged. + +## The login-hardening layer + +Bot scoring with automatic IP banning, TOTP 2FA, a honeypot field, and rate limiting with +backoff. The admin *Bot Activity* panel is deliberately **read plus emergency-unban only** — +it is a window onto an automatic system, not a control surface for it. + +## Where core's boundaries stop + +Core's security boundaries end at **authentication, roles and the session**. + +A module that serves game data brings its **own** audience rules, and core does not police +them beyond the gates it hands over — `requireAuth`, `requireRole`, and the tier group +gates. See [The module API](/docs/modules/the-module-api/#registerroutes-and-the-tier-gate). + +`module-uo`'s is the worked example, and it is a real boundary rather than a convenience +filter: an admin-configurable, per-feature and per-field audience ladder with **fail-closed +defaults**, applied at routes, at SSE subscribe time, *and* at the navigation. All three, +because a surface that is filtered in only two of those places leaks through the third. + +## Content Security Policy + +`script-src 'self'` with **no inline script**, which is why [module chunks are served +same-origin](/docs/modules/building-a-module/) and why an import map was never an option. + +`form-action 'self'` is pinned explicitly rather than inherited, because it blocks an +injected form POSTing credentials off-origin — an exfiltration path `connect-src` does not +cover. + +Violation reports go to a **same-origin** sink that stores nothing: reports describe attacks +against this site and are not handed to a third-party collector. It parses both wire formats +(browsers disagree), and always answers `204` even for malformed input — a `4xx` would make +the error handler log attacker-supplied bodies and turn an open endpoint into a log-flood +primitive. + +## Canonical document + +[`BACKEND_DESIGN.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/BACKEND_DESIGN.md) +§6 is normative for everything on this page. diff --git a/src/content/docs/docs/architecture/protocol-versions.mdx b/src/content/docs/docs/architecture/protocol-versions.mdx new file mode 100644 index 0000000..c14d451 --- /dev/null +++ b/src/content/docs/docs/architecture/protocol-versions.mdx @@ -0,0 +1,100 @@ +--- +title: Protocol versions +description: One number, declared in three repositories, that decides whether a shard and a sidecar are allowed to talk to each other. +--- + +import { Aside } from '@astrojs/starlight/components'; + +The loopback wire protocol between the game plugin and the sidecar is a **versioned +compatibility contract**, not a build dependency. Nothing compiles the three sides together, +so the number is what stops a mismatch from being discovered as corrupted data. + +The current protocol is **4**. + +## Three declaration sites + +The same number is written down in three places, and they must move together. + +| Where | What declares it | +|---|---| +| `link/sidecar/src/main.rs` | `pub const PROTOCOL_VERSION: u32 = 4` — what the sidecar speaks | +| `servuo-plugins/overlay.toml` | `protocol = 4` — what the plugin overlay speaks | +| The bundle manifest | Copied from `overlay.toml` by CI, so a released pair carries its own claim | + + + +## How a mismatch is caught + +Two independent mechanisms, at two different boundaries. + +**Sidecar ↔ website.** Every sidecar response carries `X-UOLink-Version`. A mismatch is +rejected with **`409`** rather than mis-parsed. The website's protocol expectation is +admin-managed, alongside the base URL and token, on the shard configuration screen. + +**Overlay ↔ sidecar.** The installer resolves a **bundle** — an exact, protocol-checked +sidecar and overlay pair published by CI — and never "latest of each". That is the whole +reason bundles exist: two independently released components that must agree cannot be +allowed to be chosen independently. + +## What a bump obliges + +Changing a message shape means editing every side plus the specification. A protocol-4 +change touched: + +| Repository | What had to change | +|---|---| +| `servuo-plugins` | The emitters, the config keys, and `overlay.toml` | +| `link` | `PROTOCOL_VERSION`, a store migration, and the projections | +| `module-uo` | The tables, the ingest, and the kind-to-feature map | +| `docs` | The protocol document and the integration guide | + +Note `link`'s entry: **a protocol bump can require a store migration**, because the sidecar +persists what it forwards. That is not automatic, and version 4 was the first bump that +needed one. + +## This is not the module API version + +Two different numbers, versioning two different contracts, and confusing them is easy. + +| | Versions | Lives in | Checked | +|---|---|---|---| +| **`PROTOCOL_VERSION`** | The game ↔ sidecar wire | `link`, `servuo-plugins`, the bundle | `X-UOLink-Version`, and the installer's pairing check | +| **`MODULE_API_VERSION`** | The website ↔ module contract | `website`, and every module's `coreApi` | At module load, before the module's code runs | + +A module that never talks to a game server has no protocol version at all. See [The module +manifest](/docs/modules/the-module-manifest/#coreapi-and-what-a-range-means). + +## When a contract owes a bump + +The rule this project settled on: **a contract owes a bump only once it has landed on +`main`.** + +While a version has only ever existed on a development branch, additions join it in place +rather than forcing a new number. Once it has shipped, it is somebody else's dependency and +a change to it is a change to a published contract. + +## If you are building a bridge for another game + +You do not inherit this protocol — you define your own between your plugin and your sidecar. +What is worth inheriting is the **shape**: + +- Declare the version on both sides, in files a release can read. +- Make a released pair carry its own compatibility claim, so a deployment tool can refuse a + bad combination rather than discovering it at runtime. +- Reject a mismatch **loudly and early**. A `409` is a good outcome; a successful parse of a + message you did not expect is not. + +## Canonical documents + +[`link/v4.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/link/v4.md) +is the protocol-4 record, including its cross-repository obligations; +[`link/PLAN.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/link/PLAN.md) +§7 is the wire protocol, and +[`link/INTEGRATION.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/link/INTEGRATION.md) +the integration guide. diff --git a/src/content/docs/docs/architecture/system-architecture.mdx b/src/content/docs/docs/architecture/system-architecture.mdx new file mode 100644 index 0000000..0d5ca86 --- /dev/null +++ b/src/content/docs/docs/architecture/system-architecture.mdx @@ -0,0 +1,141 @@ +--- +title: System architecture +description: The whole platform in one place — what each repository is, what talks to what, and the invariants that hold across all of them. +--- + +import { Aside } from '@astrojs/starlight/components'; + +The drawn version of this, for evaluators, is on +[Architecture](/architecture/). This page is the detailed account. + +## Ten repositories, deployed independently + +Nothing here is a monorepo. Each repository has its own history, its own CI and its own +release cadence; what binds them is a set of **versioned contracts**, not a build. + +| Repository | What it is | +|---|---| +| `website` | The Node/Express + MariaDB + React site. The only internet-facing web app | +| `Module-uo` | All the *Ultima Online* code, installed into the site as a module | +| `link` | The **uo-link sidecar**, in Rust — the only network-facing bridge component | +| `servuo-plugins` | The in-game plugin, C#, that feeds the sidecar | +| `installer` | Deploys the shard side: sidecar plus plugin overlay | +| `Android-app` | Native Android client of the website API | +| `Integration-kit` | The instruction book for putting a different game on the platform | +| `docs` | Canonical design docs and the protocol spec | +| `runicgateway.com` | This site | +| `.profile` | The organisation landing page | + +## The layers + +``` + Browser (React SPA) Native Android app + │ cookie │ bearer + └──────────┬─────────────────┘ + ▼ + ┌────────────────────────┐ + │ website (Node) │ + │ middleware → router │ + │ → controller → model │ + │ → db │ + └───────┬────────────┬───┘ + │ │ loads at boot + ▼ ▼ + MariaDB modules// ← installed, never built + │ + ▼ + the game, via whatever + bridge that module owns +``` + +The backend is strictly layered — `middleware → router → controller → model → db` — with +models in `.model.js` (logic) and `.db.js` (SQL) pairs, and **raw parameterised queries with +no ORM anywhere**. + +## Core is game-agnostic + +Since the module system shipped on **2026-08-12**, nothing in core knows about any +particular game. Routes, tables, pages, navigation and push streams for a game arrive from +[a module](/docs/modules/the-module-system/) the operator installed. Core provides the seams; +the module fills them. + +That is why the architecture below describes `module-uo` as *the worked example* rather than +as part of the platform. It is the module every other module is measured against, not a +component core depends on. + +## The invariants + +These hold across repository boundaries, and every one of them is load-bearing. + +### The game is never network-reachable + +The ServUO shard **dials out** over loopback TCP `127.0.0.1:7788`, newline-delimited JSON, +to the sidecar. The sidecar is the listener; the game opens no port. Only the sidecar is +exposed, and only the website's backend talks to it. + +See [The bridge](/docs/architecture/the-bridge/). + +### A wedged sidecar can never stall the game + +On the C# side, `Emit()` enqueues onto a **bounded, drop-oldest** queue and returns +immediately. It never touches the socket from the game's core thread. Every world read +happens on the core thread; a dedicated writer thread drains the queue. + +Dropping game events is strictly better than pausing the game to deliver them. + +### The website degrades rather than fails + +The sidecar REST client never throws — every call returns `{ ok, data, status }`. The public +site still renders with the shard shown offline. + +That guarantee covers **reading the configuration too**: resolving the admin-managed config +decrypts a stored token, which throws if the ciphertext cannot be authenticated (a rotated +`SECRET_ENC_KEY`, or a database dump restored under a different key). That is caught inside +the client and reported as unavailable, so a wrong key degrades the shard surface instead of +500-ing it — and the admin config screen keeps working, which is the screen you need in order +to recover. + +### Sensitive events never reach the public + +Ingested events fan out over two SSE channels: a **public allowlist** stream, and an +**admin-only** stream that additionally carries staff audit, cheat detection and login +attempts with IPs. + +**The catalog is the module's; the boundary is core's.** A module declares which of its +kinds are public-safe, and core enforces the split. A sensitive kind cannot reach the public +channel. + +### A failed module never takes the site down + +The loader catches failures across a module's entire lifecycle and marks it +`startup_failed`. The site comes up with that module's routes and navigation absent, and the +admin panel says why. See [Module +lifecycle](/docs/modules/module-lifecycle/#failure-is-contained-by-construction). + +### Secrets are encrypted at rest + +OAuth client secrets, the sidecar token and the Gmail refresh token are AES-256-GCM +encrypted, keyed by `SECRET_ENC_KEY`. **The sidecar token is write-only in the API** — it is +never returned to any client. + + + +## A deploy is two independent installs + +Worth stating plainly, because it is the single most common misunderstanding: **the +installer binary sets up the shard side only, and never contacts the website.** The website +is a separate Docker deployment on, usually, a different machine. + +The [installation path](/docs/getting-started/requirements/) walks both in order. + +## Canonical documents + +[`ARCHITECTURE.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/ARCHITECTURE.md) +holds the canonical diagram, and +[`BACKEND_DESIGN.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/BACKEND_DESIGN.md) +is the full API, schema and security contract. See [Canonical +documents](/docs/reference/canonical-documents/) for the whole map. diff --git a/src/content/docs/docs/architecture/teams-architecture.mdx b/src/content/docs/docs/architecture/teams-architecture.mdx new file mode 100644 index 0000000..2ea685e --- /dev/null +++ b/src/content/docs/docs/architecture/teams-architecture.mdx @@ -0,0 +1,153 @@ +--- +title: Teams architecture +description: Teams is a contract, not a surface — how core owns guilds, clans and corporations without ever learning what one is called. +--- + +import { Aside } from '@astrojs/starlight/components'; + +Most games have groups: guilds, clans, corporations, tribes, crews. Runic Gateway supports +them as a **core platform primitive**, while core itself never learns what yours is called. + +The administrator's view is [Teams](/docs/administration/teams/). + +## The sentence the design turns on + +**Teams is a contract, not a surface.** + +Core owns the tables, the sync, the access rules and the activity feed. It does **not** own +the word for a Team, and therefore does not own the Team *page*. The module that owns the +vocabulary owns the page. + +That was not the first design. Core originally rendered Team pages with slots a module +filled. It was inverted, and the inversion is the interesting part: instead of core naming +places for a module's content, **a module declares a place on its own page for core to +fill** — `registry.declareModuleSlot(id, name, { core })`, with core offering contributions +rather than naming slots. + + + +## Six invariants + +Each has a test named against it. + +1. **Module unavailability is staleness, never emptiness.** No Team subsystem may apply a + destructive result derived from a failed, timed-out or unanswered module call. +2. **Four authority paths stay four.** Game membership, leadership, forum access and + external-platform access are separate tables answering separate questions, resolved by + separate predicates. **No predicate reads another's table.** +3. **Non-contamination.** A manual forum grant never writes the membership projection, in + either direction, ever. Both facts coexist; neither migrates into the other. +4. **A Team's name is immutable for the life of its record.** A rename is an archive plus a + create. +5. **Core never interprets module vocabulary.** Activity kinds, Team metadata and capability + strings are opaque. Core stores, gates and displays; it never branches on content it does + not own. +6. **The game never touches the website.** Everything crosses the sidecar. + +Invariant 1 deserves emphasis, because it is the one a naive implementation gets wrong: if +the module fails to answer "who is in this Team?", the answer is **not** "nobody". Treating +a timeout as an empty roster would silently disband every Team on the site. + +## The rename rule + +Core's key is **(`module_id`, `external_id`, `name`) taken together** — not `external_id` +alone. + +| Situation | What core does | +|---|---| +| New `external_id` | Create a Team | +| Known id, same name | Update in place | +| Known id, **different name** | **Archive** the row and create a new one | +| Id absent from an authoritative full list | Archive as disbanded, subject to invariant 1 | + +The archived Team keeps its forum, activity history, grants and integration record; all +become read-only. It stays reachable at its old slug, `noindex`, with a banner linking to +the successor — so a Discord message from before the rename lands somewhere that explains +itself instead of 404-ing. + +This puts the whole of *"is this a rename or a different group?"* **inside the module**. If +your game has no persistent group id, synthesise `external_id` from whatever is stable, or +fold the name into it so every rename is a fresh id. Core only ever sees "an id appeared / +an id's name changed / an id is gone". + +## The module-facing interface + +A module registers a provider: + +```js +api.registerTeamProvider({ getTeams, getTeamMembers, getTeamLeaders }) +``` + +and pushes through `ctx.teams`: + +| Call | What it does | +|---|---| +| `ctx.teams.publish(event)` | An optimisation — makes a membership change visible at once | +| `ctx.teams.reconcile({ reason })` | A debounced *request*; returns immediately | +| `ctx.teams.activity.push(items)` | Writes the per-Team feed | + +**`ctx.teams` is push-only, and that is the contract.** There is no reader. A module +*answers* questions about Teams; it does not ask them. A `getTeamRoster` would be core +offering to read back the module's own answer — which the module already holds. + +All three are fire-and-forget and never reject, because they are called from inside +game-event handlers and a storage problem of core's must not become the module's control +flow. Correctness comes from reconciliation either way. + +### The six event kinds + +`team.created` · `team.disbanded` · `team.member.added` · `team.member.removed` · +`team.leader.added` · `team.leader.removed` + +Six rather than four because **leadership is its own authority path**: a leadership change +has to be expressible without pretending someone joined or left. + +**`team.created` and `team.disbanded` only ask for a reconciliation.** Core will not invent +a Team from a delta — it would have no name, no roster and no leaders — and will not archive +one from a delta either, because an archive driven by a message that may simply have been +repeated is destruction on no evidence. + +### The activity feed + +Each item carries an already-**rendered** `summary`, which core stores verbatim. Core cannot +phrase "gained 15,000 gold" for a game whose vocabulary it does not know, and a core that +templated it would have re-acquired exactly the semantics the module system exists to +remove. + +`visibility` defaults to `'members'` — **fail closed**. The module chooses it per item; core +enforces it on read. + +A `dedupeKey` collision is a **successful no-op**, which is what makes a sidecar reconnect +backfill safe to replay. + +## Untrusted game data becomes a public page + +This is the sharpest edge in the whole subsystem: a group name chosen by a player becomes a +page on a public website. + +So game-sourced names go through **reserved-name screening**, and game-sourced overrides +through an **approval gate**. Neither is optional, and neither is something a module can +waive. + +## What is deliberately out of scope + +Multi-module namespacing, Team hierarchies and alliances, cross-Team messaging, and +platform-only Teams with no game backing. + +**Matrix is research, not a roadmap item.** Of the five capabilities a shared interface +would name, a Matrix implementation could honestly provide two — it has no +channel-with-overwrites, no role object, no voice channel, and no slash-command +registration. The settled outcome was a *capability contract*, not an integration. + +## Canonical document + +[`TEAMS.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/TEAMS.md) +is normative — Part 1 for the invariants, Part 2 for the core, Parts 3–4 for pages and the +activity feed. diff --git a/src/content/docs/docs/architecture/the-bridge.mdx b/src/content/docs/docs/architecture/the-bridge.mdx new file mode 100644 index 0000000..4d8773d --- /dev/null +++ b/src/content/docs/docs/architecture/the-bridge.mdx @@ -0,0 +1,131 @@ +--- +title: The bridge +description: How a game server reaches the website without ever being reachable itself — the sidecar, the loopback socket, and the rules that keep the game running. +--- + +import { Aside } from '@astrojs/starlight/components'; + +The bridge exists to answer one question safely: **how does a private game server's live +state reach a public website?** + +The answer is a **sidecar** — a small service that owns the connection to the game and the +durable copy of what the game said. It is not optional, and the reasons are worth +understanding before you build one for another game. + +## The shape + +``` + ServUO shard ──dials out──▶ uo-link sidecar ──HTTP + WS──▶ website + (C# plugin) 127.0.0.1:7788 (Rust) bearer + version (module) + newline JSON + ▲ │ + └──────── the game opens NO port ──────┘ +``` + +Three properties fall out of that diagram, and each is a rule rather than an +implementation detail. + +## 1. The game dials out + +**The sidecar is the listener. The game connects to it.** The shard opens no port at all, +and nothing on the internet can reach it even in principle. + +This inverts the intuitive design — you would expect the thing with the data to serve it — +and the inversion is the whole security argument. Only the sidecar is exposed, and only the +website's backend talks to the sidecar. + +The transport is deliberately boring: **newline-delimited JSON, one object per line**, over +loopback TCP. + +## 2. A wedged sidecar must never stall the game + +This is the constraint the plugin is built around. + +On the C# side, `Emit()` **enqueues onto a bounded, drop-oldest queue and returns +immediately**. It never touches the socket from the game's core thread. Every world read +happens on the core thread; a dedicated writer thread drains the queue. + + + +Inbound commands get the mirror rule: **every inbound handler marshals to the core thread +before touching world state.** + +## 3. The sidecar persists before it forwards + +The sidecar owns a durable store. It is not a proxy that translates and forgets — if the +website is down, the game's events are still recorded, and a reconnecting website catches +up. + +This is what "a *thin* sidecar" means in the Integration Kit: thin in *logic*, not thin in +responsibility. The sidecar is a **dumb forwarder** — it makes no access-control decisions +and holds no policy. Access control and the admin-toggleable visibility scope live on the +**website**, where an administrator can see and change them. + +## Two ways in + +**Live events** arrive over an outbound **WebSocket** and are routed by the module's ingest +dispatcher. Kinds are handled differently by nature: state-changing kinds update tables, +notable kinds append to an events log, and high-frequency kinds only update state rather +than accumulating history. + +**Point-in-time reads and commands** go over **REST**, through a client that never throws. + +Every call carries `Authorization: Bearer ` and an `X-UOLink-Version` header. **A +protocol mismatch fails fast with `409`** rather than being mis-parsed — see [Protocol +versions](/docs/architecture/protocol-versions/). + +## What the shard can say + +The catalog spans sessions and identity, character state, economy and commerce, housing and +IDOC, combat and PvP, progression, cheat detection and staff audit, and server lifecycle. +A representative line looks like: + +```json +{"t":1752,"kind":"vendor.sale", + "buyer":{"serial":"0x1A2B","acct":"PerryAdimn"}, + "owner":{"serial":"0x33C1","acct":"Feng"}, + "item":{"serial":"0x4001A2","type":"Longsword","amount":1}, + "price":75000,"commission":3750} +``` + +The full catalog is [Event catalog](/docs/reference/event-catalog/). + +## Two design details worth stealing + +**`server.hello` is per-connection, not per-boot.** The sidecar restarts independently of +the game, so anything it needs up front must be re-sent on **every** connect. An earlier +draft emitted a "started" event once at boot; a sidecar that came up second never received +it and had no idea which shard it was attached to. + +It carries a `bootId` — a GUID generated at server start, stable across sidecar reconnects +and changed on every game restart. That is how the sidecar tells *"I reconnected"* (keep +cached state) from *"the game restarted"* (discard it). + +**Rosters are sets, not signatures.** Guild membership is compared as a set rather than +folded into a checksum, because a sum can collide: one member joining and another leaving +between two sweeps offset each other, and the guild reads as unchanged. A set can also be +*differenced*, which is what makes per-member leave events possible for a game that raises +no event for leaving. + +On a guild's **first** sweep there is no prior set, so nothing is reported as leaving — an +unknown roster becoming known is not 155 people leaving at once. + +## Building one for another game + +The bridge is not UO-specific in shape, only in vocabulary. Chapters 3 and 4 of [the +Integration Kit](/docs/modules/the-integration-kit/) cover the sidecar and the game-side +plugin, and they are the two parts where the mistakes are most expensive. + +## Canonical documents + +[`link/PLAN.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/link/PLAN.md) +§5 and §7 are the data catalog and the wire protocol; +[`link/INTEGRATION.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/link/INTEGRATION.md) +is the integration guide. Both are normative; this page is not. diff --git a/src/content/docs/docs/modules/building-a-module.mdx b/src/content/docs/docs/modules/building-a-module.mdx new file mode 100644 index 0000000..530243c --- /dev/null +++ b/src/content/docs/docs/modules/building-a-module.mdx @@ -0,0 +1,161 @@ +--- +title: Building a module +description: The repository layout, the server half, and the client build — including the three things about bundling that everyone gets wrong once. +--- + +import { Aside } from '@astrojs/starlight/components'; + +Start from [the Integration Kit's template](/docs/modules/the-integration-kit/) rather than +an empty directory. This page explains what the template is doing and why, so that when you +change something you know what you are changing. + +## The layout + +One repository, both halves, versioned together: + +``` +module.json id, version, coreApi, mounts, extensions +server/index.js the entry point — exports register(ctx, api) +server/db/schema.sql idempotent fragment, replayed every boot +server/db/purge.sql destructive; only ever run by an explicit purge +server/router/ routers and controllers +server/model/ *.model.js (logic) + *.db.js (SQL) pairs +client/src/entry.jsx registers routes, nav, providers +client/src/shim/ the shared-dependency shims — see below +client/dist/entry.js PREBUILT chunk, published by your CI +``` + +`client/dist/` is committed by your **release**, not by hand — the operator never builds, +so the built chunk has to be in the bundle. + +## The server half + +`server/index.js` exports one function, called once during core's require phase: + +```js +module.exports = function register(ctx, api) { + const log = ctx.log('examplegame') + + api.registerRoutes({ + public: { '/world': worldRouter(ctx) }, + }) + + api.onBoot(async (ctx) => { + // anything that needs a live database goes HERE, not above + }) +} +``` + +Follow core's own layering — `router → controller → model → db`, with `.model.js` (logic) +and `.db.js` (SQL) pairs, and raw parameterised queries. There is no ORM anywhere in this +project, and a module that introduces one is a module nobody else can read. + +### The rule CI enforces + +**Zero `require`/`import` may reach outside your own directory.** Not "few". Zero. + +```bash +npm run check:imports --prefix server +``` + +If you need something from core that `ctx` does not offer, that is a gap in the contract — +raise it, so the surface grows deliberately. Reaching into core's internals is how a module +breaks on a refactor it had no part in. + +## The client half + +Your chunk is built with Vite in **library mode**, emitting one unhashed `dist/entry.js`. +Unhashed deliberately: `module.json` names that file, and a hashed name would have to be +discovered at runtime. Core answers the caching question instead, serving it `no-cache`. + +Then three things about the bundling, each of which has already cost somebody a day. + +### 1. Aliases replace `external` — they do not accompany it + +This is the one that looks most like it should work. + +Rollup asks `external` **before** Vite's alias resolver runs, so a specifier listed there is +marked external and **never aliased**. The chunk then ships bare `import 'react'` +specifiers, which a browser cannot resolve without an import map — and an import map has to +be inline, which `script-src 'self'` forbids. + +The first real module shipped with both, **built cleanly**, and emitted exactly that chunk. + +```js +rollupOptions: { external: [] }, // deliberately empty +``` + +Alias only. Nothing in `external`. (`output.globals` does not rescue this either — it covers +iife/umd and does nothing for an ES module.) + +### 2. Use the array form of `resolve.alias`, with anchored regexes + +Vite's **object** form does *prefix* matching, so a `react` key also rewrites +`react/jsx-runtime` — silently, to the wrong shim. The chunk then fails at its first element +with a message about `jsx` not being a function, which points nowhere near the cause. + +```js +alias: SHARED.map(({ specifier, shim }) => ({ + find: new RegExp(`^${escape(specifier)}$`), + replacement: shim, +})) +``` + +`^react$` and `^react/jsx-runtime$` cannot collide. + +### 3. Assert at resolution time, not by grepping the output + +The risk is a missed alias welding a **second React** into your chunk. That loads fine and +then throws about an invalid hook call somewhere unrelated. + +The template fails the build if any shared package resolves into `node_modules`. Two details +of how it does that are not interchangeable: + +- It hooks **`transform`, not `load`**. `load` is first-wins, so an earlier plugin returning + the module's contents means the guard is never called. Written against `load`, it sat in + the build doing nothing while a deliberately-broken alias produced a green build with + react-router welded in. +- The list of packages that may not be bundled is stated **independently** of the alias + list. Deriving one from the other means deleting an alias also deletes the guard against + what that alias prevented. + + + +Verify with: + +```bash +npm run build --prefix client # build BEFORE the tests — two of them read the chunk +npm run check:externals --prefix client +``` + +## Registering the client half + +```js +const { registry } = window.__rg + +registry.registerRoutes(ID, { + public: [{ path: 'world', element: }], + admin: [{ path: 'link', element: }], +}) +registry.registerNav(ID, { … }) +``` + +Paths are **relative to your module's segment** — `path: 'link'` under `admin` becomes +`/admin//link`. Check `window.__rg.version` against your `coreApi` range and refuse to +register on a mismatch. + +## Then + +[Testing and release](/docs/modules/testing-and-release/) covers CI, the checks, and +publishing the bundle and its manifest. diff --git a/src/content/docs/docs/modules/installing-modules.mdx b/src/content/docs/docs/modules/installing-modules.mdx new file mode 100644 index 0000000..1812508 --- /dev/null +++ b/src/content/docs/docs/modules/installing-modules.mdx @@ -0,0 +1,115 @@ +--- +title: Installing modules +description: How a module reaches a deployment — the install manifest, the two surfaces that can install one, and which of them wins. +--- + +import { Aside } from '@astrojs/starlight/components'; + +There is no catalog, and there is no marketplace. A module is installed by **naming the +URL of a release's install manifest**. + +That is a design decision rather than an unfinished feature: a catalog would make core's +release cadence decide which modules exist, and the whole point of the module system is +that it does not. + + + +## What a release publishes + +Two artifacts: + +- **`-.tar.gz`** — the bundle: `module.json`, the server half, the prebuilt + client chunk, and the SQL fragments. +- **An install manifest** — small JSON carrying the bundle's URL and its **`sha256`**. + +The manifest URL is the thing an operator pastes. The bundle is downloaded, **verified +against the `sha256`**, and unpacked into `modules//` on the mounted volume. + +Nothing is compiled at any point in that sequence. + +## The two surfaces + +Both write the same `installed_modules` row, and neither needs a build step. + +### The admin panel + +Paste the manifest URL, press Install, then **restart** — a button on the same screen, not +an instruction to go and restart the container. It runs the lifecycle shutdown and exits, +and the supervisor declared in the shipped Compose file brings the process back. + +That is why `restart: unless-stopped` is called out as load-bearing on [Install the +site](/docs/getting-started/install-the-site/). Without a supervisor, that button takes the +site down and leaves it down. + +### The `MODULES` environment variable + +For hosts managed by Compose rather than by clicking. Each entry is: + +``` +@= +``` + +Resolution runs **inside the server process**, before the volume is scanned — which is what +lets it write the same provenance columns a panel install writes. A module already unpacked +at the declared version is a no-op that makes **no network call at all**. + +### By hand + +`./modules` is a bind mount, deliberately rather than a named volume, so placing a module +directory there yourself is a **supported install**. A named volume would have routed that +through `docker cp`. + +The image's own copy of `modules/` is excluded by `.dockerignore`, so a module sitting in a +builder's working tree can never ship inside an image. + +## Which surface wins + +They govern different things, and the split is worth memorising: + +- **The declaration owns what is on the volume.** +- **The row owns whether a module runs.** + +So uninstalling a declared module from the admin panel **returns its files at the next +start and leaves it disabled**. The files come back because `MODULES` still declares them; +it stays off because the row says so. That is the intended outcome, not a bug — but it +surprises people who expect the panel to be the last word. + +## Upgrades + +Paste the new release's manifest URL and install over the top. The bundle is verified, +unpacked over the old one, and takes effect at the restart. + +An upgrade **deliberately leaves the state alone** — upgrading an enabled module must not +silently switch it off, and re-installing a disabled one must not silently switch it on. + + + +## Removal + +Covered in full on [Managing modules](/docs/administration/managing-modules/); the shape +matters here because it constrains what you ship. + +**Uninstall** is non-destructive: the row goes to `disabled`, the directory is removed, and +the module's **tables and data are retained**. + +**Purge** is separate, explicit, and destructive — it runs your `purge.sql`. It is offered +in two places, and both are while the file is still on disk: as a standalone action on an +installed module, and as an opt-in checkbox in the uninstall dialog. + +That second placement exists because of a real ordering trap: **`purge.sql` lives inside the +directory uninstall deletes**, so "purge afterwards" was never actually possible — it would +have left a disabled row whose Purge button had nothing to run. + +The consequence, accepted and stated: an operator who uninstalls without ticking the box +keeps the tables, and getting rid of them later means reinstalling the module first. Write +`purge.sql` on the assumption it may be run long after anyone remembers what it drops. diff --git a/src/content/docs/docs/modules/module-lifecycle.mdx b/src/content/docs/docs/modules/module-lifecycle.mdx new file mode 100644 index 0000000..ae8c2cf --- /dev/null +++ b/src/content/docs/docs/modules/module-lifecycle.mdx @@ -0,0 +1,103 @@ +--- +title: Module lifecycle +description: What core does to your module on boot, in what order, and what happens when any step of it throws. +--- + +import { Aside } from '@astrojs/starlight/components'; + +The five states are on [Managing modules](/docs/administration/managing-modules/), from the +operator's side. This is the same machine from inside the module — what core calls, when, +and what it does with a throw. + +## The scan + +The loader reads `modules/*/module.json` from the filesystem **synchronously, at require +time**. The database is not consulted: what is on the volume determines what mounts. + +`MODULES_DIR` defaults to `/modules`, and Compose sets it to `/app/modules`. **A +missing modules directory is not an error** — "no modules installed" is the normal state of +bare core, and the loader must not make the mount mandatory to boot. + +Modules load **alphabetically by `id`**, deterministically. There is no dependency +resolution between modules, and alphabetical order is the honest way of saying so: any +other order would imply a precedence nobody is computing. Do not build a module that needs +to load before or after another one. + +## Validation, in order + +Each step runs against your module. A failure at any step is **your module's failure and +nobody else's**. + +1. `module.json` parses, has no unknown keys, and its `id` matches the directory name. +2. `coreApi` is satisfied by core's `MODULE_API_VERSION`. +3. Declared `mounts` prefixes are well-formed and collide with nothing. +4. Declared `extensions` slots all exist. +5. `schema` and `purge` files exist and are readable, and their table names are namespaced + or allowlisted. +6. `require()` of your server entry succeeds and exports a function. +7. `register(ctx, api)` returns without throwing, **and registers exactly what + `module.json` declared**. + +Step 7 is worth reading twice. The manifest is not documentation of what you register — it +is a claim core holds you to. Registering something you did not declare fails, and so does +declaring something you do not register. + + + +## Then the module runs + +For each module that passed: + +1. **Schema replay** — your `schema.sql` fragment is applied. It must be idempotent; it runs + on every boot. +2. **Routes and registrations** mount. +3. **`onBoot(ctx)`** is called, if you export one. This is where long-lived work belongs: + opening a stream, starting a poller, connecting to something. + +On shutdown, **`onShutdown()`** is called. Disabling a module from the panel dispatches it +too, so the module actually stops — releases its sockets, closes its streams — rather than +merely becoming unreachable. + +Enabling is deliberately **not** the mirror image: there is no `onBoot` re-dispatch, so the +panel offers a restart instead. If your `onBoot` is expensive or stateful, that asymmetry is +in your favour. + +## Failure is contained, by construction + +**A module that fails to load never takes the site down.** + +The loader try/catches the module's **entire** lifecycle — require, validation, registration, +schema replay, `onBoot` — not merely failures that surface after a router object was +returned. Any failure at any point marks that module `startup_failed`, records the reason, +and the site comes up with that module's routes and navigation absent. + +Two consequences to design around: + +- **A failed module is retried on every restart.** There is no backoff and no quarantine. + A deterministically broken module re-records its failure each boot, which is the honest + thing for it to do. +- **`disabled` is the only state a boot leaves alone.** Every other non-disabled module is + reset to `enabled` at boot and then recorded as `started` or `startup_failed`. Disabling + is an operator's decision rather than an outcome, so it survives restarts untouched. + + + +## Where the loader is specified + +[`MODULE_API.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_API.md) +Part 4 is the normative account of everything on this page, including the exact position of +the `load()` call in `app.js` and why it is load-bearing in both directions. diff --git a/src/content/docs/docs/modules/testing-and-release.mdx b/src/content/docs/docs/modules/testing-and-release.mdx new file mode 100644 index 0000000..b522fed --- /dev/null +++ b/src/content/docs/docs/modules/testing-and-release.mdx @@ -0,0 +1,104 @@ +--- +title: Testing and release +description: The checks a module should run before it ships, what a release artifact actually is, and how the version that ships gets decided. +--- + +import { Aside } from '@astrojs/starlight/components'; + +## The checks + +Four, and each exists because something got past review without it. + +```bash +npm run check:imports --prefix server # zero imports leave the module directory +npm run build --prefix client # build FIRST — two tests read the chunk +npm run check:externals --prefix client # no shared dependency welded into the chunk +npm test --prefix server && npm test --prefix client +``` + +**`check:imports`** enforces [the zero-internal-imports +rule](/docs/modules/building-a-module/#the-rule-ci-enforces). It is the mechanical form of +the module boundary — without it, the boundary is a convention, and conventions lose. + +**`check:externals`** is the one that catches a chunk shipping bare `import 'react'` +specifiers, or a second React welded in. Both build cleanly. Neither works in a browser. + + + +Also worth running your OpenAPI fragment check if you publish one — the filename is fixed +at `swagger-fragment.json` in the bundle root, so a module cannot point core at some other +file. + +## What a release artifact is + +**Not source.** An operator never builds anything, and that constraint shapes everything +here. + +A release is **the directory core's loader expects to find at `modules//`, already +assembled** — the prebuilt client chunk, any runtime dependency installed, the schema +fragment, the OpenAPI fragment — packed exactly as it will be unpacked. + +Two artifacts ship: + +- `-.tar.gz` +- an **install manifest** carrying that tarball's URL and its `sha256` + +The admin install downloads the tarball, verifies the hash, and unpacks it. **Nothing runs +`npm` on the way.** + +## The version that ships is the tag + +The template derives the next version from conventional-commit subjects since the newest +`v*` tag: + +| Commits since the last tag | Result | +|---|---| +| `feat!:` or `BREAKING CHANGE` | major | +| `feat:` | minor | +| `fix:` / `perf:` | patch | +| Nothing releasable | **no release is cut** | +| First ever run, no tag | releases what `module.json` declares | + +Your committed `module.json` version is a **floor and a starting point, not a record of the +last release**. Name a version there above the newest tag and that version is what releases +— which is still the natural way to say "this one is a minor" when a `coreApi` bump forces +the question. + + + +## Pin the core you build against + +Keep a `ci/core-ref.json` naming the exact core commit your module is written against, and +have CI assert your declared `coreApi` still holds against that core's +`MODULE_API_VERSION`. + +Moving that sha is the moment someone re-reads what changed. It is the same mechanism [the +Integration Kit uses](/docs/modules/the-integration-kit/#the-pin-that-forces-a-re-read), and +the reason a contract bump upstream becomes a visible decision in your repository rather +than a silent one. + +## Before you tag + +A short list, all of it learned rather than invented: + +- **The module boots on a real deployment**, not just in tests. [Failure is + contained](/docs/modules/module-lifecycle/#failure-is-contained-by-construction), so a + broken module is easy to not notice — the site comes up and one section is missing. +- **`schema.sql` is genuinely idempotent.** It runs on every boot, not once. +- **`purge.sql` still makes sense to someone who has forgotten your module**, because + [that is who will run it](/docs/modules/installing-modules/#removal). +- **Your `coreApi` range covers the oldest core you actually test against**, not just the + newest one you have. +- **Every capability string you publish is one you intend to keep.** Something outside your + repository is branching on them. diff --git a/src/content/docs/docs/modules/the-integration-kit.mdx b/src/content/docs/docs/modules/the-integration-kit.mdx new file mode 100644 index 0000000..fbde318 --- /dev/null +++ b/src/content/docs/docs/modules/the-integration-kit.mdx @@ -0,0 +1,107 @@ +--- +title: The Integration Kit +description: The instruction book for putting a different game on the platform — four chapters, a buildable template, and an honest account of its status. +--- + +import { Aside } from '@astrojs/starlight/components'; + +The [Integration +Kit](https://gitea.whitlocktech.com/RunicGateway/Integration-kit) is a separate repository +whose entire job is teaching someone **outside this project** how to put a different game on +the platform. + + + +## What it covers + +Three things, because the reasons live in the joins between them: + +``` +your game server ──dials out──▶ your sidecar ──HTTP + WS──▶ website core +(plugin: bounded queue, (owns the socket, (loads your module, + writer thread) persists, then forwards) serves the pages) +``` + +| Part | What it is | +|---|---| +| **The website module** | A bundle core loads at boot. The bulk of the work, and the only part every module needs | +| **The sidecar** | A small service owning the connection to your game server, and the durable copy of what the game said. **Not optional** | +| **The game-side plugin** | Whatever runs inside your game and feeds the sidecar, without ever letting the sidecar stall the game | + +## The four chapters + +| # | Chapter | What it covers | +|---|---|---| +| 1 | Your first module in twenty minutes | Copy the template, rename it, build it, install it, see a page. No theory | +| 2 | The website module | `module.json`, `register(ctx, api)`, the schema fragment, the client chunk, packaging, and what a module must never do | +| 3 | The sidecar | Why the website never talks to a game server, what "persist before you forward" means, and what a *thin* sidecar is | +| 4 | The game-side plugin | The least code and the highest stakes: never block the game thread | + +Before any of them, the kit points at the [Rust dry +run](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/modules/rust-dryrun.md) +— a complete module designed on paper for a second game, and the shortest honest picture of +the whole job. + +## The template is built, not just quoted + +Chapters 1 and 2 quote `template/`, a real module that CI builds against a pinned core. The +code in those chapters is **a tree that is proved rather than prose that looks like one**. + +Chapters 3 and 4 cite `uo-link` and `servuo-plugins` by file and identifier rather than by +line number, deliberately: those repositories move for their own reasons, and a line number +in a book is wrong the moment they do. + +## The kit never re-specifies a contract + +This is its governing rule, and it is the same one this site follows. + +> Nothing in these chapters is normative. Where a chapter and one of these documents +> disagree, the document is right and the chapter has a bug. + +| Authority | For | +|---|---| +| `MODULE_API.md` | Everything a module may do | +| `MODULE_SYSTEM.md` | Why the module system is shaped this way, and how a module is installed and removed | +| `link/PLAN.md` + `INTEGRATION.md` | The game ↔ sidecar wire protocol, as one real sidecar implements it | + +The chapters teach the order to do things in, the reasoning, and **the mistakes that cost +this project time**. + +## The pin that forces a re-read + +`ci/core-ref.json` pins the exact core commit the kit is written against, and CI asserts +that the version `template/module.json` declares **equals** that core's +`MODULE_API_VERSION`. + +Equality, not "satisfies". That is the mechanism, not a bug: a contract bump in the website +repository is *meant* to turn the kit red, so that someone re-reads the chapters before the +pin moves. + + + +## Running its checks + +Dependency-free Node scripts, from the repository root — which is also how a reader runs +them: + +```bash +node scripts/checkLinks.js # every relative link resolves; no commit permalinks +node scripts/checkRenameSites.js # the rename checklist matches the template tree +node scripts/checkChapterPaths.js # every path a chapter names in backticks still exists +``` diff --git a/src/content/docs/docs/modules/the-module-api.mdx b/src/content/docs/docs/modules/the-module-api.mdx new file mode 100644 index 0000000..25da118 --- /dev/null +++ b/src/content/docs/docs/modules/the-module-api.mdx @@ -0,0 +1,171 @@ +--- +title: The module API +description: The two arguments core hands your module — what you can reach, what you can register, and the rules that govern both. +--- + +import { Aside } from '@astrojs/starlight/components'; + +Your server entry point exports one function: + +```js +module.exports = function register(ctx, api) { /* … */ } +``` + +`ctx` is what core lends you. `api` is what you register with it. Everything crossing the +module boundary goes through one of the two. + +The contract version is **`MODULE_API_VERSION`**, currently **1.6.0**, and your manifest's +[`coreApi` range](/docs/modules/the-module-manifest/#coreapi-and-what-a-range-means) is +checked against it before your code is required. + +## The entry point runs early + +`register()` is called **once, synchronously, during core's require phase — not after the +database is up.** + +It must not `await`, must not touch the database, and must not throw for a reason a retry +would fix. Everything needing a live database belongs in `onBoot`. + + + +## `ctx` — what you can reach + +Every member exists because a real module needed it. The surface is grown from demonstrated +need, never speculation. + +| Member | What it gives you | +|---|---| +| `ctx.express`, `ctx.validator` | Core's own `express` and `express-validator` namespaces | +| `ctx.db.query`, `ctx.db.pool` | Parameterised SQL, and the pool for streaming work | +| `ctx.log(namespace)` | `error` / `warn` / `info` / `debug`, each `(msg, meta?)` | +| `ctx.settings` | `get`, `set`, `getInstanceName` | +| `ctx.auth.getUserFromRequest(req)` | `{ id, username, role }` or `null` | +| `ctx.push.publish` | Notification fan-out | +| `ctx.secretBox` | `encrypt` / `decrypt` for secrets at rest | +| `ctx.middleware` | `requireAuth`, `requireRole`, `siteMode`, `validate`, `noindex`, `rateLimit`, `accountChangeLimiter` | +| `ctx.uploads` | `upload`, `UPLOAD_DIR`, `MIME_EXT` | +| `ctx.posts` | `listAll`, `getById`, `linkAnnounceJob`, `markAnnounced` | +| `ctx.paths.moduleRoot` | Absolute path to your own directory | +| `ctx.activity.log` | The admin audit trail | +| `ctx.users.getById` | Read a user | +| `ctx.site.baseUrl` | Absolute base URL, no trailing slash | +| `ctx.moduleId` | Your id, from the manifest | +| `ctx.teams` | `publish`, `reconcile`, `activity.push` — see below | + +`ctx` is frozen one level deep before you get it. That is a guard against accident, not +against a hostile module — the boundary is organisational, [not a security +boundary](/docs/modules/the-module-system/#the-boundary-is-not-a-sandbox). + +### Three narrowings worth knowing + +Core deliberately hands you **less** than the underlying utility exports. + +- **`ctx.auth` is one function.** The full facade can mint sessions; minting is core's job. + A module that needs an identity needs to *read* one. +- **`ctx.settings` is three functions**, not the model's 24 — most of those are registration + and app-links policy that is core's business. +- **`ctx.posts` is four functions.** `create` / `update` / `remove` are the CMS, and the CMS + is not a module's. + + + +### `ctx.teams` is push-only, on purpose + +There is no reader. A module **answers** questions about Teams; it does not ask them. Every +Team table is core-internal, and a `getTeamRoster` would be core offering to read back the +module's own answer — which the module already holds. + +All three members are fire-and-forget and never reject, because they are called from inside +game-event handlers and a storage problem of core's must not become your control flow. + +See [Teams architecture](/docs/architecture/teams-architecture/) for the whole shape. + +## `api` — what you register + +```js +api.registerRoutes({ public: {…}, admin: {…}, player: {…} }) +api.registerExtension(slot, router) +api.registerNotificationStreams(streams) +api.registerAnnounceLeg({ leg, label, dispatch, classify }) +api.registerPostHook({ onSaved, onDeleted }) +api.registerTeamProvider({ getTeams, getTeamMembers, getTeamLeaders }) +api.registerSlashCommands([{ name, description, options, access, handler }]) +api.onBoot(async (ctx) => {}) +api.onShutdown(async () => {}) +``` + +Every call is synchronous, and **calling one twice is an error** rather than a +last-one-wins overwrite. + +### Everything stages; nothing commits until you are known good + +A claim's *shape* is checked at the call, so a malformed one throws with your own stack. +Whether a name is *taken* can only be answered once the whole batch is in, and is checked +when the loader commits. + +The consequence is the one that matters: a module that registers two streams and then +throws **has left nothing behind**. A half-registered catalog would be worse than a missing +one — it is a subscribable stream that nothing will ever publish to. + +### `registerRoutes` and the tier gate + +One `express.Router()` per prefix per tier. The keys must match `module.json`'s `mounts` +exactly, and prefixes are one segment — no nesting, no parameters. + +**The tier gate is already applied.** A router registered under `admin` sits behind +`noindex, isLoggedIn, requireRole('admin','editor','moderator')`; under `player`, behind +`noindex, requireAuth`; under `public`, behind nothing, by design. + +Add per-route gates on top of that. **Never re-implement the tier gate** — a module that +rolls its own is a module whose access rules drift from core's. + +Your router is mounted *inside* the tier, so it structurally cannot reach above its prefix. + +## The client half + +The client contract is its own thing. Core populates a global before it renders, and +freezes it afterwards: + +```js +window.__rg = { + version, // MODULE_API_VERSION — the same number as the server's + react, // the React namespace + reactDom, // react-dom/client + router, // react-router-dom namespace + jsxRuntime, // react/jsx-runtime + registry, // routes, nav, feature providers, slots + ui, // the shared component kit + api, // the request primitive +} +``` + +Your chunk declares `react`, `react-dom` and `react-router-dom` as **externals** resolving +to that global — a global rather than an import map precisely because an import map must be +inline and `script-src 'self'` forbids inline script. + +**`jsxRuntime` is not decoration.** Your bundler compiles every `.jsx` file to imports from +`react/jsx-runtime` under the modern automatic runtime, and those must resolve to *core's* +React like everything else. Without it on the global you would have to build with +`jsxRuntime: 'classic'`; with it, you use the default your tooling already assumes. + +**`version` is there so your entry can check it.** A module entry compares +`window.__rg.version` against its own `coreApi` range and refuses to register on a +mismatch, logging once — the client-side twin of the boot-time check. + +You register routes, navigation and feature providers through `registry`. See [Building a +module](/docs/modules/building-a-module/). + +## The contract itself + +[`MODULE_API.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_API.md) +is normative and complete — Part 2 for the server contract, Part 3 for the client, Part 4 +for the loader's obligations and Part 5 for how they are enforced. This page is a map of +it, not a substitute. diff --git a/src/content/docs/docs/modules/the-module-manifest.mdx b/src/content/docs/docs/modules/the-module-manifest.mdx new file mode 100644 index 0000000..1f67175 --- /dev/null +++ b/src/content/docs/docs/modules/the-module-manifest.mdx @@ -0,0 +1,114 @@ +--- +title: The module manifest +description: Every key in module.json, what the loader does with each, and why a typo is a boot failure rather than an inert setting. +--- + +import { Aside } from '@astrojs/starlight/components'; + +`module.json` sits at the root of your bundle. The loader reads it synchronously, before +anything else about your module runs. + +**Unknown top-level keys are rejected, not ignored.** A misspelled key is a loud failure +rather than a silently-inert setting — which is the right trade when the alternative is a +module that boots and mysteriously does half its job. + +## A complete manifest + +```json +{ + "id": "uo", + "name": "Ultima Online", + "version": "1.0.0", + "coreApi": "^1.0.0", + "server": "server/index.js", + "client": { "entry": "client/dist/entry.js" }, + "schema": "server/db/schema.sql", + "purge": "server/db/purge.sql", + "mounts": { + "public": ["/shard", "/atlas"], + "admin": ["/shard", "/uo-link"], + "player": ["/shard"] + }, + "extensions": ["admin.users.detail"], + "capabilities": ["shard", "atlas", "market"] +} +``` + +## The keys + +| Key | Required | Meaning | +|---|---|---| +| `id` | yes | `^[a-z][a-z0-9-]{1,31}$`. The directory name, the `installed_modules` key, the URL segment, and the client registry key — all at once. **Must equal the directory it was read from.** | +| `name` | yes | Human label for the admin Modules screen | +| `version` | yes | Semver. Recorded on install; shown on failure | +| `coreApi` | yes | Semver **range**, checked against core's `MODULE_API_VERSION` | +| `server` | no | Server entry point, relative to the module root. Absent means a client-only module | +| `client.entry` | no | The prebuilt ESM chunk, **in a subdirectory** — the directory it sits in is what gets served. Absent means a server-only module; present-but-empty is rejected, because it claims a client half and delivers none | +| `schema` | no | Idempotent SQL fragment, replayed every boot | +| `purge` | no | Destructive teardown. **Required if `schema` is present** | +| `mounts` | no | Declared route prefixes per tier | +| `extensions` | no | Core extension slots this module mounts into | +| `capabilities` | no | Opaque strings published to clients for feature detection | + +## `mounts` is a claim, not a description + +The loader compares your declaration against what your module **actually registers**, and +rejects a mismatch in either direction. Declaring a prefix you never mount fails; mounting +one you never declared fails too. + +Prefixes are validated against `^/[a-z0-9][a-z0-9-]*$`, and the keys must match what you +register exactly. + + + +## `capabilities` is for feature detection + +Opaque strings, published by `GET /api/v1/public/modules` — and **only while the module is +`started`**. Clients like the SPA and the Android app read them to decide what to show. + +They are not permissions and not mount prefixes. Keep them stable: something outside your +repository is branching on them. + +## `coreApi` and what a range means + +Core exports a single semver string, currently **1.6.0**. Your range is checked at boot, +before your code is required. + +A **minor** bump adds members without removing any or changing a signature, so `^1.3.0` +keeps resolving against 1.6.0 — which is exactly why `module-uo` still declares `^1.3.0` +and runs fine. + +Use a caret range against the oldest core you actually support and test against. Pinning +exactly buys nothing and strands you on the next additive release. + + + +## Schema and purge + +`schema` runs on **every boot**, so it must be idempotent — `CREATE TABLE IF NOT EXISTS`, +and additive migrations written so a replay is harmless. Table names must be namespaced or +allowlisted; the loader checks. + +`purge` is required whenever `schema` is present, because a module that can create tables +must offer a way to remove them. It is only ever run by an explicit purge — never as part +of an uninstall. + +Remember [where `purge.sql` lives](/docs/modules/installing-modules/#removal): inside the +directory an uninstall deletes. Write it to be run by someone who no longer remembers what +your module created. + +## The full specification + +[`MODULE_API.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_API.md) +§2.1 is normative for the manifest, and §2.6 for the schema fragments. diff --git a/src/content/docs/docs/modules/the-module-system.mdx b/src/content/docs/docs/modules/the-module-system.mdx new file mode 100644 index 0000000..e924fc7 --- /dev/null +++ b/src/content/docs/docs/modules/the-module-system.mdx @@ -0,0 +1,103 @@ +--- +title: The module system +description: What a module is, why the platform is built this way, and the one rule about URLs that catches everybody once. +--- + +import { Aside } from '@astrojs/starlight/components'; + +Runic Gateway's core knows nothing about any particular game. Everything that makes the +site a *Ultima Online* site — the shard status, the atlas, the market, the guild pages — +lives in a **module**, installed onto a running deployment. + +This section is the builder's track. If you only want to install one, that is +[Install a game module](/docs/getting-started/install-a-game-module/) and +[Managing modules](/docs/administration/managing-modules/). + +## What a module is + +One repository producing one bundle, with a server half and a client half that version +together — so a route and the screen that calls it can never be mismatched. + +A module owns: + +- **Its routes**, server and client +- **Its schema**, as a fragment core replays on boot +- **Its navigation entries**, interleaved into core's groups rather than parked in a + section of their own +- **Its vocabulary** — the words a player of *that* game expects + +Core owns the account, the session, the roles, the posts, the uploads, notifications and +Teams. A module reaches all of that through a defined surface, [the module +API](/docs/modules/the-module-api/). + +## Why it is built this way + +Three constraints had to hold at the same time, and between them they determined almost +everything else: + +1. **Production is a prebuilt, pull-only image.** Operators do not build. There is no + compile step anywhere in installing a module. +2. **Modules live on a mounted volume**, not inside the image — a bind mount of + `./modules`. That is what lets a module be added to an image that knows nothing about + it. +3. **`script-src 'self'`.** The content-security policy forbids inline script, which rules + out an import map and is why core shares React on a global instead. See [Building a + module](/docs/modules/building-a-module/). + +Install and uninstall need a **restart** — never a rebuild. + + + +## The boundary is not a sandbox + +A module runs **in the same Node process, with full access**. Say that plainly, because +the word "module" invites the opposite assumption. + +The boundary is a **code-organisation and distribution boundary, not a security +boundary**. For a self-hosted operator installing software they chose, that is the same +trust category as running its schema fragment — which they are also doing. + +What the boundary buys is that modules talk to core through a *defined* surface, so a core +refactor cannot silently break a module. That rule is enforced mechanically rather than by +review: **a module must run with zero `require`/`import` reaching outside its own +directory**, and CI checks it. A gap in the surface extends the surface; it is never +worked around with a deeper import. + +## The URL rule, and its one exception + +**A module owns one path segment wherever it appears.** For a module with id `uo`: + +| Surface | Path | +|---|---| +| Public pages | `/uo/shard`, `/uo/atlas`, `/uo/market` | +| Admin pages | `/admin/uo/link`, `/admin/uo/visibility` | +| Player pages | `/player/uo/…` | + +**API routes are the exception, and keep their exact paths.** The shard admin API is still +`/api/v1/admin/shard/*`, not `/api/v1/admin/uo/shard/*`. This is why the Android app and +the Discord bot needed no API changes at the cutover. + + + +Old paths are **not** redirected. That was a deliberate call — a visible boundary in the URL +rather than a hidden one — taken while the platform had no public deployments to break. + +## Where the design of record lives + +This page summarises. The normative document is +[`MODULE_SYSTEM.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_SYSTEM.md), +and the contract itself is +[`MODULE_API.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_API.md). +Where this site and those documents disagree, they are right and this is a bug. diff --git a/src/content/docs/docs/reference/bridge-cfg.mdx b/src/content/docs/docs/reference/bridge-cfg.mdx new file mode 100644 index 0000000..46c3777 --- /dev/null +++ b/src/content/docs/docs/reference/bridge-cfg.mdx @@ -0,0 +1,73 @@ +--- +title: Bridge.cfg +description: Every key the in-game plugin reads — the connection, the sweep intervals, the feature switches and the caps that keep untrusted game data bounded. +--- + +import { Aside } from '@astrojs/starlight/components'; +import { bridgeCfg } from '../../../../data/reference.mjs'; + +`Config/Bridge.cfg` in the ServUO tree configures the plugin — what it connects to, how +often it sweeps the world, and which features it publishes. + +[The installer](/docs/reference/installer-cli/) puts it there. Editing it is a shard +operator's job, not a builder's. + +## How to read this file + +Three kinds of key, and they carry very different risk: + +- **Connection** — where the sidecar is, and how much the plugin may buffer. +- **Sweep intervals** — how often the plugin walks part of the world. **These are the + performance dial.** Every sweep runs on the game's core thread, so shortening one costs + the game, not the sidecar. +- **Caps and switches** — feature toggles, and the bounds on anything a player can + influence. + + + +## Every key + +{Object.entries(bridgeCfg).map(([group, keys]) => ( +
+

{group}

+ + + + {Object.entries(keys).map(([name, why]) => ( + + ))} + +
KeyWhat it is for
{name}{why}
+
+))} + +This list is checked against the shipped `Bridge.cfg` on every build, so a key added by a +protocol change turns this page red rather than going undocumented. + +## Two that deserve their own note + +**`QueueCap`** bounds the drop-oldest queue between the game and the writer thread. When it +fills, the **oldest events are discarded** — which is the correct behaviour, because the +alternative is a game server that stutters when a sidecar is slow. Raising it buys tolerance +for longer sidecar outages at the cost of memory; it never buys correctness. + +**`GuildRosterMembersPerLine`** exists because a roster is the only fat frame this bridge +emits — a real 155-member guild measured about 10.8 KB. Rosters are **split** across lines +rather than sent oversized. See [The bridge](/docs/architecture/the-bridge/). + +## Canonical documents + +The shipped +[`Bridge.cfg`](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/src/branch/main/overlay/Config/Bridge.cfg) +is the authority; +[`link/PLAN.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/link/PLAN.md) +§10 documents the config keys and +[`SHARD_PREREQS.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/link/SHARD_PREREQS.md) +covers what a shard needs before any of this works. diff --git a/src/content/docs/docs/reference/canonical-documents.mdx b/src/content/docs/docs/reference/canonical-documents.mdx new file mode 100644 index 0000000..9d34fa8 --- /dev/null +++ b/src/content/docs/docs/reference/canonical-documents.mdx @@ -0,0 +1,70 @@ +--- +title: Canonical documents +description: Where the normative specifications live — the documents that win whenever this site disagrees with them. +--- + +import { Aside } from '@astrojs/starlight/components'; +import { canonicalDocs } from '../../../../data/reference.mjs'; + +Everything on this site is a **summary**. These are the documents it summarises, and where +the two disagree, **they are right and this site has a bug**. + + + +## The documents + + + + + {Object.entries(canonicalDocs).map(([docPath, why]) => ( + + + + + ))} + +
DocumentAnswers
+ + {docPath} + + {why}
+ +Every path above is checked to still exist on every build, so a document that is renamed or +moved turns this page red rather than leaving a dead link. + +## Which document answers which question + +- **"May a module do this?"** → `MODULE_API.md`. It is the contract, and it is the only thing + that can answer yes. +- **"Why is the module system like this?"** → `MODULE_SYSTEM.md`. +- **"What does this API return?"** → your own deployment's `/api/docs`, then + `BACKEND_DESIGN.md` §4. +- **"What can the shard send?"** → `link/PLAN.md` §5, and `v4.md` for the current protocol. +- **"Who may see this?"** → `SHARD_VISIBILITY.md` for the administrator's view, + `modules/uo/API.md` §4 for the specification. +- **"How do I set a shard up?"** → `installer/INSTALL.md`. + +## Where they live + +All of them are in +[`RunicGateway/docs`](https://gitea.whitlocktech.com/RunicGateway/docs), which is Markdown +only and versioned independently of the code it describes. + +**A code change is not complete until `docs` reflects it.** That is a rule in the +project's own contributor guidance, not an aspiration — a change to behaviour, protocol, +endpoints, schema, configuration or the deployment model requires a matching edit there. + +## Two things that are not in `docs` + +**The Integration Kit** is its own repository, because its audience is outside this project +and it teaches rather than specifies. See [The Integration +Kit](/docs/modules/the-integration-kit/). + +**The OpenAPI specification** is generated and committed in `website` itself, because it is +derived from the routes rather than written alongside them. diff --git a/src/content/docs/docs/reference/environment-variables.mdx b/src/content/docs/docs/reference/environment-variables.mdx new file mode 100644 index 0000000..6c40b32 --- /dev/null +++ b/src/content/docs/docs/reference/environment-variables.mdx @@ -0,0 +1,66 @@ +--- +title: Environment variables +description: Every variable the site reads, what each is for, and the four it refuses to start without. +--- + +import { Aside } from '@astrojs/starlight/components'; +import { envVars } from '../../../../data/reference.mjs'; + +Every variable in `website`'s root `.env.example` — **the file a Compose deployment actually +reads**, which is not the same file local development copies. + +This list is checked against that file on every build, in both directions: a variable that +disappears upstream fails, and a variable added upstream that is missing here fails too. + + + +## Every variable + + + + + {Object.entries(envVars).map(([name, why]) => ( + + ))} + +
VariableWhat it is for
{name}{why}
+ +## The three worth reading twice + +**`SECRET_ENC_KEY`** encrypts secrets at rest — OAuth client secrets, the Discord bot token, +the shard's auth token. Changing it does **not** re-encrypt anything: what was stored under +the old key can no longer be read, and every stored secret has to be entered again. + +**`COOKIE_SECURE=auto`** decides `Secure` per request, which is what lets one deployment +work both over HTTPS through a proxy and over plain HTTP on a LAN address. Forcing it either +way breaks one of those. + +**`TRUST_PROXY`** is required behind a reverse proxy for secure cookies, real client IPs and +rate limiting to work at all. Without it, every request appears to come from the proxy — so +rate limiting and IP bans apply to your whole user base at once. + +## Where to set them + +A first install is [Install the site](/docs/getting-started/install-the-site/), which prints +a complete `.env` alongside its Compose file. Afterwards, +[Configuration](/docs/administration/configuration/) covers what is env-configured and what +is not. + +**Most settings are not here.** Branding, navigation, theming and the shard connection are +**admin-managed and live in the database**, deliberately — so changing them does not mean +redeploying a container. + +## Canonical source + +`website`'s +[`.env.example`](https://gitea.whitlocktech.com/RunicGateway/website/src/branch/main/.env.example) +is the authority, and +[`BACKEND_DESIGN.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/BACKEND_DESIGN.md) +§8 covers deployment. diff --git a/src/content/docs/docs/reference/event-catalog.mdx b/src/content/docs/docs/reference/event-catalog.mdx new file mode 100644 index 0000000..8ae00db --- /dev/null +++ b/src/content/docs/docs/reference/event-catalog.mdx @@ -0,0 +1,104 @@ +--- +title: Event catalog +description: What a game server can tell the website, how those events are grouped, and the five-rung ladder that decides who may see each one. +--- + +import { Aside } from '@astrojs/starlight/components'; +import { visibilityLadder } from '../../../../data/reference.mjs'; + +The events a shard emits, and the mechanism that decides who may see them. + +The exact wire shapes are in the protocol specification and are **not** restated here — a +copy of a wire format is a copy that will be wrong after the next bump. This page is the map +and the security model. + +## What the shard can say + +Nine groups, from the data catalog: + +| Group | Covers | +|---|---| +| Session & identity | Logins, logouts, account linking | +| Character state | Vitals, stats, skills, position | +| Economy & commerce | Gold movement, vendor sales, supply totals | +| Housing / IDOC | Decay stages, ownership, coordinates | +| Combat, death, PvP | Kills, deaths, notable fights | +| Progression & activity | Skill gains, points, leaderboards | +| Cheat detection & staff audit | Fastwalk and friends; staff property edits | +| Lifecycle | `server.hello`, shutdown, crash | +| Known gaps | Things ServUO offers no clean hook for | + +A representative line: + +```json +{"t":1752,"kind":"cheat.fastwalk","serial":"0x1A2B","acct":"PerryAdimn"} +``` + +Note that one. **Cheat and audit events exist, and they are exactly what must never reach a +public page.** + +## How events are handled + +Not all alike, and the difference is deliberate: + +- **State-changing kinds** update tables. The current state is what a page renders. +- **Notable kinds** additionally append to an events log, because a history is worth + keeping. +- **High-frequency kinds** only update state. Accumulating history for something that fires + constantly buys nothing and costs a table that grows forever. + +## The visibility ladder + +Five rungs, in order, least privileged first: + +
    + {visibilityLadder.map((rung) => (
  1. {rung}
    2. ))} +
+ +Every feature declares the rung it is visible from, and individual **fields** can require a +higher rung than the feature that carries them — a character's presence may be public while +its *location* is staff-only. + +This list and its **order** are checked against the module that enforces it on every build. +Order matters as much as membership: reasoning about "staff and above" depends on the rungs +being in the right sequence. + + + +## Two SSE channels + +Ingested events fan out to browsers over two streams: + +- a **public** stream, carrying only allowlisted kinds; +- an **admin** stream, which additionally carries staff audit, cheat detection and login + attempts with IP addresses. + +**The catalog is the module's; the boundary is core's.** A module declares which of its kinds +are public-safe, and core enforces the split — a sensitive kind cannot reach the public +channel. + +A viewer's rung is resolved **once, when the stream opens, and frozen for its life**. A +long-lived connection must not silently gain privilege because the session changed +underneath it. Configuration changes, by contrast, *do* take effect live. + +## Administering it + +[The shard connection](/docs/administration/the-shard-connection/) covers the admin screens, +and the visibility ladder is administrator-configurable per feature and per field. + +## Canonical documents + +[`link/PLAN.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/link/PLAN.md) +§5 is the data catalog and §7 the wire protocol; +[`link/v4.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/link/v4.md) +is the current protocol; +[`SHARD_VISIBILITY.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/SHARD_VISIBILITY.md) +is the administrator's guide to the ladder, and +[`modules/uo/API.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/modules/uo/API.md) +§4 specifies it. diff --git a/src/content/docs/docs/reference/http-api.mdx b/src/content/docs/docs/reference/http-api.mdx new file mode 100644 index 0000000..2ce4d87 --- /dev/null +++ b/src/content/docs/docs/reference/http-api.mdx @@ -0,0 +1,85 @@ +--- +title: HTTP API +description: How the site's API is organised, where the live specification is, and the gate each tier sits behind. +--- + +import { Aside } from '@astrojs/starlight/components'; + +The site's backend API is **OpenAPI 3.0**, and the specification is generated from the routes +themselves rather than maintained beside them. + + + +## The tiers + +Every route lives under `/api/v1//`, and **the tier decides the gate**. + +| Tier | Routes | Sits behind | +|---|---|---| +| `admin` | ~93 | `noindex`, `isLoggedIn`, `requireRole('admin','editor','moderator')` | +| `auth` | ~38 | Public by necessity; heavily rate-limited and bot-scored | +| `player` | ~24 | `noindex`, `requireAuth` — role-agnostic self-service | +| `public` | ~19 | Nothing, by design | +| `settings` | 2 | `requireAuth` + `noindex`, no role gate | + +Plus two outside the versioned surface: **`/api/health`** and **`/api/csp-report`**. + +Those two are deliberately not under `/api/v1`. A browser learns the CSP report path from the +policy header rather than from a client build, so it is not part of the versioned client +contract. + +## Two things the tier table implies + +**`player` is role-agnostic.** It is self-service for whoever is signed in, gated on +`requireAuth` alone and never on "is not staff". Staff are a *superset* of players — an +administrator has characters too, and a `player` route that excluded them would 403 an admin +off their own account. + +**A module's routes inherit their tier's gate** and add their own on top. A module never +re-implements the tier gate; see [The module +API](/docs/modules/the-module-api/#registerroutes-and-the-tier-gate). + +## Authentication + +Three ways in, [one session model](/docs/architecture/authentication-architecture/): + +- **Cookie** — `httpOnly` JWT, for the browser. +- **Bearer** — short access tokens plus rotated, hashed, revocable refresh tokens, for the + native app. +- **SSO** — OAuth2/OIDC with PKCE, and **link-only**: an external identity must already be + attached to an existing account. + +Admin roles are **re-validated against the database on every request**, so a demoted user +loses access immediately rather than at token expiry. + +## The sidecar's API is a different thing + +The uo-link sidecar exposes its own small REST and WebSocket surface, reached **only** by the +website's backend. It carries `X-UOLink-Version` and answers `409` on a protocol mismatch. + +It is not part of this API and is not served from your site. See [The +bridge](/docs/architecture/the-bridge/). + +## Keeping the spec current + +For contributors: the specification is generated from `#swagger.*` annotations next to each +route, and the output is committed. + +```bash +cd website/server && npm run swagger +``` + +A route that is not in the specification is not finished. Modules publish their own +fragment, at a fixed filename in the bundle root, so a module's routes appear in the same +documentation as core's. + +## Canonical document + +[`BACKEND_DESIGN.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/BACKEND_DESIGN.md) +§4 is the API contract, including §4.0's authoritative route list. diff --git a/src/content/docs/docs/reference/installer-cli.mdx b/src/content/docs/docs/reference/installer-cli.mdx new file mode 100644 index 0000000..f3ab7ff --- /dev/null +++ b/src/content/docs/docs/reference/installer-cli.mdx @@ -0,0 +1,85 @@ +--- +title: Installer CLI +description: The four commands the installer offers, what each does to a host, and the environment variable that makes a full run safe to rehearse. +--- + +import { Aside } from '@astrojs/starlight/components'; +import { installerCommands } from '../../../../data/reference.mjs'; + +The installer is one binary per operating system that deploys the **shard side only**. It +never contacts the website. + +Downloads and the walkthrough are [Connect a game +server](/docs/getting-started/connect-a-game-server/). This page is the command surface. + +## The commands + + + + + {Object.entries(installerCommands).map(([name, why]) => ( + + ))} + +
CommandWhat it does
{name.toLowerCase()}{why}
+ +`doctor`, `update` and `uninstall` are the day-two commands. + +## Rehearsing a run + +Two mechanisms, and they answer different questions. + +```bash +runicgateway-installer install --servuo /path/to/ServUO --verify +``` + +**`--verify` writes nothing.** It reports what would change — the diff against the ServUO +tree — which is the right thing to run first against a shard that has players on it. + +```bash +RUNICGATEWAY_STATE_DIR=/tmp/rehearsal runicgateway-installer install --servuo … +``` + +**`RUNICGATEWAY_STATE_DIR` relocates everything the installer writes** — state, data, and +the sidecar binary — *and suppresses service registration*. That is how a full run is +exercised without root, and it is what the project's own tests use. + +## What an install actually does + +1. Resolves a **bundle** — an exact, protocol-checked sidecar and overlay pair published by + CI. Never "latest of each"; see [Protocol + versions](/docs/architecture/protocol-versions/). +2. Syncs the plugin overlay into the ServUO tree, backing up whatever it is about to + overwrite. +3. Offers the opt-in patch tier. +4. Installs the sidecar and registers its service. +5. Prints four values to paste into the site's shard screen. + + + +## Platforms + +Linux `x86_64`, Linux `aarch64`, and Windows `x86_64`. + +**macOS and Windows-on-ARM are deliberately absent**: the game server and the sidecar must +share a host, and no ServUO host is either. + +Releases are **unsigned**, and `SHA256SUMS` is the trust anchor — verify before running. +Windows will show a SmartScreen prompt, which is expected for an unsigned binary. + + + +## Canonical documents + +[`INSTALL.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/installer/INSTALL.md) +is the operator guide — including Appendix A, hand deployment, for hosts that cannot run the +binary — and +[`PLAN.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/installer/PLAN.md) +is the design of record. diff --git a/src/content/docs/docs/reference/sidecar-toml.mdx b/src/content/docs/docs/reference/sidecar-toml.mdx new file mode 100644 index 0000000..9c153e2 --- /dev/null +++ b/src/content/docs/docs/reference/sidecar-toml.mdx @@ -0,0 +1,68 @@ +--- +title: sidecar.toml +description: The sidecar's entire configuration — four keys — and why the file is generated rather than shipped. +--- + +import { Aside } from '@astrojs/starlight/components'; +import { sidecarConfig } from '../../../../data/reference.mjs'; + +The uo-link sidecar's configuration. It is deliberately tiny: the sidecar is a **dumb +forwarder**, and policy lives on the website where an administrator can see it. + +## The file is written, not shipped + +The sidecar **writes `sidecar.toml` on first run**, including a generated auth token. There +is no committed sample that is authoritative, and nothing is compiled into the binary. + +Point it elsewhere with `$UOLINK_CONFIG`. + + + +## The keys + + + + + {Object.entries(sidecarConfig).map(([name, why]) => ( + + ))} + +
KeyWhat it is for
{name}{why}
+ +This list is checked against the sidecar's own config structs on every build, so a key added +upstream turns this page red rather than quietly going undocumented. + +## What is *not* in here + +Worth stating, because the absences are the design: + +- **No allowlist, no audience rules, no visibility settings.** Those are the website's, and + admin-toggleable. The sidecar forwards; the site decides who may see what. +- **No website URL.** The website reaches the sidecar, not the other way round. +- **No protocol version.** It is compiled in, because a sidecar that could be *configured* + to claim a different protocol would defeat the check. See [Protocol + versions](/docs/architecture/protocol-versions/). + +## Running it + +```bash +cargo run # writes sidecar.toml on first run +RUST_LOG=debug cargo run # verbose, including heartbeats +``` + +Normally you do not run it by hand — [the installer](/docs/reference/installer-cli/) +installs it and registers its service. + +## Canonical documents + +The structs in +[`sidecar/src/config.rs`](https://gitea.whitlocktech.com/RunicGateway/link/src/branch/main/sidecar/src/config.rs) +are the authority; +[`link/PLAN.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/link/PLAN.md) +is the design of record and +[`ADMIN_CONTROLS.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/link/ADMIN_CONTROLS.md) +covers what the site may command the game to do. diff --git a/src/data/reference.mjs b/src/data/reference.mjs new file mode 100644 index 0000000..4164ff5 --- /dev/null +++ b/src/data/reference.mjs @@ -0,0 +1,190 @@ +/** + * The Reference section's enumerations. + * + * §1 says a Reference page is "a navigable summary plus a link to the canonical document — + * never a re-specification". This file is the line between those two things, and it is + * worth being explicit about where it falls: + * + * * The NAMES are here — every environment variable, every config key, every command, + * every event kind. A reference section that cannot answer "what variables are there?" + * without a click-through is a link farm. + * * The SEMANTICS are not. One terse line each, saying what a thing is FOR. Shapes, + * defaults that matter, interactions, and every "why" stay in the canonical document. + * + * Everything below is checked against its source by `scripts/checkReference.mjs`, in both + * directions — a name that disappears upstream fails, and a name that appears upstream and + * is missing here fails too. That is the whole reason it is safe to write names down at + * all: the enumeration cannot rot into fiction without turning the build red. + * + * Descriptions are NOT checked, and cannot be. They are the part a human has to keep + * honest, which is why they are kept short enough to re-read. + */ + +/** `website` root `.env.example` — the file a Compose deployment actually reads. */ +export const envVars = { + IMAGE_TAG: 'Which published image tag to run', + NODE_ENV: 'production or development — several refusals are production-only', + PORT: 'The port the app listens on', + INTERNAL_PORT: 'The internal-only listener, for the bot channel', + UPLOAD_DIR: 'Where uploads are written', + + LOG_LEVEL: 'Console log level', + FILE_LOG_LEVEL: 'File log level, set separately', + LOG_TO_FILE: 'Whether to write a log file at all', + LOG_DIR: 'Directory for the log file', + LOG_FILE: 'Log file name', + + BRAND_NAME: 'Site name — branding is data, not a build', + BRAND_SHORT_NAME: 'Short form, for tight spaces', + BRAND_TAGLINE: 'One line under the name', + BRAND_DESCRIPTION: 'Meta description', + BRAND_CONTACT_EMAIL: 'Published contact address', + BRAND_URL: 'Canonical public URL', + BRAND_ACCENT_COLOR: 'Accent colour', + BRAND_LOGO: 'Logo path', + BRAND_HERO: 'Hero image path', + BRAND_FAVICON: 'Favicon path', + + DB_HOST: 'Database host', + DB_PORT: 'Database port', + DB_NAME: 'Database name', + DB_USER: 'Database user', + DB_PASSWORD: 'Database password', + DB_ROOT_PASSWORD: "The database container's root password", + + JWT_SECRET: 'Signs session tokens. Rotating it logs everyone out', + SECRET_ENC_KEY: + 'Encrypts secrets at rest. Required in production, and rotating it ORPHANS every stored secret', + JWT_EXPIRES_IN: 'Session lifetime', + COOKIE_SECURE: 'auto decides Secure per request, so HTTPS and LAN HTTP both work', + COOKIE_NAME: 'Session cookie name. Changing it invalidates existing sessions', + TRUST_PROXY: 'Needed behind a reverse proxy for secure cookies, real IPs and rate limiting', + DEBUG_TRUST_PROXY: 'Diagnostic for the above', + TOTP_CHALLENGE_TTL: 'How long a pending 2FA challenge is valid', + + ADMIN_USERNAME: 'First admin, created only when no users exist', + ADMIN_PASSWORD: 'First admin password. Set it before the first boot, not after', + + CLIENT_ORIGIN: 'Dev only — the Vite origin allowed through CORS', + + BOT_INTERNAL_URL: 'Where the Discord bot listens', + BOT_INTERNAL_KEY: + 'Authenticates the site↔bot channel. Required in production EVEN IF you run no bot', + + NTFY_BASE_URL: 'Push notification relay base URL', +}; + +/** `link/sidecar/src/config.rs` → the TOML the sidecar writes on first run. */ +export const sidecarConfig = { + 'shard.bind': 'Loopback address the game plugin dials out to', + 'web.bind': 'Address the website reaches the sidecar on', + 'web.auth_token': 'Shared secret the website must present. Generated on first run if blank', + 'store.path': "The sidecar's own durable store", +}; + +/** `installer` — `src/cli.rs`'s `Command`. */ +export const installerCommands = { + Install: 'Set up the shard side: sync the overlay, install the sidecar, register its service', + Doctor: 'Diagnose an existing install', + Update: 'Move to a newer bundle', + Uninstall: 'Remove what install put there', +}; + +/** `servuo-plugins/overlay/Config/Bridge.cfg` — the plugin's config, grouped for reading. */ +export const bridgeCfg = { + Connection: { + Host: 'Sidecar address the shard dials out to', + Port: 'Sidecar port', + QueueCap: 'Bounded queue depth. Full means drop-oldest — never block the game', + PublicConnectAddress: 'Address players connect to, published to the site', + LinkUrl: 'Where in-game account linking sends a player', + }, + Sweeps: { + StatSweepSeconds: 'Character stat sweep interval', + DecaySweepSeconds: 'House decay sweep', + EconomySweepSeconds: 'Economy totals sweep', + ChampSweepSeconds: 'Champion spawn sweep', + PageSweepSeconds: 'Staff page sweep', + GuildSweepSeconds: 'Guild roster sweep', + CitySweepSeconds: 'City / governor sweep', + PresenceSweepSeconds: 'Who is online', + HousingSweepSeconds: 'Housing sweep', + }, + Guilds: { + GuildRosterMembersPerLine: 'Frame cap — a roster is split rather than sent oversized', + GuildRosterGuildsPerTick: 'How many guilds are swept per tick', + }, + Points: { + PointsSweepSeconds: 'Points sweep interval', + PointsLeaderboardEnabled: 'Publish a leaderboard at all', + PointsTopN: 'Leaderboard length', + PointsSystems: 'Which point systems to include', + PointsProfileEnabled: 'Show points on a character profile', + PointsProfileRank: 'Show rank as well as total', + }, + Market: { + MarketEnabled: 'Publish player vendor listings', + MarketSweepSeconds: 'Market sweep interval', + MarketSweepBatch: 'Vendors per sweep', + MarketMaxListings: 'Cap on listings published', + }, + Ruleset: { + RulesetEnabled: 'Publish the shard ruleset', + RulesetIncludeSchedule: 'Include the event schedule with it', + }, + 'Town crier': { + TownCrierMaxLines: 'Lines per notice', + TownCrierMaxLineLength: 'Characters per line', + TownCrierMaxActive: 'Concurrent notices', + TownCrierMaxDurationSec: 'Longest a notice may run', + }, + News: { + NewsMaxTitleLength: 'Title cap', + NewsMaxBodyLength: 'Body cap', + NewsMaxExternal: 'How many site posts are carried in-game', + NewsAnnounceDurationSec: 'How long an announcement shows', + }, + 'Admin commands': { + AdminWriteEnabled: 'Whether the site may write to the game at all. Off by default', + AdminAccessFloor: 'Minimum in-game access level for admin actions', + AdminBroadcastMaxLength: 'Broadcast cap', + AdminReasonMaxLength: 'Reason field cap', + AdminBanMaxDurationSec: 'Longest ban the site may set', + }, + Accounts: { + SignupMode: 'How game accounts may be created', + AccountCreateEnabled: 'Allow creation at all', + RequireIpForCreate: 'Require a real client IP', + AccountNameMaxLength: 'Account name cap', + AccountPasswordMaxLength: 'Account password cap', + }, +}; + +/** + * The five-rung visibility ladder, from `module-uo`'s `server/utils/shardVisibility.js`. + * + * This one is a SECURITY boundary, not a convenience filter, which is why it is enumerated + * rather than described: a reader needs to see the whole ladder at once to reason about it. + */ +export const visibilityLadder = ['anonymous', 'logged_in', 'player', 'staff', 'admin']; + +/** Canonical documents, by the question each answers. Checked to still exist in `docs`. */ +export const canonicalDocs = { + 'website/ARCHITECTURE.md': 'How the website fits together — the canonical diagram', + 'website/BACKEND_DESIGN.md': 'The API, schema and security contract', + 'website/MODULE_SYSTEM.md': 'Why the module system is shaped this way', + 'website/MODULE_API.md': 'Everything a module may do — the contract', + 'website/TEAMS.md': 'Teams as a platform primitive', + 'website/SHARD_VISIBILITY.md': 'The audience ladder, for administrators', + 'website/THEMING_AND_NAV.md': 'Admin-configurable theme, assets and navigation', + 'website/TRUSTED_DEVICES_MFA.md': 'Trusted devices and the second factor', + 'link/PLAN.md': 'The sidecar design of record, the data catalog and the wire protocol', + 'link/INTEGRATION.md': 'Integrating with the sidecar', + 'link/v4.md': 'Protocol 4, and its cross-repository obligations', + 'link/ADMIN_CONTROLS.md': 'What the site may command the game to do', + 'installer/INSTALL.md': 'The operator guide for setting a shard up', + 'installer/PLAN.md': "The installer's design of record", + 'modules/rust-dryrun.md': 'A second module designed on paper, to test that the contract generalises', + 'modules/uo/API.md': "module-uo's own API, including its audience rules", + 'android/PLAN.md': 'The Android app', +};