From 89aff5c26bd9dac5ef317abed2aaed9d2e84eab1 Mon Sep 17 00:00:00 2001 From: wtclaude Date: Wed, 12 Aug 2026 09:38:07 -0500 Subject: [PATCH] =?UTF-8?q?docs(modules):=20settle=20Phase=205=20=E2=80=94?= =?UTF-8?q?=20the=20Integration=20Kit,=20and=20the=20sidecar=20rule?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 4 closed; this is the plan for Phase 5, the Integration Kit, settled with the org lead on six decisions. MODULE_SYSTEM.md - §2.11.1 (new): Phase 5's shape — measured starting state, the six decisions, the four slices, the acceptance mechanism. - §2.11: layer 2 rewritten. The sidecar is the default and only path to a game, and it is a non-blocking dumb forwarder that persists before it forwards. A game that already ships a remote-control surface gets a thin sidecar, not none. - §2.10: a MODULE_API_VERSION bump now carries a pass over the kit. - Part 3: decisions 28–33. - The Phase 5 line in Part 2's phase list: STARTED 2026-08-12. MODULE_API.md - §2.7 gains one prohibition: a module does not open a connection to a game server from the website process — a game socket, RCON, a query port, an engine admin API — with the three reasons it exists (the website is the internet-facing process, the sidecar owns the durable copy, neither side can stall the other). Normative prose with no CI behind it, stated so that every second module does not re-decide it. - §1.1: the 1.4.0 entry. No member changed; minor deliberately. modules/rust-dryrun.md - §2 "Talking to the game" rewritten from "No sidecar" to a thin RCON sidecar, with a dated Correction note recording what the exercise originally concluded and why it was overruled. Corrected in view rather than silently rewritten. Pairs with website#… (the 1.4.0 bump, into edge); either order. Co-Authored-By: Claude --- modules/rust-dryrun.md | 36 ++++++++---- website/MODULE_API.md | 39 ++++++++++++- website/MODULE_SYSTEM.md | 116 +++++++++++++++++++++++++++++++++++---- 3 files changed, 169 insertions(+), 22 deletions(-) diff --git a/modules/rust-dryrun.md b/modules/rust-dryrun.md index 8c9a401..39f3448 100644 --- a/modules/rust-dryrun.md +++ b/modules/rust-dryrun.md @@ -111,19 +111,33 @@ model gets wrong, and it is worth writing down for whoever builds this. ### Talking to the game -**No sidecar.** Rust ships RCON over WebSocket, so the module dials the server directly with the -token an admin saved, encrypted at rest through `ctx.secretBox`. +**A thin sidecar.** Rust ships RCON over WebSocket, so `rust-link` is small: it holds the RCON +connection to each server with the token an admin saved, and presents the website the same shape +`uo-link` does — a bearer-authed HTTP + WebSocket API in front of a SQLite store. The module talks +only to it, never to a game server. -This is the sharpest test of whether the module system's boundary is drawn in the right place, and it -passes: core has no opinion about how a module reaches its game. What core owns is that the module -never blocks a request on it, that its secrets are encrypted, and that a game being down degrades to -a page saying so. The *shard-dials-out* invariant that shapes -[`../link/PLAN.md`](../link/PLAN.md) is a property of ServUO — a game engine with no remote-control -surface, whose plugin must not stall on a socket — not of the platform. A game that ships RCON -already answers the question the sidecar exists to answer. +The store is the reason it exists even though the game is already remote-controllable. RCON is a +live channel with no memory: what it tells you while nobody is listening is gone. So the sidecar +appends every kill, wipe and chat line, keeps the latest snapshot of each server's state, and +answers the website's reads from disk — a website that is down, restarting or mid-deploy loses +nothing, and a leaderboard renders the last thing the server said rather than an error. It also +keeps the RCON token, the reconnect loop and the per-server fan-out out of an Express process, where +a stalled socket is a stalled request handler. -The Integration Kit ([`MODULE_SYSTEM.md`](../website/MODULE_SYSTEM.md) §2.11) should say this -plainly, or its second reader will build a sidecar they did not need. +**`wipe_id` makes the durable copy load-bearing rather than a nicety.** A wipe is the moment the +game forgets; the sidecar is the only thing that remembers the shape of the map that just ended. + +> **Correction, 2026-08-12.** This section originally concluded **"No sidecar"** — the module dialling +> RCON directly — and offered it as evidence that core has no opinion about how a module reaches its +> game. That was overruled by the org lead when Phase 5 (§2.11.1 d3/d4) settled the kit's stance, and +> the prohibition is now contract: [`../website/MODULE_API.md`](../website/MODULE_API.md) §2.7, as of +> `MODULE_API_VERSION` 1.4.0, a module does not open a connection to a game server from the website +> process. The original reasoning was not wrong about *ServUO* — the shard-dials-out invariant in +> [`../link/PLAN.md`](../link/PLAN.md) really is a property of an engine with no remote-control +> surface — but it mistook that for the whole reason a sidecar exists. The other reason is durability: +> the website is not the right place to hold a game connection, because it is the process most likely +> to be restarted and the one facing the internet. The finding is left in view rather than edited out; +> what a dry run concluded is worth more than a tidy document. ## 3. The client half diff --git a/website/MODULE_API.md b/website/MODULE_API.md index a5d0885..988df6e 100644 --- a/website/MODULE_API.md +++ b/website/MODULE_API.md @@ -26,13 +26,23 @@ here extends the contract first, in this file, before the module is written agai Core exports a single integer-major semver string from `server/src/modules/version.js`: ```js -const MODULE_API_VERSION = '1.3.0' +const MODULE_API_VERSION = '1.4.0' ``` The client half carries the same number (`client/src/modules/version.js`) and a test asserts the two agree. Duplicated rather than fetched because the value has to be on `window.__rg` before the first module chunk evaluates, which is earlier than any network round trip could answer. +**1.4.0 — Phase 5, the sidecar rule.** §2.7 gained one prohibition: a module does not open a +connection to a game server from the website process. It talks to a **sidecar**, which owns the +durable copy of the game's state. No member was added, removed or changed — the surface is identical +to 1.3.0 — and the bump exists because a module written against 1.3.0 could conform to every member +and still be built the wrong way round. It is **minor rather than major** deliberately: nothing that +satisfied 1.3.0's *surface* stops working, `module-uo`'s `coreApi: "^1.3.0"` still resolves, and +module-uo already complies because the sidecar is where it came from. The reasoning a module author +needs is [`../../Integration-kit`](https://gitea.whitlocktech.com/RunicGateway/Integration-kit) +chapter 3; the rule itself is below, because the kit teaches and never re-specifies. + **1.3.0 — Phase 3 slice 3, the client half's move.** Three client additions, each because the extraction needed it: a nav item may carry an **`icon`** component (§3.3), core declares a third slot **`player.invite.accepted`** (§3.7), and `window.__rg.api` gained **`BASE`** — which §3.5 specified @@ -441,6 +451,33 @@ them leaves an operator with orphaned data and no supported way to remove it. - Read `process.env` for core configuration. Its own config is a `settings` key or its own table. - Call `process.exit`, install signal handlers, or start a listener. - Write outside `ctx.paths.moduleRoot` and the upload directory. +- **Open a connection to a game server from the website process** — a game socket, an RCON channel, + a query port, an engine's admin API. A module talks to a **sidecar**, and the sidecar talks to the + game. Added in 1.4.0. + +**Why the sidecar is not optional** (added 2026-08-12; the reasoning, and the worked example, are +Integration Kit chapter 3 — [`MODULE_SYSTEM.md`](MODULE_SYSTEM.md) §2.11): + +- **The website is the internet-facing process and the game is not.** A module dialling the game + directly makes the public web app the thing the game trusts, and puts the game's address in the + same process as every request from the internet. With a sidecar, the game dials **out** and opens + no listening port at all, which is the invariant [`../link/PLAN.md`](../link/PLAN.md) exists to + hold. +- **The sidecar owns the durable copy.** It is a non-blocking dumb forwarder that **persists before + it forwards** — `uo-link` writes every event and every board snapshot to SQLite (`store.rs`) and + serves its REST reads from there. So a website that is down, restarting or mid-deploy loses + nothing, and a page renders the last thing the game said instead of going blank. A module holding + the connection itself has nowhere to put what arrives while the website is not running. +- **Neither side can stall the other.** The game's plugin enqueues onto a bounded drop-oldest queue + and a writer thread drains it; the live feed is best-effort and lossy on purpose (a lagging + consumer drops frames) because durability is the store's job, not the socket's. A module that owns + the socket inherits both problems inside an Express process, where the failure mode is a wedged + request handler. + +This one is **normative prose with no CI behind it** — an outbound socket is not statically +detectable the way an internal `require` is (§5.1), so it is enforced in review. Stated as a rule +anyway, because the alternative is that every second module re-decides it, and the first one to +decide wrong finds out during an outage. ### 2.8 The OpenAPI fragment diff --git a/website/MODULE_SYSTEM.md b/website/MODULE_SYSTEM.md index db24075..b3a0da5 100644 --- a/website/MODULE_SYSTEM.md +++ b/website/MODULE_SYSTEM.md @@ -1404,10 +1404,12 @@ all being debugged at once. Its shape, the six decisions it turned on, the per-s acceptance table with its results are in §2.7.2 below — all four criteria met, each against the real published `module-uo` release rather than a fixture. -**Phase 5 — The Integration Kit.** `RunicGateway/Integration-kit`, the instruction book for building -a module for a game that is not UO — the website module, the sidecar and why it exists, and the -game-side plugin that feeds it. Scaffolded when Phase 2 lands, written against Phase 3's extraction, -finished alongside Phase 4. Full shape and its acceptance test in §2.11. +**Phase 5 — The Integration Kit. STARTED 2026-08-12.** `RunicGateway/Integration-kit`, the +instruction book for building a module for a game that is not UO — the website module, the sidecar +and why it is mandatory, and the game-side plugin that feeds it, on top of a `template/` module that +really builds. Scaffolded when Phase 2 lands, written against Phase 3's extraction, started once +Phase 4 closed. Its spec is §2.11; the six decisions it turned on, the slice table and the acceptance +mechanism are §2.11.1. #### 2.7.2 Phase 4's shape — settled 2026-08-12 @@ -1926,6 +1928,10 @@ merely regenerated) and `npm test`, and carries a matching edit to `BACKEND_DESI documentation aggregates in this repo under `docs/modules//` rather than living in module repos. Conventional Commits, the AI-disclosure trailer, branches cut from an up-to-date `main`. +**A `MODULE_API_VERSION` bump carries a pass over the Integration Kit** (§2.11.1 d2). The kit's CI +fails on the version number alone, which is the half a machine can check; the half it cannot is +whether a chapter has quietly become untrue, and that is the author's obligation in the same PR. + ### 2.11 The Integration Kit — the instruction book for building a module **`RunicGateway/Integration-kit`** — `https://gitea.whitlocktech.com/RunicGateway/Integration-kit.git`, @@ -1945,11 +1951,18 @@ repo: 1. **The website module.** `module.json`, the server entry point and what `ctx` hands you, the `register*` calls, the schema fragment, the prebuilt client chunk and the shared-dependency rule, packaging and release CI. The bulk of it. -2. **The sidecar** — what it is and, more importantly, *why*. The shard is never network-reachable; - the shard dials **out** and the sidecar is the listener; the wire is a versioned compatibility - contract rather than a build dependency; only the website's backend talks to it. A new game needs - its own sidecar or an adapter into the existing one, and neither can be designed by someone who has - been handed the message list and none of the reasoning. +2. **The sidecar** — what it is, *why*, and that it is **not optional**. The website process never + opens a connection to a game server; that is a rule in the contract as of MODULE_API 1.4.0 + ([`MODULE_API.md`](MODULE_API.md) §2.7), not a recommendation the kit makes. The game is never + network-reachable; it dials **out** and the sidecar is the listener; the wire is a versioned + compatibility contract rather than a build dependency; only the website's backend talks to it. + The part a message list cannot convey is that the sidecar is a **non-blocking dumb forwarder that + persists before it forwards** — `uo-link` writes every event and every board snapshot into SQLite + and answers its REST reads from that store, so a website that is down or restarting loses nothing + and a page shows the last thing the game said rather than going blank. A new game needs its own + sidecar or an adapter into the existing one; a game that already ships a remote-control surface + (Rust's RCON over WebSocket) needs a **thin** one — RCON on the game side, the uo-link-shaped + HTTP+WS API and the store on the website side — not none. 3. **The game-side plugin** — how a shard feeds the sidecar without ever letting the sidecar stall the game: the bounded drop-oldest queue, the dedicated writer thread, world reads only on the game's own thread. `servuo-plugins/` is the worked example; the constraints are general, and a plugin that @@ -1974,11 +1987,88 @@ leading it: first chapter — and doubles as the honest test that the contract generalises past its first module. **Acceptance:** someone builds a trivial working module for a second game by following the kit alone, -without reading core's source. Until that has happened it is a draft, however finished it looks. +without reading core's source. Until that has happened it is a draft, however finished it looks — and +it says so on its front page. The org landing page (`RunicGateway/.profile`) and the workspace's `CLAUDE.md` repo table both gain a row for it — when it has content, not while it is an empty repo. +#### 2.11.1 Phase 5's shape — settled 2026-08-12 + +**Measured starting state.** `Integration-kit` has **no branches and no initial commit** — still +exactly where `Module-uo` was at the start of Phase 0, two years of this plan's phases later. +Everything it will describe now exists and is proven: the loader, the four registries, the client +registry and its slots, the delivery path with its admin screen and `MODULES` variable, a real +published module (`module-uo` v0.3.0) and a written dry run for a second game. Nothing about the kit +is blocked; what remains is that nobody outside this org has ever tried to use any of it. + +**One constraint that shapes every slice: the module system is on `edge`, not `main`.** Decision 11 +holds the whole website workstream on `edge` until one cutover, and `main` has no +`server/src/modules/` at all. So the kit's CI pins a core ref **on `edge`**, and the pin is one of +the things the cutover has to revisit — recorded here because a pinned ref that quietly points at an +abandoned branch is exactly the failure this kit is supposed to be immune to. + +##### Six decisions, settled by the org lead + +1. **The kit ships a buildable `template/`, not prose alone.** A minimal module that really compiles + and really loads: `module.json`, a server entry registering one public route, a client chunk with + the Vite library build, the anchored alias array and the two boundary guards, a schema fragment + and release CI. The reader copies it, renames it, and has a working module before they have read + anything. Prose alone would leave a newcomer reverse-engineering build configuration out of a + 5,332-line real module, and the acceptance test asks for a *working* module, not an understood + one. The honest cost is that a template is code and code rots, which is what decision 2 answers. +2. **CI is the anti-rot mechanism, and it checks three things.** The kit's `pr-checks` clones + `website` at a **pinned ref** — the same trick `module-uo`'s frozen-manifest job already uses + (API §5.3) — then: asserts the kit's declared `coreApi` **equals** that core's + `MODULE_API_VERSION`, builds the template against it and runs the template's own guards, and + link-checks every out-link in the book. A contract bump in core therefore breaks the kit's build + *loudly*, which is the mechanised form of §2.11's own fear: a guide that restates a contract + diverges from it silently. The rule in prose (§2.10 gains it too) is kept as well, because CI can + only fail on a number, not on a paragraph that has become untrue. +3. **The sidecar is the default path and the website never contacts a game server.** This overrules + the "you may not need one" framing that `rust-dryrun.md` arrived at. A sidecar is a **non-blocking + dumb forwarder that persists before it forwards**: it writes into SQLite and answers reads from + that store, so a website that is down, restarting or mid-deploy loses no data and a page shows the + last thing the game said instead of blank. That is not a UO detail — `uo-link`'s `store.rs` holds + event history, every board's latest snapshot, the economy series and the published ruleset, and + its live WebSocket feed is deliberately lossy (a lagging consumer drops frames) precisely because + durability is the store's job and not the socket's. A game that already ships a remote-control + surface gets a **thin** sidecar, not none. +4. **That rule is contract, not kit teaching** — [`MODULE_API.md`](MODULE_API.md) §2.7, bumping + `MODULE_API_VERSION` to **1.4.0**. Minor rather than major: no member was added, removed or + changed, `module-uo`'s `coreApi: "^1.3.0"` still resolves, and module-uo already complies. It is + the one prohibition in §2.7 with **no CI behind it** — an outbound socket is not statically + detectable the way an internal `require` is — and it is written down anyway, because otherwise + every second module re-decides it and the first one to decide wrong finds out during an outage. +5. **Acceptance gets a proxy now and a draft banner until a human replaces it.** The real criterion + needs a person outside the org and cannot be manufactured. The proxy: a **cold agent given only + the kit and the contract it links to — never core's source and never `module-uo`** — builds a + trivial module for a second game, which then goes through API §7.7's browser smoke against a real + core. It lands as `docs/modules/kit-acceptance.md`, verdict included whichever way it goes. Every + phase of this plan found its worst defect in that browser smoke rather than in a test; this is the + same bet, aimed at the documentation instead of the code. +6. **`rust-dryrun.md` §2 is amended with a dated correction.** Its "**No sidecar** — the module dials + the server directly" is now non-conforming, and §2.11 points newcomers straight at that document + as the book's first chapter. It is rewritten to a thin RCON sidecar and carries a dated note + saying what the exercise originally concluded and why it was overruled — corrected rather than + silently rewritten, because the value of a dry run is the record of what it found. + +##### Slices + +| # | What | Repos | +| --- | --- | --- | +| — | This plan | `docs` (§2.11.1, API §1.1 + §2.7, `rust-dryrun.md`) + `website` (the 1.4.0 bump, → `edge`) | +| 0 | Scaffold: README with the draft banner, licence, contributing + AI disclosure, PR/issue templates, the book's outline — and the CI of decision 2 | `Integration-kit` | +| 1 | The `template/` module, building green against the pinned core | `Integration-kit` | +| 2 | The book: the website module, the sidecar, the game-side plugin | `Integration-kit` | +| 3 | Close: the cold-agent acceptance run, `.profile` and `CLAUDE.md` rows, the result recorded here | `Integration-kit`, `docs`, `.profile` | + +**The template is written before the book, and that is a reversal worth stating.** §2.11 says a kit +whose examples are invented is a kit whose examples do not compile; the strongest form of that is to +build the example first and then write the chapters *out of* a tree CI already proves. It also means +the book's code blocks are quotations with a path next to them rather than prose that happens to look +like code. + --- ## Part 3 — Settled decisions @@ -2015,3 +2105,9 @@ row for it — when it has content, not while it is an empty repo. | 25 | Purge is offered **inside** the uninstall flow, because `purge.sql` lives in the directory being deleted | §2.7.2 d5 | | 26 | The host allowlist **bootstraps from `MODULE_SOURCE_HOSTS`** into a settings row and is DB-owned thereafter | §2.7.2 d6 | | 27 | Documentation follows the code out: what core's reference described and no longer serves moves to `docs/modules//`, text unchanged | §2.7.2 slice 4 | +| 28 | The Integration Kit ships a **buildable `template/` module**, written before the book so the chapters quote a tree CI proves | §2.11.1 d1 | +| 29 | The kit's CI clones core at a **pinned ref** and fails on `coreApi` ≠ `MODULE_API_VERSION`, on a template that stops building, and on a dead link | §2.11.1 d2 | +| 30 | **The sidecar is the default and only path to a game**: a non-blocking dumb forwarder that persists to its own store before forwarding | §2.11.1 d3 | +| 31 | That is **contract, not advice** — API §2.7, `MODULE_API_VERSION` 1.4.0, normative prose with no CI behind it | API §1.1, §2.7 | +| 32 | The kit is a **draft on its own front page** until a person outside the org passes its acceptance; a cold-agent run is the interim proxy | §2.11.1 d5 | +| 33 | `rust-dryrun.md` is **corrected with a dated note**, not silently rewritten — a dry run's value is the record of what it found | §2.11.1 d6 |