RunicGateway/Integration-kit
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(book): the four chapters — Phase 5 slice 2
All checks were successful
PR Checks / prose (pull_request) Successful in 8s
Details
PR Checks / template (pull_request) Successful in 27s
Details

Browse Source 
The book, written out of the tree slice 1 proved. Four chapters in the order the
work happens: the first module in twenty minutes, the website module, the sidecar,
and the game-side plugin.

Shape, settled with the org lead:

  * template/README.md stays the REFERENCE — it travels with a copied template and
    CI holds it against the tree — and chapter 1 is the narration: what you should
    see after each step, the state your module lands in, and the four ways it fails.
    The chapter links to the checklist rather than restating it.
  * chapters 3 and 4 cite link/ and servuo-plugins/ by FILE AND IDENTIFIER, never by
    line. Those repositories move for their own reasons and checkLinks already
    forbids commit permalinks, so a line number in this book is wrong the moment
    they do. The template stays the only code quoted verbatim.
  * one PR: the outline's status table and the link check are only coherent when the
    whole set lands.

scripts/checkChapterPaths.js is the anti-rot half a machine can answer: every path
a chapter names in backticks must exist. None of those mentions is a markdown link,
so checkLinks never looked at them, and none is code, so nothing else did either —
renaming one template file would have left four chapters quietly pointing at
nothing. Its anchor list is STATED rather than derived from the tree, for the reason
the template's own build guard states it: a list derived from what exists cannot
fail when what exists changes, and an anchor that stops matching is a check that has
silently stopped checking. So each anchor must exist or the check fails. Eleven
tests, every "must not catch" case a span that really appears in the book.

stripFences moved to scripts/lib/markdown.js and both checks use it — shared code,
not a shared description.

CHAPTER 1 WAS RUN, NOT REASONED ABOUT. The template was copied into a real core on
edge, booted against the dev database, and every claim in "what you should see"
checked: the five log lines, /examplegame/status with its injected
<script type="module" src="/modules/examplegame/entry.js">, the chunk served
no-cache while module.json 404s, /api/v1/public/world/status, the capabilities in
/api/v1/public/modules, and the route in the merged /api/docs.json. Then the three
failures the chapter tells a reader to cause on purpose, because a chapter that
predicts the wrong debugging heuristic is worse than one that predicts none:

  * an undeclared prefix  -> stage `register`, "declared public/extra but never
    registered it", routes 404 and absent from /public/modules;
  * a table without the id prefix -> stage `schema`, at LOAD time, before mounting;
  * a throwing onBoot -> after mounting, so the same route answers 503 "Module
    unavailable" rather than vanishing.

All three came out exactly as written, and the messages in the chapter are that
core's own. Two small corrections fell out of the run: the log sample now shows the
real interleaving of core's three lines with the module's two, and the section on
failure adds that a module disappears from /api/v1/public/modules in every failure
case — a check that needs no login.

MODULE_SYSTEM.md 2.11.1 slice 2. Docs half: docs#146.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
claude
2026-08-12 13:20:29 -05:00
parent 24b9d30a16
commit f41ff92c67
11 changed files with 1438 additions and 121 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
.gitea/workflows/pr-checks.yml
View File

@@ -15,8 +15,14 @@
# tree, in both directions — an unlisted file that still carries the # tree, in both directions — an unlisted file that still carries the
# placeholder, and a listed file that no longer does, are both failures. That # placeholder, and a listed file that no longer does, are both failures. That
# checklist is the only instruction a reader has for the first thing they do # checklist is the only instruction a reader has for the first thing they do
# with the template, and it is prose, so it rots the way prose does. The two # with the template, and it is prose, so it rots the way prose does.
# checks in `scripts/` have their own unit tests, run in the same job. #
# And it checks that every path the book names in backticks still exists. The
# chapters teach out of `template/`, none of those mentions is a markdown link,
# and nothing else in this repo would ever look at them — so renaming one
# template file would leave four chapters quietly pointing at nothing. That is
# the cheap half of "is the book still true"; the other half is a reviewer's.
# All three checks in `scripts/` have their own unit tests, run in the same job.
# #
# • `template` — the interesting one, and the anti-rot mechanism of the whole # • `template` — the interesting one, and the anti-rot mechanism of the whole
# repo (MODULE_SYSTEM.md §2.11.1 d2). It clones CORE at the ref pinned in # repo (MODULE_SYSTEM.md §2.11.1 d2). It clones CORE at the ref pinned in
@@ -86,11 +92,17 @@ jobs:
- name: Check the rename checklist against the template - name: Check the rename checklist against the template
run: node scripts/checkRenameSites.js run: node scripts/checkRenameSites.js
- name: Check every path the book names still exists
run: node scripts/checkChapterPaths.js
# The checks, checked. A check that has never been shown to fail is a check # The checks, checked. A check that has never been shown to fail is a check
# nobody knows the state of — and this one gates the instructions for the # nobody knows the state of — and these gate the instructions for the first
# first thing a reader does. # thing a reader does. Named file by file rather than `node --test scripts/`:
# directory mode is not portable across the Node versions people run this on.
- name: Test the checks themselves - name: Test the checks themselves
run: node --test scripts/checkRenameSites.test.js run: |
node --test scripts/checkRenameSites.test.js
node --test scripts/checkChapterPaths.test.js
template: template:
runs-on: ubuntu-latest runs-on: ubuntu-latest

11
README.md
View File

@@ -84,9 +84,14 @@ scripts/ the checks CI runs over both
CI clones core at a **pinned commit**, asserts the version the template declares CI clones core at a **pinned commit**, asserts the version the template declares
still matches that core's `MODULE_API_VERSION`, builds the template and runs its still matches that core's `MODULE_API_VERSION`, builds the template and runs its
guards, checks every link in the book, and holds the template's rename checklist guards, checks every link in the book, holds the template's rename checklist
against the template's own tree. So a change to the contract breaks this repo's against the template's own tree, and checks that every path a chapter names is
build loudly instead of leaving a chapter quietly wrong. still there. So a change to the contract breaks this repo's build loudly instead
of leaving a chapter quietly wrong.
None of that can tell you whether a paragraph has become untrue about a file that
still exists. That is a reviewer's job on every pull request, and a
`MODULE_API_VERSION` bump is when it is owed in full.
## Licence ## Licence

244
book/01-first-module.md Normal file
View File

@@ -0,0 +1,244 @@
# 1. Your first module in twenty minutes
No theory in this chapter. You will copy a module that already works, rename it,
build it, install it into a running core, and load a page it serves. Everything
after this chapter is a change to something that runs, rather than a step toward
something that might.
That order is deliberate. The module system has a lot of seams — a server entry
point, a client chunk, a schema fragment, a nav registration, an OpenAPI fragment
— and each one is easy to understand and unpleasant to debug in the abstract. Get
all of them working at once with almost no content in them, and you can then break
exactly one at a time on purpose.
**What you need:** a Runic Gateway core you can restart, Node 20 or newer, and
about twenty minutes. You do not need core's source, and you should not read it —
if this chapter cannot be followed without it, that is a bug in this chapter and
[worth telling us about][issues].
---
## The pieces you are about to copy
`template/` is a whole module, in the shape a real one has. Nine things matter and
the rest is filling:
| Piece | What it is |
| --- | --- |
| `template/module.json` | The first thing core reads. Your id, your version, the core API range you need, and a declaration of every prefix you will mount. |
| `template/server/index.js` | The server-side handshake: one exported function, called once with `(ctx, api)`. |
| `template/server/core.js` | Lazy accessors over `ctx`, so the rest of your server code can reach core the way ordinary code reaches a library. |
| `template/server/boot.js` | `onBoot` and `onShutdown` — where anything needing a live database goes. |
| `template/server/db/schema.sql` | Your tables. Idempotent, replayed at every boot. |
| `template/server/db/purge.sql` | The same tables, dropped. Run only when an operator explicitly purges you. |
| `template/client/src/entry.jsx` | The client-side handshake: registers your routes and your nav rows into core's SPA. |
| `template/client/vite.config.js` | The library build that produces the chunk core serves — and the aliases that make your React core's React. |
| `template/swagger-fragment.json` | Generated. Core merges it into its own API documentation. |
Two of those have a reputation. `vite.config.js` is the highest-risk mechanical
detail in the whole system and chapter 2 spends real time on why; `module.json`'s
`mounts` is the one field people fill in wrong and discover at boot. Neither
matters yet — the template has both right.
## Copy it, and make it yours
```bash
cp -r template/ ~/my-module
cd ~/my-module
```
Your module id is the single most load-bearing string in it: it is the directory
core loads you from, the key in core's database, the URL segment every one of your
pages hangs under, and the prefix every one of your tables must carry. It must
match `^[a-z][a-z0-9-]{1,31}$`, and you want no hyphen in it unless you enjoy
backticking table names.
Change `id` in `module.json` first, then work down the checklist in
`template/README.md` — it names every file that still carries the placeholder,
and it is [verified by CI][renamecheck] in both directions, so it is not the kind
of checklist that is wrong by the second edit.
**The placeholder is `examplegame`, not `example`, and that is not an aesthetic
choice.** A check for a leftover `example` fires on the phrase "for example" in
ordinary prose, and a check that cries wolf is a check people learn to ignore. If
you build your own checks later, pick placeholder names that cannot occur by
accident.
## Build it
```bash
npm ci --prefix server
npm test --prefix server
npm ci --prefix client
npm run build --prefix client # → client/dist/entry.js
npm test --prefix client
```
Build **before** you run the client tests. Two of them read the built chunk and
skip when there is none, so a run in the other order passes while asking nothing
about the artifact that actually ships. That ordering has bitten this project
twice in two different repositories, which is why it is called out here rather
than left to a CI file.
What you have now is `client/dist/entry.js` — a prebuilt ES module — and a server
tree that has never been compiled at all, because it does not need to be.
**An operator never builds anything.** That is the constraint the whole delivery
path is designed around: a module arrives as a tarball with the chunk already in
it, and core serves that file untouched. Your build machine is the only place a
bundler ever runs.
## Install it
Three supported ways, and for the next twenty minutes you want the third:
1. **Admin → Modules**, pasting the URL of an install manifest — the JSON your
release workflow publishes beside your tarball. This is how a real operator
installs you.
2. **The `MODULES` environment variable**, `<id>@<version>=<manifest URL>`, for a
deployment that declares its module set instead of clicking it.
3. **A directory on the volume.** Copy your whole module tree to
`<website>/modules/<your-id>/` and restart core.
```bash
cp -r ~/my-module <website>/modules/my-id
# restart core
```
**Copy it. Do not symlink it.** The loader lists directory entries and asks each
whether it is a directory; a symlink answers no, and your module is skipped in
complete silence. This is the single most common way a first install appears to do
nothing at all.
Two more things that look like your module failing and are not:
- If core is running in a container, your files have to be on the volume core sees
— `MODULES_DIR` (`/app/modules` under the shipped Compose file), not the
repository directory next to it.
- A core with a fresh database boots in **maintenance mode**, and public module
pages sit behind the same maintenance gate core's own do. Your page will look
broken while the site is not live yet.
## What you should see
Restart core and read the log. A module that loaded says so:
```
INFO [examplegame] registered {"version":"0.1.0","routes":"public:/world"}
INFO [modules] registered module "examplegame" v0.1.0 {"mounts":{"public":["/world"]}}
INFO [modules] schema ensured for module "examplegame" {"statements":2}
INFO [examplegame:boot] booted {"refreshMs":30000}
INFO [modules] module "examplegame" started
```
Your two lines and core's three, interleaved: core narrates each step of your load
in its own `[modules]` namespace, and your logger is namespaced with your id. That
alternation is the quickest way to see how far a load got.
Then, in the browser:
- **`/examplegame/status`** renders your page, with a **World** row in the public
header pointing at it. That row is now an ordinary nav row: an operator can
reorder it, relabel it or hide it from the nav editor exactly as they can core's.
- **`/api/v1/public/world/status`** answers JSON.
- **`/api/v1/public/modules`** lists you, with the `capabilities` array from your
`module.json`. This is how a client — core's SPA, the Android app, anything —
feature-detects you.
- **`/api/docs`** shows your route under its own tag, merged out of the OpenAPI
fragment you committed. (`/api/docs.json` is the raw merged document, if you
would rather grep it.)
- **Admin → Modules** shows you as `started`.
Open the browser console while you are there. Your entry logs the core API version
it registered against, and any complaint the client half has to make will be sitting
next to it.
## The state your module is in
Core keeps one row per module and its `state` column has five values. Four are
outcomes and one is an operator's decision:
| State | Means |
| --- | --- |
| `installed` | Files are on the volume; the row was just created. |
| `enabled` | Cleared for this boot to try. Every non-disabled row is reset to this at each boot. |
| `started` | Loaded, registered, schema replayed, `onBoot` returned. This is the one you want. |
| `startup_failed` | Something went wrong; the panel shows the stage and the reason. The site came up anyway. |
| `disabled` | An operator switched you off. Nothing else — not a failure, not a reinstall — moves this. |
The important half of that table is what it implies: **a module that fails to
load never takes the site down.** Core try/catches your entire lifecycle, records
where you broke, and serves everything else. You are debugging from an admin
screen, not from a stack trace in a crash loop.
**A retry is a restart.** Every boot resets non-disabled rows to `enabled` and
writes that boot's outcome, so the panel always describes the run you are looking
at rather than a run from last week.
## The four ways it fails
When something is wrong, the shape of the failure tells you where to look before
you read a single message.
**1. Your module is not in the panel at all.** The loader never saw a directory
worth scanning. It is a symlink; or it is in the wrong place; or it has no
`module.json` at the top of it. Note the bundle shape here — a release tarball's
top-level directory is `<name>-<version>`, so an unpacked bundle copied wholesale
leaves core looking at a directory with nothing in it but another directory.
**2. It is `startup_failed`, and your routes and nav are simply absent.** The
failure happened before anything was mounted: a malformed `module.json`, an
unsatisfiable `coreApi`, a prefix that collides with core's, a schema fragment
breaking a rule. Nothing of yours is on the URL surface, so nothing of yours can
half-work.
**3. It is `startup_failed`, and your routes answer `503`.** The failure happened
after mounting — the database rejected a statement in your fragment, or your
`onBoot` threw. Your routes stay mounted deliberately: the URL surface is a
property of what is installed, not of whether a boot hook succeeded on this
machine. A module that failed to warm up says it is down; it does not serve half
its data.
**4. It answers `404` everywhere.** Someone disabled you. Same mechanism — mounted
and guarded, never unmounted.
The panel names the stage each failure happened in, and the stages are the
loader's own validation steps, listed in [`MODULE_API.md`][api] §4.3 and §4.4. Read
the stage first; it is usually enough. Core logs the same thing at boot —
`module "…" failed to load — continuing without it {"stage":…,"reason":…}` — so you
do not need the panel to debug this.
**In all four cases you disappear from `/api/v1/public/modules`.** That endpoint
answers what this backend is *serving*, so a client feature-detecting your
capability renders a site without it rather than one advertising something that
`503`s. It is also a quick check with no login: if you are not in that list, you
are not running, whatever the page looks like.
## What to do next
You have a module. Now break it on purpose, once each, and watch what the panel
says:
- Add a prefix to `module.json`'s `mounts` and do not register it. → stage
`register`, *"declared public/extra but never registered it"*. What you declared
and what you registered must match, in both directions.
- Rename one of your tables so it no longer starts with your id. → stage `schema`,
at **load** time, before anything is mounted: your routes answer `404`.
- Throw inside `onBoot`. → after mounting, so the same route answers `503` with
*"Module unavailable"* instead of vanishing.
Those are the three outcomes above, and the messages are what this core actually
prints for them — they were run to write this paragraph rather than predicted.
Twenty minutes of that is worth more than any chapter, because every one of those
failures is one you will cause accidentally later, and you will recognise it.
Then read [chapter 2](02-website-module.md), which is the same module explained —
what `ctx` hands you and why it is handed rather than imported, what each
`register*` call is for, why the client half is built the way it is, and what a
module must never do.
[api]: https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_API.md
[issues]: https://gitea.whitlocktech.com/RunicGateway/Integration-kit/issues
[renamecheck]: ../scripts/checkRenameSites.js

487
book/02-website-module.md Normal file
View File

@@ -0,0 +1,487 @@
# 2. The website module
The module you built in [chapter 1](01-first-module.md), explained. This is the
longest chapter in the book because the website module is most of the work, and
because almost every part of it is shaped by a constraint that is invisible until
you hit it.
Nothing here is normative. [`MODULE_API.md`][api] is the contract; where this
chapter and the contract disagree, the contract is right and this chapter has a
bug. What is here is the reasoning — which is exactly what a contract cannot carry
without becoming unreadable.
---
## The shape of the whole thing
A module is a directory core reads at boot. Core loads it, hands it two objects,
and takes back whatever it registers.
```
core boots (its own schema and seed have already run)
└─ scans modules/*/module.json
└─ validates yours ← nothing mounted yet: a failure here leaves
│ nothing of yours on the URL surface at all
└─ require(server entry)
└─ register(ctx, api) ← your one synchronous handshake
└─ second pass: mounts everything that survived
└─ replays your schema fragment
└─ onBoot(ctx) ← the first moment a database exists
└─ HTTP listener binds
```
Two properties of that sequence explain most of the rules that follow.
**It is synchronous and it happens during `require`.** Core's route-manifest
generator and its OpenAPI generator both require the app with the database pool
pointed at a dead port — that is how they introspect a real Express app without a
database. So `register()` may not `await` and may not query. A module that did
would hang both build tools, and the symptom would be a CI job that never
finishes rather than an error anyone can read.
**Mounting is a second pass.** Every module is validated before any module is
mounted. If mounting happened inside the scan loop, the first module's layers
would be sitting on the tier router while the second was validated —
indistinguishable from core's own — and the second would be told it collided with
*core*, naming the wrong culprit. You will never see this; it is why the failure
messages you do see are trustworthy.
## `module.json`
Every field is documented in [§2.1][api]. Three of them decide whether your module
loads at all.
**`id`** is the directory core loads you from, the key of your database row, the
URL segment your pages hang under, and the required prefix of every table you
create. It must equal its own directory name — a module renamed by copying it to a
different directory is rejected rather than quietly mounted under a name nothing
else agrees with.
**`coreApi`** is a semver range against core's `MODULE_API_VERSION`. Set it to the
version you developed against and let it drift upward deliberately. This is the
one number that decides whether a module written today loads against a core
shipped next year, and a range that is too loose does not fail — it half-works.
**`mounts`** declares every prefix you will register, per tier. **The loader
compares it with what you actually register and rejects a mismatch in both
directions.** A prefix you declared and never registered fails just as loudly as a
route you registered without declaring. That is the point: the file is a statement
of your URL surface that cannot rot, because it is checked against reality at
every boot.
**Choosing prefixes is the part to slow down on.** They share one namespace with
core's own, so `/status` is not available to you — and the loader's collision
probe cannot see all of core's, because several of core's endpoints are mounted at
the tier root rather than under a prefix of their own. `template/server/index.js`
carries the current list of what core answers on the public tier in a comment
beside the registration. Read it before you choose, and choose a noun from your
own domain rather than a generic one.
`capabilities` is the opposite kind of field: opaque strings core never
interprets, published by `GET /api/v1/public/modules` while you are `started`, so
that a client — the SPA, the Android app — can feature-detect you. Two modules may
declare the same one. A client must treat an unknown capability as absent, and must
never infer a URL from one.
## The server entry point
One exported function, called once: `register(ctx, api)`. `ctx` is what core hands
you; `api` is what you hand back. Read `template/server/index.js` — it is short,
and every comment in it is load-bearing.
### Why `ctx` is handed over rather than imported
Your module lives at `<website>/modules/<id>/`, outside core's `server/`. Node's
resolver walks *up* from a file looking for `node_modules`, so it never reaches
core's — and `require('express')` from inside a module simply fails.
That is the mechanical reason, and it is the shallow one. The real reason is that
there is exactly one of certain things in the process and core owns them: one
express, so there is one `Router` prototype; one database pool; one logger; one
session reader. A second express resolved from your own dependencies would work
for about a week and then produce a routing bug nobody can reproduce.
So the rule generalises past the two obvious cases: **anything shared between core
and a module is owned by core and handed over, never resolved by the module.** On
the server that is `express` and `express-validator`; on the client it is React,
`react-dom`, `react-router-dom` and the JSX runtime. Both halves of the system
have one mechanism for it, and it is the same rule twice.
[§2.3][api] lists every member of `ctx`. It is a curated list, not core's
internals: `ctx.auth` is one function rather than core's whole auth facade,
because minting sessions is core's job and a module that needs an identity needs
to *read* one. `ctx.settings` is three functions rather than a settings model with
two dozen. Expect the narrowing, and expect to occasionally want something that is
not there — that is a conversation about a minor version bump, not a reason to
reach around it.
### The lazy-accessor pattern, and the require order it forces
`ctx` exists only from the moment `register()` is called. But the code underneath
— models, controllers, routers — is ordinary Node that requires its dependencies
at file scope, and *that* runs before `register()` does.
`template/server/core.js` is what makes both true at once: every member is an
accessor that resolves `ctx` **when it is called**, so a model can write
`const { query } = require('../../core')` at the top of the file, exactly as
ordinary code does.
Two consequences, and both have cost this project time:
**Require order is load-bearing.** A router writes `const express = core.express`
at *its* file scope, and that runs the moment the router is required. So
`core.init(ctx)` has to happen before the first `require` of anything under
`router/`. This is why `template/server/index.js` requires its routers *inside*
the register function instead of at the top of the file. Hoist them and the module
breaks with an error about a missing `ctx`, thrown from a file that never mentions
one.
**Never destructure a getter at init time.** Core is free to hand over an accessor
rather than a value — `ctx.site.baseUrl` is one — and a value captured once at
startup is a value that cannot change afterwards.
`template/server/core.js` is also deliberately a *narrowing*: it re-exports only
what the module actually uses. Copy that discipline. It makes the file an honest
statement of your dependencies, and it makes a test double for it — see
`template/server/test/_fakes.js` — a complete one rather than a guess.
## What you register
Seven calls, all synchronous, all documented in [§2.4][api]. What is worth knowing
is not their signatures but the model behind them.
**Every call stages; nothing is committed until your whole module is known good.**
The shape of a claim is checked at the call, so a malformed one throws with your
own stack trace. Whether a *name* is taken can only be answered once your batch is
complete, and is checked when the loader commits it. So a module that registers two
notification streams and then throws leaves nothing behind. That matters more than
it sounds: a half-registered catalog is a stream a user can subscribe to and
nothing will ever publish to, which is worse than a missing one because it looks
like it works.
### Routes
`api.registerRoutes({ public, admin, player })` — one router per prefix per tier.
**The tier gate is already applied.** A router registered under `admin` sits
behind core's own `noindex, isLoggedIn, requireRole(...)`; under `player`, behind
`noindex, requireAuth`; under `public`, behind nothing, by design. You add
per-route gates on top of that and you never re-implement the tier gate. A module
cannot supply its own auth wrapper, and that restriction is one of the few places
the boundary is genuinely load-bearing rather than organisational: the server's
route table and the client's sidebar have to agree about who may see what, and
they only do if one thing decides.
Your router is mounted *inside* the tier router, so it structurally cannot reach
above its own prefix. This is not enforcement by review; there is no path
expressible from inside your router that escapes it.
### Extension slots
Sometimes what you have to add is not a page of your own but a section of core's.
An operator looking at a user in the admin panel wants that user's characters
right there, not on a separate screen.
`api.registerExtension(slot, router)` mounts your routes under a core resource,
and its client twin renders your component inside a core page. **Only core may
declare a slot; a module may only fill one**, and one module per slot.
The naming rule is worth internalising, because it is what keeps a game-agnostic
core game-agnostic: **a slot is named for a PLACE, never for a meaning.**
`site.footer.status` is "the status-ish spot in the footer" — not a declaration
that core knows what a game server's status is. Core supplies the position and the
styling; the module owns the label, the target, the data, and whether it renders
anything at all. The moment core types a slot by its content, it has re-acquired
the semantics the module system exists to remove.
### Notification streams, announce legs, post hooks
Three registries for three genuinely different things, and the distinctions are
easy to get wrong:
- **A notification stream** is a subscribable channel. You register the catalog
entry — id, label, whether it is personal, whether it needs a linked game
account — and core uses it for the subscribe endpoint and its gates. You publish
to it yourself with `ctx.push.publish`. Core never maps your events to your
streams; you have already resolved the id, and it follows that the safety rule
about which of your events may reach a *public* stream lives in your module too
— which is right, because the event kinds, the stream list and the filter are
then one file that moves together.
- **An announce leg** is one-shot delivery with retry. Core's CMS publishes a post,
every registered leg tries to deliver it somewhere, and your `classify` maps your
own result to `done` / `retry` / `terminal`. A leg that throws is caught,
classified as a retry, and never blocks another leg.
- **A post hook** maintains idempotent state, runs on delete as well as save, and
refreshes silently on an edit.
The last two fire on the same transition and are deliberately not one call. A leg
that must not be retried and a `classify` that means nothing would be the cost of
merging them.
Every hook is awaited and none may throw past core: a subscriber's failure costs
neither another subscriber nor the save itself. A hiccup in your sidecar breaking
somebody's blog post edit would be a worse bug than a stale mirror.
### The lifecycle hooks
`api.onBoot(fn)` runs after core's schema, after your schema fragment, and
**before the HTTP listener binds**. It is the first moment a database exists, so
it is where everything that needs one goes: warming a cache, backfilling,
connecting to your sidecar.
**`onBoot` has no timeout, deliberately.** A slow boot delays the listener, and
that is the guarantee rather than a problem to be timed out — a module that must
not serve traffic before it has warmed up gets exactly that. If your `onBoot`
throws, you are `startup_failed`: your routes stay mounted and answer `503`, and
the site comes up without you.
`api.onShutdown(fn)` runs while core's pool, push dispatcher and event fan-out are
all still open, because flushing through them is the only thing it is for. It has
a five-second budget and is abandoned past it — the process is exiting anyway, and
the alternative is a host where stopping the service waits for a kill.
A module whose `onBoot` threw gets **no** `onShutdown`. It is part-way through a
warm-up it never finished, and handing it a half-built world to tear down is worse
than not closing cleanly. A module with no hooks at all still reaches `started`:
having nothing to warm up is not the same as never having started.
## The schema fragment
`template/server/db/schema.sql` is your tables. It is replayed **in full, at every
boot**, statement by statement, right after core's own schema.
**There is no migration runner anywhere in this project, and that is a decision
rather than an omission.** Core's own schema is one idempotent file replayed the
same way. What you get in exchange is that a module's schema is a single readable
statement of what its tables are, with no ordering history to reconstruct and no
migration table to get out of step with the tables themselves.
What it costs you is that **changing a table is an `ALTER`, never an edit to its
`CREATE`.** `CREATE TABLE IF NOT EXISTS` no-ops against an existing table, so an
edited column definition lands on fresh installs only — and your development
database is usually the fresh one, which is what makes this bite six months later
on somebody else's instance. Add the column with
`ALTER TABLE … ADD COLUMN IF NOT EXISTS`, leave the `CREATE` alone, and both paths
converge.
Two rules the loader enforces before your module is mounted at all:
**Leading verbs are an allowlist: `CREATE`, `ALTER`, `INSERT`, `UPDATE`.** Not a
`DROP` denylist — because the file is replayed at every boot, `TRUNCATE` and
`DELETE` would empty a table at every restart and `RENAME` would fail at the
second one. A denylist only ever bans what somebody thought of.
**Table names are namespaced `<id>_` and collision-checked** against core's tables
and every other module's. A `CREATE TABLE` missing `IF NOT EXISTS` is rejected on
the same grounds as the rest: it succeeds exactly once and fails every boot after,
which presents to an operator as a module that broke on restart.
Both are checked by *reading the file*, before anything mounts, and that split is
the design: everything knowable without a database costs you the mount, so a
rule-breaking fragment never half-applies; what only a database can answer — an
unknown column type, a bad foreign key — happens later and answers `503`.
`purge.sql` is the destructive counterpart, and it is required whenever you ship a
schema. It runs **only** when an operator explicitly purges you, never on
uninstall. A module that can create tables and cannot drop them leaves an operator
with orphaned data and no supported way to remove it.
One more thing about a file that replays: **a guard and the statement it guards
must live in the same file.** If you write a one-shot data fix conditioned on a
marker, put both the marker and the fix in your own fragment. Core's schema
replays in full before any module's, so a marker core writes has already been
written by the time your guard reads it — a real defect this project shipped and
did not notice, because it is latent until the day someone installs on an older
version.
## The client half
Your client half is a **prebuilt ES module**. Core serves it from your module's
directory as a same-origin script and injects a `<script type="module" src>` for
it before `</body>`. There is no bundling step on the operator's machine, ever.
### One React, and core owns it
`window.__rg` is core's published set of shared dependencies plus the registry,
the UI kit and a request primitive ([§3.2][api]). Your build does not bundle React
— it aliases every shared specifier to a two-line shim that re-exports from that
global.
The failure this prevents is specific and nasty: a second React in the page is a
second hook dispatcher, so your component throws about an invalid hook call
somewhere unrelated to the mistake, in a page that otherwise loads fine.
`template/client/vite.config.js` is the whole mechanism, and its comments are the
most valuable prose in the template. Three things there were wrong first and are
now contract:
- **The aliases use the array form with anchored regexes.** Vite's object form does
prefix matching, so a `react` key also silently rewrites `react/jsx-runtime` — to
the wrong shim.
- **The aliases replace `external`; they do not accompany it.** Rollup asks
`external` *before* Vite's alias resolver runs, so a specifier in both is never
aliased and the chunk ships bare `import 'react'` specifiers. A browser cannot
resolve those without an import map, and core's `script-src 'self'` forbids the
inline script an import map has to be. The first real module shipped exactly that
chunk, from a clean green build.
- **The build guard hooks `transform`, not `load`**, and its forbidden-package list
is stated rather than derived from the alias list. `load` is first-wins, so
written against it the guard sat in the build doing nothing. Deriving the list
means deleting an alias also deletes the guard against what that alias prevented
— precisely when it is needed.
`template/client/scripts/checkExternals.js` asks the **built chunk** whether any
bare specifier survived. That question cannot be asked of source:
`import { useState } from 'react'` is correct in every file, and which React it
becomes is decided by the build config. Run it in your CI.
### Registration happens at evaluation time
`template/client/src/entry.jsx` registers your routes and nav rows with plain
top-level calls. There is no subscription and no late registration: module chunks
are deferred scripts that execute after core's bundle and before core's first
render, so everything you register is present in that first render.
**So every page is a static import, and lazy-loading your routes is the one thing
this seam cannot have.** A module that registered asynchronously would register
after the route table had been read, and the symptom is a page that redirects home
with nothing logged anywhere — indistinguishable from a module that failed to
load.
That timing is also where this project's most instructive client-side bug lived.
Core's own render used to wait on `document.readyState === 'loading'`; but a
deferred script runs *after* the document is parsed, so `readyState` is already
`'interactive'`, and core mounted immediately — before any module chunk had
evaluated. Every unit test passed. It was found by loading a real chunk in a real
browser, which is the only place it was visible.
### Nav, and what a registered row becomes
`registry.registerNav` interleaves your rows into **core's** navigation groups,
and from that moment your row is an ordinary row: an operator can reorder,
relabel or hide it from the nav editor exactly as they can core's. That works
because the interleave happens *before* the admin override merge — the override
layer is keyed by a row's `to`, and it drops keys its base does not declare, so a
row appended afterwards would be unorderable, unrelabellable and unhideable.
Three details worth knowing before you need them:
- **A row with no `order` appends after core's rows** rather than defaulting to
zero. "I didn't ask for a position" must not mean "put me first".
- **An unknown `group` name appends a new group** rather than dropping your row.
- **`icon` has no core fallback.** Public header rows carry no icons, so a public
row needs none; an admin or player row without one is the only glyph-less row in
its sidebar, which reads as breakage. Match the nav you land in rather than
shipping one glyph for everywhere.
`registerFeatureProvider` is how a row can be conditional: core keeps a generic
flag context and you supply the hook that fills your namespace. **The namespace
comes from the registration, not from parsing the string**, so a typo'd namespace
is not a thing that can exist.
**Everything in this layer fails open.** No provider, an answer still in flight, a
malformed row — all of them show the link. This is presentation and the server is
the gate: a UI mistake that hides a page from someone entitled to it is worse in
every case than one that shows a link which then answers `403`.
### The UI kit
Core publishes a small set of components and hooks on `window.__rg.ui`
([§3.4][api] is the list): the public layout, a page header, the loading, error
and empty states, the async hook every data page uses, and read-only access to the
session and site settings. Enough to build a page that looks like the site it is
installed in, and nothing else.
**It is curated and closed, not a re-export of core's component library.** Adding
to it is a minor version bump; changing a kit component's props is a major one.
That is a real constraint on core, and it is the price of the boundary being worth
anything.
So: when you want an eighth thing, bundle it. Tables, chips, tabs, editors — those
are yours, and your chunk carries them. Reaching into core's tree for a component
is the one thing that is never available, and `template/server/scripts/checkImports.js`
exists to make sure a moment of weakness fails the build instead of shipping.
One thing that surprises everyone once: **core's public pages render the public
layout themselves** — it is a component, not a route wrapper. A public page of
yours that does not use `ui.PublicLayout` renders bare, with no site chrome. That
is the contract working as intended, not a bug to hunt.
## The OpenAPI fragment
Every module that registers routes ships `swagger-fragment.json` in its bundle
root, and core merges the fragments of started modules into its own API document.
The filename is fixed rather than declared, like `module.json` itself.
**Generate it from your own registrations** — `template/server/scripts/swaggerFragment.js`
is a working generator. It runs your `register()` against a recording `api` and
resolves each router back to its source file, so a mount prefix exists in exactly
one place rather than being retyped into a generator that then drifts.
Two rules and one trap:
**Namespace what you define; reference core's shared schemas by core's name.** A
schema you invented gets your prefix. `Error` and `ValidationError` are core's:
reference them and do not redefine them. They resolve in the merged document,
which is the only place both halves exist — and shipping your own copy is a
collision core drops, arriving at the same result the expensive way.
**Commit the generated file and check it is current in CI.** Core merges it
verbatim, so a stale fragment documents a URL surface you do not serve, and
nothing at runtime will ever say so.
The trap: **swagger-autogen reports a broken annotation and then prints
`Success`.** It logs a syntax error, drops that annotation, and exits zero. The
template's generator captures those diagnostics and fails on them — keep that.
Two ways an annotation breaks are an object literal one brace short, and a `"` or
a backtick inside a single-quoted description. A third is only visible in a
rendered page: an escaped apostrophe (`\'`) survives literally into the output,
because the annotation is never evaluated as JavaScript. Use a typographic `’`.
## Packaging and release
`template/.gitea/workflows/release.yml` (and its GitHub twin) is a working release
pipeline. Copy it to the root of your module's repository — a workflow file is
only read from a repository root, which is why it does nothing where it sits
inside the kit.
**A release is not source.** It is the directory core's loader expects to find at
`modules/<id>/`, already assembled: the prebuilt chunk, your runtime dependencies
installed, the schema fragment, the OpenAPI fragment. Core downloads the tarball,
verifies it against the `sha256` in the install manifest, and unpacks it. Nothing
runs `npm` on the way.
**The version is declared in `module.json`, not computed from commit subjects.**
You already have one authoritative version — it is what core records and what the
admin panel shows — and two sources for one number is how they drift. A release
happens when a push to `main` leaves a version that has no release yet, so
bumping is an ordinary reviewed change and publishing is the workflow's business.
The workflow tags and publishes and never writes to a branch, so a protected
`main` needs no exception.
The install manifest is the JSON your workflow publishes beside the tarball. Its
URL is what an operator pastes into Admin → Modules, and the host it lives on has
to be on that core's allowlist — an operator-controlled setting, so tell your users
where you publish.
## Boundaries
[§2.7][api] is the list. Each item has a failure behind it:
| Rule | What it prevents |
| --- | --- |
| No `require` outside your own directory (bar built-ins and your own dependencies) | Two copies of a thing there must be one of; and a module that survives a core refactor only by luck. |
| Do not mutate `ctx`, `req.user`, or anything core handed you | A module changing another module's world, invisibly. |
| No app-level middleware, no Express error handler | One module deciding how every other module's errors are rendered. |
| Do not read `process.env` for core configuration | Configuration with two sources and no panel. Your own config is a settings key or your own table. |
| No `process.exit`, no signal handlers, no listeners | A module taking the site down, or racing core's shutdown. |
| Write only inside your module root and the upload directory | A module that cannot be uninstalled cleanly. |
| **Never open a connection to a game server from the website process** | The whole of [chapter 3](03-sidecar.md). |
That last one is newer than the others and is the reason this kit is three
chapters and not one. It is also the only rule in the list with **no CI behind
it** — an outbound socket is not statically detectable the way an internal
`require` is — so it is enforced in review and by understanding it, which is what
the next chapter is for.
[api]: https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_API.md

185
book/03-sidecar.md Normal file
View File

@@ -0,0 +1,185 @@
# 3. The sidecar
Your module may not open a connection to a game server. Not a game socket, not an
RCON channel, not a query port, not an engine's admin API. It talks to a
**sidecar**, and the sidecar talks to the game.
That is a rule in the contract ([`MODULE_API.md`][api] §2.7, as of
`MODULE_API_VERSION` 1.4.0) rather than advice this kit is offering. It is also
the rule most likely to feel like ceremony when your game already exposes a
perfectly good remote-control protocol and your module is fifty lines from
working. This chapter is why it is not.
**It is the one rule in that list with no CI behind it.** An outbound socket is
not statically detectable the way an internal `require` is. So it is enforced by
review, and by you having read this.
---
## What a sidecar is
A small, separate service that owns the connection to your game, keeps a durable
copy of what the game said, and exposes an HTTP + WebSocket API that the website's
backend reads.
```
your game server ──dials out──▶ your sidecar ──HTTP + WS──▶ website core
│ (your module)
▼
its own store
```
Three properties, and each is doing real work.
## 1. The game dials out; the sidecar listens
The sidecar binds the listener. The game connects **to it**, and the game opens no
listening port at all.
This is the inversion people find surprising and it is the load-bearing one. The
website is the internet-facing process; your game is not, and must not become
reachable because a web app knows how to reach it. A module holding the connection
makes the public web app the thing the game trusts, and puts the game's address
inside the same process as every request from the internet.
In `uo-link`, that listener is `sidecar/src/shard.rs` — `serve` binds a loopback
address and accepts shard connections forever, handling one at a time and looping
back to accept the next. The game plugin does the dialling, with its own backoff.
Loopback, in that deployment, because the sidecar runs on the game host: the only
socket the game speaks over never leaves the machine.
Only the website's backend talks to the sidecar, and it authenticates. `uo-link`'s
`web.rs` requires a token on every request — accepted as a bearer header, an API-key
header, or a query parameter, that last one only because browser WebSocket clients
cannot set handshake headers — and compares it in constant time. Auth is always on;
there is no unauthenticated mode to accidentally deploy.
## 2. Persist before you forward
This is the property that makes a sidecar worth having even when your game is
already remote-controllable, and the one a message-passing diagram never conveys.
**The sidecar owns the durable copy.** It writes what the game said into its own
store, and answers reads from that store — not by round-tripping the game.
`uo-link` does this in `sidecar/src/store.rs`: SQLite, holding event history, the
latest snapshot of every board the site renders, the economy series and the
published ruleset. `insert_event` is called for every live event as it is
broadcast; the `upsert_*` functions keep one current row per board; the REST read
paths query that store.
What it buys, concretely:
- **A website that is down, restarting or mid-deploy loses nothing.** Events that
arrive while nothing is listening are still recorded. Without a store they are
simply gone, and your first deploy of the week is a hole in your data.
- **A page renders the last thing the game said rather than going blank.** A rules
page that empties itself because the game restarted is worse than a stale one.
- **The live feed is allowed to be lossy.** `uo-link`'s WebSocket fan-out drops
frames for a consumer that has fallen behind and logs that it did — deliberately,
because durability is the store's job and not the socket's. A feed that instead
buffered without limit for a slow client would eventually take the sidecar down.
That last point is the reasoning to carry into your own design. Once the store is
authoritative, every other component is allowed to be best-effort, and each of them
gets simpler. Skip the store and you find yourself trying to make a socket reliable,
which is the hard version of this problem.
A module cannot do any of this from inside the website process. There is nowhere to
put what arrives while the website is not running, because the website not running
is exactly the case.
## 3. The wire is a versioned contract, not a build dependency
Your sidecar and your module ship separately, on different schedules, to hosts you
do not control. So the wire between them is a compatibility contract with a version
on it.
`uo-link` declares `PROTOCOL_VERSION` in `sidecar/src/main.rs`, stamps
`X-UOLink-Version` onto every response from `web.rs`, and **refuses a request whose
declared version does not match** rather than parsing it optimistically. A refusal
is a clear failure an operator can act on; a mis-parse is a wrong number on a page
with nobody to tell.
Two habits come with that:
- **Bump the version in the same change that changes a message shape**, on every
side that declares it. In this project a protocol bump has three declaration
sites — the sidecar, the game-side overlay's manifest, and the documented spec —
and the tooling refuses to pair components that disagree.
- **Version the *shape*, not the content.** Adding a new event kind that old
consumers ignore is not a break. Changing what a field means is, even when the
JSON still parses.
## The worked example
`uo-link` is a complete implementation of everything above, and it is small enough
to read:
| File | What it owns |
| --- | --- |
| `sidecar/src/shard.rs` | The listener the game dials into; one connection at a time, then accept the next. |
| `sidecar/src/store.rs` | SQLite: event history, per-board snapshots, the series and the ruleset. |
| `sidecar/src/web.rs` | HTTP + WebSocket for the website, the auth middleware, the version header and the lossy live fan-out. |
| `sidecar/src/rpc.rs` | Request/reply correlation, so a website read can ask the game a question and match the answer. |
| `sidecar/src/config.rs` | The config file, including a token generated on first run rather than defaulted. |
The protocol it speaks is specified in [`link/PLAN.md`][linkplan] and
[`link/INTEGRATION.md`][linkint]. Those are normative for that sidecar; your game
is not Ultima Online and your messages will not be its messages. What transfers is
the structure — a listener the game dials into, a store written before anything is
forwarded, a lossy live feed, an authenticated read API with a version on it.
## "But my game already speaks a remote-control protocol"
Then your sidecar is **thin**, not absent.
Rust — the survival game — is the worked example here, in
[`rust-dryrun.md`][dryrun]: a module designed on paper for a game chosen for how
little it shares with Ultima Online. Rust ships RCON over WebSocket, so a
`rust-link` has no protocol to invent and no game-side plugin to write at all. It
keeps:
- the RCON connection, its credentials and its reconnect loop, **out of an Express
process** — where the failure mode is a wedged request handler;
- a store, so the site is not blank whenever the game is restarting, which for that
genre is a daily scheduled event;
- an HTTP + WS API with a version on it, so the module talks to one shape of thing
regardless of what the game speaks.
It drops the bespoke wire protocol and the plugin. That is what "thin" means: less
code, not a different architecture.
That document originally concluded the opposite — "no sidecar, the module dials
RCON directly" — and it carries a dated correction saying so, rather than having
been quietly rewritten. The value of a dry run is the record of what it found,
including where it was overruled.
## Building yours
There is no template for a sidecar in this kit; it is your program, in your
language, and the surface it must expose is the surface your module reads. What to
settle before writing code:
1. **Which direction does the connection go?** The game dials out. If your game
cannot — if it only accepts connections — then your sidecar is the client to the
game and the listener for the website, and the rule that stands is the one that
matters: the address of the game is known to the sidecar and to nothing else.
2. **What is durable?** Everything a page must still render when the game is down.
Write it before you forward it.
3. **What is a snapshot and what is an event?** They are different storage
problems: an event is appended and read back as history, a board is one current
row per subject that you overwrite. `uo-link`'s store holds both, and keeping
them separate is why a restart does not replay a year of events at a page.
4. **What is the version, and where is it declared?** One place, on every response,
refused on mismatch.
5. **How does the website authenticate?** A token, generated rather than defaulted,
always required.
Then chapter 4, if your game needs code inside it — which is the part where getting
it wrong takes the game down rather than the website.
[api]: https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_API.md
[linkplan]: https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/link/PLAN.md
[linkint]: https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/link/INTEGRATION.md
[dryrun]: https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/modules/rust-dryrun.md

179
book/04-game-plugin.md Normal file
View File

@@ -0,0 +1,179 @@
# 4. The game-side plugin
The chapter with the least code and the highest stakes. Everything else in this
book fails by showing an operator a broken web page; this part fails by taking the
game down while people are playing it.
If your game already speaks a remote-control protocol, you may not need any of
this — see the end of [chapter 3](03-sidecar.md). If it does not, something has to
run inside the game and feed your sidecar, and the rules below are what keep that
something from being the reason the server froze.
The worked example is `servuo-plugins`, the Ultima Online shard plugin, whose link
layer is one file: `overlay/Scripts/Custom/Bridge/BridgeLink.cs`. It is C# against
a specific game engine and none of that transfers. The threading contract at the
top of it does, entirely.
---
## The one rule: never block the game
A game server is a loop. Whatever thread runs the world is the thread that must
not stop, and every rule in this chapter is a restatement of that.
**Emitting an event must enqueue and return.** It formats nothing expensive, waits
on nothing, and touches no socket. In `BridgeLink.cs`, `Emit` is called from the
game's own thread, appends a line to a queue, signals a waiting writer, and
returns. A sidecar that is slow, wedged, restarting or entirely absent cannot stall
the game, because the game never touches the connection.
The failure this prevents is not hypothetical, and it is not a small one: a socket
write from the world thread against a peer that has stopped reading blocks until
the OS buffer drains. That is a frozen game server, caused by a monitoring
feature, at exactly the moment something else is already wrong.
## The queue is bounded, and it drops the oldest
An unbounded queue in front of an absent consumer is a memory leak with a delay
timer on it. So the queue has a cap, and when it is full **the oldest record is
dropped and counted**.
`Emit` bounds first and enqueues second, so the queue can sit transiently one over
the cap and never grows without limit. `BridgeLink` exposes counters — sent,
dropped, received, connects, write errors, current depth — and those counters are
what an operator debugs from later.
Dropping is correct here, and it is worth being explicit about why: **telemetry is
worth less than the game's memory.** If your sidecar has been unreachable for ten
minutes, the useful thing is the most recent state of the world, not a
ten-minute-old backlog delivered before it. Newest-wins is the honest policy, and
"stall the game rather than lose an event" is never the trade to make.
Where losing events is genuinely unacceptable, the answer is the sidecar's store
([chapter 3](03-sidecar.md)), not a bigger queue inside the game.
## One writer thread owns the socket
A dedicated thread drains the queue and owns the connection. It connects,
reconnects with backoff, and writes.
**A single writer is also what keeps event ordering intact** — with two, the order
events reach the sidecar is the order two threads happened to be scheduled in, and
you find out from a board that says a player logged out before they logged in.
The reconnect loop in `LinkLoop` backs off with a low ceiling — a few seconds,
because a loopback reconnect is cheap and a sidecar restart should cost a few
seconds of buffering rather than half a minute of blindness. Pick your ceiling from
what the connection actually costs, not from a habit borrowed from internet
clients.
One detail there is subtle enough to be worth stealing: `BridgeLink` tags each
connection attempt with an **epoch**, so a reader thread from a previous connection
cannot tear down the connection that replaced it. Joining a thread can time out;
the stale thread's cleanup then runs against whatever is current. If you write a
reconnect loop, write it so a late-arriving cleanup from a dead connection is a
no-op.
## Read the world only on the game's thread
Inbound is the mirror image. A reader thread parses lines off the socket, and then
**hands each one to the game's own thread** to act on — `BridgeLink`'s `Dispatch`
does it by scheduling a zero-delay callback on the game's timer, which is the
engine's supported way in. The reader itself never touches the world's objects.
Two rules fall out and both are absolute:
- **Every read of the world happens on the world's thread.** Game engines are
overwhelmingly single-threaded about their state, and reading a collection while
the loop mutates it is a crash or, worse, a corruption you notice a week later.
- **The writer thread only ever sees plain data.** Format your line — a string, a
buffer, whatever your wire is — on the game's thread while the objects are safe
to read, and hand the finished bytes over. Never hand the writer a live game
object to serialise.
And an error boundary at the seam: a malformed command from the sidecar must never
escape into a game code path. `BridgeLink` wraps the inbound handler and logs
anything it throws, because the alternative is an exception unwinding somewhere in
the engine's main loop.
## Reconnect, and what to send on connect
Your sidecar restarts independently of your game. It comes back with an empty
picture, and it cannot ask the game for one without an inbound path you may not
have built yet.
So **anything the sidecar needs up front is re-sent on every connect, not once at
startup.** In `servuo-plugins` that is an explicit event: the link exposes a
"connected" hook that runs on the game thread, and each feature area subscribes to
it and re-emits its current state — the hello line, the house registry, the guild
and governor boards, the market, the ruleset. A sidecar that has just started is
therefore fully populated within one connection, with no negotiation.
The general form: **for every board your website renders, have exactly one place
that can produce its current state, and call it on connect.** If you cannot name
that place for some piece of state, your sidecar will eventually be missing it and
nobody will know why.
## Events, snapshots, and the state that has neither
You will end up emitting two different kinds of thing, and confusing them is a
design mistake that shows up as a bad page.
**An event** is something that happened, at a time: a player logged in, a house
fell, a trade completed. Events are appended and read back as history.
**A snapshot** is the current state of a subject: this board's rows, this guild's
membership, the published ruleset. Snapshots overwrite; nobody wants the history of
a leaderboard's every intermediate ordering.
Send both, and be clear at the wire about which a message is — your sidecar's store
handles them differently ([chapter 3](03-sidecar.md)), and a snapshot appended as
history is a table that grows forever.
Then there is the state your engine gives you no hook for at all. Player vitals,
decay timers, money supply: nothing fires when they change. `servuo-plugins` polls
those on the game's own thread with repeating timers, in
`overlay/Scripts/Custom/Bridge/BridgeSweeps.cs`, and the comment worth copying is
that this is only acceptable **because the cost was measured**. A full pass of all
three sweeps is well under a millisecond at that shard's scale. Measure yours
before you add a timer to a game loop, and if a sweep is expensive, sample it —
never move it off the game thread.
Two practical notes from that file, both general:
- **Emit a transition, not a level.** The decay sweep keeps the last known level per
house and emits only when one changes, and it takes a silent baseline at startup
so a restart does not re-announce every house's current state as news.
- **Know when your engine suspends timers.** These do not fire during a world save,
so a sweep that would have landed mid-save simply happens a few seconds later.
That is fine for all three — but it is fine because someone checked, not by
default.
## A checklist for the plugin you are about to write
1. The emit path enqueues and returns. Nothing on the game thread touches a socket.
2. The queue is bounded and drops the oldest, and something counts the drops.
3. One writer thread owns the connection; ordering is therefore intact.
4. Reconnect with a bounded backoff; a stale connection's cleanup cannot affect a
newer one.
5. Inbound lines are marshalled onto the game thread before touching the world, and
a handler that throws cannot escape into the engine.
6. Every board's current state has exactly one producer, and all of them run on
connect.
7. Every world read is on the world's thread; the writer sees only plain data.
8. Anything polled has had its cost measured against a realistic world.
If all eight hold, the worst a broken sidecar can do to your game is nothing at
all — which is the entire point of the arrangement.
---
That is the book. The three parts are a module core loads, a sidecar that owns the
game connection and the durable copy of what it said, and a plugin that feeds the
sidecar without ever waiting on it.
If you got this far and built something, the places you got stuck are the most
valuable thing this repo can receive — [tell us][issues], and please say where you
left the kit and what you did next.
[issues]: https://gitea.whitlocktech.com/RunicGateway/Integration-kit/issues

117
book/README.md
View File

@@ -1,99 +1,40 @@
# The book # The book
Four chapters, in the order the work happens. **None of them are written yet** — Four chapters, in the order the work happens.
this is the outline, landed first so the shape can be argued with before the prose
exists. Chapter status is in the table; a chapter that is not there yet is not
there yet, rather than a stub that reads like an answer.
Read [the dry run][dryrun] before any of them. Read [the dry run][dryrun] before any of them — a complete module designed on
paper for a second game, and the shortest honest picture of the whole job.
| # | Chapter | File | Status | | # | Chapter | What it covers |
| --- | --- | --- | --- | | --- | --- | --- |
| 1 | Your first module in twenty minutes | `01-first-module.md` | not written | | 1 | [Your first module in twenty minutes](01-first-module.md) | Copy the template, rename it, build it, install it, see a page. No theory. |
| 2 | The website module | `02-website-module.md` | not written | | 2 | [The website module](02-website-module.md) | The bulk of the work: `module.json`, `register(ctx, api)`, the schema fragment, the client chunk, packaging, and what a module must never do. |
| 3 | The sidecar | `03-sidecar.md` | not written | | 3 | [The sidecar](03-sidecar.md) | Why the website never talks to a game server, what "persist before you forward" means, and what a *thin* sidecar is. |
| 4 | The game-side plugin | `04-game-plugin.md` | not written | | 4 | [The game-side plugin](04-game-plugin.md) | The least code and the highest stakes: never block the game thread. |
They are named but not linked on purpose: a link to a file that does not exist is Chapters 1 and 2 quote `template/`, which CI builds against a pinned core, so their
the thing this repo's link check is for, and an outline should not be the first code is a tree that is proved rather than prose that looks like one. Chapters 3 and
thing to fail it. 4 cite `uo-link` and `servuo-plugins` by file and identifier rather than by line, on
purpose: those repositories move for their own reasons and a line number in a book
is wrong the moment they do.
## 1. Your first module in twenty minutes ## What is normative, and what is here
Copy `template/`, rename it, build it, install it, see a page. No theory. The point Nothing in these chapters is. Where a chapter and one of these disagree, the
is to reach a working module before learning anything, so that everything after it document is right and the chapter has a bug — [say so][issues]:
is a change to something that already runs rather than a step toward something that
might.
- What the pieces of `template/` are, one paragraph each. | Authority | For |
- `module.json`: the fields you must change, and `coreApi`. | --- | --- |
- Building the client chunk. Why a module ships **prebuilt** and an operator never | [`MODULE_API.md`][api] | Everything a module may do. |
builds anything. | [`MODULE_SYSTEM.md`][system] | Why the module system is shaped this way, and how a module is installed and removed. |
- Installing it: the admin panel, the `MODULES` environment variable, or a directory | [`link/PLAN.md`][linkplan] + [`INTEGRATION.md`][linkint] | The game↔sidecar wire protocol, as one real sidecar implements it. |
on the volume.
- Reading the state your module lands in, and the four ways it can fail to load.
## 2. The website module The chapters teach: the order to do things in, the reasoning, and the mistakes that
cost this project time.
The bulk of the kit.
- **`module.json`** — every field, and which are load-bearing at boot.
- **The server entry point.** `register(ctx, api)`; what `ctx` hands you and why
each member is handed rather than imported; the lazy-accessor pattern that lets a
ported file keep a file-scope `require`, and the require-order rule that comes
with it.
- **The `register*` calls** — routes per tier, notification streams, announce legs,
post hooks, extension slots. Worked examples of each, with the distinctions that
are easy to get wrong (a leg is one-shot delivery with retry; a post hook is
idempotent state that also runs on delete).
- **The schema fragment.** Idempotent, replayed every boot, leading-verb allowlist,
the table-prefix rule, and why there is no migration runner anywhere in this
project. What belongs in `purge.sql` instead.
- **The client half.** The prebuilt ESM chunk; `window.__rg`; the shared-dependency
rule (core owns React and hands it over — a module that resolves its own gets two
Reacts and a broken page); the Vite library build with anchored aliases and
`external: []`, and *why* that combination rather than the obvious one.
- **Routes, nav and features on the client**, and how a module's nav row becomes an
ordinary row an operator can reorder, relabel or hide.
- **The UI kit** — seven members, closed on purpose. What to do about the eighth
thing you want.
- **The OpenAPI fragment**, and how to generate it from your own registrations.
- **Packaging and release CI**: the tarball, the install manifest, the checksum,
and the version living in `module.json`.
- **Boundaries.** What a module must not do, each with the failure it prevents.
## 3. The sidecar
Why it exists, why it is **not optional**, and what "thin" means for a game that
already speaks a remote-control protocol.
- The invariant: your game is never network-reachable; it **dials out**, the
sidecar listens, and only the website's backend talks to the sidecar.
- **Persist before you forward.** The sidecar owns the durable copy — event
history, the latest snapshot of every board, whatever a page must still be able
to render when the game or the website is down. A live feed is allowed to be
lossy *because* the store is not.
- The wire as a **versioned compatibility contract** rather than a build
dependency: a version on every response, a mismatch refused rather than
mis-parsed, and what a bump obliges you to change in the same commit.
- Auth, and why the sidecar is the only exposed part.
- `uo-link` as the worked example, and what a *thin* sidecar for an RCON-style game
keeps and drops.
## 4. The game-side plugin
The chapter with the least code and the highest stakes: a plugin that gets this
wrong takes the game down when the sidecar wedges.
- **Never block the game thread.** Enqueue and return; a bounded, drop-oldest queue;
a dedicated writer thread that drains it. Dropping the oldest event is correct,
and stalling the game to avoid it is not.
- **Read the world only on the game's own thread**, and hand plain data to the
writer.
- Reconnect, backoff, and what to send on connect so the sidecar can rebuild its
picture without asking.
- What to emit at all: the difference between an event stream and a state snapshot,
and why both exist.
- `servuo-plugins` as the worked example. The constraints are general; the C# is not.
[api]: https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_API.md
[system]: https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/website/MODULE_SYSTEM.md
[dryrun]: https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/modules/rust-dryrun.md [dryrun]: https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/modules/rust-dryrun.md
[linkplan]: https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/link/PLAN.md
[linkint]: https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/link/INTEGRATION.md
[issues]: https://gitea.whitlocktech.com/RunicGateway/Integration-kit/issues

142
scripts/checkChapterPaths.js Normal file
View File

@@ -0,0 +1,142 @@
#!/usr/bin/env node
// Every path in this repo that a chapter names in backticks must exist.
//
// The book teaches out of `template/`: it says "open
// `template/server/index.js`", "the aliases are in `template/client/vite.config.js`",
// "your tables go in `template/server/db/schema.sql`". None of that is a markdown
// link, so `checkLinks.js` never looks at it — and none of it is code, so nothing
// else does either. Rename one template file and four chapters quietly point at
// nothing, which is the exact rot this repo exists to be immune to.
//
// This is the cheap half of "is the book still true", and it is honest about
// being only the half a machine can answer. Whether a paragraph has become wrong
// about a file that still exists is a reviewer's job (MODULE_SYSTEM.md §2.10).
//
// ── What counts as a claim about this repo ────────────────────────────────────
//
// An inline code span whose text begins with one of this repo's own top-level
// directories, `ANCHORS` below. That is what makes the check answerable: a
// chapter also quotes `server/index.js` loosely, and `sidecar/src/store.rs`,
// which lives in another repo entirely and cannot be resolved here. Anchoring on
// our own directory names means every token this check reads is a claim it can
// actually settle.
//
// **The anchors are stated, not derived from the tree**, and that is deliberate
// for the reason core's own build guard states it (MODULE_API.md §3.6): a list
// derived from what exists cannot fail when what exists changes. Rename
// `template/` and a derived anchor set would simply stop checking every
// `template/…` mention in the book, silently, at the moment they all became
// wrong. So the anchors are written down — and each one must exist, or this check
// fails. An anchor that has stopped matching is a check that has stopped
// checking, the same rule the identifier exemptions in core's CI follow.
//
// Fenced blocks are excluded (`lib/markdown.js`). A fence in this book is often a
// listing of the reader's own future tree, and their files are not ours.
//
// Usage: node scripts/checkChapterPaths.js (from the repo root)
const fs = require('fs')
const path = require('path')
const { codeSpans } = require('./lib/markdown')
const ROOT = path.resolve(__dirname, '..')
// This repo's own top-level directories. See the note above on why this is a list
// and not a directory scan.
const ANCHORS = ['template/', 'book/', 'scripts/', 'ci/']
const SKIP_DIRS = new Set(['.git', 'node_modules', 'dist'])
/** Every markdown file in the repo, repo-relative, sorted. */
function markdownFiles(dir = ROOT, out = []) {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
if (entry.isDirectory()) {
if (SKIP_DIRS.has(entry.name)) continue
markdownFiles(path.join(dir, entry.name), out)
} else if (entry.name.toLowerCase().endsWith('.md')) {
out.push(path.relative(ROOT, path.join(dir, entry.name)).split(path.sep).join('/'))
}
}
return out.sort()
}
/**
* The repo paths a document claims, from its inline code spans.
*
* A span is a claim when it starts with an anchor and names something a
* filesystem could answer for. Three kinds are skipped, each because the answer
* would be "no" for a reason that is not a mistake:
*
* • a placeholder — `template/<id>/…`, `scripts/*.js` — which is a shape rather
* than a path;
* • a span with whitespace in it, which is a phrase or a command line
* (`npm ci --prefix template/server` is not a path and its first word is not
* an anchor either, but a span like `cd template/server && npm test` would
* slip through on its first token without this);
* • trailing prose punctuation, stripped rather than skipped, so `template/`
* ending a sentence still resolves.
*/
function claimedPaths(markdown) {
const found = []
for (const { text, line } of codeSpans(markdown)) {
const token = text.trim()
if (/\s/.test(token)) continue
if (!ANCHORS.some((a) => token.startsWith(a))) continue
if (/[<>*?]|\.\.\./.test(token)) continue
// A path may legitimately end in `/` (a directory); anything else in this set
// is the sentence around it, not part of the name.
const cleaned = token.replace(/[.,;:)\]]+$/, '')
if (cleaned) found.push({ path: cleaned, line })
}
return found
}
/** Everything wrong, as sentences. Empty means every claim resolves. */
function problems({ claims, exists }) {
const out = []
for (const anchor of ANCHORS) {
const dir = anchor.replace(/\/$/, '')
if (!exists(dir)) {
out.push(
`${anchor} is listed as an anchor and does not exist. ` +
'Either restore it or update ANCHORS — an anchor that matches nothing is a ' +
'check that has silently stopped checking.',
)
}
}
for (const { file, path: claimed, line } of claims) {
if (!exists(claimed)) {
out.push(`${file}:${line}: no such path — ${claimed}`)
}
}
return out
}
module.exports = { ANCHORS, claimedPaths, problems, markdownFiles }
if (require.main !== module) return
const files = markdownFiles()
const claims = []
for (const file of files) {
const text = fs.readFileSync(path.join(ROOT, file), 'utf8')
for (const claim of claimedPaths(text)) claims.push({ file, ...claim })
}
const exists = (p) => fs.existsSync(path.join(ROOT, p))
const found = problems({ claims, exists })
if (found.length) {
console.error(`\ncheckChapterPaths: ${found.length} problem(s):\n`)
for (const p of found) console.error(` - ${p}`)
console.error('')
process.exit(1)
}
console.log(
`checkChapterPaths: ${claims.length} path(s) claimed across ${files.length} markdown file(s) — all present.`,
)

81
scripts/checkChapterPaths.test.js Normal file
View File

@@ -0,0 +1,81 @@
// The chapter-path check, checked.
//
// Same rule as the rename check's own suite: a check written when the thing it
// guards is already clean never fires again, and nothing distinguishes "still
// checking" from "quietly broken" without cases it is required to reject. Every
// "must not catch" case below is a real span that appears in the book.
//
// No filesystem — `problems()` takes `exists` as an argument precisely so it can
// be tested this way, and `claimedPaths()` is pure.
const test = require('node:test')
const assert = require('node:assert')
const { ANCHORS, claimedPaths, problems } = require('./checkChapterPaths')
/** `problems()` over a fixture set of paths that exist. */
const check = (claims, present) =>
problems({ claims, exists: (p) => new Set([...present, ...ANCHORS.map((a) => a.replace(/\/$/, ''))]).has(p) })
test('a claim that resolves is not a problem', () => {
assert.deepStrictEqual(check([{ file: 'book/01.md', line: 3, path: 'template/module.json' }],
['template/module.json']), [])
})
test('a claim that does not resolve fails, naming the file and line', () => {
const found = check([{ file: 'book/02-website-module.md', line: 41, path: 'template/server/gone.js' }], [])
assert.strictEqual(found.length, 1)
assert.match(found[0], /book\/02-website-module\.md:41.*template\/server\/gone\.js/)
})
test('a missing anchor fails on its own', () => {
// The half that keeps this check honest: if `template/` is renamed, every
// template path in the book is wrong AND the check would stop looking at them.
const found = problems({ claims: [], exists: (p) => p !== 'template' })
assert.strictEqual(found.length, 1)
assert.match(found[0], /template\/ is listed as an anchor and does not exist/)
})
test('paths are read only from inline code spans', () => {
const md = 'Open the entry point and read it: template/server/index.js, then stop.'
assert.deepStrictEqual(claimedPaths(md), [])
})
test('a code span inside a fenced block is not a claim', () => {
// A fence is usually the reader's own future tree, and their files are not ours.
const md = ['```', '`template/nope.js`', 'template/also-nope.js', '```'].join('\n')
assert.deepStrictEqual(claimedPaths(md), [])
})
test('a path in another repo is not this check\'s business', () => {
const md = 'The store is `sidecar/src/store.rs`, and the plugin is `overlay/Scripts/Custom/Bridge/BridgeLink.cs`.'
assert.deepStrictEqual(claimedPaths(md), [])
})
test('a placeholder shape is not a path', () => {
const md = 'Your copy lands at `template/<id>/module.json`, and the checks are `scripts/*.js`.'
assert.deepStrictEqual(claimedPaths(md), [])
})
test('a command line is not a path', () => {
// The first token is an anchor in neither case, but a span that BEGINS with one
// and carries arguments would otherwise be read as a filename with spaces in it.
const md = 'Run `npm ci --prefix template/server`, or `template/server && npm test` if you must.'
assert.deepStrictEqual(claimedPaths(md), [])
})
test('trailing sentence punctuation is stripped, not skipped', () => {
const md = 'It all lives under `template/`.'
assert.deepStrictEqual(claimedPaths(md), [{ path: 'template/', line: 1 }])
})
test('a claim on a later line reports that line', () => {
const md = ['# Title', '', 'See `template/server/boot.js`.'].join('\n')
assert.deepStrictEqual(claimedPaths(md), [{ path: 'template/server/boot.js', line: 3 }])
})
test('every anchor is a directory of this repo', () => {
// Stated, not derived (see the header) — so this asserts the stated list is
// still the real one at the moment it is written down.
assert.ok(ANCHORS.every((a) => a.endsWith('/')), 'anchors are directory prefixes')
})

31
scripts/checkLinks.js
View File

@@ -19,6 +19,8 @@
const fs = require('fs') const fs = require('fs')
const path = require('path') const path = require('path')
const { stripFences } = require('./lib/markdown')
const ROOT = path.resolve(__dirname, '..') const ROOT = path.resolve(__dirname, '..')
const QUIET = process.argv.includes('--quiet') const QUIET = process.argv.includes('--quiet')
@@ -38,31 +40,10 @@ function markdownFiles(dir = ROOT, out = []) {
return out.sort() return out.sort()
} }
// Fenced code blocks are stripped before links are read: a fence can legitimately // Fenced code blocks are stripped before links are read (`lib/markdown.js`): a
// contain a path that does not exist (a directory listing of a project the reader // fence can legitimately contain a path that does not exist — a directory listing
// has not created yet), and flagging those would make the check useless in exactly // of a project the reader has not created yet — and flagging those would make the
// the document type this repo is made of. Stripped by walking lines and toggling // check useless in exactly the document type this repo is made of.
// on a fence marker, rather than by regexp — a fence's own content can contain
// anything, including a line that looks like the end of one.
function stripFences(text) {
const out = []
let fence = null
for (const line of text.split(/\r?\n/)) {
const m = /^\s*(```+|~~~+)/.exec(line)
if (fence) {
if (m && m[1][0] === fence[0] && m[1].length >= fence.length) fence = null
out.push('')
continue
}
if (m) {
fence = m[1]
out.push('')
continue
}
out.push(line)
}
return out.join('\n')
}
/** Inline `[text](target)` links and `[ref]: target` definitions, with line numbers. */ /** Inline `[text](target)` links and `[ref]: target` definitions, with line numbers. */
function linksIn(text) { function linksIn(text) {

60
scripts/lib/markdown.js Normal file
View File

@@ -0,0 +1,60 @@
// The two pieces of markdown handling both checks in this directory need, in one
// place rather than two copies that drift.
//
// Shared code, not a shared description. Core's own loader and its schema replay
// use one splitter for the same reason (MODULE_API.md §2.6): two implementations
// of "what counts as a code fence" would disagree eventually, and the check that
// disagreed quietly would be the one still reporting green.
/**
* The text with every fenced code block blanked out, line count preserved.
*
* Fenced blocks are stripped before either check reads anything, because a fence
* can legitimately contain a path or a link that does not exist: a directory
* listing of the project the reader has not created yet, a URL in an example. In
* a repo made entirely of that document type, flagging them makes the check
* useless.
*
* Done by walking lines and toggling on a fence marker rather than by regexp — a
* fence's own content can contain anything, including a line that looks like the
* end of one. Lines are replaced by empty strings rather than removed so that
* line numbers in a report still point at the right place.
*/
function stripFences(text) {
const out = []
let fence = null
for (const line of text.split(/\r?\n/)) {
const m = /^\s*(```+|~~~+)/.exec(line)
if (fence) {
if (m && m[1][0] === fence[0] && m[1].length >= fence.length) fence = null
out.push('')
continue
}
if (m) {
fence = m[1]
out.push('')
continue
}
out.push(line)
}
return out.join('\n')
}
/**
* Every inline code span outside a fenced block, with the 1-based line it is on.
*
* `[a](b)` inside backticks is an example rather than a link, and `template/x.js`
* inside backticks is a claim about this repo's tree — which is why one check
* throws these away and the other reads only these.
*/
function codeSpans(text) {
const found = []
stripFences(text).split(/\r?\n/).forEach((line, i) => {
for (const m of line.matchAll(/`([^`]+)`/g)) {
found.push({ text: m[1], line: i + 1 })
}
})
return found
}
module.exports = { stripFences, codeSpans }