diff --git a/README.md b/README.md index 9840453..9574930 100644 --- a/README.md +++ b/README.md @@ -85,8 +85,9 @@ share a shape and nothing else, so neither document is a fallback for the other. | Doc | What it covers | |---|---| -| [PROTOCOL.md](rust-link/PROTOCOL.md) | **Canonical** — the game link and the website API, the four declaration sites of the wire version, and what protocol 1 defines | +| [PROTOCOL.md](rust-link/PROTOCOL.md) | **Canonical** — the game link and the website API, the four declaration sites of the wire version, and what each protocol version defines: 1 the transport, 2 the read path | | [INTEGRATION.md](rust-link/INTEGRATION.md) | Standing the bridge up by hand, and which of the three components is wrong when it does not work | +| [PLAYER_WALK.md](rust-link/PLAYER_WALK.md) | The half of the read path a console cannot reach: ten minutes on a rig with a player, step by step, with what each hook should produce | ### `android/` | Doc | What it covers | diff --git a/modules/rust/PLAN.md b/modules/rust/PLAN.md index 30e70aa..178f473 100644 --- a/modules/rust/PLAN.md +++ b/modules/rust/PLAN.md @@ -980,7 +980,7 @@ Each phase ends with its findings written down, as every workstream here does. | 0 | **The rig.** ✅ **Done 2026-09-15 — as built and findings in §12.** Updated to the current wipe (the script was fixed *again*, properly), Oxide re-laid, base set installed, the grant path proven end to end and both zone transitions observed live with a player connected. **Both criteria met** | docs | A current server boots with all four loaded, `oxide.grant` demonstrably gates something, and a test zone reports who is standing in it | | 1 | **Protocol 1, three skeletons, and every bundle seam at once.** ✅ **Done 2026-09-15 — as built and findings in §13.** Plugin, sidecar and module all exist and all three were exercised against the live rig; three org-lead decisions (§13.0), five defects only a running server found (§13.3), and a correction to §11.3 (§13.2). **Both criteria met** | all 3 + docs | One hello line travels game -> sidecar -> module; killing the sidecar does not stall the game; all five guards green on an untouched skeleton | | 2 | **Packaging and release.** ✅ **Done 2026-09-16 — as built and findings in §15.** `release.yml` *and* the gate that was missing entirely (`pr-checks.yml`, including the frozen-manifest job); the include list with two readers; `v0.1.0` published and installed into a running core from its manifest URL. Three org-lead decisions (§15.0), and the first proof by a core that `/rust` collides with nothing (§15.2). **Criterion met** | Module-Rust + docs | An operator installs the empty module from Admin -> Modules and it reaches `started` | -| 3 | **The read path, on both frameworks.** First hook wave from [`HOOKS.md`](HOOKS.md); events and snapshots distinct at the wire; `wipe_id` **and server id** on every row (R8); all-time rollups (R12); every board re-emitted on connect. **First phase to run against the Carbon rig (R19/R21)** — it turns [`CARBON.md`](CARBON.md) from a source-read hypothesis into tested fact, including whether the 13 unlisted hook names are renames or holes | all 3 + docs | A restarted sidecar is fully populated within one connection, a wipe does not erase a player's history, and **the same plugin file does all of that on Oxide and on Carbon** | +| 3 | **The read path, on both frameworks.** ✅ **Built and largely proven 2026-09-16 — as built and findings in §16.** Protocol 2: fifteen hooks, an envelope every frame carries, boards, a cursor feed and bounded history; four org-lead decisions (§16.0), two defects only a booted server could find (§16.2), and CI for the two bridge repositories that had none. **The player half of the catalogue is written down as a walk to run rather than measured** — see §16.7. First hook wave from [`HOOKS.md`](HOOKS.md); events and snapshots distinct at the wire; `wipe_id` **and server id** on every row (R8); all-time rollups (R12); every board re-emitted on connect. **First phase to run against the Carbon rig (R19/R21)** — it turns [`CARBON.md`](CARBON.md) from a source-read hypothesis into tested fact, including whether the 13 unlisted hook names are renames or holes | all 3 + docs | A restarted sidecar is fully populated within one connection, a wipe does not erase a player's history, and **the same plugin file does all of that on Oxide and on Carbon** | | 4 | **The first pages.** Server list as the landing page, `/rust/servers/:id` beneath it, killfeed, leaderboard; nav rows; the UI kit (`PublicLayout` `shell`, `PageHeader` props); `capabilities`; the `site.footer.status` slot (R13) | Module-Rust | The site renders the last thing each server said while every server is off | | 5 | **Android leg A** (R10). Capability-driven shell from `GET /api/v1/public/modules`, plus the phase-4 screens | Android-app | The app renders a Rust site it has never seen, and a UO site unchanged | | 6 | **Identity** (R1), and the `admin.users.detail` slot (R13) | 3 + docs | A player links an account in-game; an operator sees the Steam id inside core's own user page | @@ -2192,6 +2192,231 @@ Linux engine was found dead — its WSL distribution stopped, the `uomm-db` cont Restarting Docker Desktop and the container fixed it. Worth writing down because the failure presents as the *website* being broken (`ECONNREFUSED` to a database that is simply not there), and because `CLAUDE.md` points every smoketest at that one container. + +## 16. Phase 3 as built — the read path, 2026-09-16 + +The first phase that had to be true on two mod frameworks, and the first with a +catalogue rather than a message. Four repositories moved: the spec here, the plugin, the sidecar, +and the module. + +**Status: the bridge half is done and proven; two proofs are queued on the org lead.** What the +plugin sends and what the sidecar does with it are built, tested and exercised against live Oxide +and Carbon servers. The player-facing half of the catalogue — deaths, chat, gathering, sessions — +cannot fire without somebody holding a mouse, and is written down as a walk to run rather than +guessed at: [`PLAYER_WALK.md`](../../rust-link/PLAYER_WALK.md). §16.7 lists everything still open. + +### 16.0 The four decisions this phase needed + +- **D7 — the widest first hook wave.** The options ran from "presence, deaths and the wipe" to + "everything read-only worth having", and the widest was chosen: fifteen hooks, including the + moderation set that carries IP addresses and player reports. The consequence is real and is + designed around rather than deferred — those frames arrive **nine phases before** the visibility + framework phase 14 builds, so the classification and its default-deny allowlist ship now (§16.4). +- **D8 — the plugin derives `wipeId`.** `PROTOCOL.md` §3.2 had reserved that for the website. By + protocol 2 three components store rows that need it and only one of them can read the value, so + the reversal is written into §8.2 rather than left as a contradiction. +- **D9 — the live feed is a cursor, and D5 stands.** Core runs Node 20, where a global `WebSocket` + is still behind a flag, so a socket means taking `ws` — against a release that asserts it ships no + runtime dependencies. The deciding argument was the other one: **a socket needs a cursor anyway** + for what it missed while the module was restarting, and the catch-up path is the one that must be + right. One mechanism exercised every five seconds beats two where the second only runs after an + outage nobody planned. +- **D10 — rollups permanent, raw bounded.** The website keeps per-player-per-wipe totals for ever + and a 30-day window of raw events; the sidecar keeps 14 days and prunes hourly. R12's "a wipe does + not erase a player's history" is met by the totals, which is the row an operator actually reads. +- **D11 — CI for both bridge repositories**, which had none at all. Phase 2 found that hole in + Module-Rust; it was still open in the two repositories that ship the half running inside somebody + else's game server. + +### 16.1 `type` is the whole of protocol 2 in the sidecar + +Protocol 1 routed on `kind`, in a `match` that needed a new arm per addition. Protocol 2 adds +**`type`** — `event`, `snapshot`, `reply`, `control` — and the sidecar files on that and nothing +else. Ten new event kinds are now zero change in Rust-Link, which is the property that matters when +the thing growing fastest is the catalogue. + +A frame whose `type` this build does not know is **dropped and counted**, never guessed at. +Defaulting an absent one to `event` would file a *board* as history — the presence board appended a +few thousand times, which nothing reports and nobody notices until they wonder why the database is +large. + +**It caught a real mismatch three seconds after it first ran**, which was not planned: a protocol 1 +plugin was still live on the retired workstation rig, dialled the new sidecar, and its `server.hello` +went straight into the counter. The game link has no version handshake by design (§2), so +`untyped_frames` on `/health` is the only place that failure is visible — and the symptom without it +is a website showing nothing while the game is plainly up. + +### 16.2 Two defects a live server found, and neither could have been found anywhere else + +**The wipe id was null for every real session.** `Init` runs *before* the save is loaded, so +`SaveRestore.SaveCreatedTime` is not yet meaningful there, and the id resolved at load time stayed +null for the life of the process — every frame shipping without the field R12 splits history on. + +It was invisible for the reason such things usually are: a **hot-reloaded** plugin reads an +already-loaded world and gets the right answer every time. Every development iteration on the +workstation rig was a hot reload. It took a server that *booted* with the plugin installed — which +is every real one — to show `wipeId=none` beside a save sitting on disk. Now resolved again at +`OnServerInitialized`, and lazily while still unknown. + +**`Unload` blocked the game's main thread for two seconds.** Carbon reported it exactly: +`hook 'Unload' took longer than 100ms [2002ms]`, next to `link thread did not stop cleanly`. That is +phase 1's stall arriving by a different road — the link thread sits in a blocking +`TcpClient.Connect`, which has no timeout of its own and cannot be woken, and `Unload` joins it from +the main thread. + +The reason two phases missed it is worth keeping: **a host that refuses answers instantly, and a +host that drops does not answer at all.** Every loopback test is the first kind. A firewalled +address, a typo, a machine that is off are all the second, and the deployment this phase was being +tested through happened to be one. The connect is now bounded and waits on a stop handle of its own +— it cannot share `Wake`, which also means "the queue has something in it" and is signalled by every +hook that fires. After the fix the same reload logs no slow-hook warning and no stranded thread. + +### 16.3 The aggregate, and the rule it generalises + +`OnDispenserGather` fires on **every swing at a tree**. A frame per swing would make the bridge the +most expensive thing on a busy server, and nobody wants a killfeed of chickens either, so gathering +and NPC kills are counted in the plugin and flushed once a minute as one `player.tally` frame. + +The rule: **if a hook can fire more than once a second per player, it is a counter, not an event.** +R17's warning about chatty zone transitions is the same rule arriving early. + +A tally is a **delta, not a running total** — what happened since the last flush — so the consumer +sums rather than diffs, and a dropped frame costs one interval instead of corrupting the series. The +outbound queue is drop-oldest by design, so frames are genuinely allowed to go missing; a running +total over a lossy link is a number that is quietly wrong for ever. + +One honest limitation, corrected in the code rather than in the comment that first claimed +otherwise: **a plugin reload loses up to a minute of one player's tally.** `Unload` enqueues the +flush, but the writer stops on the same flag and the queue is cleared after the join. Draining it +first would mean waiting on a socket from the main thread — the stall §16.2 just removed — so the +loss is taken deliberately. A real shutdown flushes at `OnServerShutdown`, and a player leaving +flushes at their disconnect. + +### 16.4 The boundary is enforced by the side that serves + +The widest hook wave brings IP addresses (`CanUserLogin`, `OnUserApproved`, `OnUserBanned`), one +player's report about another, and the grid reference of somebody's base — nine phases before the +visibility framework §11.2 costed. So the classification ships with the catalogue. + +**It is not a field on the wire.** The plugin could have stamped a class on every frame; it +deliberately does not. A boundary declared by the *sender* is one a compromised — or merely +out-of-date — game host can widen. The website's own shard fan-out works the same way: a public +stream with an allowlist of kinds and an admin stream that adds the rest, and what makes it +trustworthy is that the decision lives on the serving side. + +So `module-rust/server/catalogue.js` holds it, **default-deny**: a kind this build has never heard +of is not public. That is the shape of the mistake it prevents — the next protocol version adds a +kind, the module stores it happily, and a deny-list filter would publish it the day it first +arrived, before anybody decided whether it should be. A test holds the list against §8.4's table, so +adding a kind to the protocol without classifying it fails a build. + +### 16.5 A login denial is not a hook, and §10 says it is + +`PLAN.md` §10 sources the `rust.login.denied` trigger from `CanUserLogin`. Reading the hook says that +cannot work: it fires on **every** connection attempt, and the only way to learn of a denial from it +is to *be* the denier — which §8.7 forbids, structurally, by declaring every read-path hook `void` so +it cannot answer. uMod publishes no `OnUserRejected`. + +What the game can tell us is two facts: an attempt, and an approval. Protocol 2 emits both, and a +denial is **the absence of an approval** — a deferred read, phase 10's to make. The trigger survives; +its source changes. (The same shape the engagement workstream hit at its own phase 10, which is +either a coincidence or a property of login paths.) + +### 16.6 What was proven, and how + +| Claim | How | Result | +|---|---|---| +| The read path compiles and loads on **Oxide** | live server, protocol 2, 15 hooks bound | ✅ | +| …and on **Carbon** 2.0.259.0, from the **byte-identical file** | the Pterodactyl rig, config read from `carbon/configs/` | ✅ | +| Every frame carries `type`, `serverId`, `wipeId` | `server.hello` and `players.online` read back off the sidecar | ✅ | +| A wipe id derived from the save, changing only with the save | `rg.link` reports `w-20260915T195817Z` against `saveCreatedAt 2026-09-15T19:58:17Z` | ✅ | +| **A restarted sidecar is fully populated within one connection** | store deleted, process restarted: both boards present **0.3 s** after the listener bound, and **zero** events in history | ✅ | +| Boards are not replayed as history | the same walk: `/events` returned 0 rows while `/boards` returned 2 | ✅ | +| Moderation frames reach the sidecar whole | `banid` / `unban` over RCON | ✅ | +| A protocol mismatch is counted, not mis-filed | a live protocol 1 plugin against the protocol 2 sidecar | ✅ | +| `rg.hooks` answers on both frameworks | Oxide and Carbon, identical output shape | ✅ | +| 44 sidecar tests, 95 module server tests, 20 client tests, every guard | locally, and now in CI on both repos | ✅ | +| The module's route manifest against a **real core** at the pinned ref | 10 routes, all documented, none of core's moved | ✅ | +| **The module ingests a live game** | the working tree installed into a running core, pointed at the workstation sidecar: `cursor started at the feed tail`, then a console `banid`/`unban` arrived as two `rust_events` rows with their wipe id, cursor advanced | ✅ | +| **The allowlist holds against real rows** | with both ban events in the table, `GET /public/rust/servers/:id/events` answered `{"events":[]}` — **and answered the same when asked for `player.banned` by name** | ✅ | +| A wipe row is created by being mentioned | `w-20260915T195817Z` appeared in `rust_wipes` from the first frame carrying it, with no "a wipe started" call anywhere | ✅ | + +**One finding about the rigs rather than the code:** the panel rigs cannot reach a sidecar running on +the workstation, because Windows Firewall holds two program-scoped **Block** rules for +`rust-link-sidecar.exe` — created by a dismissed prompt at some point — and a program-level block +beats any port-level allow. Removing them needs elevation. It is a rig problem only: the shipped +design puts the sidecar on the game host's own loopback (D2, R20), where it is the deployment that +never needs a rule at all. + +The local workstation rig, which does reach its sidecar on loopback, is what proved everything in the +table above that needs a live socket. **`D:\rust` is therefore not as retired as R21 assumed** — it +survives as the fast loop (a saved file is a reloaded plugin in about ten seconds, against nine +minutes of world generation on the panel), and the panel rigs are what answer "on both frameworks". + +### 16.7 What is still open, and who it is waiting on + +Three things, all of them measurements rather than decisions, and all of them the org lead's to run: + +1. **The player walk** — [`PLAYER_WALK.md`](../../rust-link/PLAYER_WALK.md). Ten minutes on a rig + with a mouse, and it closes `OnPlayerDeath`, `OnPlayerChat`, `OnDispenserGather`, + `OnPlayerRespawned`, `OnEntityDeath`-by-a-player, and `sessionSec`. The document says what each + step should produce, so it can be run without anybody watching the output live. +2. **The wipe walk.** `OnNewSave` fires when a server starts with no save — the panel rigs wipe + through the egg's own `REMOVE_FILES`, so this is the rig's own mechanism rather than a special + test. What it proves is the second half of the acceptance criterion: that the old wipe's rows are + still queryable by `?wipe=` afterwards. +3. **The Carbon socket leg.** Everything up to the socket is proven on Carbon; what is not is frames + actually arriving over a live link, which is one elevated firewall command away + (`Remove-NetFirewallRule -DisplayName "rust-link-sidecar.exe"`, then an allow for the rig). + +Until 1 and 2 are run, the honest statement of this phase is: **the transport, the envelope, the +boards and the classification are proven on both frameworks; the player half of the catalogue is +built, reviewed against the hook documentation, and unmeasured.** That is written here rather than +in a commit message because it is the kind of thing a later phase will want to know it inherited. + +### 16.7b The ingest walk, and the trap it walked into + +The module half was exercised against the live rig rather than only against its own tests: the +working tree installed into a running core, pointed at the workstation sidecar with a real Rust +server behind it. It logged `cursor started at the feed tail {at: 1}` — the fresh-install path, +starting at the end rather than replaying — and a console `banid`/`unban` pair then arrived as two +`rust_events` rows carrying the wipe id, with the cursor advancing to 3 and `events_seen` at 2. + +Then the boundary, on real rows rather than fixtures: with both ban events sitting in the table, the +public events route answered `{"events":[]}`, **and answered the same when asked for +`player.banned` by name**. That is the difference between a filter and a refusal — the kinds are, as +far as a public caller is concerned, not there. + +**The trap, and it is the same one the Pterodactyl work recorded.** The server-state row read as +`online=1` with `hostname=NULL` and `protocol=NULL`, which is a shape no code path writes. The cause +was two cores sharing one database: a second instance left running from earlier in the day still +holds the **phase 2 release** in memory, speaks protocol 1, is refused `409` by a protocol 2 sidecar, +and writes the row back as unreachable every thirty seconds. Nothing was wrong with either of them. + +What made it *look* like a defect was the measurement: two `SELECT`s in two round trips, assuming +the state did not move between them. A single atomic read caught the coherent row a moment later — +`online: 1, protocol: 2, hostname: Test Server` — alternating with the other writer's. +**A differential diagnosis across two calls is only valid if nothing else holds the controls**, and +on a shared rig something usually does. + +### 16.8 Smaller things worth keeping + +- **`rg.hooks` collides with `RGProbe`**, the phase-0 rig plugin, which registered the same console + command first. Oxide warns and the last loaded wins, which happens to be the bridge. Left alone: + `RGProbe` is rig scaffolding that never ships, and renaming the shipping command to avoid a + test tool would be the wrong way round. +- **The IP a console ban reports is the literal string `"0"`**, not an address and not a null, when + the banned id is offline. Observed, not guessed. The plugin omits the field instead of forwarding + it — a column full of `"0"` survives every is-it-missing test a reader writes and then fails + whatever parses it. +- **`--print-config` reports the *effective* configuration and writes the *file* one.** Environment + variables override the file (R22 depends on that), and the written file never contains them. Not a + bug, but the two are not the same document and an installer reading one should not assume the + other. +- **A `cargo clippy` run does not produce a binary.** Two rig readings disagreed with the source + because the sidecar under test was an older `cargo build`; `clippy` and `test` compile without + writing one. Rebuild before believing a rig. + --- [rl]: https://gitea.whitlocktech.com/RunicGateway/Rust-Link diff --git a/modules/rust/README.md b/modules/rust/README.md index a78dbc8..14ba167 100644 --- a/modules/rust/README.md +++ b/modules/rust/README.md @@ -18,7 +18,7 @@ differ. | [`DEFINITIONS.md`](DEFINITIONS.md) | **What things are called.** 678 items (short name, id, display name) and 2,590 workshop skin ids across 104 items. | | [`OPERATING.md`](OPERATING.md) | **How it gets run.** The 6 operator pages — installing Oxide on a server, then installing, configuring and permissioning plugins. | | [`agent/`](agent/README.md) | The same facts in **machine shape** — TSV and JSONL, ~46% of the tokens. Generated in the same pass, so it cannot drift. | -| [`CARBON.md`](CARBON.md) | **The other framework.** Where Carbon diverges from Oxide and nowhere else — file layout, the permission store, the `c.` commands, 30 Carbon-only hooks and 13 uMod names its catalogue omits. Sourced from Carbon's own metadata and source, **not yet proven on a live Carbon server.** | +| [`CARBON.md`](CARBON.md) | **The other framework.** Where Carbon diverges from Oxide and nowhere else — file layout, the permission store, the `c.` commands, 30 Carbon-only hooks and 13 uMod names its catalogue omits. Sourced from Carbon's own metadata and source, and **proven on a live Carbon 2.0.259.0 server** — R19 at phase 0, and the whole read path at phase 3. | **The one file here that is ours:** [`PLAN.md`](PLAN.md) — the schedule and the decisions of record for actually building `module-rust`. Everything else in this directory is copied from uMod; that one diff --git a/rust-link/INTEGRATION.md b/rust-link/INTEGRATION.md index aca817a..1389c52 100644 --- a/rust-link/INTEGRATION.md +++ b/rust-link/INTEGRATION.md @@ -164,6 +164,17 @@ Two failures that look alike and are not: says what was true when it was written; nothing has written it since. Either the poll is failing (the website's log) or the sidecar stopped (its own). +### 3.0 `untyped_frames` on `/health` is not zero + +**The plugin and the sidecar are on different protocol versions.** The game link has no handshake +to catch that at connect time (`PROTOCOL.md` §2), so it shows up here instead: the sidecar files a +frame by its `type`, a frame from the wrong version does not carry one it recognises, and it is +dropped and counted rather than guessed at. + +The symptom without this counter is the confusing one — a game server plainly up, a sidecar plainly +healthy, and a website showing nothing. Check the plugin's `rg.link` (it prints its protocol) against +the sidecar's `/health` (which prints its own) and upgrade whichever is behind. + ### 3.1 The failures that are supposed to happen Three things look like breakage and are the design: @@ -193,6 +204,29 @@ you get a warning naming both ids. --- +## 4.1 What the bridge sends, and how much of it is kept + +From protocol 2 the plugin sends the read path: connects and disconnects, deaths, chat, gathering, +bans and reports, and the wipe. Two things about the volume are worth knowing before you size +anything. + +**Gathering and NPC kills are counted, not forwarded.** `OnDispenserGather` fires on every swing at +a tree; sending one frame per swing would make the bridge the most expensive thing on the server. The +plugin keeps a per-player tally and flushes it once a minute as a single `player.tally` frame. So the +leaderboard is exact and the wire is quiet. + +**The sidecar's history is bounded; the website's is not.** `[store].retain_days` (default 14) is +how long the sidecar keeps raw events. The permanent record — per-wipe totals that survive a wipe — +lives in the website's own tables, so shortening this loses recent detail and never loses a player's +history. Set it to `0` to keep everything, if the host's disk is yours to spend. + +**Every row carries its wipe.** The plugin derives a `wipeId` from the save's creation time and +stamps it on every frame, so a wipe splits the history rather than ending it. That is also why +**the sidecar's database must never be in a wipe script's delete list** — see the Pterodactyl egg's +`REMOVE_FILES`. + +--- + ## 5. Upgrading The four declaration sites in [`PROTOCOL.md` §2](PROTOCOL.md#2-versioning) must agree. In practice diff --git a/rust-link/PLAYER_WALK.md b/rust-link/PLAYER_WALK.md new file mode 100644 index 0000000..a438574 --- /dev/null +++ b/rust-link/PLAYER_WALK.md @@ -0,0 +1,100 @@ +# The player walk — proving the half of the read path a console cannot reach + +Protocol 2's catalogue divides cleanly in two, and the line is not about importance: it is about +whether a hook can fire without somebody holding a mouse. + +Everything in the first half was proven from a console and a REST client while phase 3 was built — +the boards, the wipe id, the envelope, bans, the server lifecycle. Everything below needs a **real +player on a real server**, because the hooks carry a `BasePlayer`, a `HitInfo` or a chat line, and +none of those three can be manufactured from a console without becoming a different test. + +This document is the walk that closes it. It takes about ten minutes, it is the same on Oxide and on +Carbon, and it is written so that the answer is readable afterwards rather than watched live. + +--- + +## Before you start + +1. A rig running, with `RunicGateway.cs` loaded — `oxide.plugins` (or `c.plugins`) lists *Runic + Gateway*, and `rg.link` answers `connected=True`. +2. A sidecar the rig can reach, with its store **empty** — that is what makes the event list at the + end readable as a transcript of the walk and nothing else. +3. The sidecar's token to hand, for the reads at the bottom. + +Run this once, before you join: + +``` +rg.hooks +``` + +Every player hook should read **silent**. That is the baseline: the point of the walk is to move +them, and starting from a run where some already fired proves less. + +--- + +## The walk + +Do these in order. The order matters only in two places, noted where it does. + +| # | Do this | Fires | The frame should carry | +|---|---|---|---| +| 1 | **Join the server** | `CanUserLogin`, `OnUserApproved`, `OnPlayerConnected` | Three frames, in that order. The first two carry your **IP address** — check it is a real address and not the string `0`. `player.connected` carries your steam id and name | +| 2 | **Wake up / spawn in** (click Respawn if you are dead) | `OnPlayerRespawned` | `player.respawned`, steam id only. It does **not** fire if you simply wake from sleeping — that is the hook's own documented behaviour, so no frame here is a pass, not a failure | +| 3 | **Say something in chat**, then **say something in team chat** if you have a team | `OnPlayerChat` | Two `player.chat` frames, with `channel` reading `Global` and `Team`. The message must arrive whole — if it is truncated or the frame is missing, the flattener ate it | +| 4 | **Chop a tree for about twenty seconds**, then **mine a node** | `OnDispenserGather` | **Nothing immediately.** This is the aggregate: one `player.tally` frame within 60 seconds, carrying `gathered` with `wood` and `stones`, summed. Seeing a frame per swing would be the bug | +| 5 | **Kill an animal or a scientist** | `OnEntityDeath` | Again nothing immediately — `npcKills` on the next `player.tally`. No `player.death`: a chicken is not a killfeed entry | +| 6 | **Die to the environment** — fall damage is easiest | `OnPlayerDeath` | `player.death` with `attackerType: "environment"`, a `grid` like `H7`, and **no** `attackerId`. Check the grid against the map: a wrong sign in the row arithmetic mirrors the whole map, and only a human with the map open can see that | +| 7 | **Kill yourself** — `kill` in the F1 console | `OnPlayerDeath` | `attackerType: "self"`, no `attackerId` | +| 8 | **If a second player is available**: kill each other once | `OnPlayerDeath` | `attackerType: "player"`, with `attackerId`, `attackerName`, a `weapon` shortname and a `distance` in metres. This is the killfeed's whole shape, and it is the one row phase 4's page is built from | +| 9 | **Build a foundation, then destroy it yourself** | `OnEntityDeath` | `entity.destroyed` with `ownerId` (yours), `prefab`, `grid` and `attackerId`. Decay must **not** produce one of these — only a player breaking it | +| 10 | **Disconnect** | `OnPlayerDisconnected` | `player.disconnected` with a `reason` and a **`sessionSec`** roughly equal to how long you were on. It also flushes your tally first, so any gathering since the last minute arrives immediately before it | + +Two ordering notes: step 4 must come before step 10 by at least a minute if you want to see the +cadence flush rather than the disconnect flush, and step 1's three frames are the only place the +order between hooks is itself part of the answer. + +--- + +## Reading the result + +From the machine running the sidecar: + +```bash +TOKEN=… # [web].auth_token from sidecar.toml, or `--print-config` +BASE=http://127.0.0.1:8090 + +# The whole walk, oldest first, as a transcript. +curl -s -H "Authorization: Bearer $TOKEN" "$BASE/feed?since=0&limit=500" \ + | python -m json.tool + +# Or one kind at a time. +curl -s -H "Authorization: Bearer $TOKEN" "$BASE/events?kind=player.death&limit=20" +``` + +And from the game console: + +``` +rg.hooks +``` + +Every hook in the walk should now read **fired**, with a count. A hook still `silent` after the step +that should have fired it is the finding — and on Carbon it is the specific question +[`CARBON.md`](../modules/rust/CARBON.md) §6 asks, since Carbon's catalogue omits thirteen uMod names +and nobody has yet checked whether they are renames or holes. + +--- + +## What counts as a pass + +Not "frames arrived". Three things, and the third is the one worth slowing down for: + +1. **Every hook in the table fired**, on both frameworks, from the same plugin file. +2. **Every frame carries the envelope** — `type`, `serverId` and `wipeId` on all of them + ([`PROTOCOL.md`](PROTOCOL.md) §8.1). A player frame without a `wipeId` cannot be attributed to a + wipe and its rollup is lost. +3. **The aggregates are aggregates.** `player.tally` is a delta since the last flush, so two minutes + of chopping is two frames that sum to the total, not two frames each carrying the total. Getting + this backwards makes every leaderboard roughly double, and it looks correct until somebody counts. + +Anything that disagrees with the table is a finding about the game or the framework rather than a +mistake in the table — record it, the same way phases 0, 1 and 2 recorded theirs. diff --git a/rust-link/PROTOCOL.md b/rust-link/PROTOCOL.md index 3eab58c..aa724b8 100644 --- a/rust-link/PROTOCOL.md +++ b/rust-link/PROTOCOL.md @@ -50,7 +50,8 @@ it is listening without one. ## 2. Versioning -The wire version is a single integer, declared in **four** places that must agree: +The wire version is a single integer — **2** as of the read path (§8) — declared in **four** places +that must agree: | Where | Repo | |---|---| @@ -77,7 +78,11 @@ Bump all four in the same change as the emitters, together with this document. ## 3. Protocol 1 — the transport -Everything phase 1 defines, and deliberately nothing more. +Everything phase 1 defines, and deliberately nothing more. It is still the floor every later version +stands on — the framing, the greeting, the heartbeat and the one correlated round trip are unchanged +— but **two things below were amended by protocol 2**: every frame now carries `type`, `serverId` +and `wipeId` (§8.1), and `server.hello` is a *board* rather than a one-off greeting (§8.3). Read §8 +beside this section rather than after it. ### 3.1 Framing @@ -128,7 +133,7 @@ independently of the game, so anything it needs up front has to be re-sent per c | `t` | epoch milliseconds, stamped when the world was read | | `serverId` | this server's stable identity across wipes and restarts, from the plugin's config. **Not derived from the hostname** — an operator renames a server for a season and the site must not lose its history for it | | `bootId` | see §3.2.1 | -| `saveCreatedAt` | when the current save was created. **Raw material for a wipe id, not a wipe id** — deriving one is the website's job and is not yet specified, and emitting a guess now would bake a wrong one into stored rows | +| `saveCreatedAt` | when the current save was created. Protocol 1 called this *raw material for a wipe id* and left deriving one to the website; **§8.2 reversed that** — the plugin derives `wipeId` from this value and stamps it on every frame | Everything from `hostname` down is read from `ConVar.Server` and `BasePlayer.activePlayerList` on the game's main thread. A field the game cannot answer is **absent**, never zero. @@ -196,7 +201,8 @@ Every response carries `X-RustLink-Version`, including `/health` and including e |---|---|---| | `GET /health` | — | **Unauthenticated**, so monitoring can reach it | | `GET /server` | the store | The last `server.hello`. **`204` when the game has never connected** | -| `GET /events?kind=&limit=` | the store | Newest first; `limit` clamped to 1–1000 | +| `GET /events?kind=&wipe=&limit=` | the store | Newest first; `limit` clamped to 1–1000. For a human | +| `GET /feed?since=&limit=` | the store | **Oldest first**, from a cursor. For a consumer that must not miss a row (§8.9) | | `GET /status` | the plugin (RPC) | A live round trip. `503` with no plugin, `504` on no reply | | `GET /ws` | broadcast | The live feed; sends `{"kind":"ws.hello","protocol":1}` on connect | @@ -304,6 +310,7 @@ override the file. | `[web].bind` | `RUSTLINK_WEB_BIND` | `127.0.0.1:8090` | | `[web].auth_token` | `RUSTLINK_WEB_TOKEN` | *(generated on first start)* | | `[store].path` | `RUSTLINK_DB_PATH` | `rust-link.db` | +| `[store].retain_days` | `RUSTLINK_RETAIN_DAYS` | `14` | Two things about those are load-bearing: @@ -323,15 +330,258 @@ writing the file and generating the token if they are missing — and prints it ## 7. What is deliberately not here yet -Protocol 1 is the transport. Every one of these arrives with the phase that needs it, and each is a -version bump: +Protocol 2 is the transport plus the read path. Every one of these arrives with the phase that needs +it, and each is a version bump: -- the read path — player events, kills, clans, presence -- identity and the in-game link code -- the permission mirror -- leases, budgets and the event actions -- the map image over the asset-bridge shape +- identity and the in-game link code (phase 6) +- the permission mirror (phase 7), and plugin configuration edited from the site (phase 7b) +- clans, for core's Team provider (phase 9) +- leases, budgets and the event actions (phases 12-13) +- the map image over the asset-bridge shape (phase 14) The rule that governs all of them: **the sidecar is a dumb forwarder.** It defines no schema for a frame's contents, so a version that adds fields to an event needs no change there — only one that -adds a new *indexed* column does. +adds a new *indexed* column does. §8.1 is what turns that from an intention into a property of the +code. + +--- + +## 8. Protocol 2 — the read path + +Protocol 1 proved a line could travel. Protocol 2 is what travels: presence, deaths, chat, gathering, +moderation and the wipe, on both mod frameworks from one plugin file. + +It is the first version with a *catalogue*, and a catalogue is the thing that grows fastest. So the +shape below is chosen to make growth free everywhere except in the one place that must stay +deliberate — what the public is allowed to see. + +### 8.1 Every frame says what it **is**, not only what it is about + +Protocol 1 routed on `kind`, in a `match` the sidecar had to learn a new arm for on every addition. +Protocol 2 adds **`type`**, and the sidecar files by `type` alone: + +| `type` | Persisted | Broadcast on `/ws` | Routed by `reqId` | Example | +|---|---|---|---|---| +| `event` | appended to the history | yes | no | `player.death` | +| `snapshot` | **replaces** the board of that `kind` | yes | no | `players.online` | +| `reply` | no | no | **yes** | `server.status` | +| `control` | no | no | no | `pong` | + +**This is the dumb-forwarder property made structural.** A protocol version that adds ten event +kinds needs no change in the sidecar at all, because the sidecar never learns a kind — it learns +four verbs, and they are the complete set of things that can be done with a frame. Only a version +that adds a new *indexed column* touches it. + +Every outbound frame therefore carries five fields before anything specific to it: + +```json +{ + "kind": "player.death", + "type": "event", + "t": 1789510452152, + "serverId": "main", + "wipeId": "w-20260915T195817Z" +} +``` + +- **`type` is required.** A frame without one is **dropped and counted**, and the sidecar says so + once per connection. It is not defaulted to `event`: guessing files a board as history, which is + invisible until somebody wonders why the presence board has four thousand rows. The game link has + no version handshake (§2), so this is the place a mismatched pair fails loudly instead of quietly. +- **`serverId` is on every frame**, not only in the server body (R8). A frame is stored beside + frames from five other servers and has to be able to say which one it came from on its own. +- **`wipeId` is on every frame** — see §8.2. + +### 8.2 `wipeId` is derived by the **plugin**, and this amends §3.2 + +§3.2 called `saveCreatedAt` *"raw material for a wipe id, not a wipe id — deriving one is the +website's job"*. That is reversed here, deliberately, and the reason is that by protocol 2 there are +**three** components storing rows that need it: + +``` +w-yyyyMMddTHHmmssZ e.g. w-20260915T195817Z +``` + +It is `SaveRestore.SaveCreatedTime` in UTC, to the second — the same instant `saveCreatedAt` already +reports, in the id-shaped spelling `bootId` uses. The plugin stamps it because the plugin is the only +component that can *read* it; every other component would be re-deriving a value it was already told, +and two derivations of one fact eventually disagree about a boundary. + +Three consequences worth stating rather than discovering: + +- **A server that has never saved has no wipe**, so `wipeId` is **absent**, never `""` and never + `w-unknown`. Absent is a fact; an empty string is a row that will sort beside every other empty + string forever. +- **The id changes on `OnNewSave` and at no other time.** It is not the boot id: a restart re-reads + the same save and reports the same wipe, which is exactly what R12 needs to keep a player's + history across a restart while splitting it across a wipe. +- **A wipe boundary is a fact about the world, not about the bridge.** The plugin re-reads the value + on `OnNewSave` and caches it otherwise; nothing about a reconnect can change it. + +### 8.3 Boards — current state, one producer, re-sent on connect + +A board is chapter 4's word: *current state with exactly one producer, re-sent on every connect*. +Protocol 2 defines two. + +| Board (`kind`) | Holds | +|---|---| +| `server.hello` | the server's own description — §3.2's body, now `type: "snapshot"` | +| `players.online` | who is connected right now: `steamId`, `name`, `connectedAt`, `sleeping` | + +**Boards are re-emitted on connect and on a 60-second cadence thereafter.** The events carry the +story — `player.connected`, `player.disconnected` — and the board is the **reconciliation point**. A +missed event is corrected within a minute rather than persisting until the next restart, and the +acceptance criterion *"a restarted sidecar is fully populated within one connection"* is met by +construction rather than by hoping no event was in flight. + +The cadence is cheap on purpose: a full board for a 100-slot server is a few kilobytes, and a server +with nobody on it emits an empty array, which is a different answer from having said nothing. + +### 8.4 The catalogue + +Every kind protocol 2 defines, and the hook behind it. **`class` is not a field on the wire** — see +§8.5 — it is what this table binds the module's allowlist to. + +| `kind` | Hook | `class` | Carries | +|---|---|---|---| +| `player.connected` | `OnPlayerConnected` | public | steamId, name | +| `player.disconnected` | `OnPlayerDisconnected` | public | steamId, name, reason, sessionSec | +| `player.respawned` | `OnPlayerRespawned` | public | steamId | +| `player.death` | `OnPlayerDeath` | public | victim, attacker, attackerType, weapon, distance, grid | +| `player.chat` | `OnPlayerChat` | public | steamId, name, channel, message | +| `player.tally` | *aggregate* — see §8.6 | public | steamId, gathered{}, npcKills, structures | +| `entity.destroyed` | `OnEntityDeath` on owned building blocks | **staff** | ownerId, prefab, grid, attacker | +| `player.reported` | `OnPlayerReported` | **staff** | reporter, target, subject, message, type | +| `player.banned` / `player.unbanned` | `OnUserBanned` / `OnUserUnbanned` | **staff** | id, name, **ip**, reason | +| `player.login.attempt` | `CanUserLogin` *(observed, never answered)* | **staff** | id, name, **ip** | +| `player.approved` | `OnUserApproved` | **staff** | id, name, **ip** | +| `server.wipe` | `OnNewSave` | public | the new `wipeId`, the one it replaced | +| `server.initialized` | `OnServerInitialized` | public | — | +| `server.shutdown` | `OnServerShutdown` | public | — | + +`grid` is the Rust map reference (`H7`), not a coordinate. A death's grid is where a fight happened +and every community site shows it; a **structure's** grid is where somebody lives, which is why +`entity.destroyed` is staff-class here and why R9 makes the same distinction for map layers. + +**Three hooks are deliberately not in this wave, and none of them is an oversight:** `OnEntityTakeDamage` +and `OnFrame`/`OnTick` fire at a rate that makes a bridge a performance regression, and nothing in +phases 3–19 needs per-hit or per-frame fidelity. R17's warning about chatty zone transitions is the +same rule: **subscribe selectively; the cost of a hook is paid on the game's main thread.** + +### 8.5 The class is enforced by the **module**, not declared on the wire + +The wire carries no visibility field, and this is a security decision rather than an economy. + +**A boundary must be enforced by the side that serves, never declared by the side that sends.** The +website's own shard fan-out works this way — a public SSE stream with an allowlist of event kinds, +and an admin stream that adds the rest — and the property that makes it trustworthy is that a +compromised or merely out-of-date sender cannot widen it. A `"class":"public"` field on the frame +would move the decision to the game host. + +So: the table in §8.4 is the specification, `module-rust` holds the allowlist, and it is +**default-deny** — a kind the allowlist has never heard of is not public. The module's own test holds +its allowlist against this document, so adding a kind here without classifying it there fails a +build rather than shipping an IP address to a public page. + +`player.login.attempt`, `player.approved` and `player.banned` carry **IP addresses**, and +`player.reported` carries the text of one player's complaint about another. They are stored because +an operator chasing ban evasion needs them and because the sidecar persists what it is told; they +reach no tier below admin, and the raw window that holds them is bounded (§8.9). + +### 8.6 Two things are aggregated in the plugin, and that is the interesting part of this phase + +`OnDispenserGather` fires on **every swing at a tree**. A single player chopping for a minute is +hundreds of hooks; ten players gathering is a frame rate problem in the bridge rather than in the +game. The same is true of animal and scientist kills, at a lower rate. + +Neither is interesting per occurrence — nobody wants a killfeed of chickens — and both are wanted +*in total*, for the leaderboard. So the plugin keeps a per-player tally on the main thread and flushes +it as one `player.tally` frame: + +- on a **60-second cadence**, for players with a non-zero tally; +- on **disconnect**, so a session's last minute is not lost; +- at `OnServerShutdown`, which is the flush that covers a restart. + +A plugin *reload* is the one case that loses a tally, by choice: `Unload` runs on the game's main +thread, and draining the outbound queue there means waiting on a socket from the main thread — the +stall phase 1 removed. Under a minute of one player's gathering is the price, and a wedged peer +would make the cure worse than the disease. + +A tally frame is a **delta, not a running total** — it reports what happened since the last flush, +so the consumer sums rather than diffs and a missed frame costs that interval instead of corrupting +the series. + +This is the general rule for every later wave: **if a hook can fire more than once a second per +player, it is a counter, not an event.** + +### 8.7 The read path never vetoes, and it is structural rather than disciplined + +Four hooks in §8.4 are documented by uMod as *"returning a non-null value overrides default +behavior"* — `OnPlayerDeath`, `OnDispenserGather` and `CanUserLogin` among them. A read-path bridge +that returned something by accident would cancel a death, swallow a player's wood, or refuse a +login, and it would do it on a production server at 3am. + +**So every vetoable hook in the read path is declared `void`.** Both frameworks bind hooks by name +and arity and take the method's return value; a `void` method returns nothing and therefore cannot +override anything. The rule is enforced by the signature rather than by remembering to write +`return null`, which is the only version of this rule that survives a year of edits. + +`CanUserLogin` is in the wave for what it *observes*, never for what it answers. + +### 8.8 A login denial is not a hook — and §10 of `PLAN.md` says it is + +`PLAN.md` §10 sources the `rust.login.denied` trigger from `CanUserLogin`. Reading the hook says that +cannot work: `CanUserLogin` is called on **every** connection attempt, and the only way to learn of a +denial from it is to *be* the denier, which §8.7 forbids. uMod publishes no `OnUserRejected`. + +What the game can actually tell us is two facts — an attempt, and an approval — so protocol 2 emits +both and **a denial is the absence of an approval** for an attempt, decided by a deferred read rather +than by a hook. Phase 10 owns that pairing; protocol 2 owes it the two frames and the `t` on each. + +Recorded here because it is a correction to a catalogue, not a defect: the trigger survives, its +source changes. + +### 8.9 History, cursors and retention + +Three changes on the sidecar's own side follow from a catalogue that actually produces volume. + +**`events` gains `server_id` and `wipe_id` as indexed columns.** This is the one migration shape the +store's own header predicted: *"only a version that adds a new indexed column ever needs a +migration"*. It is applied as an `ALTER` guarded by a column check, never as an edit to the `CREATE` +— the same rule the website's schema fragments live under, for the same reason. + +**A new route, `GET /feed?since=&limit=`, is the ingest cursor**, and it is deliberately *not* +`/events` with a flag: + +| Route | Order | For | +|---|---|---| +| `GET /events?kind=&wipe=&limit=` | newest first | a human, an admin screen, a point-in-time look | +| `GET /feed?since=&limit=` | **oldest first**, from a cursor | a consumer that must not miss a row | + +One route with two orderings depending on a query parameter is a trap: every caller that forgets the +parameter gets the other one silently, and for the ingesting caller that means it advances its cursor +past rows it never read. Two routes, one ordering each. + +`/feed` items are wrapped rather than bare, because a cursor needs the row's identity: + +```json +{ "items": [ { "id": 1041, "t": 1789…, "kind": "player.death", "frame": { … } } ], + "lastId": 1041, "more": false } +``` + +`more` is `true` when the page filled, so a consumer that has fallen an hour behind drains at its own +pace instead of guessing from a count. + +**Omitting `since` asks where the end is** — no rows, and the current `lastId`. `since=0` is the +other question entirely: replay everything retained. That is deliberate, because the two intentions +must not be separated by whether somebody typed a parameter: a module installed today against a +month-old sidecar wants what happens next, not a fortnight of deaths it has no rollups for. + +**The store prunes.** `[store].retain_days` (default 14) bounds the event history, swept hourly. +Three things make that safe rather than lossy: the website holds the permanent per-wipe rollups +(R12), boards are never pruned because they hold exactly one row per kind, and the sidecar's database +lives inside a game container whose disk is the operator's (R20). A store that grows without bound on +a game host is a wipe-day outage waiting for a busy month. + +--- +