feat: ingest protocol 2, and keep the record a wipe cannot erase
All checks were successful
PR Checks / client-build (pull_request) Successful in 17s
PR Checks / frozen-manifest (pull_request) Successful in 44s
PR Checks / server-tests (pull_request) Successful in 7m57s

The module half of the read path. Seven tables, an ingest cursor, four public
routes, and one file whose only job is deciding who may see what.

**The record and the window are different things.** `rust_player_wipe_stats` and
`rust_gather_totals` are permanent and per-wipe, so all-time is those rows SUMmed
rather than a second set of counters that can disagree with them — that is R12's
"per-wipe detail plus all-time rollups" in one table instead of two.
`rust_events` is a bounded 30-day window of raw frames for the killfeed, and
`rust_presence` is a board: replaced wholesale, never appended.

**The feed is a cursor, not a socket, and the header says why.** Core runs Node
20, where a global WebSocket is still behind a flag, so a socket means taking
`ws` — against a release that asserts it has no runtime dependencies (D5). The
deciding argument is the other one though: a socket needs a cursor anyway, for
whatever it missed while the module was restarting, and the catch-up path is the
one that has to be right. A cursor alone is one mechanism exercised every five
seconds rather than two where the second only runs after an outage.

**The cursor advances after the batch, never before.** A crash between the two
re-reads events already counted, which inflates a total; the other order loses
them silently and for ever. One is visible and bounded, the other is invisible
and permanent, so the code fails in the visible direction. A server with no
cursor starts at the sidecar's current END rather than at zero — replaying a
fortnight of deaths into stats for wipes the site never saw is not a catch-up.

**`catalogue.js` is a security boundary, default-deny.** Protocol 2 carries IP
addresses (login attempts, approvals, bans), one player's report about another,
and the grid reference of somebody's base. They are stored, because an operator
chasing ban evasion needs them; they are not served below the admin tier. The
allowlist lives here rather than as a field on the wire, because a boundary
declared by the sender is one a compromised or merely out-of-date game host can
widen — the same reason core's own shard fan-out filters on the serving side. A
kind this build has never heard of is not public, and a test holds the list
against PROTOCOL.md §8.4 so that adding a kind to the protocol without
classifying it fails a build.

`PROTOCOL_VERSION` goes to 2 here in the same change as the emitters, though this
module consumes none of the new frames yet: the sidecar refuses a mismatched
client with a 409, so a module left on 1 would stop being able to read the board
it has been reading all along. A constant that lags the deployment is an outage
with a version number on it.

95 server tests, 20 client tests, every guard green, and `routes.manifest.json`
regenerated against a real core at the pinned ref: 10 routes, all documented,
none of core's moved.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
This commit is contained in:
2026-09-16 08:37:16 -05:00
parent 9018e55488
commit f211969ee1
17 changed files with 2054 additions and 27 deletions

View File

@@ -14,14 +14,20 @@
-- Every table here is prefixed `rust_`, which is this module's id and the only
-- prefix it may create under.
--
-- ── Two tables, and the split between them is the whole design ────────────
-- ── Four kinds of table, and the split between them is the whole design ───
--
-- `rust_servers` is CONFIGURATION: rows an operator writes, from Admin → Rust.
-- `rust_server_state` is OBSERVED STATE: rows this module writes from what a
-- sidecar reported. They are separate tables rather than columns on one because
-- they have different writers, different lifetimes and different audiences —
-- and because a purge of observed state while keeping the configuration is a
-- thing an operator will eventually want.
-- CONFIGURATION `rust_servers` — rows an operator writes, from Admin → Rust.
-- OBSERVED STATE `rust_server_state`, `rust_presence` — what a sidecar last
-- reported, replaced rather than appended.
-- THE RECORD `rust_wipes`, `rust_players`, `rust_player_wipe_stats`,
-- `rust_gather_totals` — permanent, and the reason a wipe does
-- not erase a player's history.
-- THE WINDOW `rust_events` — recent detail, bounded by a sweep.
--
-- They are separate tables rather than columns on one because they have
-- different writers, different lifetimes and different audiences — and because
-- a purge of observed state while keeping the configuration is a thing an
-- operator will eventually want.
--
-- Teardown is `purge.sql`, which no boot ever runs.
@@ -97,3 +103,198 @@ CREATE TABLE IF NOT EXISTS rust_server_state (
CONSTRAINT fk_rust_server_state_server
FOREIGN KEY (server_id) REFERENCES rust_servers (id) ON DELETE CASCADE
);
-- ── The read path ─────────────────────────────────────────────────────────
--
-- Protocol 2 turned the bridge from a greeting into a catalogue, and these are
-- the tables that hold it. They divide on one line, and it is the line R12 drew:
--
-- PERMANENT `rust_wipes`, `rust_players`, `rust_player_wipe_stats`,
-- `rust_gather_totals` — a player's record, kept for ever. All-time
-- is a SUM across wipes rather than a second set of counters, so
-- there is no second number that can disagree with the first.
--
-- BOUNDED `rust_events` — the recent raw window the killfeed reads, pruned
-- on a sweep. It is detail, not record: losing last month's
-- individual deaths costs a scroll-back, losing last month's
-- totals costs a player their history.
--
-- DERIVED `rust_presence` — who is on right now, replaced wholesale from
-- the `players.online` board. Never a history, never appended.
--
-- The sidecar keeps its own bounded copy of the same events (default 14 days),
-- so shortening either window loses recent detail and neither loses a total.
-- ── Wipes ─────────────────────────────────────────────────────────────────
--
-- One row per (server, wipe). The id is the plugin's, derived from the save's
-- creation time and stamped on every frame (PROTOCOL.md §8.2) — this module
-- never derives one, because two derivations of one fact eventually disagree
-- about a boundary.
--
-- Rows appear by being MENTIONED: the first frame carrying a wipe id this module
-- has not seen creates it. There is no "start a wipe" call and there must not be
-- one, because the website is not present when a wipe happens — a wipe is a fact
-- about a world that was restarted while nobody was watching.
CREATE TABLE IF NOT EXISTS rust_wipes (
server_id VARCHAR(64) NOT NULL,
wipe_id VARCHAR(48) NOT NULL,
save_created_at VARCHAR(32) NULL,
first_seen DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_seen DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (server_id, wipe_id),
CONSTRAINT fk_rust_wipes_server
FOREIGN KEY (server_id) REFERENCES rust_servers (id) ON DELETE CASCADE
);
-- ── Players ───────────────────────────────────────────────────────────────
--
-- Identity, and deliberately nothing else. It is keyed on the Steam id alone
-- and carries no server: a player is the same person on all six of a community's
-- servers, and everything that is per-server lives in the stats table.
--
-- `user_id` is NOT here. Linking a Steam id to a website account is phase 6's
-- work (R1), and a column waiting for it would be a column every read has to
-- remember is always null.
CREATE TABLE IF NOT EXISTS rust_players (
steam_id VARCHAR(32) NOT NULL PRIMARY KEY,
name VARCHAR(191) NULL,
first_seen DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_seen DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
);
-- ── The permanent record ──────────────────────────────────────────────────
--
-- One row per player per wipe per server, and the only counters this module
-- keeps. R12's "per-wipe detail plus all-time rollups" is satisfied by SUMming
-- this rather than by maintaining a second all-time row, because two counters
-- for one fact drift the first time an ingest is replayed.
--
-- Every column is a COUNT that only ever goes up within a wipe, which is what
-- makes ingest idempotent-ish in the only way that matters: the cursor advances
-- only after the batch commits, so a crash re-reads a batch it has not counted.
--
-- `playtime_sec` comes from `sessionSec` on a disconnect, and a session whose
-- start this module never saw contributes NOTHING rather than zero — the plugin
-- omits the field, the ingest skips it, and the number stays honestly short
-- instead of quietly wrong.
CREATE TABLE IF NOT EXISTS rust_player_wipe_stats (
server_id VARCHAR(64) NOT NULL,
wipe_id VARCHAR(48) NOT NULL,
steam_id VARCHAR(32) NOT NULL,
kills INT UNSIGNED NOT NULL DEFAULT 0,
deaths INT UNSIGNED NOT NULL DEFAULT 0,
suicides INT UNSIGNED NOT NULL DEFAULT 0,
npc_kills INT UNSIGNED NOT NULL DEFAULT 0,
structures INT UNSIGNED NOT NULL DEFAULT 0,
sessions INT UNSIGNED NOT NULL DEFAULT 0,
playtime_sec BIGINT UNSIGNED NOT NULL DEFAULT 0,
last_seen DATETIME NULL,
PRIMARY KEY (server_id, wipe_id, steam_id),
KEY idx_rust_stats_kills (server_id, wipe_id, kills DESC),
KEY idx_rust_stats_player (steam_id)
);
-- ── What they gathered ────────────────────────────────────────────────────
--
-- A row per resource rather than a JSON blob on the stats row, for one reason:
-- the leaderboard question is "who gathered the most sulfur this wipe", and that
-- is an ORDER BY over a column in every SQL engine and a JSON function call in
-- exactly one. The resource name is the game's own shortname, unknown in advance
-- and not worth a lookup table.
CREATE TABLE IF NOT EXISTS rust_gather_totals (
server_id VARCHAR(64) NOT NULL,
wipe_id VARCHAR(48) NOT NULL,
steam_id VARCHAR(32) NOT NULL,
resource VARCHAR(64) NOT NULL,
amount BIGINT UNSIGNED NOT NULL DEFAULT 0,
PRIMARY KEY (server_id, wipe_id, steam_id, resource),
KEY idx_rust_gather_top (server_id, wipe_id, resource, amount DESC)
);
-- ── The recent raw window ─────────────────────────────────────────────────
--
-- Every ingested event, whole, for as long as the retention sweep keeps it. The
-- killfeed reads this; so does an admin looking at what happened.
--
-- `raw` holds the entire frame and the columns beside it are only what a query
-- needs to reach — the same rule the sidecar's own store follows, one hop along:
-- a protocol version that adds a field needs no migration here.
--
-- **`kind` is a security boundary, not a label.** Some kinds carry IP addresses
-- and player reports (PROTOCOL.md §8.4), and what makes them safe is that the
-- public read is filtered by an allowlist this module holds, default-deny. The
-- rows are stored either way, because an operator chasing ban evasion needs them.
CREATE TABLE IF NOT EXISTS rust_events (
id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
server_id VARCHAR(64) NOT NULL,
wipe_id VARCHAR(48) NULL,
kind VARCHAR(64) NOT NULL,
t BIGINT NOT NULL,
steam_id VARCHAR(32) NULL,
raw LONGTEXT NOT NULL,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
KEY idx_rust_events_server (server_id, id DESC),
KEY idx_rust_events_kind (server_id, kind, id DESC),
KEY idx_rust_events_wipe (server_id, wipe_id, id DESC),
KEY idx_rust_events_created (created_at)
);
-- ── Who is on right now ───────────────────────────────────────────────────
--
-- Replaced wholesale every time the `players.online` board arrives, which is on
-- every bridge connect and every 60 seconds. It is a BOARD, and the reason it is
-- its own table rather than rows in `rust_events` is that a board answers "now"
-- and an event answers "then"; storing a board as history is the mistake the
-- wire's `type` field exists to prevent, and it would be a shame to make it here
-- after the sidecar went to the trouble of not making it there.
CREATE TABLE IF NOT EXISTS rust_presence (
server_id VARCHAR(64) NOT NULL,
steam_id VARCHAR(32) NOT NULL,
name VARCHAR(191) NULL,
sleeping TINYINT(1) NOT NULL DEFAULT 0,
connected_at DATETIME NULL,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (server_id, steam_id)
);
-- ── The ingest cursor ─────────────────────────────────────────────────────
--
-- Where this module has read up to in each sidecar's feed. One row per server.
--
-- It is persisted rather than held in memory because the alternative is a module
-- that re-reads everything on every boot or nothing at all, and both are wrong in
-- a way that only shows up in production. The cursor advances **after** the batch
-- is written, never before: a crash mid-batch re-reads rows it has not counted,
-- which is the safe direction to be wrong in.
--
-- A NEW server starts at the sidecar's current end rather than at zero (see
-- `GET /feed` with no `since`). A module installed today against a sidecar that
-- has been running a month wants what happens next — replaying a fortnight of
-- deaths into stats whose wipes it never saw is not a catch-up, it is a
-- fabrication of history it was not present for.
CREATE TABLE IF NOT EXISTS rust_ingest_cursor (
server_id VARCHAR(64) NOT NULL PRIMARY KEY,
last_event_id BIGINT UNSIGNED NOT NULL DEFAULT 0,
events_seen BIGINT UNSIGNED NOT NULL DEFAULT 0,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT fk_rust_cursor_server
FOREIGN KEY (server_id) REFERENCES rust_servers (id) ON DELETE CASCADE
);
-- ── Changes to tables that already shipped ────────────────────────────────
--
-- An ALTER below the CREATE, never an edit to it: `CREATE TABLE IF NOT EXISTS`
-- does nothing against a database that already has the table, so an edited column
-- would reach fresh installs only — which is the worst possible distribution for
-- a schema change, because it works everywhere it is tested.
ALTER TABLE rust_server_state ADD COLUMN IF NOT EXISTS wipe_id VARCHAR(48) NULL;