feat: ingest protocol 2, and keep the record a wipe cannot erase
The module half of the read path. Seven tables, an ingest cursor, four public routes, and one file whose only job is deciding who may see what. **The record and the window are different things.** `rust_player_wipe_stats` and `rust_gather_totals` are permanent and per-wipe, so all-time is those rows SUMmed rather than a second set of counters that can disagree with them — that is R12's "per-wipe detail plus all-time rollups" in one table instead of two. `rust_events` is a bounded 30-day window of raw frames for the killfeed, and `rust_presence` is a board: replaced wholesale, never appended. **The feed is a cursor, not a socket, and the header says why.** Core runs Node 20, where a global WebSocket is still behind a flag, so a socket means taking `ws` — against a release that asserts it has no runtime dependencies (D5). The deciding argument is the other one though: a socket needs a cursor anyway, for whatever it missed while the module was restarting, and the catch-up path is the one that has to be right. A cursor alone is one mechanism exercised every five seconds rather than two where the second only runs after an outage. **The cursor advances after the batch, never before.** A crash between the two re-reads events already counted, which inflates a total; the other order loses them silently and for ever. One is visible and bounded, the other is invisible and permanent, so the code fails in the visible direction. A server with no cursor starts at the sidecar's current END rather than at zero — replaying a fortnight of deaths into stats for wipes the site never saw is not a catch-up. **`catalogue.js` is a security boundary, default-deny.** Protocol 2 carries IP addresses (login attempts, approvals, bans), one player's report about another, and the grid reference of somebody's base. They are stored, because an operator chasing ban evasion needs them; they are not served below the admin tier. The allowlist lives here rather than as a field on the wire, because a boundary declared by the sender is one a compromised or merely out-of-date game host can widen — the same reason core's own shard fan-out filters on the serving side. A kind this build has never heard of is not public, and a test holds the list against PROTOCOL.md §8.4 so that adding a kind to the protocol without classifying it fails a build. `PROTOCOL_VERSION` goes to 2 here in the same change as the emitters, though this module consumes none of the new frames yet: the sidecar refuses a mismatched client with a 409, so a module left on 1 would stop being able to read the board it has been reading all along. A constant that lags the deployment is an outage with a version number on it. 95 server tests, 20 client tests, every guard green, and `routes.manifest.json` regenerated against a real core at the pinned ref: 10 routes, all documented, none of core's moved. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
This commit is contained in:
@@ -14,14 +14,20 @@
|
||||
-- Every table here is prefixed `rust_`, which is this module's id and the only
|
||||
-- prefix it may create under.
|
||||
--
|
||||
-- ── Two tables, and the split between them is the whole design ────────────
|
||||
-- ── Four kinds of table, and the split between them is the whole design ───
|
||||
--
|
||||
-- `rust_servers` is CONFIGURATION: rows an operator writes, from Admin → Rust.
|
||||
-- `rust_server_state` is OBSERVED STATE: rows this module writes from what a
|
||||
-- sidecar reported. They are separate tables rather than columns on one because
|
||||
-- they have different writers, different lifetimes and different audiences —
|
||||
-- and because a purge of observed state while keeping the configuration is a
|
||||
-- thing an operator will eventually want.
|
||||
-- CONFIGURATION `rust_servers` — rows an operator writes, from Admin → Rust.
|
||||
-- OBSERVED STATE `rust_server_state`, `rust_presence` — what a sidecar last
|
||||
-- reported, replaced rather than appended.
|
||||
-- THE RECORD `rust_wipes`, `rust_players`, `rust_player_wipe_stats`,
|
||||
-- `rust_gather_totals` — permanent, and the reason a wipe does
|
||||
-- not erase a player's history.
|
||||
-- THE WINDOW `rust_events` — recent detail, bounded by a sweep.
|
||||
--
|
||||
-- They are separate tables rather than columns on one because they have
|
||||
-- different writers, different lifetimes and different audiences — and because
|
||||
-- a purge of observed state while keeping the configuration is a thing an
|
||||
-- operator will eventually want.
|
||||
--
|
||||
-- Teardown is `purge.sql`, which no boot ever runs.
|
||||
|
||||
@@ -97,3 +103,198 @@ CREATE TABLE IF NOT EXISTS rust_server_state (
|
||||
CONSTRAINT fk_rust_server_state_server
|
||||
FOREIGN KEY (server_id) REFERENCES rust_servers (id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
|
||||
-- ── The read path ─────────────────────────────────────────────────────────
|
||||
--
|
||||
-- Protocol 2 turned the bridge from a greeting into a catalogue, and these are
|
||||
-- the tables that hold it. They divide on one line, and it is the line R12 drew:
|
||||
--
|
||||
-- PERMANENT `rust_wipes`, `rust_players`, `rust_player_wipe_stats`,
|
||||
-- `rust_gather_totals` — a player's record, kept for ever. All-time
|
||||
-- is a SUM across wipes rather than a second set of counters, so
|
||||
-- there is no second number that can disagree with the first.
|
||||
--
|
||||
-- BOUNDED `rust_events` — the recent raw window the killfeed reads, pruned
|
||||
-- on a sweep. It is detail, not record: losing last month's
|
||||
-- individual deaths costs a scroll-back, losing last month's
|
||||
-- totals costs a player their history.
|
||||
--
|
||||
-- DERIVED `rust_presence` — who is on right now, replaced wholesale from
|
||||
-- the `players.online` board. Never a history, never appended.
|
||||
--
|
||||
-- The sidecar keeps its own bounded copy of the same events (default 14 days),
|
||||
-- so shortening either window loses recent detail and neither loses a total.
|
||||
|
||||
|
||||
-- ── Wipes ─────────────────────────────────────────────────────────────────
|
||||
--
|
||||
-- One row per (server, wipe). The id is the plugin's, derived from the save's
|
||||
-- creation time and stamped on every frame (PROTOCOL.md §8.2) — this module
|
||||
-- never derives one, because two derivations of one fact eventually disagree
|
||||
-- about a boundary.
|
||||
--
|
||||
-- Rows appear by being MENTIONED: the first frame carrying a wipe id this module
|
||||
-- has not seen creates it. There is no "start a wipe" call and there must not be
|
||||
-- one, because the website is not present when a wipe happens — a wipe is a fact
|
||||
-- about a world that was restarted while nobody was watching.
|
||||
CREATE TABLE IF NOT EXISTS rust_wipes (
|
||||
server_id VARCHAR(64) NOT NULL,
|
||||
wipe_id VARCHAR(48) NOT NULL,
|
||||
save_created_at VARCHAR(32) NULL,
|
||||
first_seen DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
last_seen DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
PRIMARY KEY (server_id, wipe_id),
|
||||
CONSTRAINT fk_rust_wipes_server
|
||||
FOREIGN KEY (server_id) REFERENCES rust_servers (id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
|
||||
-- ── Players ───────────────────────────────────────────────────────────────
|
||||
--
|
||||
-- Identity, and deliberately nothing else. It is keyed on the Steam id alone
|
||||
-- and carries no server: a player is the same person on all six of a community's
|
||||
-- servers, and everything that is per-server lives in the stats table.
|
||||
--
|
||||
-- `user_id` is NOT here. Linking a Steam id to a website account is phase 6's
|
||||
-- work (R1), and a column waiting for it would be a column every read has to
|
||||
-- remember is always null.
|
||||
CREATE TABLE IF NOT EXISTS rust_players (
|
||||
steam_id VARCHAR(32) NOT NULL PRIMARY KEY,
|
||||
name VARCHAR(191) NULL,
|
||||
first_seen DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
last_seen DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
|
||||
-- ── The permanent record ──────────────────────────────────────────────────
|
||||
--
|
||||
-- One row per player per wipe per server, and the only counters this module
|
||||
-- keeps. R12's "per-wipe detail plus all-time rollups" is satisfied by SUMming
|
||||
-- this rather than by maintaining a second all-time row, because two counters
|
||||
-- for one fact drift the first time an ingest is replayed.
|
||||
--
|
||||
-- Every column is a COUNT that only ever goes up within a wipe, which is what
|
||||
-- makes ingest idempotent-ish in the only way that matters: the cursor advances
|
||||
-- only after the batch commits, so a crash re-reads a batch it has not counted.
|
||||
--
|
||||
-- `playtime_sec` comes from `sessionSec` on a disconnect, and a session whose
|
||||
-- start this module never saw contributes NOTHING rather than zero — the plugin
|
||||
-- omits the field, the ingest skips it, and the number stays honestly short
|
||||
-- instead of quietly wrong.
|
||||
CREATE TABLE IF NOT EXISTS rust_player_wipe_stats (
|
||||
server_id VARCHAR(64) NOT NULL,
|
||||
wipe_id VARCHAR(48) NOT NULL,
|
||||
steam_id VARCHAR(32) NOT NULL,
|
||||
kills INT UNSIGNED NOT NULL DEFAULT 0,
|
||||
deaths INT UNSIGNED NOT NULL DEFAULT 0,
|
||||
suicides INT UNSIGNED NOT NULL DEFAULT 0,
|
||||
npc_kills INT UNSIGNED NOT NULL DEFAULT 0,
|
||||
structures INT UNSIGNED NOT NULL DEFAULT 0,
|
||||
sessions INT UNSIGNED NOT NULL DEFAULT 0,
|
||||
playtime_sec BIGINT UNSIGNED NOT NULL DEFAULT 0,
|
||||
last_seen DATETIME NULL,
|
||||
PRIMARY KEY (server_id, wipe_id, steam_id),
|
||||
KEY idx_rust_stats_kills (server_id, wipe_id, kills DESC),
|
||||
KEY idx_rust_stats_player (steam_id)
|
||||
);
|
||||
|
||||
|
||||
-- ── What they gathered ────────────────────────────────────────────────────
|
||||
--
|
||||
-- A row per resource rather than a JSON blob on the stats row, for one reason:
|
||||
-- the leaderboard question is "who gathered the most sulfur this wipe", and that
|
||||
-- is an ORDER BY over a column in every SQL engine and a JSON function call in
|
||||
-- exactly one. The resource name is the game's own shortname, unknown in advance
|
||||
-- and not worth a lookup table.
|
||||
CREATE TABLE IF NOT EXISTS rust_gather_totals (
|
||||
server_id VARCHAR(64) NOT NULL,
|
||||
wipe_id VARCHAR(48) NOT NULL,
|
||||
steam_id VARCHAR(32) NOT NULL,
|
||||
resource VARCHAR(64) NOT NULL,
|
||||
amount BIGINT UNSIGNED NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, wipe_id, steam_id, resource),
|
||||
KEY idx_rust_gather_top (server_id, wipe_id, resource, amount DESC)
|
||||
);
|
||||
|
||||
|
||||
-- ── The recent raw window ─────────────────────────────────────────────────
|
||||
--
|
||||
-- Every ingested event, whole, for as long as the retention sweep keeps it. The
|
||||
-- killfeed reads this; so does an admin looking at what happened.
|
||||
--
|
||||
-- `raw` holds the entire frame and the columns beside it are only what a query
|
||||
-- needs to reach — the same rule the sidecar's own store follows, one hop along:
|
||||
-- a protocol version that adds a field needs no migration here.
|
||||
--
|
||||
-- **`kind` is a security boundary, not a label.** Some kinds carry IP addresses
|
||||
-- and player reports (PROTOCOL.md §8.4), and what makes them safe is that the
|
||||
-- public read is filtered by an allowlist this module holds, default-deny. The
|
||||
-- rows are stored either way, because an operator chasing ban evasion needs them.
|
||||
CREATE TABLE IF NOT EXISTS rust_events (
|
||||
id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
|
||||
server_id VARCHAR(64) NOT NULL,
|
||||
wipe_id VARCHAR(48) NULL,
|
||||
kind VARCHAR(64) NOT NULL,
|
||||
t BIGINT NOT NULL,
|
||||
steam_id VARCHAR(32) NULL,
|
||||
raw LONGTEXT NOT NULL,
|
||||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
KEY idx_rust_events_server (server_id, id DESC),
|
||||
KEY idx_rust_events_kind (server_id, kind, id DESC),
|
||||
KEY idx_rust_events_wipe (server_id, wipe_id, id DESC),
|
||||
KEY idx_rust_events_created (created_at)
|
||||
);
|
||||
|
||||
|
||||
-- ── Who is on right now ───────────────────────────────────────────────────
|
||||
--
|
||||
-- Replaced wholesale every time the `players.online` board arrives, which is on
|
||||
-- every bridge connect and every 60 seconds. It is a BOARD, and the reason it is
|
||||
-- its own table rather than rows in `rust_events` is that a board answers "now"
|
||||
-- and an event answers "then"; storing a board as history is the mistake the
|
||||
-- wire's `type` field exists to prevent, and it would be a shame to make it here
|
||||
-- after the sidecar went to the trouble of not making it there.
|
||||
CREATE TABLE IF NOT EXISTS rust_presence (
|
||||
server_id VARCHAR(64) NOT NULL,
|
||||
steam_id VARCHAR(32) NOT NULL,
|
||||
name VARCHAR(191) NULL,
|
||||
sleeping TINYINT(1) NOT NULL DEFAULT 0,
|
||||
connected_at DATETIME NULL,
|
||||
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
PRIMARY KEY (server_id, steam_id)
|
||||
);
|
||||
|
||||
|
||||
-- ── The ingest cursor ─────────────────────────────────────────────────────
|
||||
--
|
||||
-- Where this module has read up to in each sidecar's feed. One row per server.
|
||||
--
|
||||
-- It is persisted rather than held in memory because the alternative is a module
|
||||
-- that re-reads everything on every boot or nothing at all, and both are wrong in
|
||||
-- a way that only shows up in production. The cursor advances **after** the batch
|
||||
-- is written, never before: a crash mid-batch re-reads rows it has not counted,
|
||||
-- which is the safe direction to be wrong in.
|
||||
--
|
||||
-- A NEW server starts at the sidecar's current end rather than at zero (see
|
||||
-- `GET /feed` with no `since`). A module installed today against a sidecar that
|
||||
-- has been running a month wants what happens next — replaying a fortnight of
|
||||
-- deaths into stats whose wipes it never saw is not a catch-up, it is a
|
||||
-- fabrication of history it was not present for.
|
||||
CREATE TABLE IF NOT EXISTS rust_ingest_cursor (
|
||||
server_id VARCHAR(64) NOT NULL PRIMARY KEY,
|
||||
last_event_id BIGINT UNSIGNED NOT NULL DEFAULT 0,
|
||||
events_seen BIGINT UNSIGNED NOT NULL DEFAULT 0,
|
||||
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
CONSTRAINT fk_rust_cursor_server
|
||||
FOREIGN KEY (server_id) REFERENCES rust_servers (id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
|
||||
-- ── Changes to tables that already shipped ────────────────────────────────
|
||||
--
|
||||
-- An ALTER below the CREATE, never an edit to it: `CREATE TABLE IF NOT EXISTS`
|
||||
-- does nothing against a database that already has the table, so an edited column
|
||||
-- would reach fresh installs only — which is the worst possible distribution for
|
||||
-- a schema change, because it works everywhere it is tested.
|
||||
ALTER TABLE rust_server_state ADD COLUMN IF NOT EXISTS wipe_id VARCHAR(48) NULL;
|
||||
|
||||
Reference in New Issue
Block a user