feat: ingest protocol 2, and keep the record a wipe cannot erase

The module half of the read path. Seven tables, an ingest cursor, four public
routes, and one file whose only job is deciding who may see what.

**The record and the window are different things.** `rust_player_wipe_stats` and
`rust_gather_totals` are permanent and per-wipe, so all-time is those rows SUMmed
rather than a second set of counters that can disagree with them — that is R12's
"per-wipe detail plus all-time rollups" in one table instead of two.
`rust_events` is a bounded 30-day window of raw frames for the killfeed, and
`rust_presence` is a board: replaced wholesale, never appended.

**The feed is a cursor, not a socket, and the header says why.** Core runs Node
20, where a global WebSocket is still behind a flag, so a socket means taking
`ws` — against a release that asserts it has no runtime dependencies (D5). The
deciding argument is the other one though: a socket needs a cursor anyway, for
whatever it missed while the module was restarting, and the catch-up path is the
one that has to be right. A cursor alone is one mechanism exercised every five
seconds rather than two where the second only runs after an outage.

**The cursor advances after the batch, never before.** A crash between the two
re-reads events already counted, which inflates a total; the other order loses
them silently and for ever. One is visible and bounded, the other is invisible
and permanent, so the code fails in the visible direction. A server with no
cursor starts at the sidecar's current END rather than at zero — replaying a
fortnight of deaths into stats for wipes the site never saw is not a catch-up.

**`catalogue.js` is a security boundary, default-deny.** Protocol 2 carries IP
addresses (login attempts, approvals, bans), one player's report about another,
and the grid reference of somebody's base. They are stored, because an operator
chasing ban evasion needs them; they are not served below the admin tier. The
allowlist lives here rather than as a field on the wire, because a boundary
declared by the sender is one a compromised or merely out-of-date game host can
widen — the same reason core's own shard fan-out filters on the serving side. A
kind this build has never heard of is not public, and a test holds the list
against PROTOCOL.md §8.4 so that adding a kind to the protocol without
classifying it fails a build.

`PROTOCOL_VERSION` goes to 2 here in the same change as the emitters, though this
module consumes none of the new frames yet: the sidecar refuses a mismatched
client with a 409, so a module left on 1 would stop being able to read the board
it has been reading all along. A constant that lags the deployment is an outage
with a version number on it.

95 server tests, 20 client tests, every guard green, and `routes.manifest.json`
regenerated against a real core at the pinned ref: 10 routes, all documented,
none of core's moved.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4

server/boot.js

@@ -21,25 +21,52 @@
 // letting that fail the boot would make installing the module before installing
 // the bridge impossible.
 //
-// ── Polling, in phase 1 ───────────────────────────────────────────────────
+// ── Three timers, and they answer three different questions ───────────────
 //
-// This is a poll, and the live feed it will become is a later phase's work. The
-// poll is not a placeholder for it: a sidecar's store-backed reads are exactly
-// what answers while a game server is off, and the module will keep reading them
-// on an interval to notice a server that went away without saying anything.
-// What the feed adds is latency, not coverage.
+//   refresh   (30s)  what is each server, and who is on it — the BOARDS
+//   ingest     (5s)  what has happened since we last looked — the CURSOR
+//   prune       (1h)  forgetting the detail we promised not to keep for ever
+//
+// The boards poll and the ingest are deliberately separate rather than one loop
+// reading both. They fail differently and they matter differently: a board that
+// is 30 seconds stale shows a player count slightly behind, and an ingest that
+// is 30 seconds behind shows a killfeed that feels broken. Splitting them lets
+// the cheap one run often and the expensive one run rarely, and it means a
+// sidecar that answers one and not the other degrades in exactly one place.
+//
+// The poll was never a placeholder for a socket: a sidecar's store-backed reads
+// are what answer while a game server is off, which is most of what this module
+// renders. See `ingest.js` for why the live feed is a cursor and not a
+// WebSocket.
 
 const core = require('./core')
 
 const db = require('./model/servers/servers.db')
+const eventsDb = require('./model/events/events.db')
+const ingest = require('./ingest')
 const servers = require('./model/servers/servers.model')
 const sidecar = require('./sidecarClient')
 
 const log = core.logger('boot')
 
 let refreshTimer = null
+let ingestTimer = null
+let pruneTimer = null
 
 const REFRESH_MS = 30 * 1000
+const INGEST_MS = 5 * 1000
+const PRUNE_MS = 60 * 60 * 1000
+
+/**
+ * How long this module keeps raw events.
+ *
+ * Longer than the sidecar's 14 days, because this is the richer store and the
+ * one a page reads — and because the sidecar lives on somebody's game host while
+ * this lives on the website's own database. What is NOT bounded by it is the
+ * record: `rust_player_wipe_stats` and `rust_gather_totals` are permanent, which
+ * is the whole of R12's "a wipe does not erase a player's history".
+ */
+const EVENT_RETENTION_DAYS = 30
 
 /**
  * Ask every configured sidecar how its server is doing, and store what it said.
@@ -63,7 +90,10 @@ async function refresh() {
 
 async function refreshOne(server) {
   try {
-    const board = await sidecar.serverBoard(server)
+    // One call for both boards. `/server` would answer the same question about
+    // the server itself, but presence would then be a second round trip to the
+    // same process for a fact it already had in hand.
+    const board = await sidecar.boards(server)
 
     // Three outcomes, and collapsing any two of them loses something an operator
     // needs:
@@ -80,12 +110,20 @@ async function refreshOne(server) {
       return
     }
 
-    const frame = board.data
+    const boards = (board.data && board.data.boards) || {}
+    const frame = boards['server.hello']
+
     if (!frame) {
+      // The sidecar is up and has never heard from the game. Presence is emptied
+      // rather than left alone: a stale list of players on a server nobody can
+      // reach is worse than an empty one, because it looks current.
       await db.putState({ serverId: server.id, reachable: true, online: false })
+      await ingest.applyBoards(server.id, {})
       return
     }
 
+    await ingest.applyBoards(server.id, boards)
+
     await db.putState({
       serverId: server.id,
       reachable: true,
@@ -102,6 +140,7 @@ async function refreshOne(server) {
       worldSize: frame.worldSize === undefined ? null : Number(frame.worldSize),
       bootId: frame.bootId || null,
       saveCreatedAt: frame.saveCreatedAt || null,
+      wipeId: frame.wipeId || null,
       protocol: frame.protocol === undefined ? null : Number(frame.protocol),
       raw: frame,
     })
@@ -119,14 +158,44 @@ async function refreshOne(server) {
  * built to look like it — so a module that only needs core at boot time can skip
  * `core.init` entirely and use this argument.
  */
+/** Runs the cursor for every configured server, independently. */
+async function ingestAll() {
+  let rows
+
+  try {
+    rows = await servers.listForPolling()
+  } catch (err) {
+    log.warn('could not read the server list', { error: err.message })
+    return
+  }
+
+  // `allSettled`, for the same reason the board poll uses it: six servers behind
+  // one unreachable host must not stop the other five being ingested.
+  await Promise.allSettled(rows.map((server) => ingest.ingestServer(server)))
+}
+
+async function prune() {
+  try {
+    const gone = await eventsDb.pruneEvents(EVENT_RETENTION_DAYS)
+    if (gone > 0) log.info('pruned old events', { events: gone, days: EVENT_RETENTION_DAYS })
+  } catch (err) {
+    log.warn('could not prune events', { error: err.message })
+  }
+}
+
 async function onBoot() {
   await refresh()
   refreshTimer = setInterval(refresh, REFRESH_MS)
+  ingestTimer = setInterval(ingestAll, INGEST_MS)
+  pruneTimer = setInterval(prune, PRUNE_MS)
   // Node keeps the process alive for a pending timer. Core's own intervals are
   // unref'd for exactly this reason: a module that forgets turns `Ctrl-C` into a
   // thirty-second wait, and on a host it turns a `systemctl stop` into a SIGKILL.
-  if (typeof refreshTimer.unref === 'function') refreshTimer.unref()
-  log.info('booted', { refreshMs: REFRESH_MS })
+  for (const timer of [refreshTimer, ingestTimer, pruneTimer]) {
+    if (timer && typeof timer.unref === 'function') timer.unref()
+  }
+
+  log.info('booted', { refreshMs: REFRESH_MS, ingestMs: INGEST_MS })
 }
 
 /**
@@ -138,9 +207,25 @@ async function onBoot() {
  * rather than cancelled, since nothing can stop a promise that is still running.
  */
 async function onShutdown() {
-  if (refreshTimer) clearInterval(refreshTimer)
+  for (const timer of [refreshTimer, ingestTimer, pruneTimer]) {
+    if (timer) clearInterval(timer)
+  }
+
   refreshTimer = null
+  ingestTimer = null
+  pruneTimer = null
+
   log.info('shut down')
 }
 
-module.exports = { onBoot, onShutdown, refresh, refreshOne, REFRESH_MS }
+module.exports = {
+  onBoot,
+  onShutdown,
+  refresh,
+  refreshOne,
+  ingestAll,
+  prune,
+  REFRESH_MS,
+  INGEST_MS,
+  EVENT_RETENTION_DAYS,
+}