4 Commits

Author SHA1 Message Date
5ce711048c Merge pull request 'feat: ingest protocol 2, and keep the record a wipe cannot erase' (#3) from feat/phase-3-protocol-2 into main
All checks were successful
Release / release (push) Successful in 20s
Reviewed-on: #3
2026-09-16 16:37:14 +00:00
f211969ee1 feat: ingest protocol 2, and keep the record a wipe cannot erase
All checks were successful
PR Checks / client-build (pull_request) Successful in 17s
PR Checks / frozen-manifest (pull_request) Successful in 44s
PR Checks / server-tests (pull_request) Successful in 7m57s
The module half of the read path. Seven tables, an ingest cursor, four public
routes, and one file whose only job is deciding who may see what.

**The record and the window are different things.** `rust_player_wipe_stats` and
`rust_gather_totals` are permanent and per-wipe, so all-time is those rows SUMmed
rather than a second set of counters that can disagree with them — that is R12's
"per-wipe detail plus all-time rollups" in one table instead of two.
`rust_events` is a bounded 30-day window of raw frames for the killfeed, and
`rust_presence` is a board: replaced wholesale, never appended.

**The feed is a cursor, not a socket, and the header says why.** Core runs Node
20, where a global WebSocket is still behind a flag, so a socket means taking
`ws` — against a release that asserts it has no runtime dependencies (D5). The
deciding argument is the other one though: a socket needs a cursor anyway, for
whatever it missed while the module was restarting, and the catch-up path is the
one that has to be right. A cursor alone is one mechanism exercised every five
seconds rather than two where the second only runs after an outage.

**The cursor advances after the batch, never before.** A crash between the two
re-reads events already counted, which inflates a total; the other order loses
them silently and for ever. One is visible and bounded, the other is invisible
and permanent, so the code fails in the visible direction. A server with no
cursor starts at the sidecar's current END rather than at zero — replaying a
fortnight of deaths into stats for wipes the site never saw is not a catch-up.

**`catalogue.js` is a security boundary, default-deny.** Protocol 2 carries IP
addresses (login attempts, approvals, bans), one player's report about another,
and the grid reference of somebody's base. They are stored, because an operator
chasing ban evasion needs them; they are not served below the admin tier. The
allowlist lives here rather than as a field on the wire, because a boundary
declared by the sender is one a compromised or merely out-of-date game host can
widen — the same reason core's own shard fan-out filters on the serving side. A
kind this build has never heard of is not public, and a test holds the list
against PROTOCOL.md §8.4 so that adding a kind to the protocol without
classifying it fails a build.

`PROTOCOL_VERSION` goes to 2 here in the same change as the emitters, though this
module consumes none of the new frames yet: the sidecar refuses a mismatched
client with a 409, so a module left on 1 would stop being able to read the board
it has been reading all along. A constant that lags the deployment is an outage
with a version number on it.

95 server tests, 20 client tests, every guard green, and `routes.manifest.json`
regenerated against a real core at the pinned ref: 10 routes, all documented,
none of core's moved.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-16 08:37:16 -05:00
9018e55488 Merge pull request 'feat(ci): packaging, release and the frozen manifest' (#2) from feat/phase-2-packaging into main
All checks were successful
Release / release (push) Successful in 21s
Reviewed-on: #2
2026-09-16 10:34:58 +00:00
b33d21d71b feat(ci): packaging, release and the frozen manifest
All checks were successful
PR Checks / server-tests (pull_request) Successful in 21s
PR Checks / client-build (pull_request) Successful in 21s
PR Checks / frozen-manifest (pull_request) Successful in 39s
Phase 2 of docs/modules/rust/PLAN.md. Phase 1 built five guards and ran them by
hand; this repo had no workflows at all, so nothing gated the branch that gets
released and there was no way to release it.

Three pieces:

- **release.yml** — the derived-version engine link, installer and Module-uo
  already run (conventional-commit subjects since the newest tag; module.json's
  version survives as a floor; workflow_dispatch as the backdoor), assembling the
  bundle from an include list and publishing the tarball, the install manifest
  carrying its sha256, and SHA256SUMS. The tag is the number that ships and CI
  stamps it into the bundle's own module.json.
- **pr-checks.yml** — server tests, check:imports, check:bundle, check:swagger,
  the client build, client tests and check:externals, plus frozen-manifest.
- **frozen-manifest** — clones core at the sha pinned in ci/core-ref.json,
  generates its route table without this module and with it, and takes the
  difference. It ran locally against that exact ref: six routes, all documented,
  no core route moved. That is the first proof by a running core that /rust
  collides with nothing — phase 1 could only check it by reading, because core
  mounts /status and /version at a tier root where the loader's own collision
  probe cannot see them.

The bundle carries no node_modules, because the shipped half declares no runtime
dependencies (org lead, phase 2). checkBundle.js holds both halves of that: the
include list still covers everything server/index.js reaches, and no dependency
has appeared without the release learning to pack it. Verified by breaking it —
dropping "model" from the list names the exact edit and exits 1.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-16 02:10:07 -05:00
26 changed files with 3820 additions and 30 deletions

View File

@@ -0,0 +1,229 @@
# Gate every pull request into `main` on a fast, DB-free check suite, so a broken
# build or a failing test can't reach the branch that gets released.
#
# Mirrors RunicGateway/website's pr-checks.yml — this module is two npm packages
# shaped like that repo's `server/` and `client/`, and it is loaded into that
# repo's process, so it is checked the same way with the same Node version.
#
# Phase 1 built all of these checks and ran them BY HAND. That is the gap this
# file closes: a guard nothing invokes is a guard whose state nobody knows.
#
# ── What each job is really asking ───────────────────────────────────────────
#
# The tests are the ordinary half. The `check:*` scripts are the interesting one,
# because they are the acceptance criteria of the module contract itself
# (docs/website/MODULE_API.md Part 5) rather than of this module's behaviour:
#
# • `server: check:imports` — no relative path escapes the module root, and no
# shipped file resolves a bare specifier. A module that reaches into core's
# tree works right up until core moves a file, and the whole boundary is
# worth exactly as much as this check is (§5.1).
#
# • `server: check:bundle` — the release ships everything the entry point can
# reach, and still declares no runtime dependency. Every other job here runs
# against the whole repo, but a release is a SUBSET of it (release.yml
# assembles from the include list in `ci/bundle.json`), and nothing else
# compares the two. Module-uo's v1.0.0 is the cautionary tale: `server/commands/`
# arrived in a cutover, the include list did not learn about it, and the
# module installed and then died at the register stage on the operator's box
# with "Cannot find module './commands/guild.command'". Green in CI, broken
# there — because the subset only exists in the release.
#
# • `client: check:externals` — the BUILT chunk has no bare imports left. That
# failure is invisible in source: `import { useState } from 'react'` is
# correct in every file, and whether it becomes core's React or a bare
# specifier no browser can resolve is decided by vite.config.js. It has to be
# asked of the artifact, so it runs after the build. (The other half — a
# shared dependency being BUNDLED — fails the build itself, from a
# resolution-time guard inside vite.config.js.)
#
# • `server: check:swagger` — `swagger-fragment.json` describes the routes this
# module registers, today. Core has no way to generate it: core is a prebuilt
# image, this module arrives on a volume afterwards, and it mounts through a
# call no static parser can follow. So the fragment core merges into
# `/api/docs.json` is whatever this repo committed, and a stale one documents
# a URL surface that does not exist (§2.8).
#
# • `frozen-manifest` — the job with the interesting shape. It clones CORE at
# the ref pinned in `ci/core-ref.json`, generates its route manifest twice
# (without this module, then with) and takes the difference. That difference
# is what this module serves, and it is checked three ways: it must match the
# committed `routes.manifest.json`, it must not have REMOVED or changed one of
# core's own routes, and every route in it must have an operation in
# `swagger-fragment.json` — the per-module form of core's rule that a route
# which isn't in the spec doesn't ship (§5.3, §2.8).
#
# Nothing else can ask those questions. Every other check here runs against
# this repo alone, where a mount prefix is a string in `server/index.js` and a
# documented path is a string in a JSON file; whether they name the same URL
# is a fact about a running core, and this is the only job that has one. It is
# also the only thing that can see the blind spot phase 1 had to check by
# reading: core answers several public routes mounted at the TIER ROOT rather
# than under a prefix (`/status`, `/version`), which the loader's own collision
# probe cannot find, so `/rust` being free is now asserted by a core.
#
# Enforcement (one-time, in the Gitea UI):
# Repository Settings → Branches → Branch Protection (rule for `main`)
# • Enable Status Check
# • Status check patterns: PR Checks / *
# Note: Gitea only lists a context in its dropdown after it has reported once,
# so let this workflow run on one PR first. The `PR Checks / *` glob matches
# without needing the dropdown, and keeps matching as jobs are added.
#
# Runner: the shared self-hosted `ubuntu-latest` runner. These jobs need only
# Node — no Docker socket, no database.
#
# Scope note: `edge` is gated as well as `main`, though this repo has no `edge`
# branch yet. Multi-phase work lands there first everywhere else in this project,
# and gating only the `main` hop would run these checks for the first time at the
# cutover — the one moment a red build is most expensive to discover. Naming the
# branch before it exists costs nothing; an Android workstream that landed nine
# PRs on an ungated `edge` is why it is here from the start.
name: PR Checks
on:
pull_request:
branches: [main, edge]
# A newer push to the same PR cancels the in-flight run.
concurrency:
group: pr-checks-${{ github.ref }}
cancel-in-progress: true
# npm's own retry, turned up. The shared runner reads ETIMEDOUT from the registry
# often enough to matter, and a red X that means "the network hiccuped" costs a
# reviewer more than it costs the runner to retry, and teaches everyone to re-run
# rather than read a failure.
env:
NPM_CONFIG_FETCH_RETRIES: 5
NPM_CONFIG_FETCH_RETRY_MINTIMEOUT: 20000
NPM_CONFIG_FETCH_RETRY_MAXTIMEOUT: 120000
jobs:
server-tests:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
cache-dependency-path: server/package-lock.json
# `npm ci` rather than `npm install`: it also proves the lockfile is in
# sync with package.json instead of silently updating it.
- name: Install server deps
run: npm ci --prefix server
- name: Run server tests
run: npm test --prefix server
- name: Check the module boundary (MODULE_API.md §5.1)
run: npm run check:imports --prefix server
- name: Check the release ships what the module requires
run: npm run check:bundle --prefix server
- name: Check the OpenAPI fragment is current (MODULE_API.md §2.8)
run: npm run check:swagger --prefix server
client-build:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
cache-dependency-path: client/package-lock.json
- name: Install client deps
run: npm ci --prefix client
# The build comes FIRST, and that ordering is load-bearing. Two of the
# client tests read `dist/entry.js` — the chunk's externals, and what it
# registers when imported against a fake `window.__rg` — and both skip when
# there is no build. Run the other way round they skip silently in CI, which
# is the worst of both: green, and not asking the question.
- name: Build the client chunk
run: npm run build --prefix client
- name: Run client tests
run: npm test --prefix client
- name: Check the built chunk's externals (MODULE_API.md §3.6)
run: npm run check:externals --prefix client
# ── The URLs this module actually serves ──────────────────────────────────
#
# Everything above proves the module against itself. This proves it against a
# real core: the one place where "the prefix I register" and "the path I
# document" are the same fact rather than two strings that ought to agree.
#
# The module is COPIED into the core checkout, never symlinked — core's loader
# filters its scan with `entry.isDirectory()`, which reports a link as a link
# and skips it silently, so a symlinked module produces a manifest with no
# module routes in it and a diff that looks like the module registering nothing.
frozen-manifest:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
with:
path: module
- uses: actions/setup-node@v4
with:
node-version: 20
# Anonymous HTTPS, and a full clone rather than a shallow one: the pin is a
# commit sha, and `--depth 1` can only fetch a branch tip.
- name: Clone core at the pinned ref (MODULE_API.md §5.3)
run: |
REPO=$(node -p "require('./module/ci/core-ref.json').repo")
REF=$(node -p "require('./module/ci/core-ref.json').ref")
echo "core: $REPO @ $REF"
git clone --quiet "$REPO" core
git -C core checkout --quiet "$REF"
- name: Install core's server deps
run: npm ci --prefix core/server
# Core alone. `--check` first, so a pin that no longer regenerates its own
# committed manifest fails HERE, naming the pin, instead of showing up below
# as this module having removed a route it never touched.
- name: Generate core's manifest without this module
run: |
npm run routes:manifest --prefix core/server -- --check
cp core/server/routes.manifest.json before.json
# The chunk has to exist before the loader will accept the module at all —
# `client.entry` is validated during the manifest step of the scan, and a
# missing one is a load failure, not a warning.
- name: Build the client chunk
run: |
npm ci --prefix module/client
npm run build --prefix module/client
# No `npm ci` on the installed copy, because the shipped half declares no
# runtime dependencies and the release packs no `node_modules` (org lead,
# phase 2). `check:bundle` in the job above is what keeps that true; if it
# ever stops being true, this step and release.yml both grow an install.
- name: Install the module into core
run: |
mkdir -p core/modules/rust
tar -C module --exclude=.git --exclude=node_modules -cf - . | tar -C core/modules/rust -xf -
- name: Generate core's manifest with this module
run: |
npm run routes:manifest --prefix core/server
cp core/server/routes.manifest.json after.json
- name: Check the frozen manifest and the fragment's coverage
working-directory: module
run: node server/scripts/frozenManifest.js --before ../before.json --after ../after.json --check

View File

@@ -0,0 +1,442 @@
# Build and publish the installable bundle: `module-rust-<version>.tar.gz` plus
# the manifest carrying its sha256 (docs/website/MODULE_SYSTEM.md §2.3, §2.5).
#
# ── What a release IS here ──────────────────────────────────────────────────
#
# **An operator never builds anything** (MODULE_SYSTEM.md §1.14 — the constraint
# the whole module system is shaped around). So a release is not source: it is the
# directory core's loader expects to find at `modules/rust/`, already assembled —
# the prebuilt client chunk, the schema fragment and the OpenAPI fragment — packed
# as it will be unpacked. Core's admin install downloads the tarball, verifies it
# against the `sha256` in the manifest, and unpacks it onto the volume. Nothing
# runs `npm` on the way.
#
# **And nothing is installed into the bundle either**, which is where this repo
# differs from Module-uo: the shipped half declares no runtime dependencies, so
# there is no `npm ci --omit=dev` and no `server/node_modules` in the tarball (org
# lead, phase 2). That is a decision worth being loud about rather than a detail —
# `server/scripts/checkBundle.js` fails the PR that adds a dependency without also
# teaching this file to install and pack it, because a bundle that declares an
# import it does not carry fails the same way a missing directory does.
#
# ── The version is DERIVED, and the declaration is a floor ──────────────────
#
# The engine `link`, `installer` and `Module-uo` already run (MODULE_SYSTEM
# §2.7.1, decision 19 as amended):
#
# feat!: / BREAKING CHANGE -> major feat: -> minor fix|perf: -> patch
# nothing releasable -> no release is cut
# (first ever run, no tag) -> releases what module.json declares
#
# Module-uo learned this the expensive way: it released only when a merge left
# `module.json` at a version with no release yet — the version DECLARED, never
# computed — and between 2026-08-12 and 2026-08-19 that cost it *every* bundle,
# because nine phases of work landed without anyone touching that line.
#
# **The declared version is kept as a floor, not deleted.** If `module.json` names
# a version above the newest tag, that version releases. Raising it by hand is how
# you say "this one is a minor, whatever the subjects imply", and it is the natural
# place to move when a `coreApi` bump forces the question.
#
# The number that ships is therefore the TAG, and CI writes it into the
# `module.json` inside the bundle at assembly time. The committed `module.json` is
# a floor and a starting point, not a record of the last release — a release engine
# that has to commit a bump back to `main` stops working the day someone protects
# the branch, and this one is protected.
#
# ── The backdoor ────────────────────────────────────────────────────────────
#
# `workflow_dispatch` publishes on demand, for the case the rules above cannot
# reach: `module.json` changed in a way worth shipping — a widened `coreApi`, a
# new mount, a capability — with no releasable code behind it. Leave `version`
# blank to bump the newest tag by `bump` (default `patch`), or name an exact
# version to publish that. A dispatch releases even when nothing in the log is
# releasable; that is the entire point of pressing the button.
#
# Re-running on a version that is already released is a no-op, so a rerun after an
# unrelated failure is safe. A tag that exists with no release behind it is NOT a
# no-op — see the recovery branch in the plan step. That state is not theoretical:
# `servuo-plugins`' first release pushed its tag and then 401'd on the release API
# because the secret was absent, and without the recovery branch the repo would
# have been stuck there permanently.
#
# This workflow never writes to a branch. It tags and publishes, so `main` needs
# no push exception.
#
# Prerequisites (Settings → Actions → Secrets on RunicGateway/Module-Rust):
# REGISTRY_TOKEN — Gitea access token with `write:repository`, to push the tag
# and create the release.
name: Release
on:
push:
branches: [main]
workflow_dispatch:
inputs:
version:
description: 'Exact version to publish (e.g. 0.1.1). Blank = bump the newest tag by the level below.'
required: false
default: ''
bump:
description: 'Bump level when version is blank: patch | minor | major'
required: false
default: 'patch'
concurrency:
group: release-module-rust
cancel-in-progress: false
env:
GITEA_HOST: gitea.whitlocktech.com
REPO: RunicGateway/Module-Rust
jobs:
release:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
# Full history: the plan step reads every tag and every subject since the
# newest one, and a shallow clone has neither.
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: actions/setup-node@v4
with:
node-version: 20
- name: Plan the release (version + changelog)
id: plan
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
EVENT: ${{ github.event_name }}
IN_VERSION: ${{ github.event.inputs.version }}
IN_BUMP: ${{ github.event.inputs.bump }}
run: |
set -euo pipefail
mkdir -p dist
git fetch --tags --force >/dev/null 2>&1 || true
DECLARED="$(node -p "require('./module.json').version")"
LAST_TAG="$(git describe --tags --match 'v*' --abbrev=0 2>/dev/null || true)"
CURRENT="${LAST_TAG#v}"
RANGE="${LAST_TAG:+${LAST_TAG}..}HEAD"
echo "module.json declares ${DECLARED}; newest tag is ${LAST_TAG:-<none>}"
SUBJECTS="$(git log --no-merges --format='%s' $RANGE || true)"
BODIES="$(git log --no-merges --format='%B' $RANGE || true)"
BUMP=none
if echo "$BODIES" | grep -qE 'BREAKING[ -]CHANGE' ; then BUMP=major; fi
if echo "$SUBJECTS" | grep -qE '^[a-z]+(\([^)]+\))?!:' ; then BUMP=major; fi
if [ "$BUMP" = none ] && echo "$SUBJECTS" | grep -qE '^feat(\([^)]+\))?:' ; then BUMP=minor; fi
if [ "$BUMP" = none ] && echo "$SUBJECTS" | grep -qE '^(fix|perf)(\([^)]+\))?:' ; then BUMP=patch; fi
bump() { # <x.y.z> <major|minor|patch> -> bumped
IFS=. read -r MA MI PA <<< "$1"
case "$2" in
major) echo "$((MA+1)).0.0" ;;
minor) echo "${MA}.$((MI+1)).0" ;;
patch) echo "${MA}.${MI}.$((PA+1))" ;;
esac
}
# `sort -V` orders version strings, so the higher of two is its last
# line. Used rather than a hand-rolled field compare because 0.10.0 vs
# 0.9.0 is exactly the comparison a string sort gets wrong.
higher() { printf '%s\n%s\n' "$1" "$2" | sort -V | tail -1; }
rank() { case "$1" in major) echo 3 ;; minor) echo 2 ;; patch) echo 1 ;; *) echo 0 ;; esac; }
bigger_bump() { if [ "$(rank "$1")" -ge "$(rank "$2")" ]; then echo "$1"; else echo "$2"; fi; }
VERSION=""
if [ -n "${IN_VERSION:-}" ]; then
# The backdoor's exact form. Deliberately unvalidated against the log:
# a human typed it, and the already-released check below is the only
# guard that matters.
VERSION="${IN_VERSION}"
echo "dispatch: publishing the requested version ${VERSION}"
else
LEVEL="$BUMP"
# A dispatch with nothing releasable in the log still releases — that
# is what the button is for. Where the log DOES say something, the
# larger of the two wins rather than the input: pressing the button on
# a log full of `feat:` without touching the dropdown would otherwise
# publish its `patch` default over a minor's worth of work, and a
# version that undersells its own contents cannot be taken back.
if [ "${EVENT:-}" = workflow_dispatch ]; then
LEVEL="$(bigger_bump "$LEVEL" "${IN_BUMP:-patch}")"
if [ "$BUMP" = none ]; then
echo "dispatch: nothing releasable in the log, bumping ${LEVEL} anyway"
elif [ "$LEVEL" != "$BUMP" ]; then
echo "dispatch: the log says ${BUMP}, the run asked for ${LEVEL} — taking ${LEVEL}"
fi
fi
if [ -z "$CURRENT" ]; then
VERSION="$DECLARED" # first ever release: ship what is declared
elif [ "$LEVEL" != none ]; then
VERSION="$(bump "$CURRENT" "$LEVEL")"
fi
# The floor. A `module.json` above the newest tag releases at that
# version even when the log says nothing and even when the log says
# patch.
if [ -n "$CURRENT" ] && [ "$DECLARED" != "$CURRENT" ] \
&& [ "$(higher "$DECLARED" "$CURRENT")" = "$DECLARED" ]; then
if [ -z "$VERSION" ] || [ "$(higher "$DECLARED" "$VERSION")" = "$DECLARED" ]; then
echo "module.json declares ${DECLARED}, above both ${CURRENT} and the derived version — releasing that."
VERSION="$DECLARED"
fi
fi
fi
RELEASE=true
if [ -z "$VERSION" ]; then
RELEASE=false
VERSION="$CURRENT"
echo "Nothing releasable since ${LAST_TAG} (no feat/fix/perf/breaking subject) — standing down."
fi
# An existing tag is NOT automatically "nothing to do". A tag with no
# release behind it means a previous run tagged and then died before
# publishing — which is what happened on servuo-plugins' first release,
# where absent secrets took the release API call to 401 after the tag had
# already been pushed. Standing down on the tag alone makes that state
# permanent. Note this deliberately OVERRIDES the RELEASE=false above:
# with the tag in place there is nothing releasable after it, so the
# normal path would stand down, which is why it could never self-heal.
# Anything other than 200/404 — a network failure, a bad token — is not
# evidence of absence, and guessing "no" would publish over a good
# release, so refuse instead.
REUSE_TAG=false
if [ -n "$VERSION" ] && git rev-parse -q --verify "refs/tags/v${VERSION}" >/dev/null; then
CI_TOKEN="$(printf '%s' "${REGISTRY_TOKEN:-}" | tr -d '\r\n')"
REL_HTTP="$(curl -s -o /dev/null -w '%{http_code}' \
-H "Authorization: token ${CI_TOKEN}" \
"https://${GITEA_HOST}/api/v1/repos/${REPO}/releases/tags/v${VERSION}" || echo 000)"
case "$REL_HTTP" in
200) echo "v${VERSION} is already released — nothing to do."; RELEASE=false ;;
404) echo "::warning::Tag v${VERSION} exists but has no release — a previous run failed after tagging. Reusing the tag and publishing the release it is missing."
REUSE_TAG=true; RELEASE=true ;;
*) echo "::error::Could not determine whether v${VERSION} is released (HTTP ${REL_HTTP}). Refusing to guess."; exit 1 ;;
esac
fi
# Changelog range. A recovery run has nothing after the tag, so
# summarize what the tag itself contains rather than emitting an empty
# list: the range that produced it, i.e. previous-tag..this-tag.
if [ "$REUSE_TAG" = true ]; then
PREV_TAG="$(git describe --tags --match 'v*' --abbrev=0 "v${VERSION}^" 2>/dev/null || true)"
CL_RANGE="${PREV_TAG:+${PREV_TAG}..}v${VERSION}"
SINCE="$PREV_TAG"
else
CL_RANGE="$RANGE"
SINCE="$LAST_TAG"
fi
CL_SUBJECTS="$(git log --no-merges --format='%s' $CL_RANGE || true)"
{
echo "## module-rust v${VERSION}"
echo
echo "Install from the website's Admin → Modules screen by pasting the URL of"
echo "\`module-rust-${VERSION}.json\`, or unpack the tarball onto the modules volume"
echo "as \`modules/rust/\`. Requires a core whose \`MODULE_API_VERSION\` satisfies"
echo "\`$(node -p "require('./module.json').coreApi")\`."
echo
echo "A Rust server also needs the other two halves of the bridge:"
echo "[Rust-Link](https://${GITEA_HOST}/RunicGateway/Rust-Link) (the sidecar) and"
echo "[Rust-Plugins](https://${GITEA_HOST}/RunicGateway/Rust-Plugins) (the Oxide/Carbon plugin)."
echo
FEATS="$(echo "$CL_SUBJECTS" | grep -E '^feat' || true)"
FIXES="$(echo "$CL_SUBJECTS" | grep -E '^(fix|perf)' || true)"
[ -n "$FEATS" ] && { echo "### Features"; echo "$FEATS" | sed 's/^/- /'; echo; }
[ -n "$FIXES" ] && { echo "### Fixes"; echo "$FIXES" | sed 's/^/- /'; echo; }
echo "### All changes"
if [ -n "$SINCE" ]; then echo "Since ${SINCE}:"; fi
echo "$CL_SUBJECTS" | sed 's/^/- /'
echo
echo "### Verifying this download"
echo
echo "Releases are **unsigned** — the \`sha256\` in \`module-rust-${VERSION}.json\` is the"
echo "trust anchor, and the website verifies it before unpacking."
echo
echo '```bash'
echo "sha256sum -c SHA256SUMS --ignore-missing"
echo '```'
} > dist/CHANGELOG.md
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
echo "tag=v${VERSION}" >> "$GITHUB_OUTPUT"
echo "release=${RELEASE}" >> "$GITHUB_OUTPUT"
echo "reuse_tag=${REUSE_TAG}" >> "$GITHUB_OUTPUT"
echo "bump=${BUMP}" >> "$GITHUB_OUTPUT"
echo "==> release=${RELEASE} version=${VERSION} bump=${BUMP} declared=${DECLARED} last_tag=${LAST_TAG:-<none>}"
# Before anything is built or tagged, so a repo without secrets fails
# legibly rather than half-publishing: the tag push can succeed on the
# credential actions/checkout left in the local git config while the release
# API call 401s, leaving the repo tagged and unreleased.
- name: Verify release credentials are configured
if: ${{ steps.plan.outputs.release == 'true' }}
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -euo pipefail
if [ -z "$(printf '%s' "${REGISTRY_TOKEN:-}" | tr -d '\r\n')" ]; then
echo "::error::Missing Actions secret REGISTRY_TOKEN (needs write:repository) on ${REPO}."
exit 1
fi
echo "Release credentials present."
- name: Build the client chunk
if: ${{ steps.plan.outputs.release == 'true' }}
run: |
npm ci --prefix client
npm run build --prefix client
# ── Assemble exactly what an operator's volume gets ──────────────────
#
# Stated as an INCLUDE list, not an exclude list. An exclude list ships
# whatever it forgot: the day someone adds `server/tools/` with a scratch
# credential in it, an exclude list packs it and nobody finds out.
#
# The list itself lives in `ci/bundle.json`, not here, because it has a
# second reader: `server/scripts/checkBundle.js` runs in PR checks and asks
# whether the list still covers everything `server/index.js` reaches. One
# declaration, two readers, so a new directory cannot go missing quietly.
- name: Assemble the bundle
if: ${{ steps.plan.outputs.release == 'true' }}
run: |
set -euo pipefail
VERSION="${{ steps.plan.outputs.version }}"
OUT="dist/module-rust-${VERSION}"
rm -rf "$OUT" && mkdir -p "$OUT"
# The manifest core reads — with the RELEASED version written into it.
# The committed `module.json` is a floor, not a record of the last
# release (see the header), so copying it verbatim would ship a bundle
# whose `installed_modules` row and admin screen disagree with the tag it
# came from. This is the one place the derived number becomes the
# module's own.
jq --arg v "$VERSION" '.version = $v' module.json > "$OUT/module.json"
# The two fragments, and the licence the code is under — a bundle that
# ships GPL code without its licence is not distributable.
for f in $(jq -r '.root[]' ci/bundle.json); do
cp "$f" "$OUT/"
done
# The server half, minus what never runs inside core's process. No
# node_modules: the shipped half declares no runtime dependencies, and
# check:bundle is what keeps that true.
mkdir -p "$OUT/server"
for d in $(jq -r '.server[]' ci/bundle.json); do
cp -r "server/$d" "$OUT/server/"
done
# The client half is the BUILT chunk only. `client/src` is source an
# operator has no use for and core will never read.
mkdir -p "$OUT/client/dist"
cp client/dist/entry.js "$OUT/client/dist/"
# Prove the bundle is loadable before it is published: these are the
# paths core's loader resolves out of module.json, and a release whose
# entry point is missing fails on an operator's box with a
# `startup_failed` row instead of here. The version assertion guards the
# rewrite above — a bundle that still carried the declared version would
# install under a number that is not the one it was released as.
node -e '
const fs = require("fs"), path = require("path");
const [root, want] = process.argv.slice(1);
const m = JSON.parse(fs.readFileSync(path.join(root, "module.json"), "utf8"));
if (m.version !== want) {
console.error(`bundle declares ${m.version}, but this is release ${want}`);
process.exit(1);
}
for (const p of [m.server, m.schema, m.purge, m.client.entry, "swagger-fragment.json"]) {
if (!fs.existsSync(path.join(root, p))) { console.error("bundle is missing " + p); process.exit(1); }
}
console.log("bundle contents check: ok");
' "$OUT" "$VERSION"
# ── And that it can actually LOAD ─────────────────────────────────
#
# The check above stats the paths `module.json` declares, which is a real
# question but a shallow one: Module-uo's v1.0.0 passed exactly that and
# was still missing `server/commands/`, because a file reached only by a
# require inside `register()` is named nowhere in `module.json`. This
# resolves every relative require in the assembled tree and asserts the
# target is in it — asked of the artifact, so it also catches a copy that
# half failed or a list naming a path that has since moved.
#
# Run from the SOURCE tree (`server/scripts/` never ships) against the
# assembled bundle.
node server/scripts/checkBundle.js --bundle "$OUT"
tar -C dist -czf "dist/module-rust-${VERSION}.tar.gz" "module-rust-${VERSION}"
rm -rf "$OUT"
SHA="$(sha256sum "dist/module-rust-${VERSION}.tar.gz" | cut -d' ' -f1)"
SIZE="$(stat -c%s "dist/module-rust-${VERSION}.tar.gz")"
# The install manifest. Same shape as the installer's bundle JSON — a
# per-asset sha256 fetched over HTTPS, no signatures — because that is
# the model this project already has and a second one would be a second
# thing to get right (MODULE_SYSTEM.md §1.11).
jq -n \
--arg id "$(node -p "require('./module.json').id")" \
--arg name "$(node -p "require('./module.json').name")" \
--arg version "$VERSION" \
--arg coreApi "$(node -p "require('./module.json').coreApi")" \
--arg artifact "module-rust-${VERSION}.tar.gz" \
--arg sha256 "$SHA" \
--argjson size "$SIZE" \
--arg url "https://${GITEA_HOST}/${REPO}/releases/download/v${VERSION}/module-rust-${VERSION}.tar.gz" \
'{schema:1, id:$id, name:$name, version:$version, coreApi:$coreApi,
artifact:$artifact, url:$url, sha256:$sha256, size:$size}' \
> "dist/module-rust-${VERSION}.json"
echo "${SHA} module-rust-${VERSION}.tar.gz" > dist/SHA256SUMS
cat "dist/module-rust-${VERSION}.json"
# Skipped on a recovery run: the tag is already there and is the thing being
# published against.
- name: Tag the release
if: ${{ steps.plan.outputs.release == 'true' && steps.plan.outputs.reuse_tag != 'true' }}
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -euo pipefail
TAG="${{ steps.plan.outputs.tag }}"
git config user.name 'Runic Gateway CI'
git config user.email 'ci@whitlocktech.net'
git tag -a "$TAG" -m "module-rust ${TAG}"
git push origin "$TAG"
- name: Create the Gitea release and upload the bundle
if: ${{ steps.plan.outputs.release == 'true' }}
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -euo pipefail
TAG="${{ steps.plan.outputs.tag }}"
VERSION="${{ steps.plan.outputs.version }}"
API="https://${GITEA_HOST}/api/v1/repos/${REPO}"
CI_TOKEN="$(printf '%s' "${REGISTRY_TOKEN}" | tr -d '\r\n')"
REL_ID="$(curl -sSf -X POST "${API}/releases" \
-H "Authorization: token ${CI_TOKEN}" \
-H "Content-Type: application/json" \
-d "$(jq -n --arg tag "$TAG" --arg body "$(cat dist/CHANGELOG.md)" \
'{tag_name:$tag, name:$tag, body:$body, draft:false, prerelease:false}')" \
| jq -r '.id')"
echo "Created release ${TAG} (id=${REL_ID})"
for f in "module-rust-${VERSION}.tar.gz" "module-rust-${VERSION}.json" SHA256SUMS; do
curl -sSf -X POST "${API}/releases/${REL_ID}/assets?name=${f}" \
-H "Authorization: token ${CI_TOKEN}" \
-F "attachment=@dist/${f}" >/dev/null
echo " uploaded ${f}"
done

View File

@@ -52,6 +52,7 @@ both surfaces an operator can configure and then wait on, which is worse than an
```bash
npm ci --prefix server && npm test --prefix server
npm run check:imports --prefix server
npm run check:bundle --prefix server
npm run check:swagger --prefix server
npm ci --prefix client && npm run build --prefix client
npm run check:externals --prefix client && npm test --prefix client
@@ -66,11 +67,66 @@ Regenerate the OpenAPI fragment whenever a route or an annotation changes:
npm run swagger --prefix server # writes swagger-fragment.json; commit it
```
`.gitea/workflows/pr-checks.yml` runs all of the above on every pull request, plus one job this
machine cannot run on its own: **frozen-manifest** clones core at the sha pinned in
[`ci/core-ref.json`](ci/core-ref.json), generates its route table without this module and then with
it, and takes the difference. That difference is the URL surface this module serves — checked
against the committed [`routes.manifest.json`](routes.manifest.json), against the OpenAPI fragment
in both directions, and against the rule that **a module may only add**. It is the only thing that
can see whether `/rust` collides with one of the routes core mounts at a tier root (`/status`,
`/version`), which the loader's own collision probe cannot find.
## How it reaches an operator
**An operator never builds anything.** A release is not source: it is the directory core's loader
expects at `modules/rust/`, already assembled — the prebuilt client chunk, the schema fragment and
the OpenAPI fragment, packed as they will be unpacked.
**Every merge to `main` carrying a releasable commit publishes a bundle.** The next version is
computed from conventional-commit subjects since the newest `v*` tag, as in `link`, `installer` and
`Module-uo`: `feat!:` or `BREAKING CHANGE` is a major, `feat:` a minor, `fix:` or `perf:` a patch,
and a `main` that gained none of those cuts no release. The number that ships is the **tag**, and CI
writes it into the `module.json` inside the bundle. `module.json`'s version survives as a **floor**:
name a version there above the newest tag and that version releases, which is how you overrule the
subjects. For a change with nothing releasable behind it — a widened `coreApi`, a new mount, a
capability — run the **Release** workflow by hand (Actions → Release → Run workflow).
Each release carries:
| Asset | What it is |
|---|---|
| `module-rust-<version>.tar.gz` | the directory core expects at `modules/rust/`, already assembled |
| `module-rust-<version>.json` | the install manifest: id, version, `coreApi`, the artifact's URL, size and **`sha256`** |
| `SHA256SUMS` | the same hash, in the shape every other repo here publishes |
Releases are **unsigned**; the `sha256` is the trust anchor, and the website verifies it before
unpacking. That is the model `installer`'s bundles already use, and a second trust model would be a
second thing to get right.
The tarball is assembled from an **include** list ([`ci/bundle.json`](ci/bundle.json)), never an
exclude list — an exclude list ships whatever it forgot. Tests, scripts, `client/src`, `ci/` and the
dev dependencies are not in it. It carries **no `node_modules`**, because the shipped half declares
no runtime dependencies: everything it needs arrives on `ctx`. `npm run check:bundle` holds both
halves of that — that the list still covers every file `server/index.js` can reach, and that no
runtime dependency has appeared without the release learning to pack it.
## Install it into a core
Copy the whole tree to `<website>/modules/rust/` and restart. **Copy, do not symlink** — the loader
lists directory entries and asks each whether it is a directory; a symlink answers no and the module
is skipped in complete silence.
**From a release**, which is the supported path: in Admin → Modules, paste the URL of that release's
`module-rust-<version>.json`, and restart when the panel offers. Core fetches the manifest, checks
every URL and redirect hop against its own host allowlist, streams the artifact under a byte cap
while hashing it, verifies the `sha256`, inspects the archive in full before unpacking it to a
temporary directory, and only then moves it into `modules/rust/`. Nothing is written into the
modules directory until every check has passed. The allowlist must contain
`gitea.whitlocktech.com` — it is seeded from `MODULE_SOURCE_HOSTS` on a fresh install and is
DB-owned from then on, edited on that same screen. **An empty allowlist forbids every install rather
than permitting all of them.**
**From a working tree**, for development: copy the whole tree to `<website>/modules/rust/` and
restart. **Copy, do not symlink** — the loader lists directory entries and asks each whether it is a
directory; a symlink answers no and the module is skipped in complete silence.
Either way, the module appears when the process restarts: the volume is read at require time.
Then, in Admin → Rust, add a server: its name, the sidecar's base URL, and the token the sidecar
printed on first start (`rust-link-sidecar --print-config`). **The token is write-only** — it is

49
ci/bundle.json Normal file
View File

@@ -0,0 +1,49 @@
{
"$comment": [
"What a release copies into the bundle, declared ONCE. Read by .gitea/workflows/release.yml when",
"it assembles the tarball, and by server/scripts/checkBundle.js when CI asks whether that list",
"still covers everything the module's entry point can reach.",
"",
"This is an INCLUDE list on purpose. An exclude list ships whatever it forgot: the day someone",
"adds server/tools/ with a scratch credential in it, an exclude list packs it and nobody finds",
"out. The cost of that choice is that a new top-level directory silently drops OUT of every",
"release instead — which is exactly what happened to Module-uo between v0.3.0 and v1.0.0, where",
"server/commands/ arrived with a cutover, the list did not learn about it, and the module",
"installed and then died at the register stage on the operator's box. checkBundle.js exists so",
"that cannot happen twice, and it runs on the PR that adds the directory.",
"",
"server[] entries are paths under server/; root[] and generated[] are paths under the module",
"root.",
"",
"node_modules is NOT here, and its absence is asserted rather than assumed: this module declares",
"no runtime dependencies (everything the shipped half needs arrives on ctx), so the release runs",
"no npm ci and packs no dependency tree. checkBundle.js fails the PR that adds a `dependencies`",
"entry to server/package.json without also teaching the release to pack it — because a module",
"whose bundle silently lacks its own dependency fails the same way the missing directory did.",
"",
"generated[] ships but is not copied — release.yml writes module.json through jq to stamp the",
"released version into it, since the committed one is a floor rather than a record of the last",
"release. It is listed because server/index.js requires it, and a check that did not know it",
"ships would report the module's own manifest as missing from the bundle."
],
"server": [
"boot.js",
"catalogue.js",
"core.js",
"db",
"index.js",
"ingest.js",
"model",
"package.json",
"router",
"sidecarClient.js"
],
"root": [
"swagger-fragment.json",
"LICENSE.md",
"README.md"
],
"generated": [
"module.json"
]
}

6
ci/core-ref.json Normal file
View File

@@ -0,0 +1,6 @@
{
"$comment": "The core this module is proved against. MODULE_API.md §5.3: the frozen-manifest job clones RunicGateway/website at this exact ref, drops this module in as modules/rust and runs CORE's own routeManifest.js — nothing else can answer whether the URLs the module claims are the URLs it actually serves, because a mount prefix is a string in server/index.js and a documented path is a string in a JSON file, and whether those name the same URL is a fact about a running core. It also answers the blind spot phase 1 had to check by hand: core mounts several routes at the TIER ROOT (/status, /version), which the loader's collision probe cannot see, so /rust being free is asserted here by a core rather than by a reading. Pinned rather than tracking a branch on purpose: core moves for reasons that have nothing to do with this module, and a bump is then a deliberate commit saying which core the module was last proved against, instead of an unexplained red X on someone else's PR. Bump it, regenerate routes.manifest.json, and commit both together. This module needs MODULE_API 1.10.0 (module.json's coreApi is ^1.10.0), which the Event System cutover put on `main` — so unlike Module-uo, which spent the Event System window pinned to `edge`, this repo starts pinned to `main` and should stay there unless it comes to depend on a contract member that has not shipped yet.",
"repo": "https://gitea.whitlocktech.com/RunicGateway/website.git",
"ref": "efa9db73304552dd8bb7a84030b258c6320f79f7",
"refName": "main @ MODULE_API 1.10.0, the Asset Bridge cutover 2 of 5 (website#202)"
}

55
routes.manifest.json Normal file
View File

@@ -0,0 +1,55 @@
{
"$comment": "Generated inventory of the URLs module-rust serves - the module half of the freeze core keeps in server/routes.manifest.json. DERIVED as the difference between a core without this module and the same core with it, both at the pinned ref in ci/core-ref.json. Regenerate with the frozen-manifest job in .gitea/workflows/pr-checks.yml; see server/scripts/frozenManifest.js.",
"routes": [
{
"method": "DELETE",
"path": "/api/v1/admin/rust/servers/:id",
"tier": "public"
},
{
"method": "GET",
"path": "/api/v1/admin/rust/servers",
"tier": "public"
},
{
"method": "GET",
"path": "/api/v1/player/rust/servers",
"tier": "public"
},
{
"method": "GET",
"path": "/api/v1/public/rust/servers",
"tier": "public"
},
{
"method": "GET",
"path": "/api/v1/public/rust/servers/:id/events",
"tier": "public"
},
{
"method": "GET",
"path": "/api/v1/public/rust/servers/:id/leaderboard",
"tier": "public"
},
{
"method": "GET",
"path": "/api/v1/public/rust/servers/:id/online",
"tier": "public"
},
{
"method": "GET",
"path": "/api/v1/public/rust/servers/:id/wipes",
"tier": "public"
},
{
"method": "POST",
"path": "/api/v1/admin/rust/servers/:id/test",
"tier": "public"
},
{
"method": "PUT",
"path": "/api/v1/admin/rust/servers/:id",
"tier": "public"
}
]
}

View File

@@ -21,25 +21,52 @@
// letting that fail the boot would make installing the module before installing
// the bridge impossible.
//
// ── Polling, in phase 1 ───────────────────────────────────────────────────
// ── Three timers, and they answer three different questions ───────────────
//
// This is a poll, and the live feed it will become is a later phase's work. The
// poll is not a placeholder for it: a sidecar's store-backed reads are exactly
// what answers while a game server is off, and the module will keep reading them
// on an interval to notice a server that went away without saying anything.
// What the feed adds is latency, not coverage.
// refresh (30s) what is each server, and who is on it — the BOARDS
// ingest (5s) what has happened since we last looked — the CURSOR
// prune (1h) forgetting the detail we promised not to keep for ever
//
// The boards poll and the ingest are deliberately separate rather than one loop
// reading both. They fail differently and they matter differently: a board that
// is 30 seconds stale shows a player count slightly behind, and an ingest that
// is 30 seconds behind shows a killfeed that feels broken. Splitting them lets
// the cheap one run often and the expensive one run rarely, and it means a
// sidecar that answers one and not the other degrades in exactly one place.
//
// The poll was never a placeholder for a socket: a sidecar's store-backed reads
// are what answer while a game server is off, which is most of what this module
// renders. See `ingest.js` for why the live feed is a cursor and not a
// WebSocket.
const core = require('./core')
const db = require('./model/servers/servers.db')
const eventsDb = require('./model/events/events.db')
const ingest = require('./ingest')
const servers = require('./model/servers/servers.model')
const sidecar = require('./sidecarClient')
const log = core.logger('boot')
let refreshTimer = null
let ingestTimer = null
let pruneTimer = null
const REFRESH_MS = 30 * 1000
const INGEST_MS = 5 * 1000
const PRUNE_MS = 60 * 60 * 1000
/**
* How long this module keeps raw events.
*
* Longer than the sidecar's 14 days, because this is the richer store and the
* one a page reads — and because the sidecar lives on somebody's game host while
* this lives on the website's own database. What is NOT bounded by it is the
* record: `rust_player_wipe_stats` and `rust_gather_totals` are permanent, which
* is the whole of R12's "a wipe does not erase a player's history".
*/
const EVENT_RETENTION_DAYS = 30
/**
* Ask every configured sidecar how its server is doing, and store what it said.
@@ -63,7 +90,10 @@ async function refresh() {
async function refreshOne(server) {
try {
const board = await sidecar.serverBoard(server)
// One call for both boards. `/server` would answer the same question about
// the server itself, but presence would then be a second round trip to the
// same process for a fact it already had in hand.
const board = await sidecar.boards(server)
// Three outcomes, and collapsing any two of them loses something an operator
// needs:
@@ -80,12 +110,20 @@ async function refreshOne(server) {
return
}
const frame = board.data
const boards = (board.data && board.data.boards) || {}
const frame = boards['server.hello']
if (!frame) {
// The sidecar is up and has never heard from the game. Presence is emptied
// rather than left alone: a stale list of players on a server nobody can
// reach is worse than an empty one, because it looks current.
await db.putState({ serverId: server.id, reachable: true, online: false })
await ingest.applyBoards(server.id, {})
return
}
await ingest.applyBoards(server.id, boards)
await db.putState({
serverId: server.id,
reachable: true,
@@ -102,6 +140,7 @@ async function refreshOne(server) {
worldSize: frame.worldSize === undefined ? null : Number(frame.worldSize),
bootId: frame.bootId || null,
saveCreatedAt: frame.saveCreatedAt || null,
wipeId: frame.wipeId || null,
protocol: frame.protocol === undefined ? null : Number(frame.protocol),
raw: frame,
})
@@ -119,14 +158,44 @@ async function refreshOne(server) {
* built to look like it — so a module that only needs core at boot time can skip
* `core.init` entirely and use this argument.
*/
/** Runs the cursor for every configured server, independently. */
async function ingestAll() {
let rows
try {
rows = await servers.listForPolling()
} catch (err) {
log.warn('could not read the server list', { error: err.message })
return
}
// `allSettled`, for the same reason the board poll uses it: six servers behind
// one unreachable host must not stop the other five being ingested.
await Promise.allSettled(rows.map((server) => ingest.ingestServer(server)))
}
async function prune() {
try {
const gone = await eventsDb.pruneEvents(EVENT_RETENTION_DAYS)
if (gone > 0) log.info('pruned old events', { events: gone, days: EVENT_RETENTION_DAYS })
} catch (err) {
log.warn('could not prune events', { error: err.message })
}
}
async function onBoot() {
await refresh()
refreshTimer = setInterval(refresh, REFRESH_MS)
ingestTimer = setInterval(ingestAll, INGEST_MS)
pruneTimer = setInterval(prune, PRUNE_MS)
// Node keeps the process alive for a pending timer. Core's own intervals are
// unref'd for exactly this reason: a module that forgets turns `Ctrl-C` into a
// thirty-second wait, and on a host it turns a `systemctl stop` into a SIGKILL.
if (typeof refreshTimer.unref === 'function') refreshTimer.unref()
log.info('booted', { refreshMs: REFRESH_MS })
for (const timer of [refreshTimer, ingestTimer, pruneTimer]) {
if (timer && typeof timer.unref === 'function') timer.unref()
}
log.info('booted', { refreshMs: REFRESH_MS, ingestMs: INGEST_MS })
}
/**
@@ -138,9 +207,25 @@ async function onBoot() {
* rather than cancelled, since nothing can stop a promise that is still running.
*/
async function onShutdown() {
if (refreshTimer) clearInterval(refreshTimer)
for (const timer of [refreshTimer, ingestTimer, pruneTimer]) {
if (timer) clearInterval(timer)
}
refreshTimer = null
ingestTimer = null
pruneTimer = null
log.info('shut down')
}
module.exports = { onBoot, onShutdown, refresh, refreshOne, REFRESH_MS }
module.exports = {
onBoot,
onShutdown,
refresh,
refreshOne,
ingestAll,
prune,
REFRESH_MS,
INGEST_MS,
EVENT_RETENTION_DAYS,
}

118
server/catalogue.js Normal file
View File

@@ -0,0 +1,118 @@
// ── What the bridge can say, and who may hear it ──────────────────────────
//
// One file, because these two questions have to be answered together or the
// second one rots: which frame kinds exist, and which of them a member of the
// public may see.
//
// ── The boundary ──────────────────────────────────────────────────────────
//
// Protocol 2's catalogue includes frames carrying **IP addresses** (a login
// attempt, an approval, a ban) and **one player's complaint about another** (a
// report), and one — a destroyed structure — that names where somebody lives.
// They are stored, because an operator chasing ban evasion needs them and
// because the sidecar persists what it is told. They must never reach a public
// page.
//
// **The boundary is enforced HERE, on the side that serves, and not on the wire.**
// The plugin could have stamped a `class` on every frame and saved this file the
// trouble; it deliberately does not (PROTOCOL.md §8.5). A boundary declared by
// the sender is a boundary a compromised — or merely out-of-date — game host can
// widen. Core's own shard fan-out works the same way: a public stream with an
// allowlist of kinds, and an admin stream that adds the rest.
//
// ── Default deny, and why it is not paranoia ──────────────────────────────
//
// `isPublic` answers `false` for a kind it has never heard of. That matters
// because of the shape of the mistake it prevents: the next protocol version
// adds a kind, this module ingests it happily (`rust_events` stores what it is
// given), and a page that filtered by a DENY list would publish it the day it
// first arrived — before anybody had decided whether it should be public. With
// an allowlist the new kind is invisible until somebody adds it here, which is
// the same moment they think about it.
//
// The test holds this list against `docs/rust-link/PROTOCOL.md` §8.4's table, so
// adding a kind to the spec without classifying it fails a build rather than
// shipping an address to a public page.
/**
* Kinds a public, signed-out visitor may see.
*
* Each entry is a decision. `player.chat` is here because a shard's chat is
* public by the same logic that makes a killfeed public — it happened in front
* of everyone who was on the server — and an operator who disagrees turns the
* feature off rather than relying on this list being wrong.
*/
const PUBLIC_KINDS = Object.freeze([
'player.connected',
'player.disconnected',
'player.respawned',
'player.death',
'player.chat',
'player.tally',
'server.wipe',
'server.initialized',
'server.shutdown',
])
/**
* Kinds an admin may see and nobody else.
*
* Listed rather than implied by absence, so that "we know about this kind and it
* is restricted" is distinguishable from "nobody has classified this kind" — the
* second is a finding, and a bare allowlist cannot tell you which you are
* looking at.
*/
const STAFF_KINDS = Object.freeze([
'entity.destroyed',
'player.reported',
'player.banned',
'player.unbanned',
'player.login.attempt',
'player.approved',
])
/** Every kind protocol 2 defines. */
const ALL_KINDS = Object.freeze([...PUBLIC_KINDS, ...STAFF_KINDS])
const PUBLIC = new Set(PUBLIC_KINDS)
const STAFF = new Set(STAFF_KINDS)
/**
* May a signed-out visitor see this kind?
*
* Default deny: an unknown kind is not public. Callers pass whatever arrived on
* the wire, including a kind from a newer protocol this build has never seen.
*/
function isPublic(kind) {
return PUBLIC.has(kind)
}
/** Is this a kind this build knows about at all? */
function isKnown(kind) {
return PUBLIC.has(kind) || STAFF.has(kind)
}
/**
* Narrows a list of requested kinds to the ones a viewer may have.
*
* Returning the allowlist itself when nothing was requested is what makes the
* public route safe by construction rather than by remembering to filter: there
* is no code path where "no filter" means "everything".
*/
function kindsFor({ admin = false, requested = null } = {}) {
const permitted = admin ? ALL_KINDS : PUBLIC_KINDS
if (!requested || requested.length === 0) return [...permitted]
const allowed = new Set(permitted)
return requested.filter((k) => allowed.has(k))
}
module.exports = {
PUBLIC_KINDS,
STAFF_KINDS,
ALL_KINDS,
isPublic,
isKnown,
kindsFor,
}

View File

@@ -19,5 +19,12 @@
-- it knows this module registered, because it is the side that knows which
-- registrant owned what.
DROP TABLE IF EXISTS rust_ingest_cursor;
DROP TABLE IF EXISTS rust_presence;
DROP TABLE IF EXISTS rust_events;
DROP TABLE IF EXISTS rust_gather_totals;
DROP TABLE IF EXISTS rust_player_wipe_stats;
DROP TABLE IF EXISTS rust_players;
DROP TABLE IF EXISTS rust_wipes;
DROP TABLE IF EXISTS rust_server_state;
DROP TABLE IF EXISTS rust_servers;

View File

@@ -14,14 +14,20 @@
-- Every table here is prefixed `rust_`, which is this module's id and the only
-- prefix it may create under.
--
-- ── Two tables, and the split between them is the whole design ────────────
-- ── Four kinds of table, and the split between them is the whole design ───
--
-- `rust_servers` is CONFIGURATION: rows an operator writes, from Admin → Rust.
-- `rust_server_state` is OBSERVED STATE: rows this module writes from what a
-- sidecar reported. They are separate tables rather than columns on one because
-- they have different writers, different lifetimes and different audiences —
-- and because a purge of observed state while keeping the configuration is a
-- thing an operator will eventually want.
-- CONFIGURATION `rust_servers` — rows an operator writes, from Admin → Rust.
-- OBSERVED STATE `rust_server_state`, `rust_presence` — what a sidecar last
-- reported, replaced rather than appended.
-- THE RECORD `rust_wipes`, `rust_players`, `rust_player_wipe_stats`,
-- `rust_gather_totals` — permanent, and the reason a wipe does
-- not erase a player's history.
-- THE WINDOW `rust_events` — recent detail, bounded by a sweep.
--
-- They are separate tables rather than columns on one because they have
-- different writers, different lifetimes and different audiences — and because
-- a purge of observed state while keeping the configuration is a thing an
-- operator will eventually want.
--
-- Teardown is `purge.sql`, which no boot ever runs.
@@ -97,3 +103,198 @@ CREATE TABLE IF NOT EXISTS rust_server_state (
CONSTRAINT fk_rust_server_state_server
FOREIGN KEY (server_id) REFERENCES rust_servers (id) ON DELETE CASCADE
);
-- ── The read path ─────────────────────────────────────────────────────────
--
-- Protocol 2 turned the bridge from a greeting into a catalogue, and these are
-- the tables that hold it. They divide on one line, and it is the line R12 drew:
--
-- PERMANENT `rust_wipes`, `rust_players`, `rust_player_wipe_stats`,
-- `rust_gather_totals` — a player's record, kept for ever. All-time
-- is a SUM across wipes rather than a second set of counters, so
-- there is no second number that can disagree with the first.
--
-- BOUNDED `rust_events` — the recent raw window the killfeed reads, pruned
-- on a sweep. It is detail, not record: losing last month's
-- individual deaths costs a scroll-back, losing last month's
-- totals costs a player their history.
--
-- DERIVED `rust_presence` — who is on right now, replaced wholesale from
-- the `players.online` board. Never a history, never appended.
--
-- The sidecar keeps its own bounded copy of the same events (default 14 days),
-- so shortening either window loses recent detail and neither loses a total.
-- ── Wipes ─────────────────────────────────────────────────────────────────
--
-- One row per (server, wipe). The id is the plugin's, derived from the save's
-- creation time and stamped on every frame (PROTOCOL.md §8.2) — this module
-- never derives one, because two derivations of one fact eventually disagree
-- about a boundary.
--
-- Rows appear by being MENTIONED: the first frame carrying a wipe id this module
-- has not seen creates it. There is no "start a wipe" call and there must not be
-- one, because the website is not present when a wipe happens — a wipe is a fact
-- about a world that was restarted while nobody was watching.
CREATE TABLE IF NOT EXISTS rust_wipes (
server_id VARCHAR(64) NOT NULL,
wipe_id VARCHAR(48) NOT NULL,
save_created_at VARCHAR(32) NULL,
first_seen DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_seen DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (server_id, wipe_id),
CONSTRAINT fk_rust_wipes_server
FOREIGN KEY (server_id) REFERENCES rust_servers (id) ON DELETE CASCADE
);
-- ── Players ───────────────────────────────────────────────────────────────
--
-- Identity, and deliberately nothing else. It is keyed on the Steam id alone
-- and carries no server: a player is the same person on all six of a community's
-- servers, and everything that is per-server lives in the stats table.
--
-- `user_id` is NOT here. Linking a Steam id to a website account is phase 6's
-- work (R1), and a column waiting for it would be a column every read has to
-- remember is always null.
CREATE TABLE IF NOT EXISTS rust_players (
steam_id VARCHAR(32) NOT NULL PRIMARY KEY,
name VARCHAR(191) NULL,
first_seen DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
last_seen DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
);
-- ── The permanent record ──────────────────────────────────────────────────
--
-- One row per player per wipe per server, and the only counters this module
-- keeps. R12's "per-wipe detail plus all-time rollups" is satisfied by SUMming
-- this rather than by maintaining a second all-time row, because two counters
-- for one fact drift the first time an ingest is replayed.
--
-- Every column is a COUNT that only ever goes up within a wipe, which is what
-- makes ingest idempotent-ish in the only way that matters: the cursor advances
-- only after the batch commits, so a crash re-reads a batch it has not counted.
--
-- `playtime_sec` comes from `sessionSec` on a disconnect, and a session whose
-- start this module never saw contributes NOTHING rather than zero — the plugin
-- omits the field, the ingest skips it, and the number stays honestly short
-- instead of quietly wrong.
CREATE TABLE IF NOT EXISTS rust_player_wipe_stats (
server_id VARCHAR(64) NOT NULL,
wipe_id VARCHAR(48) NOT NULL,
steam_id VARCHAR(32) NOT NULL,
kills INT UNSIGNED NOT NULL DEFAULT 0,
deaths INT UNSIGNED NOT NULL DEFAULT 0,
suicides INT UNSIGNED NOT NULL DEFAULT 0,
npc_kills INT UNSIGNED NOT NULL DEFAULT 0,
structures INT UNSIGNED NOT NULL DEFAULT 0,
sessions INT UNSIGNED NOT NULL DEFAULT 0,
playtime_sec BIGINT UNSIGNED NOT NULL DEFAULT 0,
last_seen DATETIME NULL,
PRIMARY KEY (server_id, wipe_id, steam_id),
KEY idx_rust_stats_kills (server_id, wipe_id, kills DESC),
KEY idx_rust_stats_player (steam_id)
);
-- ── What they gathered ────────────────────────────────────────────────────
--
-- A row per resource rather than a JSON blob on the stats row, for one reason:
-- the leaderboard question is "who gathered the most sulfur this wipe", and that
-- is an ORDER BY over a column in every SQL engine and a JSON function call in
-- exactly one. The resource name is the game's own shortname, unknown in advance
-- and not worth a lookup table.
CREATE TABLE IF NOT EXISTS rust_gather_totals (
server_id VARCHAR(64) NOT NULL,
wipe_id VARCHAR(48) NOT NULL,
steam_id VARCHAR(32) NOT NULL,
resource VARCHAR(64) NOT NULL,
amount BIGINT UNSIGNED NOT NULL DEFAULT 0,
PRIMARY KEY (server_id, wipe_id, steam_id, resource),
KEY idx_rust_gather_top (server_id, wipe_id, resource, amount DESC)
);
-- ── The recent raw window ─────────────────────────────────────────────────
--
-- Every ingested event, whole, for as long as the retention sweep keeps it. The
-- killfeed reads this; so does an admin looking at what happened.
--
-- `raw` holds the entire frame and the columns beside it are only what a query
-- needs to reach — the same rule the sidecar's own store follows, one hop along:
-- a protocol version that adds a field needs no migration here.
--
-- **`kind` is a security boundary, not a label.** Some kinds carry IP addresses
-- and player reports (PROTOCOL.md §8.4), and what makes them safe is that the
-- public read is filtered by an allowlist this module holds, default-deny. The
-- rows are stored either way, because an operator chasing ban evasion needs them.
CREATE TABLE IF NOT EXISTS rust_events (
id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
server_id VARCHAR(64) NOT NULL,
wipe_id VARCHAR(48) NULL,
kind VARCHAR(64) NOT NULL,
t BIGINT NOT NULL,
steam_id VARCHAR(32) NULL,
raw LONGTEXT NOT NULL,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
KEY idx_rust_events_server (server_id, id DESC),
KEY idx_rust_events_kind (server_id, kind, id DESC),
KEY idx_rust_events_wipe (server_id, wipe_id, id DESC),
KEY idx_rust_events_created (created_at)
);
-- ── Who is on right now ───────────────────────────────────────────────────
--
-- Replaced wholesale every time the `players.online` board arrives, which is on
-- every bridge connect and every 60 seconds. It is a BOARD, and the reason it is
-- its own table rather than rows in `rust_events` is that a board answers "now"
-- and an event answers "then"; storing a board as history is the mistake the
-- wire's `type` field exists to prevent, and it would be a shame to make it here
-- after the sidecar went to the trouble of not making it there.
CREATE TABLE IF NOT EXISTS rust_presence (
server_id VARCHAR(64) NOT NULL,
steam_id VARCHAR(32) NOT NULL,
name VARCHAR(191) NULL,
sleeping TINYINT(1) NOT NULL DEFAULT 0,
connected_at DATETIME NULL,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (server_id, steam_id)
);
-- ── The ingest cursor ─────────────────────────────────────────────────────
--
-- Where this module has read up to in each sidecar's feed. One row per server.
--
-- It is persisted rather than held in memory because the alternative is a module
-- that re-reads everything on every boot or nothing at all, and both are wrong in
-- a way that only shows up in production. The cursor advances **after** the batch
-- is written, never before: a crash mid-batch re-reads rows it has not counted,
-- which is the safe direction to be wrong in.
--
-- A NEW server starts at the sidecar's current end rather than at zero (see
-- `GET /feed` with no `since`). A module installed today against a sidecar that
-- has been running a month wants what happens next — replaying a fortnight of
-- deaths into stats whose wipes it never saw is not a catch-up, it is a
-- fabrication of history it was not present for.
CREATE TABLE IF NOT EXISTS rust_ingest_cursor (
server_id VARCHAR(64) NOT NULL PRIMARY KEY,
last_event_id BIGINT UNSIGNED NOT NULL DEFAULT 0,
events_seen BIGINT UNSIGNED NOT NULL DEFAULT 0,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT fk_rust_cursor_server
FOREIGN KEY (server_id) REFERENCES rust_servers (id) ON DELETE CASCADE
);
-- ── Changes to tables that already shipped ────────────────────────────────
--
-- An ALTER below the CREATE, never an edit to it: `CREATE TABLE IF NOT EXISTS`
-- does nothing against a database that already has the table, so an edited column
-- would reach fresh installs only — which is the worst possible distribution for
-- a schema change, because it works everywhere it is tested.
ALTER TABLE rust_server_state ADD COLUMN IF NOT EXISTS wipe_id VARCHAR(48) NULL;

242
server/ingest.js Normal file
View File

@@ -0,0 +1,242 @@
// ── Reading a sidecar's feed, and turning it into a record ────────────────
//
// One job: move each server's cursor forward, and apply what it passed.
//
// ── Why a cursor and not a socket ─────────────────────────────────────────
//
// The obvious design is a WebSocket — the sidecar has one, and module-uo takes
// exactly that route for the UO bridge. This module polls a cursor instead, and
// the reason is not laziness about latency.
//
// Core runs on Node 20, where a global `WebSocket` is still behind a flag, so a
// socket means taking `ws` as a runtime dependency — and this module's release
// asserts that it has none (D5: everything it needs arrives on `ctx`, and the
// bundle ships no `node_modules`). That is a cost worth paying for latency, but
// the deciding argument is the other one: **a socket needs a cursor anyway.**
// Whatever a feed misses while a module is restarting has to be caught up from
// somewhere, and the catch-up path is the one that must be right. A socket on
// top of a cursor is two mechanisms where the second is load-bearing; a cursor
// alone is one mechanism that is exercised every few seconds rather than only
// after an outage nobody planned.
//
// What it costs is seconds of latency on a killfeed. What it buys is that the
// path which recovers from a five-hour outage is the same path that ran a moment
// ago.
//
// ── The ordering the whole thing rests on ─────────────────────────────────
//
// **The cursor advances after the batch is written, never before.** A crash
// between the two re-reads events already counted, which inflates a total; a
// crash the other way round loses them silently and for ever. Neither is good and
// they are not equally bad — one is visible and bounded, the other is invisible
// and permanent — so the code is arranged to fail in the visible direction.
const core = require('./core')
const db = require('./model/events/events.db')
const sidecar = require('./sidecarClient')
const log = core.logger('ingest')
/** How many events to ask for at once. */
const BATCH = 200
/**
* How many batches one tick will drain before letting the loop breathe.
*
* A module that has been down for a day has thousands of events waiting, and
* draining them in one unbounded loop would hold the tick — and a pool
* connection — for as long as that takes. Bounded, it catches up over several
* ticks and the site stays responsive while it does.
*/
const MAX_BATCHES_PER_TICK = 10
/**
* Applies one feed item.
*
* Every frame is stored raw, and only some of them move a counter. That split is
* deliberate: the raw row is what an admin reads and what a later phase can
* re-derive from, and the counters are what a leaderboard sums. A kind this
* build has never heard of still lands in `rust_events` — it costs nothing and
* the alternative is losing the one copy of an event the next version will know
* how to read.
*/
async function apply(serverId, item) {
const frame = (item && item.frame) || {}
const kind = item.kind || frame.kind
const wipeId = frame.wipeId || null
// A wipe exists because something mentioned it. There is no "a wipe started"
// call and there must not be one: the website is not there when a wipe happens.
await db.touchWipe(serverId, wipeId, frame.saveCreatedAt || null)
await db.insertEvent({
serverId,
wipeId,
kind,
t: Number(frame.t) || item.t || Date.now(),
steamId: frame.steamId || null,
raw: frame,
})
const at = { serverId, wipeId, steamId: frame.steamId }
switch (kind) {
case 'player.connected':
await db.touchPlayer(frame.steamId, frame.name || null)
break
case 'player.disconnected': {
await db.touchPlayer(frame.steamId, frame.name || null)
// `sessionSec` is ABSENT when the plugin never saw the connect — a player
// already on the server when it loaded. Absent is not zero: adding a zero
// would be recording a session of no length, which is a different claim
// from recording no session, and it is the one that quietly under-reports
// playtime for ever.
const seconds = Number(frame.sessionSec)
await db.addStats(at, {
sessions: Number.isFinite(seconds) ? 1 : 0,
playtimeSec: Number.isFinite(seconds) && seconds > 0 ? seconds : 0,
})
break
}
case 'player.death': {
await db.touchPlayer(frame.steamId, frame.name || null)
// A suicide is a death AND a suicide, not one instead of the other: the
// deaths column is "how many times did this player die", and a leaderboard
// that silently omitted self-inflicted ones would disagree with the
// killfeed sitting next to it on the same page.
await db.addStats(at, { deaths: 1, suicides: frame.attackerType === 'self' ? 1 : 0 })
// Only a real player's kill counts. `npc` and `environment` have no
// attacker to credit, and `self` must not credit the victim with a kill —
// which is the one line here that would look right in review and produce a
// leaderboard topped by whoever died the most.
if (frame.attackerType === 'player' && frame.attackerId) {
await db.touchPlayer(frame.attackerId, frame.attackerName || null)
await db.addStats({ ...at, steamId: frame.attackerId }, { kills: 1 })
}
break
}
case 'player.tally': {
await db.touchPlayer(frame.steamId, frame.name || null)
await db.addStats(at, {
npcKills: Number(frame.npcKills) || 0,
structures: Number(frame.structures) || 0,
})
// A tally is a DELTA since the last flush, which is what makes adding it
// correct. If it ever becomes a running total this loop doubles every
// number in it, slowly, and looks right the whole time.
const gathered = frame.gathered || {}
for (const [resource, amount] of Object.entries(gathered)) {
await db.addGathered(at, resource, Number(amount) || 0)
}
break
}
case 'player.chat':
case 'player.respawned':
await db.touchPlayer(frame.steamId, frame.name || null)
break
default:
// Stored, not counted. Moderation frames, the server lifecycle, and
// anything a newer protocol sends that this build does not understand.
break
}
}
/**
* Brings one server's cursor up to date.
*
* Returns the number of events applied, for the log and for the tests.
*/
async function ingestServer(server) {
const cursor = await db.getCursor(server.id)
// A server this module has never ingested starts at the sidecar's CURRENT end,
// not at zero. A module installed today against a sidecar that has been running
// for a month should read what happens next — replaying a fortnight of deaths
// into stats for wipes it never saw is not a catch-up, it is inventing a
// history it was not present for. `/feed` with no `since` asks exactly that
// question, which is why the sidecar answers it that way.
if (!cursor) {
const tail = await sidecar.feedTail(server)
if (!tail.ok || !tail.data) {
// Unreachable. Write nothing: a cursor of 0 written now would replay the
// whole retained history the moment the sidecar came back.
return 0
}
await db.setCursor(server.id, Number(tail.data.lastId) || 0, 0)
log.info('cursor started at the feed tail', { server: server.id, at: tail.data.lastId })
return 0
}
let since = Number(cursor.lastEventId) || 0
let applied = 0
for (let batch = 0; batch < MAX_BATCHES_PER_TICK; batch += 1) {
const res = await sidecar.feed(server, since, BATCH)
if (!res.ok || !res.data) return applied
const items = Array.isArray(res.data.items) ? res.data.items : []
for (const item of items) {
try {
await apply(server.id, item)
applied += 1
} catch (err) {
// One malformed event must not wedge a server's cursor for ever. It is
// logged with its id so it can be found, and the cursor moves past it:
// the alternative is an ingest that stops at a single bad row and then
// silently stops being a feed at all.
log.warn('could not apply an event', {
server: server.id,
id: item && item.id,
kind: item && item.kind,
error: err.message,
})
}
}
const lastId = Number(res.data.lastId)
if (Number.isFinite(lastId) && lastId > since) {
// AFTER the batch. See the header.
await db.setCursor(server.id, lastId, items.length)
since = lastId
}
if (!res.data.more) break
}
if (applied > 0) log.info('ingested', { server: server.id, events: applied, cursor: since })
return applied
}
/**
* Applies the boards: what is true right now, rather than what happened.
*
* `players.online` replaces the presence rows wholesale, because that is what a
* board is. Storing it as history is the mistake the wire's `type` field exists
* to prevent, and it would be a poor return for the sidecar's trouble to make it
* here after it went out of its way not to make it there.
*/
async function applyBoards(serverId, boards) {
const presence = boards && boards['players.online']
if (presence && Array.isArray(presence.players)) {
await db.replacePresence(serverId, presence.players)
}
}
module.exports = { apply, applyBoards, ingestServer, BATCH, MAX_BATCHES_PER_TICK }

View File

@@ -0,0 +1,289 @@
// ── SQL for the read path ─────────────────────────────────────────────────
//
// Writes come from one caller (`server/ingest.js`) and reads from the routers.
// They live together because they are the same tables and the invariants are
// easier to keep true when the UPDATE and the SELECT are on the same screen.
//
// Raw parameterised SQL through `core.query`, no ORM. Placeholders always —
// except for one place where a list of kinds is expanded into placeholders, and
// that expansion is checked in `events.model.js` before it ever reaches here.
const core = require('../../core')
const EVENTS = 'rust_events'
const STATS = 'rust_player_wipe_stats'
const GATHER = 'rust_gather_totals'
const PLAYERS = 'rust_players'
const WIPES = 'rust_wipes'
const PRESENCE = 'rust_presence'
const CURSOR = 'rust_ingest_cursor'
// ── The cursor ────────────────────────────────────────────────────────────
async function getCursor(serverId) {
const rows = await core.query(
`SELECT server_id AS serverId, last_event_id AS lastEventId, events_seen AS eventsSeen
FROM ${CURSOR} WHERE server_id = ?`,
[serverId],
)
return rows[0] || null
}
/**
* Moves a server's cursor forward, counting what it passed.
*
* **Called only after the batch it describes has been written.** The whole
* correctness of the ingest is in that ordering: if this ran first, a crash
* between the two would skip events for ever, silently, with no way to notice.
* Running it last means a crash re-reads events it has already counted at worst
* — see `ingest.js` for what makes that survivable.
*/
async function setCursor(serverId, lastEventId, seen = 0) {
await core.query(
`INSERT INTO ${CURSOR} (server_id, last_event_id, events_seen, updated_at)
VALUES (?, ?, ?, CURRENT_TIMESTAMP)
ON DUPLICATE KEY UPDATE
last_event_id = VALUES(last_event_id),
events_seen = events_seen + VALUES(events_seen),
updated_at = CURRENT_TIMESTAMP`,
[serverId, lastEventId, seen],
)
}
// ── Writes ────────────────────────────────────────────────────────────────
async function insertEvent({ serverId, wipeId, kind, t, steamId, raw }) {
await core.query(
`INSERT INTO ${EVENTS} (server_id, wipe_id, kind, t, steam_id, raw)
VALUES (?, ?, ?, ?, ?, ?)`,
[serverId, wipeId || null, kind, t, steamId || null, JSON.stringify(raw)],
)
}
/**
* Notes that a wipe exists, from any frame that mentions it.
*
* There is no "a wipe started" call, because the website is not there when one
* does — a wipe happens to a game server that was restarted while nobody was
* watching. A wipe is therefore created by being mentioned, and `last_seen`
* moves every time it is mentioned again.
*/
async function touchWipe(serverId, wipeId, saveCreatedAt = null) {
if (!wipeId) return
await core.query(
`INSERT INTO ${WIPES} (server_id, wipe_id, save_created_at, first_seen, last_seen)
VALUES (?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
ON DUPLICATE KEY UPDATE
last_seen = CURRENT_TIMESTAMP,
save_created_at = COALESCE(VALUES(save_created_at), save_created_at)`,
[serverId, wipeId, saveCreatedAt],
)
}
/**
* Notes that a player exists and what they were last called.
*
* `name` is COALESCEd rather than overwritten so that a frame which carries no
* name — a ban by id, a tally — cannot blank out the name every other frame
* supplied.
*/
async function touchPlayer(steamId, name = null) {
if (!steamId) return
await core.query(
`INSERT INTO ${PLAYERS} (steam_id, name, first_seen, last_seen)
VALUES (?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
ON DUPLICATE KEY UPDATE
name = COALESCE(VALUES(name), name),
last_seen = CURRENT_TIMESTAMP`,
[steamId, name],
)
}
/**
* Adds to one player's counters for one wipe.
*
* Every column is a running total that only rises within a wipe, so this is an
* upsert that ADDS rather than sets. `deltas` names only what moved; a `+ 0` on
* everything else is what keeps the caller from having to read the row first.
*/
async function addStats({ serverId, wipeId, steamId }, deltas = {}) {
if (!serverId || !steamId) return
const cols = ['kills', 'deaths', 'suicides', 'npc_kills', 'structures', 'sessions', 'playtime_sec']
const values = {
kills: deltas.kills || 0,
deaths: deltas.deaths || 0,
suicides: deltas.suicides || 0,
npc_kills: deltas.npcKills || 0,
structures: deltas.structures || 0,
sessions: deltas.sessions || 0,
playtime_sec: deltas.playtimeSec || 0,
}
await core.query(
`INSERT INTO ${STATS} (server_id, wipe_id, steam_id, ${cols.join(', ')}, last_seen)
VALUES (?, ?, ?, ${cols.map(() => '?').join(', ')}, CURRENT_TIMESTAMP)
ON DUPLICATE KEY UPDATE
${cols.map((c) => `${c} = ${c} + VALUES(${c})`).join(',\n ')},
last_seen = CURRENT_TIMESTAMP`,
[serverId, wipeId || '', steamId, ...cols.map((c) => values[c])],
)
}
async function addGathered({ serverId, wipeId, steamId }, resource, amount) {
if (!serverId || !steamId || !resource || !(amount > 0)) return
await core.query(
`INSERT INTO ${GATHER} (server_id, wipe_id, steam_id, resource, amount)
VALUES (?, ?, ?, ?, ?)
ON DUPLICATE KEY UPDATE amount = amount + VALUES(amount)`,
[serverId, wipeId || '', steamId, resource, amount],
)
}
/**
* Replaces a server's presence rows with exactly what the board said.
*
* Two statements, delete then insert, because a board is a REPLACEMENT: a player
* who left between two boards has to disappear, and an upsert alone would leave
* them online for ever. It is not wrapped in a transaction on purpose — the
* window between the two is a fraction of a second of a page possibly showing an
* empty player list, against holding a lock on a table two routes read.
*/
async function replacePresence(serverId, players = []) {
await core.query(`DELETE FROM ${PRESENCE} WHERE server_id = ?`, [serverId])
for (const p of players) {
if (!p || !p.steamId) continue
await core.query(
`INSERT INTO ${PRESENCE} (server_id, steam_id, name, sleeping, connected_at, updated_at)
VALUES (?, ?, ?, ?, ${p.connectedAt ? 'FROM_UNIXTIME(? / 1000)' : 'NULL'}, CURRENT_TIMESTAMP)
ON DUPLICATE KEY UPDATE
name = VALUES(name), sleeping = VALUES(sleeping), updated_at = CURRENT_TIMESTAMP`,
p.connectedAt
? [serverId, p.steamId, p.name || null, p.sleeping ? 1 : 0, p.connectedAt]
: [serverId, p.steamId, p.name || null, p.sleeping ? 1 : 0],
)
}
}
/** Deletes raw events older than `days`. Totals are never touched — that is the point of them. */
async function pruneEvents(days) {
if (!(days > 0)) return 0
const res = await core.query(
`DELETE FROM ${EVENTS} WHERE created_at < DATE_SUB(CURRENT_TIMESTAMP, INTERVAL ? DAY)`,
[days],
)
return (res && res.affectedRows) || 0
}
// ── Reads ─────────────────────────────────────────────────────────────────
/**
* Recent events, newest first, restricted to `kinds`.
*
* **`kinds` is never optional.** A default of "all kinds" is one forgotten
* argument away from publishing an IP address, so the caller is made to say it
* every time; `events.model.js` builds the list from the catalogue's allowlist
* and an empty list answers with no rows rather than with everything.
*/
async function recentEvents({ serverId, kinds, wipeId = null, limit = 50 }) {
if (!Array.isArray(kinds) || kinds.length === 0) return []
const holes = kinds.map(() => '?').join(', ')
const params = [serverId, ...kinds]
let sql = `SELECT id, server_id AS serverId, wipe_id AS wipeId, kind, t, steam_id AS steamId, raw
FROM ${EVENTS}
WHERE server_id = ? AND kind IN (${holes})`
if (wipeId) {
sql += ' AND wipe_id = ?'
params.push(wipeId)
}
sql += ' ORDER BY id DESC LIMIT ?'
params.push(limit)
return core.query(sql, params)
}
/**
* The leaderboard for one wipe, or across every wipe when `wipeId` is null.
*
* All-time is a SUM over the per-wipe rows rather than a separate set of
* counters, which is what makes it impossible for the two to disagree — there
* is only ever one number, added up differently.
*/
async function leaderboard({ serverId, wipeId = null, sort = 'kills', limit = 25 }) {
const column = { kills: 'kills', deaths: 'deaths', npcKills: 'npc_kills', playtime: 'playtime_sec' }[sort] || 'kills'
const params = [serverId]
let where = 's.server_id = ?'
if (wipeId) {
where += ' AND s.wipe_id = ?'
params.push(wipeId)
}
params.push(limit)
return core.query(
`SELECT s.steam_id AS steamId,
p.name AS name,
SUM(s.kills) AS kills,
SUM(s.deaths) AS deaths,
SUM(s.npc_kills) AS npcKills,
SUM(s.structures) AS structures,
SUM(s.playtime_sec) AS playtimeSec,
MAX(s.last_seen) AS lastSeen
FROM ${STATS} s
LEFT JOIN ${PLAYERS} p ON p.steam_id = s.steam_id
WHERE ${where}
GROUP BY s.steam_id, p.name
ORDER BY SUM(s.${column}) DESC, MAX(s.last_seen) DESC
LIMIT ?`,
params,
)
}
async function listWipes(serverId) {
return core.query(
`SELECT wipe_id AS wipeId, save_created_at AS saveCreatedAt,
first_seen AS firstSeen, last_seen AS lastSeen
FROM ${WIPES}
WHERE server_id = ?
ORDER BY wipe_id DESC`,
[serverId],
)
}
async function presenceFor(serverId) {
return core.query(
`SELECT steam_id AS steamId, name, sleeping, connected_at AS connectedAt
FROM ${PRESENCE}
WHERE server_id = ?
ORDER BY name ASC`,
[serverId],
)
}
module.exports = {
getCursor,
setCursor,
insertEvent,
touchWipe,
touchPlayer,
addStats,
addGathered,
replacePresence,
pruneEvents,
recentEvents,
leaderboard,
listWipes,
presenceFor,
}

View File

@@ -0,0 +1,162 @@
// ── The read path's logic ─────────────────────────────────────────────────
//
// Everything that decides WHAT a caller gets, separated from the SQL that
// fetches it, so this file can be tested with no database and `events.db.js` has
// no branching to test.
//
// The decision that matters here is not a business rule, it is a boundary: what
// a signed-out visitor may see. Protocol 2 carries IP addresses and player
// reports, and the only thing standing between them and a public page is
// `catalogue.js`'s allowlist and the fact that **every read on this file takes an
// explicit viewer**. There is no default, because a default is what a caller
// gets when they forget — and the safe value is never the one that is easier to
// type.
const catalogue = require('../../catalogue')
const db = require('./events.db')
/** Hard ceiling on a page, whatever a caller asks for. */
const MAX_LIMIT = 200
function boundedLimit(requested, fallback = 50) {
const n = Number(requested)
if (!Number.isFinite(n) || n <= 0) return fallback
return Math.min(Math.trunc(n), MAX_LIMIT)
}
/**
* Parses a `kind` query parameter into a list.
*
* Accepts `?kind=player.death` and `?kind=player.death,player.chat`, and answers
* `null` for anything empty — which means "whatever this viewer may see" rather
* than "nothing", and is then narrowed by the catalogue.
*/
function parseKinds(raw) {
if (!raw) return null
const list = String(raw)
.split(',')
.map((k) => k.trim())
.filter(Boolean)
return list.length > 0 ? list : null
}
/**
* Recent events for one server, already narrowed to what this viewer may see.
*
* **`admin` is a parameter, not a default.** A route that forgets it gets the
* public list, which is the direction it is safe to be wrong in. And a kind the
* caller asked for that they may not see is dropped silently rather than
* refused: naming it in an error would confirm the kind exists, which is a small
* thing to leak and a free one to avoid.
*/
async function recent({ serverId, admin = false, kind = null, wipeId = null, limit }) {
const kinds = catalogue.kindsFor({ admin, requested: parseKinds(kind) })
// Every requested kind was refused. Answering with an empty list is right —
// the events they asked for are, as far as they are concerned, not there.
if (kinds.length === 0) return []
const rows = await db.recentEvents({
serverId,
kinds,
wipeId,
limit: boundedLimit(limit),
})
return rows.map(shape)
}
/**
* One stored row as an API object.
*
* `raw` comes back from the database as text and is parsed here rather than in
* the db layer, because a row whose JSON will not parse is a reporting problem
* and not a query problem: it answers with the envelope it does know and an
* empty body, instead of failing a whole page over one bad row.
*/
function shape(row) {
let frame = {}
try {
frame = typeof row.raw === 'string' ? JSON.parse(row.raw) : row.raw || {}
} catch {
frame = {}
}
return {
id: Number(row.id),
kind: row.kind,
t: Number(row.t),
wipeId: row.wipeId || null,
steamId: row.steamId || null,
frame,
}
}
/**
* The leaderboard for a server, per wipe or all-time.
*
* All-time is the same rows summed differently rather than a second set of
* counters, so the two can never disagree — which is the whole reason R12's
* "per-wipe detail plus all-time rollups" is one table and not two.
*/
async function leaderboard({ serverId, wipeId = null, sort = 'kills', limit }) {
const rows = await db.leaderboard({
serverId,
wipeId,
sort,
limit: boundedLimit(limit, 25),
})
return rows.map((r) => ({
steamId: r.steamId,
name: r.name || null,
kills: Number(r.kills) || 0,
deaths: Number(r.deaths) || 0,
npcKills: Number(r.npcKills) || 0,
structures: Number(r.structures) || 0,
playtimeSec: Number(r.playtimeSec) || 0,
lastSeen: r.lastSeen || null,
}))
}
/**
* Every wipe this server has had, newest first.
*
* The list is what makes the per-wipe view navigable, and it is also the proof
* R12 asks for: a wipe that ended is still here, with its stats still attached.
*/
async function wipes(serverId) {
const rows = await db.listWipes(serverId)
return rows.map((r) => ({
wipeId: r.wipeId,
saveCreatedAt: r.saveCreatedAt || null,
firstSeen: r.firstSeen,
lastSeen: r.lastSeen,
}))
}
/**
* Who is on the server right now.
*
* Read from the presence board rather than counted from connect and disconnect
* events: the board is re-sent on every bridge connect and every minute, so it
* is right even after this module has missed something. Counting transitions
* instead would drift, and drift in exactly the direction people notice —
* players who never left.
*/
async function online(serverId) {
const rows = await db.presenceFor(serverId)
return rows.map((r) => ({
steamId: r.steamId,
name: r.name || null,
sleeping: Boolean(r.sleeping),
connectedAt: r.connectedAt || null,
}))
}
module.exports = { recent, leaderboard, wipes, online, parseKinds, boundedLimit, MAX_LIMIT }

View File

@@ -79,7 +79,8 @@ async function listState() {
return core.query(
`SELECT server_id AS serverId, reachable, online, players, max_players AS maxPlayers,
hostname, level, seed, world_size AS worldSize, boot_id AS bootId,
save_created_at AS saveCreatedAt, protocol, updated_at AS updatedAt
save_created_at AS saveCreatedAt, wipe_id AS wipeId, protocol,
updated_at AS updatedAt
FROM ${STATE}`,
)
}
@@ -99,13 +100,14 @@ async function putState(state) {
await core.query(
`INSERT INTO ${STATE}
(server_id, reachable, online, players, max_players, hostname, level, seed,
world_size, boot_id, save_created_at, protocol, raw, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, CURRENT_TIMESTAMP)
world_size, boot_id, save_created_at, wipe_id, protocol, raw, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, CURRENT_TIMESTAMP)
ON DUPLICATE KEY UPDATE
reachable = VALUES(reachable), online = VALUES(online), players = VALUES(players),
max_players = VALUES(max_players), hostname = VALUES(hostname), level = VALUES(level),
seed = VALUES(seed), world_size = VALUES(world_size), boot_id = VALUES(boot_id),
save_created_at = VALUES(save_created_at), protocol = VALUES(protocol),
save_created_at = VALUES(save_created_at), wipe_id = VALUES(wipe_id),
protocol = VALUES(protocol),
raw = VALUES(raw), updated_at = CURRENT_TIMESTAMP`,
[
state.serverId,
@@ -119,6 +121,7 @@ async function putState(state) {
state.worldSize === undefined ? null : state.worldSize,
state.bootId || null,
state.saveCreatedAt || null,
state.wipeId || null,
state.protocol === undefined ? null : state.protocol,
state.raw ? JSON.stringify(state.raw) : null,
],

View File

@@ -8,6 +8,7 @@
"scripts": {
"test": "node --test",
"check:imports": "node scripts/checkImports.js",
"check:bundle": "node scripts/checkBundle.js",
"swagger": "node scripts/swaggerFragment.js",
"check:swagger": "node scripts/swaggerFragment.js --check"
},

View File

@@ -11,6 +11,7 @@
const core = require('../../core')
const events = require('../../model/events/events.model')
const servers = require('../../model/servers/servers.model')
const log = core.logger('public')
@@ -24,4 +25,62 @@ async function listServers(req, res) {
}
}
module.exports = { listServers }
/**
* The killfeed, and everything else public that happened on one server.
*
* **`admin` is not passed, and that is the whole security posture of this
* handler.** `events.recent` takes the viewer explicitly and defaults to the
* public allowlist, so the way to leak an IP address from here is to add an
* argument rather than to forget one.
*/
async function listEvents(req, res) {
try {
res.json({
events: await events.recent({
serverId: req.params.id,
kind: req.query.kind,
wipeId: req.query.wipe || null,
limit: req.query.limit,
}),
})
} catch (err) {
log.error('failed to read events', { server: req.params.id, error: err.message })
res.status(500).json({ error: 'Failed to read events' })
}
}
async function listLeaderboard(req, res) {
try {
res.json({
leaderboard: await events.leaderboard({
serverId: req.params.id,
wipeId: req.query.wipe || null,
sort: req.query.sort,
limit: req.query.limit,
}),
})
} catch (err) {
log.error('failed to read the leaderboard', { server: req.params.id, error: err.message })
res.status(500).json({ error: 'Failed to read the leaderboard' })
}
}
async function listWipes(req, res) {
try {
res.json({ wipes: await events.wipes(req.params.id) })
} catch (err) {
log.error('failed to read wipes', { server: req.params.id, error: err.message })
res.status(500).json({ error: 'Failed to read wipes' })
}
}
async function listOnline(req, res) {
try {
res.json({ players: await events.online(req.params.id) })
} catch (err) {
log.error('failed to read presence', { server: req.params.id, error: err.message })
res.status(500).json({ error: 'Failed to read who is online' })
}
}
module.exports = { listServers, listEvents, listLeaderboard, listWipes, listOnline }

View File

@@ -41,4 +41,65 @@ rustRouter.get(
servers.listServers,
)
// ── One server's read path ────────────────────────────────────────────────
//
// Every route below is public, and every one of them answers from this module's
// own tables — never from a live call to a sidecar. That is what lets the
// killfeed and the leaderboard render while every game server in the fleet is
// off, which is the same promise the server list makes.
//
// **The events route serves an ALLOWLIST, default-deny** (`catalogue.js`).
// Protocol 2 carries IP addresses and player reports; they are stored, and they
// do not come out here.
rustRouter.get(
'/servers/:id/events',
// #swagger.tags = ['Public · Rust']
// #swagger.summary = 'Recent events on one Rust server'
// #swagger.description = 'The killfeed and everything else public that happened on a server, newest first. Narrow with `kind` (comma-separated) and `wipe`. Only publicly classified kinds are ever returned — moderation events, login attempts and anything carrying an IP address are stored but never served here.'
// #swagger.parameters['id'] = { in: 'path', required: true, description: 'The servers slug', schema: { type: 'string' } }
// #swagger.parameters['kind'] = { in: 'query', required: false, description: 'One kind, or several comma-separated', schema: { type: 'string' } }
// #swagger.parameters['wipe'] = { in: 'query', required: false, description: 'Restrict to one wipe id', schema: { type: 'string' } }
// #swagger.parameters['limit'] = { in: 'query', required: false, description: 'Rows to return, capped at 200', schema: { type: 'integer' } }
/* #swagger.responses[200] = { description: 'Recent events, newest first' } */
siteMode,
servers.listEvents,
)
rustRouter.get(
'/servers/:id/leaderboard',
// #swagger.tags = ['Public · Rust']
// #swagger.summary = 'The leaderboard for one Rust server'
// #swagger.description = 'Per-wipe when `wipe` is given, all-time otherwise. All-time is the per-wipe rows summed rather than a second set of counters, so a wipe splits a players history without ending it.'
// #swagger.parameters['id'] = { in: 'path', required: true, description: 'The servers slug', schema: { type: 'string' } }
// #swagger.parameters['wipe'] = { in: 'query', required: false, description: 'Restrict to one wipe id', schema: { type: 'string' } }
// #swagger.parameters['sort'] = { in: 'query', required: false, description: 'kills, deaths, npcKills or playtime', schema: { type: 'string' } }
// #swagger.parameters['limit'] = { in: 'query', required: false, description: 'Rows to return, capped at 200', schema: { type: 'integer' } }
/* #swagger.responses[200] = { description: 'The leaderboard' } */
siteMode,
servers.listLeaderboard,
)
rustRouter.get(
'/servers/:id/wipes',
// #swagger.tags = ['Public · Rust']
// #swagger.summary = 'Every wipe this server has had'
// #swagger.description = 'Newest first. A wipe id is derived by the bridge plugin from the saves creation time and stamped on every frame, so it is the same id the events and the leaderboard are filtered by.'
// #swagger.parameters['id'] = { in: 'path', required: true, description: 'The servers slug', schema: { type: 'string' } }
/* #swagger.responses[200] = { description: 'The wipes' } */
siteMode,
servers.listWipes,
)
rustRouter.get(
'/servers/:id/online',
// #swagger.tags = ['Public · Rust']
// #swagger.summary = 'Who is on one Rust server right now'
// #swagger.description = 'Read from the presence board the bridge re-sends on every connect and every minute, rather than counted from connect and disconnect events — so it is correct even after the website has missed one.'
// #swagger.parameters['id'] = { in: 'path', required: true, description: 'The servers slug', schema: { type: 'string' } }
/* #swagger.responses[200] = { description: 'Who is online' } */
siteMode,
servers.listOnline,
)
module.exports = rustRouter

View File

@@ -0,0 +1,301 @@
#!/usr/bin/env node
// ── Does the release actually ship everything the module needs? ────────────
//
// `ci/bundle.json` says what a release copies. `server/index.js` says what the
// module requires. Nothing keeps two lists in agreement on its own, and the first
// module this project shipped proved it: `Module-uo` added `server/commands/` in a
// cutover, its include list did not learn about it, and v1.0.0 installed cleanly
// and then died on the operator's box with
//
// module "uo" failed to load — {"stage":"register","reason":"Cannot find
// module './commands/guild.command'"}
//
// Nothing caught it, because the PR checks install the module by copying the
// WHOLE repo into core — they only ever exercised a tree that had the file. **The
// subset only exists in the release**, and the release had no check that the
// subset was complete. This module has that check from its first release rather
// than after its first outage.
//
// It asks the question in the two places it can be asked:
//
// --check (PR checks) Every file reachable from the entry point by a
// relative require lives under something ci/bundle.json
// lists. Source-tree only, so it is fast and needs no
// assembled bundle — it fails on the PR that adds the
// directory, which is where the fix is cheapest.
//
// --bundle <dir> (release) Every relative specifier inside an ASSEMBLED bundle
// resolves to a file that is in it. Asked of the
// artifact rather than of the source, so it also
// catches a copy that half-failed, a list that names a
// path that has moved, and anything else between the
// declaration and the tarball.
//
// The two are deliberately not the same question. The first is about the list
// being right; the second is about the tarball being right. A release runs both.
//
// ── The third question, which is this module's own ─────────────────────────
//
// `server/package.json` declares **no runtime dependencies**, and the release
// therefore runs no `npm ci` and packs no `node_modules`. That is a decision, not
// an accident (org lead, phase 2), and the whole value of it is that the day it
// stops being true is a loud day. So both modes also assert the declaration is
// still empty: add a `dependencies` entry without teaching release.yml to install
// and pack it, and the bundle ships an import of something that is not there —
// the missing-directory failure again, wearing a different hat.
//
// ── Why reachability, and not "require the entry point" ────────────────────
//
// The obvious check — require the bundle's entry and see if it throws — does not
// work here, and the reason is in index.js's own header: its requires are inside
// `register()` because require order is load-bearing (`core.init(ctx)` has to run
// before anything under `router/` is required). So requiring the entry evaluates
// exactly one line, `require('./core')`, and reports success on a bundle missing
// every router it has. Calling `register()` for real would need a fake `ctx`
// complete enough to satisfy the whole module — which is what `test/` is for, and
// `test/` does not ship. Walking the requires statically asks the same question
// without needing either.
const fs = require('fs')
const path = require('path')
const { stripCommentsAndTemplates } = require('./checkImports')
const MODULE_ROOT = path.resolve(__dirname, '..', '..')
// Only relative specifiers. A bare one is checkImports.js's question, not this
// one, and the two failures want different advice.
const RELATIVE = /(?:require\(|from\s+|import\()\s*['"](\.[^'"]+)['"]/g
/**
* Resolve a relative specifier the way Node would, for the file cases that can
* appear here: an exact path, `+.js`/`+.json`, or a directory's `index.js`.
*
* Returns null when nothing exists — which is the finding, not an error.
*/
function resolveFile(fromDir, specifier) {
const base = path.resolve(fromDir, specifier)
const candidates = [base, `${base}.js`, `${base}.json`, path.join(base, 'index.js')]
for (const c of candidates) {
if (fs.existsSync(c) && fs.statSync(c).isFile()) return c
}
return null
}
/**
* Every file reachable from `entry` by following relative requires, plus every
* specifier that resolved to nothing.
*
* Exported so the test can point it at fixtures — the same reason checkImports.js
* exports `scan`. A check that has never been shown to fail is a check nobody
* knows the state of, and this one is load-bearing for every release.
*/
function reachable(entry) {
const seen = new Set()
const missing = []
const queue = [entry]
while (queue.length) {
const file = queue.shift()
if (seen.has(file)) continue
seen.add(file)
// A .json dependency is a leaf: it is reached, it ships, and it has no
// requires of its own to follow.
if (file.endsWith('.json')) continue
const source = stripCommentsAndTemplates(fs.readFileSync(file, 'utf8'))
for (const [, specifier] of source.matchAll(RELATIVE)) {
const target = resolveFile(path.dirname(file), specifier)
if (target) queue.push(target)
else missing.push({ file, specifier })
}
}
return { files: [...seen], missing }
}
/**
* Everything ci/bundle.json says ends up in the bundle, as absolute paths:
* `server[]` relative to server/, `root[]` and `generated[]` relative to the
* module root. All three are equally "in the tarball" as far as a require is
* concerned — the only difference is how they get there.
*/
function declaredServerPaths(moduleRoot = MODULE_ROOT) {
const manifest = JSON.parse(fs.readFileSync(path.join(moduleRoot, 'ci', 'bundle.json'), 'utf8'))
return [
...manifest.server.map((p) => path.join(moduleRoot, 'server', p)),
...(manifest.root || []).map((p) => path.join(moduleRoot, p)),
...(manifest.generated || []).map((p) => path.join(moduleRoot, p)),
]
}
/**
* The runtime dependencies the shipped half declares.
*
* Empty is the shape this module is built around, and the release packs no
* `node_modules` because of it. Returned rather than asserted so both modes can
* report it with their own advice.
*/
function runtimeDependencies(moduleRoot = MODULE_ROOT) {
const pkgPath = path.join(moduleRoot, 'server', 'package.json')
if (!fs.existsSync(pkgPath)) return []
return Object.keys(JSON.parse(fs.readFileSync(pkgPath, 'utf8')).dependencies || {})
}
const DEPENDENCY_ADVICE =
'The release packs no node_modules, because this module declared none. A dependency\n' +
'listed here but not installed and copied by .gitea/workflows/release.yml ships as an\n' +
'import of something that is not in the tarball — the module installs and then dies at\n' +
'the register stage on the operator\'s box.\n\n' +
'Either drop the dependency (everything the shipped half needs arrives on ctx — §2.3),\n' +
'or add the `npm ci --omit=dev` + copy steps to release.yml and list "node_modules" in\n' +
'ci/bundle.json\'s server[], then update this check.\n'
const covers = (declared, file) => declared.some((d) => file === d || file.startsWith(d + path.sep))
/**
* --check: is ci/bundle.json's list sufficient for what the entry point reaches?
*
* Reports the top-level entry to ADD rather than the individual files, because
* that is the edit: the list is stated in top-level paths, and a new directory
* arrives with a dozen files in it.
*/
function checkDeclaration(moduleRoot = MODULE_ROOT) {
const serverRoot = path.join(moduleRoot, 'server')
const entry = path.join(serverRoot, 'index.js')
const { files, missing } = reachable(entry)
const declared = declaredServerPaths(moduleRoot)
// Grouped by the entry that would have to be added, which is the top-level
// path under server/ — or, for the rare reachable file outside it, the path
// itself, since that one belongs in root[] instead.
const uncovered = new Map()
for (const file of files) {
if (covers(declared, file)) continue
const inServer = file.startsWith(serverRoot + path.sep)
const key = inServer
? `server/${path.relative(serverRoot, file).split(path.sep)[0]}`
: path.relative(moduleRoot, file).split(path.sep).join('/')
if (!uncovered.has(key)) uncovered.set(key, [])
uncovered.get(key).push(file)
}
return { uncovered, missing, reached: files.length, dependencies: runtimeDependencies(moduleRoot) }
}
/**
* --bundle: does every relative specifier inside an assembled bundle resolve?
*
* Walks the bundle's own server tree rather than starting from the entry point,
* so a file that ships but is broken is caught too.
*/
function checkBundle(bundleRoot) {
const serverRoot = path.join(bundleRoot, 'server')
const missing = []
const files = []
const walk = (dir) => {
for (const e of fs.readdirSync(dir, { withFileTypes: true })) {
const p = path.join(dir, e.name)
if (e.isDirectory()) {
// An installed dependency tree would be npm's business, not this
// check's. This module ships none; the skip stays so that the day one
// arrives, this is not also the thing that breaks.
if (e.name !== 'node_modules') walk(p)
} else if (/\.(js|mjs|cjs)$/.test(e.name)) {
files.push(p)
}
}
}
walk(serverRoot)
for (const file of files) {
const source = stripCommentsAndTemplates(fs.readFileSync(file, 'utf8'))
for (const [, specifier] of source.matchAll(RELATIVE)) {
if (!resolveFile(path.dirname(file), specifier)) missing.push({ file, specifier })
}
}
return { missing, scanned: files.length, dependencies: runtimeDependencies(bundleRoot) }
}
module.exports = {
reachable,
resolveFile,
checkDeclaration,
checkBundle,
declaredServerPaths,
runtimeDependencies,
}
// Required by a test, or run as the check? Only the second one exits.
if (require.main !== module) return
const bundleFlag = process.argv.indexOf('--bundle')
if (bundleFlag !== -1) {
const root = process.argv[bundleFlag + 1]
if (!root) {
console.error('--bundle needs the path to an assembled bundle')
process.exit(2)
}
const { missing, scanned, dependencies } = checkBundle(path.resolve(root))
if (missing.length) {
console.error(`\nThe assembled bundle is incomplete — ${missing.length} require(s) resolve to nothing:\n`)
for (const m of missing) {
console.error(` ${path.relative(root, m.file)}\n requires "${m.specifier}" — not in the bundle`)
}
console.error('\nAdd the missing path to ci/bundle.json.\n')
process.exit(1)
}
if (dependencies.length) {
console.error(`\nThe assembled bundle declares ${dependencies.length} runtime dependency(ies) it does not carry:\n`)
for (const d of dependencies) console.error(` ${d}`)
console.error(`\n${DEPENDENCY_ADVICE}`)
process.exit(1)
}
console.log(`OK — every relative require in the bundle resolves (${scanned} files scanned), and it needs no node_modules.`)
} else {
const { uncovered, missing, reached, dependencies } = checkDeclaration()
if (missing.length) {
console.error(`\n${missing.length} require(s) resolve to nothing in the source tree:\n`)
for (const m of missing) {
console.error(` ${path.relative(MODULE_ROOT, m.file)}\n requires "${m.specifier}"`)
}
console.error('')
process.exit(1)
}
if (uncovered.size) {
console.error('\nci/bundle.json does not ship everything server/index.js reaches.\n')
console.error('A release built from this list would install and then fail at the')
console.error("register stage with \"Cannot find module\", on the operator's box.\n")
for (const [key, files] of uncovered) {
console.error(` ${key} (${files.length} file${files.length === 1 ? '' : 's'} reachable)`)
for (const f of files.slice(0, 5)) console.error(` ${path.relative(MODULE_ROOT, f)}`)
if (files.length > 5) console.error(` … and ${files.length - 5} more`)
}
// server[] is written relative to server/, so name the entry to add rather
// than the path just displayed — they differ by exactly that prefix.
const toServer = [...uncovered.keys()].filter((k) => k.startsWith('server/'))
const toRoot = [...uncovered.keys()].filter((k) => !k.startsWith('server/'))
if (toServer.length) {
console.error(`\nAdd ${toServer.map((k) => `"${k.slice('server/'.length)}"`).join(', ')} to ci/bundle.json's server[].`)
}
if (toRoot.length) {
console.error(`\nAdd ${toRoot.map((k) => `"${k}"`).join(', ')} to ci/bundle.json's root[].`)
}
console.error('')
process.exit(1)
}
if (dependencies.length) {
console.error(`\nserver/package.json declares ${dependencies.length} runtime dependency(ies):\n`)
for (const d of dependencies) console.error(` ${d}`)
console.error(`\n${DEPENDENCY_ADVICE}`)
process.exit(1)
}
console.log(`OK — ci/bundle.json ships every file server/index.js reaches (${reached} files), and no runtime dependency is declared.`)
}

View File

@@ -0,0 +1,196 @@
#!/usr/bin/env node
// ── §5.3 — this module's frozen route manifest ─────────────────────────────
//
// Core freezes its URL surface in `server/routes.manifest.json` by walking the
// live Express stack and committing the result; a PR that moves a URL has to
// commit the new manifest, which puts the change in front of a reviewer. The URLs
// this module serves are not in that file. They are here, frozen the same way and
// by the same generator.
//
// **The module's routes are DERIVED, never listed.** This script is handed two
// manifests generated from the SAME core at the pinned ref — one without this
// module on the volume, one with — and the difference is what this module serves.
// Nothing here says "/api/v1/public/rust/*"; a mount prefix appears in exactly one
// place, `server/index.js`'s `registerRoutes` call, which is where an operator's
// core reads it from too.
//
// Taking the difference rather than filtering by prefix buys the other half of
// §5.3 for free, and it is the half that matters most: **no core URL may move.**
// A module that shadowed a core route, or whose mount displaced one, shows up here
// as a removal or a change, not merely as an addition somewhere else. That is the
// promise §1.2 makes to the shipped Android app and the Discord bot.
//
// It is also the only check that can see the blind spot §13's own registration
// comment names: core answers several public routes mounted at the TIER ROOT
// rather than under a prefix — `/status` and `/version` among them — and the
// loader's collision probe cannot find those. `/rust` was checked against core's
// mount tables by hand when phase 1 chose it. From here it is checked by a core.
//
// The third thing it checks is the OpenAPI fragment (§2.8). `swagger-fragment.json`
// is generated from the module's own registrations against §2.4's stated tier
// bases — the one place a constant could be wrong. Here there is ground truth: a
// real core with this module loaded, reporting the URLs it actually serves. Every
// route must have a documented operation and every documented operation must be a
// route. That is the per-module form of core's standing rule, never ship a route
// that isn't in the spec — and it is what stops a wrong constant in the generator
// from producing a fragment that is internally consistent and describes nothing
// core will ever serve.
//
// Usage (the workflow does the cloning; see .gitea/workflows/pr-checks.yml):
// node scripts/frozenManifest.js --before core-only.json --after core-plus-rust.json
// node scripts/frozenManifest.js --before … --after … --check
const fs = require('fs')
const path = require('path')
const MODULE_ROOT = path.resolve(__dirname, '..', '..')
const MANIFEST = path.join(MODULE_ROOT, 'routes.manifest.json')
const FRAGMENT = path.join(MODULE_ROOT, 'swagger-fragment.json')
const COMMENT =
'Generated inventory of the URLs module-rust serves - the module half of the freeze ' +
'core keeps in server/routes.manifest.json. DERIVED as the difference between a core ' +
'without this module and the same core with it, both at the pinned ref in ci/core-ref.json. ' +
'Regenerate with the frozen-manifest job in .gitea/workflows/pr-checks.yml; see ' +
'server/scripts/frozenManifest.js.'
const key = (r) => `${r.method} ${r.path}`
/**
* The module's routes, plus proof that core's own surface did not move.
*
* @param {object} before routes.manifest.json from core alone
* @param {object} after routes.manifest.json from the same core with this module
* @returns {{ added: object[], removed: string[] }}
*/
function diffManifests(before, after) {
const added = []
const removed = []
for (const tier of ['public', 'internal']) {
const was = new Set((before[tier] || []).map(key))
for (const route of after[tier] || []) {
if (!was.has(key(route))) added.push({ ...route, tier })
was.delete(key(route))
}
for (const gone of was) removed.push(`${tier} ${gone}`)
}
added.sort((a, b) => (key(a) < key(b) ? -1 : 1))
return { added, removed }
}
/**
* Which of the module's routes the fragment fails to document, and vice versa.
*
* Express `:id` is OpenAPI `{id}`; the fragment is already in OpenAPI's spelling
* because that is what core merges, so the manifest's paths are converted here
* rather than the other way round.
*/
function coverage(added, fragment) {
const documented = new Set()
for (const [p, item] of Object.entries(fragment.paths || {})) {
for (const method of Object.keys(item)) documented.add(`${method.toUpperCase()} ${p}`)
}
const undocumented = []
for (const route of added) {
const oas = `${route.method} ${route.path.replace(/:([A-Za-z0-9_]+)/g, '{$1}')}`
if (documented.has(oas)) documented.delete(oas)
else undocumented.push(oas)
}
// Whatever is left is documented and not served: a route that moved or was
// deleted while its annotation stayed behind. Core's own spec has no equivalent
// check and grew four orphan tags and thirty-three orphan schemas because of it.
return { undocumented, unserved: [...documented].sort() }
}
function serialize(routes) {
return `${JSON.stringify(
{
$comment: COMMENT,
routes: routes.map(({ method, path: p, tier }) => ({ method, path: p, tier })),
},
null,
2,
)}\n`
}
function main() {
const arg = (name) => {
const i = process.argv.indexOf(name)
return i === -1 ? null : process.argv[i + 1]
}
const beforePath = arg('--before')
const afterPath = arg('--after')
if (!beforePath || !afterPath) {
process.stderr.write('usage: frozenManifest.js --before <manifest> --after <manifest> [--check]\n')
process.exit(2)
}
const before = JSON.parse(fs.readFileSync(beforePath, 'utf8'))
const after = JSON.parse(fs.readFileSync(afterPath, 'utf8'))
const { added, removed } = diffManifests(before, after)
let failed = false
if (removed.length > 0) {
process.stderr.write(
`\nLoading this module REMOVED or CHANGED ${removed.length} of core's own route(s):\n` +
`${removed.map((r) => ` - ${r}`).join('\n')}\n` +
'A module may only add. This is the frozen-URL promise (MODULE_SYSTEM.md §1.2) breaking.\n',
)
failed = true
}
if (added.length === 0) {
process.stderr.write(
'\nLoading this module added NO routes. Either it failed to load in the core checkout\n' +
'(check the boot log for a startup_failed line) or the two manifests are the same file.\n',
)
process.exit(1)
}
const fragment = JSON.parse(fs.readFileSync(FRAGMENT, 'utf8'))
const { undocumented, unserved } = coverage(added, fragment)
if (undocumented.length > 0) {
process.stderr.write(
`\n${undocumented.length} route(s) this module serves have no operation in swagger-fragment.json:\n` +
`${undocumented.map((r) => ` - ${r}`).join('\n')}\n` +
'Run `npm run swagger --prefix server` and commit the result (MODULE_API.md §2.8).\n',
)
failed = true
}
if (unserved.length > 0) {
process.stderr.write(
`\n${unserved.length} operation(s) in swagger-fragment.json are not routes this module serves:\n` +
`${unserved.map((r) => ` - ${r}`).join('\n')}\n` +
'A documented URL nobody serves is a client following the docs into a 404.\n',
)
failed = true
}
if (failed) process.exit(1)
const contents = serialize(added)
if (process.argv.includes('--check')) {
const current = fs.existsSync(MANIFEST) ? fs.readFileSync(MANIFEST, 'utf8').replace(/\r\n/g, '\n') : null
if (current !== contents) {
process.stderr.write(
'\nroutes.manifest.json is stale. The URLs this module serves changed — regenerate it and\n' +
'commit the result so the move is reviewed rather than merged as mechanical.\n',
)
process.exit(1)
}
process.stdout.write(`routes.manifest.json is current — ${added.length} routes, all documented\n`)
return
}
fs.writeFileSync(MANIFEST, contents)
process.stdout.write(`wrote routes.manifest.json — ${added.length} routes, all documented\n`)
}
if (require.main === module) main()
module.exports = { diffManifests, coverage, serialize, MANIFEST, FRAGMENT }

View File

@@ -48,14 +48,22 @@ const log = core.logger('sidecar')
const TIMEOUT_MS = 12000
/**
* The wire version this module speaks. Declared in three places that must agree:
* here, `PROTOCOL_VERSION` in the sidecar, and `overlay.toml` in Rust-Plugins.
* The wire version this module speaks. Declared in FOUR places that must agree:
* here, `PROTOCOL_VERSION` in the sidecar, `ProtocolVersion` in the bridge
* plugin, and `protocol` in its `overlay.toml`.
*
* **2 — the read path.** The bump lands here in the same change as the emitters,
* even though this module does not yet consume any of the new frames: the
* sidecar refuses a client declaring a different version with a `409`, so a
* module left on 1 would stop being able to read the server board it has been
* reading all along. A constant that lags the deployment is not a safe default;
* it is an outage with a version number on it.
*
* It is sent on every request as `X-RustLink-Version`, which turns a mismatched
* deployment into a `409` naming both numbers instead of a parse failure three
* layers further in.
*/
const PROTOCOL_VERSION = 1
const PROTOCOL_VERSION = 2
/** What a caller gets back. Shaped once so every call site reads the same. */
function reply(ok, status, data = null) {
@@ -163,6 +171,24 @@ const serverBoard = (server) => request(server, '/server')
/** A live round trip through the sidecar to the game. Fails when the game is down, by design. */
const liveStatus = (server) => request(server, '/status')
/** Every board at once: what is true now, before following what happens next. */
const boards = (server) => request(server, '/boards')
/**
* The ingest cursor: events after `since`, oldest first.
*
* **`since` is required here, unlike on the wire.** The sidecar treats an omitted
* cursor as "tell me where the end is", which is a genuinely useful question and
* a catastrophic default for an ingest loop that would silently store nothing
* and advance past everything. So the question is asked explicitly, by name, and
* a caller cannot get it by forgetting an argument.
*/
const feed = (server, since, limit = 200) =>
request(server, `/feed?since=${encodeURIComponent(since)}&limit=${encodeURIComponent(limit)}`)
/** Where the sidecar's history currently ends. What a new server's cursor starts at. */
const feedTail = (server) => request(server, '/feed')
module.exports = {
TIMEOUT_MS,
PROTOCOL_VERSION,
@@ -170,5 +196,8 @@ module.exports = {
health,
serverBoard,
liveStatus,
boards,
feed,
feedTail,
joinUrl,
}

View File

@@ -0,0 +1,110 @@
// ── The boundary, asserted ────────────────────────────────────────────────
//
// `catalogue.js` is the only thing standing between a frame carrying an IP
// address and a public page, so it gets a suite of its own rather than being
// covered incidentally by a route test.
//
// The most valuable test here is the last one: it holds the classification
// against the specification in `docs/rust-link/PROTOCOL.md` §8.4. Without it the
// two drift the first time somebody adds a kind to the protocol, and the drift
// is silent in the direction that matters — a new kind is simply never served,
// until the day somebody "fixes" that by adding it to the wrong list.
const test = require('node:test')
const assert = require('node:assert')
const catalogue = require('../catalogue')
test('an unknown kind is not public — the default is deny', () => {
assert.equal(catalogue.isPublic('player.death'), true)
assert.equal(catalogue.isPublic('something.new'), false)
assert.equal(catalogue.isPublic(''), false)
assert.equal(catalogue.isPublic(undefined), false)
// The shape of the mistake this prevents: a kind a LATER protocol adds, which
// this build ingests happily and would publish on the day it first arrived if
// the filter were a deny list.
assert.equal(catalogue.isKnown('player.location'), false)
assert.equal(catalogue.isPublic('player.location'), false)
})
test('nothing carrying an IP address or a report is public', () => {
for (const kind of [
'player.login.attempt',
'player.approved',
'player.banned',
'player.unbanned',
'player.reported',
'entity.destroyed',
]) {
assert.equal(catalogue.isPublic(kind), false, `${kind} must not be public`)
assert.ok(catalogue.STAFF_KINDS.includes(kind), `${kind} must be classified, not merely absent`)
}
})
test('a viewer with no kinds asked for gets the allowlist, never everything', () => {
const asPublic = catalogue.kindsFor({})
const asAdmin = catalogue.kindsFor({ admin: true })
assert.deepEqual(asPublic, [...catalogue.PUBLIC_KINDS])
assert.equal(asAdmin.length, catalogue.ALL_KINDS.length)
// The property that makes the route safe by construction: there is no argument
// a caller can omit that turns the filter off.
assert.ok(asPublic.length > 0)
assert.ok(!asPublic.includes('player.banned'))
})
test('a kind a viewer may not see is dropped, not refused', () => {
const asked = catalogue.kindsFor({ requested: ['player.death', 'player.banned'] })
assert.deepEqual(asked, ['player.death'])
// Asking for only forbidden kinds answers with nothing to select, which the
// model turns into an empty list — the events are, as far as this viewer is
// concerned, not there.
assert.deepEqual(catalogue.kindsFor({ requested: ['player.banned'] }), [])
// And an admin gets what they asked for.
assert.deepEqual(catalogue.kindsFor({ admin: true, requested: ['player.banned'] }), [
'player.banned',
])
})
test('every kind is classified exactly once', () => {
const seen = new Set()
for (const kind of catalogue.ALL_KINDS) {
assert.ok(!seen.has(kind), `${kind} appears in both lists`)
seen.add(kind)
}
assert.equal(seen.size, catalogue.PUBLIC_KINDS.length + catalogue.STAFF_KINDS.length)
})
test('the classification covers exactly the kinds protocol 2 defines', () => {
// The spec lives in another repository, so the list is restated here rather
// than parsed — and restating it is the point: adding a kind to the protocol
// without deciding who may see it has to fail somewhere, and this is where.
//
// Sourced from docs/rust-link/PROTOCOL.md §8.4.
const PROTOCOL_2 = [
'player.connected',
'player.disconnected',
'player.respawned',
'player.death',
'player.chat',
'player.tally',
'entity.destroyed',
'player.reported',
'player.banned',
'player.unbanned',
'player.login.attempt',
'player.approved',
'server.wipe',
'server.initialized',
'server.shutdown',
]
assert.deepEqual([...catalogue.ALL_KINDS].sort(), [...PROTOCOL_2].sort())
})

View File

@@ -0,0 +1,273 @@
// The bundle check, checked.
//
// `scripts/checkBundle.js` exists because of a failure this module has not had
// and does not intend to: Module-uo's v1.0.0 shipped without `server/commands/`
// and died at the register stage on the operator's box. A check written in
// response to one bug is worth exactly as much as its coverage of that bug, so
// the first two tests below are that bug, in both modes — a list that has stopped
// covering what the entry point reaches, and a tarball with the file missing from
// it — and the third pair is this module's own version of it, a runtime
// dependency declared and not packed.
//
// **Every fixture is a template literal, and that is load-bearing** — the same
// reason checkImports.test.js gives. `scripts/checkImports.js` scans this
// directory too, so an ordinary quoted string holding a relative require would
// make this file fail that check. Templates are blanked by the stripper.
const test = require('node:test')
const assert = require('node:assert')
const fs = require('node:fs')
const os = require('node:os')
const path = require('node:path')
const {
reachable,
resolveFile,
checkDeclaration,
checkBundle,
declaredServerPaths,
runtimeDependencies,
} = require('../scripts/checkBundle')
/**
* Write a throwaway module tree: `files` under server/, `bundle` as its
* ci/bundle.json, `pkg` as its server/package.json. Returns the module root.
*/
function fixture(files, bundle = { server: ['index.js'] }, pkg = null) {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'module-rust-bundle-'))
for (const [name, source] of Object.entries(files)) {
const file = path.join(root, 'server', name)
fs.mkdirSync(path.dirname(file), { recursive: true })
fs.writeFileSync(file, source)
}
fs.mkdirSync(path.join(root, 'ci'), { recursive: true })
fs.writeFileSync(path.join(root, 'ci', 'bundle.json'), JSON.stringify(bundle))
if (pkg) {
fs.mkdirSync(path.join(root, 'server'), { recursive: true })
fs.writeFileSync(path.join(root, 'server', 'package.json'), JSON.stringify(pkg))
}
return root
}
const cleanup = (root) => fs.rmSync(root, { recursive: true, force: true })
// ── The regression this script was written for ─────────────────────────────
test('--check catches a directory the include list has stopped covering', () => {
const root = fixture(
{
'index.js': `const r = require('./router/public/rust.router')`,
'router/public/rust.router.js': `module.exports = {}`,
},
{ server: ['index.js'] }, // `router` missing — exactly Module-uo's v1.0.0
)
try {
const { uncovered } = checkDeclaration(root)
assert.strictEqual(uncovered.size, 1)
assert.ok(uncovered.has('server/router'))
} finally {
cleanup(root)
}
})
test('--bundle catches the file missing from an assembled tarball', () => {
const root = fixture({ 'index.js': `require('./router/public/rust.router')` })
try {
const { missing } = checkBundle(root)
assert.strictEqual(missing.length, 1)
assert.strictEqual(missing[0].specifier, './router/public/rust.router')
} finally {
cleanup(root)
}
})
// ── This module's own version of that failure ──────────────────────────────
//
// The release packs no node_modules because the shipped half declares no
// dependencies (org lead, phase 2). The value of that decision is entirely in
// the day it stops being true being a LOUD day, so both modes ask.
test('--check reports a runtime dependency the release would not pack', () => {
const root = fixture({ 'index.js': `module.exports = 1` }, { server: ['index.js'] }, {
name: 'x',
dependencies: { ws: '^8.21.0' },
})
try {
assert.deepStrictEqual(checkDeclaration(root).dependencies, ['ws'])
} finally {
cleanup(root)
}
})
test('--bundle reports a dependency the assembled bundle declares and does not carry', () => {
const root = fixture({ 'index.js': `module.exports = 1` }, { server: ['index.js'] }, {
name: 'x',
dependencies: { ws: '^8.21.0' },
})
try {
assert.deepStrictEqual(checkBundle(root).dependencies, ['ws'])
} finally {
cleanup(root)
}
})
test('devDependencies are not runtime dependencies', () => {
// express, express-validator and swagger-autogen are all here and none of them
// ships: the shipped half is handed express on `ctx` (§2.3). A check that
// confused the two would fail on a correct repo, which is the one way to make
// everyone stop reading it.
const root = fixture({ 'index.js': `module.exports = 1` }, { server: ['index.js'] }, {
name: 'x',
devDependencies: { express: '^4.19.2' },
})
try {
assert.deepStrictEqual(runtimeDependencies(root), [])
} finally {
cleanup(root)
}
})
// ── It has to reach requires that are not at the top level ─────────────────
test('follows requires written inside a function', () => {
// index.js requires inside `register()` because require order is load-bearing:
// `core.init(ctx)` has to run before anything under router/ is required. A
// check that only saw file-scope requires would miss every router this module
// has.
const root = fixture(
{
'index.js': `module.exports = function register(ctx) { const r = require('./router/a') }`,
'router/a.js': `module.exports = {}`,
},
{ server: ['index.js', 'router'] },
)
try {
assert.strictEqual(checkDeclaration(root).uncovered.size, 0)
assert.strictEqual(checkBundle(root).missing.length, 0)
} finally {
cleanup(root)
}
})
test('follows requires transitively, not just one hop', () => {
const root = fixture(
{
'index.js': `require('./a')`,
'a.js': `require('./b')`,
'b.js': `require('./deep/c')`,
'deep/c.js': `module.exports = {}`,
},
{ server: ['index.js', 'a.js', 'b.js'] }, // `deep` missing
)
try {
assert.ok(checkDeclaration(root).uncovered.has('server/deep'))
} finally {
cleanup(root)
}
})
// ── Resolution has to match Node's, or it invents failures ─────────────────
test('resolves a directory to its index.js', () => {
const root = fixture(
{ 'index.js': `require('./boot')`, 'boot/index.js': `module.exports = {}` },
{ server: ['index.js', 'boot'] },
)
try {
assert.strictEqual(checkDeclaration(root).uncovered.size, 0)
} finally {
cleanup(root)
}
})
test('resolves a .json dependency, and does not try to parse it for requires', () => {
// server/index.js's last line requires ../module.json, which is why this case
// is not hypothetical and why `generated` is in the declared list at all.
const root = fixture(
{ 'index.js': `require('./data/atlas.json')`, 'data/atlas.json': `{"a":1}` },
{ server: ['index.js', 'data'] },
)
try {
const { uncovered, missing } = checkDeclaration(root)
assert.strictEqual(missing.length, 0)
assert.strictEqual(uncovered.size, 0)
} finally {
cleanup(root)
}
})
test('survives a require cycle', () => {
const root = fixture(
{ 'index.js': `require('./a')`, 'a.js': `require('./index')` },
{ server: ['index.js', 'a.js'] },
)
try {
assert.strictEqual(checkDeclaration(root).uncovered.size, 0)
} finally {
cleanup(root)
}
})
test('a specifier that resolves to nothing is reported, not thrown', () => {
const root = fixture({ 'index.js': `require('./gone')` })
try {
const { missing } = checkDeclaration(root)
assert.strictEqual(missing.length, 1)
assert.strictEqual(missing[0].specifier, './gone')
} finally {
cleanup(root)
}
})
test('prose describing a require is not a require', () => {
// The failure mode checkImports.js hit the first time it ran: index.js's own
// header explains why it must never require express, and comments in this repo
// name module paths constantly.
const root = fixture(
{ 'index.js': `// this file used to require('./router/gone')\nmodule.exports = 1` },
{ server: ['index.js'] },
)
try {
assert.strictEqual(checkDeclaration(root).missing.length, 0)
} finally {
cleanup(root)
}
})
test("node_modules inside a bundle is npm's business, not this check's", () => {
// Nothing ships one today. The skip stays so that the day a dependency does
// arrive, this is not also the thing that breaks.
const root = fixture({
'index.js': `module.exports = 1`,
'node_modules/ws/index.js': `require('./lib/that-npm-owns')`,
})
try {
assert.strictEqual(checkBundle(root).missing.length, 0)
} finally {
cleanup(root)
}
})
// ── And the real repo, which is the check that actually gates a release ────
test('the real ci/bundle.json covers everything the real entry point reaches', () => {
const { uncovered, missing, reached, dependencies } = checkDeclaration()
assert.deepStrictEqual([...uncovered.keys()], [])
assert.deepStrictEqual(missing, [])
assert.deepStrictEqual(dependencies, [])
assert.ok(reached > 1, 'the walk should reach more than the entry point itself')
})
test('every path ci/bundle.json declares exists', () => {
// A list naming a path that has moved packs nothing and says nothing — `cp` in
// the release would fail, but only after the tag had been pushed.
for (const p of declaredServerPaths()) {
assert.ok(fs.existsSync(p), `ci/bundle.json names ${p}, which does not exist`)
}
})
test('the entry point is reachable from the declared list', () => {
const entry = path.resolve(__dirname, '..', 'index.js')
assert.ok(reachable(entry).files.includes(entry))
assert.ok(resolveFile(path.dirname(entry), './core'))
})

144
server/test/events.test.js Normal file
View File

@@ -0,0 +1,144 @@
// ── The read path's logic ─────────────────────────────────────────────────
//
// The model decides what a caller gets. Two properties are worth more than the
// rest, and both are about a caller who did something slightly wrong:
//
// • a route that forgets to say who is asking gets the PUBLIC view;
// • a caller asking for a million rows gets two hundred.
const test = require('node:test')
const assert = require('node:assert')
const { fakeCtx } = require('./_fakes')
function withCore() {
require('../core')._reset()
require('../core').init(fakeCtx())
}
test('the limit is bounded, whatever was asked for', () => {
withCore()
const model = require('../model/events/events.model')
assert.equal(model.boundedLimit(10), 10)
assert.equal(model.boundedLimit(undefined), 50)
assert.equal(model.boundedLimit('nonsense'), 50)
assert.equal(model.boundedLimit(-5), 50)
assert.equal(model.boundedLimit(0), 50)
assert.equal(model.boundedLimit(1e9), model.MAX_LIMIT)
assert.equal(model.boundedLimit(12.9), 12)
})
test('kinds parse from one name or a list, and nothing means "not specified"', () => {
withCore()
const model = require('../model/events/events.model')
assert.deepEqual(model.parseKinds('player.death'), ['player.death'])
assert.deepEqual(model.parseKinds('player.death, player.chat'), ['player.death', 'player.chat'])
// Null rather than an empty list: "I did not ask" and "I asked for nothing"
// are different, and only the first means "whatever I am allowed".
assert.equal(model.parseKinds(''), null)
assert.equal(model.parseKinds(undefined), null)
assert.equal(model.parseKinds(' , , '), null)
})
test('a reader who does not say who they are gets the public view', async () => {
withCore()
const db = require('../model/events/events.db')
const model = require('../model/events/events.model')
const original = db.recentEvents
let asked = null
db.recentEvents = async (args) => {
asked = args
return []
}
try {
await model.recent({ serverId: 'main' })
assert.ok(!asked.kinds.includes('player.banned'), 'no IP-carrying kind by default')
assert.ok(asked.kinds.includes('player.death'))
await model.recent({ serverId: 'main', admin: true })
assert.ok(asked.kinds.includes('player.banned'), 'an admin who says so gets them')
} finally {
db.recentEvents = original
}
})
test('asking only for kinds you may not see answers with nothing, and queries nothing', async () => {
withCore()
const db = require('../model/events/events.db')
const model = require('../model/events/events.model')
const original = db.recentEvents
let called = false
db.recentEvents = async () => {
called = true
return []
}
try {
const rows = await model.recent({ serverId: 'main', kind: 'player.banned,player.approved' })
assert.deepEqual(rows, [])
assert.equal(called, false, 'a query with no permitted kinds must not reach the database')
} finally {
db.recentEvents = original
}
})
test('a row whose stored frame will not parse still answers with its envelope', async () => {
withCore()
const db = require('../model/events/events.db')
const model = require('../model/events/events.model')
const original = db.recentEvents
db.recentEvents = async () => [
{ id: 7, kind: 'player.death', t: 12, wipeId: 'w-1', steamId: 'p1', raw: '{not json' },
]
try {
const [row] = await model.recent({ serverId: 'main' })
// One unreadable row must not fail a whole page. What is known is still
// reported; the body is empty rather than absent.
assert.equal(row.id, 7)
assert.equal(row.kind, 'player.death')
assert.deepEqual(row.frame, {})
} finally {
db.recentEvents = original
}
})
test('the leaderboard answers numbers, never nulls', async () => {
withCore()
const db = require('../model/events/events.db')
const model = require('../model/events/events.model')
const original = db.leaderboard
// SUM() over no rows is NULL in SQL, and a JOIN with no player row gives a
// null name. A page that has to defend against both is a page with the
// defence in three places.
db.leaderboard = async () => [
{ steamId: 'p1', name: null, kills: null, deaths: '3', npcKills: null, playtimeSec: null },
]
try {
const [row] = await model.leaderboard({ serverId: 'main' })
assert.equal(row.kills, 0)
assert.equal(row.deaths, 3)
assert.equal(row.npcKills, 0)
assert.equal(row.playtimeSec, 0)
assert.equal(row.name, null)
} finally {
db.leaderboard = original
}
})

View File

@@ -0,0 +1,177 @@
// The frozen manifest's derivation, checked.
//
// `scripts/frozenManifest.js` runs in one place — a CI job with a whole core
// checked out beside it — so it is the least-exercised piece of machinery in this
// repo, and it is the piece that decides whether the URLs this module claims are
// the URLs it serves (MODULE_API.md §5.3). Its three answers are pure functions of
// two manifests and a fragment, so all three are asked here, with fixtures rather
// than a clone.
//
// What is deliberately NOT asserted here: the numbers. `routes.manifest.json`'s
// routes are proved by the job that generates them from a real core, and a copy of
// that count in this file would only ever be a second thing to update.
const test = require('node:test')
const assert = require('node:assert')
const fs = require('node:fs')
const path = require('node:path')
const { diffManifests, coverage, MANIFEST, FRAGMENT } = require('../scripts/frozenManifest')
const manifest = (public_ = [], internal = []) => ({ public: public_, internal })
const get = (p) => ({ method: 'GET', path: p })
test("the module's routes are the ones a core gains by loading it", () => {
const before = manifest([get('/api/v1/public/settings')])
const after = manifest([get('/api/v1/public/settings'), get('/api/v1/public/rust/servers')])
const { added, removed } = diffManifests(before, after)
assert.deepStrictEqual(removed, [])
assert.deepStrictEqual(added, [{ method: 'GET', path: '/api/v1/public/rust/servers', tier: 'public' }])
})
test('a route core loses to the module is reported, not quietly absorbed', () => {
// The failure this exists for, and the one phase 1 could only check by reading:
// core mounts several routes at the TIER ROOT (/status, /version) that the
// loader's collision probe cannot see, so a module whose mount displaced one
// would not show up as an addition — the URL is unchanged — and a check that
// only looked at what appeared would call it clean.
const before = manifest([get('/api/v1/public/settings'), get('/api/v1/public/status')])
const after = manifest([get('/api/v1/public/settings')])
const { removed } = diffManifests(before, after)
assert.deepStrictEqual(removed, ['public GET /api/v1/public/status'])
})
test('a route whose METHOD changed counts as removed and added', () => {
const { added, removed } = diffManifests(
manifest([{ method: 'POST', path: '/api/v1/admin/thing' }]),
manifest([{ method: 'PUT', path: '/api/v1/admin/thing' }]),
)
assert.deepStrictEqual(removed, ['public POST /api/v1/admin/thing'])
assert.strictEqual(added.length, 1)
})
test('the internal app is diffed too, and keeps its own tier', () => {
const { added } = diffManifests(
manifest([], [get('/internal/health')]),
manifest([], [get('/internal/health'), get('/internal/rust/thing')]),
)
assert.deepStrictEqual(added, [{ method: 'GET', path: '/internal/rust/thing', tier: 'internal' }])
})
test('added routes are sorted, so the committed file does not churn on traversal order', () => {
const { added } = diffManifests(
manifest([]),
manifest([get('/b'), get('/a'), { method: 'POST', path: '/a' }]),
)
assert.deepStrictEqual(
added.map((r) => `${r.method} ${r.path}`),
['GET /a', 'GET /b', 'POST /a'],
)
})
// ── coverage: the route ⇄ fragment agreement ────────────────────────────────
const fragment = (paths) => ({ paths })
test('a served route with no documented operation is named', () => {
const { undocumented, unserved } = coverage([get('/api/v1/public/rust/servers')], fragment({}))
assert.deepStrictEqual(undocumented, ['GET /api/v1/public/rust/servers'])
assert.deepStrictEqual(unserved, [])
})
test('a documented operation nobody serves is named too', () => {
// The direction core's own spec has no check for, which is how it accumulated
// orphan tags and schemas describing routes that had moved out of it. A
// documented URL nobody serves is a client following the docs into a 404.
const { undocumented, unserved } = coverage([], fragment({ '/api/v1/public/rust/gone': { get: {} } }))
assert.deepStrictEqual(undocumented, [])
assert.deepStrictEqual(unserved, ['GET /api/v1/public/rust/gone'])
})
test('express :params and OpenAPI {params} are the same route', () => {
const { undocumented, unserved } = coverage(
[{ method: 'DELETE', path: '/api/v1/admin/rust/servers/:id' }],
fragment({ '/api/v1/admin/rust/servers/{id}': { delete: {} } }),
)
assert.deepStrictEqual(undocumented, [])
assert.deepStrictEqual(unserved, [])
})
test('methods are matched, not just paths', () => {
const { undocumented, unserved } = coverage(
[{ method: 'POST', path: '/api/v1/admin/rust/servers/:id/test' }],
fragment({ '/api/v1/admin/rust/servers/{id}/test': { get: {} } }),
)
assert.deepStrictEqual(undocumented, ['POST /api/v1/admin/rust/servers/{id}/test'])
assert.deepStrictEqual(unserved, ['GET /api/v1/admin/rust/servers/{id}/test'])
})
// ── the committed artifacts, against each other ─────────────────────────────
//
// These two files are generated together by a job that has a real core; here
// there is no core, so what can still be asked is whether they agree with each
// other. If they do not, one of them was committed without the other.
test('every route in the committed manifest has a committed operation', () => {
const { routes } = JSON.parse(fs.readFileSync(MANIFEST, 'utf8'))
const spec = JSON.parse(fs.readFileSync(FRAGMENT, 'utf8'))
const { undocumented, unserved } = coverage(routes, spec)
assert.deepStrictEqual(undocumented, [], 'routes.manifest.json lists routes swagger-fragment.json does not document')
assert.deepStrictEqual(unserved, [], 'swagger-fragment.json documents operations routes.manifest.json does not list')
})
test('the fragment carries only the three sections §6.1a allows', () => {
const spec = JSON.parse(fs.readFileSync(FRAGMENT, 'utf8'))
assert.deepStrictEqual(Object.keys(spec).sort(), ['components', 'paths', 'tags'])
assert.deepStrictEqual(Object.keys(spec.components), ['schemas'])
})
test("the fragment defines only namespaced schemas, and redefines none of core's", () => {
const spec = JSON.parse(fs.readFileSync(FRAGMENT, 'utf8'))
for (const name of Object.keys(spec.components.schemas)) {
assert.match(name, /^Rust[A-Z]/, `${name} is not namespaced — core wins the collision and drops it (§6.1a)`)
}
// Anything this fragment REFERENCES and does not define has to be one of
// core's shared schemas, which resolve in the merged document — that is the
// whole point of a fragment. A typo'd $ref is otherwise invisible until a
// reader opens /api/docs.json and finds a dangling pointer.
const refs = JSON.stringify(spec.paths).match(/#[/]components[/]schemas[/]([A-Za-z0-9_]+)/g) || []
const shared = ['Error', 'ValidationError']
for (const name of new Set(refs.map((r) => r.split('/').pop()))) {
const resolvable = Object.hasOwn(spec.components.schemas, name) || shared.includes(name)
assert.ok(resolvable, `$ref to ${name} resolves to nothing — not defined here, not one of core's shared schemas`)
}
})
test('every path in the fragment is fully qualified', () => {
const spec = JSON.parse(fs.readFileSync(FRAGMENT, 'utf8'))
for (const p of Object.keys(spec.paths)) {
// §6.1a: core merges the fragment verbatim and never re-derives a prefix, so
// a router-relative path here is a path nothing serves.
assert.match(p, /^\/api\/v1\/(public|admin|player)\//, `${p} is not a fully-qualified URL`)
assert.doesNotMatch(p, /\/$/, `${p} has a trailing slash — no client calls that URL`)
}
})
test("the manifest and the module's declared mounts agree", () => {
const { routes } = JSON.parse(fs.readFileSync(MANIFEST, 'utf8'))
const { mounts } = JSON.parse(fs.readFileSync(path.join(__dirname, '..', '..', 'module.json'), 'utf8'))
const declared = []
for (const [tier, prefixes] of Object.entries(mounts)) {
for (const prefix of prefixes) declared.push(`/api/v1/${tier}${prefix}/`)
}
// Every route this module serves is under a prefix it declared. There is no
// exception here yet, and that is the point of asserting it now: phase 6 adds
// the `admin.users.detail` extension slot, whose routes live under core's
// `/api/v1/admin/users/` rather than under any mount of ours (§2.4). When that
// arrives this test must grow the exception deliberately, rather than a route
// outside every declared mount arriving unnoticed.
for (const route of routes) {
const under = declared.some((d) => route.path.startsWith(d))
assert.ok(under, `${route.method} ${route.path} is served from outside every mount module.json declares`)
}
})

329
server/test/ingest.test.js Normal file
View File

@@ -0,0 +1,329 @@
// ── The ingest ────────────────────────────────────────────────────────────
//
// Every test here is about one of three things, and all three are mistakes that
// look correct in review:
//
// • **who gets credited.** A suicide must not credit the victim with a kill.
// That single line would produce a leaderboard topped by whoever died most,
// and it would look plausible for a whole wipe.
// • **the cursor's ordering.** It advances AFTER the batch, never before, so a
// crash re-reads rather than skips. Skipping is silent and permanent.
// • **absent is not zero.** A session whose start was never seen contributes
// no playtime rather than zero playtime.
//
// The database is a recorder. Asserting the SQL exactly would be a test of the
// SQL's punctuation, so each case asserts the *statement shape* and the values —
// which table was written, and with what.
const test = require('node:test')
const assert = require('node:assert')
const { fakeCtx } = require('./_fakes')
/** Installs a core whose `db.query` records every statement. */
function withRecorder() {
const statements = []
const ctx = fakeCtx({
db: {
query: (sql, params = []) => {
statements.push({ sql, params })
return Promise.resolve([])
},
pool: {},
},
})
require('../core')._reset()
require('../core').init(ctx)
return {
statements,
/** Every statement that touched a table, with its parameters. */
touching(table) {
return statements.filter((s) => s.sql.includes(table))
},
}
}
const frame = (over = {}) => ({
type: 'event',
t: 1789560564452,
serverId: 'main',
wipeId: 'w-20260915T195817Z',
...over,
})
const item = (kind, over = {}) => ({ id: 1, t: 1, kind, frame: frame({ kind, ...over }) })
test('every frame is stored, whether or not this build understands it', async () => {
const rec = withRecorder()
const { apply } = require('../ingest')
await apply('main', item('player.death', { steamId: '76561198000000001' }))
await apply('main', item('something.from.protocol.9'))
const stored = rec.touching('rust_events')
assert.equal(stored.length, 2, 'an unrecognised kind must still be stored')
// The one copy of an event a later version will know how to read is the one
// this version chose not to throw away.
assert.ok(stored[1].params.includes('something.from.protocol.9'))
})
test('a wipe exists because a frame mentioned it', async () => {
const rec = withRecorder()
const { apply } = require('../ingest')
await apply('main', item('player.chat', { steamId: '1', message: 'hello' }))
const wipes = rec.touching('rust_wipes')
assert.equal(wipes.length, 1)
assert.deepEqual(wipes[0].params.slice(0, 2), ['main', 'w-20260915T195817Z'])
})
test('a kill credits the attacker and a death the victim', async () => {
const rec = withRecorder()
const { apply } = require('../ingest')
await apply(
'main',
item('player.death', {
steamId: 'victim',
attackerType: 'player',
attackerId: 'killer',
attackerName: 'Killer',
}),
)
const stats = rec.touching('rust_player_wipe_stats')
assert.equal(stats.length, 2, 'one row for the victim, one for the attacker')
// The parameter order is (server, wipe, steam, kills, deaths, suicides, ...).
const victim = stats.find((s) => s.params[2] === 'victim')
const killer = stats.find((s) => s.params[2] === 'killer')
assert.ok(victim && killer)
assert.equal(victim.params[3], 0, 'the victim scored no kill')
assert.equal(victim.params[4], 1, 'the victim died once')
assert.equal(killer.params[3], 1, 'the attacker scored one kill')
assert.equal(killer.params[4], 0, 'the attacker did not die')
})
test('a suicide is a death and a suicide, and credits nobody with a kill', async () => {
const rec = withRecorder()
const { apply } = require('../ingest')
await apply('main', item('player.death', { steamId: 'victim', attackerType: 'self' }))
const stats = rec.touching('rust_player_wipe_stats')
assert.equal(stats.length, 1, 'nobody is credited with the kill')
assert.equal(stats[0].params[4], 1, 'it is still a death')
assert.equal(stats[0].params[5], 1, 'and a suicide')
assert.equal(stats[0].params[3], 0)
})
test('an environment or NPC death credits no attacker', async () => {
for (const attackerType of ['environment', 'npc']) {
const rec = withRecorder()
const { apply } = require('../ingest')
await apply('main', item('player.death', { steamId: 'victim', attackerType }))
const stats = rec.touching('rust_player_wipe_stats')
assert.equal(stats.length, 1, `${attackerType} must credit nobody`)
assert.equal(stats[0].params[4], 1)
}
})
test('an absent session length adds no playtime and no session', async () => {
const rec = withRecorder()
const { apply } = require('../ingest')
// A player who was already on the server when the plugin loaded: the plugin
// omits `sessionSec` rather than sending 0, and the difference has to survive
// all the way to the column. Adding a zero would record a session of no
// length, which is a different claim from recording no session.
await apply('main', item('player.disconnected', { steamId: 'p1', reason: 'quit' }))
const stats = rec.touching('rust_player_wipe_stats')
assert.equal(stats[0].params[8], 0, 'no session counted')
assert.equal(stats[0].params[9], 0, 'no playtime added')
const rec2 = withRecorder()
await require('../ingest').apply(
'main',
item('player.disconnected', { steamId: 'p1', sessionSec: 600 }),
)
const counted = rec2.touching('rust_player_wipe_stats')
assert.equal(counted[0].params[8], 1)
assert.equal(counted[0].params[9], 600)
})
test('a tally is added per resource, as a delta', async () => {
const rec = withRecorder()
const { apply } = require('../ingest')
await apply(
'main',
item('player.tally', {
steamId: 'p1',
gathered: { wood: 1200, stones: 300 },
npcKills: 3,
structures: 2,
}),
)
const gathered = rec.touching('rust_gather_totals')
assert.equal(gathered.length, 2)
assert.deepEqual(
gathered.map((g) => [g.params[3], g.params[4]]),
[
['wood', 1200],
['stones', 300],
],
)
const stats = rec.touching('rust_player_wipe_stats')
assert.equal(stats[0].params[6], 3, 'npc kills')
assert.equal(stats[0].params[7], 2, 'structures')
// `amount = amount + VALUES(amount)` is what makes a delta correct. A running
// total on the wire would double every number here, slowly, looking right.
assert.match(gathered[0].sql, /amount = amount \+ VALUES\(amount\)/)
})
test('a new server starts at the feed tail, not at the beginning of history', async () => {
withRecorder()
const sidecar = require('../sidecarClient')
const db = require('../model/events/events.db')
const ingest = require('../ingest')
const originalTail = sidecar.feedTail
const originalCursor = db.getCursor
const originalSet = db.setCursor
const written = []
db.getCursor = async () => null
db.setCursor = async (...args) => written.push(args)
sidecar.feedTail = async () => ({ ok: true, status: 'ok', data: { lastId: 4021, items: [] } })
try {
const applied = await ingest.ingestServer({ id: 'main' })
assert.equal(applied, 0, 'nothing is replayed')
assert.deepEqual(written, [['main', 4021, 0]], 'the cursor starts at the end')
} finally {
sidecar.feedTail = originalTail
db.getCursor = originalCursor
db.setCursor = originalSet
}
})
test('an unreachable sidecar writes no cursor at all', async () => {
withRecorder()
const sidecar = require('../sidecarClient')
const db = require('../model/events/events.db')
const ingest = require('../ingest')
const originalTail = sidecar.feedTail
const originalCursor = db.getCursor
const originalSet = db.setCursor
const written = []
db.getCursor = async () => null
db.setCursor = async (...args) => written.push(args)
sidecar.feedTail = async () => ({ ok: false, status: 'transport-error', data: null })
try {
await ingest.ingestServer({ id: 'main' })
// A cursor of 0 written here would replay the sidecar's whole retained
// history the moment it came back — which is the failure that looks like a
// working catch-up until somebody reads the leaderboard.
assert.deepEqual(written, [])
} finally {
sidecar.feedTail = originalTail
db.getCursor = originalCursor
db.setCursor = originalSet
}
})
test('the cursor advances after the batch, and one bad event does not wedge it', async () => {
withRecorder()
const sidecar = require('../sidecarClient')
const db = require('../model/events/events.db')
const ingest = require('../ingest')
const originals = {
feed: sidecar.feed,
getCursor: db.getCursor,
setCursor: db.setCursor,
insertEvent: db.insertEvent,
}
const order = []
db.getCursor = async () => ({ lastEventId: 10 })
db.setCursor = async (_id, last) => order.push(`cursor:${last}`)
db.insertEvent = async (row) => {
order.push(`event:${row.kind}`)
if (row.kind === 'player.chat') throw new Error('malformed')
}
sidecar.feed = async (_server, since) =>
since === 10
? {
ok: true,
status: 'ok',
data: {
items: [item('player.chat'), item('player.connected', { steamId: 'p1' })],
lastId: 12,
more: false,
},
}
: { ok: true, status: 'ok', data: { items: [], lastId: since, more: false } }
try {
const applied = await ingest.ingestServer({ id: 'main' })
// The bad row is logged and skipped; the good one still counts.
assert.equal(applied, 1)
// And the ordering the whole design rests on: every event is written before
// the cursor moves past it.
assert.deepEqual(order, ['event:player.chat', 'event:player.connected', 'cursor:12'])
} finally {
Object.assign(db, {
getCursor: originals.getCursor,
setCursor: originals.setCursor,
insertEvent: originals.insertEvent,
})
sidecar.feed = originals.feed
}
})
test('a board replaces presence rather than appending to it', async () => {
const rec = withRecorder()
const ingest = require('../ingest')
await ingest.applyBoards('main', {
'players.online': {
kind: 'players.online',
type: 'snapshot',
count: 1,
players: [{ steamId: 'p1', name: 'One', sleeping: false }],
},
})
const presence = rec.touching('rust_presence')
// The DELETE is what makes it a board. Without it a player who left stays
// online for ever, which is the exact drift the board exists to correct.
assert.match(presence[0].sql, /^DELETE FROM rust_presence/)
assert.match(presence[1].sql, /INSERT INTO rust_presence/)
})

View File

@@ -195,6 +195,172 @@
}
}
}
},
"/api/v1/public/rust/servers/{id}/events": {
"get": {
"tags": [
"Public · Rust"
],
"summary": "Recent events on one Rust server",
"description": "The killfeed and everything else public that happened on a server, newest first. Narrow with `kind` (comma-separated) and `wipe`. Only publicly classified kinds are ever returned — moderation events, login attempts and anything carrying an IP address are stored but never served here.",
"parameters": [
{
"name": "id",
"in": "path",
"required": true,
"schema": {
"type": "string"
},
"description": "The servers slug"
},
{
"name": "kind",
"in": "query",
"required": false,
"description": "One kind, or several comma-separated",
"schema": {
"type": "string"
}
},
{
"name": "wipe",
"in": "query",
"required": false,
"description": "Restrict to one wipe id",
"schema": {
"type": "string"
}
},
{
"name": "limit",
"in": "query",
"required": false,
"description": "Rows to return, capped at 200",
"schema": {
"type": "integer"
}
}
],
"responses": {
"200": {
"description": "Recent events, newest first"
},
"500": {
"description": "Internal Server Error"
}
}
}
},
"/api/v1/public/rust/servers/{id}/leaderboard": {
"get": {
"tags": [
"Public · Rust"
],
"summary": "The leaderboard for one Rust server",
"description": "Per-wipe when `wipe` is given, all-time otherwise. All-time is the per-wipe rows summed rather than a second set of counters, so a wipe splits a players history without ending it.",
"parameters": [
{
"name": "id",
"in": "path",
"required": true,
"schema": {
"type": "string"
},
"description": "The servers slug"
},
{
"name": "wipe",
"in": "query",
"required": false,
"description": "Restrict to one wipe id",
"schema": {
"type": "string"
}
},
{
"name": "sort",
"in": "query",
"required": false,
"description": "kills, deaths, npcKills or playtime",
"schema": {
"type": "string"
}
},
{
"name": "limit",
"in": "query",
"required": false,
"description": "Rows to return, capped at 200",
"schema": {
"type": "integer"
}
}
],
"responses": {
"200": {
"description": "The leaderboard"
},
"500": {
"description": "Internal Server Error"
}
}
}
},
"/api/v1/public/rust/servers/{id}/online": {
"get": {
"tags": [
"Public · Rust"
],
"summary": "Who is on one Rust server right now",
"description": "Read from the presence board the bridge re-sends on every connect and every minute, rather than counted from connect and disconnect events — so it is correct even after the website has missed one.",
"parameters": [
{
"name": "id",
"in": "path",
"required": true,
"schema": {
"type": "string"
},
"description": "The servers slug"
}
],
"responses": {
"200": {
"description": "Who is online"
},
"500": {
"description": "Internal Server Error"
}
}
}
},
"/api/v1/public/rust/servers/{id}/wipes": {
"get": {
"tags": [
"Public · Rust"
],
"summary": "Every wipe this server has had",
"description": "Newest first. A wipe id is derived by the bridge plugin from the saves creation time and stamped on every frame, so it is the same id the events and the leaderboard are filtered by.",
"parameters": [
{
"name": "id",
"in": "path",
"required": true,
"schema": {
"type": "string"
},
"description": "The servers slug"
}
],
"responses": {
"200": {
"description": "The wipes"
},
"500": {
"description": "Internal Server Error"
}
}
}
}
},
"tags": [