feat(ci): packaging, release and the frozen manifest #2
229
.gitea/workflows/pr-checks.yml
Normal file
229
.gitea/workflows/pr-checks.yml
Normal file
@@ -0,0 +1,229 @@
|
||||
# Gate every pull request into `main` on a fast, DB-free check suite, so a broken
|
||||
# build or a failing test can't reach the branch that gets released.
|
||||
#
|
||||
# Mirrors RunicGateway/website's pr-checks.yml — this module is two npm packages
|
||||
# shaped like that repo's `server/` and `client/`, and it is loaded into that
|
||||
# repo's process, so it is checked the same way with the same Node version.
|
||||
#
|
||||
# Phase 1 built all of these checks and ran them BY HAND. That is the gap this
|
||||
# file closes: a guard nothing invokes is a guard whose state nobody knows.
|
||||
#
|
||||
# ── What each job is really asking ───────────────────────────────────────────
|
||||
#
|
||||
# The tests are the ordinary half. The `check:*` scripts are the interesting one,
|
||||
# because they are the acceptance criteria of the module contract itself
|
||||
# (docs/website/MODULE_API.md Part 5) rather than of this module's behaviour:
|
||||
#
|
||||
# • `server: check:imports` — no relative path escapes the module root, and no
|
||||
# shipped file resolves a bare specifier. A module that reaches into core's
|
||||
# tree works right up until core moves a file, and the whole boundary is
|
||||
# worth exactly as much as this check is (§5.1).
|
||||
#
|
||||
# • `server: check:bundle` — the release ships everything the entry point can
|
||||
# reach, and still declares no runtime dependency. Every other job here runs
|
||||
# against the whole repo, but a release is a SUBSET of it (release.yml
|
||||
# assembles from the include list in `ci/bundle.json`), and nothing else
|
||||
# compares the two. Module-uo's v1.0.0 is the cautionary tale: `server/commands/`
|
||||
# arrived in a cutover, the include list did not learn about it, and the
|
||||
# module installed and then died at the register stage on the operator's box
|
||||
# with "Cannot find module './commands/guild.command'". Green in CI, broken
|
||||
# there — because the subset only exists in the release.
|
||||
#
|
||||
# • `client: check:externals` — the BUILT chunk has no bare imports left. That
|
||||
# failure is invisible in source: `import { useState } from 'react'` is
|
||||
# correct in every file, and whether it becomes core's React or a bare
|
||||
# specifier no browser can resolve is decided by vite.config.js. It has to be
|
||||
# asked of the artifact, so it runs after the build. (The other half — a
|
||||
# shared dependency being BUNDLED — fails the build itself, from a
|
||||
# resolution-time guard inside vite.config.js.)
|
||||
#
|
||||
# • `server: check:swagger` — `swagger-fragment.json` describes the routes this
|
||||
# module registers, today. Core has no way to generate it: core is a prebuilt
|
||||
# image, this module arrives on a volume afterwards, and it mounts through a
|
||||
# call no static parser can follow. So the fragment core merges into
|
||||
# `/api/docs.json` is whatever this repo committed, and a stale one documents
|
||||
# a URL surface that does not exist (§2.8).
|
||||
#
|
||||
# • `frozen-manifest` — the job with the interesting shape. It clones CORE at
|
||||
# the ref pinned in `ci/core-ref.json`, generates its route manifest twice
|
||||
# (without this module, then with) and takes the difference. That difference
|
||||
# is what this module serves, and it is checked three ways: it must match the
|
||||
# committed `routes.manifest.json`, it must not have REMOVED or changed one of
|
||||
# core's own routes, and every route in it must have an operation in
|
||||
# `swagger-fragment.json` — the per-module form of core's rule that a route
|
||||
# which isn't in the spec doesn't ship (§5.3, §2.8).
|
||||
#
|
||||
# Nothing else can ask those questions. Every other check here runs against
|
||||
# this repo alone, where a mount prefix is a string in `server/index.js` and a
|
||||
# documented path is a string in a JSON file; whether they name the same URL
|
||||
# is a fact about a running core, and this is the only job that has one. It is
|
||||
# also the only thing that can see the blind spot phase 1 had to check by
|
||||
# reading: core answers several public routes mounted at the TIER ROOT rather
|
||||
# than under a prefix (`/status`, `/version`), which the loader's own collision
|
||||
# probe cannot find, so `/rust` being free is now asserted by a core.
|
||||
#
|
||||
# Enforcement (one-time, in the Gitea UI):
|
||||
# Repository Settings → Branches → Branch Protection (rule for `main`)
|
||||
# • Enable Status Check
|
||||
# • Status check patterns: PR Checks / *
|
||||
# Note: Gitea only lists a context in its dropdown after it has reported once,
|
||||
# so let this workflow run on one PR first. The `PR Checks / *` glob matches
|
||||
# without needing the dropdown, and keeps matching as jobs are added.
|
||||
#
|
||||
# Runner: the shared self-hosted `ubuntu-latest` runner. These jobs need only
|
||||
# Node — no Docker socket, no database.
|
||||
#
|
||||
# Scope note: `edge` is gated as well as `main`, though this repo has no `edge`
|
||||
# branch yet. Multi-phase work lands there first everywhere else in this project,
|
||||
# and gating only the `main` hop would run these checks for the first time at the
|
||||
# cutover — the one moment a red build is most expensive to discover. Naming the
|
||||
# branch before it exists costs nothing; an Android workstream that landed nine
|
||||
# PRs on an ungated `edge` is why it is here from the start.
|
||||
|
||||
name: PR Checks
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: [main, edge]
|
||||
|
||||
# A newer push to the same PR cancels the in-flight run.
|
||||
concurrency:
|
||||
group: pr-checks-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
# npm's own retry, turned up. The shared runner reads ETIMEDOUT from the registry
|
||||
# often enough to matter, and a red X that means "the network hiccuped" costs a
|
||||
# reviewer more than it costs the runner to retry, and teaches everyone to re-run
|
||||
# rather than read a failure.
|
||||
env:
|
||||
NPM_CONFIG_FETCH_RETRIES: 5
|
||||
NPM_CONFIG_FETCH_RETRY_MINTIMEOUT: 20000
|
||||
NPM_CONFIG_FETCH_RETRY_MAXTIMEOUT: 120000
|
||||
|
||||
jobs:
|
||||
server-tests:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 20
|
||||
cache: npm
|
||||
cache-dependency-path: server/package-lock.json
|
||||
|
||||
# `npm ci` rather than `npm install`: it also proves the lockfile is in
|
||||
# sync with package.json instead of silently updating it.
|
||||
- name: Install server deps
|
||||
run: npm ci --prefix server
|
||||
|
||||
- name: Run server tests
|
||||
run: npm test --prefix server
|
||||
|
||||
- name: Check the module boundary (MODULE_API.md §5.1)
|
||||
run: npm run check:imports --prefix server
|
||||
|
||||
- name: Check the release ships what the module requires
|
||||
run: npm run check:bundle --prefix server
|
||||
|
||||
- name: Check the OpenAPI fragment is current (MODULE_API.md §2.8)
|
||||
run: npm run check:swagger --prefix server
|
||||
|
||||
client-build:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 20
|
||||
cache: npm
|
||||
cache-dependency-path: client/package-lock.json
|
||||
|
||||
- name: Install client deps
|
||||
run: npm ci --prefix client
|
||||
|
||||
# The build comes FIRST, and that ordering is load-bearing. Two of the
|
||||
# client tests read `dist/entry.js` — the chunk's externals, and what it
|
||||
# registers when imported against a fake `window.__rg` — and both skip when
|
||||
# there is no build. Run the other way round they skip silently in CI, which
|
||||
# is the worst of both: green, and not asking the question.
|
||||
- name: Build the client chunk
|
||||
run: npm run build --prefix client
|
||||
|
||||
- name: Run client tests
|
||||
run: npm test --prefix client
|
||||
|
||||
- name: Check the built chunk's externals (MODULE_API.md §3.6)
|
||||
run: npm run check:externals --prefix client
|
||||
|
||||
# ── The URLs this module actually serves ──────────────────────────────────
|
||||
#
|
||||
# Everything above proves the module against itself. This proves it against a
|
||||
# real core: the one place where "the prefix I register" and "the path I
|
||||
# document" are the same fact rather than two strings that ought to agree.
|
||||
#
|
||||
# The module is COPIED into the core checkout, never symlinked — core's loader
|
||||
# filters its scan with `entry.isDirectory()`, which reports a link as a link
|
||||
# and skips it silently, so a symlinked module produces a manifest with no
|
||||
# module routes in it and a diff that looks like the module registering nothing.
|
||||
frozen-manifest:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
path: module
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 20
|
||||
|
||||
# Anonymous HTTPS, and a full clone rather than a shallow one: the pin is a
|
||||
# commit sha, and `--depth 1` can only fetch a branch tip.
|
||||
- name: Clone core at the pinned ref (MODULE_API.md §5.3)
|
||||
run: |
|
||||
REPO=$(node -p "require('./module/ci/core-ref.json').repo")
|
||||
REF=$(node -p "require('./module/ci/core-ref.json').ref")
|
||||
echo "core: $REPO @ $REF"
|
||||
git clone --quiet "$REPO" core
|
||||
git -C core checkout --quiet "$REF"
|
||||
|
||||
- name: Install core's server deps
|
||||
run: npm ci --prefix core/server
|
||||
|
||||
# Core alone. `--check` first, so a pin that no longer regenerates its own
|
||||
# committed manifest fails HERE, naming the pin, instead of showing up below
|
||||
# as this module having removed a route it never touched.
|
||||
- name: Generate core's manifest without this module
|
||||
run: |
|
||||
npm run routes:manifest --prefix core/server -- --check
|
||||
cp core/server/routes.manifest.json before.json
|
||||
|
||||
# The chunk has to exist before the loader will accept the module at all —
|
||||
# `client.entry` is validated during the manifest step of the scan, and a
|
||||
# missing one is a load failure, not a warning.
|
||||
- name: Build the client chunk
|
||||
run: |
|
||||
npm ci --prefix module/client
|
||||
npm run build --prefix module/client
|
||||
|
||||
# No `npm ci` on the installed copy, because the shipped half declares no
|
||||
# runtime dependencies and the release packs no `node_modules` (org lead,
|
||||
# phase 2). `check:bundle` in the job above is what keeps that true; if it
|
||||
# ever stops being true, this step and release.yml both grow an install.
|
||||
- name: Install the module into core
|
||||
run: |
|
||||
mkdir -p core/modules/rust
|
||||
tar -C module --exclude=.git --exclude=node_modules -cf - . | tar -C core/modules/rust -xf -
|
||||
|
||||
- name: Generate core's manifest with this module
|
||||
run: |
|
||||
npm run routes:manifest --prefix core/server
|
||||
cp core/server/routes.manifest.json after.json
|
||||
|
||||
- name: Check the frozen manifest and the fragment's coverage
|
||||
working-directory: module
|
||||
run: node server/scripts/frozenManifest.js --before ../before.json --after ../after.json --check
|
||||
442
.gitea/workflows/release.yml
Normal file
442
.gitea/workflows/release.yml
Normal file
@@ -0,0 +1,442 @@
|
||||
# Build and publish the installable bundle: `module-rust-<version>.tar.gz` plus
|
||||
# the manifest carrying its sha256 (docs/website/MODULE_SYSTEM.md §2.3, §2.5).
|
||||
#
|
||||
# ── What a release IS here ──────────────────────────────────────────────────
|
||||
#
|
||||
# **An operator never builds anything** (MODULE_SYSTEM.md §1.14 — the constraint
|
||||
# the whole module system is shaped around). So a release is not source: it is the
|
||||
# directory core's loader expects to find at `modules/rust/`, already assembled —
|
||||
# the prebuilt client chunk, the schema fragment and the OpenAPI fragment — packed
|
||||
# as it will be unpacked. Core's admin install downloads the tarball, verifies it
|
||||
# against the `sha256` in the manifest, and unpacks it onto the volume. Nothing
|
||||
# runs `npm` on the way.
|
||||
#
|
||||
# **And nothing is installed into the bundle either**, which is where this repo
|
||||
# differs from Module-uo: the shipped half declares no runtime dependencies, so
|
||||
# there is no `npm ci --omit=dev` and no `server/node_modules` in the tarball (org
|
||||
# lead, phase 2). That is a decision worth being loud about rather than a detail —
|
||||
# `server/scripts/checkBundle.js` fails the PR that adds a dependency without also
|
||||
# teaching this file to install and pack it, because a bundle that declares an
|
||||
# import it does not carry fails the same way a missing directory does.
|
||||
#
|
||||
# ── The version is DERIVED, and the declaration is a floor ──────────────────
|
||||
#
|
||||
# The engine `link`, `installer` and `Module-uo` already run (MODULE_SYSTEM
|
||||
# §2.7.1, decision 19 as amended):
|
||||
#
|
||||
# feat!: / BREAKING CHANGE -> major feat: -> minor fix|perf: -> patch
|
||||
# nothing releasable -> no release is cut
|
||||
# (first ever run, no tag) -> releases what module.json declares
|
||||
#
|
||||
# Module-uo learned this the expensive way: it released only when a merge left
|
||||
# `module.json` at a version with no release yet — the version DECLARED, never
|
||||
# computed — and between 2026-08-12 and 2026-08-19 that cost it *every* bundle,
|
||||
# because nine phases of work landed without anyone touching that line.
|
||||
#
|
||||
# **The declared version is kept as a floor, not deleted.** If `module.json` names
|
||||
# a version above the newest tag, that version releases. Raising it by hand is how
|
||||
# you say "this one is a minor, whatever the subjects imply", and it is the natural
|
||||
# place to move when a `coreApi` bump forces the question.
|
||||
#
|
||||
# The number that ships is therefore the TAG, and CI writes it into the
|
||||
# `module.json` inside the bundle at assembly time. The committed `module.json` is
|
||||
# a floor and a starting point, not a record of the last release — a release engine
|
||||
# that has to commit a bump back to `main` stops working the day someone protects
|
||||
# the branch, and this one is protected.
|
||||
#
|
||||
# ── The backdoor ────────────────────────────────────────────────────────────
|
||||
#
|
||||
# `workflow_dispatch` publishes on demand, for the case the rules above cannot
|
||||
# reach: `module.json` changed in a way worth shipping — a widened `coreApi`, a
|
||||
# new mount, a capability — with no releasable code behind it. Leave `version`
|
||||
# blank to bump the newest tag by `bump` (default `patch`), or name an exact
|
||||
# version to publish that. A dispatch releases even when nothing in the log is
|
||||
# releasable; that is the entire point of pressing the button.
|
||||
#
|
||||
# Re-running on a version that is already released is a no-op, so a rerun after an
|
||||
# unrelated failure is safe. A tag that exists with no release behind it is NOT a
|
||||
# no-op — see the recovery branch in the plan step. That state is not theoretical:
|
||||
# `servuo-plugins`' first release pushed its tag and then 401'd on the release API
|
||||
# because the secret was absent, and without the recovery branch the repo would
|
||||
# have been stuck there permanently.
|
||||
#
|
||||
# This workflow never writes to a branch. It tags and publishes, so `main` needs
|
||||
# no push exception.
|
||||
#
|
||||
# Prerequisites (Settings → Actions → Secrets on RunicGateway/Module-Rust):
|
||||
# REGISTRY_TOKEN — Gitea access token with `write:repository`, to push the tag
|
||||
# and create the release.
|
||||
|
||||
name: Release
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version:
|
||||
description: 'Exact version to publish (e.g. 0.1.1). Blank = bump the newest tag by the level below.'
|
||||
required: false
|
||||
default: ''
|
||||
bump:
|
||||
description: 'Bump level when version is blank: patch | minor | major'
|
||||
required: false
|
||||
default: 'patch'
|
||||
|
||||
concurrency:
|
||||
group: release-module-rust
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
GITEA_HOST: gitea.whitlocktech.com
|
||||
REPO: RunicGateway/Module-Rust
|
||||
|
||||
jobs:
|
||||
release:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
# Full history: the plan step reads every tag and every subject since the
|
||||
# newest one, and a shallow clone has neither.
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 20
|
||||
|
||||
- name: Plan the release (version + changelog)
|
||||
id: plan
|
||||
env:
|
||||
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
|
||||
EVENT: ${{ github.event_name }}
|
||||
IN_VERSION: ${{ github.event.inputs.version }}
|
||||
IN_BUMP: ${{ github.event.inputs.bump }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
mkdir -p dist
|
||||
git fetch --tags --force >/dev/null 2>&1 || true
|
||||
|
||||
DECLARED="$(node -p "require('./module.json').version")"
|
||||
LAST_TAG="$(git describe --tags --match 'v*' --abbrev=0 2>/dev/null || true)"
|
||||
CURRENT="${LAST_TAG#v}"
|
||||
RANGE="${LAST_TAG:+${LAST_TAG}..}HEAD"
|
||||
echo "module.json declares ${DECLARED}; newest tag is ${LAST_TAG:-<none>}"
|
||||
|
||||
SUBJECTS="$(git log --no-merges --format='%s' $RANGE || true)"
|
||||
BODIES="$(git log --no-merges --format='%B' $RANGE || true)"
|
||||
|
||||
BUMP=none
|
||||
if echo "$BODIES" | grep -qE 'BREAKING[ -]CHANGE' ; then BUMP=major; fi
|
||||
if echo "$SUBJECTS" | grep -qE '^[a-z]+(\([^)]+\))?!:' ; then BUMP=major; fi
|
||||
if [ "$BUMP" = none ] && echo "$SUBJECTS" | grep -qE '^feat(\([^)]+\))?:' ; then BUMP=minor; fi
|
||||
if [ "$BUMP" = none ] && echo "$SUBJECTS" | grep -qE '^(fix|perf)(\([^)]+\))?:' ; then BUMP=patch; fi
|
||||
|
||||
bump() { # <x.y.z> <major|minor|patch> -> bumped
|
||||
IFS=. read -r MA MI PA <<< "$1"
|
||||
case "$2" in
|
||||
major) echo "$((MA+1)).0.0" ;;
|
||||
minor) echo "${MA}.$((MI+1)).0" ;;
|
||||
patch) echo "${MA}.${MI}.$((PA+1))" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# `sort -V` orders version strings, so the higher of two is its last
|
||||
# line. Used rather than a hand-rolled field compare because 0.10.0 vs
|
||||
# 0.9.0 is exactly the comparison a string sort gets wrong.
|
||||
higher() { printf '%s\n%s\n' "$1" "$2" | sort -V | tail -1; }
|
||||
|
||||
rank() { case "$1" in major) echo 3 ;; minor) echo 2 ;; patch) echo 1 ;; *) echo 0 ;; esac; }
|
||||
bigger_bump() { if [ "$(rank "$1")" -ge "$(rank "$2")" ]; then echo "$1"; else echo "$2"; fi; }
|
||||
|
||||
VERSION=""
|
||||
if [ -n "${IN_VERSION:-}" ]; then
|
||||
# The backdoor's exact form. Deliberately unvalidated against the log:
|
||||
# a human typed it, and the already-released check below is the only
|
||||
# guard that matters.
|
||||
VERSION="${IN_VERSION}"
|
||||
echo "dispatch: publishing the requested version ${VERSION}"
|
||||
else
|
||||
LEVEL="$BUMP"
|
||||
# A dispatch with nothing releasable in the log still releases — that
|
||||
# is what the button is for. Where the log DOES say something, the
|
||||
# larger of the two wins rather than the input: pressing the button on
|
||||
# a log full of `feat:` without touching the dropdown would otherwise
|
||||
# publish its `patch` default over a minor's worth of work, and a
|
||||
# version that undersells its own contents cannot be taken back.
|
||||
if [ "${EVENT:-}" = workflow_dispatch ]; then
|
||||
LEVEL="$(bigger_bump "$LEVEL" "${IN_BUMP:-patch}")"
|
||||
if [ "$BUMP" = none ]; then
|
||||
echo "dispatch: nothing releasable in the log, bumping ${LEVEL} anyway"
|
||||
elif [ "$LEVEL" != "$BUMP" ]; then
|
||||
echo "dispatch: the log says ${BUMP}, the run asked for ${LEVEL} — taking ${LEVEL}"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -z "$CURRENT" ]; then
|
||||
VERSION="$DECLARED" # first ever release: ship what is declared
|
||||
elif [ "$LEVEL" != none ]; then
|
||||
VERSION="$(bump "$CURRENT" "$LEVEL")"
|
||||
fi
|
||||
|
||||
# The floor. A `module.json` above the newest tag releases at that
|
||||
# version even when the log says nothing and even when the log says
|
||||
# patch.
|
||||
if [ -n "$CURRENT" ] && [ "$DECLARED" != "$CURRENT" ] \
|
||||
&& [ "$(higher "$DECLARED" "$CURRENT")" = "$DECLARED" ]; then
|
||||
if [ -z "$VERSION" ] || [ "$(higher "$DECLARED" "$VERSION")" = "$DECLARED" ]; then
|
||||
echo "module.json declares ${DECLARED}, above both ${CURRENT} and the derived version — releasing that."
|
||||
VERSION="$DECLARED"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
RELEASE=true
|
||||
if [ -z "$VERSION" ]; then
|
||||
RELEASE=false
|
||||
VERSION="$CURRENT"
|
||||
echo "Nothing releasable since ${LAST_TAG} (no feat/fix/perf/breaking subject) — standing down."
|
||||
fi
|
||||
|
||||
# An existing tag is NOT automatically "nothing to do". A tag with no
|
||||
# release behind it means a previous run tagged and then died before
|
||||
# publishing — which is what happened on servuo-plugins' first release,
|
||||
# where absent secrets took the release API call to 401 after the tag had
|
||||
# already been pushed. Standing down on the tag alone makes that state
|
||||
# permanent. Note this deliberately OVERRIDES the RELEASE=false above:
|
||||
# with the tag in place there is nothing releasable after it, so the
|
||||
# normal path would stand down, which is why it could never self-heal.
|
||||
# Anything other than 200/404 — a network failure, a bad token — is not
|
||||
# evidence of absence, and guessing "no" would publish over a good
|
||||
# release, so refuse instead.
|
||||
REUSE_TAG=false
|
||||
if [ -n "$VERSION" ] && git rev-parse -q --verify "refs/tags/v${VERSION}" >/dev/null; then
|
||||
CI_TOKEN="$(printf '%s' "${REGISTRY_TOKEN:-}" | tr -d '\r\n')"
|
||||
REL_HTTP="$(curl -s -o /dev/null -w '%{http_code}' \
|
||||
-H "Authorization: token ${CI_TOKEN}" \
|
||||
"https://${GITEA_HOST}/api/v1/repos/${REPO}/releases/tags/v${VERSION}" || echo 000)"
|
||||
case "$REL_HTTP" in
|
||||
200) echo "v${VERSION} is already released — nothing to do."; RELEASE=false ;;
|
||||
404) echo "::warning::Tag v${VERSION} exists but has no release — a previous run failed after tagging. Reusing the tag and publishing the release it is missing."
|
||||
REUSE_TAG=true; RELEASE=true ;;
|
||||
*) echo "::error::Could not determine whether v${VERSION} is released (HTTP ${REL_HTTP}). Refusing to guess."; exit 1 ;;
|
||||
esac
|
||||
fi
|
||||
|
||||
# Changelog range. A recovery run has nothing after the tag, so
|
||||
# summarize what the tag itself contains rather than emitting an empty
|
||||
# list: the range that produced it, i.e. previous-tag..this-tag.
|
||||
if [ "$REUSE_TAG" = true ]; then
|
||||
PREV_TAG="$(git describe --tags --match 'v*' --abbrev=0 "v${VERSION}^" 2>/dev/null || true)"
|
||||
CL_RANGE="${PREV_TAG:+${PREV_TAG}..}v${VERSION}"
|
||||
SINCE="$PREV_TAG"
|
||||
else
|
||||
CL_RANGE="$RANGE"
|
||||
SINCE="$LAST_TAG"
|
||||
fi
|
||||
CL_SUBJECTS="$(git log --no-merges --format='%s' $CL_RANGE || true)"
|
||||
|
||||
{
|
||||
echo "## module-rust v${VERSION}"
|
||||
echo
|
||||
echo "Install from the website's Admin → Modules screen by pasting the URL of"
|
||||
echo "\`module-rust-${VERSION}.json\`, or unpack the tarball onto the modules volume"
|
||||
echo "as \`modules/rust/\`. Requires a core whose \`MODULE_API_VERSION\` satisfies"
|
||||
echo "\`$(node -p "require('./module.json').coreApi")\`."
|
||||
echo
|
||||
echo "A Rust server also needs the other two halves of the bridge:"
|
||||
echo "[Rust-Link](https://${GITEA_HOST}/RunicGateway/Rust-Link) (the sidecar) and"
|
||||
echo "[Rust-Plugins](https://${GITEA_HOST}/RunicGateway/Rust-Plugins) (the Oxide/Carbon plugin)."
|
||||
echo
|
||||
FEATS="$(echo "$CL_SUBJECTS" | grep -E '^feat' || true)"
|
||||
FIXES="$(echo "$CL_SUBJECTS" | grep -E '^(fix|perf)' || true)"
|
||||
[ -n "$FEATS" ] && { echo "### Features"; echo "$FEATS" | sed 's/^/- /'; echo; }
|
||||
[ -n "$FIXES" ] && { echo "### Fixes"; echo "$FIXES" | sed 's/^/- /'; echo; }
|
||||
echo "### All changes"
|
||||
if [ -n "$SINCE" ]; then echo "Since ${SINCE}:"; fi
|
||||
echo "$CL_SUBJECTS" | sed 's/^/- /'
|
||||
echo
|
||||
echo "### Verifying this download"
|
||||
echo
|
||||
echo "Releases are **unsigned** — the \`sha256\` in \`module-rust-${VERSION}.json\` is the"
|
||||
echo "trust anchor, and the website verifies it before unpacking."
|
||||
echo
|
||||
echo '```bash'
|
||||
echo "sha256sum -c SHA256SUMS --ignore-missing"
|
||||
echo '```'
|
||||
} > dist/CHANGELOG.md
|
||||
|
||||
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
|
||||
echo "tag=v${VERSION}" >> "$GITHUB_OUTPUT"
|
||||
echo "release=${RELEASE}" >> "$GITHUB_OUTPUT"
|
||||
echo "reuse_tag=${REUSE_TAG}" >> "$GITHUB_OUTPUT"
|
||||
echo "bump=${BUMP}" >> "$GITHUB_OUTPUT"
|
||||
echo "==> release=${RELEASE} version=${VERSION} bump=${BUMP} declared=${DECLARED} last_tag=${LAST_TAG:-<none>}"
|
||||
|
||||
# Before anything is built or tagged, so a repo without secrets fails
|
||||
# legibly rather than half-publishing: the tag push can succeed on the
|
||||
# credential actions/checkout left in the local git config while the release
|
||||
# API call 401s, leaving the repo tagged and unreleased.
|
||||
- name: Verify release credentials are configured
|
||||
if: ${{ steps.plan.outputs.release == 'true' }}
|
||||
env:
|
||||
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "$(printf '%s' "${REGISTRY_TOKEN:-}" | tr -d '\r\n')" ]; then
|
||||
echo "::error::Missing Actions secret REGISTRY_TOKEN (needs write:repository) on ${REPO}."
|
||||
exit 1
|
||||
fi
|
||||
echo "Release credentials present."
|
||||
|
||||
- name: Build the client chunk
|
||||
if: ${{ steps.plan.outputs.release == 'true' }}
|
||||
run: |
|
||||
npm ci --prefix client
|
||||
npm run build --prefix client
|
||||
|
||||
# ── Assemble exactly what an operator's volume gets ──────────────────
|
||||
#
|
||||
# Stated as an INCLUDE list, not an exclude list. An exclude list ships
|
||||
# whatever it forgot: the day someone adds `server/tools/` with a scratch
|
||||
# credential in it, an exclude list packs it and nobody finds out.
|
||||
#
|
||||
# The list itself lives in `ci/bundle.json`, not here, because it has a
|
||||
# second reader: `server/scripts/checkBundle.js` runs in PR checks and asks
|
||||
# whether the list still covers everything `server/index.js` reaches. One
|
||||
# declaration, two readers, so a new directory cannot go missing quietly.
|
||||
- name: Assemble the bundle
|
||||
if: ${{ steps.plan.outputs.release == 'true' }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
VERSION="${{ steps.plan.outputs.version }}"
|
||||
OUT="dist/module-rust-${VERSION}"
|
||||
rm -rf "$OUT" && mkdir -p "$OUT"
|
||||
|
||||
# The manifest core reads — with the RELEASED version written into it.
|
||||
# The committed `module.json` is a floor, not a record of the last
|
||||
# release (see the header), so copying it verbatim would ship a bundle
|
||||
# whose `installed_modules` row and admin screen disagree with the tag it
|
||||
# came from. This is the one place the derived number becomes the
|
||||
# module's own.
|
||||
jq --arg v "$VERSION" '.version = $v' module.json > "$OUT/module.json"
|
||||
|
||||
# The two fragments, and the licence the code is under — a bundle that
|
||||
# ships GPL code without its licence is not distributable.
|
||||
for f in $(jq -r '.root[]' ci/bundle.json); do
|
||||
cp "$f" "$OUT/"
|
||||
done
|
||||
|
||||
# The server half, minus what never runs inside core's process. No
|
||||
# node_modules: the shipped half declares no runtime dependencies, and
|
||||
# check:bundle is what keeps that true.
|
||||
mkdir -p "$OUT/server"
|
||||
for d in $(jq -r '.server[]' ci/bundle.json); do
|
||||
cp -r "server/$d" "$OUT/server/"
|
||||
done
|
||||
|
||||
# The client half is the BUILT chunk only. `client/src` is source an
|
||||
# operator has no use for and core will never read.
|
||||
mkdir -p "$OUT/client/dist"
|
||||
cp client/dist/entry.js "$OUT/client/dist/"
|
||||
|
||||
# Prove the bundle is loadable before it is published: these are the
|
||||
# paths core's loader resolves out of module.json, and a release whose
|
||||
# entry point is missing fails on an operator's box with a
|
||||
# `startup_failed` row instead of here. The version assertion guards the
|
||||
# rewrite above — a bundle that still carried the declared version would
|
||||
# install under a number that is not the one it was released as.
|
||||
node -e '
|
||||
const fs = require("fs"), path = require("path");
|
||||
const [root, want] = process.argv.slice(1);
|
||||
const m = JSON.parse(fs.readFileSync(path.join(root, "module.json"), "utf8"));
|
||||
if (m.version !== want) {
|
||||
console.error(`bundle declares ${m.version}, but this is release ${want}`);
|
||||
process.exit(1);
|
||||
}
|
||||
for (const p of [m.server, m.schema, m.purge, m.client.entry, "swagger-fragment.json"]) {
|
||||
if (!fs.existsSync(path.join(root, p))) { console.error("bundle is missing " + p); process.exit(1); }
|
||||
}
|
||||
console.log("bundle contents check: ok");
|
||||
' "$OUT" "$VERSION"
|
||||
|
||||
# ── And that it can actually LOAD ─────────────────────────────────
|
||||
#
|
||||
# The check above stats the paths `module.json` declares, which is a real
|
||||
# question but a shallow one: Module-uo's v1.0.0 passed exactly that and
|
||||
# was still missing `server/commands/`, because a file reached only by a
|
||||
# require inside `register()` is named nowhere in `module.json`. This
|
||||
# resolves every relative require in the assembled tree and asserts the
|
||||
# target is in it — asked of the artifact, so it also catches a copy that
|
||||
# half failed or a list naming a path that has since moved.
|
||||
#
|
||||
# Run from the SOURCE tree (`server/scripts/` never ships) against the
|
||||
# assembled bundle.
|
||||
node server/scripts/checkBundle.js --bundle "$OUT"
|
||||
|
||||
tar -C dist -czf "dist/module-rust-${VERSION}.tar.gz" "module-rust-${VERSION}"
|
||||
rm -rf "$OUT"
|
||||
|
||||
SHA="$(sha256sum "dist/module-rust-${VERSION}.tar.gz" | cut -d' ' -f1)"
|
||||
SIZE="$(stat -c%s "dist/module-rust-${VERSION}.tar.gz")"
|
||||
|
||||
# The install manifest. Same shape as the installer's bundle JSON — a
|
||||
# per-asset sha256 fetched over HTTPS, no signatures — because that is
|
||||
# the model this project already has and a second one would be a second
|
||||
# thing to get right (MODULE_SYSTEM.md §1.11).
|
||||
jq -n \
|
||||
--arg id "$(node -p "require('./module.json').id")" \
|
||||
--arg name "$(node -p "require('./module.json').name")" \
|
||||
--arg version "$VERSION" \
|
||||
--arg coreApi "$(node -p "require('./module.json').coreApi")" \
|
||||
--arg artifact "module-rust-${VERSION}.tar.gz" \
|
||||
--arg sha256 "$SHA" \
|
||||
--argjson size "$SIZE" \
|
||||
--arg url "https://${GITEA_HOST}/${REPO}/releases/download/v${VERSION}/module-rust-${VERSION}.tar.gz" \
|
||||
'{schema:1, id:$id, name:$name, version:$version, coreApi:$coreApi,
|
||||
artifact:$artifact, url:$url, sha256:$sha256, size:$size}' \
|
||||
> "dist/module-rust-${VERSION}.json"
|
||||
|
||||
echo "${SHA} module-rust-${VERSION}.tar.gz" > dist/SHA256SUMS
|
||||
cat "dist/module-rust-${VERSION}.json"
|
||||
|
||||
# Skipped on a recovery run: the tag is already there and is the thing being
|
||||
# published against.
|
||||
- name: Tag the release
|
||||
if: ${{ steps.plan.outputs.release == 'true' && steps.plan.outputs.reuse_tag != 'true' }}
|
||||
env:
|
||||
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
TAG="${{ steps.plan.outputs.tag }}"
|
||||
git config user.name 'Runic Gateway CI'
|
||||
git config user.email 'ci@whitlocktech.net'
|
||||
git tag -a "$TAG" -m "module-rust ${TAG}"
|
||||
git push origin "$TAG"
|
||||
|
||||
- name: Create the Gitea release and upload the bundle
|
||||
if: ${{ steps.plan.outputs.release == 'true' }}
|
||||
env:
|
||||
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
TAG="${{ steps.plan.outputs.tag }}"
|
||||
VERSION="${{ steps.plan.outputs.version }}"
|
||||
API="https://${GITEA_HOST}/api/v1/repos/${REPO}"
|
||||
CI_TOKEN="$(printf '%s' "${REGISTRY_TOKEN}" | tr -d '\r\n')"
|
||||
|
||||
REL_ID="$(curl -sSf -X POST "${API}/releases" \
|
||||
-H "Authorization: token ${CI_TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$(jq -n --arg tag "$TAG" --arg body "$(cat dist/CHANGELOG.md)" \
|
||||
'{tag_name:$tag, name:$tag, body:$body, draft:false, prerelease:false}')" \
|
||||
| jq -r '.id')"
|
||||
echo "Created release ${TAG} (id=${REL_ID})"
|
||||
|
||||
for f in "module-rust-${VERSION}.tar.gz" "module-rust-${VERSION}.json" SHA256SUMS; do
|
||||
curl -sSf -X POST "${API}/releases/${REL_ID}/assets?name=${f}" \
|
||||
-H "Authorization: token ${CI_TOKEN}" \
|
||||
-F "attachment=@dist/${f}" >/dev/null
|
||||
echo " uploaded ${f}"
|
||||
done
|
||||
62
README.md
62
README.md
@@ -52,6 +52,7 @@ both surfaces an operator can configure and then wait on, which is worse than an
|
||||
```bash
|
||||
npm ci --prefix server && npm test --prefix server
|
||||
npm run check:imports --prefix server
|
||||
npm run check:bundle --prefix server
|
||||
npm run check:swagger --prefix server
|
||||
npm ci --prefix client && npm run build --prefix client
|
||||
npm run check:externals --prefix client && npm test --prefix client
|
||||
@@ -66,11 +67,66 @@ Regenerate the OpenAPI fragment whenever a route or an annotation changes:
|
||||
npm run swagger --prefix server # writes swagger-fragment.json; commit it
|
||||
```
|
||||
|
||||
`.gitea/workflows/pr-checks.yml` runs all of the above on every pull request, plus one job this
|
||||
machine cannot run on its own: **frozen-manifest** clones core at the sha pinned in
|
||||
[`ci/core-ref.json`](ci/core-ref.json), generates its route table without this module and then with
|
||||
it, and takes the difference. That difference is the URL surface this module serves — checked
|
||||
against the committed [`routes.manifest.json`](routes.manifest.json), against the OpenAPI fragment
|
||||
in both directions, and against the rule that **a module may only add**. It is the only thing that
|
||||
can see whether `/rust` collides with one of the routes core mounts at a tier root (`/status`,
|
||||
`/version`), which the loader's own collision probe cannot find.
|
||||
|
||||
## How it reaches an operator
|
||||
|
||||
**An operator never builds anything.** A release is not source: it is the directory core's loader
|
||||
expects at `modules/rust/`, already assembled — the prebuilt client chunk, the schema fragment and
|
||||
the OpenAPI fragment, packed as they will be unpacked.
|
||||
|
||||
**Every merge to `main` carrying a releasable commit publishes a bundle.** The next version is
|
||||
computed from conventional-commit subjects since the newest `v*` tag, as in `link`, `installer` and
|
||||
`Module-uo`: `feat!:` or `BREAKING CHANGE` is a major, `feat:` a minor, `fix:` or `perf:` a patch,
|
||||
and a `main` that gained none of those cuts no release. The number that ships is the **tag**, and CI
|
||||
writes it into the `module.json` inside the bundle. `module.json`'s version survives as a **floor**:
|
||||
name a version there above the newest tag and that version releases, which is how you overrule the
|
||||
subjects. For a change with nothing releasable behind it — a widened `coreApi`, a new mount, a
|
||||
capability — run the **Release** workflow by hand (Actions → Release → Run workflow).
|
||||
|
||||
Each release carries:
|
||||
|
||||
| Asset | What it is |
|
||||
|---|---|
|
||||
| `module-rust-<version>.tar.gz` | the directory core expects at `modules/rust/`, already assembled |
|
||||
| `module-rust-<version>.json` | the install manifest: id, version, `coreApi`, the artifact's URL, size and **`sha256`** |
|
||||
| `SHA256SUMS` | the same hash, in the shape every other repo here publishes |
|
||||
|
||||
Releases are **unsigned**; the `sha256` is the trust anchor, and the website verifies it before
|
||||
unpacking. That is the model `installer`'s bundles already use, and a second trust model would be a
|
||||
second thing to get right.
|
||||
|
||||
The tarball is assembled from an **include** list ([`ci/bundle.json`](ci/bundle.json)), never an
|
||||
exclude list — an exclude list ships whatever it forgot. Tests, scripts, `client/src`, `ci/` and the
|
||||
dev dependencies are not in it. It carries **no `node_modules`**, because the shipped half declares
|
||||
no runtime dependencies: everything it needs arrives on `ctx`. `npm run check:bundle` holds both
|
||||
halves of that — that the list still covers every file `server/index.js` can reach, and that no
|
||||
runtime dependency has appeared without the release learning to pack it.
|
||||
|
||||
## Install it into a core
|
||||
|
||||
Copy the whole tree to `<website>/modules/rust/` and restart. **Copy, do not symlink** — the loader
|
||||
lists directory entries and asks each whether it is a directory; a symlink answers no and the module
|
||||
is skipped in complete silence.
|
||||
**From a release**, which is the supported path: in Admin → Modules, paste the URL of that release's
|
||||
`module-rust-<version>.json`, and restart when the panel offers. Core fetches the manifest, checks
|
||||
every URL and redirect hop against its own host allowlist, streams the artifact under a byte cap
|
||||
while hashing it, verifies the `sha256`, inspects the archive in full before unpacking it to a
|
||||
temporary directory, and only then moves it into `modules/rust/`. Nothing is written into the
|
||||
modules directory until every check has passed. The allowlist must contain
|
||||
`gitea.whitlocktech.com` — it is seeded from `MODULE_SOURCE_HOSTS` on a fresh install and is
|
||||
DB-owned from then on, edited on that same screen. **An empty allowlist forbids every install rather
|
||||
than permitting all of them.**
|
||||
|
||||
**From a working tree**, for development: copy the whole tree to `<website>/modules/rust/` and
|
||||
restart. **Copy, do not symlink** — the loader lists directory entries and asks each whether it is a
|
||||
directory; a symlink answers no and the module is skipped in complete silence.
|
||||
|
||||
Either way, the module appears when the process restarts: the volume is read at require time.
|
||||
|
||||
Then, in Admin → Rust, add a server: its name, the sidecar's base URL, and the token the sidecar
|
||||
printed on first start (`rust-link-sidecar --print-config`). **The token is write-only** — it is
|
||||
|
||||
47
ci/bundle.json
Normal file
47
ci/bundle.json
Normal file
@@ -0,0 +1,47 @@
|
||||
{
|
||||
"$comment": [
|
||||
"What a release copies into the bundle, declared ONCE. Read by .gitea/workflows/release.yml when",
|
||||
"it assembles the tarball, and by server/scripts/checkBundle.js when CI asks whether that list",
|
||||
"still covers everything the module's entry point can reach.",
|
||||
"",
|
||||
"This is an INCLUDE list on purpose. An exclude list ships whatever it forgot: the day someone",
|
||||
"adds server/tools/ with a scratch credential in it, an exclude list packs it and nobody finds",
|
||||
"out. The cost of that choice is that a new top-level directory silently drops OUT of every",
|
||||
"release instead — which is exactly what happened to Module-uo between v0.3.0 and v1.0.0, where",
|
||||
"server/commands/ arrived with a cutover, the list did not learn about it, and the module",
|
||||
"installed and then died at the register stage on the operator's box. checkBundle.js exists so",
|
||||
"that cannot happen twice, and it runs on the PR that adds the directory.",
|
||||
"",
|
||||
"server[] entries are paths under server/; root[] and generated[] are paths under the module",
|
||||
"root.",
|
||||
"",
|
||||
"node_modules is NOT here, and its absence is asserted rather than assumed: this module declares",
|
||||
"no runtime dependencies (everything the shipped half needs arrives on ctx), so the release runs",
|
||||
"no npm ci and packs no dependency tree. checkBundle.js fails the PR that adds a `dependencies`",
|
||||
"entry to server/package.json without also teaching the release to pack it — because a module",
|
||||
"whose bundle silently lacks its own dependency fails the same way the missing directory did.",
|
||||
"",
|
||||
"generated[] ships but is not copied — release.yml writes module.json through jq to stamp the",
|
||||
"released version into it, since the committed one is a floor rather than a record of the last",
|
||||
"release. It is listed because server/index.js requires it, and a check that did not know it",
|
||||
"ships would report the module's own manifest as missing from the bundle."
|
||||
],
|
||||
"server": [
|
||||
"boot.js",
|
||||
"core.js",
|
||||
"db",
|
||||
"index.js",
|
||||
"model",
|
||||
"package.json",
|
||||
"router",
|
||||
"sidecarClient.js"
|
||||
],
|
||||
"root": [
|
||||
"swagger-fragment.json",
|
||||
"LICENSE.md",
|
||||
"README.md"
|
||||
],
|
||||
"generated": [
|
||||
"module.json"
|
||||
]
|
||||
}
|
||||
6
ci/core-ref.json
Normal file
6
ci/core-ref.json
Normal file
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"$comment": "The core this module is proved against. MODULE_API.md §5.3: the frozen-manifest job clones RunicGateway/website at this exact ref, drops this module in as modules/rust and runs CORE's own routeManifest.js — nothing else can answer whether the URLs the module claims are the URLs it actually serves, because a mount prefix is a string in server/index.js and a documented path is a string in a JSON file, and whether those name the same URL is a fact about a running core. It also answers the blind spot phase 1 had to check by hand: core mounts several routes at the TIER ROOT (/status, /version), which the loader's collision probe cannot see, so /rust being free is asserted here by a core rather than by a reading. Pinned rather than tracking a branch on purpose: core moves for reasons that have nothing to do with this module, and a bump is then a deliberate commit saying which core the module was last proved against, instead of an unexplained red X on someone else's PR. Bump it, regenerate routes.manifest.json, and commit both together. This module needs MODULE_API 1.10.0 (module.json's coreApi is ^1.10.0), which the Event System cutover put on `main` — so unlike Module-uo, which spent the Event System window pinned to `edge`, this repo starts pinned to `main` and should stay there unless it comes to depend on a contract member that has not shipped yet.",
|
||||
"repo": "https://gitea.whitlocktech.com/RunicGateway/website.git",
|
||||
"ref": "efa9db73304552dd8bb7a84030b258c6320f79f7",
|
||||
"refName": "main @ MODULE_API 1.10.0, the Asset Bridge cutover 2 of 5 (website#202)"
|
||||
}
|
||||
35
routes.manifest.json
Normal file
35
routes.manifest.json
Normal file
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"$comment": "Generated inventory of the URLs module-rust serves - the module half of the freeze core keeps in server/routes.manifest.json. DERIVED as the difference between a core without this module and the same core with it, both at the pinned ref in ci/core-ref.json. Regenerate with the frozen-manifest job in .gitea/workflows/pr-checks.yml; see server/scripts/frozenManifest.js.",
|
||||
"routes": [
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/rust/servers/:id",
|
||||
"tier": "public"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/rust/servers",
|
||||
"tier": "public"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/player/rust/servers",
|
||||
"tier": "public"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/rust/servers",
|
||||
"tier": "public"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/rust/servers/:id/test",
|
||||
"tier": "public"
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/rust/servers/:id",
|
||||
"tier": "public"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -8,6 +8,7 @@
|
||||
"scripts": {
|
||||
"test": "node --test",
|
||||
"check:imports": "node scripts/checkImports.js",
|
||||
"check:bundle": "node scripts/checkBundle.js",
|
||||
"swagger": "node scripts/swaggerFragment.js",
|
||||
"check:swagger": "node scripts/swaggerFragment.js --check"
|
||||
},
|
||||
|
||||
301
server/scripts/checkBundle.js
Normal file
301
server/scripts/checkBundle.js
Normal file
@@ -0,0 +1,301 @@
|
||||
#!/usr/bin/env node
|
||||
// ── Does the release actually ship everything the module needs? ────────────
|
||||
//
|
||||
// `ci/bundle.json` says what a release copies. `server/index.js` says what the
|
||||
// module requires. Nothing keeps two lists in agreement on its own, and the first
|
||||
// module this project shipped proved it: `Module-uo` added `server/commands/` in a
|
||||
// cutover, its include list did not learn about it, and v1.0.0 installed cleanly
|
||||
// and then died on the operator's box with
|
||||
//
|
||||
// module "uo" failed to load — {"stage":"register","reason":"Cannot find
|
||||
// module './commands/guild.command'"}
|
||||
//
|
||||
// Nothing caught it, because the PR checks install the module by copying the
|
||||
// WHOLE repo into core — they only ever exercised a tree that had the file. **The
|
||||
// subset only exists in the release**, and the release had no check that the
|
||||
// subset was complete. This module has that check from its first release rather
|
||||
// than after its first outage.
|
||||
//
|
||||
// It asks the question in the two places it can be asked:
|
||||
//
|
||||
// --check (PR checks) Every file reachable from the entry point by a
|
||||
// relative require lives under something ci/bundle.json
|
||||
// lists. Source-tree only, so it is fast and needs no
|
||||
// assembled bundle — it fails on the PR that adds the
|
||||
// directory, which is where the fix is cheapest.
|
||||
//
|
||||
// --bundle <dir> (release) Every relative specifier inside an ASSEMBLED bundle
|
||||
// resolves to a file that is in it. Asked of the
|
||||
// artifact rather than of the source, so it also
|
||||
// catches a copy that half-failed, a list that names a
|
||||
// path that has moved, and anything else between the
|
||||
// declaration and the tarball.
|
||||
//
|
||||
// The two are deliberately not the same question. The first is about the list
|
||||
// being right; the second is about the tarball being right. A release runs both.
|
||||
//
|
||||
// ── The third question, which is this module's own ─────────────────────────
|
||||
//
|
||||
// `server/package.json` declares **no runtime dependencies**, and the release
|
||||
// therefore runs no `npm ci` and packs no `node_modules`. That is a decision, not
|
||||
// an accident (org lead, phase 2), and the whole value of it is that the day it
|
||||
// stops being true is a loud day. So both modes also assert the declaration is
|
||||
// still empty: add a `dependencies` entry without teaching release.yml to install
|
||||
// and pack it, and the bundle ships an import of something that is not there —
|
||||
// the missing-directory failure again, wearing a different hat.
|
||||
//
|
||||
// ── Why reachability, and not "require the entry point" ────────────────────
|
||||
//
|
||||
// The obvious check — require the bundle's entry and see if it throws — does not
|
||||
// work here, and the reason is in index.js's own header: its requires are inside
|
||||
// `register()` because require order is load-bearing (`core.init(ctx)` has to run
|
||||
// before anything under `router/` is required). So requiring the entry evaluates
|
||||
// exactly one line, `require('./core')`, and reports success on a bundle missing
|
||||
// every router it has. Calling `register()` for real would need a fake `ctx`
|
||||
// complete enough to satisfy the whole module — which is what `test/` is for, and
|
||||
// `test/` does not ship. Walking the requires statically asks the same question
|
||||
// without needing either.
|
||||
|
||||
const fs = require('fs')
|
||||
const path = require('path')
|
||||
const { stripCommentsAndTemplates } = require('./checkImports')
|
||||
|
||||
const MODULE_ROOT = path.resolve(__dirname, '..', '..')
|
||||
|
||||
// Only relative specifiers. A bare one is checkImports.js's question, not this
|
||||
// one, and the two failures want different advice.
|
||||
const RELATIVE = /(?:require\(|from\s+|import\()\s*['"](\.[^'"]+)['"]/g
|
||||
|
||||
/**
|
||||
* Resolve a relative specifier the way Node would, for the file cases that can
|
||||
* appear here: an exact path, `+.js`/`+.json`, or a directory's `index.js`.
|
||||
*
|
||||
* Returns null when nothing exists — which is the finding, not an error.
|
||||
*/
|
||||
function resolveFile(fromDir, specifier) {
|
||||
const base = path.resolve(fromDir, specifier)
|
||||
const candidates = [base, `${base}.js`, `${base}.json`, path.join(base, 'index.js')]
|
||||
for (const c of candidates) {
|
||||
if (fs.existsSync(c) && fs.statSync(c).isFile()) return c
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
/**
|
||||
* Every file reachable from `entry` by following relative requires, plus every
|
||||
* specifier that resolved to nothing.
|
||||
*
|
||||
* Exported so the test can point it at fixtures — the same reason checkImports.js
|
||||
* exports `scan`. A check that has never been shown to fail is a check nobody
|
||||
* knows the state of, and this one is load-bearing for every release.
|
||||
*/
|
||||
function reachable(entry) {
|
||||
const seen = new Set()
|
||||
const missing = []
|
||||
const queue = [entry]
|
||||
|
||||
while (queue.length) {
|
||||
const file = queue.shift()
|
||||
if (seen.has(file)) continue
|
||||
seen.add(file)
|
||||
|
||||
// A .json dependency is a leaf: it is reached, it ships, and it has no
|
||||
// requires of its own to follow.
|
||||
if (file.endsWith('.json')) continue
|
||||
|
||||
const source = stripCommentsAndTemplates(fs.readFileSync(file, 'utf8'))
|
||||
for (const [, specifier] of source.matchAll(RELATIVE)) {
|
||||
const target = resolveFile(path.dirname(file), specifier)
|
||||
if (target) queue.push(target)
|
||||
else missing.push({ file, specifier })
|
||||
}
|
||||
}
|
||||
|
||||
return { files: [...seen], missing }
|
||||
}
|
||||
|
||||
/**
|
||||
* Everything ci/bundle.json says ends up in the bundle, as absolute paths:
|
||||
* `server[]` relative to server/, `root[]` and `generated[]` relative to the
|
||||
* module root. All three are equally "in the tarball" as far as a require is
|
||||
* concerned — the only difference is how they get there.
|
||||
*/
|
||||
function declaredServerPaths(moduleRoot = MODULE_ROOT) {
|
||||
const manifest = JSON.parse(fs.readFileSync(path.join(moduleRoot, 'ci', 'bundle.json'), 'utf8'))
|
||||
return [
|
||||
...manifest.server.map((p) => path.join(moduleRoot, 'server', p)),
|
||||
...(manifest.root || []).map((p) => path.join(moduleRoot, p)),
|
||||
...(manifest.generated || []).map((p) => path.join(moduleRoot, p)),
|
||||
]
|
||||
}
|
||||
|
||||
/**
|
||||
* The runtime dependencies the shipped half declares.
|
||||
*
|
||||
* Empty is the shape this module is built around, and the release packs no
|
||||
* `node_modules` because of it. Returned rather than asserted so both modes can
|
||||
* report it with their own advice.
|
||||
*/
|
||||
function runtimeDependencies(moduleRoot = MODULE_ROOT) {
|
||||
const pkgPath = path.join(moduleRoot, 'server', 'package.json')
|
||||
if (!fs.existsSync(pkgPath)) return []
|
||||
return Object.keys(JSON.parse(fs.readFileSync(pkgPath, 'utf8')).dependencies || {})
|
||||
}
|
||||
|
||||
const DEPENDENCY_ADVICE =
|
||||
'The release packs no node_modules, because this module declared none. A dependency\n' +
|
||||
'listed here but not installed and copied by .gitea/workflows/release.yml ships as an\n' +
|
||||
'import of something that is not in the tarball — the module installs and then dies at\n' +
|
||||
'the register stage on the operator\'s box.\n\n' +
|
||||
'Either drop the dependency (everything the shipped half needs arrives on ctx — §2.3),\n' +
|
||||
'or add the `npm ci --omit=dev` + copy steps to release.yml and list "node_modules" in\n' +
|
||||
'ci/bundle.json\'s server[], then update this check.\n'
|
||||
|
||||
const covers = (declared, file) => declared.some((d) => file === d || file.startsWith(d + path.sep))
|
||||
|
||||
/**
|
||||
* --check: is ci/bundle.json's list sufficient for what the entry point reaches?
|
||||
*
|
||||
* Reports the top-level entry to ADD rather than the individual files, because
|
||||
* that is the edit: the list is stated in top-level paths, and a new directory
|
||||
* arrives with a dozen files in it.
|
||||
*/
|
||||
function checkDeclaration(moduleRoot = MODULE_ROOT) {
|
||||
const serverRoot = path.join(moduleRoot, 'server')
|
||||
const entry = path.join(serverRoot, 'index.js')
|
||||
const { files, missing } = reachable(entry)
|
||||
const declared = declaredServerPaths(moduleRoot)
|
||||
|
||||
// Grouped by the entry that would have to be added, which is the top-level
|
||||
// path under server/ — or, for the rare reachable file outside it, the path
|
||||
// itself, since that one belongs in root[] instead.
|
||||
const uncovered = new Map()
|
||||
for (const file of files) {
|
||||
if (covers(declared, file)) continue
|
||||
const inServer = file.startsWith(serverRoot + path.sep)
|
||||
const key = inServer
|
||||
? `server/${path.relative(serverRoot, file).split(path.sep)[0]}`
|
||||
: path.relative(moduleRoot, file).split(path.sep).join('/')
|
||||
if (!uncovered.has(key)) uncovered.set(key, [])
|
||||
uncovered.get(key).push(file)
|
||||
}
|
||||
|
||||
return { uncovered, missing, reached: files.length, dependencies: runtimeDependencies(moduleRoot) }
|
||||
}
|
||||
|
||||
/**
|
||||
* --bundle: does every relative specifier inside an assembled bundle resolve?
|
||||
*
|
||||
* Walks the bundle's own server tree rather than starting from the entry point,
|
||||
* so a file that ships but is broken is caught too.
|
||||
*/
|
||||
function checkBundle(bundleRoot) {
|
||||
const serverRoot = path.join(bundleRoot, 'server')
|
||||
const missing = []
|
||||
const files = []
|
||||
|
||||
const walk = (dir) => {
|
||||
for (const e of fs.readdirSync(dir, { withFileTypes: true })) {
|
||||
const p = path.join(dir, e.name)
|
||||
if (e.isDirectory()) {
|
||||
// An installed dependency tree would be npm's business, not this
|
||||
// check's. This module ships none; the skip stays so that the day one
|
||||
// arrives, this is not also the thing that breaks.
|
||||
if (e.name !== 'node_modules') walk(p)
|
||||
} else if (/\.(js|mjs|cjs)$/.test(e.name)) {
|
||||
files.push(p)
|
||||
}
|
||||
}
|
||||
}
|
||||
walk(serverRoot)
|
||||
|
||||
for (const file of files) {
|
||||
const source = stripCommentsAndTemplates(fs.readFileSync(file, 'utf8'))
|
||||
for (const [, specifier] of source.matchAll(RELATIVE)) {
|
||||
if (!resolveFile(path.dirname(file), specifier)) missing.push({ file, specifier })
|
||||
}
|
||||
}
|
||||
|
||||
return { missing, scanned: files.length, dependencies: runtimeDependencies(bundleRoot) }
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
reachable,
|
||||
resolveFile,
|
||||
checkDeclaration,
|
||||
checkBundle,
|
||||
declaredServerPaths,
|
||||
runtimeDependencies,
|
||||
}
|
||||
|
||||
// Required by a test, or run as the check? Only the second one exits.
|
||||
if (require.main !== module) return
|
||||
|
||||
const bundleFlag = process.argv.indexOf('--bundle')
|
||||
|
||||
if (bundleFlag !== -1) {
|
||||
const root = process.argv[bundleFlag + 1]
|
||||
if (!root) {
|
||||
console.error('--bundle needs the path to an assembled bundle')
|
||||
process.exit(2)
|
||||
}
|
||||
const { missing, scanned, dependencies } = checkBundle(path.resolve(root))
|
||||
if (missing.length) {
|
||||
console.error(`\nThe assembled bundle is incomplete — ${missing.length} require(s) resolve to nothing:\n`)
|
||||
for (const m of missing) {
|
||||
console.error(` ${path.relative(root, m.file)}\n requires "${m.specifier}" — not in the bundle`)
|
||||
}
|
||||
console.error('\nAdd the missing path to ci/bundle.json.\n')
|
||||
process.exit(1)
|
||||
}
|
||||
if (dependencies.length) {
|
||||
console.error(`\nThe assembled bundle declares ${dependencies.length} runtime dependency(ies) it does not carry:\n`)
|
||||
for (const d of dependencies) console.error(` ${d}`)
|
||||
console.error(`\n${DEPENDENCY_ADVICE}`)
|
||||
process.exit(1)
|
||||
}
|
||||
console.log(`OK — every relative require in the bundle resolves (${scanned} files scanned), and it needs no node_modules.`)
|
||||
} else {
|
||||
const { uncovered, missing, reached, dependencies } = checkDeclaration()
|
||||
|
||||
if (missing.length) {
|
||||
console.error(`\n${missing.length} require(s) resolve to nothing in the source tree:\n`)
|
||||
for (const m of missing) {
|
||||
console.error(` ${path.relative(MODULE_ROOT, m.file)}\n requires "${m.specifier}"`)
|
||||
}
|
||||
console.error('')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
if (uncovered.size) {
|
||||
console.error('\nci/bundle.json does not ship everything server/index.js reaches.\n')
|
||||
console.error('A release built from this list would install and then fail at the')
|
||||
console.error("register stage with \"Cannot find module\", on the operator's box.\n")
|
||||
for (const [key, files] of uncovered) {
|
||||
console.error(` ${key} (${files.length} file${files.length === 1 ? '' : 's'} reachable)`)
|
||||
for (const f of files.slice(0, 5)) console.error(` ${path.relative(MODULE_ROOT, f)}`)
|
||||
if (files.length > 5) console.error(` … and ${files.length - 5} more`)
|
||||
}
|
||||
// server[] is written relative to server/, so name the entry to add rather
|
||||
// than the path just displayed — they differ by exactly that prefix.
|
||||
const toServer = [...uncovered.keys()].filter((k) => k.startsWith('server/'))
|
||||
const toRoot = [...uncovered.keys()].filter((k) => !k.startsWith('server/'))
|
||||
if (toServer.length) {
|
||||
console.error(`\nAdd ${toServer.map((k) => `"${k.slice('server/'.length)}"`).join(', ')} to ci/bundle.json's server[].`)
|
||||
}
|
||||
if (toRoot.length) {
|
||||
console.error(`\nAdd ${toRoot.map((k) => `"${k}"`).join(', ')} to ci/bundle.json's root[].`)
|
||||
}
|
||||
console.error('')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
if (dependencies.length) {
|
||||
console.error(`\nserver/package.json declares ${dependencies.length} runtime dependency(ies):\n`)
|
||||
for (const d of dependencies) console.error(` ${d}`)
|
||||
console.error(`\n${DEPENDENCY_ADVICE}`)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
console.log(`OK — ci/bundle.json ships every file server/index.js reaches (${reached} files), and no runtime dependency is declared.`)
|
||||
}
|
||||
196
server/scripts/frozenManifest.js
Normal file
196
server/scripts/frozenManifest.js
Normal file
@@ -0,0 +1,196 @@
|
||||
#!/usr/bin/env node
|
||||
// ── §5.3 — this module's frozen route manifest ─────────────────────────────
|
||||
//
|
||||
// Core freezes its URL surface in `server/routes.manifest.json` by walking the
|
||||
// live Express stack and committing the result; a PR that moves a URL has to
|
||||
// commit the new manifest, which puts the change in front of a reviewer. The URLs
|
||||
// this module serves are not in that file. They are here, frozen the same way and
|
||||
// by the same generator.
|
||||
//
|
||||
// **The module's routes are DERIVED, never listed.** This script is handed two
|
||||
// manifests generated from the SAME core at the pinned ref — one without this
|
||||
// module on the volume, one with — and the difference is what this module serves.
|
||||
// Nothing here says "/api/v1/public/rust/*"; a mount prefix appears in exactly one
|
||||
// place, `server/index.js`'s `registerRoutes` call, which is where an operator's
|
||||
// core reads it from too.
|
||||
//
|
||||
// Taking the difference rather than filtering by prefix buys the other half of
|
||||
// §5.3 for free, and it is the half that matters most: **no core URL may move.**
|
||||
// A module that shadowed a core route, or whose mount displaced one, shows up here
|
||||
// as a removal or a change, not merely as an addition somewhere else. That is the
|
||||
// promise §1.2 makes to the shipped Android app and the Discord bot.
|
||||
//
|
||||
// It is also the only check that can see the blind spot §13's own registration
|
||||
// comment names: core answers several public routes mounted at the TIER ROOT
|
||||
// rather than under a prefix — `/status` and `/version` among them — and the
|
||||
// loader's collision probe cannot find those. `/rust` was checked against core's
|
||||
// mount tables by hand when phase 1 chose it. From here it is checked by a core.
|
||||
//
|
||||
// The third thing it checks is the OpenAPI fragment (§2.8). `swagger-fragment.json`
|
||||
// is generated from the module's own registrations against §2.4's stated tier
|
||||
// bases — the one place a constant could be wrong. Here there is ground truth: a
|
||||
// real core with this module loaded, reporting the URLs it actually serves. Every
|
||||
// route must have a documented operation and every documented operation must be a
|
||||
// route. That is the per-module form of core's standing rule, never ship a route
|
||||
// that isn't in the spec — and it is what stops a wrong constant in the generator
|
||||
// from producing a fragment that is internally consistent and describes nothing
|
||||
// core will ever serve.
|
||||
//
|
||||
// Usage (the workflow does the cloning; see .gitea/workflows/pr-checks.yml):
|
||||
// node scripts/frozenManifest.js --before core-only.json --after core-plus-rust.json
|
||||
// node scripts/frozenManifest.js --before … --after … --check
|
||||
|
||||
const fs = require('fs')
|
||||
const path = require('path')
|
||||
|
||||
const MODULE_ROOT = path.resolve(__dirname, '..', '..')
|
||||
const MANIFEST = path.join(MODULE_ROOT, 'routes.manifest.json')
|
||||
const FRAGMENT = path.join(MODULE_ROOT, 'swagger-fragment.json')
|
||||
|
||||
const COMMENT =
|
||||
'Generated inventory of the URLs module-rust serves - the module half of the freeze ' +
|
||||
'core keeps in server/routes.manifest.json. DERIVED as the difference between a core ' +
|
||||
'without this module and the same core with it, both at the pinned ref in ci/core-ref.json. ' +
|
||||
'Regenerate with the frozen-manifest job in .gitea/workflows/pr-checks.yml; see ' +
|
||||
'server/scripts/frozenManifest.js.'
|
||||
|
||||
const key = (r) => `${r.method} ${r.path}`
|
||||
|
||||
/**
|
||||
* The module's routes, plus proof that core's own surface did not move.
|
||||
*
|
||||
* @param {object} before routes.manifest.json from core alone
|
||||
* @param {object} after routes.manifest.json from the same core with this module
|
||||
* @returns {{ added: object[], removed: string[] }}
|
||||
*/
|
||||
function diffManifests(before, after) {
|
||||
const added = []
|
||||
const removed = []
|
||||
|
||||
for (const tier of ['public', 'internal']) {
|
||||
const was = new Set((before[tier] || []).map(key))
|
||||
for (const route of after[tier] || []) {
|
||||
if (!was.has(key(route))) added.push({ ...route, tier })
|
||||
was.delete(key(route))
|
||||
}
|
||||
for (const gone of was) removed.push(`${tier} ${gone}`)
|
||||
}
|
||||
|
||||
added.sort((a, b) => (key(a) < key(b) ? -1 : 1))
|
||||
return { added, removed }
|
||||
}
|
||||
|
||||
/**
|
||||
* Which of the module's routes the fragment fails to document, and vice versa.
|
||||
*
|
||||
* Express `:id` is OpenAPI `{id}`; the fragment is already in OpenAPI's spelling
|
||||
* because that is what core merges, so the manifest's paths are converted here
|
||||
* rather than the other way round.
|
||||
*/
|
||||
function coverage(added, fragment) {
|
||||
const documented = new Set()
|
||||
for (const [p, item] of Object.entries(fragment.paths || {})) {
|
||||
for (const method of Object.keys(item)) documented.add(`${method.toUpperCase()} ${p}`)
|
||||
}
|
||||
|
||||
const undocumented = []
|
||||
for (const route of added) {
|
||||
const oas = `${route.method} ${route.path.replace(/:([A-Za-z0-9_]+)/g, '{$1}')}`
|
||||
if (documented.has(oas)) documented.delete(oas)
|
||||
else undocumented.push(oas)
|
||||
}
|
||||
|
||||
// Whatever is left is documented and not served: a route that moved or was
|
||||
// deleted while its annotation stayed behind. Core's own spec has no equivalent
|
||||
// check and grew four orphan tags and thirty-three orphan schemas because of it.
|
||||
return { undocumented, unserved: [...documented].sort() }
|
||||
}
|
||||
|
||||
function serialize(routes) {
|
||||
return `${JSON.stringify(
|
||||
{
|
||||
$comment: COMMENT,
|
||||
routes: routes.map(({ method, path: p, tier }) => ({ method, path: p, tier })),
|
||||
},
|
||||
null,
|
||||
2,
|
||||
)}\n`
|
||||
}
|
||||
|
||||
function main() {
|
||||
const arg = (name) => {
|
||||
const i = process.argv.indexOf(name)
|
||||
return i === -1 ? null : process.argv[i + 1]
|
||||
}
|
||||
const beforePath = arg('--before')
|
||||
const afterPath = arg('--after')
|
||||
if (!beforePath || !afterPath) {
|
||||
process.stderr.write('usage: frozenManifest.js --before <manifest> --after <manifest> [--check]\n')
|
||||
process.exit(2)
|
||||
}
|
||||
|
||||
const before = JSON.parse(fs.readFileSync(beforePath, 'utf8'))
|
||||
const after = JSON.parse(fs.readFileSync(afterPath, 'utf8'))
|
||||
const { added, removed } = diffManifests(before, after)
|
||||
|
||||
let failed = false
|
||||
|
||||
if (removed.length > 0) {
|
||||
process.stderr.write(
|
||||
`\nLoading this module REMOVED or CHANGED ${removed.length} of core's own route(s):\n` +
|
||||
`${removed.map((r) => ` - ${r}`).join('\n')}\n` +
|
||||
'A module may only add. This is the frozen-URL promise (MODULE_SYSTEM.md §1.2) breaking.\n',
|
||||
)
|
||||
failed = true
|
||||
}
|
||||
|
||||
if (added.length === 0) {
|
||||
process.stderr.write(
|
||||
'\nLoading this module added NO routes. Either it failed to load in the core checkout\n' +
|
||||
'(check the boot log for a startup_failed line) or the two manifests are the same file.\n',
|
||||
)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
const fragment = JSON.parse(fs.readFileSync(FRAGMENT, 'utf8'))
|
||||
const { undocumented, unserved } = coverage(added, fragment)
|
||||
if (undocumented.length > 0) {
|
||||
process.stderr.write(
|
||||
`\n${undocumented.length} route(s) this module serves have no operation in swagger-fragment.json:\n` +
|
||||
`${undocumented.map((r) => ` - ${r}`).join('\n')}\n` +
|
||||
'Run `npm run swagger --prefix server` and commit the result (MODULE_API.md §2.8).\n',
|
||||
)
|
||||
failed = true
|
||||
}
|
||||
if (unserved.length > 0) {
|
||||
process.stderr.write(
|
||||
`\n${unserved.length} operation(s) in swagger-fragment.json are not routes this module serves:\n` +
|
||||
`${unserved.map((r) => ` - ${r}`).join('\n')}\n` +
|
||||
'A documented URL nobody serves is a client following the docs into a 404.\n',
|
||||
)
|
||||
failed = true
|
||||
}
|
||||
|
||||
if (failed) process.exit(1)
|
||||
|
||||
const contents = serialize(added)
|
||||
if (process.argv.includes('--check')) {
|
||||
const current = fs.existsSync(MANIFEST) ? fs.readFileSync(MANIFEST, 'utf8').replace(/\r\n/g, '\n') : null
|
||||
if (current !== contents) {
|
||||
process.stderr.write(
|
||||
'\nroutes.manifest.json is stale. The URLs this module serves changed — regenerate it and\n' +
|
||||
'commit the result so the move is reviewed rather than merged as mechanical.\n',
|
||||
)
|
||||
process.exit(1)
|
||||
}
|
||||
process.stdout.write(`routes.manifest.json is current — ${added.length} routes, all documented\n`)
|
||||
return
|
||||
}
|
||||
|
||||
fs.writeFileSync(MANIFEST, contents)
|
||||
process.stdout.write(`wrote routes.manifest.json — ${added.length} routes, all documented\n`)
|
||||
}
|
||||
|
||||
if (require.main === module) main()
|
||||
|
||||
module.exports = { diffManifests, coverage, serialize, MANIFEST, FRAGMENT }
|
||||
273
server/test/checkBundle.test.js
Normal file
273
server/test/checkBundle.test.js
Normal file
@@ -0,0 +1,273 @@
|
||||
// The bundle check, checked.
|
||||
//
|
||||
// `scripts/checkBundle.js` exists because of a failure this module has not had
|
||||
// and does not intend to: Module-uo's v1.0.0 shipped without `server/commands/`
|
||||
// and died at the register stage on the operator's box. A check written in
|
||||
// response to one bug is worth exactly as much as its coverage of that bug, so
|
||||
// the first two tests below are that bug, in both modes — a list that has stopped
|
||||
// covering what the entry point reaches, and a tarball with the file missing from
|
||||
// it — and the third pair is this module's own version of it, a runtime
|
||||
// dependency declared and not packed.
|
||||
//
|
||||
// **Every fixture is a template literal, and that is load-bearing** — the same
|
||||
// reason checkImports.test.js gives. `scripts/checkImports.js` scans this
|
||||
// directory too, so an ordinary quoted string holding a relative require would
|
||||
// make this file fail that check. Templates are blanked by the stripper.
|
||||
|
||||
const test = require('node:test')
|
||||
const assert = require('node:assert')
|
||||
const fs = require('node:fs')
|
||||
const os = require('node:os')
|
||||
const path = require('node:path')
|
||||
|
||||
const {
|
||||
reachable,
|
||||
resolveFile,
|
||||
checkDeclaration,
|
||||
checkBundle,
|
||||
declaredServerPaths,
|
||||
runtimeDependencies,
|
||||
} = require('../scripts/checkBundle')
|
||||
|
||||
/**
|
||||
* Write a throwaway module tree: `files` under server/, `bundle` as its
|
||||
* ci/bundle.json, `pkg` as its server/package.json. Returns the module root.
|
||||
*/
|
||||
function fixture(files, bundle = { server: ['index.js'] }, pkg = null) {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'module-rust-bundle-'))
|
||||
for (const [name, source] of Object.entries(files)) {
|
||||
const file = path.join(root, 'server', name)
|
||||
fs.mkdirSync(path.dirname(file), { recursive: true })
|
||||
fs.writeFileSync(file, source)
|
||||
}
|
||||
fs.mkdirSync(path.join(root, 'ci'), { recursive: true })
|
||||
fs.writeFileSync(path.join(root, 'ci', 'bundle.json'), JSON.stringify(bundle))
|
||||
if (pkg) {
|
||||
fs.mkdirSync(path.join(root, 'server'), { recursive: true })
|
||||
fs.writeFileSync(path.join(root, 'server', 'package.json'), JSON.stringify(pkg))
|
||||
}
|
||||
return root
|
||||
}
|
||||
|
||||
const cleanup = (root) => fs.rmSync(root, { recursive: true, force: true })
|
||||
|
||||
// ── The regression this script was written for ─────────────────────────────
|
||||
|
||||
test('--check catches a directory the include list has stopped covering', () => {
|
||||
const root = fixture(
|
||||
{
|
||||
'index.js': `const r = require('./router/public/rust.router')`,
|
||||
'router/public/rust.router.js': `module.exports = {}`,
|
||||
},
|
||||
{ server: ['index.js'] }, // `router` missing — exactly Module-uo's v1.0.0
|
||||
)
|
||||
try {
|
||||
const { uncovered } = checkDeclaration(root)
|
||||
assert.strictEqual(uncovered.size, 1)
|
||||
assert.ok(uncovered.has('server/router'))
|
||||
} finally {
|
||||
cleanup(root)
|
||||
}
|
||||
})
|
||||
|
||||
test('--bundle catches the file missing from an assembled tarball', () => {
|
||||
const root = fixture({ 'index.js': `require('./router/public/rust.router')` })
|
||||
try {
|
||||
const { missing } = checkBundle(root)
|
||||
assert.strictEqual(missing.length, 1)
|
||||
assert.strictEqual(missing[0].specifier, './router/public/rust.router')
|
||||
} finally {
|
||||
cleanup(root)
|
||||
}
|
||||
})
|
||||
|
||||
// ── This module's own version of that failure ──────────────────────────────
|
||||
//
|
||||
// The release packs no node_modules because the shipped half declares no
|
||||
// dependencies (org lead, phase 2). The value of that decision is entirely in
|
||||
// the day it stops being true being a LOUD day, so both modes ask.
|
||||
|
||||
test('--check reports a runtime dependency the release would not pack', () => {
|
||||
const root = fixture({ 'index.js': `module.exports = 1` }, { server: ['index.js'] }, {
|
||||
name: 'x',
|
||||
dependencies: { ws: '^8.21.0' },
|
||||
})
|
||||
try {
|
||||
assert.deepStrictEqual(checkDeclaration(root).dependencies, ['ws'])
|
||||
} finally {
|
||||
cleanup(root)
|
||||
}
|
||||
})
|
||||
|
||||
test('--bundle reports a dependency the assembled bundle declares and does not carry', () => {
|
||||
const root = fixture({ 'index.js': `module.exports = 1` }, { server: ['index.js'] }, {
|
||||
name: 'x',
|
||||
dependencies: { ws: '^8.21.0' },
|
||||
})
|
||||
try {
|
||||
assert.deepStrictEqual(checkBundle(root).dependencies, ['ws'])
|
||||
} finally {
|
||||
cleanup(root)
|
||||
}
|
||||
})
|
||||
|
||||
test('devDependencies are not runtime dependencies', () => {
|
||||
// express, express-validator and swagger-autogen are all here and none of them
|
||||
// ships: the shipped half is handed express on `ctx` (§2.3). A check that
|
||||
// confused the two would fail on a correct repo, which is the one way to make
|
||||
// everyone stop reading it.
|
||||
const root = fixture({ 'index.js': `module.exports = 1` }, { server: ['index.js'] }, {
|
||||
name: 'x',
|
||||
devDependencies: { express: '^4.19.2' },
|
||||
})
|
||||
try {
|
||||
assert.deepStrictEqual(runtimeDependencies(root), [])
|
||||
} finally {
|
||||
cleanup(root)
|
||||
}
|
||||
})
|
||||
|
||||
// ── It has to reach requires that are not at the top level ─────────────────
|
||||
|
||||
test('follows requires written inside a function', () => {
|
||||
// index.js requires inside `register()` because require order is load-bearing:
|
||||
// `core.init(ctx)` has to run before anything under router/ is required. A
|
||||
// check that only saw file-scope requires would miss every router this module
|
||||
// has.
|
||||
const root = fixture(
|
||||
{
|
||||
'index.js': `module.exports = function register(ctx) { const r = require('./router/a') }`,
|
||||
'router/a.js': `module.exports = {}`,
|
||||
},
|
||||
{ server: ['index.js', 'router'] },
|
||||
)
|
||||
try {
|
||||
assert.strictEqual(checkDeclaration(root).uncovered.size, 0)
|
||||
assert.strictEqual(checkBundle(root).missing.length, 0)
|
||||
} finally {
|
||||
cleanup(root)
|
||||
}
|
||||
})
|
||||
|
||||
test('follows requires transitively, not just one hop', () => {
|
||||
const root = fixture(
|
||||
{
|
||||
'index.js': `require('./a')`,
|
||||
'a.js': `require('./b')`,
|
||||
'b.js': `require('./deep/c')`,
|
||||
'deep/c.js': `module.exports = {}`,
|
||||
},
|
||||
{ server: ['index.js', 'a.js', 'b.js'] }, // `deep` missing
|
||||
)
|
||||
try {
|
||||
assert.ok(checkDeclaration(root).uncovered.has('server/deep'))
|
||||
} finally {
|
||||
cleanup(root)
|
||||
}
|
||||
})
|
||||
|
||||
// ── Resolution has to match Node's, or it invents failures ─────────────────
|
||||
|
||||
test('resolves a directory to its index.js', () => {
|
||||
const root = fixture(
|
||||
{ 'index.js': `require('./boot')`, 'boot/index.js': `module.exports = {}` },
|
||||
{ server: ['index.js', 'boot'] },
|
||||
)
|
||||
try {
|
||||
assert.strictEqual(checkDeclaration(root).uncovered.size, 0)
|
||||
} finally {
|
||||
cleanup(root)
|
||||
}
|
||||
})
|
||||
|
||||
test('resolves a .json dependency, and does not try to parse it for requires', () => {
|
||||
// server/index.js's last line requires ../module.json, which is why this case
|
||||
// is not hypothetical and why `generated` is in the declared list at all.
|
||||
const root = fixture(
|
||||
{ 'index.js': `require('./data/atlas.json')`, 'data/atlas.json': `{"a":1}` },
|
||||
{ server: ['index.js', 'data'] },
|
||||
)
|
||||
try {
|
||||
const { uncovered, missing } = checkDeclaration(root)
|
||||
assert.strictEqual(missing.length, 0)
|
||||
assert.strictEqual(uncovered.size, 0)
|
||||
} finally {
|
||||
cleanup(root)
|
||||
}
|
||||
})
|
||||
|
||||
test('survives a require cycle', () => {
|
||||
const root = fixture(
|
||||
{ 'index.js': `require('./a')`, 'a.js': `require('./index')` },
|
||||
{ server: ['index.js', 'a.js'] },
|
||||
)
|
||||
try {
|
||||
assert.strictEqual(checkDeclaration(root).uncovered.size, 0)
|
||||
} finally {
|
||||
cleanup(root)
|
||||
}
|
||||
})
|
||||
|
||||
test('a specifier that resolves to nothing is reported, not thrown', () => {
|
||||
const root = fixture({ 'index.js': `require('./gone')` })
|
||||
try {
|
||||
const { missing } = checkDeclaration(root)
|
||||
assert.strictEqual(missing.length, 1)
|
||||
assert.strictEqual(missing[0].specifier, './gone')
|
||||
} finally {
|
||||
cleanup(root)
|
||||
}
|
||||
})
|
||||
|
||||
test('prose describing a require is not a require', () => {
|
||||
// The failure mode checkImports.js hit the first time it ran: index.js's own
|
||||
// header explains why it must never require express, and comments in this repo
|
||||
// name module paths constantly.
|
||||
const root = fixture(
|
||||
{ 'index.js': `// this file used to require('./router/gone')\nmodule.exports = 1` },
|
||||
{ server: ['index.js'] },
|
||||
)
|
||||
try {
|
||||
assert.strictEqual(checkDeclaration(root).missing.length, 0)
|
||||
} finally {
|
||||
cleanup(root)
|
||||
}
|
||||
})
|
||||
|
||||
test("node_modules inside a bundle is npm's business, not this check's", () => {
|
||||
// Nothing ships one today. The skip stays so that the day a dependency does
|
||||
// arrive, this is not also the thing that breaks.
|
||||
const root = fixture({
|
||||
'index.js': `module.exports = 1`,
|
||||
'node_modules/ws/index.js': `require('./lib/that-npm-owns')`,
|
||||
})
|
||||
try {
|
||||
assert.strictEqual(checkBundle(root).missing.length, 0)
|
||||
} finally {
|
||||
cleanup(root)
|
||||
}
|
||||
})
|
||||
|
||||
// ── And the real repo, which is the check that actually gates a release ────
|
||||
|
||||
test('the real ci/bundle.json covers everything the real entry point reaches', () => {
|
||||
const { uncovered, missing, reached, dependencies } = checkDeclaration()
|
||||
assert.deepStrictEqual([...uncovered.keys()], [])
|
||||
assert.deepStrictEqual(missing, [])
|
||||
assert.deepStrictEqual(dependencies, [])
|
||||
assert.ok(reached > 1, 'the walk should reach more than the entry point itself')
|
||||
})
|
||||
|
||||
test('every path ci/bundle.json declares exists', () => {
|
||||
// A list naming a path that has moved packs nothing and says nothing — `cp` in
|
||||
// the release would fail, but only after the tag had been pushed.
|
||||
for (const p of declaredServerPaths()) {
|
||||
assert.ok(fs.existsSync(p), `ci/bundle.json names ${p}, which does not exist`)
|
||||
}
|
||||
})
|
||||
|
||||
test('the entry point is reachable from the declared list', () => {
|
||||
const entry = path.resolve(__dirname, '..', 'index.js')
|
||||
assert.ok(reachable(entry).files.includes(entry))
|
||||
assert.ok(resolveFile(path.dirname(entry), './core'))
|
||||
})
|
||||
177
server/test/frozenManifest.test.js
Normal file
177
server/test/frozenManifest.test.js
Normal file
@@ -0,0 +1,177 @@
|
||||
// The frozen manifest's derivation, checked.
|
||||
//
|
||||
// `scripts/frozenManifest.js` runs in one place — a CI job with a whole core
|
||||
// checked out beside it — so it is the least-exercised piece of machinery in this
|
||||
// repo, and it is the piece that decides whether the URLs this module claims are
|
||||
// the URLs it serves (MODULE_API.md §5.3). Its three answers are pure functions of
|
||||
// two manifests and a fragment, so all three are asked here, with fixtures rather
|
||||
// than a clone.
|
||||
//
|
||||
// What is deliberately NOT asserted here: the numbers. `routes.manifest.json`'s
|
||||
// routes are proved by the job that generates them from a real core, and a copy of
|
||||
// that count in this file would only ever be a second thing to update.
|
||||
|
||||
const test = require('node:test')
|
||||
const assert = require('node:assert')
|
||||
const fs = require('node:fs')
|
||||
const path = require('node:path')
|
||||
|
||||
const { diffManifests, coverage, MANIFEST, FRAGMENT } = require('../scripts/frozenManifest')
|
||||
|
||||
const manifest = (public_ = [], internal = []) => ({ public: public_, internal })
|
||||
const get = (p) => ({ method: 'GET', path: p })
|
||||
|
||||
test("the module's routes are the ones a core gains by loading it", () => {
|
||||
const before = manifest([get('/api/v1/public/settings')])
|
||||
const after = manifest([get('/api/v1/public/settings'), get('/api/v1/public/rust/servers')])
|
||||
|
||||
const { added, removed } = diffManifests(before, after)
|
||||
assert.deepStrictEqual(removed, [])
|
||||
assert.deepStrictEqual(added, [{ method: 'GET', path: '/api/v1/public/rust/servers', tier: 'public' }])
|
||||
})
|
||||
|
||||
test('a route core loses to the module is reported, not quietly absorbed', () => {
|
||||
// The failure this exists for, and the one phase 1 could only check by reading:
|
||||
// core mounts several routes at the TIER ROOT (/status, /version) that the
|
||||
// loader's collision probe cannot see, so a module whose mount displaced one
|
||||
// would not show up as an addition — the URL is unchanged — and a check that
|
||||
// only looked at what appeared would call it clean.
|
||||
const before = manifest([get('/api/v1/public/settings'), get('/api/v1/public/status')])
|
||||
const after = manifest([get('/api/v1/public/settings')])
|
||||
|
||||
const { removed } = diffManifests(before, after)
|
||||
assert.deepStrictEqual(removed, ['public GET /api/v1/public/status'])
|
||||
})
|
||||
|
||||
test('a route whose METHOD changed counts as removed and added', () => {
|
||||
const { added, removed } = diffManifests(
|
||||
manifest([{ method: 'POST', path: '/api/v1/admin/thing' }]),
|
||||
manifest([{ method: 'PUT', path: '/api/v1/admin/thing' }]),
|
||||
)
|
||||
assert.deepStrictEqual(removed, ['public POST /api/v1/admin/thing'])
|
||||
assert.strictEqual(added.length, 1)
|
||||
})
|
||||
|
||||
test('the internal app is diffed too, and keeps its own tier', () => {
|
||||
const { added } = diffManifests(
|
||||
manifest([], [get('/internal/health')]),
|
||||
manifest([], [get('/internal/health'), get('/internal/rust/thing')]),
|
||||
)
|
||||
assert.deepStrictEqual(added, [{ method: 'GET', path: '/internal/rust/thing', tier: 'internal' }])
|
||||
})
|
||||
|
||||
test('added routes are sorted, so the committed file does not churn on traversal order', () => {
|
||||
const { added } = diffManifests(
|
||||
manifest([]),
|
||||
manifest([get('/b'), get('/a'), { method: 'POST', path: '/a' }]),
|
||||
)
|
||||
assert.deepStrictEqual(
|
||||
added.map((r) => `${r.method} ${r.path}`),
|
||||
['GET /a', 'GET /b', 'POST /a'],
|
||||
)
|
||||
})
|
||||
|
||||
// ── coverage: the route ⇄ fragment agreement ────────────────────────────────
|
||||
|
||||
const fragment = (paths) => ({ paths })
|
||||
|
||||
test('a served route with no documented operation is named', () => {
|
||||
const { undocumented, unserved } = coverage([get('/api/v1/public/rust/servers')], fragment({}))
|
||||
assert.deepStrictEqual(undocumented, ['GET /api/v1/public/rust/servers'])
|
||||
assert.deepStrictEqual(unserved, [])
|
||||
})
|
||||
|
||||
test('a documented operation nobody serves is named too', () => {
|
||||
// The direction core's own spec has no check for, which is how it accumulated
|
||||
// orphan tags and schemas describing routes that had moved out of it. A
|
||||
// documented URL nobody serves is a client following the docs into a 404.
|
||||
const { undocumented, unserved } = coverage([], fragment({ '/api/v1/public/rust/gone': { get: {} } }))
|
||||
assert.deepStrictEqual(undocumented, [])
|
||||
assert.deepStrictEqual(unserved, ['GET /api/v1/public/rust/gone'])
|
||||
})
|
||||
|
||||
test('express :params and OpenAPI {params} are the same route', () => {
|
||||
const { undocumented, unserved } = coverage(
|
||||
[{ method: 'DELETE', path: '/api/v1/admin/rust/servers/:id' }],
|
||||
fragment({ '/api/v1/admin/rust/servers/{id}': { delete: {} } }),
|
||||
)
|
||||
assert.deepStrictEqual(undocumented, [])
|
||||
assert.deepStrictEqual(unserved, [])
|
||||
})
|
||||
|
||||
test('methods are matched, not just paths', () => {
|
||||
const { undocumented, unserved } = coverage(
|
||||
[{ method: 'POST', path: '/api/v1/admin/rust/servers/:id/test' }],
|
||||
fragment({ '/api/v1/admin/rust/servers/{id}/test': { get: {} } }),
|
||||
)
|
||||
assert.deepStrictEqual(undocumented, ['POST /api/v1/admin/rust/servers/{id}/test'])
|
||||
assert.deepStrictEqual(unserved, ['GET /api/v1/admin/rust/servers/{id}/test'])
|
||||
})
|
||||
|
||||
// ── the committed artifacts, against each other ─────────────────────────────
|
||||
//
|
||||
// These two files are generated together by a job that has a real core; here
|
||||
// there is no core, so what can still be asked is whether they agree with each
|
||||
// other. If they do not, one of them was committed without the other.
|
||||
|
||||
test('every route in the committed manifest has a committed operation', () => {
|
||||
const { routes } = JSON.parse(fs.readFileSync(MANIFEST, 'utf8'))
|
||||
const spec = JSON.parse(fs.readFileSync(FRAGMENT, 'utf8'))
|
||||
const { undocumented, unserved } = coverage(routes, spec)
|
||||
assert.deepStrictEqual(undocumented, [], 'routes.manifest.json lists routes swagger-fragment.json does not document')
|
||||
assert.deepStrictEqual(unserved, [], 'swagger-fragment.json documents operations routes.manifest.json does not list')
|
||||
})
|
||||
|
||||
test('the fragment carries only the three sections §6.1a allows', () => {
|
||||
const spec = JSON.parse(fs.readFileSync(FRAGMENT, 'utf8'))
|
||||
assert.deepStrictEqual(Object.keys(spec).sort(), ['components', 'paths', 'tags'])
|
||||
assert.deepStrictEqual(Object.keys(spec.components), ['schemas'])
|
||||
})
|
||||
|
||||
test("the fragment defines only namespaced schemas, and redefines none of core's", () => {
|
||||
const spec = JSON.parse(fs.readFileSync(FRAGMENT, 'utf8'))
|
||||
for (const name of Object.keys(spec.components.schemas)) {
|
||||
assert.match(name, /^Rust[A-Z]/, `${name} is not namespaced — core wins the collision and drops it (§6.1a)`)
|
||||
}
|
||||
// Anything this fragment REFERENCES and does not define has to be one of
|
||||
// core's shared schemas, which resolve in the merged document — that is the
|
||||
// whole point of a fragment. A typo'd $ref is otherwise invisible until a
|
||||
// reader opens /api/docs.json and finds a dangling pointer.
|
||||
const refs = JSON.stringify(spec.paths).match(/#[/]components[/]schemas[/]([A-Za-z0-9_]+)/g) || []
|
||||
const shared = ['Error', 'ValidationError']
|
||||
for (const name of new Set(refs.map((r) => r.split('/').pop()))) {
|
||||
const resolvable = Object.hasOwn(spec.components.schemas, name) || shared.includes(name)
|
||||
assert.ok(resolvable, `$ref to ${name} resolves to nothing — not defined here, not one of core's shared schemas`)
|
||||
}
|
||||
})
|
||||
|
||||
test('every path in the fragment is fully qualified', () => {
|
||||
const spec = JSON.parse(fs.readFileSync(FRAGMENT, 'utf8'))
|
||||
for (const p of Object.keys(spec.paths)) {
|
||||
// §6.1a: core merges the fragment verbatim and never re-derives a prefix, so
|
||||
// a router-relative path here is a path nothing serves.
|
||||
assert.match(p, /^\/api\/v1\/(public|admin|player)\//, `${p} is not a fully-qualified URL`)
|
||||
assert.doesNotMatch(p, /\/$/, `${p} has a trailing slash — no client calls that URL`)
|
||||
}
|
||||
})
|
||||
|
||||
test("the manifest and the module's declared mounts agree", () => {
|
||||
const { routes } = JSON.parse(fs.readFileSync(MANIFEST, 'utf8'))
|
||||
const { mounts } = JSON.parse(fs.readFileSync(path.join(__dirname, '..', '..', 'module.json'), 'utf8'))
|
||||
|
||||
const declared = []
|
||||
for (const [tier, prefixes] of Object.entries(mounts)) {
|
||||
for (const prefix of prefixes) declared.push(`/api/v1/${tier}${prefix}/`)
|
||||
}
|
||||
|
||||
// Every route this module serves is under a prefix it declared. There is no
|
||||
// exception here yet, and that is the point of asserting it now: phase 6 adds
|
||||
// the `admin.users.detail` extension slot, whose routes live under core's
|
||||
// `/api/v1/admin/users/` rather than under any mount of ours (§2.4). When that
|
||||
// arrives this test must grow the exception deliberately, rather than a route
|
||||
// outside every declared mount arriving unnoticed.
|
||||
for (const route of routes) {
|
||||
const under = declared.some((d) => route.path.startsWith(d))
|
||||
assert.ok(under, `${route.method} ${route.path} is served from outside every mount module.json declares`)
|
||||
}
|
||||
})
|
||||
Reference in New Issue
Block a user