From 4f268f7ffb5a4bf06734418a8df03a3937028c5a Mon Sep 17 00:00:00 2001 From: wtclaude Date: Fri, 25 Sep 2026 22:49:47 -0500 Subject: [PATCH] ci(bundle): compose schema 2 for ServUO and Rust beside schema 1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Module-rust phase 18, step 2 of docs/modules/rust/PLAN.md §34.2.7 (D146, D147). The compose job now writes three streams on the bundles branch: current.json schema 1, ServUO, exactly as before v2/servuo/ schema 2, game "servuo" v2/rust/ schema 2, game "rust" (Rust-Link + Rust-Plugins) The gates move into .gitea/scripts/compose-bundles.sh, so they can be run by hand against the real release API or a mock without a runner. Each game composes independently: a Rust failure still lets a ServUO bundle publish, and the run goes red afterwards. A game whose repos have never released composes nothing and does not fail, which is where Rust is until its cutover. A ServUO matrix carries the same tag at both schemas, so the first run publishes v2/servuo/bundle-2026.09.15.json rather than a second name for the pair. Schema 1 stops being composed on 2027-01-01 and is left frozen. Tested in debian:12 against the real releases (v2/servuo published as 2026.09.15, schema 1 unchanged, Rust skipped; rerun is a no-op; past the retirement date schema 1 is frozen) and against a mock Gitea for Rust (document composed, protocol mismatch refused, missing manifest key refused, rerun a no-op, ServUO failing independently). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01E14m6SuuY6i1vASFeGDBeY --- .gitea/scripts/compose-bundles.sh | 437 ++++++++++++++++++++++++++++++ .gitea/workflows/bundle.yml | 415 +++++++--------------------- bundles/README.md | 75 ++++- 3 files changed, 600 insertions(+), 327 deletions(-) create mode 100755 .gitea/scripts/compose-bundles.sh diff --git a/.gitea/scripts/compose-bundles.sh b/.gitea/scripts/compose-bundles.sh new file mode 100755 index 0000000..f40dfae --- /dev/null +++ b/.gitea/scripts/compose-bundles.sh @@ -0,0 +1,437 @@ +#!/usr/bin/env bash +# Compose every bundle stream from the latest component releases. +# +# Called by .gitea/workflows/bundle.yml, which owns checkout, publishing and the +# stale-component check; this script owns the gates and the documents. It lives +# in a file rather than inline so it can be run by hand against the real release +# API — which is how it is tested — without a runner. +# +# ── The streams (docs/modules/rust/PLAN.md §34.2.2, D146/D147) ─────────────── +# +# current.json, bundle-.json schema 1, ServUO (until SCHEMA1_RETIRES) +# v2/servuo/current.json, bundle-.json schema 2, game "servuo" +# v2/rust/current.json, bundle-.json schema 2, game "rust" +# +# A schema-2 document names ONE game. The two games release on their own +# schedules, and a document naming both would hand a ServUO host a new bundle +# every time a Rust plugin shipped. +# +# Schema 1 is still composed because every installer already in the field reads +# only `current.json` and refuses any schema but 1. After SCHEMA1_RETIRES it is +# left FROZEN at its last bundle rather than deleted, so an old installer still +# resolves something and every bundle-.json stays pinnable. +# +# ── One matrix, one tag ────────────────────────────────────────────────────── +# A ServUO matrix published at both schemas carries the SAME tag in both, so a +# `--bundle ` an operator wrote down means one pair whichever installer +# reads it. The first run after schema 2 lands is the case this is for: the +# ServUO matrix has not changed, so v2/servuo takes the tag schema 1 already +# gave it rather than inventing a second name for the same pair. +# +# ── Failure is per game ────────────────────────────────────────────────────── +# A Rust release with a protocol mismatch must not stop a ServUO bundle from +# publishing, and the reverse. Each game composes in its own subshell; what +# succeeded is written, and the script exits 1 at the end if anything failed, so +# the run is still red. +# +# A game none of whose components has released yet composes nothing, and that is +# NOT a failure — it is the state Rust is in until its first cutover. +# +# ── Interface ──────────────────────────────────────────────────────────────── +# PUBLISHED the bundles-branch worktree (read, and written on change) +# WORK scratch directory +# GITEA_HOST default gitea.whitlocktech.com (GITEA_BASE overrides the whole URL) +# TODAY YYYY-MM-DD, default today UTC (tests override it) +# SCHEMA1_RETIRES YYYY-MM-DD, default 2027-01-01 (PLAN.md §34.4) +# +# Results, for the workflow: +# $WORK/result.env changed=true|false, commit_subject=… +# $WORK/summary.md the job summary's body +# $WORK/stale.tsv \t, one per resolved component + +set -euo pipefail + +: "${PUBLISHED:?PUBLISHED must name the bundles-branch worktree}" +: "${WORK:?WORK must name a scratch directory}" +GITEA_HOST="${GITEA_HOST:-gitea.whitlocktech.com}" +# Overridable so the script can be pointed at a mock Gitea (how the Rust path was +# tested before either Rust repository had released). +GITEA_BASE="${GITEA_BASE:-https://${GITEA_HOST}}" +TODAY="${TODAY:-$(date -u +%Y-%m-%d)}" +SCHEMA1_RETIRES="${SCHEMA1_RETIRES:-2027-01-01}" + +SERVUO_LINK_REPO="RunicGateway/link" +SERVUO_OVERLAY_REPO="RunicGateway/servuo-plugins" +RUST_LINK_REPO="RunicGateway/Rust-Link" +RUST_PLUGIN_REPO="RunicGateway/Rust-Plugins" + +# Fixed top-level directories inside each payload tarball. Deliberately NOT +# versioned — a versioned prefix would mean parsing the version out of a path in +# order to read the manifest that declares the version. +SERVUO_OVERLAY_PREFIX="runicgateway-overlay" +RUST_PLUGIN_PREFIX="runicgateway-rust-plugin" + +mkdir -p "$WORK" "$PUBLISHED" +: > "$WORK/summary.md" +: > "$WORK/stale.tsv" +: > "$WORK/published.txt" + +fail() { echo "::error::$*" >&2; exit 1; } +note() { printf -- '%s\n' "$*" >> "$WORK/summary.md"; } + +# ── Release resolution ─────────────────────────────────────────────────────── +# Read ANONYMOUSLY, on purpose: these are exactly the requests the shipped +# installer and the egg make, from a host with no Gitea credentials. A repo +# flipped to private fails CI here instead of on an operator's machine. +# +# Returns 0 with $WORK/-release.json written, or 2 when the repo has never +# released (Gitea answers /releases/latest with 404). Anything else is fatal. +resolve_latest() { + local slug="$1" key="$2" code tag + code="$(curl -sS -o "$WORK/${key}-release.json" -w '%{http_code}' \ + "${GITEA_BASE}/api/v1/repos/${slug}/releases/latest")" || fail "could not reach ${slug}'s releases" + case "$code" in + 200) ;; + 404) return 2 ;; + *) fail "${slug} /releases/latest answered HTTP ${code}" ;; + esac + tag="$(jq -r '.tag_name' "$WORK/${key}-release.json")" + [ -n "$tag" ] && [ "$tag" != "null" ] || fail "${slug}'s latest release has no tag" + printf '%s\t%s\n' "$slug" "$tag" >> "$WORK/stale.tsv" + echo "==> ${slug} latest: ${tag}" +} + +release_tag() { jq -r '.tag_name' "$WORK/$1-release.json"; } + +# ── GATE 2 (installer PLAN.md §7.1): assets exist, checksums match ─────────── +# Every asset a bundle will reference is downloaded and verified against the +# SHA256SUMS published beside it. SHA256SUMS is the trust anchor for these +# deliberately UNSIGNED artifacts, and the installer and the egg verify against +# the hashes THIS job records; a hash copied from a file nobody checked would +# make the chain decorative. +# +# `sha256sum -c` catches a SHA256SUMS entry with no asset. The reverse — an asset +# with no entry — is checked by name, because -c would pass right over it. The +# `\*?` matches sha256sum's binary-mode marker. +verify_assets() { + local key="$1" dir="$WORK/$1" sums_url name url + rm -rf "$dir"; mkdir -p "$dir" + sums_url="$(jq -r '.assets[] | select(.name == "SHA256SUMS") | .browser_download_url' "$WORK/${key}-release.json")" + [ -n "$sums_url" ] && [ "$sums_url" != "null" ] \ + || fail "${key} release has no SHA256SUMS asset — nothing to verify against" + curl -sSfL -o "${dir}/SHA256SUMS" "$sums_url" + + jq -r '.assets[] | select(.name != "SHA256SUMS") | "\(.name)\t\(.browser_download_url)"' \ + "$WORK/${key}-release.json" > "${dir}/asset-list.tsv" + [ -s "${dir}/asset-list.tsv" ] || fail "${key} release carries no assets besides SHA256SUMS" + + while IFS=$'\t' read -r name url; do + [ -n "$name" ] || continue + echo " fetching ${key}/${name}" + curl -sSfL -o "${dir}/${name}" "$url" + grep -qE "[ \t]\*?${name}\$" "${dir}/SHA256SUMS" \ + || fail "${key} asset ${name} has no entry in that release's SHA256SUMS" + done < "${dir}/asset-list.tsv" + + ( cd "$dir" && sha256sum -c SHA256SUMS ) \ + || fail "${key} assets do not match the SHA256SUMS published with them" + echo "==> ${key}: all assets present and verified" +} + +sha_of() { sha256sum "$1" | cut -d' ' -f1; } + +# One asset as a bundle `{name,url,sha256}` object. +asset_json() { + local key="$1" name="$2" url + url="$(awk -F'\t' -v n="$name" '$1 == n { print $2 }' "$WORK/${key}/asset-list.tsv")" + jq -n --arg name "$name" --arg url "$url" --arg sha "$(sha_of "$WORK/${key}/${name}")" \ + '{ name: $name, url: $url, sha256: $sha }' +} + +# The single tarball a payload release carries, by name. Exactly one: a second +# .tar.gz would leave the installer guessing which one it deploys. +single_tarball() { + local key="$1" count + count="$(awk -F'\t' '$1 ~ /\.tar\.gz$/' "$WORK/${key}/asset-list.tsv" | wc -l)" + [ "$count" -eq 1 ] || fail "expected exactly 1 .tar.gz in the ${key} release, found ${count}" + awk -F'\t' '$1 ~ /\.tar\.gz$/ { printf "%s", $1 }' "$WORK/${key}/asset-list.tsv" +} + +# ── GATE 1 (installer PLAN.md §7.1): both halves speak one protocol ────────── +# The sidecar side is PROTOCOL_VERSION in sidecar/src/main.rs, read at the +# RELEASE TAG — not from the binary, which would mean executing a downloaded +# artifact (and, for older releases, one with no way to answer). Both link and +# Rust-Link keep the constant at that path. +sidecar_protocol() { + local slug="$1" tag="$2" key="$3" p + curl -sSfL -o "$WORK/${key}-main.rs" \ + "${GITEA_BASE}/${slug}/raw/tag/${tag}/sidecar/src/main.rs" + p="$(sed -nE 's/^[[:space:]]*pub const PROTOCOL_VERSION[^=]*=[[:space:]]*([0-9]+).*/\1/p' "$WORK/${key}-main.rs" | head -1)" + # Empty means the constant moved. "Could not read" must never read as "matches". + [ -n "$p" ] || fail "could not read PROTOCOL_VERSION from ${slug}@${tag}:sidecar/src/main.rs — has the constant moved? Gate 1 cannot be skipped." + printf '%s' "$p" +} + +# Unpack a payload tarball and return the path of its manifest.json, having +# checked the manifest names the version it was released under. +payload_manifest() { + local key="$1" tarball="$2" prefix="$3" tag="$4" m v + rm -rf "$WORK/${key}-x"; mkdir -p "$WORK/${key}-x" + tar -xzf "$WORK/${key}/${tarball}" -C "$WORK/${key}-x" + m="$WORK/${key}-x/${prefix}/manifest.json" + [ -f "$m" ] || fail "the ${key} tarball has no ${prefix}/manifest.json — the installer and the egg resolve it at that exact path" + v="$(jq -r '.version' "$m")" + # A manifest that disagrees with its tag means the release stamped one version + # and tagged another; every record of "what is installed" would then be wrong. + [ "$v" = "${tag#v}" ] || fail "${key} manifest says version ${v} but the release is tagged ${tag}" + printf '%s' "$m" +} + +# ── Publishing helpers ─────────────────────────────────────────────────────── +# `bundle` and `generated` describe the RUN, not the matrix. Comparing them would +# make every nightly cron look like a change and commit a dated duplicate of the +# same matrix forever, so only what an installer acts on is compared. +same_content() { + local published="$1" content="$2" + [ -f "$published" ] || return 1 + cmp -s <(jq -S 'del(.bundle, .generated)' "$published") <(jq -S '.' "$content") +} + +# A fresh date tag that names no bundle in any of the given directories. Two +# bundles on one day get .2, .3, … so a tag always names exactly one matrix. +fresh_tag() { + local base tag n=1 d taken + base="$(date -u -d "$TODAY" +%Y.%m.%d)" + tag="$base" + while :; do + taken=false + for d in "$@"; do [ -f "${d}/bundle-${tag}.json" ] && taken=true; done + [ "$taken" = false ] && break + n=$((n + 1)); tag="${base}.${n}" + done + printf '%s' "$tag" +} + +# Write /bundle-.json and /current.json from a content document. +# The header keys go first so a person reading the file sees what it is. +publish_doc() { + local dir="$1" tag="$2" content="$3" + mkdir -p "$dir" + jq --arg bundle "$tag" --arg generated "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \ + '(if has("game") then { schema: .schema, game: .game } else { schema: .schema } end) + + { bundle: $bundle, generated: $generated } + del(.schema, .game)' \ + "$content" > "${dir}/bundle-${tag}.json" + cp "${dir}/bundle-${tag}.json" "${dir}/current.json" + local rel="${dir#"$PUBLISHED"}"; rel="${rel#/}" + echo "${rel:+${rel}/}bundle-${tag}.json" >> "$WORK/published.txt" +} + +# ── ServUO: schema 1 and v2/servuo ─────────────────────────────────────────── +compose_servuo() { + resolve_latest "$SERVUO_LINK_REPO" servuo-link \ + || fail "${SERVUO_LINK_REPO} has no published release" + resolve_latest "$SERVUO_OVERLAY_REPO" servuo-overlay \ + || fail "${SERVUO_OVERLAY_REPO} has no published release" + local link_tag overlay_tag + link_tag="$(release_tag servuo-link)"; overlay_tag="$(release_tag servuo-overlay)" + + verify_assets servuo-link + verify_assets servuo-overlay + + # link's binaries onto platform keys. An unrecognized name is a hard failure, + # so a new target in link's release.yml reddens this job instead of silently + # vanishing from every bundle; a missing required key fails the same way. + local assets='{}' name url plat + while IFS=$'\t' read -r name url; do + [ -n "$name" ] || continue + case "$name" in + *-linux-x86_64) plat=linux-x86_64 ;; + *-linux-aarch64) plat=linux-aarch64 ;; + *-windows-x86_64.exe) plat=windows-x86_64 ;; + *) fail "unrecognized link asset '${name}' — teach compose-bundles.sh this name or the bundle would silently omit it." ;; + esac + assets="$(jq --arg p "$plat" --argjson a "$(asset_json servuo-link "$name")" '. + { ($p): $a }' <<<"$assets")" + done < "$WORK/servuo-link/asset-list.tsv" + for plat in linux-x86_64 linux-aarch64 windows-x86_64; do + jq -e --arg p "$plat" 'has($p)' <<<"$assets" >/dev/null \ + || fail "link release is missing a ${plat} binary; the installer ships for all three" + done + + local tarball manifest overlay_protocol link_protocol + [ "$(wc -l < "$WORK/servuo-overlay/asset-list.tsv")" -eq 1 ] \ + || fail "expected exactly 1 overlay asset besides SHA256SUMS" + tarball="$(single_tarball servuo-overlay)" + manifest="$(payload_manifest servuo-overlay "$tarball" "$SERVUO_OVERLAY_PREFIX" "$overlay_tag")" + overlay_protocol="$(jq -r '.protocol' "$manifest")" + link_protocol="$(sidecar_protocol "$SERVUO_LINK_REPO" "$link_tag" servuo-link)" + echo "==> servuo: sidecar ${link_tag} protocol=${link_protocol} | overlay ${overlay_tag} protocol=${overlay_protocol}" + [ "$link_protocol" = "$overlay_protocol" ] || fail \ + "PROTOCOL MISMATCH — sidecar ${link_tag} speaks ${link_protocol}, overlay ${overlay_tag} declares ${overlay_protocol}. Refusing a bundle whose shard the sidecar rejects with 409. Fix: land the matching half and let its release cut, or bump servuo-plugins/overlay.toml." + + local common + common="$(jq -n \ + --arg link_repo "$SERVUO_LINK_REPO" --arg link_tag "$link_tag" --argjson assets "$assets" \ + --arg ov_repo "$SERVUO_OVERLAY_REPO" --arg ov_tag "$overlay_tag" \ + --arg ov_commit "$(jq -r '.commit' "$manifest")" \ + --arg ov_min "$(jq -r '.servuo.min_version' "$manifest")" \ + --arg ov_patched "$(jq -r '.servuo.patches_verified_against' "$manifest")" \ + --argjson ov_asset "$(asset_json servuo-overlay "$tarball")" \ + --argjson protocol "$link_protocol" \ + '{ protocol: $protocol, + sidecar: { repo: $link_repo, tag: $link_tag, version: ($link_tag | ltrimstr("v")), + protocol: $protocol, assets: $assets }, + overlay: { repo: $ov_repo, tag: $ov_tag, version: ($ov_tag | ltrimstr("v")), + commit: $ov_commit, protocol: $protocol, + servuo: { min_version: $ov_min, patches_verified_against: $ov_patched }, + asset: $ov_asset } }')" + + # Schema 1: byte-for-byte the shape every shipped installer parses. + jq '{ schema: 1, protocol: .protocol, link: .sidecar, overlay: .overlay }' \ + <<<"$common" > "$WORK/servuo-s1.json" + # Schema 2: the same pair, in the game-neutral shape (§34.2.2). The payload's + # `compat` is schema 1's `servuo` block. + jq '{ schema: 2, game: "servuo", protocol: .protocol, sidecar: .sidecar, + payload: ({ kind: "overlay" } + (.overlay | del(.servuo)) + { compat: .overlay.servuo }) }' \ + <<<"$common" > "$WORK/servuo-s2.json" + + local s1_active=false s1_changed=false s2_changed=false tag="" + [[ "$TODAY" < "$SCHEMA1_RETIRES" ]] && s1_active=true + if [ "$s1_active" = true ] && ! same_content "$PUBLISHED/current.json" "$WORK/servuo-s1.json"; then + s1_changed=true + fi + same_content "$PUBLISHED/v2/servuo/current.json" "$WORK/servuo-s2.json" || s2_changed=true + + if [ "$s1_changed" = false ] && [ "$s2_changed" = false ]; then + note "- **ServUO**: no change (link \`${link_tag}\`, overlay \`${overlay_tag}\`, protocol ${link_protocol})." + [ "$s1_active" = true ] || note " Schema 1 retired on ${SCHEMA1_RETIRES}; its \`current.json\` stays frozen." + return 0 + fi + + # One matrix, one tag: when schema 1 already names this pair, v2 borrows its + # tag — unless v2 somehow already holds that tag for different content, since + # a published bundle-.json is never rewritten. + if [ "$s1_active" = true ] && [ "$s1_changed" = false ] && [ -f "$PUBLISHED/current.json" ]; then + tag="$(jq -r '.bundle' "$PUBLISHED/current.json")" + if [ -f "$PUBLISHED/v2/servuo/bundle-${tag}.json" ]; then tag=""; fi + fi + [ -n "$tag" ] || tag="$(fresh_tag "$PUBLISHED" "$PUBLISHED/v2/servuo")" + + [ "$s1_changed" = true ] && publish_doc "$PUBLISHED" "$tag" "$WORK/servuo-s1.json" + [ "$s2_changed" = true ] && publish_doc "$PUBLISHED/v2/servuo" "$tag" "$WORK/servuo-s2.json" + local which="" + [ "$s1_changed" = true ] && which="schema 1" + [ "$s2_changed" = true ] && which="${which:+${which} + }schema 2" + note "- **ServUO**: published \`${tag}\` (${which}) — link \`${link_tag}\`, overlay \`${overlay_tag}\`, protocol ${link_protocol}." + echo "servuo ${tag}" >> "$WORK/published-games.txt" +} + +# ── Rust: v2/rust ──────────────────────────────────────────────────────────── +compose_rust() { + local rc=0 missing="" + resolve_latest "$RUST_LINK_REPO" rust-link || { rc=$?; [ "$rc" -eq 2 ] && missing="${RUST_LINK_REPO}"; } + rc=0 + resolve_latest "$RUST_PLUGIN_REPO" rust-plugin || { rc=$?; [ "$rc" -eq 2 ] && missing="${missing:+${missing} and }${RUST_PLUGIN_REPO}"; } + if [ -n "$missing" ]; then + echo "==> rust: no release yet from ${missing}, so nothing to compose (not a failure)" + note "- **Rust**: skipped — no release yet from ${missing}." + return 0 + fi + local link_tag plugin_tag + link_tag="$(release_tag rust-link)"; plugin_tag="$(release_tag rust-plugin)" + + verify_assets rust-link + verify_assets rust-plugin + + # Rust-Link's release: a binary per platform, the egg's launcher, and the egg + # itself. The egg is verified (gate 2 covers every asset) but is not part of + # the bundle — a panel admin imports it; nothing resolves it from here. + local assets='{}' launcher="" name url plat + while IFS=$'\t' read -r name url; do + [ -n "$name" ] || continue + case "$name" in + *-linux-x86_64) plat=linux-x86_64 ;; + *-windows-x86_64.exe) plat=windows-x86_64 ;; + with-sidecar.sh) launcher="$(asset_json rust-link "$name")"; continue ;; + egg-*.json) continue ;; + *) fail "unrecognized Rust-Link asset '${name}' — teach compose-bundles.sh this name or the bundle would silently omit it." ;; + esac + assets="$(jq --arg p "$plat" --argjson a "$(asset_json rust-link "$name")" '. + { ($p): $a }' <<<"$assets")" + done < "$WORK/rust-link/asset-list.tsv" + # No linux-aarch64: RustDedicated has no arm64 build (D149). + for plat in linux-x86_64 windows-x86_64; do + jq -e --arg p "$plat" 'has($p)' <<<"$assets" >/dev/null \ + || fail "Rust-Link release is missing a ${plat} binary (D149)" + done + [ -n "$launcher" ] || fail "Rust-Link release has no with-sidecar.sh — the egg's startup runs it (§34.2.6)" + + local tarball manifest plugin_protocol link_protocol + tarball="$(single_tarball rust-plugin)" + manifest="$(payload_manifest rust-plugin "$tarball" "$RUST_PLUGIN_PREFIX" "$plugin_tag")" + [ -f "$(dirname "$manifest")/RunicGateway.cs" ] \ + || fail "the Rust-Plugins tarball has no ${RUST_PLUGIN_PREFIX}/RunicGateway.cs" + plugin_protocol="$(jq -r '.protocol' "$manifest")" + link_protocol="$(sidecar_protocol "$RUST_LINK_REPO" "$link_tag" rust-link)" + echo "==> rust: sidecar ${link_tag} protocol=${link_protocol} | plugin ${plugin_tag} protocol=${plugin_protocol}" + [ "$link_protocol" = "$plugin_protocol" ] || fail \ + "PROTOCOL MISMATCH — Rust-Link ${link_tag} speaks ${link_protocol}, Rust-Plugins ${plugin_tag} declares ${plugin_protocol}. The game link has no 409: a mismatched plugin would mis-parse. Fix: land the matching half and let its release cut, or bump Rust-Plugins/overlay.toml." + + jq -n \ + --arg link_repo "$RUST_LINK_REPO" --arg link_tag "$link_tag" \ + --argjson assets "$assets" --argjson launcher "$launcher" \ + --arg p_repo "$RUST_PLUGIN_REPO" --arg p_tag "$plugin_tag" \ + --slurpfile m "$manifest" \ + --argjson p_asset "$(asset_json rust-plugin "$tarball")" \ + --argjson protocol "$link_protocol" \ + '{ schema: 2, game: "rust", protocol: $protocol, + sidecar: { repo: $link_repo, tag: $link_tag, version: ($link_tag | ltrimstr("v")), + protocol: $protocol, assets: $assets, launcher: $launcher }, + payload: { kind: "plugin", repo: $p_repo, tag: $p_tag, version: ($p_tag | ltrimstr("v")), + commit: $m[0].commit, protocol: $protocol, + compat: { frameworks: { oxide: { min_version: $m[0].min_oxide_version }, + carbon: { min_version: $m[0].min_carbon_version } }, + requires_plugins: $m[0].requires_plugins }, + asset: $p_asset } }' > "$WORK/rust-s2.json" + # A manifest missing a key would compose a `null` the installer then trusts. + jq -e '([.payload.compat.frameworks[].min_version, .payload.commit] | all(type == "string")) + and (.payload.compat.requires_plugins | type == "array")' "$WORK/rust-s2.json" >/dev/null \ + || fail "the Rust-Plugins manifest is missing commit, a framework floor, or requires_plugins" + + if same_content "$PUBLISHED/v2/rust/current.json" "$WORK/rust-s2.json"; then + note "- **Rust**: no change (Rust-Link \`${link_tag}\`, Rust-Plugins \`${plugin_tag}\`, protocol ${link_protocol})." + return 0 + fi + local tag + tag="$(fresh_tag "$PUBLISHED/v2/rust")" + publish_doc "$PUBLISHED/v2/rust" "$tag" "$WORK/rust-s2.json" + note "- **Rust**: published \`${tag}\` — Rust-Link \`${link_tag}\`, Rust-Plugins \`${plugin_tag}\`, protocol ${link_protocol}." + echo "rust ${tag}" >> "$WORK/published-games.txt" +} + +# ── Run both, independently ────────────────────────────────────────────────── +: > "$WORK/published-games.txt" +FAILED="" +for game in servuo rust; do + echo "────────── ${game} ──────────" + # NOT `if ! ( … )`: a subshell in a condition runs with errexit OFF, so an + # unchecked failed download inside it would sail on. Capture the status instead. + set +e + ( set -e; "compose_${game}" ) + rc=$? + set -e + if [ "$rc" -ne 0 ]; then + FAILED="${FAILED:+${FAILED}, }${game}" + note "- **${game}**: **FAILED** — see the log. Nothing was published for it." + fi +done + +CHANGED=false +[ -s "$WORK/published.txt" ] && CHANGED=true +{ + echo "changed=${CHANGED}" + echo "commit_subject=chore(bundle): publish $(paste -sd ',' "$WORK/published-games.txt" | sed 's/,/, /g') [skip ci]" +} > "$WORK/result.env" + +echo "────────── result ──────────" +cat "$WORK/summary.md" +[ "$CHANGED" = true ] && { echo "files:"; sed 's/^/ /' "$WORK/published.txt"; } +[ -z "$FAILED" ] || fail "compose failed for: ${FAILED}" diff --git a/.gitea/workflows/bundle.yml b/.gitea/workflows/bundle.yml index 6e0dc4f..695549b 100644 --- a/.gitea/workflows/bundle.yml +++ b/.gitea/workflows/bundle.yml @@ -1,50 +1,64 @@ -# Compose and publish the bundle manifest. +# Compose and publish the bundle manifests. # -# This is Phase 0 item 3 of docs/installer/PLAN.md (§7.1–§7.3). +# This is Phase 0 item 3 of docs/installer/PLAN.md (§7.1–§7.3), extended by +# module-rust phase 18 (docs/modules/rust/PLAN.md §34.2.2) to a second game. # # ── What a bundle is ───────────────────────────────────────────────────────── # The bundle IS the compat matrix. The installer does not hardcode component # versions and does not resolve "latest" at run time; it fetches one small JSON -# document naming an exact, protocol-checked combination of a uo-link release -# and a servuo-plugins overlay release, and installs that. Because the bundle is +# document naming an exact, protocol-checked combination of a sidecar release +# and a game-side payload release, and installs that. Because the bundle is # data, a new sidecar release regenerates ~30 lines of JSON and leaves the # installer binary untouched: operators do not re-download the installer to pick # up a sidecar patch, and this repo does not accumulate releases whose code is # byte-identical. # -# ── Where it is published, and why not as a release ────────────────────────── -# Bundles are COMMITTED to this repo, on their own `bundles` branch, at its root: +# ── The streams ────────────────────────────────────────────────────────────── +# Bundles are COMMITTED to this repo, on their own `bundles` branch: # -# current.json the bundle the installer uses by default -# bundle-.json every bundle ever published, kept for --bundle +# current.json, bundle-.json schema 1, ServUO: link + overlay +# v2/servuo/current.json, bundle-.json schema 2, game "servuo" +# v2/rust/current.json, bundle-.json schema 2, game "rust": Rust-Link + Rust-Plugins # -# so the installer's two fetches are plain anonymous raw URLs on a public repo: +# Schema 2 (D146) names ONE game, with a `game` discriminant and a `payload` +# that is an overlay for ServUO and a plugin for Rust. Schema 1 (D147) is still +# composed beside it because every installer in the field reads only the root +# `current.json` and refuses any other schema; on 2027-01-01 it stops being +# composed and is left frozen, never deleted, so old installers still resolve +# and every bundle-.json stays pinnable. A ServUO matrix carries the same +# tag at both schemas. +# +# The installer's fetches are plain anonymous raw URLs on a public repo: # # https://gitea.whitlocktech.com/RunicGateway/installer/raw/branch/bundles/current.json -# https://gitea.whitlocktech.com/RunicGateway/installer/raw/branch/bundles/bundle-2026.08.04.json +# https://gitea.whitlocktech.com/RunicGateway/installer/raw/branch/bundles/v2/rust/current.json # # A BRANCH, not `main`, because `main` is protected and this job is unattended: # the pre-receive hook declines a push from CI, which is not a thing a nightly # cron can resolve. Publishing to a branch of its own keeps everything the # original choice was for — a reviewable diff, a git history of the compat # matrix, plain raw URLs, no auth on the shard host — and needs no protection -# exception. The alternative, whitelisting a scheduled job for pushes to the -# default branch, buys nothing this does not. +# exception. # # The obvious alternative — one Gitea release per bundle — was rejected because # it collides with this repo's own product. release.yml publishes the installer # BINARIES as v* releases, and `/releases/latest` returns whichever release is # newest regardless of kind; interleaving bundle releases would make "latest" -# intermittently resolve to a release containing no installer binary. Committing -# also gets a reviewable diff and a git history of the compat matrix for free. +# intermittently resolve to a release containing no installer binary. # # `main` is never pushed to by this workflow. (release.yml does not push to it # either — it tags and lets the release API do the rest.) # +# ── Where the logic lives ──────────────────────────────────────────────────── +# .gitea/scripts/compose-bundles.sh holds the gates and the documents, so it can +# be run by hand against the real release API (or a mock) without a runner. This +# file owns what needs the runner: checkout, the stale-component dispatch, and +# the push. +# # ── Triggers (PLAN.md §7.2) ────────────────────────────────────────────────── -# workflow_dispatch — POSTed by link's and servuo-plugins' release workflows -# as their final step, so a new release recomposes the -# bundle immediately. +# workflow_dispatch — POSTed by link's, servuo-plugins', Rust-Link's and +# Rust-Plugins' release workflows as their final step, +# so a new release recomposes the bundle immediately. # schedule (nightly) — recomputes from whatever the latest releases actually # are, so a missed or failed dispatch self-heals instead # of silently pinning operators to a stale sidecar. @@ -55,11 +69,11 @@ # ── Prerequisites (Settings → Actions → Secrets on RunicGateway/installer) ─── # REGISTRY_USER — Gitea username the token below belongs to # REGISTRY_TOKEN — Gitea access token with `write:repository`. It needs write -# on THIS repo (to push the bundle commit) and on -# RunicGateway/link + RunicGateway/servuo-plugins (to fire -# their release workflows for the stale case below). A token -# without the latter degrades to a warning, not a failure — -# the bundle it composes is still valid. +# on THIS repo (to push the bundle commit) and on the four +# component repos (to fire their release workflows for the +# stale case below). A token without the latter degrades to +# a warning, not a failure — the bundle it composes is still +# valid. # # The bundle commit carries `[skip ci]`, so it does not re-trigger release.yml. @@ -73,7 +87,7 @@ on: # Two component releases landing together dispatch this twice. Serialize rather # than cancel: a cancelled run is a bundle that never got composed, and the -# second run would otherwise race the first on the push to main. +# second run would otherwise race the first on the push. concurrency: group: compose-bundle cancel-in-progress: false @@ -81,13 +95,6 @@ concurrency: env: GITEA_HOST: gitea.whitlocktech.com REPO: RunicGateway/installer - LINK_REPO: RunicGateway/link - OVERLAY_REPO: RunicGateway/servuo-plugins - # Fixed top-level directory inside the overlay tarball. Deliberately NOT - # versioned (servuo-plugins/.gitea/workflows/release.yml) — a versioned prefix - # would mean parsing the version out of a path in order to read the manifest - # that declares the version. - OVERLAY_PREFIX: runicgateway-overlay jobs: compose: @@ -104,9 +111,8 @@ jobs: # The published bundles live on their own branch (see the header), so they # are materialized into a worktree rather than being part of the checkout. - # Everything downstream reads and writes `published/`, which means the - # ".2 suffix" scan and the idempotence check both see what is actually - # published rather than a stale copy on main. + # The compose script reads and writes `published/`, so the ".2 suffix" + # scan and the idempotence check both see what is actually published. - name: Materialize the bundles branch run: | set -euo pipefail @@ -121,7 +127,7 @@ jobs: if git ls-remote --exit-code --heads origin bundles >/dev/null 2>&1; then git fetch origin bundles git worktree add -B bundles published origin/bundles - echo "==> bundles branch: $(ls published/*.json 2>/dev/null | wc -l) published bundle(s)" + echo "==> bundles branch: $(find published -name 'bundle-*.json' | wc -l) published bundle(s)" else # First run. A root commit with an empty tree gives the worktree a # branch to sit on without inheriting main's history, which has @@ -140,276 +146,27 @@ jobs: $SUDO apt-get update -qq $SUDO apt-get install -y -qq --no-install-recommends jq curl ca-certificates - # ── Resolve the two component releases ─────────────────────────────── - # Read ANONYMOUSLY, on purpose. These are exactly the requests the shipped - # installer makes on an operator's machine, which has no Gitea credentials - # (PLAN.md §1: no git and no token on the shard host). Authenticating here - # would hide a repo flipped to private until an operator hit it; this way - # the visibility regression fails CI instead. - - name: Resolve the latest release of each component - id: resolve - run: | - set -euo pipefail - mkdir -p work - for pair in "link:${LINK_REPO}" "overlay:${OVERLAY_REPO}"; do - KEY="${pair%%:*}"; SLUG="${pair#*:}" - curl -sSfL -o "work/${KEY}-release.json" \ - "https://${GITEA_HOST}/api/v1/repos/${SLUG}/releases/latest" - TAG="$(jq -r '.tag_name' "work/${KEY}-release.json")" - [ -n "$TAG" ] && [ "$TAG" != "null" ] || { echo "::error::${SLUG} has no published release"; exit 1; } - echo "${KEY}_tag=${TAG}" >> "$GITHUB_OUTPUT" - echo "${KEY}_version=${TAG#v}" >> "$GITHUB_OUTPUT" - echo "==> ${SLUG} latest: ${TAG}" - done - - # ── GATE 2 (PLAN.md §7.1): assets exist, checksums match ───────────── - # Every asset the bundle will reference is downloaded and verified against - # the SHA256SUMS the publishing repo shipped beside it. This is not - # ceremony: SHA256SUMS is the trust anchor for these deliberately UNSIGNED - # artifacts (PLAN.md §3), and the installer will verify against the hashes - # THIS job records. A hash copied from a file nobody checked would make the - # whole chain decorative. + # ── Resolve, gate and compose every stream ─────────────────────────── + # Release resolution, gate 2 (every asset present and matching its + # SHA256SUMS), gate 1 (the sidecar's PROTOCOL_VERSION at its release tag + # against the payload's declared protocol) and the documents all live in + # the script. Each game composes independently: a Rust failure still lets + # a ServUO bundle publish, and this step goes red afterwards. A game whose + # repos have never released composes nothing, and that is not a failure. # - # `sha256sum -c` without --ignore-missing fails when SHA256SUMS names a - # file the release does not actually carry. The reverse — an asset with no - # SHA256SUMS entry — is checked separately below, because -c would not - # notice it. - - name: 'Gate 2: download assets and verify checksums' - id: assets - run: | - set -euo pipefail - fail() { echo "::error::$*"; exit 1; } - - for KEY in link overlay; do - DIR="work/${KEY}"; mkdir -p "$DIR" - SUMS_URL="$(jq -r '.assets[] | select(.name == "SHA256SUMS") | .browser_download_url' "work/${KEY}-release.json")" - [ -n "$SUMS_URL" ] && [ "$SUMS_URL" != "null" ] \ - || fail "${KEY} release has no SHA256SUMS asset — nothing to verify against" - curl -sSfL -o "${DIR}/SHA256SUMS" "$SUMS_URL" - - jq -r '.assets[] | select(.name != "SHA256SUMS") | "\(.name)\t\(.browser_download_url)"' \ - "work/${KEY}-release.json" > "${DIR}/asset-list.tsv" - [ -s "${DIR}/asset-list.tsv" ] || fail "${KEY} release carries no assets besides SHA256SUMS" - - while IFS="$(printf '\t')" read -r NAME URL; do - [ -n "$NAME" ] || continue - echo " fetching ${KEY}/${NAME}" - curl -sSfL -o "${DIR}/${NAME}" "$URL" - # An asset absent from SHA256SUMS is unverifiable, and `-c` below - # would pass right over it. The `\*?` is not paranoia: sha256sum - # marks binary mode by prefixing the path with `*` instead of the - # two-space text separator, so a naive match on " ${NAME}" would - # miss every entry on a host that defaults to binary mode. - grep -qE "[ \t]\*?${NAME}\$" "${DIR}/SHA256SUMS" \ - || fail "${KEY} asset ${NAME} has no entry in that release's SHA256SUMS" - done < "${DIR}/asset-list.tsv" - - ( cd "$DIR" && sha256sum -c SHA256SUMS ) \ - || fail "${KEY} assets do not match the SHA256SUMS published with them" - echo "==> ${KEY}: all assets present and verified" - done - - # Map link's binaries onto platform keys. The pattern is asserted, not - # assumed: an unrecognized asset name is a hard failure so that adding - # a target to link's release.yml (macOS, a Windows arm64) surfaces here - # as a red run, rather than being silently dropped from every bundle. - # - # linux-aarch64 was recognized here one merge BEFORE link published one - # (PLAN.md §5.2, steps 1 and 3). That order was forced by the two rules - # below being strict in opposite directions: an unknown name fails the - # run, and a missing REQUIRED key fails it too. So the name had to be - # taught before the release that carried it, and the key could only be - # required after — requiring it first would have failed every bundle - # for as long as the gap lasted. link v1.1.1 ships the binary, so the - # key is now required: a dropped target reddens this job instead of - # vanishing from every bundle. - : > work/link-platforms.tsv - while IFS="$(printf '\t')" read -r NAME URL; do - [ -n "$NAME" ] || continue - case "$NAME" in - *-linux-x86_64) PLAT=linux-x86_64 ;; - *-linux-aarch64) PLAT=linux-aarch64 ;; - *-windows-x86_64.exe) PLAT=windows-x86_64 ;; - *) fail "unrecognized link asset '${NAME}' — bundle.yml does not know what platform to file it under. Teach it this name or the bundle would silently omit the asset." ;; - esac - printf '%s\t%s\t%s\t%s\n' "$PLAT" "$NAME" "$URL" \ - "$(sha256sum "work/link/${NAME}" | cut -d' ' -f1)" >> work/link-platforms.tsv - done < work/link/asset-list.tsv - for REQUIRED in linux-x86_64 linux-aarch64 windows-x86_64; do - grep -q "^${REQUIRED}$(printf '\t')" work/link-platforms.tsv \ - || fail "link release is missing a ${REQUIRED} binary; the installer ships for all three" - done - - # The overlay release carries exactly one artifact: the tarball. - OVERLAY_COUNT="$(wc -l < work/overlay/asset-list.tsv)" - [ "$OVERLAY_COUNT" -eq 1 ] \ - || fail "expected exactly 1 overlay asset besides SHA256SUMS, found ${OVERLAY_COUNT}" - OVERLAY_NAME="$(cut -f1 work/overlay/asset-list.tsv)" - case "$OVERLAY_NAME" in - *.tar.gz) ;; - *) fail "overlay asset '${OVERLAY_NAME}' is not the .tar.gz the installer expects" ;; - esac - { - echo "overlay_name=${OVERLAY_NAME}" - echo "overlay_url=$(cut -f2 work/overlay/asset-list.tsv)" - echo "overlay_sha=$(sha256sum "work/overlay/${OVERLAY_NAME}" | cut -d' ' -f1)" - } >> "$GITHUB_OUTPUT" - - # ── GATE 1 (PLAN.md §7.1): the two halves speak the same protocol ──── - # This is the check the whole bundle exists for. The sidecar rejects a - # protocol mismatch with 409 rather than mis-parsing, so a mismatched pair - # is not a subtle bug — it is a shard that emits into a void. Catching it - # here costs one HTTP GET; catching it on an operator's box costs them an - # evening. - # - # The two sides are read from genuinely different places because they ARE - # genuinely different: - # - # overlay — manifest.json inside the tarball. The C# plugin announces no - # version on the wire and none is queryable before ServUO - # boots (PLAN.md §2.6), so this hand-maintained declaration is - # the only statement of it that exists. - # sidecar — PROTOCOL_VERSION in sidecar/src/main.rs, read at the RELEASE - # TAG. Not from the binary: `--print-config` would answer, but - # only for releases from v1.1.0 on (it did not exist before - # Phase 0.2), and `--bundle ` has to be able to recompose - # an older bundle. Reading the tag the release was built from - # works uniformly, needs no execution of a downloaded - # artifact, and does not provision a throwaway config and - # print its auth token into a CI log. - - name: 'Gate 1: sidecar and overlay protocol versions agree' - id: protocol - run: | - set -euo pipefail - fail() { echo "::error::$*"; exit 1; } - - LINK_TAG="${{ steps.resolve.outputs.link_tag }}" - OVERLAY_TAG="${{ steps.resolve.outputs.overlay_tag }}" - - tar -xzf "work/overlay/${{ steps.assets.outputs.overlay_name }}" -C work - MANIFEST="work/${OVERLAY_PREFIX}/manifest.json" - [ -f "$MANIFEST" ] || fail "the overlay tarball has no ${OVERLAY_PREFIX}/manifest.json — the installer resolves it at that exact path" - - OVERLAY_PROTOCOL="$(jq -r '.protocol' "$MANIFEST")" - OVERLAY_COMMIT="$(jq -r '.commit' "$MANIFEST")" - MANIFEST_VERSION="$(jq -r '.version' "$MANIFEST")" - # A manifest that disagrees with the tag it shipped under means the - # release workflow stamped one version and tagged another; every - # downstream record of "what is installed" would then be wrong. - [ "$MANIFEST_VERSION" = "${OVERLAY_TAG#v}" ] \ - || fail "overlay manifest says version ${MANIFEST_VERSION} but the release is tagged ${OVERLAY_TAG}" - - curl -sSfL -o work/main.rs \ - "https://${GITEA_HOST}/${LINK_REPO}/raw/tag/${LINK_TAG}/sidecar/src/main.rs" - LINK_PROTOCOL="$(sed -nE 's/^[[:space:]]*pub const PROTOCOL_VERSION[^=]*=[[:space:]]*([0-9]+).*/\1/p' work/main.rs | head -1)" - # Empty means the constant moved or was renamed. Fail loudly: silently - # treating "could not read" as "matches" is how a mismatched pair ships. - [ -n "$LINK_PROTOCOL" ] \ - || fail "could not read PROTOCOL_VERSION from ${LINK_REPO}@${LINK_TAG}:sidecar/src/main.rs — has the constant moved? Gate 1 cannot be skipped." - - echo "==> sidecar ${LINK_TAG} protocol=${LINK_PROTOCOL} | overlay ${OVERLAY_TAG} protocol=${OVERLAY_PROTOCOL}" - [ "$LINK_PROTOCOL" = "$OVERLAY_PROTOCOL" ] || fail \ - "PROTOCOL MISMATCH — sidecar ${LINK_TAG} speaks ${LINK_PROTOCOL}, overlay ${OVERLAY_TAG} declares ${OVERLAY_PROTOCOL}. Refusing to publish a bundle that would install a shard whose events the sidecar rejects with 409. Fix: land the matching half and let its release cut, or bump servuo-plugins/overlay.toml." - - { - echo "protocol=${LINK_PROTOCOL}" - echo "overlay_commit=${OVERLAY_COMMIT}" - echo "overlay_min_servuo=$(jq -r '.servuo.min_version' "$MANIFEST")" - echo "overlay_patched_against=$(jq -r '.servuo.patches_verified_against' "$MANIFEST")" - } >> "$GITHUB_OUTPUT" - - # ── Compose ────────────────────────────────────────────────────────── - # Note the shape difference from PLAN.md §7.1's sketch: `link` carries a - # per-platform asset map rather than one sha256. link publishes a Linux - # binary and a Windows .exe, and the installer runs on both — a single - # hash could only ever have described one of them. - - name: Compose bundle.json + # The outputs are written even when the script fails, so the publish step + # (on `always()`) can still ship the game that did compose. + - name: Compose the bundles id: compose run: | set -euo pipefail - LINK_ASSETS="$(jq -R -s ' - split("\n") | map(select(length > 0)) | map(split("\t")) - | map({ (.[0]): { name: .[1], url: .[2], sha256: .[3] } }) | add - ' work/link-platforms.tsv)" - - jq -n \ - --arg link_repo "${LINK_REPO}" \ - --arg link_tag "${{ steps.resolve.outputs.link_tag }}" \ - --arg link_version "${{ steps.resolve.outputs.link_version }}" \ - --argjson link_assets "${LINK_ASSETS}" \ - --arg ov_repo "${OVERLAY_REPO}" \ - --arg ov_tag "${{ steps.resolve.outputs.overlay_tag }}" \ - --arg ov_version "${{ steps.resolve.outputs.overlay_version }}" \ - --arg ov_commit "${{ steps.protocol.outputs.overlay_commit }}" \ - --arg ov_name "${{ steps.assets.outputs.overlay_name }}" \ - --arg ov_url "${{ steps.assets.outputs.overlay_url }}" \ - --arg ov_sha "${{ steps.assets.outputs.overlay_sha }}" \ - --arg ov_min "${{ steps.protocol.outputs.overlay_min_servuo }}" \ - --arg ov_patched "${{ steps.protocol.outputs.overlay_patched_against }}" \ - --argjson protocol "${{ steps.protocol.outputs.protocol }}" \ - '{ - schema: 1, - protocol: $protocol, - link: { - repo: $link_repo, - tag: $link_tag, - version: $link_version, - protocol: $protocol, - assets: $link_assets - }, - overlay: { - repo: $ov_repo, - tag: $ov_tag, - version: $ov_version, - commit: $ov_commit, - protocol: $protocol, - servuo: { - min_version: $ov_min, - patches_verified_against: $ov_patched - }, - asset: { name: $ov_name, url: $ov_url, sha256: $ov_sha } - } - }' > work/content.json - - echo "----- composed content -----" - cat work/content.json - - # Idempotence. `bundle` and `generated` are metadata ABOUT this run, so - # comparing them would make every nightly cron look like a change and - # commit a dated duplicate of the same matrix forever. Compare only - # what the installer would actually act on. - CHANGED=true - if [ -f published/current.json ]; then - if jq -S 'del(.bundle, .generated)' published/current.json > work/old-content.json \ - && jq -S '.' work/content.json > work/new-content.json \ - && cmp -s work/old-content.json work/new-content.json; then - CHANGED=false - fi + rc=0 + PUBLISHED="$PWD/published" WORK="$PWD/work" GITEA_HOST="$GITEA_HOST" \ + bash .gitea/scripts/compose-bundles.sh || rc=$? + if [ -f work/result.env ]; then + grep '^changed=' work/result.env >> "$GITHUB_OUTPUT" fi - echo "changed=${CHANGED}" >> "$GITHUB_OUTPUT" - - if [ "$CHANGED" = false ]; then - echo "==> identical to the published current.json — nothing to publish." - exit 0 - fi - - # Bundle tags are dates (PLAN.md §7.1). Two bundles on one day — a - # sidecar release in the morning and an overlay release in the - # afternoon is the normal way that happens — get .2, .3, … so a tag - # always names exactly one matrix and `--bundle` stays reproducible. - BASE="$(date -u +%Y.%m.%d)" - TAG="$BASE"; N=1 - while [ -f "published/bundle-${TAG}.json" ]; do - N=$((N+1)); TAG="${BASE}.${N}" - done - - jq --arg bundle "$TAG" --arg generated "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \ - '{ schema: .schema, bundle: $bundle, generated: $generated } + del(.schema)' \ - work/content.json > "published/bundle-${TAG}.json" - cp "published/bundle-${TAG}.json" published/current.json - - echo "bundle_tag=${TAG}" >> "$GITHUB_OUTPUT" - echo "==> composed bundle ${TAG}" + exit "$rc" # ── Stale-component check: dispatch, don't wait (PLAN.md §7.3) ─────── # Each component self-releases on merge to its own main, so by the time @@ -424,10 +181,14 @@ jobs: # counts would report every README fix as a stuck release and re-dispatch # a workflow that correctly declines to run, every single night. # - # This runs even when the bundle is unchanged: an unchanged bundle is the - # exact symptom of a component release that never happened. + # This runs even when the bundle is unchanged, or a game failed: an + # unchanged bundle is the exact symptom of a component release that never + # happened. work/stale.tsv names each component the compose resolved; a + # repo that has never released is absent from it, and has no release to + # be stale against. - name: Check for components with unreleased work, and dispatch them id: stale + if: always() continue-on-error: true env: REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }} @@ -440,11 +201,12 @@ jobs: # message change what that script does. (Do not write that token # literally in a comment: the runner parses it, fails, and silently # skips the whole step.) + mkdir -p work : > work/stale-warnings.md + [ -f work/stale.tsv ] || exit 0 - for pair in "${LINK_REPO}:${{ steps.resolve.outputs.link_tag }}" \ - "${OVERLAY_REPO}:${{ steps.resolve.outputs.overlay_tag }}"; do - SLUG="${pair%:*}"; TAG="${pair##*:}" + while IFS="$(printf '\t')" read -r SLUG TAG; do + [ -n "$SLUG" ] || continue if ! curl -sSfL -o work/compare.json \ "https://${GITEA_HOST}/api/v1/repos/${SLUG}/compare/${TAG}...main"; then echo "::warning::could not compare ${SLUG} ${TAG}...main; skipping its stale check" @@ -483,7 +245,7 @@ jobs: else echo "==> ${SLUG}: nothing releasable after ${TAG}" fi - done + done < work/stale.tsv # ── Publish ────────────────────────────────────────────────────────── # Preflighted for the same reason the release workflows are: actions/ @@ -491,8 +253,11 @@ jobs: # config, so a push can succeed on that leftover even with the secrets # empty. That makes "the push worked" no evidence at all that the repo is # configured, and the failure surfaces somewhere less obvious later. + # + # `always()`: when one game failed and the other composed, the one that + # composed still ships. The compose step has already made the run red. - name: Verify publish credentials are configured - if: ${{ steps.compose.outputs.changed == 'true' }} + if: ${{ always() && steps.compose.outputs.changed == 'true' }} env: REGISTRY_USER: ${{ secrets.REGISTRY_USER }} REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }} @@ -502,19 +267,18 @@ jobs: [ -n "$(printf '%s' "${REGISTRY_USER:-}" | tr -d '\r\n')" ] || MISSING="${MISSING} REGISTRY_USER" [ -n "$(printf '%s' "${REGISTRY_TOKEN:-}" | tr -d '\r\n')" ] || MISSING="${MISSING} REGISTRY_TOKEN" if [ -n "$MISSING" ]; then - echo "::error::Missing Actions secret(s):${MISSING}. Set them under Settings → Actions → Secrets on ${REPO}. REGISTRY_TOKEN needs write:repository to push the bundle commit to main." + echo "::error::Missing Actions secret(s):${MISSING}. Set them under Settings → Actions → Secrets on ${REPO}. REGISTRY_TOKEN needs write:repository to push the bundle commit." exit 1 fi echo "Publish credentials present." - - name: Commit and push the bundle - if: ${{ steps.compose.outputs.changed == 'true' }} + - name: Commit and push the bundles + if: ${{ always() && steps.compose.outputs.changed == 'true' }} env: REGISTRY_USER: ${{ secrets.REGISTRY_USER }} REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }} run: | set -euo pipefail - TAG="${{ steps.compose.outputs.bundle_tag }}" # Secrets can arrive with a trailing newline depending on how they were # pasted, and a stray CR/LF corrupts the remote URL ("credential url # cannot be parsed"). @@ -522,9 +286,14 @@ jobs: CI_TOKEN="$(printf '%s' "${REGISTRY_TOKEN}" | tr -d '\r\n')" git remote set-url origin "https://${CI_USER}:${CI_TOKEN}@${GITEA_HOST}/${REPO}.git" + # The subject is the script's ("publish servuo 2026.09.27, rust + # 2026.09.27 [skip ci]"), read from a file rather than interpolated + # from a template value, so nothing in it can change this script. + grep '^commit_subject=' work/result.env | cut -d= -f2- > work/commit-msg.txt + cd published git add -A - git commit -m "chore(bundle): publish ${TAG} (link ${{ steps.resolve.outputs.link_tag }}, overlay ${{ steps.resolve.outputs.overlay_tag }}, protocol ${{ steps.protocol.outputs.protocol }}) [skip ci]" + git commit -F ../work/commit-msg.txt # Two runs can compose at once — a component release dispatches this # while the nightly cron is mid-flight — so losing the race is normal @@ -538,7 +307,8 @@ jobs: git rebase origin/bundles git push origin bundles fi - echo "==> published bundle-${TAG}.json and current.json on the bundles branch" + echo "==> published on the bundles branch:" + sed 's/^/ /' ../work/published.txt - name: Job summary if: always() @@ -547,16 +317,17 @@ jobs: { echo "## Bundle compose" echo - echo "| Component | Release | Protocol |" - echo "|---|---|---|" - echo "| uo-link sidecar | \`${{ steps.resolve.outputs.link_tag }}\` | ${{ steps.protocol.outputs.protocol }} |" - echo "| servuo-plugins overlay | \`${{ steps.resolve.outputs.overlay_tag }}\` | ${{ steps.protocol.outputs.protocol }} |" - echo - case "${{ steps.compose.outputs.changed }}" in - true) echo "**Published \`${{ steps.compose.outputs.bundle_tag }}\`** → \`bundles/current.json\`" ;; - false) echo "No change — \`bundles/current.json\` already names this combination." ;; - *) echo "Compose did not complete — see the failing step above." ;; - esac + if [ -s work/summary.md ]; then + cat work/summary.md + else + echo "Compose did not complete — see the failing step above." + fi + if [ -s work/published.txt ]; then + echo + echo "Written to the bundles branch:" + echo + sed 's/^/- /' work/published.txt + fi if [ -s work/stale-warnings.md ]; then echo echo "### ⚠ Components with unreleased work" diff --git a/bundles/README.md b/bundles/README.md index 4985d99..2e62b7e 100644 --- a/bundles/README.md +++ b/bundles/README.md @@ -3,26 +3,45 @@ **These files are generated. Do not edit them by hand.** A *bundle* names one exact, protocol-checked combination of the two components the installer -deploys — a `uo-link` sidecar release and a `servuo-plugins` overlay release. The installer does not +deploys for one game: a sidecar release and a game-side payload release. For ServUO that is `link` +and a `servuo-plugins` overlay; for Rust it is `Rust-Link` and a `Rust-Plugins` plugin. The installer does not hardcode versions and does not resolve "latest" at run time; it fetches one of these documents and installs what it names. **The bundle is the compat matrix.** -They are written by [`.gitea/workflows/bundle.yml`](../.gitea/workflows/bundle.yml), which composes +They are written by [`.gitea/workflows/bundle.yml`](../.gitea/workflows/bundle.yml) (the logic is +in [`.gitea/scripts/compose-bundles.sh`](../.gitea/scripts/compose-bundles.sh)), which composes one whenever a component publishes a release (dispatched by that release's own workflow) and nightly, so a missed dispatch self-heals. A run that finds nothing changed writes nothing. -See `docs/installer/PLAN.md` §7 for the design. +See `docs/installer/PLAN.md` §7 for the design, and `docs/modules/rust/PLAN.md` §34.2.2 for +schema 2. ## Where they live: the `bundles` branch **The JSON documents are not in this directory.** They are published to a branch of their own, -[`bundles`](https://gitea.whitlocktech.com/RunicGateway/installer/src/branch/bundles), at its root: +[`bundles`](https://gitea.whitlocktech.com/RunicGateway/installer/src/branch/bundles). There is one +stream per game and schema, each a directory holding the same two kinds of file: + +| Directory | Stream | +|---|---| +| *(root)* | Schema 1, ServUO. What every installer up to v0.2.x reads. **Retires 2027-01-01** | +| `v2/servuo/` | Schema 2, `"game": "servuo"` | +| `v2/rust/` | Schema 2, `"game": "rust"` | | File | What it is | |---|---| | `current.json` | The bundle the installer uses by default. Always a copy of the newest `bundle-*.json`. | | `bundle-.json` | Every bundle ever published, kept forever so `--bundle ` stays reproducible. | +**Schema 1 retires on 2027-01-01.** Until then it is composed exactly as before, beside schema 2, so +every installer already in the field keeps updating. After that date it stops being composed and is +left **frozen at its last bundle, never deleted**: an old installer still resolves something, and +every `bundle-.json` stays pinnable. A ServUO matrix carries the **same tag** at both schemas. + +A schema-2 document names one game rather than both, because the games release on their own +schedules: a document naming both would hand a ServUO host a new bundle every time a Rust plugin +shipped. A game none of whose repos has released composes nothing, and that is not a failure. + Tags are UTC dates — `2026.08.04`. A second bundle on the same day (a sidecar release in the morning, an overlay release in the afternoon) becomes `2026.08.04.2`, so one tag always names exactly one matrix. @@ -45,6 +64,7 @@ Plain anonymous `GET`s against a public repo. The shard host gets no git and no ``` https://gitea.whitlocktech.com/RunicGateway/installer/raw/branch/bundles/current.json https://gitea.whitlocktech.com/RunicGateway/installer/raw/branch/bundles/bundle-2026.08.04.json +https://gitea.whitlocktech.com/RunicGateway/installer/raw/branch/bundles/v2/rust/current.json ``` Bundles are committed rather than published as Gitea releases because this repo's *own* releases are @@ -52,7 +72,7 @@ the installer binaries, and `/releases/latest` returns whichever release is newe kind — interleaving the two would make "latest" intermittently resolve to a release containing no installer binary. -## Schema +## Schema 1 (ServUO, retiring) `schema` is the version of *this document's* shape, and is unrelated to `protocol` (the uo-link wire protocol) or to either component's release version. All three move independently. @@ -104,3 +124,48 @@ rejects a protocol mismatch with `409` rather than mis-parsing, so a mismatched emitting into a void. CI refuses to publish one: it reads `PROTOCOL_VERSION` from the sidecar's source at its release tag and the declared `protocol` from the overlay tarball's `manifest.json`, and fails if they differ. The top-level `protocol` is that agreed value. + +## Schema 2 + +The same document for either game: a `game` discriminant, a `sidecar`, and a `payload` whose `kind` +says what it is. A reader switches on `game`, and refuses a `schema` it does not know. + +```jsonc +{ + "schema": 2, + "game": "rust", // or "servuo" + "bundle": "2026.09.27", + "generated": "2026-09-27T…Z", + "protocol": 12, + + "sidecar": { + "repo": "RunicGateway/Rust-Link", "tag": "v…", "version": "…", "protocol": 12, + "assets": { // Rust: no linux-aarch64 — RustDedicated has no arm64 build + "linux-x86_64": { "name": "…", "url": "…", "sha256": "…" }, + "windows-x86_64": { "name": "…", "url": "…", "sha256": "…" } + }, + "launcher": { "name": "with-sidecar.sh", "url": "…", "sha256": "…" } // Rust only: the egg's startup + }, + + "payload": { + "kind": "plugin", // "overlay" for ServUO + "repo": "RunicGateway/Rust-Plugins", "tag": "v…", "version": "…", "commit": "…", + "protocol": 12, + "compat": { // per game; for ServUO it is schema 1's `servuo` block + "frameworks": { "oxide": { "min_version": "2.0.7585" }, + "carbon": { "min_version": "2.0.259" } }, + "requires_plugins": ["Kits", "ZoneManager"] + }, + "asset": { "name": "runicgateway-rust-plugin-….tar.gz", "url": "…", "sha256": "…" } + } +} +``` + +A ServUO schema-2 document is schema 1's content re-shaped: `link` becomes `sidecar` (no +`launcher`), `overlay` becomes a `payload` of `"kind": "overlay"`, and its `servuo` block becomes +`compat` (`min_version`, `patches_verified_against`). + +The two gates hold for both games. For Rust, gate 1 reads `PROTOCOL_VERSION` from `Rust-Link`'s +`sidecar/src/main.rs` at its release tag and `protocol` from the `manifest.json` inside the plugin +tarball (`runicgateway-rust-plugin/manifest.json`). The stakes are higher there than for ServUO: the +Rust game link has no `409`, so a mismatched plugin would mis-parse rather than be refused.