fix(bundle): publish to a bundles branch, and unbreak the stale check

Three things, all found by the first compose run that ever had a bundle to write. 1. `main` is protected, so the push was declined by the pre-receive hook -- twice, since the retry rebases and pushes to the same place. Every bundle since v1.1.1 has been composed correctly and thrown away. Bundles now go to a `bundles` branch of their own, at its root, which needs no protection exception and keeps everything the original choice was for: a reviewable diff, a git history of the compat matrix, plain anonymous raw URLs, no credentials on the shard host. The header's claim that this push "needs no new branch-protection exception" was simply false. 2. A `${{ }}` written literally in a shell comment silently disabled the entire stale-component check. The runner scans a step's script for template expressions before running it, fails to parse the empty one, and skips the step WITHOUT failing the job -- so the dispatch that is supposed to fire a component's release workflow has never run once. Reworded, with a warning not to write that token in a comment again. (link/release.yml and this repo's release.yml carry the same bug in their bump-and-tag step; handled separately.) 3. linux-aarch64 is now a REQUIRED platform key, which was step 3 of PLAN.md §5.2 and was waiting on link publishing one. v1.1.1 does, so from here a dropped target reddens this job instead of vanishing from every bundle. The published bundles are materialized into a worktree at `published/`, so the ".2 suffix" scan and the idempotence check read what is actually published rather than a stale copy on main. The branch is created from an empty-tree root commit on first use, so it carries no history that has nothing to do with the compat matrix; it has been seeded already with bundle 2026.08.04, because every bundle is kept forever and the move must not lose the one that exists. bundles/*.json is deleted from main -- it is now a stale copy of data that lives elsewhere, and a wrong "current" is worse than none. The README stays and documents the branch. Verified by running the whole job in a container against a bare repo standing in for the remote: first run creates the branch and publishes both files with all three asset keys, second and third runs report "identical to the published current.json -- nothing to publish" and push nothing, and the stale check now runs and reports both components as having nothing releasable. Co-Authored-By: Claude <noreply@anthropic.com>
Merge pull request 'ci(bundle): recognize a linux-aarch64 link asset' (#9 ) from ci/bundle-aarch64-key into main
2026-08-05 17:14:57 -05:00 · 2026-08-05 17:51:06 +00:00 · 2026-08-05 05:22:59 -05:00
4 changed files with 106 additions and 124 deletions
--- a/.gitea/workflows/bundle.yml
+++ b/.gitea/workflows/bundle.yml
@@ -13,15 +13,23 @@
 # byte-identical.
 #
 # ── Where it is published, and why not as a release ──────────────────────────
-# Bundles are COMMITTED to this repo under bundles/:
+# Bundles are COMMITTED to this repo, on their own `bundles` branch, at its root:
 #
-#   bundles/current.json          the bundle the installer uses by default
-#   bundles/bundle-<tag>.json     every bundle ever published, kept for --bundle
+#   current.json          the bundle the installer uses by default
+#   bundle-<tag>.json     every bundle ever published, kept for --bundle
 #
 # so the installer's two fetches are plain anonymous raw URLs on a public repo:
 #
-#   https://gitea.whitlocktech.com/RunicGateway/installer/raw/branch/main/bundles/current.json
-#   https://gitea.whitlocktech.com/RunicGateway/installer/raw/branch/main/bundles/bundle-2026.08.04.json
+#   https://gitea.whitlocktech.com/RunicGateway/installer/raw/branch/bundles/current.json
+#   https://gitea.whitlocktech.com/RunicGateway/installer/raw/branch/bundles/bundle-2026.08.04.json
+#
+# A BRANCH, not `main`, because `main` is protected and this job is unattended:
+# the pre-receive hook declines a push from CI, which is not a thing a nightly
+# cron can resolve. Publishing to a branch of its own keeps everything the
+# original choice was for — a reviewable diff, a git history of the compat
+# matrix, plain raw URLs, no auth on the shard host — and needs no protection
+# exception. The alternative, whitelisting a scheduled job for pushes to the
+# default branch, buys nothing this does not.
 #
 # The obvious alternative — one Gitea release per bundle — was rejected because
 # it collides with this repo's own product. release.yml publishes the installer
@@ -30,8 +38,8 @@
 # intermittently resolve to a release containing no installer binary. Committing
 # also gets a reviewable diff and a git history of the compat matrix for free.
 #
-# The push to `main` needs no new branch-protection exception: release.yml's
-# version-bump commit already requires REGISTRY_USER to be able to push here.
+# `main` is never pushed to by this workflow. (release.yml does not push to it
+# either — it tags and lets the release API do the rest.)
 #
 # ── Triggers (PLAN.md §7.2) ──────────────────────────────────────────────────
 #   workflow_dispatch  — POSTed by link's and servuo-plugins' release workflows
@@ -86,14 +94,44 @@ jobs:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    steps:
-      # Full history: the push step rebases onto main if release.yml's version
-      # bump landed while this job was composing, and a depth-1 clone has no
-      # base to rebase onto.
-      - name: Check out the bundles directory
+      # Full history: the publish step rebases onto the bundles branch if another
+      # run landed while this one was composing, and a depth-1 clone has no base
+      # to rebase onto.
+      - name: Check out the repository
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

+      # The published bundles live on their own branch (see the header), so they
+      # are materialized into a worktree rather than being part of the checkout.
+      # Everything downstream reads and writes `published/`, which means the
+      # ".2 suffix" scan and the idempotence check both see what is actually
+      # published rather than a stale copy on main.
+      - name: Materialize the bundles branch
+        run: |
+          set -euo pipefail
+          git config user.name  "installer-ci"
+          git config user.email "ci@whitlocktech.com"
+          # `prune` matters on a re-run in an existing checkout: removing the
+          # directory leaves the worktree registered, and `worktree add` then
+          # refuses the path. CI checks out fresh every time, so this only shows
+          # up when driving the job by hand — which is how it is tested.
+          rm -rf published
+          git worktree prune
+          if git ls-remote --exit-code --heads origin bundles >/dev/null 2>&1; then
+            git fetch origin bundles
+            git worktree add -B bundles published origin/bundles
+            echo "==> bundles branch: $(ls published/*.json 2>/dev/null | wc -l) published bundle(s)"
+          else
+            # First run. A root commit with an empty tree gives the worktree a
+            # branch to sit on without inheriting main's history, which has
+            # nothing to do with the compat matrix.
+            EMPTY_TREE="$(git hash-object -t tree /dev/null)"
+            ROOT="$(git commit-tree "$EMPTY_TREE" -m 'chore(bundle): start the bundles branch')"
+            git worktree add -B bundles published "$ROOT"
+            echo "==> bundles branch does not exist yet; it will be created by the first publish"
+          fi
+
      - name: Install jq and curl
        run: |
          set -euo pipefail
@@ -173,22 +211,33 @@ jobs:

          # Map link's binaries onto platform keys. The pattern is asserted, not
          # assumed: an unrecognized asset name is a hard failure so that adding
-          # a target to link's release.yml (aarch64, macOS) surfaces here as a
-          # red run, rather than being silently dropped from every bundle.
+          # a target to link's release.yml (macOS, a Windows arm64) surfaces here
+          # as a red run, rather than being silently dropped from every bundle.
+          #
+          # linux-aarch64 was recognized here one merge BEFORE link published one
+          # (PLAN.md §5.2, steps 1 and 3). That order was forced by the two rules
+          # below being strict in opposite directions: an unknown name fails the
+          # run, and a missing REQUIRED key fails it too. So the name had to be
+          # taught before the release that carried it, and the key could only be
+          # required after — requiring it first would have failed every bundle
+          # for as long as the gap lasted. link v1.1.1 ships the binary, so the
+          # key is now required: a dropped target reddens this job instead of
+          # vanishing from every bundle.
          : > work/link-platforms.tsv
          while IFS="$(printf '\t')" read -r NAME URL; do
            [ -n "$NAME" ] || continue
            case "$NAME" in
              *-linux-x86_64)       PLAT=linux-x86_64 ;;
+              *-linux-aarch64)      PLAT=linux-aarch64 ;;
              *-windows-x86_64.exe) PLAT=windows-x86_64 ;;
              *) fail "unrecognized link asset '${NAME}' — bundle.yml does not know what platform to file it under. Teach it this name or the bundle would silently omit the asset." ;;
            esac
            printf '%s\t%s\t%s\t%s\n' "$PLAT" "$NAME" "$URL" \
              "$(sha256sum "work/link/${NAME}" | cut -d' ' -f1)" >> work/link-platforms.tsv
          done < work/link/asset-list.tsv
-          for REQUIRED in linux-x86_64 windows-x86_64; do
+          for REQUIRED in linux-x86_64 linux-aarch64 windows-x86_64; do
            grep -q "^${REQUIRED}$(printf '\t')" work/link-platforms.tsv \
-              || fail "link release is missing a ${REQUIRED} binary; the installer ships for both"
+              || fail "link release is missing a ${REQUIRED} binary; the installer ships for all three"
          done

          # The overlay release carries exactly one artifact: the tarball.
@@ -330,8 +379,8 @@ jobs:
          # commit a dated duplicate of the same matrix forever. Compare only
          # what the installer would actually act on.
          CHANGED=true
-          if [ -f bundles/current.json ]; then
-            if jq -S 'del(.bundle, .generated)' bundles/current.json > work/old-content.json \
+          if [ -f published/current.json ]; then
+            if jq -S 'del(.bundle, .generated)' published/current.json > work/old-content.json \
               && jq -S '.' work/content.json > work/new-content.json \
               && cmp -s work/old-content.json work/new-content.json; then
              CHANGED=false
@@ -340,7 +389,7 @@ jobs:
          echo "changed=${CHANGED}" >> "$GITHUB_OUTPUT"

          if [ "$CHANGED" = false ]; then
-            echo "==> identical to bundles/current.json — nothing to publish."
+            echo "==> identical to the published current.json — nothing to publish."
            exit 0
          fi

@@ -350,15 +399,14 @@ jobs:
          # always names exactly one matrix and `--bundle` stays reproducible.
          BASE="$(date -u +%Y.%m.%d)"
          TAG="$BASE"; N=1
-          while [ -f "bundles/bundle-${TAG}.json" ]; do
+          while [ -f "published/bundle-${TAG}.json" ]; do
            N=$((N+1)); TAG="${BASE}.${N}"
          done

-          mkdir -p bundles
          jq --arg bundle "$TAG" --arg generated "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
             '{ schema: .schema, bundle: $bundle, generated: $generated } + del(.schema)' \
-             work/content.json > "bundles/bundle-${TAG}.json"
-          cp "bundles/bundle-${TAG}.json" bundles/current.json
+             work/content.json > "published/bundle-${TAG}.json"
+          cp "published/bundle-${TAG}.json" published/current.json

          echo "bundle_tag=${TAG}" >> "$GITHUB_OUTPUT"
          echo "==> composed bundle ${TAG}"
@@ -387,9 +435,11 @@ jobs:
          set -euo pipefail
          CI_TOKEN="$(printf '%s' "${REGISTRY_TOKEN:-}" | tr -d '\r\n')"
          # Warnings go to a FILE, not a step output. The job summary below
-          # reads it with `cat`; interpolating a multi-line `${{ }}` value into
+          # reads it with `cat`; interpolating a multi-line template value into
          # a shell string there would let any character in a commit-derived
-          # message change what that script does.
+          # message change what that script does. (Do not write that token
+          # literally in a comment: the runner parses it, fails, and silently
+          # skips the whole step.)
          : > work/stale-warnings.md

          for pair in "${LINK_REPO}:${{ steps.resolve.outputs.link_tag }}" \
@@ -470,27 +520,25 @@ jobs:
          # cannot be parsed").
          CI_USER="$(printf '%s' "${REGISTRY_USER}" | tr -d '\r\n')"
          CI_TOKEN="$(printf '%s' "${REGISTRY_TOKEN}" | tr -d '\r\n')"
-          git config user.name  "installer-ci"
-          git config user.email "ci@whitlocktech.com"
          git remote set-url origin "https://${CI_USER}:${CI_TOKEN}@${GITEA_HOST}/${REPO}.git"

-          git add bundles
+          cd published
+          git add -A
          git commit -m "chore(bundle): publish ${TAG} (link ${{ steps.resolve.outputs.link_tag }}, overlay ${{ steps.resolve.outputs.overlay_tag }}, protocol ${{ steps.protocol.outputs.protocol }}) [skip ci]"

-          # The checkout is a detached snapshot of main; push the commit at HEAD
-          # to the branch the installer reads its raw URLs from. release.yml
-          # pushes its version-bump commit to the same branch, so losing the
-          # race is normal rather than exceptional — rebase and retry once
-          # instead of failing and leaving the bundle unpublished until the
-          # next cron. Only bundles/ is touched here, so a rebase over a bump
-          # commit cannot conflict.
-          if ! git push origin "HEAD:main"; then
-            echo "::warning::push rejected (main moved during compose) — rebasing and retrying once"
-            git fetch origin main
-            git rebase origin/main
-            git push origin "HEAD:main"
+          # Two runs can compose at once — a component release dispatches this
+          # while the nightly cron is mid-flight — so losing the race is normal
+          # rather than exceptional. Rebase and retry once instead of failing and
+          # leaving the bundle unpublished until tomorrow. Every file here is a
+          # bundle nobody else edits, and a bundle tag names exactly one matrix,
+          # so a rebase cannot conflict.
+          if ! git push origin bundles; then
+            echo "::warning::push rejected (the bundles branch moved during compose) — rebasing and retrying once"
+            git fetch origin bundles
+            git rebase origin/bundles
+            git push origin bundles
          fi
-          echo "==> published bundles/bundle-${TAG}.json and bundles/current.json"
+          echo "==> published bundle-${TAG}.json and current.json on the bundles branch"

      - name: Job summary
        if: always()
--- a/bundles/README.md
+++ b/bundles/README.md
@@ -13,7 +13,10 @@ nightly, so a missed dispatch self-heals. A run that finds nothing changed write

 See `docs/installer/PLAN.md` §7 for the design.

-## Layout
+## Where they live: the `bundles` branch
+
+**The JSON documents are not in this directory.** They are published to a branch of their own,
+[`bundles`](https://gitea.whitlocktech.com/RunicGateway/installer/src/branch/bundles), at its root:

 | File | What it is |
 |---|---|
@@ -24,14 +27,24 @@ Tags are UTC dates — `2026.08.04`. A second bundle on the same day (a sidecar
 morning, an overlay release in the afternoon) becomes `2026.08.04.2`, so one tag always names
 exactly one matrix.

+**Why a branch rather than `main`.** `main` is protected and this job is unattended: the pre-receive
+hook declines a push from CI, which is not something a nightly cron can resolve. A branch of its own
+keeps everything the original choice was for — a reviewable diff, a git history of the compat
+matrix, plain anonymous raw URLs, no credentials on the shard host — and needs no protection
+exception. Whitelisting a scheduled job for pushes to the default branch would buy nothing this does
+not.
+
+This directory keeps the documentation, because that is what belongs on `main`: the branch carries
+data, and only data.
+
 ## How the installer fetches these

 Plain anonymous `GET`s against a public repo. The shard host gets no git and no Gitea credentials
 (`PLAN.md` §1), so nothing here may require auth:

 ```
-https://gitea.whitlocktech.com/RunicGateway/installer/raw/branch/main/bundles/current.json
-https://gitea.whitlocktech.com/RunicGateway/installer/raw/branch/main/bundles/bundle-2026.08.04.json
+https://gitea.whitlocktech.com/RunicGateway/installer/raw/branch/bundles/current.json
+https://gitea.whitlocktech.com/RunicGateway/installer/raw/branch/bundles/bundle-2026.08.04.json
 ```

 Bundles are committed rather than published as Gitea releases because this repo's *own* releases are
@@ -56,8 +69,9 @@ protocol) or to either component's release version. All three move independently
    "tag": "v1.1.0",
    "version": "1.1.0",
    "protocol": 3,
-    "assets": {                        // per-platform: the installer runs on both
+    "assets": {                        // per-platform: the installer runs on each
      "linux-x86_64":   { "name": "…", "url": "…", "sha256": "…" },
+      "linux-aarch64":  { "name": "…", "url": "…", "sha256": "…" },
      "windows-x86_64": { "name": "…", "url": "…", "sha256": "…" }
    }
  },
--- a/bundles/bundle-2026.08.04.json
+++ b/bundles/bundle-2026.08.04.json
@@ -1,40 +0,0 @@
-{
-  "schema": 1,
-  "bundle": "2026.08.04",
-  "generated": "2026-08-04T16:07:13Z",
-  "protocol": 3,
-  "link": {
-    "repo": "RunicGateway/link",
-    "tag": "v1.1.0",
-    "version": "1.1.0",
-    "protocol": 3,
-    "assets": {
-      "linux-x86_64": {
-        "name": "uo-link-sidecar-linux-x86_64",
-        "url": "https://gitea.whitlocktech.com/RunicGateway/link/releases/download/v1.1.0/uo-link-sidecar-linux-x86_64",
-        "sha256": "27d491efda3fc6859dd38da9b2aa3b97b5fdf1dc5fc488a8916bb88b03443ad9"
-      },
-      "windows-x86_64": {
-        "name": "uo-link-sidecar-windows-x86_64.exe",
-        "url": "https://gitea.whitlocktech.com/RunicGateway/link/releases/download/v1.1.0/uo-link-sidecar-windows-x86_64.exe",
-        "sha256": "fbefd886af0355bf128f1f4c65657b772a58b128d32438061adb0069978e0b8f"
-      }
-    }
-  },
-  "overlay": {
-    "repo": "RunicGateway/servuo-plugins",
-    "tag": "v0.1.1",
-    "version": "0.1.1",
-    "commit": "3a52abbd77047e7c94883934533edcfef3ede555",
-    "protocol": 3,
-    "servuo": {
-      "min_version": "57.4",
-      "patches_verified_against": "57.4"
-    },
-    "asset": {
-      "name": "runicgateway-overlay-0.1.1.tar.gz",
-      "url": "https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/releases/download/v0.1.1/runicgateway-overlay-0.1.1.tar.gz",
-      "sha256": "75dc6d6ce08322b753a30303b3b2df6f15cf1e84658507d97430af44ec4d34d7"
-    }
-  }
-}
--- a/bundles/current.json
+++ b/bundles/current.json
@@ -1,40 +0,0 @@
-{
-  "schema": 1,
-  "bundle": "2026.08.04",
-  "generated": "2026-08-04T16:07:13Z",
-  "protocol": 3,
-  "link": {
-    "repo": "RunicGateway/link",
-    "tag": "v1.1.0",
-    "version": "1.1.0",
-    "protocol": 3,
-    "assets": {
-      "linux-x86_64": {
-        "name": "uo-link-sidecar-linux-x86_64",
-        "url": "https://gitea.whitlocktech.com/RunicGateway/link/releases/download/v1.1.0/uo-link-sidecar-linux-x86_64",
-        "sha256": "27d491efda3fc6859dd38da9b2aa3b97b5fdf1dc5fc488a8916bb88b03443ad9"
-      },
-      "windows-x86_64": {
-        "name": "uo-link-sidecar-windows-x86_64.exe",
-        "url": "https://gitea.whitlocktech.com/RunicGateway/link/releases/download/v1.1.0/uo-link-sidecar-windows-x86_64.exe",
-        "sha256": "fbefd886af0355bf128f1f4c65657b772a58b128d32438061adb0069978e0b8f"
-      }
-    }
-  },
-  "overlay": {
-    "repo": "RunicGateway/servuo-plugins",
-    "tag": "v0.1.1",
-    "version": "0.1.1",
-    "commit": "3a52abbd77047e7c94883934533edcfef3ede555",
-    "protocol": 3,
-    "servuo": {
-      "min_version": "57.4",
-      "patches_verified_against": "57.4"
-    },
-    "asset": {
-      "name": "runicgateway-overlay-0.1.1.tar.gz",
-      "url": "https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/releases/download/v0.1.1/runicgateway-overlay-0.1.1.tar.gz",
-      "sha256": "75dc6d6ce08322b753a30303b3b2df6f15cf1e84658507d97430af44ec4d34d7"
-    }
-  }
-}