ci(release): show the error body, retry the POST, and sweep for orphan tags
installer#22's release run built every artifact, pushed its tag, then took a 500
from POST /releases one second later and exited 22 -- leaving the tag orphaned
with no binaries. Re-running published it unchanged, so the 500 was a race with
the tag push rather than a bad request.
This repo's release step has the same two gaps verbatim, and it is where the
whole problem was first seen.
`curl -sSf` prints no response body on an error status, so such a failure leaves
only "curl: (22) ... error: 500" in the log and the cause has to be inferred
from timestamps. Every call now captures the body and prints it on failure.
Nothing retried, so a transient 5xx became a permanent orphan. The POST now
retries five times with a 5/10/15/20s backoff. 4xx is deliberately not retried:
a bad token or a malformed body will not improve by being sent again.
The asset upload gets the same treatment. That matters more here than anywhere
else: this release ships an overlay tarball and a SHA256SUMS, and a release
whose checksums do not cover the tarball they advertise is worse than no
release, because that file is the trust anchor and the installer verifies
against it.
The third gap is the one this repo proves. The orphan-tag recovery in the plan
step is VERSION-SCOPED -- it computes VERSION from the newest tag plus the bump,
then only checks refs/tags/v${VERSION}. That recovers an orphan on the very next
run and is useless afterwards, because once any releasable commit lands the next
run computes a NEW version and never looks at the old tag again.
v0.1.0 was the proof, and the proof is pointed: the commit that ADDED that
recovery was itself typed "fix(release): preflight credentials and recover the
orphaned v0.1.0 tag", so it bumped the version to v0.1.1 -- and the run that
introduced the recovery stepped straight past the tag it was written to rescue.
The tag stayed orphaned from 2026-08-04 until today.
So the plan step now sweeps every v* tag and warns about any without a release.
It warns rather than recovers, deliberately: publishing an old version would
mean building today's tree and shipping it under a tag whose tree it is not,
which is worse than the inconsistency it fixes. It never fails the run either --
a sweep that can break a good release is a sweep someone will delete.
v0.1.0 itself is deleted, on the org lead's decision. Nothing referenced it: it
is three releases behind, and no published bundle names it -- not even
bundle-2026.08.04, because the tag never had a release for a bundle to point at.
It was 724262548b, the merge of #7, recorded here
so the tag can be recreated if that turns out to be wrong.
Verified by extracting every run block from the YAML: bash -n clean across all
of them, the YAML parses, no empty template token, the asset loop still the
tarball-and-checksums pair rather than link's three binaries, the retry loop
exercised against a stubbed curl across seven cases, and the sweep run against
the real repositories -- reporting v0.1.0 before the deletion and clean after.
Typed ci(...) rather than fix(...) on purpose: the plan step bumps on feat/fix,
and this changes no artifact, so a release here would be an empty one. That is
the same rule the fix commit above tripped over.
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -176,6 +176,39 @@ jobs:
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── Orphan sweep ────────────────────────────────────────────────
|
||||
#
|
||||
# The check above is VERSION-SCOPED: it only ever asks about the one
|
||||
# version this run computed. That is enough to recover an orphan on
|
||||
# the very next run, and useless afterwards — once any releasable
|
||||
# commit lands, the next run computes a NEW version, never looks at
|
||||
# the old tag again, and the orphan becomes permanent and silent.
|
||||
#
|
||||
# servuo-plugins v0.1.0 is the proof, and the proof is pointed: the
|
||||
# commit that ADDED the recovery above was itself typed
|
||||
# `fix(release): ... recover the orphaned v0.1.0 tag`, so it bumped to
|
||||
# v0.1.1 — and the run that introduced the recovery stepped straight
|
||||
# past the tag it was written to rescue. That tag is still orphaned.
|
||||
#
|
||||
# So every v* tag is checked, and anything missing a release is
|
||||
# WARNED about. Deliberately not recovered: publishing an old version
|
||||
# would mean building today's tree and shipping it under a tag whose
|
||||
# tree it is not, which is worse than the inconsistency it fixes.
|
||||
# A human decides whether to recover or drop it.
|
||||
#
|
||||
# Never fails the run. A sweep that can break a good release is a
|
||||
# sweep someone will delete.
|
||||
ORPHANS=""
|
||||
for T in $(git tag -l 'v*' --sort=-v:refname); do
|
||||
T_HTTP="$(curl -s -o /dev/null -w '%{http_code}' \
|
||||
-H "Authorization: token $(printf '%s' "${REGISTRY_TOKEN:-}" | tr -d '\r\n')" \
|
||||
"https://${GITEA_HOST}/api/v1/repos/${REPO}/releases/tags/${T}" || echo 000)"
|
||||
[ "$T_HTTP" = "404" ] && ORPHANS="${ORPHANS} ${T}"
|
||||
done
|
||||
if [ -n "${ORPHANS}" ]; then
|
||||
echo "::warning::Tags with no release:${ORPHANS} — a run failed after tagging. Publish or delete them; this job will not do either."
|
||||
fi
|
||||
|
||||
# Changelog range. A recovery run has nothing after the tag, so
|
||||
# summarize what the tag itself contains rather than emitting an empty
|
||||
# list: the range that produced it, i.e. previous-tag..this-tag.
|
||||
@@ -477,18 +510,73 @@ jobs:
|
||||
# corrupt the Authorization header.
|
||||
CI_TOKEN="$(printf '%s' "${REGISTRY_TOKEN}" | tr -d '\r\n')"
|
||||
|
||||
REL_ID="$(curl -sSf -X POST "${API}/releases" \
|
||||
-H "Authorization: token ${CI_TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$(jq -n --arg tag "$TAG" --arg body "$BODY" \
|
||||
'{tag_name:$tag, name:$tag, body:$body, draft:false, prerelease:false}')" \
|
||||
| jq -r '.id')"
|
||||
PAYLOAD="$(jq -n --arg tag "$TAG" --arg body "$BODY" \
|
||||
'{tag_name:$tag, name:$tag, body:$body, draft:false, prerelease:false}')"
|
||||
|
||||
# installer#22's release run failed exactly here: it landed one second
|
||||
# after the tag push and Gitea answered 500, having not finished
|
||||
# processing the pushed tag. Re-running published the same artifacts
|
||||
# untouched, so it was a race, not a bad request — but the tag sat
|
||||
# orphaned until a human noticed.
|
||||
#
|
||||
# Two things made that worse than it needed to be.
|
||||
#
|
||||
# 1. `curl -sSf` prints NO response body on an error status, so all the
|
||||
# log carried was "curl: (22) ... error: 500" and the cause had to be
|
||||
# inferred from timestamps. Capture the body and print it.
|
||||
# 2. Nothing retried, so a transient 5xx became a permanent orphan.
|
||||
#
|
||||
# 4xx is deliberately NOT retried: a bad token or a malformed body does
|
||||
# not improve by being sent again, and retrying only turns a clear
|
||||
# failure into a slow one.
|
||||
REL_ID=""
|
||||
for attempt in 1 2 3 4 5; do
|
||||
HTTP="$(curl -s -o /tmp/rel.json -w '%{http_code}' -X POST "${API}/releases" \
|
||||
-H "Authorization: token ${CI_TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "${PAYLOAD}" || echo 000)"
|
||||
|
||||
if [ "$HTTP" = "201" ] || [ "$HTTP" = "200" ]; then
|
||||
REL_ID="$(jq -r '.id' /tmp/rel.json)"
|
||||
break
|
||||
fi
|
||||
|
||||
echo "::warning::POST /releases attempt ${attempt} returned HTTP ${HTTP}"
|
||||
echo "--- response body ---"
|
||||
cat /tmp/rel.json || true
|
||||
echo
|
||||
echo "---------------------"
|
||||
|
||||
case "$HTTP" in
|
||||
4*) echo "::error::HTTP ${HTTP} is a client error - not retrying."; exit 1 ;;
|
||||
esac
|
||||
|
||||
if [ "$attempt" = 5 ]; then
|
||||
echo "::error::POST /releases still failing after 5 attempts. Tag ${TAG} is pushed but has no release."
|
||||
echo "::error::Re-run this workflow - the plan step detects the orphan tag and republishes it."
|
||||
exit 1
|
||||
fi
|
||||
sleep $(( attempt * 5 ))
|
||||
done
|
||||
|
||||
if [ -z "$REL_ID" ] || [ "$REL_ID" = "null" ]; then
|
||||
echo "::error::Release created but no id came back; refusing to upload assets blind."
|
||||
exit 1
|
||||
fi
|
||||
echo "Created release ${TAG} (id=${REL_ID})"
|
||||
|
||||
for f in "${TARBALL}" SHA256SUMS; do
|
||||
curl -sSf -X POST "${API}/releases/${REL_ID}/assets?name=${f}" \
|
||||
# Same treatment. An upload that fails quietly leaves a release whose
|
||||
# SHA256SUMS does not cover every artifact it advertises, which is
|
||||
# worse than no release at all -- that file is the trust anchor.
|
||||
HTTP="$(curl -s -o /tmp/asset.json -w '%{http_code}' -X POST "${API}/releases/${REL_ID}/assets?name=${f}" \
|
||||
-H "Authorization: token ${CI_TOKEN}" \
|
||||
-F "attachment=@dist/${f}" >/dev/null
|
||||
-F "attachment=@dist/${f}" || echo 000)"
|
||||
if [ "$HTTP" != "201" ] && [ "$HTTP" != "200" ]; then
|
||||
echo "::error::uploading ${f} returned HTTP ${HTTP}"
|
||||
cat /tmp/asset.json || true
|
||||
exit 1
|
||||
fi
|
||||
echo " uploaded ${f}"
|
||||
done
|
||||
|
||||
|
||||
Reference in New Issue
Block a user