ci(release): show the error body, and retry the release POST #23

wtclaude · 2026-08-24T16:55:10Z

wtclaude commented

2026-08-24 16:55:10 +00:00

What happened

The release run for #22 (run 75) built everything and then failed at the last step. Steps 1–12 green, including Push the release tag; then Create Gitea release and upload assets took a 500 from POST /releases one second later and exited 22.

That left tag v0.1.1 at 9cc1099 with no release and no binaries — so the handoff fix in #22 reached no operator at all, and v0.1.0 stayed the current download. Found while writing runicgateway.com's installation docs, where I went to write "fixed in v0.1.1" and there was no such release.

Re-running the workflow published it — same four assets, untouched, via the orphan-tag recovery the plan step already has. So the 500 was a race with the tag push, not a bad request: Gitea had not finished processing the pushed tag when the POST arrived. v0.1.1 is now live.

The two gaps that made it worse than it needed to be

1. The failure was unreadable. curl -sSf prints no response body on an error status, so the entire log was:

curl: (22) The requested URL returned error: 500
##[error]Process completed with exit code 22.

The cause had to be inferred from timestamps. Every call in this step now captures the body and prints it on failure.

2. Nothing retried, and the recovery does not fire on its own. The plan step can adopt an orphan tag — but only on a run that reaches it, and a later push with no releasable commits sets RELEASE=false and stands down first. In practice the tag sits there until a human notices, which is exactly what happened. The POST now retries 5 times with a 5/10/15/20s backoff.

4xx is deliberately not retried — a bad token or a malformed body does not improve by being sent again, and retrying only turns a clear failure into a slow one. The give-up message names the orphan tag and says that a re-run republishes it.

The asset uploads get the same treatment: a release whose SHA256SUMS does not cover every binary it advertises is worse than no release, because that file is the trust anchor for an unsigned download.

Verification

No Rust changed. The step's shell was extracted from the YAML and the loop run against a stubbed curl:

PASS  first try succeeds                     rc=0, 0 retried attempt(s)
PASS  500 then success (the v0.1.1 race)     rc=0, 1 retried attempt(s)
PASS  two 500s then success                  rc=0, 2 retried attempt(s)
PASS  500 five times gives up                rc=1, 5 retried attempt(s)
PASS  403 aborts immediately                 rc=1, 1 retried attempt(s)
PASS  404 aborts immediately                 rc=1, 1 retried attempt(s)
PASS  000 (network) is retried               rc=0, 1 retried attempt(s)

bash -n clean, YAML parses, and the step contains no empty ${{ }} token — the trap that silently skipped three workflows for months.

Two notes

Typed ci(...) rather than fix(...) on purpose. The plan step bumps on feat/fix subjects; this changes no binary, so a fix(...) here would cut an empty v0.1.2 (and turn runicgateway.com's checkFacts red for nothing). Merging this should cut no release.

This file is link's release.yml retargeted, as its own header says — so the same two gaps very likely exist in link and servuo-plugins. Not touched here; say the word and it is a small pass across both.

AI-assisted: written with Claude Code (Opus 5).

🤖 Generated with Claude Code

## What happened The release run for #22 (run 75) **built everything and then failed at the last step**. Steps 1–12 green, including *Push the release tag*; then *Create Gitea release and upload assets* took a `500` from `POST /releases` **one second later** and exited 22. That left tag `v0.1.1` at `9cc1099` with **no release and no binaries** — so the handoff fix in #22 reached no operator at all, and `v0.1.0` stayed the current download. Found while writing runicgateway.com's installation docs, where I went to write "fixed in v0.1.1" and there was no such release. **Re-running the workflow published it** — same four assets, untouched, via the orphan-tag recovery the plan step already has. So the `500` was a **race with the tag push**, not a bad request: Gitea had not finished processing the pushed tag when the POST arrived. v0.1.1 is now live. ## The two gaps that made it worse than it needed to be **1. The failure was unreadable.** `curl -sSf` prints no response body on an error status, so the entire log was: ``` curl: (22) The requested URL returned error: 500 ##[error]Process completed with exit code 22. ``` The cause had to be inferred from timestamps. Every call in this step now captures the body and prints it on failure. **2. Nothing retried, and the recovery does not fire on its own.** The plan step *can* adopt an orphan tag — but only on a run that reaches it, and a later push with no releasable commits sets `RELEASE=false` and stands down first. In practice the tag sits there until a human notices, which is exactly what happened. The POST now retries **5 times with a 5/10/15/20s backoff**. `4xx` is deliberately **not** retried — a bad token or a malformed body does not improve by being sent again, and retrying only turns a clear failure into a slow one. The give-up message names the orphan tag and says that a re-run republishes it. The **asset uploads get the same treatment**: a release whose `SHA256SUMS` does not cover every binary it advertises is worse than no release, because that file is the trust anchor for an unsigned download. ## Verification No Rust changed. The step's shell was extracted from the YAML and the loop run against a stubbed `curl`: ``` PASS first try succeeds rc=0, 0 retried attempt(s) PASS 500 then success (the v0.1.1 race) rc=0, 1 retried attempt(s) PASS two 500s then success rc=0, 2 retried attempt(s) PASS 500 five times gives up rc=1, 5 retried attempt(s) PASS 403 aborts immediately rc=1, 1 retried attempt(s) PASS 404 aborts immediately rc=1, 1 retried attempt(s) PASS 000 (network) is retried rc=0, 1 retried attempt(s) ``` `bash -n` clean, YAML parses, and the step contains no empty `${{ }}` token — the trap that silently skipped three workflows for months. ## Two notes **Typed `ci(...)` rather than `fix(...)` on purpose.** The plan step bumps on `feat`/`fix` subjects; this changes no binary, so a `fix(...)` here would cut an empty v0.1.2 (and turn runicgateway.com's `checkFacts` red for nothing). Merging this should cut **no release**. **This file is `link`'s `release.yml` retargeted**, as its own header says — so the same two gaps very likely exist in `link` and `servuo-plugins`. Not touched here; say the word and it is a small pass across both. --- AI-assisted: written with Claude Code (Opus 5). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

wtclaude added 1 commit 2026-08-24 16:55:11 +00:00

ci(release): show the error body, and retry the release POST

PR Checks / rust-gates (pull_request) Successful in 1m32s

Details

188e6eb882

Run 75 built every artifact, pushed tag v0.1.1, then took a 500 from
POST /releases one second later and exited 22. The tag was left orphaned with
no release and no binaries, so the handoff fix in #22 reached no operator until
the workflow was re-run by hand today.

Re-running published the same four assets untouched, via the orphan-tag
recovery the plan step already has. So the 500 was a race with the tag push --
Gitea had not finished processing the pushed tag when the POST arrived -- and
not a bad request.

Two separate gaps made that worse than it needed to be.

`curl -sSf` prints no response body on an error status. All the log carried was
"curl: (22) ... error: 500", so the cause had to be inferred from timestamps
rather than read. Every call in this step now captures the body and prints it
on failure, including the asset uploads.

And nothing retried. The plan step can recover an orphan tag, but only on a run
that reaches it, and a later push with no releasable commits stands down before
it gets there -- which is why this one sat until someone looked. The POST now
retries five times with a 5/10/15/20s backoff.

4xx is deliberately not retried: a bad token or a malformed body will not
improve by being sent again, and retrying would turn a clear failure into a
slow one. A give-up message names the orphan tag and says a re-run republishes.

The asset uploads get the same treatment, because a release whose SHA256SUMS
does not cover every binary it advertises is worse than no release -- that file
is the trust anchor for an unsigned download.

Verified by extracting the step's shell from the YAML and running the loop
against a stubbed curl: first-try success, 500-then-success (the case that
actually happened), two 500s then success, five 500s giving up, 403 and 404
aborting without retrying, and a 000 network failure being retried. bash -n
clean and the YAML parses.

Typed ci(...) rather than fix(...) on purpose: the plan step bumps on feat/fix,
and this changes no binary, so a release here would be an empty one.

Co-Authored-By: Claude <noreply@anthropic.com>

whitlocktech approved these changes 2026-08-24 17:32:23 +00:00

whitlocktech merged commit 094da1776b into main

2026-08-24 17:32:29 +00:00

whitlocktech referenced this issue from a commit

2026-08-24 17:32:30 +00:00

Merge pull request 'ci(release): show the error body, and retry the release POST' (#23) from ci/release-post-retry-and-error-body into main

whitlocktech deleted branch ci/release-post-retry-and-error-body

2026-08-24 17:32:30 +00:00

wtclaude referenced this pull request

2026-08-24 18:58:33 +00:00

ci(release): sweep every tag for a missing release, not just this run's #24

Sign in to join this conversation.