RunicGateway/docs
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(events): the cutover, the outage it survived, and the defect it exposed (Phase 16b cutover, 6 of 6) #231

Merged
whitlocktech merged 2 commits from docs/events-p16b-cutover into edge 2026-09-10 02:14:25 +00:00
Conversation 0 Commits 2 Files Changed 1 +141 -1
1 changed files with 141 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

142
website/EVENTS_PLAN.md
View File

@@ -1879,9 +1879,149 @@ core, docs, then the kit's re-pin and `runicgateway.com`.
> a refusal releases its key, with the rule that pays for it written down — *do not answer `*.error` > a refusal releases its key, with the rule that pays for it written down — *do not answer `*.error`
> after changing the world*. > after changing the world*.
> **16b CUT OVER (2026-09-09/10) — six steps, and core before the module.** The protocol pair
> (`link#40` + `servuo-plugins#26`) is ONE step, not two: `bundle.yml`'s Gate 1 reads the protocol
> number out of both released artefacts and refuses a pair that disagrees, so whichever lands first
> leaves a compose that cannot run. Then core (`website#199`), the module carrying its own re-pin
> (`Module-uo#34`), the app (`Android-app#46`), the kit's re-pin (`Integration-kit#11`), and docs.
>
> **Core lands before the module**, which departs from the sentence above and matches what the
> engagement cutover actually did: `Module-uo`'s `ci/core-ref.json` has to name a website `main` sha
> carrying MODULE_API 1.10.0, and that sha does not exist until core has landed. Four decisions, all
> as recommended (org lead, 2026-09-09): that order; the app merges with **no `v*` tag**, so no APK
> was cut; the re-verify walks the whole delivery path; and **`edge` stays standing** in every repo
> rather than being deleted as the module-system cutover deleted its own.
>
> Releases cut: sidecar **v2.2.0**, overlay **v1.2.0**, bundle **2026.09.10 (protocol 7)**. `website`
> never releases. `MODULE_API_VERSION` and `EVENTS.md` are untouched by this leg — the cutover moves
> no contract.
>
> **`Integration-kit#10` had been merged early**, on 2026-09-08, though it was written to be held —
> so the kit's `main` was red on `checkCoreApi` for two days. That is what step 5 closes, and it is
> the reason the re-pin is a repair rather than only a date.
>
> #### The outage, and what it did not break
>
> Gitea was unreachable for about ten minutes in the middle of the window (Cloudflare 1033/530) and
> killed **both** release runs. `link`'s built every binary and wrote `SHA256SUMS`, then died pushing
> the tag: `fatal: unable to access … The requested URL returned error: 530`. `servuo-plugins`' died
> inside `Set up job` after 11m52s with no step ever executing — which is why that job's log route
> answers 500 while its predecessor's serves fine: **there is no log blob, and that absence is
> evidence.** No tag was pushed either time, so the orphaned-tag failure mode did not occur, and
> re-running both by `workflow_dispatch` published them. The first to land left the pair mismatched
> and compose run 102 failed exactly as the PRs predicted; the second dispatched it again and 103
> composed. `link`'s `rust-gates` reds on three earlier PRs were `curl: (6) Could not resolve host:
> sh.rustup.rs` inside the runner — infrastructure, not code, on all four counts.
>
> **A job's log IS readable on this instance, through the web route rather than the API:**
> `/{owner}/{repo}/actions/runs/<n>/jobs/<j>/logs` with an API token, served as `text/plain`; step
> statuses come from the UI's own POST endpoint with a `_csrf` cookie. Every earlier phase diagnosed
> CI by reproducing jobs locally, on the belief that logs were unreachable. They are not, and reading
> one is what turned four red X's into four known causes in about ten minutes.
>
> #### A seventh defect, red on every events PR since Phase 10
>
> `website`'s `server-tests` job had been failing since `#192` — eight PRs, every one reporting
> `# fail 1`, always **the same single test**, so nothing else was ever hiding behind it. The
> workstream merged over it eight times.
>
> `events/announce.js` asked `Intl.DateTimeFormat('en-GB', { …, hour12: true })`, and **that is not
> the same request as a 12-hour clock.** For a locale whose default cycle is h23 — `en-GB` is one —
> Node 20 resolves `hour12: true` to **`h11`**, whose hours run 0–11, so midnight renders `0:00 am`;
> Node 22 and later resolve it to `h12` and it renders `12:00 am`. **Same ICU (78.2) on both sides**,
> so this is V8's ECMA-402 behaviour and not locale data — no amount of matching the runner's locale
> would have found it.
>
> The image ships `node:20-alpine` and CI runs Node 20, while a dev machine is newer. So the mail
> every real recipient got said **"0:00 am"** beside a schedule editor saying "12:00 AM" — one
> instant, two spellings, the exact contradiction that option was added to prevent — and it rendered
> correctly in front of everyone who reviewed it. Fixed to `hourCycle: 'h12'` (`website#200`), which
> is the form `recurrence.js` had already adopted for the mirror-image case (`h23` **rather than**
> `hour12: false`); `announce.js` was the last `hour12` in either repo.
>
> **The rule: `hour12` is a request about a locale's preference, `hourCycle` is a request about the
> clock. Ask for the clock.** And the test now says so out loud, because it can only fail on Node 20:
> a green run on a dev machine is not evidence, and CI is what holds that line.
>
> #### The re-verify, from artefacts an operator would download
>
> This is the leg 16a could not do — a locally built bundle cannot go through core's module installer,
> which is https-only with a host allowlist.
>
> | | |
> |---|---|
> | installer | released `v0.1.1` binary, checksum matched against the release's own `SHA256SUMS` |
> | bundle | resolved **2026.09.10, protocol 7**; both component checksums verified by the installer |
> | overlay sync | a **first install** into a stock 57.4 tree — `add=30 change=1 unchanged=0` |
> | script build | `0 Warning(s) 0 Error(s)` — the released overlay compiles on a stock tree, which no release had ever been asked to prove |
> | shard boot | `[Bridge] enabled=True … adminWrite=True … events=True`, then `connected to 127.0.0.1:7788` |
> | sidecar | `server.hello` for **208,568 items / 42,871 mobiles**; `x-uolink-version: 7` |
> | event plane | `lease.list.ok` (config **and** targeted property leases), `item.catalog.ok` with its bounds, `GET /world/<run>` an empty list rather than a 404 |
> | core | released `main` on a throwaway database, `capabilities: ["events"]` on `/public/version` |
> | one event | published, run, **`completed` / `health: ok`**, results published |
> | the page | `/site/events` reads *"Everything scheduled, live and recently finished"* and lists a run that finished two minutes earlier |
>
> The last row is 16a's calendar fix holding on `main`: before it, a run that had already started or
> finished was absent and the page rendered `entries: []`.
>
> **A fresh `Bridge.cfg` still ships `EventsEnabled=false` and `AdminWriteEnabled=false`** — the
> operator's real first-boot state, and the released config confirms it rather than a working tree's.
>
> One thing checked and deliberately **not** reported as a defect: a **cancelled** run appears on the
> public calendar. It is meant to. The entry carries its own `status`, and the page renders a past
> cancelled run as **"Did not happen"** — the honest label, not a silent omission.
>
> **The module's own install was walked too**, once step 3 cut `Module-uo` **v1.2.1**. The module
> arrived the way an operator's would: `POST /admin/modules` naming the release's **manifest** (not
> its tarball — core answers a tarball with *"the install manifest is larger than 262144 bytes"*,
> which is the size guard doing its job), core fetched the artifact over https from the allowlisted
> host, verified its `sha256`, and mounted it on the next boot with 12 event actions and 27 triggers.
> Then `PUT /admin/uo-link/config` with the four values the installer printed answered
> **`status: connected`, `pluginConnected: true`, `protocol: 7`** — released core, released module,
> released sidecar, released overlay, all four talking.
>
> On that rig the two Phase 16a fixes were confirmed in the shipped artefacts rather than in a working
> tree: the atlas **imported off a stock tree** (309 decor types, 6,455 points, 800 creatures, 558
> landmarks, 387 regions, 25 champions — the import that used to die at 313), and a world verb ran and
> **tore down for real** — three orcs spawned, ledger `reverted` ×3, `cleanup: complete`, and the shard
> itself answering `world.owned → owned: [], pruned: 0`. That last check is the one 16a's no-op
> teardown hid behind. The enablement gate and the cap behaved as specified on the way past: the dry
> run refused the action before it was enabled, then priced it `uo.creatures 3 of 10`.
>
> **And the leg found two more defects, both in the released bundle and neither visible to any test**
> (`Module-uo#35`).
>
> **The aggregator discarded the `UniqueId`, so no Phase 12b property lease was authorable at all.**
> All 6,455 spawn points imported with `unique_id` NULL; `listSpawners` filters
> `unique_id IS NOT NULL`, so `uo.options.spawners` — the only source those leases have — was an empty
> dropdown with nothing to explain itself. Every part of the path was right except one line: the files
> carry `<UniqueId>`, `parsePoints` returns it, the column exists, the insert passes it. `buildAtlas`
> rebuilds each point from an explicit field list and the field was not on it. **`PARSER_VERSION = 4`'s
> own note says a point keeps its `UniqueId` and names Phase 12b as the reason** — that bump exists to
> re-read trees for this field, and the field was dropped one function later. The intent shipped as a
> comment. Fixing it needs `PARSER_VERSION` 5 as well, because the tree's hashes have not changed —
> only what is kept from them — so nothing would re-read an existing install.
>
> **A landmark option value named 23 places at once.** 558 landmarks, 320 distinct `facet/name`:
> `Trammel/Entrance` is Blighted Grove, Covetous, Deceit, Despise, Destard and 18 more, and
> `landmarkPoint` resolves with `.find()`. So 22 of the 23 were unreachable and an author who picked
> "Entrance — Destard" got Blighted Grove, with a successful run and no warning. **The group was
> already the disambiguator** — shown in the dropdown, left out of the value. Now `facet/group/name`,
> distinct across all 558, with the two-part read kept as a fallback because a published version is
> immutable and those stored values are the authored record. A three-part value whose group is gone
> refuses rather than falling back: it asked for one place.
>
> Both are the same failure shape as 16a's blocking defect and worth naming as a class: **an option
> source that answers empty, or answers with a value that does not identify one thing, disables a
> feature silently.** Nothing errors, the form simply cannot express the thing — and a test that
> checks the parser, or the query, or the column in isolation passes throughout. The atlas fixture had
> no `<UniqueId>` in it at all until this phase, which is why a green suite said nothing for two.
**Two documents that are cutover-window work by construction.** **Two documents that are cutover-window work by construction.**
- **`runicgateway.com`** — `checkFacts` reads `main`, so any claim about events is unverifiable until - **`runicgateway.com`** — `checkFacts` reads `main`, so any claim about events is unverifiable until
the cutover lands. Same 12a/12b split the engagement workstream needed. the cutover lands. Same 12a/12b split the engagement workstream needed. **16b landed it**, so both
of these are now unblocked: `main` carries the engine, the module and the app, and the bundle triple
the site quotes is sidecar **v2.2.0** / overlay **v1.2.0** / bundle **2026.09.10**.
- **`.profile`** — the org landing page is updated when the *shape* of the project changes, which a new - **`.profile`** — the org landing page is updated when the *shape* of the project changes, which a new
subsystem is. subsystem is.