feat(sidecar)!: protocol 4 — guild rosters, and a way to migrate the store (Teams cutover 2/6) #32

wtclaude · 2026-08-19T08:52:18Z

wtclaude commented

2026-08-19 08:52:18 +00:00

Cutover 2 of 6, and the pair to RunicGateway/servuo-plugins#14. PROTOCOL_VERSION goes to 4; the overlay declares protocol = 4. The installer refuses to pair a sidecar and an overlay that disagree, so these two merge together or the next bundle silently fails to compose.

What lands

members JSON on the guilds board, served from GET /guilds — the snapshot rule, so a website that connects fresh gets a roster without waiting for the next membership change.
A roster that arrives in several frames is reassembled rather than each frame overwriting the last. Found after the first cut; a 155-member guild does not fit one frame.
The store grew a migration mechanism, and this is the first change that needed one. SCHEMA was CREATE TABLE IF NOT EXISTS, which can add a table and cannot add a column — every schema change up to protocol 3 happened to add whole tables, so ALTER TABLE appears nowhere in this repo's history and the gap was invisible until guilds.members. Settled as PRAGMA user_version stepped migrations: each step transactional with the bump recording it, a failure aborts startup (already the behaviour, and safe because the shard dials out), and a database from a newer sidecar warns and continues so a binary rollback stays a recovery path. Not sqlx::migrate!, whose per-file checksums hard-fail startup if a released migration is ever edited.

Spec of record: docs/link/v4.md.

AI disclosure

Written with Claude Code (Opus 5). Commits carry the Co-Authored-By trailer.

**Cutover 2 of 6**, and the pair to `RunicGateway/servuo-plugins#14`. `PROTOCOL_VERSION` goes to **4**; the overlay declares `protocol = 4`. The installer refuses to pair a sidecar and an overlay that disagree, so **these two merge together** or the next bundle silently fails to compose. ### What lands - **`members` JSON on the `guilds` board**, served from `GET /guilds` — the snapshot rule, so a website that connects fresh gets a roster without waiting for the next membership change. - **A roster that arrives in several frames is reassembled** rather than each frame overwriting the last. Found after the first cut; a 155-member guild does not fit one frame. - **The store grew a migration mechanism, and this is the first change that needed one.** `SCHEMA` was `CREATE TABLE IF NOT EXISTS`, which can add a table and cannot add a column — every schema change up to protocol 3 happened to add whole tables, so `ALTER TABLE` appears nowhere in this repo's history and the gap was invisible until `guilds.members`. Settled as **`PRAGMA user_version` stepped migrations**: each step transactional with the bump recording it, a failure aborts startup (already the behaviour, and safe because the shard dials *out*), and a database from a *newer* sidecar warns and continues so a binary rollback stays a recovery path. Not `sqlx::migrate!`, whose per-file checksums hard-fail startup if a released migration is ever edited. Spec of record: [`docs/link/v4.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/edge/link/v4.md). ### AI disclosure Written with Claude Code (Opus 5). Commits carry the `Co-Authored-By` trailer.

wtclaude added 3 commits 2026-08-19 08:52:20 +00:00

feat(sidecar)!: guild rosters on the guild board, and a way to migrate the store 9216006208

Protocol 4 gives the guild board a real member list instead of the member
*count* that was all Protocol 2 could express. `guild.roster` carries the set;
`guild.leave` is forwarded but deliberately not projected.

The roster lives in its own `members` column rather than as a field folded into
`json`. That column holds the verbatim `guild.update` line, so a roster write
into it would clobber the snapshot — name, abbreviation, leader, online count —
that `guild.update` owns. Two writers across two columns of one row means both
stay plain upserts: neither reads the other's value first, so there is no
read-modify-write and no ordering requirement between the two kinds. `GET
/guilds` folds the roster back in as `roster` at read time.

`guild.leave` gets no board arm on purpose. The shard re-emits `guild.roster`
whenever the member set changes, so the board self-corrects within one sweep,
and keeping the delta out of the projection is what keeps the sidecar a
forwarder rather than a thing that maintains state.

This is also the repo's first store migration, and the reason it needed one:
`SCHEMA` is `CREATE TABLE IF NOT EXISTS`, which can add a table but cannot add a
column to a table that already exists. Every schema change up to and including
Protocol 3.0 happened to add whole tables, so `ALTER TABLE` appears nowhere in
this repo's history and the gap was invisible. `guilds.members` is the first
column added to an existing table, so without a mechanism the column would
simply never reach an installed sidecar and every roster write would fail.

The counter is SQLite's own `PRAGMA user_version` — an integer in the database
header, so it costs no table and cannot drift from the file it describes. Each
step runs in a transaction together with the bump recording it, so a step lands
completely or not at all. A database written by a *newer* sidecar warns and
continues rather than failing: every step is additive, so a newer schema has only
columns an older reader ignores, and refusing to start would turn rolling the
binary back — a recovery path — into a dead end.

A migration failure aborts startup, which was already the behaviour and is the
right one: a half-migrated store answers the website with confusing partial data,
and the shard dials *out*, so a sidecar that refuses to start never stalls the
game.

store.rs had no tests before this. The six added here cover the upgrade path that
matters (an existing pre-Protocol-4 database gains the column and lands at the
current version), that a restart re-running the migration is a no-op, that a
roster does not clobber the snapshot, that the two writers work in either order,
and that a guild with no roster yet has no `roster` key at all — "not known" and
"known to be empty" must not be conflated, or a website renders an empty roster
as fact.

Also gates PRs into `edge`, not just `main`. This workstream lands ten phases
there, and gating only the `main` hop would run these checks for the first time
at the cutover. The precedent and the reasoning are already in
RunicGateway/installer's copy of this workflow.

Refs: docs/website/TEAMS.md Part 12 Phase 1

Co-Authored-By: Claude <noreply@anthropic.com>

fix(sidecar): reassemble a guild roster that arrived in several frames

PR Checks / rust-gates (pull_request) Successful in 2m30s

Details

b00f2719a4

The shard caps members per `guild.roster` frame, so a guild over that cap emits
several frames carrying `seq`/`more`/`total`. The board's upsert wrote whichever
array it was handed, so each frame overwrote the last and only the final chunk
survived: a live 155-member guild, split 50/50/50/5, landed on the board with 5
members while `guild.update` correctly reported 155 beside it.

Every unit test passed through this, because they all exercised a single-frame
roster. Only the live rig caught it — the case does not arise until a guild
exceeds the cap.

Frames are now reassembled in memory and written once, on the frame that closes
the roster. The alternative — appending to the `members` column per frame — was
rejected twice over: it would make the write a read-modify-write, which is the
exact thing splitting the board across two columns exists to avoid, and it would
publish a torn roster, since a reader hitting GET /guilds between frames would
see a partial member list presented as the whole truth.

Buffering here does not make the sidecar stateful in the sense that matters. This
is transport-level reassembly — the same category of work as turning bytes into a
line — and it holds nothing once a roster is complete.

The ordinary case is unchanged and untouched by the buffer: a guild inside the
cap arrives as `seq` 0 with `more` false and is returned immediately, never
entering the map. What the buffer adds is the handling of everything that can go
wrong around a split roster: a fresh `seq` 0 supersedes an abandoned partial, an
out-of-order frame discards the partial rather than storing one with an
undetectable hole, a continuation with no start is ignored, a reconnect drops
every partial (the shard restarts each roster at 0), and accumulation is bounded
so a shard that never sends a closing frame cannot grow this map without limit.

Re-verified on the live rig: four frames reassembled to 153 entries after two
members were removed, with both departed serials absent.

Co-Authored-By: Claude <noreply@anthropic.com>

Merge pull request 'feat(sidecar)!: guild rosters on the guild board, and a way to migrate the store' (#31 ) from feat/teams-phase1-guild-roster into edge

PR Checks / rust-gates (pull_request) Successful in 3m11s

Details

2408d31ff7

Reviewed-on: #31

whitlocktech approved these changes 2026-08-19 08:55:26 +00:00

whitlocktech merged commit 7499e099f4 into main

2026-08-19 08:55:35 +00:00

whitlocktech referenced this issue from a commit

2026-08-19 08:55:36 +00:00

Merge pull request 'feat(sidecar)!: protocol 4 — guild rosters, and a way to migrate the store (Teams cutover 2/6)' (#32) from edge into main

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: RunicGateway/link#32