RunicGateway/docs
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(link): protocol 4 — guild membership on the wire

Browse Source 
Adds v4.md as the spec of record for `guild.roster` and `guild.leave`, and
corrects the two older documents that Protocol 4 makes wrong.

PROTOCOL_2.md §10.1 already described this design — hold a member-serial set,
diff it each sweep, emit join/leave — and 2.0 then shipped only the half needing
no new state, folding membership into the board signature as a serial *sum*. The
section has read ever since as though the whole thing were built. It now says
which half shipped, and carries the correction that doing it produced: a sum is
not a safe stand-in for a set, because one member joining and another leaving
between two sweeps offset each other and the guild reads as unchanged.

INTEGRATION.md gains both kinds in the event catalogue, the `roster` key on
GET /guilds, and the three things an integrator gets wrong otherwise — that
`guild.leave`'s `who` is a bare serial rather than an actor object (the mobile has
already left, so there is nothing to attribute), that `acct` is genuinely optional
on a member, and that a guild with no `roster` key is not the same as one with an
empty roster.

v4.md documents what the phase found as well as what it built: the missing store
migration and the user_version decision, why the roster lives in its own column
rather than inside the guild.update snapshot, why a split roster is reassembled in
memory rather than appended to the column, and why guild.leave gets no board
projection at all. §6 records that the reassembly bug was invisible to every unit
test — they all exercised single-frame rosters — and only the live rig caught it.

TEAMS.md is amended where this phase disagreed with it: Phase 1 spans five repos,
not four, because installer/backup.rs justifies skipping the sidecar database on
reasoning the migration falsifies. The user_version decision is recorded there too,
since the design of record did not contemplate a migration mechanism at all.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
claude
2026-08-17 13:01:05 -05:00
parent 3bfa309a40
commit b369e728c2
4 changed files with 293 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
link/INTEGRATION.md
View File

@@ -283,11 +283,19 @@ Guilds expose only one in-game event (a member joining), so the roster is polled
| kind | fields | notes |
|------|--------|-------|
| `guild.update` | `id`, `name`, `abbr`, `members`, `online`, `alliance` (or null), `leader` (actor object or null) | A guild's roster/leader/alliance changed, or its first sight this connection. A **leave** shows up here as `members` dropping. |
| `guild.update` | `id`, `name`, `abbr`, `members`, `online`, `alliance` (or null), `leader` (actor object or null) | A guild's leader/alliance/name changed, its member count moved, or its first sight this connection. |
| `guild.remove` | `id` | The guild disbanded (leader gone) or was removed. Drop the row. |
| `guild.join` | `id`, `name`, `abbr`, `who` (actor object) | Real-time: a player joined a guild (`EventSink.JoinGuild`). |
| `guild.roster` **(4)** | `id`, `name`, `abbr`, `total`, `seq`, `more`, `members` (array of actor objects) | The full member list. Emitted whenever the member set changes. **`seq` 0 supersedes whatever roster you hold for that guild; `more: false` ends it.** |
| `guild.leave` **(4)** | `id`, `name`, `who` (serial string) | Real-time: a member left. Advisory — see below. |
The `leader`/`who` **actor object** is `{serial, name, acct?, webId?, player}` — `acct`/`webId` present when the mobile has an account / a linked website user.
The `leader`/`who` **actor object** is `{serial, name, acct?, webId?, player}` — `acct`/`webId` present when the mobile has an account / a linked website user. Note `guild.leave`'s `who` is a bare **serial string**, not an actor object: the mobile has already left, so there is nothing to attribute.
**On Protocol 4.** Before it, a guild's membership was a *count* and a leave surfaced only as that count dropping. `guild.roster` carries the members themselves, and `guild.leave` names who went.
`guild.leave` is **advisory**: any change to the member set re-emits the whole roster, so a consumer holding a membership table stays correct even if it ignores every leave event. Handle it when you want a "so-and-so left" feed to update without waiting for the sweep.
**Rosters can arrive in several frames.** Members per frame are capped so a large guild cannot produce an unbounded line (~69 bytes per member; the default cap is 500). Every realistic guild arrives as one frame with `seq: 0, more: false` and needs no special handling — but if you consume the raw stream, accumulate from `seq` 0 and apply on `more: false`, discarding a partial roster if a frame arrives out of order or the shard reconnects. `GET /guilds` hands you rosters already reassembled. A guild with no members emits one frame with an empty array, so an emptied roster is distinguishable from an absent one.
```json
{"kind":"guild.update","id":1042,"name":"The Silver Hand","abbr":"TSH","members":14,
@@ -296,8 +304,16 @@ The `leader`/`who` **actor object** is `{serial, name, acct?, webId?, player}`
"t":1752489280000}
{"kind":"guild.join","id":1042,"name":"The Silver Hand","abbr":"TSH",
"who":{"serial":"0x77","name":"Bran","acct":"bran","player":true},"t":1752489281000}
{"kind":"guild.roster","id":1042,"name":"The Silver Hand","abbr":"TSH",
"total":14,"seq":0,"more":false,
"members":[{"serial":"0x1A2B","name":"Darrow","acct":"whitlocktech","webId":"9931","player":true},
{"serial":"0x77","name":"Bran","acct":"bran","player":true}],
"t":1752489282000}
{"kind":"guild.leave","id":1042,"name":"The Silver Hand","who":"0x77","t":1752489283000}
```
`acct` is genuinely optional on a member — a character can have no account at all — so do not assume it is present.
Render the current board from `GET /guilds` (§6) on connect, then keep it live with these events.
#### Town governors (Protocol 2.0)
@@ -853,7 +869,17 @@ GET /guilds
"t":1752489280000}, ... ] }
```
Every guild's latest roster snapshot at once — the live board. Served from the sidecar's projection (no shard round-trip), kept current by the `guild.*` stream (§4). Render on load, then subscribe. Each entry is exactly a `guild.update` payload; ordered by name. Survives a sidecar restart.
Every guild's latest snapshot at once — the live board. Served from the sidecar's projection (no shard round-trip), kept current by the `guild.*` stream (§4). Render on load, then subscribe. Ordered by name. Survives a sidecar restart.
Each entry is a `guild.update` payload **plus, from Protocol 4, a `roster` key** holding the member list — already reassembled, so the frame-splitting described in §4 never reaches this endpoint:
```
→ { "guilds": [ {"kind":"guild.update","id":1042, ..., "roster":[
{"serial":"0x1A2B","name":"Darrow","acct":"whitlocktech","webId":"9931","player":true},
{"serial":"0x77","name":"Bran","acct":"bran","player":true} ]}, ... ] }
```
A guild that has had a `guild.update` but no roster yet has **no `roster` key at all** — deliberately distinct from `"roster": []`, which means the guild is genuinely empty. Do not conflate "not known" with "known to be empty".
### Governor board (Protocol 2.0)

8
link/PROTOCOL_2.md
View File

@@ -236,6 +236,14 @@ These are **outbound** streams (shard → website), the natural extension of `PL
### 10.1 Guilds
> **Built in two halves.** 2.0 shipped the board (`guild.update` / `guild.remove`) plus the live
> `guild.join`, and folded membership into the board signature as a serial *sum* — so the site got a
> member **count** and no roster, and the `guild.leave` sketched below was never emitted. **Protocol 4
> builds the rest of what this section describes**: the member-serial set, its diff, `guild.roster`
> and `guild.leave`. See [`v4.md`](v4.md). One correction from doing it — a *sum* is not a safe stand-in
> for a set, because one member joining and another leaving between two sweeps offset each other and
> the guild reads as unchanged. v4 holds the real set.
**Hook reality (verified):**
- `EventSink.JoinGuild` is real — raised at `Scripts/Misc/Guild.cs:1597` when a mobile joins a guild. Usable as a live `guild.join`.

232
link/v4.md Normal file
View File

@@ -0,0 +1,232 @@
# Protocol 4 — Guild membership on the wire
**Status:** Feature-complete on `edge`. All work lands on an `edge` branch in each repo; `edge` → `main` is the v4 cutover.
**Date:** 2026-08-17
**Codebase:** ServUO 57.4, `<servuo>`, net48 / x64, Expansion **EJ**.
**Companion to** [`PLAN.md`](PLAN.md) (1.0 read/event plane), [`PROTOCOL_2.md`](PROTOCOL_2.md) (2.0 provisioning + world-state streams), [`v3.md`](v3.md) (3.0 shard content + the visibility framework), [`INTEGRATION.md`](INTEGRATION.md) (website API).
---
## 1. Why 4
Protocol 2 gave the website a guild board: name, abbreviation, leader, alliance, and a member
**count**. It could say a guild had 155 members. It could not say who they were, and there was no
event for anyone leaving one.
That gap was known when 2.0 shipped. [`PROTOCOL_2.md` §10.1](PROTOCOL_2.md) sketched exactly this
design — hold a member-serial set, diff it each sweep, emit `guild.join`/`guild.leave` — and then
2.0 shipped only the half that needed no new state, folding membership into the board signature as a
serial *sum*. Protocol 4 builds the rest of what §10.1 described.
The immediate consumer is a richer public Guilds page, which is worth the bump on its own merit. The
reason it is being built **now** is that [`../website/TEAMS.md`](../website/TEAMS.md) needs it:
platform Teams are sourced from game guilds, and nothing in Team core can be built against a count.
That is a dependency, not a coupling — nothing in this protocol version knows what a Team is.
**Two kinds, both additive:**
| Kind | Shape | Purpose |
|---|---|---|
| `guild.roster` | full member list, possibly split across frames | the membership itself; supersedes whatever was held |
| `guild.leave` | one departing member | the real-time counterpart to the existing `guild.join` |
Nothing existing changed shape. `GET /guilds` grows a `roster` key; a v3 consumer that ignores it
keeps working.
---
## 2. The shard side
`BridgeSocial.cs` holds each guild's member serial **set** instead of folding it into the signature
as a sum. Two consequences, both the point of the change:
- **A set comparison cannot collide.** The old sum could: one member joining and another leaving
between two sweeps offset each other, and the guild read as unchanged.
- **A set can be differenced.** Departures are the prior set minus the current one — which is what
makes a per-member `guild.leave` possible without a core tap, since ServUO raises no event for
leaving, disbanding, or a leader change.
A changed set re-emits `guild.roster`, the whole member list. That is what lets `guild.leave` stay
**advisory**: a consumer building a "so-and-so left" feed wants the individual events, but a consumer
holding a membership table only needs the roster, so nothing downstream has to replay deltas to
remain correct. A dropped `guild.leave` costs latency, never accuracy.
On a guild's **first** sweep there is no prior set, so nothing is reported as leaving. An unknown
roster becoming known is not 155 people leaving at once.
### 2.1 Frames are capped, and split when they must be
A roster is the only fat frame this bridge emits. Measured against a real 155-member guild: **10,812
bytes, about 69 bytes per member.** The sidecar's `read_line` has no length bound, so an uncapped
roster is an unbounded line.
Members per frame are therefore capped (`Bridge.GuildRosterMembersPerLine`, default 500 ≈ 35 KB), and
a guild over the cap is split into frames carrying `seq`, `more` and `total`:
```jsonc
{"t":1786988708513,"kind":"guild.roster","id":1,"name":"The Silver Hand","abbr":"TSH",
"total":155,"seq":0,"more":true,"members":[ /* 50 actor objects */ ]}
{"t":1786988708514,"kind":"guild.roster","id":1,"name":"The Silver Hand","abbr":"TSH",
"total":155,"seq":1,"more":true,"members":[ /* 50 */ ]}
{"t":1786988708514,"kind":"guild.roster","id":1,"name":"The Silver Hand","abbr":"TSH",
"total":155,"seq":2,"more":true,"members":[ /* 50 */ ]}
{"t":1786988708514,"kind":"guild.roster","id":1,"name":"The Silver Hand","abbr":"TSH",
"total":155,"seq":3,"more":false,"members":[ /* 5 */ ]}
```
Reading the flags: **`seq` 0 begins a roster and supersedes whatever was held for that guild**;
`more: false` ends it. Every realistic guild is inside the cap and emits exactly one frame with
`seq: 0, more: false` — the same shape as if chunking did not exist. A guild with no members still
emits one frame with an empty array, or a consumer could never learn that a roster it holds has
emptied.
`guild.leave` is a small frame naming the departed serial:
```jsonc
{"t":1786988770099,"kind":"guild.leave","id":1,"name":"The Silver Hand","who":"0x1F5"}
```
### 2.2 The reconnect baseline is spread
`BridgeLink.OnConnected` clears the diff caches, so after a reconnect **every** guild reads as
changed at once. The outbound queue is not the risk — the cap is 10,000 *lines* and a few hundred
guilds is a few hundred lines — but building hundreds of fat JSON frames in a single Core-thread tick
is exactly the stall this bridge exists to avoid.
So at most `Bridge.GuildRosterGuildsPerTick` guilds (default 25) emit a roster per sweep. A guild over
budget keeps its previously held member set, so it still reads as changed on the next pass; its
`guild.update` has already gone, so the board's counts are current either way. While a baseline is
draining the sweep re-arms itself after 2 s rather than waiting a full `GuildSweepSeconds`, so
catch-up takes seconds instead of one sweep interval per batch.
### 2.3 What a roster member carries
Each entry is the standard actor object — `serial`, `name`, `player`, plus `acct` when the mobile has
an account and `webId` when that account is linked:
```jsonc
{"serial":"0x1F5","name":"Seed000A","acct":"seed_000","webId":"42","player":true}
```
`acct` is **genuinely optional**: a `PlayerMobile` can have no `Account` at all, and the local test
world contains such mobiles. Consumers must not assume it is present.
These identity fields are emitted unconditionally, by design — the sidecar is a forwarder, and
deciding who may see them is the website's job. See §4.
---
## 3. The sidecar side
`guild.roster` writes a **`members` column on the `guilds` board**, not a field inside the existing
`json` column. That column holds the verbatim `guild.update` line, and a roster written into it would
clobber the snapshot — name, abbreviation, leader, online count — that `guild.update` owns. Two
writers across two columns of one row keeps both as plain upserts: neither reads the other's value
first, so there is no read-modify-write and no ordering requirement between the two kinds. Either can
arrive first.
`GET /guilds` folds the roster back in as `roster` at read time. A guild that has had a
`guild.update` but no `guild.roster` yet simply has **no `roster` key** — the honest representation
of "not known", and deliberately distinct from a guild whose roster is genuinely empty.
`guild.leave` gets **no board projection at all**. The sidecar persists and broadcasts it like any
event and leaves the board alone; the roster self-corrects on the next `guild.roster`, which the
shard re-emits whenever the member set changes. Keeping the delta out of the projection is what keeps
the sidecar a forwarder rather than a thing that maintains state.
### 3.1 Split rosters are reassembled in memory
A roster split across frames is reassembled **before** it is stored, and written once on the frame
that closes it. Appending to the column per frame was rejected twice over: it would make the write a
read-modify-write — the exact thing the two-column split exists to avoid — and it would publish a
torn roster, since a reader hitting `GET /guilds` between frames would see a partial member list
presented as the whole truth.
This is transport-level reassembly, the same category of work as turning bytes into a line, and it
holds nothing once a roster is complete. The ordinary single-frame case never enters the buffer at
all. Around it: a fresh `seq: 0` supersedes an abandoned partial, an out-of-order frame discards the
partial rather than storing one with an undetectable hole, a continuation with no start is ignored, a
`server.hello` drops every partial (a reconnected shard restarts each roster at 0), and accumulation
is bounded so a shard that never sends a closing frame cannot grow the buffer without limit.
### 3.2 This is the first bump that needed a store migration
`store.rs`'s `SCHEMA` is `CREATE TABLE IF NOT EXISTS`, which can add a table but **cannot add a
column to a table that already exists**. Every schema change up to and including Protocol 3.0 added
whole tables, so this never mattered and `ALTER TABLE` appeared nowhere in the repo's history.
`guilds.members` is the first column added to an existing table: without a mechanism it would simply
never reach an installed sidecar, and every roster write would fail.
The counter is SQLite's own **`PRAGMA user_version`** — an integer in the database header, so it
costs no table and cannot drift from the file it describes. Each step runs in a transaction together
with the bump recording it, so a step lands completely or not at all and an interrupted run resumes
in the right place.
- **A failure aborts startup**, which was already the behaviour. A half-migrated store answers the
website with confusing partial data, which is worse than being plainly absent — and the shard dials
*out*, so a sidecar that refuses to start never stalls the game.
- **A database written by a newer sidecar warns and continues.** Every step is additive, so a newer
schema has only columns an older reader ignores; refusing to start would turn rolling the binary
back — a recovery path — into a dead end.
`sqlx::migrate!` was considered and rejected: it checksums each migration file, so editing an
already-released migration hard-fails startup, which is a poor trade for a schema of eleven
JSON-blob tables.
---
## 4. Visibility
Both kinds map to the existing **`guilds`** feature in the website's visibility framework
([`v3.md` §3](v3.md)). That mapping is required, not cosmetic: rule 2 fails an *unmapped* kind closed
to admin-only, which would have quietly kept rosters off the public Guilds page forever.
Mapping them is safe because of how the framework strips fields. A roster is the first frame carrying
locked fields inside an **array** of actors rather than a single nested actor — but the projection
walker already recurses into arrays and matches `acct`/`webId` **by suffix, on meaning rather than
spelling**. So a member's account name is stripped below `admin` by exactly the rule that already
strips `guild.leader.acct`.
The website stores `acct`/`web_id` per member, because that is what lets a linked member be matched to
a site user at all. It never projects them below `admin`. The difference this rule makes is a public
page listing character names versus one publishing 150 account names, so it carries a test of its own.
---
## 5. Cross-repo obligations
| Repo | Change |
|---|---|
| `servuo-plugins` | member-set diff, `guild.roster` + `guild.leave`, `BridgeJson.Actors`, two `Bridge.cfg` keys, **`overlay.toml` protocol → 4** |
| `link` | `PROTOCOL_VERSION` → 4, `members` column + `user_version` migration, roster reassembly, `GET /guilds` projection |
| `module-uo` | `shard_guild_members`, `guild.roster`/`guild.leave` ingest, the kind→feature map entries |
| `installer` | `backup.rs`'s stated reason for skipping the sidecar DB (§3.2 falsifies it) — docs only |
| `docs` | this file, `PROTOCOL_2.md` §10.1, `INTEGRATION.md` |
**`overlay.toml` must move in the same PR as the emitters.** CI folds it into the release manifest and
the installer refuses to pair an overlay and a sidecar whose protocol numbers disagree, so a bump
landing separately would silently fail to compose into a bundle.
---
## 6. Verification
`servuo-plugins` has no CI build — the plugin compiles only inside ServUO — so "it compiled" is not
evidence and neither is a clean boot. What was actually run:
1. **A throwaway one-guild spike first** (TEAMS.md Phase 0), against the real Rust sidecar rather than
a stub, to retire the unknowns before committing to a four-repo bump. It proved the two-column
board shape, measured the line, and surfaced §3.2 — the missing migration mechanism — which no
amount of reading would have.
2. **A live end-to-end run**: 155 members seeded from real `PlayerMobile`s on the local ServUO tree,
the cap forced down to 50 so the split path actually fired, producing 50/50/50/5 across four
frames; two members then removed on a timer, producing exactly two `guild.leave` frames with the
correct serials and a re-emitted roster at `total: 153`.
3. **A restart test with the shard stopped first**, so its reconnect could not re-emit and fake
persistence. The board's row came back byte-identical.
Step 2 is what caught the reassembly bug in §3.1: every unit test passed through it, because they all
exercised a single-frame roster. The case does not arise until a guild exceeds the cap.
Still outstanding for the cutover: the five-rung shard visibility walk against a live shard, confirming
`acct`/`webId` never reach a caller below their rung.