fix(shard): enforce visibility on the REST reads that bypassed it #110

wtclaude · 2026-07-28T15:54:05Z

wtclaude commented

2026-07-28 15:54:05 +00:00

What & why

Protocol 3.0 Part A follow-up. Found by running the v3.md §11 visibility smoke test live — all five rungs against a real MariaDB and a stub sidecar.

Part A implemented the framework correctly on the SSE path and on /guilds + /governors, but the remaining public REST reads never called into it. The result: the same event was projected live and served verbatim from stored history.

The four gaps

#	Gap	Impact
1	`GET /public/shard/feed` returned the stored payload as-is	`actor.acct` / `actor.webId` readable anonymously for every logged kind — `player.death`, `player.murdered`, `mob.killed`, `quest.complete`, `skill.gain`, `fame`/`karma.change`, `mob.login`/`logout`, `guild.join`. Broader than the §3.1 leak Part A closed, which only covered board holders.
2	`GET /public/shard/idoc` returned `ownerAcct`	The house owner's game account name on an anonymous endpoint.
3	The `houses` field rules were dead config	Neither `getIdoc` nor `getHouses` projected — the admin panel offered toggles that did nothing.
4	`/feed` filtered on `PUBLIC_KINDS`	A module-load constant derived from compiled defaults, so live audience changes never reached it. With `guilds` moved to `staff`, `/guilds` 403'd while `/feed` still served `guild.join` to anonymous.

Plus a latent fail-open: shardEvents.db.list treated an empty kinds array as "no filter" and fell through to an unfiltered SELECT. A fully-gated config would have dumped the entire event log, staff audit included.

The fixes, at the root rather than per-route

Rule 1 now matches a field's meaning, not one spelling. The wire nests actors (leader.acct); the read models flatten them (shapeHouse → ownerAcct, shapeGuild → leaderWebId), and an exact-key check missed every flattened one. isLockedField() locks a key that is or ends in acct/webId, case-insensitively — so it fails closed for shapes nobody has written yet. The admin PUT rejects those spellings too.
visibleKinds(level, config) resolves readable kinds from the live config; getFeed uses it and projects each row against its own kind's feature. Deliberately independent of the stream flag, which governs SSE fan-out only — so market history stays readable with its firehose off.
getIdoc / getHouses / getChamps / getPresence project, so every shard surface honours one config.
An empty allowlist serves nothing, never everything.

Also fixes a bug introduced while wiring this up: projectValue recursed into any object, so a Date column came back as {}. It now walks arrays and plain objects only. The unit tests used JSON fixtures and could not have caught it — the live /idoc read did, which is the argument for the §11 smoke test over tests alone.

Docs: RunicGateway/docs#65.

How it was tested

Live, five rungs (anonymous / logged_in / player / staff / admin) against local MariaDB + a stub sidecar WS pushing real frames, with seeded guild/governor/house/online fixtures carrying acct/webId:

13 routes × 5 rungs. Defaults reproduce pre-v3 access exactly. Zero acct/webId occurrences below admin across every shard read (was 90 on /feed alone).
editor (unlinked) correctly resolves to logged_in — no shard privilege.
Field projection: /online location gates at staff; house.decay gives anonymous no owner, staff owner without acct/webId, admin everything.
SSE: unmapped kinds (staff.command, cheat.detect, login.attempt) reach only admin — rule 2 fails closed.
Live config changes take effect on an already-open stream (verified guild.update stopping mid-connection); stream=false suppresses SSE while REST stays 200; enabled=false → 404; out-of-rung → 403.
Admin write path: locked fields, unknown features, unknown rungs and unknown field rungs all rejected 400 — including the flattened ownerAcct.
Admin SSE channel remains adminOnly (401/403/403/403/200).

Automated: DB_HOST=127.0.0.1 DB_PORT=59999 npm test → 487 pass / 0 fail (+9 new). Swagger regenerated; route manifest unchanged (no route signatures changed). Also cut the two slowest tests in shardControllerPublic from ~10s each to instant by stubbing the model rather than the util's exports — an exports-level stub doesn't intercept the module-internal getConfig.

Checklist

I have read CONTRIBUTING.md.
The change builds and existing tests/checks pass locally.
I have added or updated tests/docs where it makes sense.
My commits are reasonably scoped with clear messages.

AI-assisted contributions (required)

AI tools were used. Tool(s): Claude Code (Opus 5). I have reviewed and understand every change, and take responsibility for it. AI-authored commits are marked with a Co-Authored-By trailer.

License

I agree that my contribution is licensed under this project's license (GNU GPL v3.0 or later), and I have the right to contribute it.

## What & why Protocol 3.0 Part A follow-up. Found by running the [`v3.md` §11](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/edge/link/v3.md) visibility smoke test live — all five rungs against a real MariaDB and a stub sidecar. Part A implemented the framework correctly on the **SSE path** and on `/guilds` + `/governors`, but the remaining public REST reads never called into it. The result: the same event was projected live and served **verbatim** from stored history. ### The four gaps | # | Gap | Impact | |---|---|---| | 1 | `GET /public/shard/feed` returned the stored payload as-is | `actor.acct` / `actor.webId` readable **anonymously** for every logged kind — `player.death`, `player.murdered`, `mob.killed`, `quest.complete`, `skill.gain`, `fame`/`karma.change`, `mob.login`/`logout`, `guild.join`. **Broader than the §3.1 leak Part A closed**, which only covered board holders. | | 2 | `GET /public/shard/idoc` returned `ownerAcct` | The house owner's **game account name** on an anonymous endpoint. | | 3 | The `houses` field rules were dead config | Neither `getIdoc` nor `getHouses` projected — the admin panel offered toggles that did nothing. | | 4 | `/feed` filtered on `PUBLIC_KINDS` | A module-load constant derived from compiled **defaults**, so live audience changes never reached it. With `guilds` moved to `staff`, `/guilds` 403'd while `/feed` still served `guild.join` to anonymous. | Plus a latent fail-open: `shardEvents.db.list` treated an **empty** `kinds` array as "no filter" and fell through to an unfiltered `SELECT`. A fully-gated config would have dumped the entire event log, staff audit included. ### The fixes, at the root rather than per-route 1. **Rule 1 now matches a field's *meaning*, not one spelling.** The wire nests actors (`leader.acct`); the read models flatten them (`shapeHouse` → `ownerAcct`, `shapeGuild` → `leaderWebId`), and an exact-key check missed every flattened one. `isLockedField()` locks a key that *is* or *ends in* `acct`/`webId`, case-insensitively — so it fails closed for shapes nobody has written yet. The admin `PUT` rejects those spellings too. 2. **`visibleKinds(level, config)`** resolves readable kinds from the **live** config; `getFeed` uses it and projects each row against **its own kind's feature**. Deliberately independent of the `stream` flag, which governs SSE fan-out only — so market history stays readable with its firehose off. 3. **`getIdoc` / `getHouses` / `getChamps` / `getPresence` project**, so every shard surface honours one config. 4. **An empty allowlist serves nothing**, never everything. Also fixes a bug introduced while wiring this up: `projectValue` recursed into *any* object, so a `Date` column came back as `{}`. It now walks arrays and plain objects only. **The unit tests used JSON fixtures and could not have caught it — the live `/idoc` read did**, which is the argument for the §11 smoke test over tests alone. Docs: RunicGateway/docs#65. ## How it was tested **Live, five rungs** (`anonymous` / `logged_in` / `player` / `staff` / `admin`) against local MariaDB + a stub sidecar WS pushing real frames, with seeded guild/governor/house/online fixtures carrying `acct`/`webId`: - **13 routes × 5 rungs.** Defaults reproduce pre-v3 access exactly. **Zero `acct`/`webId` occurrences below `admin` across every shard read** (was 90 on `/feed` alone). - `editor` (unlinked) correctly resolves to `logged_in` — no shard privilege. - **Field projection:** `/online` location gates at `staff`; `house.decay` gives anonymous no `owner`, staff `owner` without `acct`/`webId`, admin everything. - **SSE:** unmapped kinds (`staff.command`, `cheat.detect`, `login.attempt`) reach **only** admin — rule 2 fails closed. - **Live config changes** take effect on an **already-open** stream (verified `guild.update` stopping mid-connection); `stream=false` suppresses SSE while REST stays `200`; `enabled=false` → `404`; out-of-rung → `403`. - **Admin write path:** locked fields, unknown features, unknown rungs and unknown field rungs all rejected `400` — including the flattened `ownerAcct`. - Admin SSE channel remains `adminOnly` (401/403/403/403/200). **Automated:** `DB_HOST=127.0.0.1 DB_PORT=59999 npm test` → **487 pass / 0 fail** (+9 new). Swagger regenerated; route manifest unchanged (no route signatures changed). Also cut the two slowest tests in `shardControllerPublic` from ~10s each to instant by stubbing the model rather than the util's exports — an exports-level stub doesn't intercept the module-internal `getConfig`. ## Checklist - [x] I have read [CONTRIBUTING.md](CONTRIBUTING.md). - [x] The change builds and existing tests/checks pass locally. - [x] I have added or updated tests/docs where it makes sense. - [x] My commits are reasonably scoped with clear messages. ## AI-assisted contributions (required) - [x] AI tools were used. Tool(s): `Claude Code (Opus 5)`. I have reviewed and understand every change, and take responsibility for it. AI-authored commits are marked with a `Co-Authored-By` trailer. ## License - [x] I agree that my contribution is licensed under this project's license (**GNU GPL v3.0 or later**), and I have the right to contribute it.

wtclaude added 1 commit 2026-07-28 15:54:07 +00:00

fix(shard): enforce visibility on the REST reads that bypassed it f30ea66fce

Protocol 3.0 Part A follow-up, found by the live five-rung smoke test.

Part A implemented the visibility framework correctly on the SSE path
and on /guilds + /governors, but the remaining public REST reads never
called into it. The result was that one event was projected live and
served verbatim from history:

  * GET /public/shard/feed returned the stored payload as-is, so
    actor.acct and actor.webId were readable ANONYMOUSLY for every
    logged kind - player.death, player.murdered, mob.killed,
    quest.complete, skill.gain, fame/karma.change, mob.login/logout,
    guild.join. Broader than the guild-leader leak Part A set out to
    close, since it covers every player rather than board holders.

  * GET /public/shard/idoc returned ownerAcct - the house owner's game
    account - to anonymous callers.

  * The `houses` field rules (owner/price -> staff) were dead config:
    neither getIdoc nor getHouses projected, so an admin could set them
    in the panel and nothing happened.

  * /feed filtered on PUBLIC_KINDS, a module-load constant derived from
    the compiled DEFAULTS, so live audience changes did not reach it.
    With `guilds` moved to staff, /guilds 403'd while /feed happily
    served guild.join to anonymous.

Four fixes, all at the root rather than per-route:

1. Rule 1 now matches a field's MEANING, not one spelling. The wire
   nests actors (leader.acct) but the read models flatten them
   (shapeHouse -> ownerAcct, shapeGuild -> leaderWebId), and an
   exact-key check missed every flattened one. isLockedField() locks a
   key that is or ends in acct/webId, case-insensitively, so it fails
   closed for shapes not yet written. The admin PUT rejects those
   spellings too - `ownerAcct` is no longer configurable.

2. visibleKinds(level, config) resolves readable kinds from the LIVE
   config; getFeed uses it and projects each row against its own kind's
   feature. Deliberately independent of the `stream` flag, which governs
   SSE fan-out only - so market history stays readable with its firehose
   off. This makes the set a superset of PUBLIC_KINDS by exactly the two
   vendor kinds.

3. getIdoc/getHouses/getChamps/getPresence project, so every shard
   surface honours the same config.

4. shardEvents.db.list treats an EMPTY kinds array as "serve nothing".
   It previously fell through to the unfiltered query, so a fully-gated
   config would have dumped the whole event log, staff audit included.

Also fixes a bug introduced while wiring this up: projectValue recursed
into any object, so a Date column came back as {}. It now walks arrays
and plain objects only. The unit tests used JSON fixtures and could not
have caught it - the live /idoc read did.

Verified live against MariaDB + a stub sidecar, all five rungs: 13
routes x 5 rungs, defaults reproducing pre-v3 access exactly, zero
acct/webId below admin on any read, unmapped kinds (staff.command,
cheat.detect, login.attempt) reaching only admin on SSE, and audience /
enabled / stream changes taking effect live on an already-open stream.

Tests: 487 server (+9). Swagger regenerated; route manifest unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>

wtclaude referenced this pull request from RunicGateway/docs

2026-07-28 15:54:29 +00:00

docs(shard): record the REST projection gap the Part A smoke test found #65

whitlocktech merged commit 6b1396dd2f into edge

2026-07-28 15:58:31 +00:00

whitlocktech deleted branch fix/shard-visibility-rest-projection

2026-07-28 15:58:32 +00:00

whitlocktech referenced this issue from a commit

2026-07-28 15:58:33 +00:00

Merge pull request 'fix(shard): enforce visibility on the REST reads that bypassed it' (#110) from fix/shard-visibility-rest-projection into edge

wtclaude referenced this pull request

2026-07-29 23:07:57 +00:00