RunicGateway/docs
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
32def88c4e33185826778f483fa1b45bf06ab48c
docs/link/v3.md
wtclaude 32def88c4e docs(link): fill in the cutover PR numbers 
The order-6 row was written before the seven PRs existed. Same follow-up as the
cliloc row got.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-29 18:09:50 -05:00

66 KiB
Raw Blame History

Protocol 3.0 — Shard content, standings & the visibility framework

Status: Feature-complete on edge; the cutover (order 6) is in review. All work lands on an edge branch in each repo; edge → main is the v3 cutover. Date: 2026-07-28 Codebase: ServUO 57.4, <servuo>, net48 / x64, Expansion EJ. Companion to PLAN.md (1.0 read/event plane), PROTOCOL_2.md (2.0 provisioning + world-state streams), ADMIN_CONTROLS.md (staff write plane), INTEGRATION.md (website API).

Progress

Each part is marked off here as it lands on edge. §9 carries the same state per sequencing row.

Order Part State Landed on edge
1 A — visibility framework + actor-leak fix (§3) ✅ Done website #109 + #110, docs #64 + #65
2 B/1 — world.ruleset (§5) ✅ Done servuo-plugins #3, link #17, website #111, docs #66
3 C — spawn atlas (§6) ✅ Done website #112 (parsers + CLI + tables) + #113 (API + pages + admin panel), docs #67 + #68
4 B/2 — points.board (§7) ✅ Done servuo-plugins #4, link #18, website #114, docs #69
5a B/3 dependency — cliloc table (§8.6) ✅ Done website #115, docs #70
5b B/3 — vendor.listing (§8) ✅ Done servuo-plugins #5, link #19, website #116, docs #71
6 Cutover — PROTOCOL_VERSION 2→3 (§4) 🟨 In review the bump: link #20, website #117, docs #72 — then edge → main: servuo-plugins #6, link #21, website #118, docs #73

Order 5 split in two once §8.6's cliloc dependency turned out to be a client-format problem rather than a parser (see §8.6). 5a is website-only and lands first so the marketplace ships with real item names; 5b is the four-repo wire change.

1. Why 3.0

A survey of the live ServUO tree against everything the bridge already surfaces end-to-end found that the bridge covers live activity well and covers shard content and standings almost not at all.

Covered by 1.0 + 2.0: presence/online, region transitions, char vitals + profile + roster, house registry + IDOC decay, champion spawns, guild board, city governors + term history, help-page queue, total gold supply, player-vendor sales log, deaths/murders/kills, skill gains, fame/karma, quest completes, staff/cheat audit, account linking + creation, town crier + news.

Not covered by anything: every leaderboard, every ruleset fact, every "where do I find X", and the entire player economy outside a player's own vendors.

3.0 has three scope areas:

  • A — The visibility framework (§3). Admin-configurable, per-feature and per-field audience control over every shard-derived surface on the website. Ships first; the rest depends on it.
  • B — Three new wire streams (§5, §7, §8). world.ruleset, points.board, vendor.listing/vendor.listing.remove.
  • C — One website-only feature (§6). The spawn atlas, built from static ServUO data files with no wire involvement at all.

2. Survey: the full gap list

Recorded so the items not scoped for 3.0 aren't re-derived later.

# Gap Source on the shard Value Cost Status
1 Points/loyalty leaderboards — 25 point currencies Scripts/Services/PointsSystems/PointsSystem.cs → static List<PointsSystem> Systems, each List<PointsEntry>{Player,Points} Very high Low 3.0 §7
2 Shard ruleset page Config/*.cfg via Server.Config.Get High Very low 3.0 §5
3 Shard-wide marketplace PlayerVendor.PlayerVendors + VendorSearch.cs Very high High 3.0 §8
4 Spawn atlas / bestiary Spawns/*.xml (6,455 spawners), RevampedSpawns/*.xml (333), Data/Regions.xml, Data/Locations/*.xml, Config/ChampionSpawns.xml, Data/teleporters.csv, Data/HarvestLocs/* High Medium 3.0 §6
5 VvV standings + battle status Services/ViceVsVirtue/{ViceVsVirtueSystem,GuildStats,VvVBattle}.cs High Medium PROTOCOL_2.md §10.4 deferred this pending "which PvP system does this shard run?" — now answered: VvV.cfg Enabled=True, Factions.cfg off. Unblocked, not scoped here
6 Skill leaderboards + shard census Services/Reports/Reports.cs → GetSkillDistribution(), CompileGeneralStats(), StaffHistory High Low Spec'd PROTOCOL_2.md §14 (Part B phase 6), unbuilt. Shares §7's UI — fold in after
7 Custom mounts/pets codex — ~35 across 4 tiers Scripts/Custom/{Companions,Legendary,Named,New Legacy} Medium-high Very low Pure wiki/CMS content, zero bridge work. The shard's most distinctive content, with zero site presence
8 Community Collections progress Services/CommunityCollections/CollectionsSystem.cs Medium Low Natural public "community goal" widget
9 Seasonal/holiday event calendar Services/Seasonal Events/SeasonalEventSystem.cs, Krampus, Forsaken Foes Medium Low "What's live now / what's next"
10 Crafting / taming / harvesting feeds EventSink.CraftSuccess / TameCreature / ResourceHarvestSuccess Medium Low Spec'd PROTOCOL_2.md §11 #3/#4/#5, unbuilt
11 Virtue progression EventSink.VirtueLevelChange, Services/Ethics/ Medium Low Spec'd §11 #6, unbuilt
12 Bulk Order Deeds + reward tables Services/BulkOrders/, Data/Bulk Orders/* Medium Low Feed spec'd §11 #7; the static reward tables are a free wiki page
13 Guild wars war state on Guild Low-medium Low Spec'd §11 #8, unbuilt
14 Astronomy discovery log Services/Astronomy/AstronomySystem.cs (104 KB save) Low Low Niche completion leaderboard
15 In-game chat relay Services/Chat/, Logs/Chat/{General,Help,Trade,LFG} Low Medium Privacy-sensitive; staff-only at most
16 Shard health telemetry Crash logs, LayerConflict.log, throttle.log, world.save.after counts, AutoSave/AutoRestart schedule Low-medium Low world.save.after is already ingested but never charted — world-size-over-time is nearly free
17 Ultima Store / Sovereigns balance Store.cfg Enabled=True, CurrencyName=Sovereigns; UltimaStore.GetCurrency ? Medium Only worth it if sovereigns are actually sold

Excluded permanently — see ADMIN_CONTROLS.md Tier H/N: firewall/IP-block, kill/resurrect, jail, item/gold grants, set-access-level, arbitrary [set/[add.

Noticed during the survey, out of scope: Scripts/Custom/PerryOwnerFix.cs hardcodes an EventSink.Login hook granting AccessLevel.Owner to account "ShardOnwerPerry". Worth reviewing independently of this work.

3. Part A — The visibility framework ✅ Done

Landed on edge: website #109 (the framework) and #110 (the REST-projection gap §3.6.1 records), docs #64 + #65. Smoke-tested across all five rungs per §11.

3.1 The leak this replaces (verified 2026-07-28)

BridgeJson.Actor() (BridgeJson.cs:85-117) writes serial, name, acct, webId, player. shardState.model.js:346 shapeGuild() returns r.payload verbatim, and GET /api/v1/public/shard/guilds (anonymous, shard.controller.js:131) serves it. A guild leader's game account name and website user id are readable on an anonymous public endpoint today. The same path exists for shapeGovernor → /public/shard/governors. Actor also feeds guild.join, city.update and region.enter, all three in PUBLIC_KINDS on the anonymous SSE stream.

The framework below is the vehicle for the fix, and the reason it ships before anything else.

3.2 Where visibility lives

On the website, never in the sidecar. The sidecar's job for 3.0 is unchanged in character: accept frames, persist them to its SQLite store, forward them verbatim over WS, and serve store-backed reads that survive a shard outage. It defines no access parameters, no audiences, no field projection, and advertises no capabilities.

3.3 The audience ladder

anonymous → logged_in → player → staff → admin, each rung implying the ones below it.

viewerLevel(req) resolves: no session ⇒ anonymous; authenticated ⇒ logged_in; authenticated with a linked shard account ⇒ player; moderator/admin role ⇒ staff/admin. Staff always satisfy the player rung even without a linked game account, consistent with the existing rule that /player/* is role-agnostic self-service.

3.4 Two limits an admin cannot override

  1. acct and webId are admin-only, always. They are not in-game-visible and are not exposed as configurable fields.
  2. A kind absent from the kind→feature map is never broadcast below admin. Fail closed. This preserves the property that today's static PUBLIC_KINDS allowlist is a security boundary rather than a convenience filter.

3.5 Configuration

CREATE TABLE IF NOT EXISTS shard_feature_visibility (
  feature     VARCHAR(48) NOT NULL PRIMARY KEY,
  enabled     TINYINT(1)  NOT NULL DEFAULT 1,
  audience    VARCHAR(20) NOT NULL DEFAULT 'anonymous',
  field_rules JSON        NULL,   -- {"<field>": "<rung>"} for sensitive fields only
  updated_by  INT         NULL,
  updated_at  DATETIME    NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

Not seeded on boot (this changed during implementation): an absent row means "use the compiled default", so the table starts empty and only ever holds rows an admin has actually touched. The defaults live in one place — FEATURES in shardVisibility.js — instead of being duplicated into a seeder that could drift from it, and a DB blip degrades to those same defaults rather than to "everything is public". All ten shard features are covered — the four new ones and the six that already ship — and every default reproduces today's behavior, so the retrofit is a no-op until an admin changes something.

Feature Default audience Sensitive fields (default rung)
status, activity, champs, guilds, governors anonymous guilds/governors: leaderAcct / leaderWebId → admin (locked)
houses anonymous owner → staff, price → staff (matches today's IDOC-only public view)
presence anonymous location → staff (matches today's staff-only, location-gated /online)
ruleset (new) anonymous connect → anonymous
atlas (new) anonymous —
leaderboards (new) anonymous characterName → anonymous
market (new) anonymous ownerName → anonymous, location → anonymous

3.6 Enforcement — three points, one config

New website/server/src/utils/shardVisibility.js:

  • LADDER = ['anonymous','logged_in','player','staff','admin'], rank(), meets(viewer, required)
  • viewerLevel(req) (§3.3)
  • KIND_FEATURE — every event kind → its feature; unmapped ⇒ admin-only (§3.4)
  • getConfig() — DB-backed, cached ~5 s like uoLinkClient's config cache, busted on admin PUT
  • requireFeature(name) — 404 when disabled (don't leak existence), 403 when enabled but the viewer is below the audience
  • projectFeature(name, payload, viewerLevel) — strips fields whose rung the viewer doesn't meet; acct/webId always stripped below admin

Applied at:

  1. Routes — requireFeature(…) on every /public/shard/*, /public/atlas/* and the shard-derived player routes; projectFeature in the controllers, replacing the ad-hoc shapeGuild-returns-payload-verbatim path.
  2. SSE — shardBroadcast.js moves from "one public channel with a static PUBLIC_KINDS allowlist plus one admin channel" to per-connection filtering: each subscriber carries its viewerLevel; each frame is mapped kind→feature, gated on enabled && meets(...), then passed through projectFeature before write. PUBLIC_KINDS becomes the seed data for KIND_FEATURE rather than a hardcoded gate. This is the largest single change in Part A and where the security boundary now lives.
  3. Nav — GET /api/v1/public/shard/features returns only the features the calling viewer can see, so the SPA hides nav entries rather than rendering links that 403.

3.6.1 What the first implementation missed (found by the §11 smoke test, fixed)

Part A shipped enforcement on the SSE path and on /guilds + /governors, but the remaining public REST reads never called into it — so the same event was projected live and served verbatim from history. Recorded because each miss is a shape the next phase can repeat:

  • /public/shard/feed returned the stored payload as-is. actor.acct / actor.webId were readable anonymously for every logged kind (player.death, mob.killed, skill.gain, guild.join, …) — broader than the §3.1 leak, which was limited to board holders.
  • /public/shard/idoc returned ownerAcct. Rule 1 keyed on the exact strings acct/webId, but shapeHouse flattens the actor into ownerAcct / ownerName / ownerSerial. The lock is now on the field's meaning — a key that is or ends in acct/webId, case-insensitively — so flattened spellings are covered and unwritten shapes fail closed.
  • The houses field rules were dead config. Neither getIdoc nor getHouses projected, so the panel offered toggles that did nothing. Every feature's declared fields must name the keys the read model actually emits, not just the wire frame's.
  • /feed filtered on PUBLIC_KINDS, a module-load constant derived from the compiled defaults, so live audience changes never reached it. visibleKinds(level, config) resolves the readable set from live config; it deliberately ignores the stream flag, which governs SSE fan-out only (market history stays readable with its firehose off).
  • shardEvents.db.list treated an empty kinds array as "no filter" and fell through to an unfiltered SELECT. A fully-gated config would have dumped the whole event log, staff audit included. An empty allowlist now serves nothing.
  • projectValue recursed into every object, so a Date column came back as {}. It walks arrays and plain objects only. The unit tests used JSON fixtures and could not have caught this — the live read did, which is the argument for §11's smoke test over tests alone.

The rule this leaves behind: a read path that returns shard data and does not call projectFeature is a bug. Every new surface in Parts B and C — /ruleset, /points, /market, /atlas — must project, and must gate its kind set on live config rather than on PUBLIC_KINDS.

3.7 Admin surface

GET / PUT /api/v1/admin/shard/visibility (admin-only). Validate feature names against the known set and rungs against the ladder; reject any attempt to set a locked field below admin — including its flattened spellings (ownerAcct, leaderWebId), see §3.6.1. Writes an admin.audit-style row so visibility changes are traceable. New client panel routes/admin/ShardVisibility.jsx at /admin/shard-visibility, linked from ShardAdmin.jsx.

4. The version bump and the rollout

PROTOCOL_VERSION 2 → 3 in link/sidecar/src/main.rs:27. v3 is defined as "adds world.ruleset, points.board, vendor.listing / vendor.listing.remove".

A bump is an operator-visible hard cutover — web.rs::gate returns 409 on every protected route on mismatch, uoLinkSocket.js::handleHello closes the WS, and the website's declared version is the admin-set uo_link_config.protocol column — so it happens exactly once, at the end:

  • Cut an edge branch from main in each of website/, link/, servuo-plugins/, docs/.
  • Every phase PRs into edge, never main. Feature branches are cut from edge.
  • Part A lands first, alone.
  • When all parts are built and tested, one edge → main PR per repo, merged together. That merge is the v3 cutover.
  • A schema migration sets uo_link_config.protocol (the existing row and the column default) from 2 to 3, so the cutover doesn't require a manual admin edit. UOLINK_PROTOCOL still overrides.
  • No feature-negotiation array anywhere — v3 implies all three kinds.

4.1 What the bump actually touches

The version lives in five places, and all five move together:

Where Change
link/sidecar/src/main.rs PROTOCOL_VERSION 2 → 3 (with the v3 note beside the v2 one), plus the sidecar README's worked example
website/server/db/schema.sql uo_link_config.protocol column default 1 → 3, plus the boot migration below
website/server/src/model/uoLinkConfig/uoLinkConfig.model.js DEFAULT_PROTOCOL — what a site with nothing saved yet declares
website/server/src/utils/uoLinkClient.js + uoLinkSocket.js the `config.protocol
website/client/.../ShardAdmin.jsx, website/.env.example the admin form's initial value and the documented env default

The migration has to be one-shot, and that is the only subtle part. schema.sql is re-run on every boot (utils/db.js::ensureSchema), and every other statement in its migration block is an idempotent ADD COLUMN IF NOT EXISTS / MODIFY. A bare UPDATE uo_link_config SET protocol = 3 would not be idempotent in the sense that matters: protocol is admin-editable, so an operator who deliberately pins an older sidecar in Admin → Shard would silently be un-pinned on the next restart. It is therefore gated on a marker row in settings:

ALTER TABLE uo_link_config MODIFY COLUMN protocol INT NOT NULL DEFAULT 3;
UPDATE uo_link_config SET protocol = 3
  WHERE id = 1 AND protocol < 3
    AND NOT EXISTS (SELECT 1 FROM settings WHERE `key` = 'uo_link_protocol_3_migrated');
INSERT IGNORE INTO settings (`key`, value) VALUES ('uo_link_protocol_3_migrated', '1');

The marker is written after the UPDATE, so the first boot on the new build migrates and every later boot is a no-op. A fresh install has no uo_link_config row to update and simply gets the marker plus the new column default. protocol < 3 rather than = 2 so an install that never left the old default of 1 is carried across too — it could not have been talking to a v2 sidecar anyway.

5. Part B/1 — world.ruleset ✅ Done

Landed on edge: servuo-plugins #3, link #17, website #111, docs #66. Implementation notes worth keeping:

  • shadowguard is derived, not configured. Shadowguard.cfg carries only ReadyDuration and RandomizeInstances — there is no Enabled key — so the systems block reports Core.TOL (the expansion gate) instead. Same shape for factions: Factions.cfg has no Enabled either, and Services/Factions/Core/Faction.cs sets Settings.Enabled = !ViceVsVirtueSystem.Enabled, so the frame reads that static rather than inventing a key. Where a system's on/off state is derived, read the system's own static; only read Config.Get where the .cfg key IS the truth.
  • caps.skill / caps.totalSkill are in tenths (1000 = 100.0), the way ServUO stores them. Documented in INTEGRATION.md and converted in the client, because the raw number is actively misleading rather than merely unhelpful.
  • Config.Get re-parses when the cached type differs. InternalGet<T> caches the parsed value on the entry and re-parses if entry.Object is T fails, so reading PlayerCaps.SkillCap as an int where ServUO reads it as a double is correct (both parse) — it just re-parses. Harmless, but worth knowing before assuming a shared cache.
  • The plugin CAN be compile-verified, contrary to "no standalone build": point Roslyn (dotnet sdk/*/Roslyn/bincore/csc.dll, /langversion:7.3, net48 reference assemblies) at the whole ServUO Scripts tree with overlay/Scripts/Custom/Bridge/*.cs substituted for the deployed copy, excluding Scripts/obj and Scripts/bin. 6,205 files, ~40 s, and it catches every signature error a boot would. Worth doing before every plugin PR.

PROTOCOL_2.md §10.4 sketches a world.systems capability frame that was never implemented (grep returns nothing across all four repos). world.ruleset subsumes it, carrying a systems sub-object with the cityLoyalty / vvv / factions booleans §10.4 asked for. §10.4 is marked superseded; no orphan kind is left behind.

5.1 Plugin

NEW servuo-plugins/overlay/Scripts/Custom/Bridge/BridgeRuleset.cs, modelled on BridgeBoot.EmitHello — not a sweep. Subscribes BridgeLink.Connected_Core += Emit so a sidecar that comes up second still learns the ruleset.

Built from an explicit allowlist of Server.Config.Get<T> calls. Never enumerate Config.Entries (Server/Config.cs:162) — it would sweep in secrets. An FNV-1a rev over the body makes an unchanged reconnect a site-side no-op (String.GetHashCode() is not stable across runs and must not be used).

BridgeConfig.cs + overlay/Config/Bridge.cfg: RulesetEnabled=true, PublicConnectAddress="", RulesetIncludeSchedule=true. BridgeBoot.cs: reload → re-emit, status → rev/bytes. Not wired to sweepnow; it isn't a sweep.

5.2 Payload

Every block optional, omitted when its system is off:

shard, expansion, connect (only from PublicConnectAddress), systems{cityLoyalty,vvv,factions,siege,chat,store,dailyRares,honesty,shadowguard,treasureMaps,vetRewards,testCenter}, caps{skill:1000,totalSkill:7000,stat:225,str/dex/int:125,strMax/dexMax/intMax:150}, housing{accountHouseLimit:1}, accounts{perIp:3,charSlots:7,autoCreate}, vetRewards{enabled,rewardIntervalDays:30}, loot{feluccaLuckBonus:1000,feluccaBudgetBonus:100,feluccaMaxProps:11}, vendors{restockDelayMinutes,maxSell,economyStockAmount}, champions{powerScrolls:6,statScrolls:16,scrollChance,transcendenceChance,rankThresholds}, treasureMaps, vvv{enabled,startSilver:2000,enhancedRules}, store{enabled,currencyName}, schedule{autoSaveFrequencyMinutes,autoRestart*}.

Excluded by name — in a code comment and here: Server.cfg (Address/Listen/Port; only PublicConnectAddress is published), Staff.cfg, Email.cfg, DataPath.cfg, Bridge.cfg, Compiler.cfg, Reports.cfg, Client.cfg.

5.3 Sidecar and website

Sidecar — store.rs: singleton ruleset(id CHECK(id=1), rev, json, updated_t) + upsert/get; main.rs: new arm in the board-projection match; web.rs: GET /ruleset served from the store, so it answers during a shard outage (PROTOCOL_2.md §12.2).

Website — uoLinkClient.getRuleset(); uoLinkSocket.backfill() (object-shaped, so follow the getPresence() block's explicit form, not the array-only snapshot() helper); shardIngest.js → shardState.setRuleset, not in LOGGED_KINDS (it re-arrives every reconnect and server.hello already marks those); KIND_FEATURE['world.ruleset'] = 'ruleset'; shard_ruleset singleton table (rev, expansion, payload JSON, t); GET /public/shard/ruleset behind requireFeature('ruleset'), returning null ⇒ "not published yet".

Client — NEW routes/public/Rules.jsx at /site/rules, alongside /site/champs|guilds|governors|houses; live via useShardFeed({ filter: new Set(['world.ruleset']) }).

5.4 Risk

Perf is nil (~3 KB per connect). The only real risk is publishing a secret, mitigated by the explicit allowlist, the no-Config.Entries rule, the named exclusion list, and a manual eyeball of the emitted frame during verification.

6. Part C — Spawn atlas / bestiary (website-only)

No plugin, no sidecar, no Bridge.cfg knob, no new kinds. Not part of the v3 wire change.

Status: complete on edge — website #112 (parsers, import CLI, tables) and #113 (the six public routes, the five admin ones, /site/atlas + /site/atlas/:slug, and the Admin → Spawn Atlas panel). Part C ships as two website PRs, not one: the parsing half is where the correctness risk lives, and burying it under routes and React would have meant reviewing it in a 10k-line diff. Full operator documentation: docs/website/SPAWN_ATLAS.md.

§6 below is the original design and is partly superseded. §6.1 records two decisions that were rejected in review and replaced (the committed artifact, and the fixed facet list); §6.2 records the corrections the real ServUO data forced. Read both before trusting §6.

Decision (revised at implementation time): the shard's ServUO tree is the single source of truth, re-derived on every server boot. The original plan here was a committed generated artifact plus an idempotent import. That was rejected in review for two reasons, recorded in §6.1: a snapshot in the repo goes stale as a shard's maps change, and the design leaned on a fixed facet list that no shard is obliged to keep. Still not a browser-served blob; still parsed server-side only.

New in website/server/:

  • src/utils/spawnAtlasParse.js — pure functions, no fs, so they are unit-testable in CI without a ServUO tree: parseObjects2(), parsePoints(), parseRegions(), parseLocations(), resolveRegion().
  • scripts/buildSpawnAtlas.js and a committed db/data/spawnAtlas.*.json artifact — dropped, see §6.1 R1. Replaced by src/utils/spawnAtlasSource.js (the only thing that reads a ServUO tree, shared by the boot path and the CLI) and a scripts/importSpawnAtlas.js that is a thin CLI over the model. package.json gains atlas:import only.
  • src/model/shardAtlas/{shardAtlas.db.js,shardAtlas.model.js} following the shardState split.
  • src/router/v1/public/atlas.{router,controller}.js; test/spawnAtlas.parse.test.js.

Two parsing notes that matter:

  • <Objects2> is Type:MX=n:SB=… segments joined by :OBJ= — verified against trammel.xml, where a single point carries six types. Split on :OBJ=; the token before the first : is the type.
  • The high-value transform: point-in-rect each spawn against the facet's Regions.xml rects (highest priority wins), falling back to the nearest Data/Locations landmark, else "Wilderness". This is what turns "lizardman at 5411,1234" into "Despise, Felucca" and is the entire reason the page is worth building. Regions.xml is genuinely nested and needs a ~120-line recursive tokenizer or one devDependency (fast-xml-parser) — the server has zero XML deps today, so that is an explicit call to make at implementation time. The flat <Points> files need only regex/streaming; do not put 10.5 MB through a DOM parser.

Tables: shard_spawn_creatures (slug PK, name, total, facets JSON), shard_spawn_points (slug, facet, x, y, region, landmark, max_count, tod_*), shard_regions, shard_landmarks, shard_champion_spawns, shard_atlas_meta. Plain INDEX on name, not FULLTEXT — ~1,500 creature rows makes a LIKE scan free, and FULLTEXT brings min-token-length trouble for names like "orc". No FKs, consistent with every existing shard_* table.

Routes at /api/v1/public/atlas, not under /shard — the atlas is static shard content, not live shard state; it must not look sidecar-dependent, and unlike /shard/* it should be siteMode-gated like /posts and /wiki. GET /creatures?q=&facet=, /creatures/:slug, /regions, /landmarks, /champions, /meta, all behind requireFeature('atlas'). Admin: GET /admin/shard/atlas/status (artifact-vs-DB drift) and POST /admin/shard/atlas/import. Build stays CLI-only.

Client: routes/public/Atlas.jsx (/site/atlas) and AtlasCreature.jsx (/site/atlas/:slug).

Payload risk — superseded by §6.1 R1; nothing is committed. The field selection it describes still applies at parse time: every <Points> field the site cannot use (UniqueId, all trigger/refractory/proximity/sequential fields, sound ids) is dropped, keeping Name/Map/X/Y/W/H/Range/MaxCount/MinDelay/MaxDelay/TOD*/types. Parsed data never reaches the browser; the browser sees only paginated API responses.

Operator re-run story — revised by §6.1 R1. Spawns changed → restart, or npm run atlas:import / POST /admin/shard/atlas/import to apply without one. shard_atlas_meta holds a sha256 per source file, so the server can tell on boot whether anything changed, and GET /admin/shard/atlas/status reports drift. If the change would remove a facet it is staged for approval rather than applied (§6.1 R3). Full detail in docs/website/SPAWN_ATLAS.md.

6.1 What implementation changed

Two design decisions in §6 were rejected in review and replaced; the rest are corrections the real ServUO data forced. Kept as a diff rather than edited in place, because each is a trap the next person would otherwise re-enter.

R1. The committed artifact is gone — the tree is re-parsed on every boot. §6 proposed building a generated artifact, committing it, and importing it. Two problems. A shard's maps change over its life, so a snapshot in the repo silently drifts from the world players actually see; and the build/ import split existed only to work around the website container not having a tree, which is a deployment question (mount it) rather than a reason to freeze data. The server now hashes the source files on boot and re-derives the atlas when they differ. scripts/buildSpawnAtlas.js, the 1.41 MB artifact, and the whole encode/decode seam it needed are deleted.

R2. Nothing may name a facet. The first implementation carried a lookup table of the six stock UO facets to reconcile the spelling drift between sources. A shard may add facets, replace them outright, or rename them when its maps are updated, and a built-in list mishandles all three silently. Reconciliation is now by matching against the facet set discovered from the shard's own spawn and region data — exact key, then prefix in either direction — with an unmatched name keeping its own rather than being forced into a wrong bucket.

R3. Two contracts on the boot path. It never blocks startup: no path, an unreadable mount, a malformed file or a database error is caught and logged, and the site comes up serving whatever atlas it had. And a refresh that would REMOVE a facet is never applied automatically — facet loss is indistinguishable at boot from a half-copied or mid-update tree, so it is staged in shard_atlas_pending for an admin to approve or reject. Only the decision is stored (source hashes

  • the facet diff, a few KB); approving re-parses, so what lands matches the tree at approval time. A rejection is remembered against those hashes so it does not re-prompt every restart.

6.2 What the build against real data changed

Six corrections to the design above, from running it against stock ServUO 57.4. Kept as a diff rather than edited in place, because each one is a trap the next person would otherwise re-enter.

1. Six facets, not thirteen. The design said spawnAtlas.<facet>.json ×13, assuming one facet per spawn file. There are 13 files but only 6 facets — Eodon.xml, GravewaterLake.xml, TreasuresOfKotl.xml and the other named-area files carry TerMur/Trammel points. The facet comes from each record's own <Map>, never the file name, and the artifact shards 6 ways.

2. The XML dependency call: hand-rolled, zero deps. §6 left fast-xml-parser vs a ~120-line tokenizer open. Resolved as the tokenizer — a deliberate subset parser covering only what these files use. The server keeps zero XML dependencies at any tier.

3. Facet names disagree between sources — a silent failure. Data/Locations/*.xml spells them Ter Mur and Tokuno Islands; <Map> and <Facet name> say TerMur and Tokuno. Unreconciled, the landmark bucket is keyed differently from the points looking it up, so the fallback never fires and every unregioned spawn in Ter Mur and Tokuno reads "Wilderness" — a plausible-looking atlas that is quietly wrong for two facets. All facet names now pass through normalizeFacet().

4. Spawn type tokens carry XmlSpawner directives. <Objects2> types are not always bare class names: Fairy,{RND,4,8}, alchemist/z/-50, Agralem/Name/Agralem, greatape,true. Taken literally they invent creatures that do not exist and split real ones in two, since Fairy and Fairy,{RND,4,8} slug apart. 71 of 845 entries were affected; stripping at the first / or , leaves 800 real creatures. (The design's "~1,500 creature rows" estimate was high; 800 only reinforces the plain-INDEX-not-FULLTEXT call.)

5. The artifact would have been 1.41 MB, not "well under 1 MB" — and is now moot. Dropping the unused <Points> fields as the design directed still left 4.40 MB; three further encodings brought it to 1.41 MB, and getting under 1 MB would have meant dropping the spawner name. The size budget in §6 was simply optimistic for 6,455 points. Superseded by §6.1 R1: there is no artifact, so there is no payload to budget and no encode/decode seam to keep in sync.

6. DELETE, not TRUNCATE. The design said "TRUNCATE + batched INSERT in one transaction", which does not hold: TRUNCATE is DDL in MariaDB and implicitly commits, so a mid-import failure would leave the atlas half-loaded. DELETE is transactional, and at ~7k rows the cost is irrelevant. Point ids are also assigned explicitly rather than by AUTO_INCREMENT, because the join rows need them and conn.batch() reports no usable insertId.

Measured result: 6,455 points, 800 creatures, 23,927 point/type rows, 387 regions, 558 landmarks, 25 champion altars. The placement transform resolves 83.2% of points (3,689 by region, 1,690 by landmark, 1,086 Wilderness).

One thing the design got exactly right: the point-in-rect transform really is the reason to build this. "Where does a lizardman spawn?" answers Shrines, Isamu-Jima, Yew across three facets.

6.3 What the API/client half added

The second website PR built the six public routes, the five admin ones, /site/atlas + /site/atlas/:slug, and the Admin → Spawn Atlas panel. Three things it changed or established:

1. Respawn delays were being read in the wrong unit — sometimes. XmlSpawner writes MinDelay/MaxDelay in minutes and switches to seconds only when a delay does not divide into whole minutes, flagging that per record with DelayInSec (XmlSpawner2.cs:7462-7480, read back at :6345-6358). So a 5 means five minutes on one spawner and five seconds on the next, both plausible, and the pipeline stored the raw number. 170 of 6,455 stock spawners are second-flagged — few enough to look like noise on a page and be believed. The parser now normalises to seconds, and the API and UI carry seconds throughout. This is the class of bug §6.2 is a list of: the atlas still builds, it is just quietly wrong.

2. The hash gate needed a parser version, and this generalises. Fixing the parse exposed that "has the tree changed?" is the wrong question on its own — an install whose maps never change would have kept serving the old readings forever, because the only thing compared was the tree. spawnAtlasSource.PARSER_VERSION is stored in shard_atlas_meta beside the source hashes, and a mismatch counts as drift. Any future parse correction lands on the next boot without an operator having to know it happened. Bump it whenever the parser derives different data from identical files.

3. points is a count; spawners is the list. The first cut of the detail route spread the creature row and then set points to the array of spawn points — the same key meaning a number on the search route and an array on the detail route. Renamed before it shipped, and worth recording because the two names are one letter apart in meaning and it reads as correct.

On projection. The atlas feature declares no sensitive fields, so projectFeature is a no-op on every one of these routes today. Every handler calls it anyway, per §3.6.1's rule — the point of the rule is that the first field that needs gating is covered by construction rather than by a retrofit nobody remembers to do.

7. Part B/2 — points.board ✅ Done

Landed on edge: servuo-plugins #4, link #18, website #114, docs #69. Verified against the real ServUO tree per §11 — see §7.5 for what that run changed.

Two deliverables: a diff sweep for the boards, and a points block folded into char.profile — the PROTOCOL_2.md §10.3 titles precedent (read-model enrichment, no new request kind).

7.1 Plugin

NEW BridgePoints.cs, copying the BridgeHousing.cs diff-sweep shape (Initialize → ServerStarted, Connected_Core += OnConnected clearing _last + Rearm(), SweepOnce(), Status(), skip when !BridgeLink.Connected, try/catch throughout).

Which systems: default to PointsSystem.Systems filtered to ShowOnLoyaltyGump == true — reuse the shard's own "this is player-facing" signal rather than inventing one. Bridge.cfg PointsSystems= overrides. Null-guard PointsSystem.Systems; it is a mutable static populated by 25 separate subsystem constructors.

The perf trap. PlayerTable is a plain List<PointsEntry>, and QueensLoyalty has AutoAdd, so it can hold an entry for every PlayerMobile that ever existed. A naive .OrderByDescending().Take(N) across 25 systems is 25 full sorts — at 20,000 historical characters, ~7.5 M comparisons, tens of ms on the Core thread. BRIDGE_PLUGIN_PLAN.md §1 found that nothing except bulk profile generation comes close to a frame budget; this would be the second thing that does.

Mitigation — single-pass bounded selection into a fixed N-element sorted array (N=10): O(n·N) with tiny constants and one allocation. Skip Player == null || Deleted and Points <= 0. ~500 k cheap iterations at a 300 s interval.

Diff signature per system: concat(serial + ":" + (long)points) over the top N, plus the entry count. No points.remove — the system set is fixed, the same argument city.update already uses.

7.2 Payload — one frame per system

25 × ~600 B rather than one 12 KB frame, matching champ.update / guild.update:

{"t":…,"kind":"points.board","system":"QueensLoyalty",
 "nameString":"Queen's Loyalty","nameNumber":1114938,
 "maxPoints":30000,"showOnGump":true,"players":842,
 "top":[{"rank":1,"serial":"0x1A2B","name":"Darrow","points":29500}, …]}

nameString and nameNumber are both emitted (a TextDefinition may be a cliloc), resolved website-side — the contract titles.reward already documents at BridgeProfile.cs:107-110.

Entries are written inline as {serial, name} — never via BridgeJson.Actor. Deliberate even though the website can now reveal fields by rung: acct/webId are not needed here, because the website resolves serial→user from its own shard_account_links mirror for staff views. Keep the wire minimal.

7.3 char.profile enrichment

BridgeProfile.cs gains WritePoints(sb, m) alongside WriteTitles: "points":[{system,nameString,points,maxPoints}], omitting systems with no entry or 0 points.

Deliberately no rank — computing it means scanning each system's PlayerTable once per profile (25 × n), which would dominate the measured 0.069 ms/profile budget. The website derives rank from the board when the character appears in the top N. Gate behind PointsProfileRank=false if it is ever wanted.

7.4 Config, sidecar, website

Bridge.cfg: PointsSweepSeconds=300, PointsLeaderboardEnabled=true, PointsTopN=10, PointsSystems= (blank ⇒ auto), PointsProfileEnabled=true, PointsProfileRank=false. BridgeBoot.cs: Rearm() in reload, SweepOnce() in sweepnow, Status() in both.

Sidecar — points_boards(system PK, name, json, updated_t); main.rs arm keyed on system; GET /points and GET /points/:system.

Website — shard_points_boards(system PK, name, name_cliloc, max_points, players, show_on_gump, payload JSON, t). The top-N list stays in payload — a fixed-size list read whole, exactly like shard_governors.candidates. Do not normalize into a shard_points_entries table until a per-character reverse lookup is actually needed. shardIngest.js → upsertPointsBoard, not in LOGGED_KINDS (board state, like guild.update). KIND_FEATURE['points.board'] = 'leaderboards', with characterName as its per-field rule. GET /public/shard/points and /points/:system behind requireFeature('leaderboards'); validate system ≤ 48 chars.

No new player route — per-character points ride inside char.profile, already served by GET /player/shard/char/:serial with its shardLinks.ownsAccount check.

Client — NEW routes/public/Leaderboards.jsx at /site/leaderboards; a "Loyalty & Points" section added to components/CharacterSheet.jsx, one edit serving both PlayerCharacter.jsx and AdminCharacter.jsx.

7.5 What the run against a real shard changed

The plan above was written from reading PointsSystem.cs. Booting the actual shard (ServUO 57.4, a 43,011-mobile world) and letting one sweep run corrected four things — all of them invisible to a fake-shard test, because a fake shard emits whatever the spec says it should.

  1. maxPoints overflowed to long.MinValue. MaxPoints is a double, and ServUO's idiom for an uncapped system is double.MaxValue — which DespiseCrystals, ShameCrystals and VoidPool all use. (long)double.MaxValue in C# is an unchecked conversion: it does not throw, it yields long.MinValue, and the first real sweep published "maxPoints": -9223372036854775808 for three of the five live boards. Fixed with Cap() / Score() converters that normalise anything unrepresentable to 0, which is now the wire's documented "uncapped" value. Worth stating plainly because it inverts the obvious reading: on a real shard, maxPoints: 0 is the common case, not an edge case, so any UI dividing by it must special-case it.
  2. nameString is usually null. Most systems define their Name as a cliloc rather than a literal: four of the five boards on the live shard came back nameString: null with only nameNumber set. The humanise-the-system-key fallback is therefore the primary display path, not a defensive nicety, and both the leaderboards page and the character sheet lead with it.
  3. GetEntry/GetPoints cannot be used in the read model. GetEntry(from, create: false) still calls AddEntry when the system has AutoAdd (PointsSystem.cs:207) — it mutates the world. Ten of the ~25 systems have AutoAdd = true, so a profile built with the obvious accessor would have appended up to ten rows to the points save file every time anyone viewed a character sheet. BridgeProfile.WritePoints hand-rolls a read-only scan instead, and says so loudly.
  4. players had to be redefined. §7.2 called for "the entry count", but those same ten AutoAdd systems hold a zero-point row per character ever created — so the raw count reports the shard's whole census as one system's participants. It is now the number of players actually holding points, which is both the honest number and a strictly better diff signal (it moves when someone scores, not when someone logs in for the first time).

One deviation from the plan as written, for the same class of reason: §7.4 named the per-field visibility rule characterName, but projectValue matches on the literal JSON key, and the wire key is name. A rule under the descriptive name would have been silently inert — an admin tightening character names would have got no enforcement and no error, exactly the failure §3.6.1 records for the flattened ownerAcct. FEATURES.leaderboards.fields therefore keys on name, with a test that fails if it is renamed back.

8. Part B/3 — vendor.listing 🟨 In review

8.1 It cannot be an RPC, and this is load-bearing

rpc.rs::try_route correlates on the first frame carrying a matching reqId and resolves a single oneshot. A chunked reply sharing one reqId would deliver chunk 1 to the HTTP caller and leak chunks 2..N onto the broadcast feed. REPLY_TIMEOUT is 10 s (the client waits 12 s), so a whole-world snapshot could not fit regardless.

⇒ a per-vendor diff sweep on the broadcast stream, like champ.update / house.update. The existing per-account vendor.snapshot RPC is untouched; the player portal keeps using it.

Kinds: vendor.listing (one frame per vendor, authoritative for that vendor) and vendor.listing.remove. Payload: serial, shopName, owner:{serial,name}, map, x, y, region, house, count, truncated, items:[{serial,itemId,hue,amount,price,name,cliloc,child}].

8.2 Two perf traps

Measured baseline (BRIDGE_PLUGIN_PLAN.md §1): 30 vendors / 1,200 listings = 0.343 ms via pack.Items + GetVendorItem; extrapolated to 500 vendors / 40,000 listings ≈ 12 ms per full pass. Except:

  1. VendorSearch.GetItemName(Item) is a packet builder, not a field read. It constructs an ObjectPropertyList, calls GetProperties, serialises, then byte-parses the packet (VendorSearch.cs:681-789) — per item. Across 40,000 items in one tick that is a multi-hundred-millisecond stall. Mandatory: never call it in the sweep. Emit itemId, hue, amount, price, item.Name (the plain field, null for most) and item.LabelNumber, resolving display names website-side — exactly what char.profile.equipment already does (BridgeProfile.cs:173).
  2. VendorSearch.GetItems(PlayerVendor) is private (:791). The reusable public API is GetItems(Container, List<Item>) (:807), which recurses into sub-containers, so real item counts run above the top-level pack.Items the 0.343 ms measurement used. Budget accordingly.

8.3 Mitigations

  • Amortized round-robin sweep — MarketSweepSeconds=60, at most MarketSweepBatch=25 vendors per tick, with a persistent cursor over PlayerVendor.PlayerVendors. Full coverage in ceil(vendors/25) × 60 s, with per-tick cost bounded independent of world size. This is the one genuinely new pattern versus the existing sweeps and should be flagged in review.
  • Per-vendor signature diff (count | Σ(serial ^ price) | x | y | shopName), as BridgeHousing does — most vendors are static, so steady-state emission is near zero.
  • MarketMaxListings=250, then "truncated":true. BridgeJson.Parse caps inbound at 1 MB; outbound is uncapped and shard.rs::read_line will allocate whatever arrives.
  • On Connected_Core, clear _last and reset the cursor; the re-emit is self-throttled by the round-robin window.

8.4 Player opt-out and privacy

Honour pv.VendorSearch — ServUO's own per-vendor opt-out, which DoSearch filters on (:62). Skip opted-out vendors entirely; the seen-set removal then drops them from the board, so a player who hid their vendor in game is hidden on the website too. Also skip Map == null || Map.Internal and Backpack == null, matching DoSearch.

A vendor's shop name, owner character name and location are already globally visible in-game — the stock Vendor Search gump surfaces exactly this set to any player — which is why they default to anonymous. They remain per-field configurable (ownerName, location) so an admin can tighten them. Account name and website user id never go on the wire.

8.5 Sidecar and website

Sidecar — one table vendors(serial PK, shop_name, owner_name, map, x, y, region, count, json, updated_t) storing the whole-vendor blob. No vendor_items table — the sidecar's job here is outage resilience (PROTOCOL_2.md §12.2), not search; search lives in MariaDB. Endpoint is GET /market, not /vendors — axum would route the latter fine, but the collision with the per-account RPC is a readability trap.

Website — shard_vendors + shard_vendor_items (indexes on vendor_serial, price, item_id, display_name; delete-then-insert per vendor in one transaction; no FKs). shardIngest.js handles both kinds; not in LOGGED_KINDS.

KIND_FEATURE['vendor.listing'] = 'market', but the market feature's SSE mapping is disabled by default: a live firehose of full vendor inventories would be the site's single biggest bandwidth consumer, and no page needs it live. The page is a paginated DB query with a staleness stamp; an admin can turn the stream on. uoLinkSocket paginates /market on reconnect, bounded by MARKET_SNAPSHOT_MAX = 5000 vendors so a pathological world cannot hang startup.

GET /public/shard/market?q=&minPrice=&maxPrice=&itemId=&map=&region=&sort=&limit=&offset= (limit 1..100, default 50; q ≤ 60 chars; sort ∈ {price_asc, price_desc, recent}) and /market/vendors/:serial, behind requireFeature('market'). Rate-limit it — this is the first genuinely expensive public endpoint; express-rate-limit is already a dependency.

8.6 The open dependency — cliloc names ✅ Resolved (shipped ahead of §8)

CharacterSheet.jsx:14-15 documented the gap ("without a cliloc table on the site we can only show literals") and rendered equipment as id {itemId}. Search-by-name needs that table.

Resolved as its own website-only change, landed BEFORE the market so /site/market ships with real item names. Full design and operator guide: docs/website/CLILOCS.md. Ingest denormalizes the resolved name into shard_vendor_items.display_name as planned.

Two things in the original recommendation above turned out to be wrong, and both are worth recording because the reasoning generalises.

1. The committed db/data/clilocs.json artifact was dropped. It predates the two Part C corrections (§6.1) and violates both: no committed snapshot of derived content, and nothing EA-derived ever shipped. UO's strings are EA's, exactly as the creature sprites are. Replaced with the §6 pattern instead — parse on every boot from an operator-configured path, hash-gated, output gitignored, PARSER_VERSION counted as drift.

2. scripts/buildClilocs.js reads the UO client's Cliloc.enu is not possible, and the reason matters. Every current client ships its cliloc files COMPRESSED — all eight Cliloc.* files open with a DWORD whose high byte is 0x8E, the "Mythic" container. The plain layout (02 00 00 00 01 00, then {int32 number, byte flag, uint16 length, UTF-8}) is what those files looked like before that change. Parsing a compressed file as plain does not fail cleanly: it yields ~19k "records" with negative ids, 1,722 distinct keys out of 19,508, one 62 KB "string", and a truncation somewhere in the middle.

Decompressing means porting an inverse-BWT coder with a 1 KB frequency header — a few hundred lines whose failure mode is plausible-looking garbage rather than an error. Two facts closed off the alternatives:

  • ServUO cannot read it either. Its bundled Ultima.StringList implements only the plain layout, so on a modern client VendorSearch.StringList is null and VendorSearch.GetItemName returns item.Name. The in-game Vendor Search gump has the same gap — which also means §8.2's warning never to call GetItemName in the sweep costs us nothing we could otherwise have had.
  • The shard therefore cannot supply names on our behalf, so this could not be pushed to the plugin.

⇒ the operator converts once, from their own client, and the site reads the result. Accepted shapes are the plain binary layout and a number<TAB|,|;>text export; the site sniffs which. server/tools/cliloc-export/ drives UOFiddler's Ultima.dll (the decompressor that already exists) and writes the plain form. A shard that never converts is fully supported — names render as ids, exactly as before.

Shards edit items and add new ones, and those carry ids no stock client table has — so this reads a set of sources, not one file, hash-gated together and re-read on every boot exactly as §6 reads the ServUO tree: a base (the converted client table) plus every overlay under custom/, later winning. Adding one custom item therefore never means re-exporting a 5 MB client file. Measured on the live shard for scale: its script tree references 16,434 cliloc ids and only 37 are absent from stock — tens against a 67k base, which is why an overlay and not a second table. custom/ is the one convention here that is ours rather than the shard's, because ServUO has no server-side notion of a custom cliloc: they live in the patched client a shard distributes, and nothing in the tree declares them.

That set also brings back a hazard a single file did not have, and §8.6 answers it the way §6 does. A corrupt source fails the parse loudly, but a source that has vanished parses perfectly and imports a table quietly missing everything it contributed — an unmounted volume is indistinguishable from a deliberate deletion. So it is staged, not applied (status: 'needsReview'), reported by both the import and status(), and accepted with {approve:true}. It is a flag rather than §6's approve/reject pair because the atlas stores a pending decision so that approving re-parses; here nothing is stored, so re-reading at approval time is automatic.

Five traps found by building it, all recorded in CLILOCS.md:

  • StringList.SaveStringList RE-COMPRESSES on save. It looks exactly like the export path and is not; its output is byte-identical to its compressed input, because its purpose is round-tripping a file back into the client.
  • Trimming a text line before splitting silently drops half the table. Roughly half of a real cliloc table is empty strings (ids the client reserves), exported as 1005008<TAB>. Trimming eats the trailing separator, leaving a bare number that then looks like a header row — 55,994 of 123,490 entries vanished, and the import still looked successful.
  • Number('') is 0, not NaN. A line starting with a separator imports as a bogus cliloc 0 unless the empty field is rejected explicitly.
  • Tidying punctuation unconditionally corrupts real names. Stripping leftover brackets is right after a placeholder is removed ([~1_stuff~] → nothing) and wrong otherwise: a shard's custom "Runic Gateway Sigil (v2)" rendered as "(v2". Same shape as the % rule. Found only by running a shard-style overlay through it — every stock-table fixture passed.
  • Source labels must be forward-slashed and root-relative, or the same directory fingerprints differently on Windows and Linux and every boot looks like a change. The identical bug §6 records.

Blank entries are dropped at import (123,490 parsed → 67,496 stored), which also makes the binary and text paths converge on identical content.

8.7 Client

routes/public/Market.jsx at /site/market, with a "prices last refreshed N minutes ago" banner driven by staleAt (the oldest shard_vendors.updated_at). The round-robin sweep means data is inherently up to one full cycle old, and the UI must say so.

Shipped with a second page, routes/public/MarketVendor.jsx at /site/market/vendors/:serial — where a search result points. It is the only surface that can render the two states the result list cannot: a truncated shop ("showing 250 of 3,104 — this shop holds more than the shard publishes") and a location an admin has gated away, which is a real answer rather than an empty coordinate.

8.8 What the build changed

Four things the implementation settled differently from §8 as written, all of them found by building against the live shard.

1. location is a nested object, not flat map/x/y/region. §8.1's payload sketch had them flat, and it would have made market.location — a rule Part A pre-wired — inert, exactly like the characterName miss §7.5 records: projectValue matches literal JSON keys, so there is no location key for the rule to match. Flat keys would have needed five rules that could drift apart. Nesting makes one rule hide the facet, the coordinates, the region and the house together, on the live frame and the stored read model alike, because both now spell it the same way.

The other pre-wired rule, market.ownerName, checked out — it is a real key on the frame. Owner is written as flat ownerSerial/ownerName rather than through BridgeJson.Actor, which would add acct and webId; same argument points.board makes. ownerSerial was added to the configurable fields alongside ownerName, because an admin who hides the owner's name and leaves a serial every other board resolves back to that name has not hidden anything.

2. The per-vendor diff signature is the full listing set, not §8.3's count | Σ(serial ^ price). That hash collides on the single most common change a shop makes: two items swapping prices, which is what re-pricing looks like. The signature is built over the same buffer the frame is written from, in the same order, so a match really does mean an identical frame.

3. There is no payload column on shard_vendors. §8.5 implied the board pattern (whole frame in JSON, columns hoisted for display). It does not apply here: the items ARE the searchable rows, so they are normalized into shard_vendor_items and there is nothing left worth duplicating. The sidecar keeps the whole blob, because outage resilience is its job and search is not.

4. Sweep cost is reported, and a slow tick warns. The batch cap is a claim about per-tick cost, and an operator tuning MarketSweepBatch was otherwise tuning blind. [bridge status now carries lastMs/maxMs, and a tick over 50 ms prints a rate-limited warning naming the knob.

Measured on the live shard (27 vendors × 40 listings, 209k items / 43k mobiles):
First tick — 25 vendors emitted cold 15.4 ms
Second tick — the remaining 2 3.4 ms
Steady state — nothing changed 0.3 ms
Website /market search over 1,040 listings 1,040 total, names resolved
Cliloc re-resolution pass over 1,040 rows 50 ms

The diff is what makes the steady state ~free; the batch cap is what bounds the cold case. Note the arithmetic the warning exists for: at the default cap of 250 listings, a batch of 25 full shops is 6,250 items ≈ 95 ms — over budget. Real shops hold tens, which is why 25 is the default, but a shard of commodity resellers should lower the batch, and now it will be told to.

Two smaller things worth not rediscovering:

  • BridgeJson.Escape takes a NON-NULL string — it dereferences value.Length immediately — and BridgeJson.Str writes its own ,"key": prefix, so neither serves a value inside a hand-built object. Nearly everything this frame writes is legitimately null (an item's plain Name is null for almost every item; a vendor in the street has no house), so that is the common path, not an edge case. BridgeMarket.Text() is the two-line writer that was missing.
  • The ServUO console writes in the OS code page, so an em dash in a Console.WriteLine renders as ??? in the log an operator would paste into an issue. Bridge console output is ASCII.

Search-side, one thing the site had to fix rather than inherit: % and _ in a user's query are LIKE metacharacters, not SQL ones, so parameterization does not neutralize them — a search for % would otherwise match every listing on the shard. shardMarket.db.js escapes them. (The atlas's LIKE searches predate this and have the same shape over a much smaller table; worth a follow-up, not a blocker here.)

9. Sequencing

Order Part Repos touched Wire change State
1 A — visibility framework + actor-leak fix website, docs none ✅ Done
2 B/1 — world.ruleset (§5) all four new kind ✅ Done
3 C — spawn atlas (§6) website, docs none ✅ Done
4 B/2 — points.board (§7) all four new kind + char.profile field ✅ Done
5a B/3 dependency — cliloc table (§8.6) website, docs none ✅ Done
5b B/3 — vendor.listing (§8) all four new kinds ✅ Done
6 Cutover — PROTOCOL_VERSION 2→3, edge → main all four the bump 🟨 In review

10. Documentation obligations

  • This file (link/v3.md) is the canonical 3.0 design.
  • PROTOCOL_2.md §10.4 gains a note that world.systems is superseded by world.ruleset, and that the deferred VvV question is answered (VvV.cfg Enabled=True, Factions off).
  • INTEGRATION.md — catalog entries and §6 consumer sections for each new kind, plus the v2→v3 upgrade note for operators.
  • PLAN.md — phasing.
  • website/BACKEND_DESIGN.md — every new table and route, and the visibility framework as a security contract: the audience ladder, the two locked rules, and the fail-closed kind map belong in the security section.
  • NEW website/SHARD_VISIBILITY.md — admin-facing: what each feature exposes, what each rung means, what cannot be loosened.
  • NEW website/SPAWN_ATLAS.md, NEW website/CLILOCS.md, NEW website/MARKETPLACE.md.
  • PROJECT_TREE.md in each touched repo.
  • npm run swagger and npm run routes:manifest on every route-touching PR — both are committed artifacts, and test/routeManifest.test.js fails on drift.

Follow-up, not scoped for 3.0: the Android app consumes the same public/player shard API and will need /public/shard/features to hide its own nav. Track separately against android-app/.

11. Verification

Plugin — servuo-plugins\deploy.ps1 -ServerPath <servuo> -Verify, inspect the ADD/CHANGE list, then re-run without -Verify (ServUO must be stopped). Boot with tools/stub_sidecar.ps1 listening and confirm the compile banner in the console, not merely the absence of errors — BRIDGE_PLUGIN_PLAN.md §1 warns that a failing build is silently ignored and the previous Scripts.dll reloads. Then [bridge status, [bridge sweepnow, [bridge reload.

  • §5: eyeball the emitted world.ruleset frame for anything sourced from Server.cfg, Staff.cfg, Email.cfg, DataPath.cfg or Bridge.cfg.
  • §8: with a seeded world, time one sweep tick and confirm the batch cap holds it under ~1 ms.

Sidecar — cargo build && cargo clippy; curl -H "Authorization: Bearer <token>" localhost:8080/ruleset (and /points, /market); confirm X-UOLink-Version: 3 and that a client declaring 2 receives a 409.

Website server — DB_HOST=127.0.0.1 DB_PORT=59999 node --test. New tests, each modelled on an existing sibling: test/shardVisibility.test.js, test/shardBroadcast.visibility.test.js, test/shardIngest.{ruleset,points,market}.test.js (after shardIngest.protocol2.test.js — stubbed deps, asserting routing and logged flags), test/spawnAtlas.parse.test.js (pure functions, inline fixtures). Then npm run routes:manifest and npm run swagger, committing both.

Full stack — against a local MariaDB: apply db/schema.sql (idempotent), start the server, confirm uoLinkSocket backfill logs the new snapshot lines and that uo_link_config.protocol migrated to 3, then load /site/rules, /site/atlas, /site/leaderboards, /site/market.

Visibility smoke test — for each of the five rungs, walk every shard page and confirm gating and field projection match the configured matrix. Same shape as the 200-routes × 5-access-levels sweep already run for the domain split.

12. Critical files

File Why
website/server/src/utils/shardBroadcast.js The security boundary; reworked from a static allowlist to per-connection audience filtering. The highest-risk file in 3.0.
website/server/src/utils/shardVisibility.js (new) Ladder, kind→feature map, projection
website/server/src/utils/shardIngest.js The dispatcher every new kind routes through
website/server/src/model/shardState/shardState.model.js The shape* projections, including the shapeGuild leak §3.1 fixes
servuo-plugins/overlay/Scripts/Custom/Bridge/BridgeHousing.cs Cleanest copy of the diff-sweep pattern; template for BridgePoints.cs and BridgeMarket.cs
link/sidecar/src/main.rs PROTOCOL_VERSION 2→3 and the board-projection match
website/server/db/schema.sql All new shard_* tables plus the uo_link_config.protocol migration